OpenAI’s GPT-4 held the top position in a highly regarded AI performance index for about a year – significantly longer than any model thereafter. Since Claude 3 Opus surpassed GPT-4 in February 2024, the lead has changed 17 times, with an average duration of only about seven weeks per model.
What the Epoch Capabilities Index Measures
The analysis is based on the Epoch Capabilities Index (ECI) of the non-profit research organization Epoch AI, an aggregated measure of the performance of language models. According to Epoch AI, the index is calculated from more than 50 benchmarks that converge on the proprietary evaluation platform, combining both internal tests and values reported by model developers and benchmark creators. Currently, the ECI covers 168 models released since January 2023. For the analysis, Epoch AI only considers models that surpassed every previously released model at their launch – the so-called ECI front.
GPT-4’s Record and the Change Thereafter
According to Epoch AI researcher Jaeho Lee, GPT-4 led the index for 352 days, from its release on March 14, 2023, until Claude 3 Opus overtook it on February 29, 2024. This is by far the longest leadership duration in the available data. The second-longest tenure was held by OpenAI’s o1 with 98 days, followed by o1-mini with 96 days, Claude 3.5 Sonnet with 84 days, GPT-5 with 61 days, and o3-pro with 58 days. Thus, GPT-4 held the top position for about 3.6 times longer than the next-ranked successor. Of the 17 models that have taken the lead since GPT-4, none held the top position for triple-digit day counts.
Two Interpretations of a Pattern
According to Epoch AI, the data can be interpreted in two ways. On one hand, they show how extraordinarily long it took other AI labs to catch up to GPT-4 – the model was, at the time of its release, a true outlier compared to what the rest of the industry had to offer. On the other hand, the series of short leadership changes observed since then illustrates that competition among AI labs has become noticeably more intense: No provider can maintain a lead for an extended period because competitors quickly catch up.
Important Limitations of the Analysis
Epoch AI identifies several methodological limitations of the evaluation in its own publication. Since the public ECI does not evaluate models prior to 2023, GPT-4 can only be compared with later released models; whether individual older models were also leading for longer periods is considered by Epoch AI to be rather unlikely based on preliminary, uncertain estimates. Furthermore, the ranking is based on point estimates: Some of the leadership changes observed since GPT-4 are based on ECI differences that fall within the respective confidence intervals – if only statistically significant changes were counted as real leadership changes, several of the mentioned periods would be moderately extended. The index is also regularly recalibrated, which can cause historical values to shift slightly between updates. Finally, the current leader GPT-5.5 Pro is only measured up to the current data status, so its actual leadership duration could still extend.
Classification
The evaluation by Epoch AI thus provides a quantified confirmation of a trend that has been described anecdotal in the industry: GPT-4 marked a real leap forward in March 2023, to which the competition had nothing to counter for almost a year. Since then, the pace of model releases has increased so significantly that no leadership position can be maintained for long – with simultaneously diminishing leaps in capabilities between the individual leadership changes, compared to the phase of the first reasoning models like o1-preview in the fall of 2024.
