Reasons time horizon is overrated and misinterpreted:

In the 9 months since the METR time horizon paper (during which AI time horizons have increased by ~6x), it’s generated lots of attention as well as various criticism on LW and elsewhere. As one of the main authors, I think much of the criticism is a valid response to misinterpretations, and want to list my beliefs about limitations of our methodology and time horizon more broadly. This is not a complete list, but rather whatever I thought of in a few hours.

• Time horizon is not the length of time AIs can work independently
  • Rather, it’s the amount of serial human labor they can replace with a 50% success rate. When AIs solve tasks they’re usually much faster than humans.
• Time horizon is not precise
  • When METR says “Claude Opus 4.5 has a 50%-time horizon of around 4 hrs 49 mins (95% confidence interval of 1 hr 49 mins to 20 hrs 25 mins)”, we mean those error bars. They were generated via bootstrapping, so if we randomly subsample harder tasks our code would spit out <1h49m 2.5% of the time. I really have no idea whether Claude’s “true” time horizon is 3.5h or 6.5h.
  • Error bars have historically been a factor of ~2 in each direction, worse with current models like Opus 4.5 as our benchmark begins to saturate.
  • Because model performance is correlated, error bars for relative comparisons between models are a bit smaller. But it still makes little sense to care about whether a model is just below frontier, 10% above the previous best model, or 20% above.
• Time horizon differs between domains by orders of magnitude
  • The original paper measured it on mostly software and research tasks. Applying the same methodology in a follow-up found that time horizons are fairly similar for math, but 40-100x lower for visual computer use tasks, due to eg poor perception.
  • Claude 4.5 Sonnet’s real-world coffee-making time horizon is only ~2 minutes
• Time horizon does not apply to every task distribution
  • On SWE-Lancer OpenAI observed that a task’s monetary value (which should be a decent proxy for engineer-hours) doesn’t correlate with a model’s success rate. I still don’t know why this is.
• Benchmark vs real-world task distribution
  • We’re making tasks just ahead of what we expect future models to be able to do, and benchmark construction has many design choices.
  • We try to make tasks representative of the real world, but as in any benchmark, there are inherent tradeoffs between realism, diversity, fixed costs (implementation), and variable costs (ease of running the benchmark). Inspect has made this easier but there will obviously be factors that cause our benchmarks to favor or disfavor models.
  • Because anything automatically gradable can be an RL environment, and models are extensively trained using RLVR [1], making gradable tasks that don’t overestimate real-world performance at all essentially means making more realistic RLVR settings than labs, which is hard.
    
    Figure 1: METR staying one step ahead of frontier models

• Our benchmarks differ from the real world in many ways, some of which are discussed in the original paper.
  • Low vs high context (low-context tasks are isolated and don’t require prior knowledge about a codebase)
  • Well-defined vs poorly defined
  • “Messy” vs non-messy tasks (see section 6.2 of original paper)
• Different conventions around human baseline times could affect time horizon by >1.25x.
  • I think we made reasonable choices, but there were certainly judgement calls here– the most important thing was to be consistent.
  • Baseliner skill level: Our baseliner pool was “skilled professionals in software engineering, machine learning, and cybersecurity”, but top engineers, e.g. lab employees, would be faster.
  • We didn’t incorporate failed baselines into time estimates because baseliners often failed for non-relevant reasons. If we used survival analysis to interpret an X-hour failed baseline as information that the task takes >X hours, we would increase measured task lengths.
  • When a task had multiple successful baselines we aggregated these using the geometric mean. Baseline times have high variance, so using the arithmetic mean would increase averages by ~25%.
• A 50% time horizon of X hours does not mean we can delegate tasks under X hours to AIs.
  • Some (reliability-critical and poorly verifiable) tasks require 98%+ success probabilities to be worth automating
  • Doubling the time horizon does not double the degree of automation. Even if the AI requires half as many human interventions, it will probably fail in more complex ways requiring more human labor per intervention.
  • To convert time horizons to research speedup, we need to measure how much time a human spends prompting AIs, waiting for generations, checking AI output, writing code manually, etc. when doing an X hour task assisted by an AI with time horizon Y hours. Then we plug this into the uplift equation. This process is nontrivial and requires a much richer data source like Cursor logs or screen recordings.
• 20% and 80% time horizons are kind of fake because there aren’t enough parameters to fit them separately.
  • We fit a two-parameter logistic model which doesn’t fit the top and bottom of the success curve separately, so improving performance on 20% horizon tasks can lower 80% horizon.
• Time horizons at 99%+ reliability levels cannot be fit at all without much larger and higher-quality benchmarks.
  • Measuring 99% time horizons would require ~300 highly diverse tasks in each time bucket. If the tasks are not highly diverse and realistic, we could fail to sample the type of task that would trip up the AI in actual use.
  • The tasks also need <<1% label noise. If they’re broken/unfair/have label noise, the benchmark could saturate at 98% and we would estimate the 99% time horizon of every model to be zero.
• Speculating about the effects of a months- or years-long time horizon is fraught.
  • The distribution of tasks from which the suite is drawn from is not super well-defined, and so reasonable different extrapolations could get quite different time-horizon trends.
  • One example: all of the tasks in METR-HRS are self-contained, whereas most months-long tasks humans do require collaboration.
    • If an AI has a 3-year time horizon, does this mean an AI can competently substitute for a human for a 3-year long project with the same level of feedback from a manager, or be able to do the human's job completely independently? We have no tasks involving multi-turn interaction with a human so there is no right answer.
  • There is a good argument that AGI would have an infinite time horizon and so time horizon will eventually start growing superexponentially. However, the AI Futures timelines model is highly sensitive to exactly how superexponential future time horizon growth will be, which we have little data on. This parameter, “Doubling Difficulty Growth Factor”, can change the date of the first Automated Coder AI between 2028 and 2050.

Despite these limitations, what conclusions do I still stand by?

• The most important numbers to estimate were the slope of the long-run trend (one doubling every 6-7 months) and a linear extrapolation of this trend predicting when AIs would reach 1 month / 167 working hours time horizon (2030), not the exact time horizon of any particular model. I think the paper did well here.
• Throughout the project we did the least work we could to establish a sufficiently robust result, because task construction and baselining were both super expensive. As a result, the data are insufficient to do some secondary and subset analyses. I still think it’s fine but have increasing worries as the benchmark nears saturation.
  • Without SWAA the error bars are super wide, and SWAA is lower quality than some easy (non-software) benchmarks like GSM8k. This might seem worrying, but it’s fine because it doesn’t actually matter for the result whether GPT-2’s time horizon is 0.5 seconds or 3 seconds; the slope of the trend is pretty similar. All that matters is that we can estimate it at all with a benchmark that isn’t super biased.
  • Some tasks have time estimates rather than actual human baselines, and the tasks that do have baselines have few of them. This is statistically ok because in our sensitivity analysis, adding IID baseline noise had minimal impact on the results, and the range of task lengths (spanning roughly a factor of 10,000) means that even baselining error correlated with task length wouldn’t affect the doubling time much.
    • However, Tom Cunningham points out that most of the longer tasks don’t have baselines, so if we're systematically over/under-estimating the length of long tasks we could be misjudging the degree of acceleration in 2025.
  • The paper had a small number of tasks (only ~170) because we prioritize quality over quantity. The dataset size was originally fine but is now becoming a problem as we lack longer, 2h+ tasks to evaluate future models.
    • I think we’re planning to update the task suite soon to include most of the HCAST tasks (the original paper had only a subset) plus some new tasks. Beyond this, we have various plans to continue measuring AI capabilities, both through benchmarks and other means like RCTs.

[1] see eg DeepSeek R1 paper: https://arxiv.org/abs/2501.12948

No background, but it's plausible to me that they actively prefer imperfect alignment because companies that care about alignment will tend to be woke, moralizing, or opposed to authoritarianism.

In the specific hedonist vs Christian case, aren't there two obvious compromises?

• "One trillion year reign of the CEV of Jesus of Nazareth over the multiverse"
• "Entire mass of universe converted to nervous tissue experiencing euphoric union with God in His loving grace"

 There are three cost sources: materials, design, and manufacturing. I claim that because phones are small and expensive to design it's just not worth it.

• Materials: Phones have very high $/kg already. At $5,000/kg, you could launch them to space and only double the cost. So unless there's a better battery design made of pure gold, phones can already afford basically whatever materials are optimal. Even a $4 million hypercar costs less per kg than an iPhone Pro. 
• IP/design/software is zero marginal cost. It doesn't make sense for companies to spend $10 billion designing an OS for its premium users only. Better to segment on hardware.
• Manufacturing: Assembly just isn't that expensive. As for components, it's impossible to make a phone with chips more than one process node ahead than average. So you get $500 phones that are one node behind, and $1500 phones that are at the frontier and 20% faster, and that's the limit.

I don't really think progress being fast is quite sufficient to explain things. If every tripling in price could get you a 20% better phone, many people would pay for an 81x more expensive, 107% better phone just like they do for private jets. If it became obsolete in a year, the billionaire would have their phone guy replace it every six months. I think it's because a phone, or gmail, starlink, etc. has super high fixed costs, and the smaller market for luxury goods generally means the design will be worse. For food or furniture better quality ingredients/materials can more than make up for it, but consumer electronics can only get slightly better components because most already designed things are within the cost/kg budget of a $1k phone.

Everything more expensive per kg than phones has an interesting way of funding their design costs without scale:

• Military tech like fighter jets, radar systems: Cost mainly from r&d. The military demands highest performance and is a large market, spending 2-5% of GDP on fairly small volume
• Pharmaceuticals: Cost mainly from r&d. Either large scale or literally lifesaving for rich people
• Luxury watches: Practical benefits of a luxury watch are nil but people buy them anyway for status and wonder, essentially paying watchmakers to arrange titanium and rubies into ever more elaborate shapes that happen to tell time.

So my prediction would be that the luxury smartphone business only starts up when lots of rich people have different problems from the average consumer that need a custom device (security maybe?) or subscribe to a status game that results in the phone equivalent of luxury watches.

• Money can be exchanged for goods and services in under a 10-year timeframe, including AI hardware and talent. To come out ahead from trade they just need to invest more of their surplus in AI than the US does.
• The #1 issue for the Chinese public is the economy, just like in the US. This includes people with influence on CCP decision-making.

Seems unlikely to me this would be in their interest.

• Regarding chips, US just started selling them H200s, and if they invade Taiwan they probably destroy TSMC and immediately lose their main source of compute. The majority of the semiconductor industry outside Taiwan would still not be friendly to China.
• They also lose 50% of their trade even if the US continues trading with them, which tanks their GDP by something like 10% [1]. If the US sanctions them too, it would be 60-70% trade reduction.
• Militarily, they are also building up faster than the US, and if they invade Japan/SK will militarize and might get nuclear weapons.

If they really believed in RSI they would do diplomacy to get more compute and just invest in their AI industry.

[1] Claude thinks the likely outcome is this:

• Lost trade: $2.4-2.8 trillion (US, EU, Japan, UK, Canada, Australia, possibly South Korea/Taiwan)
• Continuing trade: $2.0-2.4 trillion (Russia, most ASEAN, Brazil, India, Middle East, Africa)
• Net trade: $2.0-2.4T vs previous $6.2T = 60-70% reduction

Sure, then just add that to the disclaimer. "I may omit claims that are risky, unpopular, may be easily misinterpreted, require lots of words to justify, etc, but aim to not outright lie for such reasons"

It seems super costly for many public intellectuals to say all of their beliefs, and for reasons that other commenters have pointed out, giving an epistemic status might not help. What's wrong with a blanket disclaimer like "Assume that all of my claims without an epistemic status are optimized to improve discourse on the margin, rather than to convey a complete picture of my all-things-considered beliefs"?

I think IBKR might let Americans trade HKEX index options out to 2030 but it would probably be a hassle. Otherwise there are options on FXI, a Chinese large-cap ETF, which look less liquid than SPX options and only go out 2 years with an IV of 32% or so. I don't think FXI is worth it because China isn't in a position to get AGI in the next 2 years if the US doesn't.

(in the usual units, this means that the plot of log(2025-FLOP per FLOP) vs log(researcher-hours) is a straight line with slope $r$.) A plot that curves downward or "hits a wall" seems like evidence against this model's applicability to the data.

Note there are no log-log plots in the data. They're performance vs LoC and log(performance) vs LoC, and same for stars. I don't think we're at an absolute ceiling since two more improvements came out in the past week, they've just gotten smaller and taken more code to implement.

I need to think about this algorithmic progress being 10x/year thing. It feels like some assumptions are violated with how much the data seem to give inconsistent answers, maybe there's a prospective vs retrospective difference. Or do you think progress has just sped up in the past couple of years?