Discussion about this post

User's avatar
Daniel Kokotajlo's avatar

Thanks for this review! Some comments:

We co-authored an article with the "AI as Normal Technology" authors you might be interested in: https://asteriskmag.substack.com/p/common-ground-between-ai-2027-and

"one of the authors recently acknowledged that “AI propaganda hasn’t been as effective as feared”."

That was about my previous scenario, written in 2021, not about AI 2027. I don't think there's been less AI propaganda so far than depicted in AI 2027, because AI 2027 didn't really depict AI propaganda happening by mid 2026. (If you are interested in the previous scenario, it's called What 2026 Looks Like. https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like)

"By their estimate, uplift is only 17% of what they’d predicted at this point. The 65% figure primarily reflects metrics that are not directly related to real-world AI capabilities, such as benchmark scores and data center construction. I should note however that frontier lab revenue is at 80% of prediction (in their scoring, which may in fact be conservative here)."

After we published that, frontier lab revenue has continued to increase and now is higher than AI 2027 predicted. This matters because frontier lab revenue is pretty directly related to real-world AI capabilities!

As for uplift: We actually think we're being pretty harsh on ourselves here. The reason it's only 17% is because at the time we wrote AI 2027 we thought uplift was higher than it actually was. In our writeup we say: "AI software R&D uplift is behind pace. This is primarily because we have updated our estimate of uplift in early 2025 downward, and thus our uplift estimates for the end of 2025 are similar to our original estimates for the start of AI 2027." So basically, the slope of the line is similar to what we expected, but the whole line is shifted down relative to what we expected. Very different from progress only being 17% as fast as we expected, which is a possible interpretation someone might have from just seeing that figure.

Antipopulist's avatar

My main issue with checking the predictions from AI 2027 is that it's really two stories: things continue walking along normally until early 2027, and then a MASSIVE discontinuity happens as AI leaps from merely doing well on arbitrary benchmarks to swallowing most white collar work in a short amount of time. But the former does not imply the latter: there's tons of schlep in peoples' jobs that AI is not showing signs of being able to overcome. It can help with individual tasks, but is still very far from serving as a drop-in replacement. The AI 2027 story doesn't do enough to show how AI gets from benchmarks to real-world impacts or how it solves for the jagged frontier, it just kinda motions at "Recursive Self Improvement" as a binary yes/no toggle that somehow solves everything like magic.

22 more comments...

No posts

Ready for more?