By 2025, “AI agents” were positioned as the inevitable next step: autonomous systems that could plan, reason, delegate, and execute complex tasks with minimal human oversight. On January 6, 2025, Sam Altman wrote “We believe that, in 2025, we may see the first AI agents ‘join the workforce’ and materially change the output of companies.”
These expectations turned out to be overly optimistic. What we got instead was an MIT report, according to which “95% of generative AI pilots at companies are failing”, and, perhaps the more interestingly, the METR study according to which “when developers are allowed to use AI tools, they take 19% longer to complete issues”.
I think this is a symptom of a fundamental misunderstanding of what AI really is. A mismatch, which stems largely from the way people tend to interpret the term “artificial intelligence”.
In the original proposal for the Dartmouth summer research project on artificial intelligence, the definition of artificial intelligence is as follows: “For the present purpose the artificial intelligence problem is taken to be that of making a machine behave in ways that would be called intelligent if a human were so behaving.” This is, either implicitly or explicitly, the definition most people use or at least allude to.
It’s quite clear that this is not what we have at the moment. LLMs are able to simulate thinking to an impressive degree; enough for many people to believe systems like ChatGPT are actually processing information in a way that is somehow on par with our own way of thinking. I believe, though, that this is largely an illusion created by our own preconception of what it takes for a system to produce text that, at least on the surface, makes sense.
I think that, at the moment, we are racing to understand what these things are. The winner is the one who figures it out first, not the one who relies on existing assumptions. I believe that to win, one has to approach these tools in an open-minded manner: taking LLMs for what they are, and not what we want them to be.
