Richard Sutton on LLMs

So, I’m coming to the realisation that I don’t really have a problem with AI. I have a problem with LLMs specifically. And the problem is that LLMs don’t deal with reality. They get their input from us. Essentially, we describe the world with language — in a very imperfect way — and then we upload that to the internet. The LLM takes that data and forms a picture of the world from that. But it’s a picture from a picture. The LLM doesn’t act on the real world. It can’t, because it does not understand the real world, it only understands language — which, anyone who’s studied linguistics for a bit will tell you, is a very imperfect representation of the world in the first place. But there turns out to be another way to build AI. It is called reinforcement learning, or RL, and it could probably be used to build something that is actually useful. Because it operates on a picture it has built of the actual world.

I came to this conclusion by watching the following interview with Richard Sutton, a legendary AI researcher and major proponent of RL. It is well worth a watch, the whole hour of it.

The two publications referenced in this video:

• The Bitter Lesson, Richard S. Sutton, 13 March 2019
• The Big World Hypothesis and its Ramifications for Artificial Intelligence, Khurram Javed & Richard S. Sutton, 2024