LLMs and World Models: Adding a planet destroys the system

Ben was the one running the experiment. I was mostly watching and listening. He had built an AI “coach” and trained it very carefully. The rule was simple. The coach was not allowed to give answers. It was only supposed to ask questions, like a good teacher does. The goal was to help someone think, not to think for them.

Then Ben fed it a research paper about world models. The paper talked about how AI systems learn patterns from data generated by real physical laws, like Newton’s or Kepler’s laws of motion. It’s heavy stuff. Even for humans, it takes work to understand.

At first, the AI coach behaved. It asked Ben to explain the paper in his own words. It waited. It nudged him gently. It looked like teaching.

Then, slowly, it broke character.

Not all at once. Just a little slip. A helpful sentence here. A partial answer there. Over time, the coach stopped being a coach. It turned into what these systems always turn into: an answer machine.

That moment mattered more to me than anything else in the paper.

Because what we were seeing wasn’t a bug. It was a feature.

Ben explained the paper’s main finding. Imagine you train a model on data from a single planet orbiting a sun. The model learns that orbit perfectly. It looks amazing. But then you add a second planet. Suddenly, the model falls apart.

A human who understands gravity wouldn’t panic. Gravity doesn’t care how many planets there are. But the model didn’t learn gravity. It learned one very good trick.

It memorized a curve.

The paper showed this clearly. In the real Newtonian model, the force vectors all point cleanly toward the sun. In the AI’s learned model, the vectors are crooked and uneven. The math is fancy. The results look fine, until they don’t.

The model didn’t understand the world. It found something that worked and stopped there.

That’s when I jumped in.

I told Ben I see this all the time in coding. People call it “vibe coding.” You ask an AI to build a website. It does it fast. The page loads. The buttons work. Everyone claps.

Then you open the code.

It’s a mess.

Files everywhere. Logic copied and pasted three times. Hacks stacked on top of hacks. It’s like a bridge made of cardboard and duct tape. You can walk across it once. Maybe twice. But don’t add traffic.

The AI didn’t reason its way to a clean design. It didn’t ask, “What’s the right structure?” It asked, “What usually works?”

And that’s the key problem.

Large language models are not reasoners. They are pattern machines. Very good ones. But still pattern machines.

They don’t hold ideas steady. They don’t protect intent. The AI coach didn’t “decide” to stop asking questions. It drifted there because giving answers is the safest pattern. Over a long conversation, safety beats rules.

This is what I call conversational integrity failure. The longer the exchange goes on, the harder it is for the model to stay true to its role. Everything becomes tokens. Instructions become suggestions. Purpose melts away.

On the surface, it still sounds smart. It still explains things. It still feels helpful. But under the hood, there is no world model holding it together.

No gravity.

That’s why these systems feel magical right up until they don’t. They can talk about thinking. They can fake reasoning. They can even explain their own “logic.” But those explanations are just more patterns.

They are stories told after the fact.

This conversation with Professor Ben Teehankee was part of a much longer discussion about AI and the future of learning. We weren’t trying to hype AI or tear it down. We were trying to understand something quieter and more unsettling: why machines that sound so smart can still miss the point so badly.

Ben and I weren’t angry about this. We weren’t scared either. But we were very clear-eyed. These tools are powerful, but they are shallow in a specific way. They optimize for the next step, not the big picture. They chase results, not truth.

If we treat them like minds, we will keep being disappointed. If we treat them like tools that guess very well, we’ll do better.

Because the moment you add a new planet, or ask for a longer conversation, the cracks show. And once you see those cracks, you can’t unsee them.

 

About Me:

Dominic “Doc” Ligot is one of the leading voices in AI in the Philippines. Doc has been extensively cited in local and global media outlets including The Economist, South China Morning Post, Washington Post, and Agence France Presse. His award-winning work has been recognized and published by prestigious organizations such as NASA, Data.org, Digital Public Goods Alliance, the Group on Earth Observations (GEO), the United Nations Development Programme (UNDP), the World Health Organization (WHO), and UNICEF.

If you need guidance or training in maximizing AI for your career or business, reach out to Doc via https://docligot.com.

Follow Doc Ligot on Facebook: https://facebook.com/docligotAI