In the 1920s, a Soviet filmmaker figured out something we’re still struggling with in AI.
Lev Kuleshov took the same shot of an actor with a blank expression and cut it together with three different scenes, a bowl of soup, a dead child and a woman lying on a sofa, and people saw hunger, grief and desire, even though the actor’s face never changed.
The meaning wasn’t in the face itself, it came from what people saw around it.
Something similar happens with LLMs, where the same token can be interpreted very differently depending on the surrounding context, and while self-attention is obviously more complicated than film editing, I think the comparison is useful when building agents.
When an agent fails, the first thing we usually do is give it more context: more chat history, more logs, more tool definitions, more instructions, and if that doesn’t help, maybe a bigger context window will.
But I’ve seen agents fail with all the information they needed already in the prompt, and the problem wasn’t that something was missing, it was that the important parts were buried under everything else.
A 2-million-token context window doesn’t solve this problem by itself, it just gives you more room to put things the model might not need, while the one constraint that actually matters for the next action gets lost somewhere in the middle.
I think we still treat context engineering too much like a storage problem, when a lot of the work is really about deciding what the model should see at a particular step, what can be removed without losing anything important, and which pieces of information are actually going to change its decision.
You can give an agent access to almost everything and it can still make a stupid decision, because having the information available and using it at the right moment are two very different things.