Stateless AI Fails Developers, and Token Maxxing Makes It Worse

The AI ​​industry has begun to confuse utility with intelligence. Windows main context has been a feature battle. Additional tokens are a token of sophistication. Quietly, the use of tokens has been a proxy for progress.

That should worry us.

We create general AI systems that repeatedly ask the same context and use the computer to solve problems that have to remember how to solve them. The result is a pattern emerging groups now describe as “token maxxing”: treating higher token usage as evidence of deeper intelligence or better productivity. That’s not the case. In most cases, it shows the opposite.

A stateless system is not smart simply because it creates more work. If anything, overuse of tokens often indicates that the underlying structure of the model is failing.

I’ve seen this pattern before. We once measured engineering productivity by lines of code written. Then we learned that more code means more complexity and more ways to break systems. Mature engineering organizations finally stopped leaking and started
rewarding beauty, efficiency, and reliability instead. I believe AI systems are headed towards the same calculation.

Stateless systems create artificial activity

Currently, many teams are building workflows where the modeler spends more time reconstructing the context than solving the actual problem. All commands start from zero, each session requires a history to rehydrate, and orchestration layers inject additional context and tools just to recreate
to understand the model already needed five minutes ago.

Ask a coding assistant about a bug you fixed yesterday, and we act like the conversation never happened. You attach the structure of the repository to many information because the system has forgotten. You repeatedly define the same internal APIs and rewrite commands, not because the function has changed, but because the model has lost its thread. Then we wonder why the token count is exploding.

A working paper from the Stanford Digital Economy Lab claims that AI operations for an agent consume 1,000x more tokens than a regular code conversation, driven by input tokens – because the agent has to re-read the entire conversation history before every action. This creates a dangerous illusion. Teams are beginning to believe that the increasing complexity of interactions itself is proof that rational thinking is possible. Big data and orchestration graphs look complicated. The use of large tokens is starting to feel like a computer priority. But usually, the program just compensates for the missing memory. And the person on the other end, the developer, the customer, the end user, is the one who incurs those costs in slow responses, broken context, and frequent restart interactions.

A surprising amount of what is marketed today as “agent intelligence” is contextual reconstruction. A workflow that requires multiple agents and rapid iterative injection just to answer a decision query does not measure intelligence. Measuring inefficiency.

The main context windows are not the same as memory

This problem becomes even more obvious in business environments where AI systems work across different tools, codes, tickets, documents, chats, and applications. Without solid memory, every interaction becomes an expensive recompilation exercise.

Ironically, software engineering has solved versions of this problem over the past decades. Databases do not compute everything from scratch for every query because rebuilding the context continuously is inefficient, expensive, and unnecessary. Yet many AI systems are as effective as goldfish with big names.

The current obsession with context windows risks making this worse. Increasing the amount of information a model can use is useful, but in the main context windows are not the same thing as memory. Feeding multiple tokens to a stateless system doesn’t magically create continuity. It simply increases the short-term information the model has to process before forgetting it again.

In their Tokenomics paper, researchers from the Data-driven Analysis of Software (DAS) Lab at Concordia University found that input tokens account for 53.9% of total usage, costs created by re-reading the collected context, not generating new responses. Developers should be careful not to confuse temporary content collection with long-lasting intelligence. At some point, developers will stop asking how many tokens a workflow uses and start asking why it needs so many in the first place.

AI development becomes a systems design problem

Instead of treating AI as a creative problem, we need to start treating it as a systems design problem. The important questions are very different. How do we reduce negative thought cycles? How do we maintain persistent context across sessions and preserve codebase memory over time?

These are questions of infrastructure and buildings. It’s not fast developer tricks. In my experience, teams making real progress have already figured that out.

Functional AI systems will likely begin to look less like chatbot assistants and more like memory-aware computer systems. They will maintain relationships between decisions, code changes, events, workflows, and performance history. They will understand the progress
without requiring developers to redefine everything over and over again. Most importantly, they will move the value equation away from the volume of interaction and towards the quality of the result. Because developers are not paid to create tokens. They are paid to solve problems.

The future belongs to systems that remember

The current cycle of AI rewards work more for visibility than results. I see organizations celebrating AI work rather than engineering results. Teams increasingly measure progress by the volume of interactions: more information, more orchestration layers, more agents, and more production. In some cases, developers spend more time managing AI than doing core work — architecture decisions, product thinking, customer impact.

The best infrastructure programs are often the ones you don’t see because they eliminate conflict instead of creating an event. A truly intelligent development system shouldn’t require developers to constantly rebuild context, direct orchestration chains, or manage quick exercises just to maintain progress. To me, the best systems are the ones you don’t notice. They remember enough to stop asking the same questions.