Enterprise AI is defined by a new, expensive reality: the token economy. Tokens are an economic unit used to measure the inputs and outputs of large-scale linguistic models (LLMs). Since input data is tokenized and LLM responds with output tokens, companies monetize and price their applications based on this usage.
The plan has led to massive spending, with the Magnificent Seven companies alone spending three billion dollars to build infrastructure capable of supporting the massive gigawatt capacity required for multiple tokens.
The costs are compounded by the new LLMs, which are starting to cost more—some five to six times more expensive than their predecessors—because they are designed to spend more time thinking and consulting. This has become known as tokenmaxxing – akin to a sale you can get from a car dealer, which – after agreeing on a price, you are put in front of a dealer who wants you to add undercoating or rim protection to the final cost. More profit for them.
Jitender Aswani, senior VP of product at data platform provider Starburst, told SD Times that he calls LLMs doing this “overzealous.”
“They’ll answer your question, but they might give you a very verbose answer, which basically means they’re tokenmaxxing. Another way they increase their token output is they ask you, ‘I’ve answered your question, but are you interested in A, B and C?’
An outcome-based strategy
To deal with this problem of spending and diminishing returns, Starburst, an intelligence platform, offers a different approach. Internally, the company fights wasteful token usage by rejecting the idea of unlimited and unaccountable AI usage. Instead of setting quotas or competitive leaderboards based on token volume, Starburst’s strategy is based solely on results. They monitor the impact of AI discovery, not the number of notifications.
For example, a developer who spends a billion tokens to achieve “an amazing result,” Aswani said, “is more valuable than someone who spends a billion tokens with little influence.” The developer metrics that matter are developer speed and cycle time – how quickly an idea can move to stable, reliable production – not immediate volume or token usage. This ensures that major investments in AI tools are focused on moving the business needle, not just on increasing usage.
Access to different data without moving it
The core strength of the platform is its ability to access and integrate all of what Aswani calls the “ground truth” of a structured customer, which is often fragmented among 200 or more systems, without requiring data to be transferred.
“AI is only as good as the knowledge it can access,” explained Aswani. It will bring answers, which if it happens in business, you can end up making wrong business decisions or unsuccessful business decisions.”
As data has exploded and different types of data have emerged, this has led to fragmentation, with different types of data stored in different silos. Some large enterprises may have data spread across hundreds of systems, providing enormous potential for applications. “They have call center data, they have customer support data, they have customer experience data, they have product analytics data. A company like Bank of America or Citibank, you can imagine how many applications they have. Each program has data, and then ultimately that data needs to be analyzed to understand, what our customers are doing, what kind of questions they’re asking, what kind of conflict our customers are facing, the biggest business challenge they’re facing, the biggest challenge our customers are facing. they are different, but they need a program, an app or a platform like Starburst that can aggregate all that data without moving the data
Meanwhile, recognizing that a single model does not provide all business needs, Starburst offers solutions that help contain the use of tokens by expanding the use of the LLM ecosystem.
Starburst’s orchestration layer allows customers to “deliver your own LLM”. This gives businesses a choice, allowing the system to decide which model is best suited for the job. In simple conversations, the orchestration layer may choose the least expensive model, while a different model may be chosen for summarization or multimodal input. By matching the right tool – and its associated costs – with the job, Starburst helps companies control the use of tokens.
