Nebius & DigitalOcean: Inference Efficiency
(NBIS, DOCN)
1. The Core AI Thesis
At the core of all AI theses there are two beliefs:
Demand for intelligence is ~infinite — this still seems to be true, at leas there is no data to the contrary.
Compute access = intelligence = value creation. As long as demand holds, compute keeps getting built.
2. However, The Market Is Moving Toward Efficiency
What is cutting edge in AI today? Multi-agent orchestration systems.
What is this? An efficient way to allocate compute. Its aim: get the best LLM for each job at the lowest price.
How does it work: when you make an API call, you speak to a chief agent, which evaluates your prompt and delegates to, and manages a customized team of AI models to achieve your task as efficiently as possible. Same quality output, but faster and lower cost.
The neo-Clouds are a play on this: they sell orchestration services. E.g. NBIS and DOCN both have this service.
Note that Anthropic also has a dedicated Managed Agents API, so the leading labs are also all going in this direction.
Another example would be the recent launch of Sakana Fugu Ultra, which sells orchestration as a cloud service (paid for / proprietary). However it is built on open-source, using open-weight models.
We think this is accurate:
“People who build agents, they even don’t want to consume tokens. They want to get the final results, the outcome of the agent.” … “They will not choose the model, they will not compare tokens from this model and tokens from that model.” — NBIS management
Inference Is Where the Action Is Moving To
There are lots of negative stories —e.g. UBER / tokenmaxxing / companies burning through annual token budgets in a couple of months, etc. Because, guess what, a new tool doesn’t always get used for the right things. We are simply at the stage where costs are making people reflect on what is a good use of the AI tool-set.
Agents and looping are the key trends
Agentic workloads consume 15–30x more tokens than human chat sessions!
Global inference traffic will grow >>10x by 2030 (ests).
Comments