Nebius & DigitalOcean: Inference Efficiency

(NBIS, DOCN)

1. The Core AI Thesis

At the core of all AI theses there are two beliefs:

• Demand for intelligence is ~infinite — this still seems to be true, at leas there is no data to the contrary.

• Compute access = intelligence = value creation. As long as demand holds, compute keeps getting built.

2. However, The Market Is Moving Toward Efficiency

What is cutting edge in AI today? Multi-agent orchestration systems.

What is this? An efficient way to allocate compute. Its aim: get the best LLM for each job at the lowest price.

• How does it work: when you make an API call, you speak to a chief agent, which evaluates your prompt and delegates to, and manages a customized team of AI models to achieve your task as efficiently as possible. Same quality output, but faster and lower cost.

• The neo-Clouds are a play on this: they sell orchestration services. E.g. NBIS and DOCN both have this service.

  • Note that Anthropic also has a dedicated Managed Agents API, so the leading labs are also all going in this direction.

  • Another example would be the recent launch of Sakana Fugu Ultra, which sells orchestration as a cloud service (paid for / proprietary). However it is built on open-source, using open-weight models.

We think this is accurate:

“People who build agents, they even don’t want to consume tokens. They want to get the final results, the outcome of the agent.” … “They will not choose the model, they will not compare tokens from this model and tokens from that model.” — NBIS management

Inference Is Where the Action Is Moving To

There are lots of negative stories —e.g. UBER / tokenmaxxing / companies burning through annual token budgets in a couple of months, etc. Because, guess what, a new tool doesn’t always get used for the right things. We are simply at the stage where costs are making people reflect on what is a good use of the AI tool-set.

• Agents and looping are the key trends

• Agentic workloads consume 15–30x more tokens than human chat sessions!

• Global inference traffic will grow >>10x by 2030 (ests).

Link to research