It's pretty obvious that Gemini 4 Carbon is not token efficient. It also has relatively low tokens-per-second compared to other models. I can think of several reasons why this is the case:
1. Google has huge amounts of cash and compute. Their goal was to train a model that that has the highest quality & frontier performance, not create a model that's efficient to run and serve. They have other models for that, like Hydrogen.
2. Google created Carbon without caring for inference costs. DeepMind is subsidized by the rest of Google's empire. Selling inference is an insignificant part of Google's revenue. But for Anthropic and OpenAI, selling inference is their entire business model. Anthropic and OpenAI have enormous incentive to sell as much inference as they can in the most efficient manner. Google doesn't have this incentive.
3. They are one of the few companies that can be inefficient. Out of all the frontier labs, Google DeepMind is probably the least constrained by compute. Chinese labs have around 10 to 100 times less compute, so they must release highly optimized models.
Google's frontend web servers routinely time out connections from agy (the antigravity cli) when using Carbon, wasting tons of work. Like any effort level more than "low" is completely unusable.
It looks as if Carbon often takes more than 240 seconds thinking, which I'd guess appears as a silent connection to Google's load balancers, which then drop what it thinks is a stalled connection.
One hopes that Gemini's serving stack is smart enough to cancel these torn-down connections, but maybe they're not and this is leading to all sorts of ghost TPU load.
Disclaimer: We spend millions on GCP per month and have a dedicated customer engineer. Even they are horrified at the glacial pace of the Gemini fixes.
4. they are so behind that they can not create effective datasets like other frontier labs do. So instead they do all the tricks the Chinese labs do: using RL with the single purpose of benchmaxxing the AA score.
Look how all Chinese models, or haiku/luna class US models behave, you can see the same pattern: they think too much and only work well with a subset of tasks
It's 1. Carbon was announced the day of Google's earnings call. It's a PR stunt.
Google has a huge flow of user data; they were the first to figure out about the Transformer, yet they’re lagging behind in the race. This is very strange, although their flash models are quite good, they’re expensive compared to the Chinese ones.
So in short, as is usually claimed about Google products from bazel to tensorflow to Kubernetes: they may work better inside Google than outside of it.
(2) isn't right. Cloud is on track to become the largest source of revenue and a lot of that comes from selling inference.
Maybe they should be more efficient and sell fewer ads.
Also, Google uses their own TPU architecture rather than GPU/Nvidia.
1) "Gemini 4 Hydrogen is adapted from Gemini 4 Carbon." From https://deepmind.google/models/model-cards/gemini-4-hydrogen... . Also, the Agy team is pushing is that Carbon is a great planner, and then use hydrogen as the worker. They want people to use it.
2) lots of people inside Google use carbon. If they didn't care about it, it wouldn't have been pushed on all of Google.
3) Deepmind is competing with cloud and others. they are the only Frontier Lab that actually has to compete with other business units within an org for compute. GCP and others want that compute as well.
(Googler, opinions are my own)