Google has TPUs, a frontier model, a completely separate and lucrative revenue stream they can call on at will, and teams working on multiple different language modeling strategies simultaneously. Did I mention the vast and ominous data centers that already serve a significant fraction of the internet? If that ain't a moat, then what exactly is a moat?

> Nobody has a moat.

Custom hardware, data centers, huge cash reserves, deep/broad talent pool, and non-AI customer base are all huge advantages if not moats.

Google, Microsoft, or Amazon are more likely to be the AI leaders than OpenAI or Anthropic.

The whole winner-take-all idea seems entirely based around Singularity/Rationalism and would require massive advances that we probably aren't close to at all.

> Nobody has a moat except nvidia

For now, for cloud training. but for consumers, nvidia vs amd reasonably close - the moat there is thin and shrinking. I suspect AMD will surprise us. nvidia has no motes in china, which may be a new source of (gpu) chip design. Huawei's Ascend 910C is about a generation behind... again: for now.

point is: moats dry up. I see nvidia's shrinking as a real possibility.

It's even worse, we are crossing over into the realm of religion. The article against GML 5.3 is the equivalent of a Papal excommunication.

My personal theory is (assuming there really is no moat) whoever starts the latest with developing AI models might actually win as they should be able to develop a competitive product with significant less resources and initial investment resulting in a higher ROI. AI might even become a commodity.

And if nobody has a moat the current valuations and costs are not going to be justifiable in the slightest.

The US companies still have trillion dollar valuations like there is a monopoly. There just isn't one. They are all within a few percent of each other on the benchmarks.

The slightly lower Chinese open models are good enough for almost everything, too, and much cheaper. Like with humans there is plenty of employment for people with below genius level IQ's.

The strongest moat I see is ownership of training data - an area where Google seems to have a clear edge

Even Google itself stated (internally at least) that nobody has a moat https://newsletter.semianalysis.com/p/google-we-have-no-moat...

Isn't the important takeaway here that Gemini 4 is not released and has no planned release date?

This is marketing from Google, not a competitive offering

Nobody has a moat, but everyone is f*&$ed. Competitors do seem to leapfrog each other, but each step is getting closer to beating humans at most tasks. Once that happens, that do we do?

I think that scenario only naively made sense if technical knowledge was entirely proprietary and talent was guarded with severe non-competes and NDAs

And Chinese labs openly publishing so much of their methodology destroyed any hope, which was inevitable

> The famous theory of Dario Amodei was that AI was this winner-takes-all field where the first team to get a head start would never cede ground back.

This is the kind of story that ones tells to investors to justify the huge amount of cash burn. :-)

And before him, Altman was explaining very calmly that no company could ever compete with OpenAI.

I'm not necessarily defending this obvious marketing speak but maybe the "starting point" was wider than assumed. So far, nobody has caught up to US and Chinese labs for example despite lots of funding in Europe. This is also despite abundant in-depth research papers being published alongside open source code and weights by some Chinese labs

Google said as much shortly after GPT3.5 was released. There is no moat.

With that said, there are some other moats and people are building them: training capacity, inference capacity, brain capacity (literally buying the best researchers and keeping them tied down), harnesses/subscriptions, etc...

So far, yeah. It doesn't eliminate the possibility over the next 5-10 years of the AI race that we wont encounter a scenario that results in a well positioned lab making a clean break

I think some in the AI industry drank their own Kool-Aid. They believed that if they had the best model and the most compute, they could tell the model, "Make a better model." And it would, and the next one could make its replacement, and so on.

So far, that's not exactly how it's played out. Humans are still necessary for the leaps in capability or efficiency. A model can grind on a problem to eke out the most performance, and models can synthesize data and iterate on various techniques to find the optimal combination. But, seems like humans still have to provide the real thinking, and the talent and drive for doing that is not concentrated in one company or city or even one country. And, (surprisingly) a lot of the people involved are in it for advancing the field more than making another billion dollars, so they're publishing their research.

So, yeah, the moat isn't deep. Even the compute moat, that OpenAI, Musk, and a bunch of other also-rans (like Oracle) bet the farm on, isn't really panning out. The Chinese makers just spent their effort on making models vastly more efficient, since they couldn't do anything about having an order of magnitude less compute available.

I just find this unlikely personally, think about the great research that's happening in the open source world, I'm sure inside anthropic + openai they've also made a bunch of discoveries and improvements (and I'd guess way more due to them attracting the best talent + the better internal models they have)

Was Dario's company winning at that point in time by any chance?

It is too early to declare that there is no moat. I think there is and we will eventually arrive at a monopoly or duopoly at the frontier.

Seems like learning rate velocty may be the ultimate moat

I have never understood the whole "this is a winner take all game" mentality - the sheer size of the pie is so great that from a purely rational standpoint companies should just be trying to productively get a slice of it and be profitable. winner-take-all is just greed/capitalism run amok, where it is not enough to be profitable, you have to own the entire market (and presumably extract rents)

I feel his theory depends on the premise that access to pure compute would the be the determining factor of success. Not the case

> Nobody has a moat. duo to LLM mislead as AGI, then famous theory is still hold, just not for LLM

Whoever gets to RSI first “wins” but also maybe ends life on earth. The incentives have never been worse.

It's hard to make predictions, especially about the future

Nobody has a moat so long as employees can move between companies

Dario has been wrong about a lot of things tbh.

It's crazy anyone ever thought there could be a moat. Almost all the research was/is published in the open. There's no secret trick, no hidden method. LLMs are a commodity technology. Anyone can read a paper, write software, and train models. How do people think llama.cpp works? It's not magic... it's software.

Hardware is the real differentiator. Not everyone has billions to make more advanced chips, and only a few companies can make them anyway. Both OpenAI and Anthropic would already be dead in the water if we had cheaper GPUs, because we'd all be running open models on local machines with 8 graphics cards. They're gonna have to force a hardware shortage to prevent a collapse in 2-3 years. My guess is it'll be tariffs or import restrictions or licenses to buy newer hardware.

The present leapfrogging is not a contraindication because companies are not necessarily releasing their best models; we know they have smarter internal models. Furthermore, humans are still involved in model creation. Human involvement is expected to decrease over time, and when model iteration is completely automated, progress will happen at the machine's pace, leading to runaway intelligence, barring any ceilings.

AI is a commodity. One that is showing to be more readily commoditized than most has anticipated. As of now, the only moats are the financing for the hardware to run it and the hardware vendors themselves - with the latter largely not yet a commodity because of ecosystem lock and a limited capacity of the most advanced fabs in the world.

Lol you are talking about Google here

And yet we're still no closer to AGI. Make it stop

isn't he explaining capitalism though? It's almost like he is...