alt Hacker NewsOpenAI unveils its first custom chip, built by Broadcom
Comments
Probably obvious but still omitted in the OpenAI post: chips are being made by TSMC [1]. Wasn't sure if Intel got it.
1. https://www.investing.com/news/stock-market-news/openai-unve...
This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level.
What's everyone think of Taalas?
They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins.
Super fast demo live at: https://chatjimmy.ai/
https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...
I wanna see an inference chip where the weights are part of the rom of the chip.
There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle.
That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second coming out the output bus.
Downside is this chip would be huuuuge - a whole wafer.
Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.
Due to the speed the industry moves, you'd want to race from model weights to production super fast, make 50 wafers, use them for a year, then bin them when that model is obsolete.
Pretty huge move. Google and their TPUs are looking infinitely more prescient as I think they are on their 7th generation, along with the offshoots it inspired like the LPU and even others, perhaps like Cerebras and their Wafer Scale Engine.
However, based off first impressions, it seems like this is meant for inference side, and not training, which is also an interesting choice.
With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old Dell desktop. Think that's crazy? Look at the size of the first hard drives. The IBM 350 was a disk with 50 platters, 24 inches in diameter, that held 3.5Mb, and was leased for today's equivalent of $35K.
https://www.computerhistory.org/storageengine/first-commerci...
Compare that to a multi-terabyte ssd. Now apply that improvement to how an LLM is architected and run now. With AI assisting, it won't be long before a leap occurs and these data centers with all their current ultra-cutting edge Nvidia cards are nearly obsolete overnight.
>designed for initial deployment by the end of 2026 and expanding in the years ahead,
So after the IPO and will be featured heavily in the IPO sales brochure as a future promise?
I'm sceptical over any pre-IPO announcements.
I haven't seen this discussed here:
So far, the accelerator is showing cost savings of roughly 50% compared with typical AI graphics processing units, Broadcom Chief Executive Officer Hock Tan said in an interview. - [0]
50% cost saving. The picture changes so quickly, there are still a lot of low hanging fruits, that I find any discussion about whether a vendor has moats, or if they can recoup investment, is moot and futile.
[0] - https://www.bloomberg.com/news/articles/2026-06-24/openai-an...
Microsoft, Google, and Amazon also do this, but they also have the hyperscaler datacenter infrastructure to host the chips. Designing and taping out the chip is one thing, packaging, cooling, deploying, powering, and managing the fleet is another stack entirely. Wonder where that will come from?
I had Opus 4.5 design an LLM inference engine in verilog, including firmware and automated verification a while ago: https://github.com/cpldcpu/smollm.c
It's of course far from optical. But lowering the implementation through the abstraction levels turned out to be extremely powerful.
We’ve entered the “if you care about software, build hardware” phase of AI
> May we scale smoothly, exponentially and uneventfully through A[SI]
That sentence sounds weird to me. I can't really put my finger on why, maybe the combination of adverbs, or just the fact of writing the desire of scaling as a company so directly. It feels (to me) like openly claiming their selfish goals. Or maybe I am just misinterpreting and they are referring to the whole humanity as "We" (but knowing Broadcom and in a lesser extent OpenAI doings, I am not convinced).
I hope to see something like this, but in a small form factor like the NVIDIA spark.
I want a super fast LLM that is Opus 4.6+, like, in ability.
So I’ve been wondering about “one or two levels back” chip design. If I understand it, 28nm chips (pre EUV) is just about suitable to run (not train just inference) frontier models.
And so if I was a mid-level State would it be worth while to take my nascent chip industry and push it out to build a 28nm foundry and supporting eco-system.
The models will come but the real challenge of the future is having enough compute power for every one and every use. Even if LLMs don’t become AGI they will still be incredible tools - and as OpenAI seems to spend 8000 for each 200 monthly subscription building one’s own data centres seems sensible
I am not sure how much of the work is done by OpenAI, or whether it is basically a Broadcom chip specifically built for OpenAI models. It is a necessary step, but building a high-performance chip is not easy. Look at companies like Groq, Amazon, and Google.
This is starting to sound like startup scope creep. Instead of making the AI model it’s now custom silicon, web browsers, and consumer electronics?
This seems like more competition for Cerebras? Am I understanding correctly?
My question is: what will this do to Ceberas? It validates them, did they just have their lunch eaten?
This is another Cerebras? fwiw, it took Cerebras many years to finally get a handle on the yield and the cooling problem. Wondering if they just hired a bunch of people from Cerebras.
Two turkeys don't make an eagle.
I don't have much confidence in either OpenAi/Sama nor Broadcom, given past history. Again this is just pre-IPO shenanigans.
As credible as the "Datacenter in Space" claim by Elmo, before the SPCX IPO.
Why even "unveil" it? Seems like giving away competitive intelligence for no reason at all... other than hyping the stock?
„ OpenAI says early results show significantly better performance-per-watt than current state-of-the-art alternatives“
would be very interesting to see any papers/data around this
OpenAI is going to close the one thing it needs to be profitable : calculation power. Love this website : https://isaiprofitable.com/, shows who wins at the AI revolution. Nvidia wins because it has instant revenue, OpenAI is going to close that gap.
cheap token is more important now than ever. Chinese open weight model is getting pretty good. the real cost of AI adaption will come down to who (China or US) can provide cheap token for consumers and companies. Microsoft consider DeepSeek for their cowork is an example and now OpenAI with its own AI inference chip.
I mean I'd love to be able to buy something like the 17k tps taalas chip as a pcie or m.2.
Imagine when we can roar along at that speed, low power. Can just have the model reason for a while about anything and everything. It reminds me of the "race to idle" for mcus etc.
Word of Advice for OpenAI:
Never underestimate Broadcom’s ability to shaft their own customers
- VMware
- CA Technologies
- Symantec Enterprise Security
- Brocade
- LSI Corporation
*requires VMWare license.
Very interested to know the distribution of effort between the two companies. Is this truly a brainchild of OpenAI engineers or did they pay to white label and use a new Broadcom chip?
I'm assuming they used LLMs to (help humans) do custom circuit design. Even pre LLM there were various computer optimizations that didn't require humans like genetic algorithms. It'd be cool to see a paper on how they did it.
The only surprising thing about this is that they didn't do it three years ago.
Is broadcom really the best business partner? 100,000 VMware customers might say no.
The new chip sounds like it's vustom made to accelerate a few specific models they really need to run fast. The advantage is it's truly and ASIC, not a xPU. There are several new startups targeting EDA tooling automation, Chip Agents is the biggest one I can think of but their are smaller players too, Silimate is one I recall. These companies are focusing on building fast AI powered tools to speed up the tape out cycle.
There is a never ending torrent of money coming, so why not make custom chips.
Whoo ... party!
Broadcom will let the entire industry leverage the decade of research done for TPUs.
The AI business of Nvidia is cooked.
owow...what gonna be next.....thei own robot????
So this mafia is driving up RAM prices. And now build their own overpriced hardware.
Either RAM prices go down, or that mafia must pay us all compensation money for this cartel build up. Why is the USA protecting this? How much does the orange man profit personally from helping drive up the prices here?
aw shucks nvda has some spicy competition
Make sure you all use that fancy ñ
So this is where all the memory they bought is going to.
If it’s really a differentiator, why announce it? Why not keep it secret and make it a competitive advantage?
I wonder how close OpenAI is getting to using the memory they purchased. Are they planning to stack a huge amount of HBM2 into these chips?
Look at the SIZE of that chip.
Cerebras stock is down nearly 20% today.
Not only is approach overlapping, OpenAI is also Cerebras's only major customer.
'braodcom' ha ha ... it's not OpenAI's chip then ...
If this is something that will hurt Nvidia, I'm all for it
Fucking Broadcom?
The only time I've ever seen that name before is when trying to solve driver issues, on both Linux and Windows.
Are they especially stingy with their IP related to drivers or something?
> Developed from design to production in nine months, accelerated by OpenAI’s models
> the use of OpenAI models to accelerate parts of the design and optimization process.
I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors.
Like, if this was as big a deal as it kind of vaguely implies, they would be making a bigger deal of it, right?