alt Hacker NewsOne benefit of an open source one is that you can, as a large corporation, run it "locally" within your own data center. Even fine tune it.
Replies
How big is this market, self-hosting a model that requires 64 GPUs, H100 or better, with good interconnects between nodes?
I suspect the overlap of those that can afford it, and those that have the talent to manage it, is a fairly thin slice of the Venn diagram. Even the large corps are gonna be getting it from the inference vendors, or more likely Bedrock and friends.
Technically, "open weight" but yeah
The problem with hosting these right now is that it's not a known quantity like lots of traditional business software is.
You can more-or-less approximate what your build out spend for a data center hosting database software is going to be. Your book of business will require a given amount of revenue to pay it off, but once that's known, it's off to the races.
With AI, things are moving so fast and new business models are being tried all the time. You would be competing with some of the wealthiest companies in the world for data center hardware capable of hosting these models in a usable state.
We've gone with Claude somehow hosted through GCP Vertex AI where I'm at.
There are very few companies that would ever be able to afford to run it themselves. But it does give you the security that it's technically possible. Puts some limits on stuff like Moonshot changing their terms/conditions/policies