alt Hacker NewsThere's probably some quantization and other processing needed to get Sol to fit on a Celebras chip, so it's not an exact 1:1 copy of the Sol model running on Nvidia GPUs, because it's not running on GPUs doing abstract math calculations anymore, so they can run benchmarks at them and say it's similar enough, but no one believes in benchmarks in this new field of performance that's more computer astrology than computer science.
Gotta build some personal benchmarks if you don’t trust the public ones. But the well-known public ones, despite their flaws, are generally high signal on model intelligence.