logoalt Hacker News

enraged_cameltoday at 5:26 PM2 repliesview on HN

>> We have an internal eval that measures performance on tasks for a handful of embedded systems repos for our mmWave radios

Okay but the parent said real-world usage, presumably meaning coding tasks.

We have a whole bunch of complex evals that Haiku 4.5 passes. That doesn't mean it is a good model for coding.


Replies

spmurrayzzztoday at 5:36 PM

Yes, these are coding tasks in the embedded systems domain (I mentioned Rust and C).

graceful6800today at 7:00 PM

They literally stated in their first sentence that it was coding tasks.