logoalt Hacker News

bel8 • yesterday at 5:17 PM • 3 replies • view on HN

So it's a Fable class LLM?

                             DSV4Pro vs Fable5
    HLE w tools              60.0 vs 63.0
    Terminal Bench 2.1       87.9 vs 88.0
    Cybergym                 83.3 vs 83.1
    DeepSWE                  62.7 vs 70.0
    Toolathlon-Verified      74.1 vs 77.9
    AutomationBench (Public) 31.8 vs 29.1
    DSBench-FullStack        71.1 vs 77.2
    DSBench-Hard             67.2 vs 68.3

Replies

eli • yesterday at 5:20 PM

Fable's guardrails would never let it do something like Cybergym so at least for that one it's measuring Opus 5

➕ show 1 reply
nikcub • yesterday at 8:24 PM

that DeepSWE result is likely most indicative of how you'll find real world usage

aftbit • yesterday at 5:19 PM

Fabble lol

➕ show 1 reply