logoalt Hacker News

Inkling 2 Mini

110 points • by jasondavies • today at 8:12 PM • 9 comments • view on HN

Comments

montroser • today at 8:40 PM

This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit:

Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha!

There are two particularly interesting aspects of this model for local users:

We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization.

This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :)

This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations!

https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin...

jakswa • today at 8:44 PM

When inkling first released I tested it out and thought it was the best prose from an LLM in my quick sampling -- easy-reading and easy to understand. Big hopes I can write code with one of these models one day.

mattnewton • today at 8:21 PM

Benchmarks look good, and inkling 2 small felt better than the benchmarks suggested, so I am very excited to play with this.

From a research perspective, I'm even more interested in the varlooped variant they're releasing https://thinkingmachines.ai/model-card/tml-d24-varloop-resea... I'm not aware of any other working open source models like that.

➕ show 2 replies
neya • today at 8:32 PM

Wow, that coding demo was super impressive. Can't wait to try it out. Just need to wait for the hardware market to cool down a bit so we can run these on our own commodity hardware.

jakswa • today at 8:41 PM

The masses lying in wait for Qwen 4 cry out in joy. A savior? Could it be?!

walrus01 • today at 8:27 PM

Looking at the link to quantizations it looks like unsloth has already quantized it:

https://huggingface.co/thinkingmachines/Inkling-2-Mini

https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

In terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B.

christkv • today at 8:33 PM

I'm loving this trend. My framework desktop is looking like a better and better investment.