Reverse-engineered Jev-like model

Any diffusion model is potentially a Jev in disguise: https://github.com/vllm-project/vllm/pull/57250

Runs ~0.2s per decision on my DGX Spark.

  10/10 programming language detection
  9/10 human language detection
  10/12 unit magnitude comparison

It (DiffusionGemma with the Jev mode) can also solve an ASCII maze.

https://x.com/harshagundal/status/2100044305536889015?s=20

> They were building in stealth for 2 years, I was building in stealth for 2 hours…

> Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.

You can ask this Redditor saying he made it. https://old.reddit.com/r/LocalLLaMA/comments/1wihgum/i_liter... I think.

From the readme

> A Jev-like model takes a piece of text and a list of N text options. It returns one probability for each option. It does this in one pass instead of writing an answer word by word.

I’ve read TypeSafe’s announcement, watched the home assistant demo, and still had no idea what it was. If instead those three sentences were in the announcement…

Out of curiosity and semi unrelated — why do so many of these projects with customized encoder-decoder setups use earlier Qwen versions like 2.5 and 3 and not the smallest 3.5? Purely the few 100m params, or something else in the latter’s arch or pretraining?

I've seen a lot of LLM uses that are really just zero/few-shot classifiers with a lot of extra steps, so it is interesting to see more models that are taking advantage of all the intelligence encoded in the latent spaces of these models with really efficient output. It feels like this is an under-explored area of LLMs right now and I'm excited to see what comes out of it.

Can play Doom too - https://x.com/vinnylarouge/status/2100281651930513460

I like it! I suspect Jev may have more going on under the hood, but I like the idea of efficient universal transformers

That was super quick.