alt Hacker NewsIntroducing System One Models and Jev
Comments
Wasn't really till seeing this home assistant demo they have (https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4) that the value really clicked for me.
Seems really cool.
This, combined with contracts, could make a lot of things so much fun now!
For those who don't know (which is probably everyone but me), I ported the design-by-contract pattern in Python and combined it with LLMs. This was early 2025. I originally wrote about it here: https://leoveanu.com/2025-03-01-dbc/ . Contracts are a core feature of SymbolicAI ever since. The community seems to have loved it too (https://news.ycombinator.com/item?id=44399234).
I think I'm starting to glimpse the implications and it's gonna change agentic workloads if it holds up to scrutiny. It's too early for me to tell anything other than jot down some rough thoughts.
In short, you get blazingly fast semantic branching you can use in control flows. For contracts, I can now directly take the data model that you have to design and convert it into Jev's expected format. Or I can use Jev for semantic branching in postconditions.
If my understanding is correct, that should be doable, but I need to think more about it. It could be that with Jev I can finally “compile contracts” and better chain them into workflows, which is something I always wanted but didn't know how to do properly.
Eager to test. On the waiting list.
This is a very promising idea - a model that takes arbitrary text input (which can be a complex json), plus a set of questions (yes/no, multiple-choice, or score) and quickly (milliseconds) and cheaply ($0.042/MTok) answers those questions.
Unfortunately, none of this is explained in the announcement, but the documentation [0] is pretty good.
[0]: https://docs.typesafe.ai/concepts/how-to-build-with-system-o...
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle.
Oh, I have one of those use cases, matching people in genealogy trees. You can ask all sorts of questions: do the names match? Do they match within some edit distance? Do they match according to soundex/ metaphone rules (which are themselves a ginormous set of rules for letters and letter combinations which may or may not result in the same sounds, hand-coded as a huge if tree by a linguist not a programmer)? What about their relatives, do they match by the same rules? Should we incorporate domain knowledge about local naming customs? Etc etc.
I pointed a coding agent to this problem, and it aggressively started coming up with complex scoring rules and testing them against real datasets. Which led to sort-of acceptable results, but it still missed lots of cases which were obvious to a human, and had false positives which were obvious to a human. Which I could trade off, and slightly improve, with more back and forth with the coding agent.
Pointing a good LLM to all the information about two people, would of course give great results. Maybe even better than human judgment. But I can't do that for 100000^2 people, it would be too expensive in all sorts of ways. I need a fast, reliable scorer. I could maybe train an embedding, but that would be a huge job and where would I get the quality data?
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
Side note: it took me more time than I would like to admit to realize that Diogo Almeida isn’t a satirical version of the name Dario Amodei
Looking at the example Jev use cases, it almost feels like Jev's incredible cost/task can make it competitive as a generalized "poor man's ranking" algorithm that can be useful for lean startups or any fast paced development org.
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto
I'm trying to parse it down to what we had before vs what is new here.
We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had
1. "unstructured state in, probabilistic decisions out" 2. "orders of magnitude faster and more efficient"
What was hard there was to train the model head without ML expertise, and considerable amount of data.
This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.
If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.
> Input tokens: $0.042 / MTok ($42 per billion tokens).
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.
I think this is a great direction -- for some kinds of users. And this makes me wonder if the 'vs' framing is misleading.
Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.
So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?
I am not an expert in this domain but as an engineer-turned-researcher, this looks a lot like GliNER with a fitting harness.
This is something I focus on in a bunch of my experiments - how to get immense value out of tiny models (<1b params). There are lots of different architectures out there and there is so much to optimize if you know what you are asking and have a grammar to constrain with.
Great to see this and I hope this is a lot on top of what is already openly available.
There's a whole lot of information on this page that doesn't tell me anything about what this actually is. Can anyone spell out what the architecture is here?
They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they emphasize parallel generation, so is it like a text diffusion model?
Wouldn't that be fun for text adventures? As you can get probabilities for finite actions relating to objects that are in the world/room
Is this the reverse of LLM? Ie, "what's the capital of France?" LLM picks from "Paris" 99.9% / "London" 0.001%, LLM then with some randomness output you "Paris". For Jev, you ask it to give probability of a set of answers "what's the capital of France?" choose from answers (Paris/London), it then gives you (99.9%/0.001%)
I am confused why they say it is not an LLM and then in the documentation it is shown as being an LLM derivative. The documentation makes it sound like they're taking a pretrained LLM and then giving it their unique post-training. How is that not an LLM?
FAQ: Is Jev just a smaller LLM?
Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.
Image in documentation: https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-prime...
https://x.com/completeskeptic/status/2099925682726002904?s=4...
The doom demo is quite cool
This is a good product but the naming/branding is pretty unfortunate.
Typesafe.AI sounds like some typescript/structured output type of tool…
What even is “system one” ?
IMO the product/tech is really there, just needs better communication.
what is the…epistemic status, for lack of a better way to put it, of the probabilities? what do they mean? what (probabilistic) guarantees do we have about, say, the responses to
- is the capital of france paris?
- it is august. is it raining in paris?
(forgive the examples; they're probably not semantically the sort of thing jev is trained to work on. but i figure the point translates to various kinds of questions that come up in "inner loop of agentic pid controller" contexts)
a normal text-generating model if asked to produce a number will also do that just fine. i assume in jev's case it was actually rled to essentially learn to express priors over things using its implicit world model, which definitely ought to help, but can we say more?
Just trying to validate my understanding - so this is a Large natural language classifier, a general purpose or zero shot classifier ?? it can recognize entities, can classify text into some pre-defined classes ? right? or did i miss something amid all the marketing terms such as system one or RLCD or whatever??
It could be used for coding if you gave it an AST.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
It doesn't produce text -> it does not hallucinate statements
This is kind of trivially satisfied and they make this sound more extraordinary than it is. Also I suppose it can still hallucinate in the sense that for out-of-distribution data it will give miscalibrated probabilities.
Anyway a great step in the direction of calibrated AGI
This has the potential to be huge for computer use.
OpenAI has been teasing how fast computer use is with their models running on Cerebras chips but the difference here is a burning hole in your pocket.
Cool approach, i think less latency and cost is the way to go.
Here's how this would have likely been made.
- Tiny transformer or equivalent model (maybe a few bn or so?), explaining latency and cost
- Questions are sent in parallel to multiple copies of it (I'm sure they're edge located)
- The model is post-trained for calibration in a wide variety of data (the recipe is relatively simple, and likely targeted on distillation of logprobs / confidence of a bigger model)
Notice how cost is ONLY for input tokens as output is merely numbers (few tokens) because input could be huge (questions and options).
At 0.042-per-million price they have, Astra estimates the model to be 3bn parameters.
One could replicate this by post training Qwen 3.5 2Bn. I expect people to do so soon!
Could you use this to build a proactive memory formation and retrieval system for LLMs that runs lightning fast?
Last 32k of connect + Summary of current task: Did we learn something useful here (true/false)? What is the category to file it under? Then notify the LLM to file it away.
What class of memory might be useful here? Model gives probability to each item in the list. Short description of all memories ordered by tagged class is used in the next round. Are any of these memories useful in the current context, such that they will inform the model and help in its task (yes/no)?
I’m sure there’s some fine tuning to be had, but this sure seems like the basis for a substantially better proactive memory system that works around an existing LLM conversation.
If I’m understanding what this does and how this works (generic input, intelligent classification with probabilities, rapid and cheap), this is absolutely nuts.
Wow, this is really cool. If this holds up to scrutiny, and has a decent context window (+16k), it suddenly changes our project's status from "cool concept, too slow and expensive to release" to "doable", just like that.
Just joined the waitlist, excited to try it out!
I wonder how many choices you can give this thing in multiple choice response mode. I'm guessing you could give it enough choices to produce Turing complete programs one symbol at a time when running in a loop, or to hold a conversation when given a vocabulary as the choices. It would be particularly hilarious to just let it choose the next ASCII character of output in a loop.
My guess is that due to its design there is no support for prompt caching, as there would normally be no reason for it. So the performance of my idea would probably be appalling as every step in the loop would reevaluate every input and output token. But it would be interesting to see the outcome.
This is actually pretty cool. I think the undertalked about part of this for TypeSafe is that they can always "extract"/distill the frontier of this type of task from the newest LLMs for cheap. Jev seems seems to be GPT-6-Astra/Fable 5.1 but I imagine a bunch of training data is from earlier models?
Then, you can serve it faster/cheaper than the frontier LLMs. It's basically distilling a small but extremely common use-case from LLMs and serving it. Then RLCD comes into play to update weights when a new model comes out, etc.
Any thoughts on what the next potential "cheap" win to be distilled from frontier LLMs is? I'm going to need to play around with this.
Is this fundamentally different from other text-based LLMs, or is it the same except with special reinforcement learning a safe guards around generating valid types?
Surely it’s still generating some kind unstructured data internally? For example, what if I told it to generate a short story, but the short story is output as a JSON string?
They never show exactly how they use it? Only a bunch of animations of it 'working'. Would like to see the actual code used for the demos!
Can I put it as Air Traffic Controller? With similar error rates as humans?
That would be the litmus test.
"Does not hallucinate" is not the same as "is never wrong".
So the ATC test could be the benchmark.
IIRC, Carmack was working on getting AIs to play Amiga games. The Doom demo suggests a very interesting direction to take this research.
I'm curious to hear his take on this approach.
We've already started using it for some pretty powerful decision tree stuff. We're just scratching the surface. We shipped an extension for Swamp[1] a few minutes ago and the combination is great!
The one downside is that the context window is very small (32k.) So some initial ideas we had for initial evaluation of code reviews won't fit yet in the window.
> [others] Output tokens: ~5x more expensive than input tokens.
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
Very cool! Can you explain when I would use this vs. training a standard ML model on my data? Suppose I had a fraud dataset with features like customer ID, amount, merchant, online or in-person, etc. - I can't imagine that a general model like Jev would predict this more accurately or cheaply than even a basic XGBoost model trained on my dataset (one that I could build in a few minutes by asking Codex to build it). Where does Jev add value here?
Spent a lot of time - but this makes zero sense to me. It can, maybe, return type safe outputs faster than larger llms - but there is little reason to believe that it will be more accurate. It does absolutely hallucinate - and seems to me that the claim is largely misleading.
You architect your systems with typesafe - because it is marginally faster, but inaccurate - to do what ? You can just wait for the next version of LLM's to get more accuracy at the same cost - or just use a faster model right now from a different provider.
We had early access and found it to be pretty useful. Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models. [0] IMHO This type of model works incredibly well in concert with LLMs, not as a replacement.
[0] https://goodstartlabs.com/research/verification-is-the-bottl...
I think what this shows is how important branding and comms are. They've captured imaginations with their demos and nomenclature, despite the arguably non-novel architecture. One forward pass, read the embedding space, train some regressors on predicate structure, [??]
This is too much for me. ML playing doom was a thing since before LLMs, decisions tree were always insanely and no one ever used then anyway, i can't see anything new in this yet everyone is treating this as a revolution. This technology was always there and quite easily accessible all along.
The Doom demo looks impressive but was it a fine-tuned model? It's the difference between a cool demo and revolutionary tech.
I was previously working on LLMs to extract key info from data rooms for energy assets, and this looks great for that use case.
"Does this contract contain ____?" is a pretty typical query for many industries, and then you can have follow up questions that nest down into further info about X, Y or Z thing.
Looks really good for that use case, especially with certainty as part of the output, as you could flag things that didn't have high enough of a certainty to human review.
I'm sure legora and the other legal AI tech softwares are all over this.
Haven't seen any docs or so. Is this actually a general model, or does it need training on the the data set it answers? Finding it suspicious you never see some kind of prompt.
Edit: never mind, found https://docs.typesafe.ai/introduction/quickstart by now
When we speak about good judgement in models and agents we are talking about humans skills such as critical thinking , judgement and decision making , emotional iq, mindfulness etc. We have been building since 2018 a structured good judgement data lake tied to ten core humans skills and sub skills levelled queaisn and answers against blooms taxonomy from a tagged community of experts , outliers , contrarians . Open ended situational questions and answers like how the real world operates in various sectors to capture tacit knowledge . We have a playground with a tiny slice of just 100 of our over 500k base good judgement scenarios that can then be synthesized across sectors and workflows . With just a tiny slice it outperforms Fable and Open Ai models. Exceptional human judgement outperforms consistently models…. Good judgement is just also rare in us humans like common sense ( no pun intended) . Check us out at lovelyhumans.ai. Holler if curious. Sallyann Dellacasa on LinkedIn .
How is that different from machine learning 101 "regression"? And why don't they just put a regression or softmax head on top of a trained transformer? (or do they?)
First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.