Rendered at 18:47:52 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
pradn 22 hours ago [-]
I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.
We've stumbled into general differentiable models..
redox99 21 hours ago [-]
Because what they did is kinda trivial. Its basically like the Dropbox comment really[0], except here you don't need petabytes of storage and infinite VC pockets.
After chatgpt everything in AI mostly became LLMs and building wrappers around them. It's like people forgot how to do ML.
To those of us who actually trained models back in the day, its kind of cute to see people wowed by a classifier. Yes, this is 0 shot and doesn't need training (most people wanting this would've used structured output, this is cool because it's cheaper and faster). But anyone with basic ML knowledge could've built this in a few hours.
The question is mostly why wasn't this productized. And it's interesting indeed that it took this long to become a finished product.
Theorizing, but I'm guessing that before LLMs, people weren't using anything for situations like these. As people started using LLMs, people's use cases grew, LLMs got slower and more expensive, and people lost the wow-factor and are now worrying about price. Great timing to launch a product like this, where certain use cases can be distilled down into something faster and cheaper.
There's probably other areas where people are using LLMs where a more tailored ML solution might work better.
andai 19 hours ago [-]
> The question is mostly why wasn't this productized.
Probably because doing it wrong (using an llm in place of a classifier) is more profitable? (For the people selling inference.)
eadwu 18 hours ago [-]
The answer for why it wasn't productized might just be pretty straightforward.
LLMs still are better than Jev at the task, just across the board slower.
Anyone who had a reason to try this already tried it (ads/recommendations) - back in 2023/2024 during the first fine tuning wave and it was accurately determined that it was not worth the effort, the results were more bogus than just using CoT, so frankly parallelism meant nothing if bogus * parallel = bogus.
So thrown into the dumpster and nobody really cared to revisit because it was already tried.
Pretty much sometime between then and now it somehow became the state where the tradeoff makes sense now.
wongarsu 10 hours ago [-]
However in the last 2-3 years LLMs became a lot more efficient. Small models without CoT are still bad, but leagues ahead of where they used to be
Maybe it's just a case of the idea just now crossing the threshold into working just good enough to be worth it
coolThingsFirst 7 hours ago [-]
Can you explain how 0 shot classifiers work how come it isn’t trained on my data how does it know when issue is urgent lets say?
dotancohen 42 minutes ago [-]
This is my burning question as well.
I have 80,000 voice recordings to classify. Very few are in English, and the classes are not in English. Many of the classes are project names or other proper nouns. How could a system not trained on data specific to the problem possibly be expected to work?
nickstinemates 18 hours ago [-]
It wasn't until recently that demand for classifiers at this scale existed. Jev exists because LLM's exist. Without them it wouldn't be (as) useful
jmalicki 17 hours ago [-]
That's not at all true.
There have been tons of applications for this. People were using earlier LLMs like BERT for classifiers long before LLMs became viable chatbots.
janalsncm 16 hours ago [-]
I think GP has a point, though. BERT models have existed for a while but OpenAI made classification via LLM convenient and accessible for regular developers.
People didn’t know they wanted classifiers until OpenAI gave them a taste.
calebkaiser 12 hours ago [-]
I don't know if that first part is true? Classifiers were/are one of the dominant applications of classic ML and neural networks, especially in production. Even today, image classification, object recognition, language detection, segmentation models etc are still super common.
I think the hype with Jev is just that, while structured generation is great, LLM judges tend to kind of suck for precise classification. And the more powerful the base model, the more accurate they can get, but they get increasingly expensive/impossible to finetune. It was specifically the latency/price point Jev offered vs. the general accuracy it claimed that generated all the excitement. Plus the promise of cheap calibration (tuning).
"Jev exists because LLMs exist" is kind of a truism, as Jev apparently is literally a Transformer model.
nickstinemates 2 hours ago [-]
How many people were doing ML work pre LLMs? And how many are using LLMs now?
calebkaiser 2 minutes ago [-]
I think the answers are "a lot" and "a lot more"? But what I'm saying is that Jev's virality isn't because people didn't have access to classifiers before it. Jev's general claims about capability and performance vs. cost would have been a very big deal 5 years ago too. In a vacuum, the idea that you can get a general classifier that is very accurate across any domain and on any modality with minimal latency and a very low price point is wild.
In early 2016, Clarifai's core product was basically just an image classifer exposed via an API. And at that point, they'd raised $40 million--the same amount as TypeSafe.ai/Jev--and they were experiencing viral growth among developers + signing contracts with a bunch of flashy logos. The demand was so high that Amazon launched Rekognition and Google launched their similar APIs to compete.
The AI hype cycle and the number of people thinking about using AI certainly puts more wind at Jev's back, but even in an alternative universe where we don't have contemporary LLMs, Jev's core claims would be remarkable and there would be a big market for it.
Fordec 18 hours ago [-]
I think the idea of a "feature startup" is dead. What used to be a niche subscription business is now an individual Epic level of work. The smallest viable business becomes what two or three years ago was a mid tier enterprise. It is no longer "look at this tool I maintain", but "we take this specific approach using these hundreds of tools merged together to solve a problem in a specific way that nobody is going to compete with. Not because they can't compete if they wanted to, but that the competitions approach diverges in fifty different chosen ways that they are targeting a different market segment essentially."
I adhere to the idea that this is software's "Tower of Babel" moment where everyone just fundamentally ships things in completely diverging architectures, because creating a ground up architecture is no longer something that needs to be avoided for an economically viable business mode that in the past two decades would have otherwise incentivized people into industry standards. In a world where "taste" is the focus, single ingredients in the recipe aren't enough.
mattstir 2 hours ago [-]
> I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly
This particular "innovative" concept already has a rich, open research background. What Jev appears to have done is scale that up a bit and isolate good training data, which results in a great product but not really something impossible to imitate. The only major difference currently is that the open source decision models need to be fine-tuned as they're not trained off of the entire internet yet.
It a paradox when the article is claiming the prior art is absurd, but then goes on to analyse the one side and compare it to another for which most of the values (except scaling the concept) are unknown. And even for scaling, it uses the first, pre-laya instance to judge the limited schema, while overlooking that Laya is just doing this scaling. Important to note that prior art is not having built the exact same thing.
kensai 11 hours ago [-]
There is definitely more to the story. There is a huge financial interest for each side to discredit the other. Fact of the matter is, we still don't know who will prevail. These are cutting edge tech stacks and they were just released.
janalsncm 21 hours ago [-]
Presumably the training recipe and training dataset itself cannot be easily copied in a week or two. So if they want to shut down these competitor models they need to make it obvious how they are better than them.
rosegroove 21 hours ago [-]
[dead]
cung 14 hours ago [-]
Wait so a startup that didn’t innovate much should have a bigger moat? If their work can be copied in 2 weeks maybe they don’t have anything?
cobzilla 18 hours ago [-]
This is precisely why we need strict government regulation of open weight models. :wink:
tukHelix 19 hours ago [-]
Today we might need to evaluate “innovation” in a new standard, and have a different expectation for what innovator would be awarded. Getting public attention in such an era where innovation happens every a few days could’ve already been something precious. And that attention would allow TypeSafe to be heard easily next time. Like OpenAI, Anthropic, or any others, they launch frequently but still each time they launch something new, that would hit headlines. I think that’s the “surplus” flown to innovators today.
_menelaus 19 hours ago [-]
The moat is the RL synthetic data pipeline they set up to train jev. Open sourcing that would be the coup, not the model architecture and training scripts, which are trivial.
cjonas 12 hours ago [-]
Is it likely not just distilled from one of the flagship models? That sounds like an afternoon of work and a few thousand dollars in tokens?
scotty79 7 hours ago [-]
Didn't llaya come first? So the entire Jev's innovation is encapsulating idea in a cheap service?
locknitpicker 11 hours ago [-]
> I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?).
My thoughts too. It sounds like a minor feature being framed as a whole new business.
Then again, Dropbox and Docker are too.
fooker 18 hours ago [-]
For everyone dismissing Jev's innovation as being trivial, no it's not.
It is definitely not the MNIST classifier you had trained in 2019.
The difference is that you only train it once and the modern LLM machinery sort of takes care of that with large contexts.
It's great that Jev proved this is a viable product. I'd expect a great many research innovations coming from making this work better/faster/cheaper, and around interfacing modern agents with it.
hodgehog11 7 hours ago [-]
Just because it is zero-shot does not mean that Jev is an architectural innovation, especially in the year 2026.
Anyone fitting MNIST in 2019 was already outdated by several years at the very least. GPT-2 was 2019! We already had zero-shot classifiers then. In fact, the paper for GPT-3 was literally
"Large Language Models are Zero-Shot Reasoners"
These kinds of zero-shot classifiers were already developed and used in-house for many years. They just weren't commercialized as a separate product, because anyone who could use an LLM proper could build layers around it to fulfill any classification task like this.
fooker 2 hours ago [-]
To get a classifier that worked in 2019, you had to train a model. Either from scratch or from a starting point.
GPT2 was absolutely unusable as a classifier. Using GPT3 as a classifier cost a few order of magnitude more than what this thing is priced at, and the context window was a few thousand tokens.
> These kinds of zero-shot classifiers were already developed and used in-house for many years
This is like Google's favorite coping mechanism for falling behind at AI. "We had everything inhouse for several years, we didn't release it for $reasons."
> anyone who could use an LLM proper could build layers around it to fulfill any classification task like this
You missed the part where it costs more than two orders of magnitude lower :)
I'm not claiming there are major architectural innovations, but that's not the point. Once you prove there's a market, there's a cambrian explosion of innovations.
syntaxing 18 hours ago [-]
Hah you’re probably dating yourself. Keras came out in 2015 and that was one of the early examples with Theano backend. You could train MNIST since 2015 pretty straight forward. But comparing Jev to image classification is an unfaithful argument. Comparing it to ELmo or BERT is analogously better.
fooker 17 hours ago [-]
You missed the point - you had to train BERT or anything similar to get useful results out of it.
Now all you need is to give it more context along with your query.
c10o 14 hours ago [-]
Exactly. And there’s a lot of situations where context change frequently and fine-tuning / re-training becomes impractical or even impossible. With a general classifier + context, you can change the context dynamically and get instant results. That capability opens up a whole lot of possibilities.
k__ 5 hours ago [-]
But aren't BERTs tiny compared to a LLM and can be trained cheaply with the help of an LLM?
fooker 38 minutes ago [-]
Suppose you want it to make a decision based on ..say.. 300KBs of somewhat changing information per query.
There's no scenario where you are training BERT online to give you an answer.
george_max 24 hours ago [-]
Has anyone actually seen better or the same results with Laya compared to Jev? From my experience, Laya performs significantly worse. It's less confident and often makes wrong decisions with more complex queries.
jonmagic 23 hours ago [-]
I've been following jevbench twice a day for the past week and that's been a lot of fun. Latest update:
Rank System Score Public / sealed accuracy Evidence
Did anyone else notice the huge gap between scores on private vs public for ALL Jev-like models compared to LLMs (such as GPT Luna)? Doesn't it mean those models aren't generalizing so not very useful on data they haven't seen?
Havoc 22 hours ago [-]
Amazing - was looking for some benchmarks around this earlier
philipodonnell 22 hours ago [-]
What the best way to see how a homegrown version compares?
scronkfinkle 24 hours ago [-]
Yes. JEV generalizes better because they probably have an enormous corpus and trained on it for a long time. Laya's out of the box model is much weaker. However, in the age of LLM's it's incredibly easy and cheap to generate large datasets to fine tune laya for your task, and the training loop is pretty quick and cheap too.
It's so easy that I question why I would ever pay for JEV when eventually I'll have done enough random things that I will also have a large corpus and likely a general model as well.
mtkd 23 hours ago [-]
Isn't the point of Jev that it generalises better?
It's a fast classifier you can use out-the-box, ~1.5bn tokens is about $40 (I've been hammering it)
It just works ... a whole bunch of low-level/low-importance workflow stuff that was getting farmed out to small/fast LLM models now has a competitive alternative ... and bits that hadn't even been considered to go into some external descision/classifier service can be tested/deployed at ~$0.00003/req
I don't get this wall of negativity on it, it's genuinely innovative/useful tech ... would expect HN to be more positive, regardless of whether it's the absolute best execution
shepardrtc 23 hours ago [-]
It really does just work. And it works so well I already integrated it into my product. Saves me about 75% of costs for the section its working in, which isn't a small amount. I see a lot of negativity and I don't really get it either. Its so cheap and so fast, why not give it a try?
DenisM 23 hours ago [-]
I think it’s the infamous Dropbox reaction - anyone can wrap an FTP server, where the innovation?
Starting from a business POV one should inflate terminology, hack together an MVP, and see if the market demands it before doing hardcore R&D.
But starting from technical/craftsman POV all you see is a hack and a lot of big words, so it’s easy to become jaded.
digitaltrees 23 hours ago [-]
I think your point is valid but many are annoyed that it is presented as groundbreaking, revolutionary, novel frontier tech when it is a known classification system. It’s the hype that feels undeserved. Honestly it was one of the best marketing campaigns I’ve seen.
not_a_bot_4sho 23 hours ago [-]
I didn't see any negativity in the post you replied to.
I think the point being made is that Jev is great but it has no competitive moat, and open source versions will very soon catch up if their secret sauce is just synthetic data.
(Whether or not that is true, I don't know.)
killingtime74 14 hours ago [-]
It's probably true because even for Frontier LLM models there are many competitors now.
ichorio 20 hours ago [-]
If you don't mind me asking, what are you using it for?
I've been unable to find a good use case for now.
taylorfinley 16 hours ago [-]
You should try building something with it, the hype is what it is, but the model is crazy useful.
I'm building a woodworking app and I've managed to create an autopilot that can take a simple instruction ("get me 5 2x4s", "cut the middle 2x4 into 4 equal pieces", "move the 2x4 3 feet left") and the action instantly happens with next to no lag. There is already an llm but now it can share an intent, and the geometry system shows jev the various actions and jev chooses the action that gets it closer to the goal until it has found a state that matches the intent or gives up. The result is the llm can think "higher level" and let the cheap fast model grind out the options in a relative blink of the eye, without the 30s of reasoning the llm would have done about the various operations it could try.
_menelaus 19 hours ago [-]
If you're so inclined it would be easy, fast and cheap to distill Jev for your task.
cobanov 24 hours ago [-]
Developer here. You're right, Laya is a lot weaker than Jev, especially on harder queries. It's a small model, so it's fast, but that's the trade-off. The open models that get close to Jev are much bigger, and running those is what I'm working on next.
adinb 21 hours ago [-]
It doesn’t to be a ton bigger, 16k and reliable 8k would be a godsend. (I run at 2k)
mikodin 23 hours ago [-]
What are the models? I am super curious in these as well
simcop2387 22 hours ago [-]
Probably Kev and/or the decider models. Kev is trained on one of the 4B qwen models, similar for decider but it ranges from 0.8B through to the 35B-A3B model so far I believe.
cobanov 10 hours ago [-]
[dead]
cjonas 12 hours ago [-]
I've been testing, for my use case laya didn't come close to decider was just as good. My experience with the 3 models matches the result here
This is just anecdotal and I might be doing it wrong but I made jev and laya versions of a simple semantic grep tool (https://github.com/lgastako/jevplay) and played with them a bit, and at first it seemed like laya was comparable (eg on queries like "this is a mans name" or "this is a womans name" on names.txt) but the more I played with it, eg. "this is a vegetable" on foods.txt the further the gap widened in favor of jev. Then I started trying variations of the query eg simply "mans name" and for the most part laya just fell apart and didn't return anything useful for a lot of stuff. I was hoping to find that laya was competitive because it's much faster to have the model running locally but it's just not, yet.
jasonjmcghee 21 hours ago [-]
In my experience it's not close and the benchmarks I've seen don't reflect my experience at all.
But I'm guessing people will find the right training regime and data mix soon to close the gap.
But big things I see are instability and inaccuracy - like pick a random problem.
draginol 4 hours ago [-]
That's the thing I think some people keep missing.
I mean, anyone here could have a decision maker just return a random number. Fast and easy.
The question really is how GOOD is it?
verdverm 23 hours ago [-]
one day, perhaps people will click through to the laya author's arxiv paper content and the why may become clearer, you won't have to read it, a skim will suffice
iamflimflam1 23 hours ago [-]
Nothing yet. Unfortunately it sometimes feels like our industry has been overrun by grifters and chancers.
I’m sure this has been a gradual and long decline. Maybe it even started with the dot com boom and accelerated with crypto. With AI it seems to have got worse.
neop1x 2 hours ago [-]
There is also a small wrapper over an unmodified llamacpp binary which implements jev decision API on any llamacpp-supported model. It is different than Leya/Kime - raw label softmax, not calibrated! But can sometimes be enough I guess. https://github.com/NakliTechie/llamacpp-jev
alex7o 23 hours ago [-]
Guys I have a real q, what is the difference between an instruct based re-ranker and laya/jev I just don't see it.
Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?
Swizec 22 hours ago [-]
> difference between an instruct based re-ranker and laya/jev I just don't see it
Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).
Right now a lot of people are doing this with LLMs and it's too slow and expensive.
Imo the right iterative approach to productionizing these systems is something like:
1. Build it with an LLM. Iterate on the prompt
2. Start building a real-world dataset
3. When the prompt works, turn it into a clear rubric for Jev or similar
4. Keep iterating until desired accuracy achieved
5. Use the real-world evals you've built to train a custom classifier fine-tuned to your needs
You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.
janalsncm 21 hours ago [-]
I don’t think that’s it. I sincerely doubt most developers are doing side by side comparisons of calibration quality.
OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.
I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision/recall tradeoffs. They want something which plausibly works and is easy to use.
Swizec 20 hours ago [-]
Yes it turns all that work of building a classifier into an api call. This is hugely valuable for prototyping and while you iterate on what the product should even do.
rosegroove 21 hours ago [-]
[dead]
kakugawa 21 hours ago [-]
Jev's value becomes more apparent when the task is a moving target. eg an auto-mode classifier.
c10o 14 hours ago [-]
Exactly. Just think about a discord or twitch moderation bot that screens messages in real-time. When the streamer starts playing a game, the whole context switch in an instant and comments like «kill them» will suddenly have a whole different meaning.
alex7o 11 hours ago [-]
Yea but a re-ranker can do that too if instruct trainrd. You update the instruction on which it is ranking the the values, a.k.a it is classifying them.
avereveard 22 hours ago [-]
Calibrated probability across multi task with zero shot I guess. A reranker is single task and tuning it make it even more narrow. And I guess some piping to make multiclass efficient since you cannot mask logprob for independent questions in the same output space without throwing calibration away.
alex7o 11 hours ago [-]
If you mean that I can compare the probilities between different tasks fair. This is a thing you can not do with a reranker. This sounds cool but not the amount of hype we got cool
solaire_oa 21 hours ago [-]
I installed it, I tried the examples, it works.... But forgive my lack of imagination... what is this useful for?
Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?
https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.
I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).
devttyeu 21 hours ago [-]
I have a lot of semi-practical examples of how you can use this model wrapped in unix-ish tools - https://github.com/aurorainfra/grev (readme links to docs of each tool with some more or less practical examples)
Really I think "smart grep" is a pretty good one ('look for an error looking vaguely like this'). Also I think sql-based shell history + decision model is quite good to make the last 'which one of those choices is best fit given users past few commands' etc.
spaniard89277 21 hours ago [-]
Isn't it better to use an LLM to train modernbert or xgboost et al?
devttyeu 21 hours ago [-]
It is /possible/ to use an LLM.
But with Jev you're just paying for input (prefill) which is really fast, and in case of Jev specifically costs 50% of Deepseek V4.1 Flash (which has famously really cheap input token pricing).
I put 250MB / 1M lines of logs through Grev and it cost ~$10USD, DSv4.1 would be at least 10x that and much, much, much slower. With Jev/Grev that 1M requests took 10 mins
Edit: completely misread your question - yeah you could finetune specialized models to do that, probably based on some decent pretrained llm base, that is true for roughly any Jev-shaped problem. Do you want to bother doing that, also having to deal with having to host a zoo of specialized models?
motoboi 8 hours ago [-]
Laya, the open source model, it’s actually modernBERT based.
solaire_oa 20 hours ago [-]
Ok, those are pretty decent examples, and clears up the utility a bit: speed and tokens. Some of it's still a bit iffy (e.g. `cutv 'email address' 'phone number' < examples/users.csv`, csv is already in columns), but I can see using it for some niche queries. Neat tool.
I very much appreciate your to-the-point, non-vibed README as well, ty for that.
solaire_oa 20 hours ago [-]
I still think that `churn_risk` above is incorrect and unacceptable (perhaps there are sensible fixes, but saying "no churn risk" about a refund, in a leading example on their homepage, flabbergasting).
But if that were solved, I could see giving ollaya/grev to LLMs themselves, giving LLMs their own massive token-saver.
cobanov 10 hours ago [-]
[dead]
devttyeu 20 hours ago [-]
Yeah, speed is the one, I believe the default TypeSafe API quota is 1.5-2k queries per second (batched in bigger requests).
On the readme I'm so sorry to tell you that, but it's 100% written by Opus 5.5 with zero "pretty please don't write slop" prompting, it's just how slop is going to look like from now on. I've been writing code for 15 years or sth like that and the code is also what I'd call pretty reasonable..
solaire_oa 20 hours ago [-]
It has some jargon hallmarks, which I noticed, but vibed or not, it's a massive improvement on other repos. Maybe it's because it's only a few commits so far... perhaps if you were to vibe 100 more commits it would devolve. Or maybe 5.5 really did improve (doubt it, still sounds like an asshole for me). But idk.
motoboi 21 hours ago [-]
Is for when you want an AI to make a decision. If you have been using gpt or claude or open source models for that, than it’s a way cheaper alternative.
And if you have not been, it’s for when you have to extract the context from text. When you have numbers or fixed options, it’s just a matter of code.
So if you find yourself having to decide if a given user comment is a refund_request, that’s for that.
It’s not perfect, you still have to fine-tune (or calibrate) using examples you have (and keep those examples updated over time). But it’s way better than trying to parse text with regexes.
20 hours ago [-]
21 hours ago [-]
colordrops 20 hours ago [-]
If you don't want to spend a lot and want low latency, e.g. home automation. "It's cold and dark in here, do something about it", it will then turn on the lights and heater nearly instantly.
maskedpirate 16 hours ago [-]
sounds reasonable but doesn’t feel natural in a way that I can’t explain easily
ranyume 24 hours ago [-]
>Run decision models locally.
>example is a text classification task instead of a decision
hbrn 23 hours ago [-]
"Decision model" is just marketing jargon.
decision model = classifier
system one model = small non-reasoning LLM
noul = boolean
confidence = f(probabilities)
It's sad to see how gullible engineers are today.
verdverm 22 hours ago [-]
> how gullible ... today
that laya is even a thing is further evidence, people took that author at face value, the paper contents are incomplete and describe something that does not sound like Jev at all
this was the period of arxiv history that led to the new vouching system, laya author contributed to that imo
hbrn 22 hours ago [-]
My understanding is that Laya (or whatever it was called in 2025) was yet another fine-tuned classifier, not a general purpose one.
That said, Typesafe false marketing caused Laya to fit perfectly into pretty much every advantage that they are claiming: "system one decision model", cheap, fast, no hallucinations, structured, confidence output, parallel, calibrated. Their BS is their own demise.
I think Laya's author genuinely bought their BS and thinks he built the same thing. Unlike Typesafe, I don't think he's intentionally misleading people.
The only unique thing about Jev is that it's a general purpose classifier. Funny enough, they were so busy spreading marketing bullshit that they forgot to mention the only real thing that makes Jev unique.
verdverm 21 hours ago [-]
Laya author is spitting more BS than Typesafe, the (incomplete) papers are nothing like Jev, they use RAG and azure hosted services for calculating embeddings, with an orchestrator. Jev is just a model, Laya was put together after Jev, almost certainly based on what the author learned from Typesafe, and then backported "his" idea
I suspect most people only read the blog post, and thought it was great how a VC company "stole" an idea and was "outdone" by a rando... without actually checking the facts. Confirmational reading bias, we live in a post-truth world with dysfunction media ecosystem
chrchr 12 hours ago [-]
I don't think people even read the blog post, because the blog post pretty clearly explains that Laya isn't really comparable to Jev.
hbrn 21 hours ago [-]
I think you're right about Laya (and confirmation bias).
But like you said, at the end of the day he's just a rando.
He's not asking for $40m, not saying "I made ChatGPT, but i hate it, so I built the next big thing". Not claiming to co-invent RLHF.
Laya is just noise. Jev's bullshit affects me today - I see people injecting it into the codebases where it has no place.
To be fair, he's just asking how to get customers. And the post is 2 days before Jev's launch date? I don't think he's trying to sell Laya there (though he probably will at this point).
verdverm 21 hours ago [-]
There's something to be gleaned from the sum of their output across GitHub, arxiv, reddit, and HN (didn't delve LI, I hear it's a hot mess)
cgio 19 hours ago [-]
If anything, that makes Jev look even less impressive. They may just be hiding their similar quality outputs behind good PR.
verdverm 19 hours ago [-]
Laya is nowhere close to Jev in the 3rd party benchmarks I've seen, rather Laya is among the lowest scoring of the scores of Jev-alikes
That is not related to the ad hominem above, or bear to the claim of innovation. Just comparing them implies they are similar. Implementation quality, training data etc. are all optimisations. I have not seen a compelling argument Jev is different, maybe one that it’s better. I don’t have huge insights in the domain, so open to learning or changing opinion.
verdverm 5 hours ago [-]
Jev is just a trained model afawk., the (incomplete) papers the Laya author uses to claim they had the idea first describe RAG systems, orchestrators, and related built on LLM APIs. Laya came out after Jev and appears to be vibe coded to look more like Jev, then the claim of "idea theft" was made with the new blog post and code base, pointing back to papers that describe nothing like Jev.
Stepping back from Laya, there are dozens of Jev-a-likes now, most built on LLMs, whereas it is speculated that Jev likely uses transformers, but is not LLM like by generating a token at a time. Most likely they are reaching into the logits to get the details and assembling the json object deterministically. This is what the various LLM spins are doing, which is probably close but not the same as Typesafe.
Their marketing language is misleading. They must still use some transformer language model backbone to encode the text input (BERT or decoder-only LLM). The biggest difference is the output, instead of auto-regressively generating tokens, they produce probabilities over a bounded set of decisions (more flexible classification).
22 hours ago [-]
cobanov 24 hours ago [-]
Fair point, that example is basically classification. I'll change it to something that looks more like a real decision.
zahlman 20 hours ago [-]
Ollama is for large language models, so this is for large language... yodels?
Cool... but this does seem undermined by the fact that Ollama can add support for decision models at any time.
cobanov 24 hours ago [-]
Fair, and I'd be happy if they did. Ollaya uses the same API as Jev, so your code isn't tied to it either way
24 hours ago [-]
accountrequired 24 hours ago [-]
and that ollama is go-llama and not rust, so it's not really the ollama of anything
carimura 14 hours ago [-]
It's hard to keep up with the pace of everything but looking at this site it seems like the Ollama people launched it? Or are they just purposely imitating everything about Ollama. Seems pretty infringey.
Jev and its equivalent have garnered too much hype.
newswangerd 3 hours ago [-]
I can't wait for a version of this to come along that supports images. I want to build a feature into Digital Carrot for creating AI goals where you can create a daily goal to, for example, "empty the dishwasher" that you would then verify by taking a picture of the empty dishwasher at the end of the day. You can do this with current LLMs, but it's too slow and expensive for my use case at the moment.
mococa 23 hours ago [-]
It would be really cool to have LLMs and System One in a single tool - in this case, if Ollama implemented it.
verdverm 21 hours ago [-]
next vLLM release will have this
if you use gateways, GoModel support the S1 endpoints, my favorite feature is the virtual models, stable name, I can swap out the backing model(s)
(the "kev" in the docs is my fault, I should have said Jev / System1 in my feature request)
handfuloflight 24 hours ago [-]
Sounds good on latency but how is its actual decision quality vs. Jev?
cobanov 24 hours ago [-]
Depends on the model. The small ones I support today are well below Jev on harder queries, but fine for simple, well-defined questions. The open models that get close to Jev are bigger, and I'm adding support for those next.
nacs 21 hours ago [-]
It would be good to list 1) zero-shot accuracy and 2) latency on the models page . The LLM-based models' latency is probably much higher than the BERT approaches I would assume.
Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?
datadrivenangel 1 days ago [-]
Are there many models that are comparable to Jev for generic decision making?
Smarter move if you have an eval set is to just train a classifier and call it a day.
<<<"i was curious to see if i could train a competitive Jev-like model completely autonomously with a swarm of agents using our internal system."
Bro is writing off the H200 lol
On a sidenote I really can't stand the term "swarm" and definately plays into AI doomerism.
lirolero 23 hours ago [-]
[dead]
cobanov 24 hours ago [-]
The link rgbrgb posted is a good overview. The best open ones are close to Jev now, but they're big models. And I agree, if you have an eval set for a fixed task, a trained classifier is the better choice.
nickstinemates 21 hours ago [-]
Laya is pretty easy to set up on its own without ollaya. I just did that and replaced my current jev API usage to laya running on a GTX 970 with 4GB of vram.
Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.
eserozvataf 24 hours ago [-]
great project for empowering open-source alternatives.
rkovashikawa 24 hours ago [-]
open-source is the only way for safe AI development. whoever doesn’t share the weights/code will lag behind.
cobanov 24 hours ago [-]
Thanks!
thih9 22 hours ago [-]
FAQ[1] says:
> It is an independent project, not affiliated with Ollama.
I am fairly confident if Jev-style decision models are seen as prominent (which, they seem to be), Ollama will support them. Surprised the team hasn't implemented this already.
vorticalbox 23 hours ago [-]
Does anyone know what laya multi lang is faster than laya en? I would have thought focusing on a single language would be faster.
gauravsapkotanp 23 hours ago [-]
I have also tried this and its really awesome
qurren 22 hours ago [-]
Would be great if you supported CUDA 12; I don't feel like paying $15K to upgrade my GPU right now
verdverm 21 hours ago [-]
wait another week or so for vLLM's next release
11 hours ago [-]
oguzhankayan 21 hours ago [-]
Nice work! Making open models easier to run locally is valuable on its own. Keeping the API compatible with Jev is a thoughtful touch, too.
We've stumbled into general differentiable models..
After chatgpt everything in AI mostly became LLMs and building wrappers around them. It's like people forgot how to do ML.
To those of us who actually trained models back in the day, its kind of cute to see people wowed by a classifier. Yes, this is 0 shot and doesn't need training (most people wanting this would've used structured output, this is cool because it's cheaper and faster). But anyone with basic ML knowledge could've built this in a few hours.
The question is mostly why wasn't this productized. And it's interesting indeed that it took this long to become a finished product.
[0] https://news.ycombinator.com/item?id=9224
There's probably other areas where people are using LLMs where a more tailored ML solution might work better.
Probably because doing it wrong (using an llm in place of a classifier) is more profitable? (For the people selling inference.)
LLMs still are better than Jev at the task, just across the board slower.
Anyone who had a reason to try this already tried it (ads/recommendations) - back in 2023/2024 during the first fine tuning wave and it was accurately determined that it was not worth the effort, the results were more bogus than just using CoT, so frankly parallelism meant nothing if bogus * parallel = bogus.
So thrown into the dumpster and nobody really cared to revisit because it was already tried.
Pretty much sometime between then and now it somehow became the state where the tradeoff makes sense now.
Maybe it's just a case of the idea just now crossing the threshold into working just good enough to be worth it
I have 80,000 voice recordings to classify. Very few are in English, and the classes are not in English. Many of the classes are project names or other proper nouns. How could a system not trained on data specific to the problem possibly be expected to work?
There have been tons of applications for this. People were using earlier LLMs like BERT for classifiers long before LLMs became viable chatbots.
People didn’t know they wanted classifiers until OpenAI gave them a taste.
I think the hype with Jev is just that, while structured generation is great, LLM judges tend to kind of suck for precise classification. And the more powerful the base model, the more accurate they can get, but they get increasingly expensive/impossible to finetune. It was specifically the latency/price point Jev offered vs. the general accuracy it claimed that generated all the excitement. Plus the promise of cheap calibration (tuning).
"Jev exists because LLMs exist" is kind of a truism, as Jev apparently is literally a Transformer model.
In early 2016, Clarifai's core product was basically just an image classifer exposed via an API. And at that point, they'd raised $40 million--the same amount as TypeSafe.ai/Jev--and they were experiencing viral growth among developers + signing contracts with a bunch of flashy logos. The demand was so high that Amazon launched Rekognition and Google launched their similar APIs to compete.
The AI hype cycle and the number of people thinking about using AI certainly puts more wind at Jev's back, but even in an alternative universe where we don't have contemporary LLMs, Jev's core claims would be remarkable and there would be a big market for it.
I adhere to the idea that this is software's "Tower of Babel" moment where everyone just fundamentally ships things in completely diverging architectures, because creating a ground up architecture is no longer something that needs to be avoided for an economically viable business mode that in the past two decades would have otherwise incentivized people into industry standards. In a world where "taste" is the focus, single ingredients in the recipe aren't enough.
This particular "innovative" concept already has a rich, open research background. What Jev appears to have done is scale that up a bit and isolate good training data, which results in a great product but not really something impossible to imitate. The only major difference currently is that the open source decision models need to be fine-tuned as they're not trained off of the entire internet yet.
My thoughts too. It sounds like a minor feature being framed as a whole new business.
Then again, Dropbox and Docker are too.
It is definitely not the MNIST classifier you had trained in 2019.
The difference is that you only train it once and the modern LLM machinery sort of takes care of that with large contexts.
It's great that Jev proved this is a viable product. I'd expect a great many research innovations coming from making this work better/faster/cheaper, and around interfacing modern agents with it.
Anyone fitting MNIST in 2019 was already outdated by several years at the very least. GPT-2 was 2019! We already had zero-shot classifiers then. In fact, the paper for GPT-3 was literally
"Large Language Models are Zero-Shot Reasoners"
These kinds of zero-shot classifiers were already developed and used in-house for many years. They just weren't commercialized as a separate product, because anyone who could use an LLM proper could build layers around it to fulfill any classification task like this.
GPT2 was absolutely unusable as a classifier. Using GPT3 as a classifier cost a few order of magnitude more than what this thing is priced at, and the context window was a few thousand tokens.
> These kinds of zero-shot classifiers were already developed and used in-house for many years
This is like Google's favorite coping mechanism for falling behind at AI. "We had everything inhouse for several years, we didn't release it for $reasons."
> anyone who could use an LLM proper could build layers around it to fulfill any classification task like this
You missed the part where it costs more than two orders of magnitude lower :)
I'm not claiming there are major architectural innovations, but that's not the point. Once you prove there's a market, there's a cambrian explosion of innovations.
Now all you need is to give it more context along with your query.
There's no scenario where you are training BERT online to give you an answer.
Rank System Score Public / sealed accuracy Evidence
1 decider-4b v2 64.13 83.5% / 34.7% Evaluator-run, offline
2 Jev 1.13 63.29 86.6% / 36.7% Evaluator-run API
3 JevK5 v0.2 62.04 85.3% / 33.1% Evaluator-run
4 Cygnet 12B 61.76 87.9% / 33.8% Evaluator-run, offline
5 Hopper 59.43 82.3% / 34.1% Evaluator-run
28 Kev 4B 36.14 66.2% / 22.4% Evaluator-run
41 Laya 421M 30.25 58.4% / 30.8% Evaluator-run
https://benchmarkheaven.com/jev-models
It's so easy that I question why I would ever pay for JEV when eventually I'll have done enough random things that I will also have a large corpus and likely a general model as well.
It's a fast classifier you can use out-the-box, ~1.5bn tokens is about $40 (I've been hammering it)
It just works ... a whole bunch of low-level/low-importance workflow stuff that was getting farmed out to small/fast LLM models now has a competitive alternative ... and bits that hadn't even been considered to go into some external descision/classifier service can be tested/deployed at ~$0.00003/req
I don't get this wall of negativity on it, it's genuinely innovative/useful tech ... would expect HN to be more positive, regardless of whether it's the absolute best execution
Starting from a business POV one should inflate terminology, hack together an MVP, and see if the market demands it before doing hardcore R&D.
But starting from technical/craftsman POV all you see is a hack and a lot of big words, so it’s easy to become jaded.
I think the point being made is that Jev is great but it has no competitive moat, and open source versions will very soon catch up if their secret sauce is just synthetic data.
(Whether or not that is true, I don't know.)
I've been unable to find a good use case for now.
I'm building a woodworking app and I've managed to create an autopilot that can take a simple instruction ("get me 5 2x4s", "cut the middle 2x4 into 4 equal pieces", "move the 2x4 3 feet left") and the action instantly happens with next to no lag. There is already an llm but now it can share an intent, and the geometry system shows jev the various actions and jev chooses the action that gets it closer to the goal until it has found a state that matches the intent or gives up. The result is the llm can think "higher level" and let the cheap fast model grind out the options in a relative blink of the eye, without the 30s of reasoning the llm would have done about the various operations it could try.
https://benchmarkheaven.com/jev-models
But I'm guessing people will find the right training regime and data mix soon to close the gap.
But big things I see are instability and inaccuracy - like pick a random problem.
I mean, anyone here could have a decision maker just return a random number. Fast and easy.
The question really is how GOOD is it?
I’m sure this has been a gradual and long decline. Maybe it even started with the dot com boom and accelerated with crypto. With AI it seems to have got worse.
Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?
Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).
Right now a lot of people are doing this with LLMs and it's too slow and expensive.
Imo the right iterative approach to productionizing these systems is something like:
You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.
https://developers.openai.com/cookbook/examples/zero-shot_cl...
I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision/recall tradeoffs. They want something which plausibly works and is easy to use.
Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?
https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.
I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).
Really I think "smart grep" is a pretty good one ('look for an error looking vaguely like this'). Also I think sql-based shell history + decision model is quite good to make the last 'which one of those choices is best fit given users past few commands' etc.
But with Jev you're just paying for input (prefill) which is really fast, and in case of Jev specifically costs 50% of Deepseek V4.1 Flash (which has famously really cheap input token pricing).
I put 250MB / 1M lines of logs through Grev and it cost ~$10USD, DSv4.1 would be at least 10x that and much, much, much slower. With Jev/Grev that 1M requests took 10 mins
Edit: completely misread your question - yeah you could finetune specialized models to do that, probably based on some decent pretrained llm base, that is true for roughly any Jev-shaped problem. Do you want to bother doing that, also having to deal with having to host a zoo of specialized models?
I very much appreciate your to-the-point, non-vibed README as well, ty for that.
But if that were solved, I could see giving ollaya/grev to LLMs themselves, giving LLMs their own massive token-saver.
On the readme I'm so sorry to tell you that, but it's 100% written by Opus 5.5 with zero "pretty please don't write slop" prompting, it's just how slop is going to look like from now on. I've been writing code for 15 years or sth like that and the code is also what I'd call pretty reasonable..
And if you have not been, it’s for when you have to extract the context from text. When you have numbers or fixed options, it’s just a matter of code.
So if you find yourself having to decide if a given user comment is a refund_request, that’s for that.
It’s not perfect, you still have to fine-tune (or calibrate) using examples you have (and keep those examples updated over time). But it’s way better than trying to parse text with regexes.
>example is a text classification task instead of a decision
decision model = classifier
system one model = small non-reasoning LLM
noul = boolean
confidence = f(probabilities)
It's sad to see how gullible engineers are today.
that laya is even a thing is further evidence, people took that author at face value, the paper contents are incomplete and describe something that does not sound like Jev at all
this was the period of arxiv history that led to the new vouching system, laya author contributed to that imo
That said, Typesafe false marketing caused Laya to fit perfectly into pretty much every advantage that they are claiming: "system one decision model", cheap, fast, no hallucinations, structured, confidence output, parallel, calibrated. Their BS is their own demise.
I think Laya's author genuinely bought their BS and thinks he built the same thing. Unlike Typesafe, I don't think he's intentionally misleading people.
The only unique thing about Jev is that it's a general purpose classifier. Funny enough, they were so busy spreading marketing bullshit that they forgot to mention the only real thing that makes Jev unique.
I suspect most people only read the blog post, and thought it was great how a VC company "stole" an idea and was "outdone" by a rando... without actually checking the facts. Confirmational reading bias, we live in a post-truth world with dysfunction media ecosystem
But like you said, at the end of the day he's just a rando.
He's not asking for $40m, not saying "I made ChatGPT, but i hate it, so I built the next big thing". Not claiming to co-invent RLHF.
Laya is just noise. Jev's bullshit affects me today - I see people injecting it into the codebases where it has no place.
Just how to "make fkn $500k ARR fast?"
https://news.ycombinator.com/item?id=49674396
too much LI/Xitter influencer consumption
To be fair, he's just asking how to get customers. And the post is 2 days before Jev's launch date? I don't think he's trying to sell Laya there (though he probably will at this point).
example ranking shared in comment on this HN post
https://benchmarkheaven.com/jev-models
Stepping back from Laya, there are dozens of Jev-a-likes now, most built on LLMs, whereas it is speculated that Jev likely uses transformers, but is not LLM like by generating a token at a time. Most likely they are reaching into the logits to get the details and assembling the json object deterministically. This is what the various LLM spins are doing, which is probably close but not the same as Typesafe.
This is the main PR (merged) for using DiffusionGemma (diffusion generally) for Jev-a-likeness in vLLM. https://github.com/vllm-project/vllm/pull/57250
if you use gateways, GoModel support the S1 endpoints, my favorite feature is the virtual models, stable name, I can swap out the backing model(s)
https://gomodel.enterpilot.io/docs/getting-started/quickstar...
(the "kev" in the docs is my fault, I should have said Jev / System1 in my feature request)
Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?
Smarter move if you have an eval set is to just train a classifier and call it a day.
top open one is trained by perplexity cto for $3k, kinda cool https://x.com/denisyarats/status/2102252088067850507
Bro is writing off the H200 lol
On a sidenote I really can't stand the term "swarm" and definately plays into AI doomerism.
Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.
> It is an independent project, not affiliated with Ollama.
[1]: https://ollaya.dev/docs/faq
(already merged)
GoModel (gateway) already supports Jev like endpoints too
https://gomodel.enterpilot.io/docs/providers/jev