Jev and small decisions inside agents

What it is, what it costs, what it's for, where it fails, and the open-source alternatives.

Jev is a model from TypeSafe AI that does not generate text. It takes a context and a set of closed questions and returns typed answers with probabilities in under a second, at a fraction of an LLM's cost. It handles the small decisions an agent currently asks an LLM to make (who gets the task next, whether an action is dangerous, whether a result is good enough), and text generation stays with the LLM.

This article synthesizes TypeSafe documentation, published experiments and open-source projects. The comparison tables preserve a research snapshot from 26 September 2026. Model availability, pricing and rankings can change; follow the linked sources before selecting a deployment. The examples and sources were reviewed on 30 September.

• essentials
• how it works
• price and numbers
• integrate
• use cases
• eval judge
• design rules
• limits
• sources
• open alternatives

The essentials

• What it is. TypeSafe calls it a System One model: you send a state (the context: a ticket, an agent trace, a web page) and questions of three kinds (pick an option, rate on a scale, yes/no). It returns probabilities your code uses directly, with no text to parse. It does not write, code, or explain its reasoning, so it does not replace the LLM behind Claude Code or Cursor.
• Price. In the 26 September snapshot, Jev 1.13 costs $0.042 per million input tokens, and output is free. A thousand decisions of about a thousand tokens each cost about 4 cents. In LangChain's experiment each call cost $0.00035 and took 0.44 s on average.
• Where it pays off. Routing (tasks, tickets, models), classifying and labeling at scale, filtering content, blocking an agent's dangerous actions before they run, choosing a browser agent's next action, and evaluating agent answers. Published cases range from 1,018 papers classified for $0.08 to an agent that finds flights in 7 seconds for $0.0039.
• The comparison against LLMs. LangChain tested it as an eval judge against GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6. Jev matched a human reviewer on 100% of pass/fail calls, its quality scores varied 92 to 913 times less, and the run cost $0.34 versus $28.17 with Claude. The test is small (five cases from a weather agent).
• Where it fails. It reads instructions literally, cannot count or do arithmetic, compares dates badly, loses accuracy when the context is padded with irrelevant material, and malicious text inside the state can move its answers. As a verifier for long coding tasks, the CLM authors measured it below picking a solution at random. It works best in English.
• Open alternatives. Dozens of open rebuilds accept the same requests as Jev, so code written for it runs on them by changing the server address. Only models of about 27 billion parameters match its quality (AutoJev-27B); the 4B ones that run on a Mac, such as Kev or decider, trail it, and the small encoders such as Laya are far behind. Details in the last section.
• Where to start. Try the TypeSafe Playground, install the official skill in Claude Code, and ask it to analyze where Jev fits in a specific project. The Python SDK is typesafe-sdk and the LangChain integration is langchain-typesafe.

How it works

One call, step by step, using a support ticket as the example.

The call

Everything goes through one endpoint, POST /v1/systemone, with three fields: model, state, and questions. The state can be text, a JSON object, or a list of text values. Input is text only: images, audio, or video have to be converted to text or fields first.

The three question types

Confidence and probability are different things

For Choice and Score, confidence measures how concentrated the distribution is, using (n × p_max − 1) / (n − 1), where n is the number of options and p_max the winner's probability. It is not an accuracy percentage. Two Choices with three options each:

Both pick billing, but only the first one is sure of it. For a Noul, a value near 0.5 means the model is torn between yes and no; it does not mean "medium intensity".

Several questions at once

All questions about the same state are evaluated in parallel. Adding questions barely increases latency and only costs those questions' tokens. Questions cannot see each other's answers: if one decision needs another's result, you need two calls.

The question's name never reaches the model

Calling a field safe_to_publish tells Jev nothing. The full meaning has to go in instructions and in the description of each option (criteria).

How it's trained and adapted

It is trained with RLCD (reinforcement learning aimed at calibrated probabilities). The same weights serve every customer: there is no fine-tuning on your data. You adapt it to a domain by putting your own material in the state and your business rules in the instructions and criteria. TypeSafe says it does not train on customer requests and offers zero data retention to enterprise customers.

Versions

jev-latest pointed on 26 September to jev-1.13.0, and so does jev-preview (no separate preview build was listed in that snapshot). If you tune confidence thresholds for one version, pin that version instead of the alias, because the alias switches models without notice when a new one ships.

Price, limits, and measured numbers

Jev 1.13 spec sheet (official documentation)

Published numbers, with their source

@0xCodila's article repeats one warning: the cost that matters is the cost per completed task. A cheap decision that sends the agent down the wrong branch can cost far more than the decision itself.

How to integrate it

No code. The Playground (console.typesafe.ai/playground) lets you paste a state, add questions, and see the answers.

Python. pip install typesafe-sdk (Python 3.12 or later) with the key in TYPESAFE_API_KEY. A minimal example:

from typesafe_sdk import Choice, Noul, TypeSafeClient

with TypeSafeClient(model="jev-1.13.0") as client:
    r = client.system_one(
        state={"ticket": "I was charged twice. Please refund the duplicate today."},
        questions={
            "team": Choice(
                instructions="Which team should handle this ticket?",
                criteria={
                    "billing": "Charges, invoices, refunds",
                    "technical": "Bugs and outages",
                    "none": "None of the above",
                },
            ),
            "urgent": Noul(instructions="The customer asks for an immediate fix"),
        },
    )

print(r.choices["team"].choice, r.choices["team"].confidence)
print(r.nouls["urgent"].noul)

There is also a JavaScript SDK (@typesafe-ai/sdk) and a plain HTTP API. The SDKs retry rate-limit errors on their own. The usual errors are 401 (bad key), 422 (malformed request field), and 429 or 529 (rate limit or overload: wait and retry).

LangChain. The langchain-typesafe package (alpha 0.0.1a2 in LangChain's experiment) exposes Jev as TypeSafeClassifier: you pass state and questions to .invoke() and get classifications back, not a chat message. The state can be text, structured data, or LangChain messages, so it can be called from any node or middleware in the agent. It ships two experimental middlewares:

• ModelRouterMiddleware uses Jev to pick a model from the latest user message (a fast one for lookups and small changes, a powerful one for architecture and high-stakes decisions) and keeps it for the whole run.
• AutoModeMiddleware checks each tool call (for example bash) and blocks risky ones before they execute. It is the same "auto mode" pattern that Claude Code, Codex, and Cursor keep in the closed part of their products, now available to any agent.

Skill for coding agents. The typesafe-ai/skills repo (MIT license; 440 stars on September 19, 2,159 in the source snapshot) teaches an agent to write code that uses Jev. Installing it in Claude Code:

claude plugin marketplace add typesafe-ai/skills
claude plugin install typesafe@typesafe-ai

For other agents: npx skills add typesafe-ai/skills --skill typesafe-ai. In Claude Code you invoke it with /typesafe:typesafe-ai. Bowe Frankema's tip: install it in a project and ask the agent to analyze where and how to use Jev there.

The skill tells the agent to read the live documentation (the index is at docs.typesafe.ai/llms.txt) and suggests six patterns:

• Route and fill arguments: pick the handler and its typed parameters in the same call.
• Select instead of generate: code finds candidates (values, text spans) and Jev picks the right one.
• Find and judge evidence: rerank RAG passages by relevance and pick the useful context.
• Turn judgments into reusable data: score dimensions once and let code change weights, thresholds, and rankings without calling the model again.
• Verify and escalate: check claims against their sources and send uncertain cases to a person or a reasoning model.
• React to changing state: code keeps goals and observations, and Jev guides the next bounded step.

Use cases, with real examples

Routing. LangChain picks between a cheap model and a powerful one based on the task. @0xCodila builds a "chief of staff" that decides whether a briefing's next step is research, writing, or review, and drops the job as JSON into a per-destination queue. Riley Brown uses it to decide what to do with each email (reply, research, wait, review). Ryan Vogel does email triage at scale.

Choosing an agent's next action. At every step, Browser Use builds the list of controls on the page, Jev picks the action and the element, and a small LLM only writes when a field needs filling. When Jev says "done", a separate step checks the result. Kyle Jeong (Browserbase) runs browser agents for fractions of a cent, and Jarrod Watts built a live trading agent.

Blocking dangerous actions. LangChain's AutoModeMiddleware decides whether a tool call is safe before it runs.

Classifying and labeling at scale. Hassan's 1,018 papers, sorted into topics defined as the options of a Choice.

Filtering content. SuperX (Rob Hallam) takes the last three days of posts on a topic with 30+ likes and asks each one eight questions at once: is it specific, is it new, is it bait, is it a plug, among others. It combines the answers with the like-to-reply ratio and sorts each post into read, skim, or pass. There is a free demo with no sign-up (superx.so/instead-of-doomscrolling), and every SuperX plan includes it as an API (POST /v1/posts/triage), an MCP tool (triage_posts), a CLI, and an agent skill, "Worth a Reply", which returns the posts worth replying to with an angle for each.

Second brain (J.B., @VibeMarketer_). One inbox for saved posts, articles, meeting notes, and ideas. Jev sorts incoming material by your projects and interests, flags duplicates, and marks which notes add something new. When you start a task, the agent searches the archive, Jev scores which notes are relevant, and those notes go to the writing agent. Every note is saved with its original link and date. He recommends starting with a single project.

Games and real-time loops. laya-mlx plays Snake on a Mac at 60 decisions per second, and CLM has demos with Chrome's dinosaur game, Super Mario, and WikiRacing.

elvis's view (@omarsar0). He criticizes that nearly every demo is a simple classification that was already possible before. For him Jev is one more building block of an agent harness (the code around the model that manages tools, context, and control flow), and it is useful for:

• Cheaper, more deterministic workflows: control flow, classifiers, labeling.
• Surfacing context on demand: which tool, which skill, or which metadata to show at each moment.
• Dynamic UIs, cheaper "councils" of several LLMs, orchestration and planning, and proactive agents.
• Reliable verifiers and evals, and generating synthetic data so model and harness improve together.

The three areas where he sees the most potential: continuous evaluation and monitoring, custom verifiers for long-horizon agents, and self-improving harnesses. He moved a team of agents that pruned and archived sessions, previously running on frontier models, onto Jev. What surprised him most was proactive agents. He has promised a full guide that was not available in the source snapshot.

Grok's list. A user asked Grok for the ten best uses of Jev in agents: real-time loops, browser actions, gating dangerous tools, model routing, detecting a stuck agent, context compaction, skill selection, output guardrails, ticket triage, and RAG reranking. It is an LLM's answer, ranked by theoretical gain with no measurements; treat it as a list of ideas.

Jev as an eval judge: LangChain's experiment

Agents are evaluated today in two ways: with code (cheap and reliable, but only for closed checks such as "did it call the tool?") or with another LLM that reads the trace and grades it (flexible, but slow, expensive, and inconsistent from run to run). LangChain tested whether Jev could be a third option.

Setup. A weather agent built with Deep Agents answered five requests. The five answers were stored, and each judge evaluated them 100 times with two signals: a continuous quality score and a pass/fail call. A person graded the same answers against the same rubric and served as the reference.

What it implies. With 10,000 production traces a day, a judge at $0.00035 per call lets you evaluate far more traces and more criteria per trace, and catch quality drops sooner.

Caveats the authors state themselves:

• Five cases from a single agent; it remains to be seen whether this holds for other agents.
• A cheap judge that is consistently wrong produces bad signal at scale; human review is still needed.
• Temperature was not set for the LLM judges (each provider used its defaults), and the Jev version that answered was not recorded.
• The lower variance is an observation; the experiment does not show that it comes from the training method.

One thing to watch when reading the original article: the examples for the three question types are swapped (the Choice example returns a number from 0 to 1, and the Noul example picks among three options). The correct definitions are the ones in the table under How it works.

Design rules for building with Jev

These come from @0xCodila's article, the official skill, and the documentation.

• Division of labor. The LLM researches, plans, and writes; Jev routes, scores, approves, or escalates; code executes. Exact rules, such as "stop after ten actions", belong in code, not in the model.
• One question, one judgment. Each question asks for one specific thing, with every option described. Include a "none" option when nothing may fit. If you catch yourself explaining what you really meant after a wrong answer, that explanation is what was missing from the instruction.
• Give evidence, not summaries. "The researcher finished" tells it less than the sources, findings, and remaining gaps. Evidence goes in named JSON fields, kept apart from the original request. Anything the question doesn't need gets filtered out first, because irrelevant context lowers accuracy.
• Options from the current moment. Build the options from the current state (the workers available now, the sources that exist) and rebuild them when a tool changes the state. With very long lists: filter the obvious mismatches in code, score what's left with Score, and pick among the shortlist with Choice.
• Ask everything together. Independent questions about the same state go in the same call, including speculative ones (ask about several possible branches and keep only the one that applies).
• Measured thresholds. The 0.85 confidence in @0xCodila's example is a starting point; tune it with your own labeled examples. Uncertain cases go to a person or an LLM.
• Agent brakes. An action limit, a spending limit, progress saved after every step, and an independent check that the task actually happened (a confident answer does not prove a file was saved). Publishing or sending anything always needs a separate approval.

Known limits and criticism

What TypeSafe acknowledges about Jev 1.13

From the known-failures page, last reviewed September 17.

Not a verifier for long tasks. The CLM team gave it several candidate solutions to coding problems (DeepSWE and Terminal-Bench 2.1) and asked it to pick the best one; Jev scored below picking one at random.

Slows down with many options. Jev can reuse the work done on the state, but not on the options. With hundreds or thousands of fixed candidates (tools, game actions), that makes it slower than alternatives such as CLM.

elvis's criticism (@omarsar0). In his tests, apart from speed, he saw few quality differences compared with what he already had, and he asks for more quality checks and fewer speed demos.

Reminders from the documentation. Correctly formatted output does not mean correct output, so validate it on data from your own domain. Rate limits change with demand.

Sources and published examples

Besides the posts, five pages of TypeSafe's documentation were read (models and pricing, Jev 1.13 known failures, confidence, Jev with coding agents, and the main index).

TypeSafe’s model reference was rechecked on 30 September 2026 for Jev 1.13 pricing, context limits and aliases. The external benchmark tables retain their 26 September snapshot; these experiments were not reproduced for this article.

Open-source alternatives

Added on September 26, 2026. New projects appear every day, so read the numbers as a snapshot of that date.

Jev is closed: its weights, architecture and training data are not published, and the only way to use it is TypeSafe's paid API. Since its release in mid-September, dozens of open projects have tried to rebuild it, and two independent leaderboards already rank them: JevBench measures 93 systems and the Decision Index 56. This section covers how the open ones work, which ones are worth trying, whether code written for Jev runs on them, and how far each one is from Jev.

The short answer

• They can be used the same way, with care. Most of them serve the same endpoint, POST /v1/systemone, with the same request and the same three question types, so code written with TypeSafe's SDK runs against them by changing the server address. In an independent conformance test, only Kev and decider passed every required check, and a threshold tuned on Jev does not carry over to any of them.
• Model size decides quality more than the project does. Open models of about 27 billion parameters tie Jev: AutoJev-27B scores 50.9 on the Decision Index and Jev 51.7. Models of 4 to 12 billion score roughly 7 to 25 points lower. The small models that run on a laptop CPU (under one billion parameters) score between 2 and 18 on that 0-to-100 scale, where 0 means guessing.
• A mixed ranking can mislead. JevBench puts decider-4b v2 first, above Jev, because its score also counts speed and cost. On reasoning alone Jev still leads, 53.1 to 49.4, and it is better calibrated.
• Most rebuilds use the same trick. An ordinary open language model reads the options labelled with letters, and the probability it gives each letter as the next token becomes the answer. A stock Qwen3 4B read this way already scores 46.4 on JevBench's reasoning axis (Jev 53.1) but only 29.1 on calibration. The trained rebuilds add a few points of reasoning and, above all, calibration (around 75).
• On a Mac. Kev runs through MLX (the 0.8B on any Apple Silicon Mac, the 4B and 9B with 32 GB), JevK5 through llama.cpp, decider's 2B on the Apple GPU through PyTorch, and Laya through laya-mlx; Ollaya installs several of them behind Jev's API with one command. laya-mlx answers in about 10 ms, the 0.8B to 2B models in 130 to 150 ms, and a 4B in 0.6 to 0.7 s.
• Language. decider, JevK5 and Von say they are English only, and the others publish no results in other languages. The exception is Laya's multilingual checkpoint (100+ languages), one of the weakest models on the English benchmarks.
• Read the READMEs as claims. Von calls itself state of the art; JevBench ranks it 46th. Laya reports beating Jev, but only with a checkpoint fine-tuned on that benchmark's own training data. CLM's authors report results similar to Jev's; both leaderboards place it near the bottom for general decisions.

How most of them work

Almost every rebuild that gets close to Jev reads an existing open model instead of training a new architecture:

SemIf, JevK5, Cygnet, decider and local-jev work this way. Kev instead adds a small trained head that scores each option directly, on the same kind of model. Laya and Von use a small encoder (a model that reads text but cannot write it) that scores each option at its own marker token. CLM compares an embedding of the state with cached embeddings of the options.

The temperature matters because a raw model is overconfident: dividing its scores by a number fitted on labelled examples spreads the probabilities without changing which option wins. The projects differ mainly in three things: which base model they read, whether they fine-tune it on decision data (a LoRA, a small add-on trained on top of frozen weights, or a full fine-tune), and how carefully they fit that temperature.

Nokia's AnyJev library measures why the care matters. Reading raw letter probabilities from Qwen3-8B on 20 banking intents, 23% of answers flipped when the option order was reversed, and only 7.7% of decisions could be automated at an error rate of 5% or less. Averaging over rotated option orders, with no labels, cut the flips to 7.3%; adding a temperature fitted on 100 to 500 labelled examples lowered the calibration error from 0.240 to 0.095 and raised the automatable share to 52.0%. Accuracy only moved from 0.747 to 0.807: the gain is in probabilities you can set a threshold on.

The candidates

JevBench's board also lists dozens of smaller rebuilds (Hopper, Winnow-12B, Jev-Omni, reflex, SemIf, local-jev, OpenSourceJev and more), most of them the same recipe on Qwen3.5-4B or Gemma 4. Two notes on them. Hopper, fifth on JevBench, is licensed for research and demos only, because part of its training data forbids commercial use. And Lev (Interfaze, Qwen3.5-4B) is one of the few that ran Jev through its own harness: 0.689 against Jev's 0.761 over 13 subsets, with a calibration error of 0.115 against 0.091. Together AI's Tev1 is a different case: a Qwen3.5-4B fine-tuned to answer one letter from 2 to 24 options, with a recipe to train your own for about $17, but no probabilities you can threshold and not Jev's API. Two other kinds of alternative show up in the measurements. A large stock model read through a shim, with no training at all, is already close to Jev: Jevfire on Qwen3.8-27B scores 45.7 on the Decision Index. And an open-weight LLM that just answers with a constrained choice also competes: in nibzard's benchmark, gpt-oss-120b served by Cerebras scored 81.3% on banking intents against Jev's 76.3%, at 303 to 346 ms. The one hosted service that looks like an alternative, classifier.dev, runs Jev itself in its fast tier.

Details on the main ones

AutoJev-27B. Its author reports 84.6% accuracy against Jev's 82.8% on its own benchmark, with better calibration, and the Decision Index confirms it essentially ties Jev (50.9 against 51.7, with a median of 105 ms against Jev's 253 ms in that run). The weights are public on Hugging Face. The cost is hardware: about 49 GB of weights in bf16, which means a data-centre GPU, not a laptop.

Kev (Jared Palmer, 7,096 stars). Four sizes, calibrated by default, and the most complete tooling. It also has a hosted API: OpenRouter serves Kev-4B over the same /v1/systemone contract at Jev's price, $0.042 per million input tokens with free output, so it can be tried without a GPU and moved in-house later. The tooling includes a local playground, a coding-agent skill that fine-tunes Kev on your own labelled data on Modal (about $1 per 4B run on an H100), and one command to deploy an endpoint that scales to zero. On the author's "new sources" (datasets it never trained on), Kev-27B scores 0.848 against Jev's 0.857, and Kev-4B and 9B are within four points. Its confidence uses the same formula as TypeSafe's reference adapter. On a 32 GB M5 Mac, Kev-4B answers five questions about a 270-token text in 721 ms (136 ms if the text repeats). The README states its limits: it trails Jev on knowledge questions (MMLU 0.74 for the 9B against 0.90), it lets you automate fewer decisions at a 5% error budget (0.45 to 0.57 against 0.70), and it was trained on states of at most 384 tokens.

decider (Mapika). Ranked first on JevBench's composite and trained on public datasets plus data labelled by local Qwen models, with nothing distilled from Jev. It runs on the Apple GPU: decider-2b answered in a median of 133 ms on an M1 Pro. Its own README locates the gap: Jev leads on knowledge questions (0.69 against 0.51 for the 35B on the Decision Index panel), and one pass cannot do multi-step arithmetic. English only.

JevK5. A Qwen3.5-4B with a LoRA distilled from Qwen3.6-27B and GPT-6 Luna. Its GGUF files run on llama.cpp on any GPU, Mac or CPU: about 0.6 s per short decision for the 4B on an M1 Pro. It answers up to 16 options per pass (more with extra passes), refuses inputs above 16,384 tokens, and is English only. Part of the v0.3 training data comes from GPT-6 Luna, so OpenAI's terms apply to those outputs.

Laya and laya-mlx. The best known (25,071 stars on GitHub) and the smallest serious option. pip install laya; its Router sends English to the ModernBERT checkpoint and other languages to the multilingual one, and laya-serve accepts Jev's requests. The laya-mlx port (pip install laya-mlx, then laya.load("aac6fef/laya-mlx"), which downloads the model once and then runs offline) answers a short question in 13.4 ms (7.4 ms multilingual) on an M3 Max, in under 1 GB of memory, and matches the original on 378 of 378 test comparisons. Its announcement says "50 times faster than Jev", but that compares a local model with a network call. Its model card lists its own limits: the base checkpoints are near chance on the typed-decisions benchmark without fine-tuning (0.362), they ship overconfident, yes/no questions can follow their true/false labels instead of the text, and accuracy collapses with more than about 20 options (0.425 on Banking77 against Jev's 0.870). The context is 512 tokens for English and 1,024 for multilingual, extendable to 8,192. The "beats Jev" figures on the card belong to a checkpoint fine-tuned on the typed-decisions training split.

Von. pip install von-sdk, then von serve exposes /v1/systemone. Version 1.2 fixed a real defect: in 1.1, 49.5% of hard answers changed when the same options were shuffled; now each option is scored alone, so order cannot matter. The README's other claims don't hold up outside it. It says "SOTA" and reports beating Jev at Doom in its own runs, but independent suites put it at 0.742 against Jev's 0.967 (jabr) and 46th on JevBench. Its figures also mix versions: it quotes two different calibration temperatures, and its benchmark table is for 1.1. English only.

CLM-8B. Two encoders, one for the state and one for the actions, trained so each state lands close to the right action. Actions are encoded once and cached, which makes each decision very cheap when the set of options is fixed (tools, buttons, game moves): its authors measure up to 9 times Jev's speed, 13 times with about 1,000 options. It was trained on 60 million question-answer pairs, 30 million synthetic negative examples and one million agent trajectories. Its authors also report, after light fine-tuning as a verifier choosing among solutions written by Opus 5 and Fable 5, 81.6% on DeepSWE (31 of 38 held-out tasks) and 87.6% on Terminal-Bench 2.1, 4 to 6 times faster than Jev. On general decisions the independent boards disagree: 78th on JevBench and 6.5 on the Decision Index. To run it: pip install contrastive-lm, start the encoder with vllm serve Qwen/Qwen3-8B --runner pooling --port 8090, then clm-serve, which opens the API and a playground at http://localhost:8700. It needs an NVIDIA GPU (measured on an RTX 4090 and an H100), not a Mac. A multimodal CLM-35B is due in early October.

Can they replace Jev in your code?

For the call itself, mostly yes. The Python example in How to integrate it runs against Kev by changing only the client:

from typesafe_sdk import TypeSafeClient

# Same state and questions as before; only the server changes.
with TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8009",
                    model="kev-latest") as client:
    r = client.system_one(state=..., questions=...)

With decider, setting TYPESAFE_BASE_URL=http://localhost:8000 is enough. The easiest way to try several is Ollaya, which does for decision models what Ollama does for chat models: one install on macOS, Linux or Windows, ollaya run laya --preset triage "…" to test a ready-made set of questions from the terminal, and a local server on port 11435 that speaks TypeSafe's API for Laya, decider, Kev, Von, Winnow and a few classic classifiers, with weights downloaded from each author's repository. It also runs as an MCP server for Claude Code (claude mcp add ollaya -- ollaya mcp). On a Mac its models run on the CPU through ONNX Runtime, except the GGUF ones (Winnow), which use the Apple GPU through llama.cpp.

What does not carry over:

• Thresholds. A 0.85 tuned on Jev means nothing on another model, and "confidence" is not even computed the same way. jevcompat, a test suite for Jev-compatible servers, found five different definitions among eight servers: applied to the same 204 answers with a 0.9 threshold, TypeSafe's formula accepted 58% and Laya's (one minus the normalised entropy) 30%. Kev and decider's current versions (1.3.0 and later) use TypeSafe's formula; Von reports the gap between the two most likely options. Laya's authors also ask you to refit its temperature on your own data. Retune every threshold on labelled examples, as you would for a new Jev version.
• Edge cases of the API. jevcompat ran 32 required checks against the eight most-starred compatible servers on September 24. Only Kev and decider passed all of them. Laya rejected a question with 128 options and returns a null in the Score legend that the SDK cannot parse; Simple Jev rejects the default model name jev-latest and more than 50 options. Running jevcompat test URL against a server before switching catches this.
• How many options. Jev takes up to 255 options per question and rejects 256 or more. JevK5 answers 16 per pass (more with extra passes), Cygnet up to 26, and Laya loses accuracy above about 20.
• How much text. Jev takes 64,000 tokens per request. decider takes 32,000, JevK5 and Cygnet 16,384, and Laya 512 in English. Kev accepts 65,536 but was trained on states of up to 384 tokens, and its accuracy drops on long documents.
• Language. Test on your own content in the language you use; most of these models have never been measured outside English.
• Option order. Shuffling the options changes some answers on almost every model, Jev included: in nibzard's test Jev changed 13% of its choices and the worst LLM 37%. Kev's server has an endpoint, /v1/systemone/permute, that measures this for you.
• Operations. You run the server. The ones close to Jev need a GPU; on a Mac they work, but in hundreds of milliseconds rather than tens.
• Licences and provenance. "Open" does not always mean usable at work: Hopper's weights are for research and demos only. Some models were trained on Jev's own answers (autotrust's JEV-9B was trained on a corpus whose largest part, 498,000 rows, has labels "distilled from Jev 1.13 (TypeSafe) via OpenRouter"), which TypeSafe's customer agreement forbids: clause 2.3(b) bars using its output "to perform model distillation, train a model to imitate the output of the Services". Kev, decider and Tev1 state that they used no Jev outputs. And some repos are traps: lucast4049/openjev, created on September 25, borrows the openjev name and only asks you to download a Windows app, the usual pattern of a malware lure.

Older open tools that do part of the job

Before Jev, open models already covered pieces of it, each with its own API rather than Jev's:

• Zero-shot classifiers such as GLiClass, GLiNER2 and the NLI models by Moritz Laurer (DeBERTa-v3 and ModernBERT) take the labels at call time and return a probability per label, on a CPU. They are the ancestors of Laya and Von, and score in the same low range on the Decision Index (GLiNER 2.5 between 3.9 and 7.1).
• Rerankers score how well each candidate matches a query. JevBench ran five of them as decision models; Qwen3-Reranker-4B came 18th with 43.5, above most of the dedicated rebuilds.
• Guard models for blocking dangerous tool calls, the job of LangChain's AutoModeMiddleware: IBM Granite Guardian, Qwen3Guard, Llama Guard and gpt-oss-safeguard. Qwen3Guard is one of the models Ollaya serves behind Jev's API.
• Evaluation judges such as Prometheus 2, Atla Selene Mini and Flow-Judge grade an answer against a rubric, like Jev in LangChain's experiment, but they write their verdict as text instead of returning a probability.

What the independent benchmarks say

The four independent evaluations agree on the shape: Jev is still ahead of anything that runs on a laptop; only the open 27B models catch it, and a 12B comes within about 7 points.

A fifth, smaller study (AbdelStark, 300 examples) compared Jev with GLiNER2.5-multi: Jev was clearly ahead on news topics (0.910 against 0.700) and banking intents (0.870 against 0.610), and the two tied on emotions, where Jev gave zero probability to the correct label on 16% of the examples.

Which one to try, if any

These are proposed evaluation paths; the deployments below have not been benchmarked for this article:

• To compare several on the Mac with one install: Ollaya, with the SDK pointed at localhost:11435, and the same questions sent to Jev as the reference.
• Closest to Jev on the Mac: Kev-4B through MLX (needs 32 GB). It serves Jev's API locally, so the same code can run against Jev and against Kev on your own examples before choosing. JevK5-4B also runs on a Mac through llama.cpp, but from its own Python class rather than Jev's API.
• For decisions in Spanish or another language: create a labelled evaluation set in that language. English benchmark rankings do not establish multilingual quality. Compare task accuracy, calibration and latency before selecting a model.
• Jev's quality without the API: AutoJev-27B or a 27B read through a shim. That needs a data-centre GPU, so it is an infrastructure decision, not something for the laptop.

Sources for this section

GitHub and Hugging Face metadata (stars, licences, dates) for these projects and for jeff, GLiNER2.5-Decide, laya-mlx and CLM were read on September 26, 2026.