# Jev, now open source: Lev URL: https://interfaze.ai/blog/jev-now-open-source-lev TypeSafe's Jev proved that a model can answer software's questions with calibrated probabilities instead of text. Today we're releasing **Lev**, an open System One model that speaks the same API and runs on a single GPU you control. Lev on GitHub ## What is classification? Classification is picking one label from a fixed set. Is this email spam, which team owns this ticket, is this comment toxic? Most of the AI inside a real product has this shape: routing, moderation, intent detection, triage, and grading, thousands of times a day. ### How it used to work | Approach | How it works | The catch | | ------------------------------------------------- | ----------------------------------------------------------- | --------------------------------------------------------------- | | **Keyword rules** | `if "refund" in text: route("billing")` | Breaks the moment a customer phrases it differently | | **Classic ML** (Naive Bayes, logistic regression) | Train on thousands of labeled examples per task | No real understanding of language, one model per task | | **Fine-tuned BERT** | Pretrained encoder plus a small classification head | Labels are baked into the head, so a new label means retraining | | **Zero-shot NLI** (`bart-large-mnli`) | Each label becomes a hypothesis: "This example is billing." | One forward pass per label, so 60 labels means 60 passes | | **LLM prompting** | Describe the labels in plain English and ask for JSON | Generates tokens, can invent labels, no calibrated confidence | [BERT](https://arxiv.org/abs/1810.04805) made classifiers understand language, but froze the label set at training time. [Zero-shot NLI](https://arxiv.org/abs/1909.00161) unfroze it, at the cost of one forward pass per label. ### Then LLMs arrived LLMs made classification feel solved: describe the labels, ask for JSON, and a frontier model gets it right most of the time. But they generate token by token, can return labels outside your list, and are overconfident about how sure they are. So you add validation, retries, and a human review queue. Fine for a chatbot, a deal-breaker for an `if` statement. ## What is Jev? [Jev](https://typesafe.ai/) is the first **System One Model** from TypeSafe AI, [launched on September 15, 2026](https://typesafe.ai/blog/introducing-system-one-models-and-jev). You give it some context and a set of questions (yes/no, pick one, or rate on a scale), and it returns a probability for every option instead of text. - **No generated text.** All answers come from one parallel pass. Output tokens are free and input costs $0.042 per million tokens. - **Type-safe.** The answer space is the option set you send, so an out-of-set label is impossible. - **Calibrated.** Gate on the probability: act above 0.9, send the rest to a person. - **Fast.** 70 to 500 ms end to end, where a reasoning LLM takes seconds. ### RLCD: the training behind it | Method | Optimizes for | Produces | | -------- | ---------------------------------------------- | --------------------------------------------- | | **RLHF** | Answers human raters prefer | Chat models, great at instruction following | | **RLVR** | Outputs that can be checked programmatically | Reasoning models, strong at math and code | | **RLCD** | Calibrated decisions with honest probabilities | System One models that return typed decisions | RLHF shapes probabilities around what raters liked, which makes models overconfident. **Reinforcement Learning for Calibrated Decisions (RLCD)** rewards honest probabilities instead: when Jev says 0.8, it should be right 80% of the time. TypeSafe hasn't published how RLCD works. But calibration is measurable, and our benchmark below measures it. ## Open source Jev: Lev Lev is our open System One model. It answers every typed question in one forward pass, reading answers from logits it already computed, and generates no tokens. | Item | Lev | Jev | | ---------------------------------------- | ------------------------------------------------ | -------------- | | Weights | Open, Apache-2.0 | Closed | | Size | 4B (Qwen3.5-4B + LoRA) | Undisclosed | | S1Bench, all 13 subsets | 68.9% | **76.1%** | | Calibration error (ECE, lower is better) | 0.115 | **0.091** | | Output tokens | 0 | 0 | | Options per `choice` | Several hundred by label code, unbounded by head | 255 | | Runs on | Your GPU (one H100 tested) | TypeSafe's API | Lev is only a 4B model which can easily run on your Macbook or small GPU while being slightly less accurate than Jev. No open model at 4B or smaller scores higher on S1Bench. ### Quickstart Needs Python 3.12+ and, for real-time use, a CUDA GPU. It also runs on CPU, but much slower: seconds per call instead of milliseconds. The first load downloads the base model (about 8 GB) and the adapter (about 200 MB), with nothing to configure. ```bash pip install "lev[serve] @ git+https://github.com/InterfazeAI/lev#subdirectory=packages/lev" ``` Questions are `noul` (yes/no), `choice` (pick one), or `score` (2 to 10 levels). Run Lev in Python, or start `lev serve` and call `/v1/systemone` over HTTP. **Lev · python** ```python import lev model = lev.load("interfaze-ai/lev") state = "Hi, I was charged twice for my order #4471 and I want a refund." questions = { "intent": { "type": "choice", "instructions": "What does the customer want?", "criteria": { "refund": "wants money back", "cancel": "wants to cancel an order", "track": "wants to know where an order is", "other": "anything else", }, }, "urgent": {"type": "noul", "instructions": "Does this need a human within the hour?"}, "frustration": { "type": "score", "instructions": "How frustrated is the customer?", "criteria": ["calm", "mildly annoyed", "annoyed", "angry"], }, } result = model.system_one(state, questions) print(result.answers["intent"].choice) # refund print(result.answers["intent"].probabilities) # {'refund': 0.84, 'cancel': 0.094, 'track': 0.012, 'other': 0.054} print(result.answers["urgent"].noul) # 0.43 print(result.answers["frustration"].score) # 1.57, between "mildly annoyed" and "annoyed" print(result.usage.output_tokens) # 0 ``` **cURL · bash** ```bash lev serve --checkpoint interfaze-ai/lev --host 0.0.0.0 --port 8000 curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{ "state": "The package arrived crushed and the screen is cracked.", "questions": {"damaged": {"type": "noul", "instructions": "Was the item damaged?"}} }' ``` These are real outputs from the released checkpoint. ### How Lev works ```mermaid flowchart LR S["State + typed questions"] --> P["One batched forward pass
Qwen3.5-4B + LoRA"] P --> A["Label-token readout
one single-token code per option"] P --> B["Candidate-path head
option text matched to the state"] A --> T["Temperature per question type,
readout mode and option count"] B --> T T --> O["Typed answers
choice, noul, score + probabilities"] ``` - **Label-token readout.** Each option gets a one-token code, and the answer is read from the logits over those codes. - **Candidate-path head.** When options run out of codes, a small head matches the state to each option's text, so there's no option limit. - **Calibration.** Fitted temperatures turn raw scores into honest probabilities. Other open re-creations pick one readout and cap the option count. Lev routes between both. ### The optimizations that mattered - **169 → 69 ms:** one batched forward pass instead of forking the cache per question. - **+51 points on 60-intent routing:** label codes instead of the head for large option sets. - **banking77 0.818 → 0.980:** skipping codes that split into two tokens. - **+5.7 points before training:** using the base model's own chat format. The evidence for each is in the [decision log](https://github.com/InterfazeAI/lev/blob/main/docs/DECISIONS.md). ## Classification in Interfaze Text classification in Interfaze runs on a similar system to Lev: the model reads the answer from the set of labels you define. The difference is that Interfaze is still token based, so it's a hybrid. | Item | Lev | Interfaze | | ------------------- | --------------------------------- | ---------------------------------------------------- | | Output | Probabilities, zero output tokens | Tokens, returned as structured output | | Questions | Typed only: yes/no, choice, score | Any JSON schema, labels included | | In the same request | Classification only | OCR, web search, transcription, extraction, and more | | Where it runs | Your GPU | [Interfaze API](https://interfaze.ai/dashboard) | Tokens cost a little speed, but they let one request classify a document while also reading, searching, and extracting from it. Define your labels as an enum in the schema, and the label comes back as a typed field. **Interfaze SDK · typescript** ```typescript import { Interfaze, responseFormat } from "interfaze"; import { z } from "zod"; const interfaze = new Interfaze({ apiKey: process.env.INTERFAZE_API_KEY }); const ticketSchema = z.object({ intent: z.enum(["refund", "cancel", "track", "other"]), urgent: z.boolean().describe("Does this need a human within the hour?"), }); const response = await interfaze.chat.completions.create({ messages: [{ role: "user", content: "Hi, I was charged twice for my order #4471 and I want a refund." }], response_format: responseFormat(z.toJSONSchema(ticketSchema), "ticket_schema"), }); console.log(JSON.parse(response.choices[0]?.message.content ?? "{}")); ``` **Vercel AI SDK · typescript** ```typescript import { createInterfaze } from "@interfaze-ai/ai-sdk"; import { generateText, Output } from "ai"; import { z } from "zod"; const interfaze = createInterfaze({ apiKey: process.env.INTERFAZE_API_KEY }); const ticketSchema = z.object({ intent: z.enum(["refund", "cancel", "track", "other"]), urgent: z.boolean().describe("Does this need a human within the hour?"), }); const { output } = await generateText({ model: interfaze("interfaze"), output: Output.object({ schema: ticketSchema }), prompt: "Hi, I was charged twice for my order #4471 and I want a refund.", }); console.log(output); ``` **LangChain SDK · typescript** ```typescript import { ChatInterfaze } from "@interfaze-ai/langchain"; import { z } from "zod"; const interfaze = new ChatInterfaze({ apiKey: process.env.INTERFAZE_API_KEY }); const ticketSchema = z.object({ intent: z.enum(["refund", "cancel", "track", "other"]), urgent: z.boolean().describe("Does this need a human within the hour?"), }); const response = await interfaze .withStructuredOutput(ticketSchema) .invoke("Hi, I was charged twice for my order #4471 and I want a refund."); console.log(response); ``` **Interfaze SDK · python** ```python import os from typing import Literal from pydantic import BaseModel, Field from interfaze import Interfaze interfaze = Interfaze(api_key=os.environ["INTERFAZE_API_KEY"]) class Ticket(BaseModel): intent: Literal["refund", "cancel", "track", "other"] urgent: bool = Field(..., description="Does this need a human within the hour?") response = interfaze.chat.completions.parse( messages=[{"role": "user", "content": "Hi, I was charged twice for my order #4471 and I want a refund."}], response_format=Ticket, ) print(response.choices[0].message.parsed) ``` **LangChain SDK · python** ```python import os from typing import Literal from pydantic import BaseModel, Field from interfaze_langchain import ChatInterfaze interfaze = ChatInterfaze(api_key=os.environ["INTERFAZE_API_KEY"]) class Ticket(BaseModel): intent: Literal["refund", "cancel", "track", "other"] urgent: bool = Field(..., description="Does this need a human within the hour?") response = interfaze.with_structured_output(Ticket).invoke( "Hi, I was charged twice for my order #4471 and I want a refund." ) print(response) ``` Read more in the [structured output docs](https://interfaze.ai/docs/structured-output). ## Benchmarks We ran Lev and Jev through the same harness on all 3,880 S1Bench items, pinned by [Nimble](https://github.com/bespokelabsai/nimble)'s manifests. Our Jev run lands within 0.8 points of TypeSafe's published figures, so the harness isn't the gap. ```chart-bar { "subtitle": "Accuracy per S1Bench subset. Higher is better. Macro: Lev 68.9%, Jev 76.1%.", "format": "percent", "domain": [0, 1], "sort": "none", "hoverValues": true, "categoryWidth": 170, "series": [ { "key": "lev", "label": "Lev", "color": "#2A78D6" }, { "key": "jev", "label": "Jev", "color": "#EE6A35" } ], "data": [ { "label": "vitaminc-dev", "lev": 0.668, "jev": 0.801 }, { "label": "massive-en-US", "lev": 0.857, "jev": 0.874 }, { "label": "massive-de-DE", "lev": 0.823, "jev": 0.871 }, { "label": "boolq", "lev": 0.827, "jev": 0.893 }, { "label": "squad2", "lev": 0.813, "jev": 0.836 }, { "label": "paws", "lev": 0.776, "jev": 0.9 }, { "label": "multinli", "lev": 0.89, "jev": 0.836 }, { "label": "civil_comments", "lev": 0.76, "jev": 0.803 }, { "label": "aegis2", "lev": 0.8, "jev": 0.804 }, { "label": "helpsteer2", "lev": 0.386, "jev": 0.341 }, { "label": "summeval-relevance", "lev": 0.358, "jev": 0.358 }, { "label": "summeval-consistency", "lev": 0.271, "jev": 0.812 }, { "label": "pubmedqa", "lev": 0.732, "jev": 0.764 } ] } ``` - **Jev leads clearly** on minimal-edit pairs (paws, vitaminc) and summary faithfulness, where Lev rates faithful summaries one level too low. - **Lev leads** on multinli and helpsteer2, though both are within noise (5 to 9 points). - **Calibration:** Lev is better calibrated on 5 of 13 subsets. **Lev is at the top of its size class.** On the six subsets every model on the public S1Bench board completed, no model at 4B or smaller scores higher: Lev ties reflex-4b at 71.9%. Only Jev and three models 6 to 9 times its size are ahead. ```chart-bar { "subtitle": "Macro accuracy over the six S1Bench subsets every listed model completed. Parameter count in brackets. † self-declared training contamination on an evaluation subset.", "format": "percent", "domain": [0.2, 0.8], "sort": "none", "hoverValues": true, "categoryWidth": 240, "data": [ { "label": "Jev (board)", "value": 0.775, "color": "#EE6A35" }, { "label": "Jev (measured here)", "value": 0.769, "color": "#EE6A35" }, { "label": "simplejev-qwen38-27b (27B)", "value": 0.758, "color": "#C9C7C1" }, { "label": "djev-full (26B)", "value": 0.749, "color": "#C9C7C1" }, { "label": "simplejev-qwen36-35b-a3b (35B)", "value": 0.744, "color": "#C9C7C1" }, { "label": "Lev (4B, measured here)", "value": 0.719, "color": "#2A78D6" }, { "label": "reflex-4b (4B)", "value": 0.719, "color": "#C9C7C1" }, { "label": "decider-2b † (2B)", "value": 0.703, "color": "#C9C7C1" }, { "label": "laya-gpu (421M)", "value": 0.625, "color": "#C9C7C1" }, { "label": "laya (421M)", "value": 0.625, "color": "#C9C7C1" }, { "label": "jeff (400M)", "value": 0.561, "color": "#C9C7C1" }, { "label": "jeff-gpu-full (400M)", "value": 0.559, "color": "#C9C7C1" }, { "label": "qwen3-8b-full (8B)", "value": 0.535, "color": "#C9C7C1" }, { "label": "open-jev-deberta † (435M)", "value": 0.524, "color": "#C9C7C1" }, { "label": "reflex-08b (800M)", "value": 0.516, "color": "#C9C7C1" }, { "label": "kev-05b † (500M)", "value": 0.493, "color": "#C9C7C1" }, { "label": "gliner-base (194M)", "value": 0.432, "color": "#C9C7C1" }, { "label": "gliner-multi (287M)", "value": 0.426, "color": "#C9C7C1" }, { "label": "gliner-small (74M)", "value": 0.4, "color": "#C9C7C1" }, { "label": "simplejev-rwkv-mid", "value": 0.38, "color": "#C9C7C1" }, { "label": "simplejev-rwkv-std", "value": 0.327, "color": "#C9C7C1" }, { "label": "simplejev-rwkv-small", "value": 0.313, "color": "#C9C7C1" } ] } ``` On held-out intent sets, Lev hits 0.980 on banking77 (77 intents) and 0.968 on clinc_oos (151 intents). ### Speed Compute is one batched pass, so it stays flat from one question to eight. ```chart-bar { "subtitle": "Lev compute per call in milliseconds, on one H100, before any network. Lower is better.", "format": "number", "decimals": 0, "domain": [0, 200], "sort": "none", "showValues": true, "directionHint": "lower", "categoryWidth": 220, "data": [ { "label": "prefill + fork (original)", "value": 169, "color": "#C9C7C1" }, { "label": "prefill + fork, conv kernel", "value": 140, "color": "#C9C7C1" }, { "label": "single batched forward", "value": 81, "color": "#C9C7C1" }, { "label": "single + conv kernel (served)", "value": 69, "color": "#2A78D6" } ] } ``` End to end from a laptop, Jev's hosted API answered in about 340 ms median and Lev on one H100 in 414 to 654 ms. ## Training - **Data:** 200,000 examples from 29 public sources, with questions paraphrased and recast so the model learns the format, not one wording. - **No contamination:** any source overlapping S1Bench is refused. - **Recipe:** LoRA r=32 on one H100, about 6 hours per run. The full pipeline is in the [GitHub repo](https://github.com/InterfazeAI/lev). ## Takeaway - **Classic classifiers:** fast and cheap, but the label set is frozen at training time - **LLMs:** any label at runtime, but tokens, retries, and no confidence you can trust - **Jev:** typed decisions with calibrated probabilities from a hosted API - **Lev:** the same protocol with open weights, 4B parameters, on your own GPU - **Interfaze:** a hybrid that classifies with tokens, alongside OCR, search, and extraction in one request Classification doesn't need a model that writes. It needs one that picks from your options and tells you how sure it is, and now there's an open one. - Weights on [Hugging Face](https://huggingface.co/interfaze-ai/lev) - Code, harness, and decision log on [GitHub](https://github.com/InterfazeAI/lev) - Try classification in the [Interfaze playground](https://interfaze.ai/dashboard) - Read TypeSafe's [Jev launch post](https://typesafe.ai/blog/introducing-system-one-models-and-jev) - Join the [Discord](https://interfaze.ai/discord) Lev is not affiliated with or endorsed by TypeSafe AI.