# Interfaze 1.1 Lite: The Truly Open Weight Model for Deterministic Work
URL: https://interfaze.ai/blog/introducing-interfaze-1-1-lite
Most models you can call today were trained to write for a person. **Interfaze 1.1 Lite** writes for your code, and it puts a box and a confidence score on the text it reads.
Every model inside 1.1 is under a license that allows commercial use. This release also adds a new document reader and guardrails that check text safety in 119 languages.
## What Lite does
Lite takes text, images, audio and files, including PDFs and Word documents, and returns text or JSON that matches your schema. It's meant for jobs with one right answer:
- [OCR](https://interfaze.ai/docs/vision/ocr) with a box and confidence for every line and word
- [Speech-to-text](https://interfaze.ai/docs/audio/speech-to-text) with [speaker diarization](https://interfaze.ai/docs/audio/stt-speaker-diarization)
- Classification and [structured extraction](https://interfaze.ai/docs/structured-output) into your schema
- [Object detection](https://interfaze.ai/docs/vision/object-detection) and [GUI detection](https://interfaze.ai/docs/vision/gui-detection)
- [Translation](https://interfaze.ai/docs/translation), [forecasting](https://interfaze.ai/docs/compute/time-series-forecasting) and [guardrails](https://interfaze.ai/docs/guardrails)
The weights are open and the whole model runs on a single 80 GB GPU. Through the API it's \$0.85 per million input tokens and \$1.50 per million output tokens.
| | Interfaze 1.1 Lite |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Input price | \$0.85 / MTok |
| Output price | \$1.50 / MTok |
| Context window | 128k tokens |
| Max output tokens | 32k tokens |
| Input | Text, images, audio, files |
| Output | Text, JSON |
| Reasoning | Available |
| PDF pages | Up to 50 per request |
| Open weights | Yes, runs on one 80 GB GPU |
| License | [Apache 2.0, MIT and CC BY 4.0, commercial use allowed](https://huggingface.co/interfaze-ai/interfaze-1-lite/blob/main/LICENSE) |
## What's new in 1.1
1.1 replaces two of the specialist models. The API, the price and the one-GPU footprint stay the same.
The new document reader supplies the text, reading order, tables and markdown for each page. The line detector still adds a box and a confidence score to every line.
The new guardrails model checks text safety in 119 languages and maps each violation to an S1 to S14 category code.
The whole bundle is open for commercial use. Our own code is Apache 2.0, and each sub-model whose weights ship in the repo is under Apache 2.0, MIT or CC BY 4.0, so you can self-host Lite in a commercial product without negotiating a separate license.
## The architecture
Lite is a reasoning core plus a set of specialist models, each built for one kind of perception.
The core reads the request, decides which specialists to run, and writes the answer from what they return. The specialists do the reading, listening and locating that a general model is weakest at.
### Why split it up
A single vision-language model can read a page well. But like a person, it gives you the words without saying where they are on the page or how sure it was.
Specialists produce that metadata as part of how they work. A text detector scores every line it finds, and a speech recognizer timestamps every word.
| Approach | Reads the content | Exact positions | Confidence per line | Handles nuance |
| ---------------------------- | ----------------- | --------------- | ------------------- | -------------- |
| General VLM alone | Yes | Approximate | No | Yes |
| Specialist models alone | Yes | Yes | Yes | No |
| **Lite: core + specialists** | **Yes** | **Yes** | **Yes** | **Yes** |
The core understands the request, follows your schema and reasons about what it read. The specialists supply the precise output and the scores.
Everything the specialists produce comes back in [precontext](https://interfaze.ai/docs/precontext), so one response carries both the JSON answer and the evidence behind it.
## Benchmarks
Here's Lite next to our flagship [interfaze-1](https://interfaze.ai/docs/models/interfaze-1), GPT-5.4-Mini, Claude-Sonnet-4.6, Gemini-3-Flash and Grok-4.3. Bold marks the best score on each benchmark.
| Benchmark | Task | Interfaze 1.1 Lite | interfaze-1 | GPT-5.4-Mini | Claude-Sonnet-4.6 | Gemini-3-Flash | Grok-4.3 |
| ----------------------------------------------------------------------------- | ---------------------- | ------------------ | ----------- | ------------ | ----------------- | -------------- | -------- |
| [MMMU-Pro](https://interfaze.ai/leaderboards/mmmu-pro) | Multimodal reasoning | **73.2%** | 71.1% | 40.4% | 46.3% | 67.6% | 68.7% |
| [RefCOCO](https://interfaze.ai/leaderboards/refcoco) (Acc@0.5) | Visual grounding | **83.8%** | 82.1% | 67.0% | 75.5% | 75.2% | 25.0% |
| [SOB value accuracy](https://interfaze.ai/leaderboards/structured-output-benchmark) | Structured output | **81.5%** | 80.5% | 75.1% | 77.9% | 77.3%\* | 78.4% |
| [olmOCR-Bench](https://interfaze.ai/leaderboards/olmocr) | Document OCR | 83.8% | **85.7%** | 80.1% | 73.9% | 75.3% | 81.9% |
| [OCRBench V2](https://interfaze.ai/leaderboards/ocrbench-v2) (English) | Text in images | 60.9% | **70.7%** | 52.7% | 54.7% | 55.8% | 54.7% |
| [VoxPopuli-Cleaned WER](https://interfaze.ai/leaderboards/voxpopuli-cleaned-aa) (lower is better) | Speech recognition | 3.01% | **2.4%** | – | – | 4.0% | – |
| [Spider 2.0-Lite](https://interfaze.ai/leaderboards/spider-2-lite) (SQLite) | Text-to-SQL | 48.9% | **52.9%** | 26.7% | 49.6% | 45.2% | 45.9% |
| [GPQA Diamond](https://interfaze.ai/leaderboards/gpqa-diamond) | Graduate-level science | 85.9% | **92.4%** | 82.8% | 89.9% | 88.5% | 73.6% |
| [MMMLU](https://interfaze.ai/leaderboards/mmmlu) | Multilingual knowledge | 87.8% | **90.9%** | 75.3% | 84.9% | 88.7% | 89.7% |
Comparison scores are from the [Interfaze leaderboard](https://interfaze.ai/leaderboards). A dash means the model has no speech recognition score. \*Gemini-3-Flash-Preview.
Lite beats every third-party model scored on six of the nine: multimodal reasoning, visual grounding, structured output, document OCR, text in images and speech. It also beats our flagship on MMMU-Pro, RefCOCO and SOB, and it ranks second of 30 models on SOB.
It trails on text-to-SQL, science and multilingual knowledge, where [interfaze-1](https://interfaze.ai/docs/models/interfaze-1) is the better pick.
[See the full benchmark breakdown](https://interfaze.ai/leaderboards).
## Self-hosting
The weights are on [Hugging Face](https://huggingface.co/interfaze-ai/interfaze-1-lite). The whole model runs on a single 80 GB GPU with no external services, so it works offline.
You need a GPU with compute capability 8.9 or newer, like an H100, and `ffmpeg` for audio.
```bash
hf download interfaze-ai/interfaze-1-lite requirements.txt --local-dir .
pip install -r requirements.txt
```
Load it with Transformers and call `chat`, the same entry point the API uses. The core plans, runs the specialists it needs and answers.
```python
from transformers import AutoModel
model = AutoModel.from_pretrained("interfaze-ai/interfaze-1-lite", trust_remote_code=True)
answer = model.chat(
[{"role": "user", "content": "What is the total, and which item is highlighted?"}],
files=["receipt.jpg"],
)
print(answer["content"])
```
To skip planning, call one capability directly. `ocr` runs the document reader and line geometry and returns the stitched page.
```python
doc = model.ocr("invoice.pdf", page_range=[1, 2])
print(doc["text"])
```
The [model card](https://huggingface.co/interfaze-ai/interfaze-1-lite) covers every capability's method, arguments and output.
Transformers is slower than a batching serving engine. For throughput, use the API.
## Use it with the Interfaze API
To try it without code, open the [playground](https://interfaze.ai/dashboard/playground) and pick Interfaze 1.1 Lite from the model picker. It shows the answer and the precontext behind it.
In code, set `model` to `interfaze-1.1-lite`. Structured output and precontext work the same as on every other Interfaze model.
**Interfaze SDK · typescript**
```typescript
import { Interfaze } from "interfaze";
const interfaze = new Interfaze({ apiKey: process.env.INTERFAZE_API_KEY });
const response = await interfaze.chat.completions.create({
model: "interfaze-1.1-lite",
messages: [
{
role: "user",
content: [
{ type: "text", text: "Extract the details from this ID" },
{ type: "image_url", image_url: { url: "https://r2public.jigsawstack.com/interfaze/examples/id.jpg" } },
],
},
],
});
console.log(response.choices[0]?.message.content);
```
**Vercel AI SDK · typescript**
```typescript
import { createInterfaze } from "@interfaze-ai/ai-sdk";
import { generateText } from "ai";
const interfaze = createInterfaze({ apiKey: process.env.INTERFAZE_API_KEY });
const { text } = await generateText({
model: interfaze("interfaze-1.1-lite"),
messages: [
{
role: "user",
content: [
{ type: "text", text: "Extract the details from this ID" },
{ type: "file", mediaType: "image/jpeg", data: new URL("https://r2public.jigsawstack.com/interfaze/examples/id.jpg") },
],
},
],
});
console.log(text);
```
**LangChain SDK · typescript**
```typescript
import { ChatInterfaze } from "@interfaze-ai/langchain";
import { HumanMessage } from "@langchain/core/messages";
const interfaze = new ChatInterfaze({ model: "interfaze-1.1-lite", apiKey: process.env.INTERFAZE_API_KEY });
const response = await interfaze.invoke([
new HumanMessage({
content: [
{ type: "text", text: "Extract the details from this ID" },
{ type: "image_url", image_url: { url: "https://r2public.jigsawstack.com/interfaze/examples/id.jpg" } },
],
}),
]);
console.log(response.content);
```
**Interfaze SDK · python**
```python
import os
from interfaze import Interfaze
interfaze = Interfaze(api_key=os.environ["INTERFAZE_API_KEY"])
response = interfaze.chat.completions.create(
model="interfaze-1.1-lite",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Extract the details from this ID"},
{"type": "image_url", "image_url": {"url": "https://r2public.jigsawstack.com/interfaze/examples/id.jpg"}},
],
}
],
)
print(response.choices[0].message.content)
```
**LangChain SDK · python**
```python
import os
from interfaze_langchain import ChatInterfaze
from langchain_core.messages import HumanMessage
interfaze = ChatInterfaze(model="interfaze-1.1-lite", api_key=os.environ["INTERFAZE_API_KEY"])
response = interfaze.invoke([
HumanMessage(content=[
{"type": "text", "text": "Extract the details from this ID"},
{"type": "image_url", "image_url": {"url": "https://r2public.jigsawstack.com/interfaze/examples/id.jpg"}},
])
])
print(response.content)
```
Interfaze speaks the Chat Completions standard, so any OpenAI-compatible SDK works too. Point it at `https://api.interfaze.ai/v1` with your key from the [dashboard](https://interfaze.ai/dashboard).
## Example: read an X-ray consult note
An orthopaedic clinic scans a consult note with the patient's ankle X-ray printed on it, and its records system needs the note as a typed record.
The note mixes print, handwriting, checkboxes, a stamp and a radiograph. One request has to read all of it and point to where each part is.
The code below does four things:
1. Extracts the patient, X-ray, diagnosis and vitals into a schema. The injured side is an enum, so only `left` or `right` can come back.
2. Finds the X-ray and its side marker with object detection, and reads the marker's letter with OCR.
3. Checks that the marker on the film matches the side written in the note. Wrong-side mistakes are a classic patient safety failure, so this runs on every note.
4. Checks the identifiers and vitals against the OCR confidence of the line each came from. Fields at 0.9 or above post automatically, and the rest go to a nurse.
**Interfaze SDK · typescript**
```typescript
import { Interfaze, responseFormat } from "interfaze";
import { z } from "zod";
const interfaze = new Interfaze({ apiKey: process.env.INTERFAZE_API_KEY });
const ConsultSchema = z.object({
mrn: z.string(),
patient_name: z.string(),
date_of_birth: z.string(),
visit_date: z.string(),
side: z.enum(["left", "right"]),
xray_views: z.array(z.string()),
xray_impression: z.string(),
diagnosis: z.string(),
vitals: z.object({
blood_pressure: z.string(),
heart_rate: z.number(),
respiratory_rate: z.number(),
temperature_c: z.number(),
}),
});
const response = await interfaze.chat.completions.create({
model: "interfaze-1.1-lite",
messages: [
{
role: "user",
content: [
{
type: "text",
text: "Extract the patient, X-ray, diagnosis and vitals from this consult note. Locate each X-ray view and its side marker.",
},
{ type: "image_url", image_url: { url: "https://r2public.jigsawstack.com/interfaze/examples/health_care_memo_example.png" } },
],
},
],
response_format: responseFormat(z.toJSONSchema(ConsultSchema), "consult_note"),
});
type Bounds = { top_left: { x: number; y: number }; bottom_right: { x: number; y: number } };
type OcrLine = { text: string; bounds: Bounds; average_confidence: number };
type Precontext = { name: string; result: { sections?: { lines: OcrLine[] }[]; detected_objects?: { label: string; bounds: Bounds }[] } }[];
const note = ConsultSchema.parse(JSON.parse(response.choices[0]?.message.content ?? "{}"));
const precontext = (response.precontext ?? []) as Precontext;
const lines = precontext.flatMap((p) => (p.name === "ocr" ? (p.result.sections ?? []) : [])).flatMap((s) => s.lines);
const films = precontext
.flatMap((p) => (p.name === "object_detection" ? (p.result.detected_objects ?? []) : []))
.filter((o) => o.label.toLowerCase().includes("x-ray"));
const inside = (inner: Bounds, outer: Bounds) =>
inner.top_left.x >= outer.top_left.x &&
inner.top_left.y >= outer.top_left.y &&
inner.bottom_right.x <= outer.bottom_right.x &&
inner.bottom_right.y <= outer.bottom_right.y;
const marker = lines.find((l) => /^[LR]$/.test(l.text) && films.some((f) => inside(l.bounds, f.bounds)));
const markerSide = marker?.text === "R" ? "right" : marker?.text === "L" ? "left" : undefined;
console.log(markerSide === note.side ? "auto" : "review", "side", note.side, marker?.text);
const REVIEW_BELOW = 0.9;
const fields = {
mrn: note.mrn,
patient_name: note.patient_name,
date_of_birth: note.date_of_birth,
blood_pressure: note.vitals.blood_pressure,
heart_rate: String(note.vitals.heart_rate),
respiratory_rate: String(note.vitals.respiratory_rate),
temperature_c: String(note.vitals.temperature_c),
};
for (const [field, value] of Object.entries(fields)) {
const confidence = lines.find((l) => l.text.includes(value))?.average_confidence ?? 0;
console.log(confidence >= REVIEW_BELOW ? "auto" : "review", field, value, confidence);
}
```
**Vercel AI SDK · typescript**
```typescript
import { createInterfaze } from "@interfaze-ai/ai-sdk";
import { generateText, Output } from "ai";
import { z } from "zod";
const interfaze = createInterfaze({ apiKey: process.env.INTERFAZE_API_KEY });
const ConsultSchema = z.object({
mrn: z.string(),
patient_name: z.string(),
date_of_birth: z.string(),
visit_date: z.string(),
side: z.enum(["left", "right"]),
xray_views: z.array(z.string()),
xray_impression: z.string(),
diagnosis: z.string(),
vitals: z.object({
blood_pressure: z.string(),
heart_rate: z.number(),
respiratory_rate: z.number(),
temperature_c: z.number(),
}),
});
const { output: note, providerMetadata } = await generateText({
model: interfaze("interfaze-1.1-lite"),
output: Output.object({ schema: ConsultSchema }),
messages: [
{
role: "user",
content: [
{
type: "text",
text: "Extract the patient, X-ray, diagnosis and vitals from this consult note. Locate each X-ray view and its side marker.",
},
{
type: "file",
mediaType: "image/png",
data: new URL("https://r2public.jigsawstack.com/interfaze/examples/health_care_memo_example.png"),
},
],
},
],
});
type Bounds = { top_left: { x: number; y: number }; bottom_right: { x: number; y: number } };
type OcrLine = { text: string; bounds: Bounds; average_confidence: number };
type Precontext = { name: string; result: { sections?: { lines: OcrLine[] }[]; detected_objects?: { label: string; bounds: Bounds }[] } }[];
const precontext = (providerMetadata?.interfaze?.precontext ?? []) as Precontext;
const lines = precontext.flatMap((p) => (p.name === "ocr" ? (p.result.sections ?? []) : [])).flatMap((s) => s.lines);
const films = precontext
.flatMap((p) => (p.name === "object_detection" ? (p.result.detected_objects ?? []) : []))
.filter((o) => o.label.toLowerCase().includes("x-ray"));
const inside = (inner: Bounds, outer: Bounds) =>
inner.top_left.x >= outer.top_left.x &&
inner.top_left.y >= outer.top_left.y &&
inner.bottom_right.x <= outer.bottom_right.x &&
inner.bottom_right.y <= outer.bottom_right.y;
const marker = lines.find((l) => /^[LR]$/.test(l.text) && films.some((f) => inside(l.bounds, f.bounds)));
const markerSide = marker?.text === "R" ? "right" : marker?.text === "L" ? "left" : undefined;
console.log(markerSide === note.side ? "auto" : "review", "side", note.side, marker?.text);
const REVIEW_BELOW = 0.9;
const fields = {
mrn: note.mrn,
patient_name: note.patient_name,
date_of_birth: note.date_of_birth,
blood_pressure: note.vitals.blood_pressure,
heart_rate: String(note.vitals.heart_rate),
respiratory_rate: String(note.vitals.respiratory_rate),
temperature_c: String(note.vitals.temperature_c),
};
for (const [field, value] of Object.entries(fields)) {
const confidence = lines.find((l) => l.text.includes(value))?.average_confidence ?? 0;
console.log(confidence >= REVIEW_BELOW ? "auto" : "review", field, value, confidence);
}
```
**LangChain SDK · typescript**
```typescript
import { ChatInterfaze } from "@interfaze-ai/langchain";
import { AIMessage, HumanMessage } from "@langchain/core/messages";
import { z } from "zod";
const interfaze = new ChatInterfaze({ model: "interfaze-1.1-lite", apiKey: process.env.INTERFAZE_API_KEY });
const ConsultSchema = z.object({
mrn: z.string(),
patient_name: z.string(),
date_of_birth: z.string(),
visit_date: z.string(),
side: z.enum(["left", "right"]),
xray_views: z.array(z.string()),
xray_impression: z.string(),
diagnosis: z.string(),
vitals: z.object({
blood_pressure: z.string(),
heart_rate: z.number(),
respiratory_rate: z.number(),
temperature_c: z.number(),
}),
});
const response = await interfaze.withStructuredOutput(ConsultSchema, { includeRaw: true }).invoke([
new HumanMessage({
content: [
{
type: "text",
text: "Extract the patient, X-ray, diagnosis and vitals from this consult note. Locate each X-ray view and its side marker.",
},
{ type: "image_url", image_url: { url: "https://r2public.jigsawstack.com/interfaze/examples/health_care_memo_example.png" } },
],
}),
]);
type Bounds = { top_left: { x: number; y: number }; bottom_right: { x: number; y: number } };
type OcrLine = { text: string; bounds: Bounds; average_confidence: number };
type Precontext = { name: string; result: { sections?: { lines: OcrLine[] }[]; detected_objects?: { label: string; bounds: Bounds }[] } }[];
const note = response.parsed;
const precontext = ((response.raw as AIMessage).response_metadata.precontext ?? []) as Precontext;
const lines = precontext.flatMap((p) => (p.name === "ocr" ? (p.result.sections ?? []) : [])).flatMap((s) => s.lines);
const films = precontext
.flatMap((p) => (p.name === "object_detection" ? (p.result.detected_objects ?? []) : []))
.filter((o) => o.label.toLowerCase().includes("x-ray"));
const inside = (inner: Bounds, outer: Bounds) =>
inner.top_left.x >= outer.top_left.x &&
inner.top_left.y >= outer.top_left.y &&
inner.bottom_right.x <= outer.bottom_right.x &&
inner.bottom_right.y <= outer.bottom_right.y;
const marker = lines.find((l) => /^[LR]$/.test(l.text) && films.some((f) => inside(l.bounds, f.bounds)));
const markerSide = marker?.text === "R" ? "right" : marker?.text === "L" ? "left" : undefined;
console.log(markerSide === note.side ? "auto" : "review", "side", note.side, marker?.text);
const REVIEW_BELOW = 0.9;
const fields = {
mrn: note.mrn,
patient_name: note.patient_name,
date_of_birth: note.date_of_birth,
blood_pressure: note.vitals.blood_pressure,
heart_rate: String(note.vitals.heart_rate),
respiratory_rate: String(note.vitals.respiratory_rate),
temperature_c: String(note.vitals.temperature_c),
};
for (const [field, value] of Object.entries(fields)) {
const confidence = lines.find((l) => l.text.includes(value))?.average_confidence ?? 0;
console.log(confidence >= REVIEW_BELOW ? "auto" : "review", field, value, confidence);
}
```
**Interfaze SDK · python**
```python
import os
from typing import List, Literal
from pydantic import BaseModel
from interfaze import Interfaze
interfaze = Interfaze(api_key=os.environ["INTERFAZE_API_KEY"])
class Vitals(BaseModel):
blood_pressure: str
heart_rate: int
respiratory_rate: int
temperature_c: float
class ConsultNote(BaseModel):
mrn: str
patient_name: str
date_of_birth: str
visit_date: str
side: Literal["left", "right"]
xray_views: List[str]
xray_impression: str
diagnosis: str
vitals: Vitals
response = interfaze.chat.completions.parse(
model="interfaze-1.1-lite",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Extract the patient, X-ray, diagnosis and vitals from this consult note. Locate each X-ray view and its side marker."},
{"type": "image_url", "image_url": {"url": "https://r2public.jigsawstack.com/interfaze/examples/health_care_memo_example.png"}},
],
}
],
response_format=ConsultNote,
)
note = response.choices[0].message.parsed
precontext = response.precontext or []
lines = [l for p in precontext if p["name"] == "ocr" for s in p["result"].get("sections", []) for l in s["lines"]]
films = [
o
for p in precontext
if p["name"] == "object_detection"
for o in p["result"].get("detected_objects", [])
if "x-ray" in o["label"].lower()
]
def inside(inner, outer):
return (
inner["top_left"]["x"] >= outer["top_left"]["x"]
and inner["top_left"]["y"] >= outer["top_left"]["y"]
and inner["bottom_right"]["x"] <= outer["bottom_right"]["x"]
and inner["bottom_right"]["y"] <= outer["bottom_right"]["y"]
)
marker = next((l for l in lines if l["text"] in ("L", "R") and any(inside(l["bounds"], f["bounds"]) for f in films)), None)
marker_side = {"L": "left", "R": "right"}.get(marker["text"]) if marker else None
print("auto" if marker_side == note.side else "review", "side", note.side, marker["text"] if marker else None)
REVIEW_BELOW = 0.9
fields = {
"mrn": note.mrn,
"patient_name": note.patient_name,
"date_of_birth": note.date_of_birth,
"blood_pressure": note.vitals.blood_pressure,
"heart_rate": str(note.vitals.heart_rate),
"respiratory_rate": str(note.vitals.respiratory_rate),
"temperature_c": str(note.vitals.temperature_c),
}
for field, value in fields.items():
line = next((l for l in lines if value in l["text"]), None)
confidence = line["average_confidence"] if line else 0
print("auto" if confidence >= REVIEW_BELOW else "review", field, value, confidence)
```
**LangChain SDK · python**
```python
import os
from typing import List, Literal
from pydantic import BaseModel
from langchain_core.messages import HumanMessage
from interfaze_langchain import ChatInterfaze
interfaze = ChatInterfaze(model="interfaze-1.1-lite", api_key=os.environ["INTERFAZE_API_KEY"])
class Vitals(BaseModel):
blood_pressure: str
heart_rate: int
respiratory_rate: int
temperature_c: float
class ConsultNote(BaseModel):
mrn: str
patient_name: str
date_of_birth: str
visit_date: str
side: Literal["left", "right"]
xray_views: List[str]
xray_impression: str
diagnosis: str
vitals: Vitals
response = interfaze.with_structured_output(ConsultNote, include_raw=True).invoke([
HumanMessage(content=[
{"type": "text", "text": "Extract the patient, X-ray, diagnosis and vitals from this consult note. Locate each X-ray view and its side marker."},
{"type": "image_url", "image_url": {"url": "https://r2public.jigsawstack.com/interfaze/examples/health_care_memo_example.png"}},
])
])
note = response["parsed"]
precontext = response["raw"].response_metadata.get("precontext", [])
lines = [l for p in precontext if p["name"] == "ocr" for s in p["result"].get("sections", []) for l in s["lines"]]
films = [
o
for p in precontext
if p["name"] == "object_detection"
for o in p["result"].get("detected_objects", [])
if "x-ray" in o["label"].lower()
]
def inside(inner, outer):
return (
inner["top_left"]["x"] >= outer["top_left"]["x"]
and inner["top_left"]["y"] >= outer["top_left"]["y"]
and inner["bottom_right"]["x"] <= outer["bottom_right"]["x"]
and inner["bottom_right"]["y"] <= outer["bottom_right"]["y"]
)
marker = next((l for l in lines if l["text"] in ("L", "R") and any(inside(l["bounds"], f["bounds"]) for f in films)), None)
marker_side = {"L": "left", "R": "right"}.get(marker["text"]) if marker else None
print("auto" if marker_side == note.side else "review", "side", note.side, marker["text"] if marker else None)
REVIEW_BELOW = 0.9
fields = {
"mrn": note.mrn,
"patient_name": note.patient_name,
"date_of_birth": note.date_of_birth,
"blood_pressure": note.vitals.blood_pressure,
"heart_rate": str(note.vitals.heart_rate),
"respiratory_rate": str(note.vitals.respiratory_rate),
"temperature_c": str(note.vitals.temperature_c),
}
for field, value in fields.items():
line = next((l for l in lines if value in l["text"]), None)
confidence = line["average_confidence"] if line else 0
print("auto" if confidence >= REVIEW_BELOW else "review", field, value, confidence)
```
### The output
In our run on Interfaze 1.1 Lite, the schema response copied the handwriting, the X-ray impression and the diagnosis word for word:
```json
{
"mrn": "WVSC-24-006721",
"patient_name": "Sarah K. Miller",
"date_of_birth": "1995-11-03",
"visit_date": "2024-05-13",
"side": "right",
"xray_views": ["AP", "Lateral", "Mortise"],
"xray_impression": "Oblique fracture of the distal fibula (lateral malleolus) at the level of the syndesmosis. Mild soft tissue swelling. Ankle mortise appears congruent.",
"diagnosis": "Weber B fracture - Right distal fibula (Stable ankle mortise)",
"vitals": {
"blood_pressure": "118/76",
"heart_rate": 78,
"respiratory_rate": 16,
"temperature_c": 36.7
}
}
```
Here's the precontext behind it, trimmed to the X-ray, its side marker and the heart rate line, without the outlines and per-word boxes:
```json
[
{
"name": "object_detection",
"result": {
"detected_objects": [
{
"label": "X-ray image",
"bounds": { "top_left": { "x": 420, "y": 367 }, "bottom_right": { "x": 934, "y": 624 }, "width": 514, "height": 257 }
},
{
"label": "side marker",
"bounds": { "top_left": { "x": 628, "y": 386 }, "bottom_right": { "x": 663, "y": 418 }, "width": 35, "height": 32 }
}
]
}
},
{
"name": "ocr",
"result": {
"sections": [
{
"lines": [
{
"text": "R",
"bounds": { "top_left": { "x": 634, "y": 388 }, "bottom_right": { "x": 656, "y": 415 } },
"average_confidence": 1
},
{
"text": "HR: 78",
"bounds": { "top_left": { "x": 1080, "y": 678 }, "bottom_right": { "x": 1166, "y": 704 } },
"average_confidence": 0.87
}
]
}
],
"width": 1448,
"height": 1086
}
}
]
```
And the checks print:
```text
auto side right R
auto mrn WVSC-24-006721 1
auto patient_name Sarah K. Miller 1
auto date_of_birth 1995-11-03 1
auto blood_pressure 118/76 0.93
review heart_rate 78 0.87
review respiratory_rate 16 0.88
auto temperature_c 36.7 0.91
```
On the note itself, the X-ray is boxed in blue and its side marker in purple. Green fields post on their own, and red fields go to review.
The side check passes. Object detection found the X-ray, and OCR read an "R" inside it at 1.0, which matches the `right` in the note.
The handwritten heart rate read at 0.87 and the respiratory rate at 0.88, so both go to a nurse. Both values were right, and the boxes show the reviewer exactly where to look.
A chat model could have returned the same JSON, but you'd have no way to tell which vital was shaky or whether the film matched the note.
- [Try it in the playground](https://interfaze.ai/dashboard/playground)
- [Interfaze 1.1 Lite model card](https://interfaze.ai/docs/models/interfaze-1.1-lite)
- [Weights on Hugging Face](https://huggingface.co/interfaze-ai/interfaze-1-lite)
- [Compare the models](https://interfaze.ai/docs/models)
- [Precontext docs](https://interfaze.ai/docs/precontext)
- [Get an API key](https://interfaze.ai/dashboard)