Interfaze

Beta

pricing

help

docs

blog

Run Tasks

copy markdown

Interfaze architecture allows you to programmatically run parts of the model or built-in tools without activating the full model making it significantly faster and cheaper.

Available tasks

Task Name	Description
`ocr`	Optical character recognition on images and documents
`object_detection`	Detect objects in images
`gui_detection`	Detect GUI elements in images
`web_search`	Web search
`scraper`	Extract structured data from web pages
`speech_to_text`	Speech to text transcription
`translate`	Translation
`forecast`	Forecast time series data

Limits

Only one task can be run at a time.
The structured output is fixed for the task and cannot be customized.

With the Interfaze SDK, use a tasks.* helper when all you need to pass is a source, or set the task parameter on a normal completion when the message has multiple parts. The result comes back on message.content under a result key.

With the LangChain SDK, put the task name in the first system message and invoke the model as usual. The result comes back as JSON on the message content, so no response format is needed.

With any other SDK:

The system prompt has to contain the task name in the format <task>task_name</task>
The structured output response format has to be a type of any or empty schema.

Example of system prompt:

<task>web_search</task>

Example of running a task

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.transcribe("https://r2public.jigsawstack.com/interfaze/examples/stt_long_audio_sample_3.mp3");

console.log(result);

Output

The output will always be a structured output with name of the task and the raw result
The result schema is different depending on the task
The result is the raw result of the specific model layer or tool
Each task will have a consistent structure on evert run

The output is truncated for this example.

Examples for other tasks

Each task uses the same structure shown above. With the Interfaze SDK, call the matching tasks.* helper. With the LangChain SDK, pass <task>task_name</task> as the system message and invoke the model as usual. With any other SDK, set the system prompt to <task>task_name</task> and pass an empty (any) response format, with the user message carrying the input for the task.

OCR (`ocr`)

Extract text from images, scanned documents and PDFs. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.ocr("https://r2public.jigsawstack.com/interfaze/examples/id.jpg");

console.log(result);

Output

{
  "object": {
    "name": "ocr",
    "result": {
      "extracted_text": "California\nUSA\nDRIVER LICENSE\nDL Y4067081\nCLASS C\nEXP 09/12/2027\nEN MUÑOZ ESTRADA\nFN IVÁN ICHET\n14223 BELGATE ST\nBALDWIN PARK CA 91706\nDOB 09/12/1987\nSEX M HAIR BLK EYES BLK\nHGT 5-02\" WGT 185lb",
      "sections": [
        {
          "text": "DRIVER LICENSE",
          "lines": [
            {
              "text": "DRIVER LICENSE",
              "bounds": {
                "top_left": { "x": 63, "y": 89 },
                "top_right": { "x": 268, "y": 89 },
                "bottom_right": { "x": 268, "y": 129 },
                "bottom_left": { "x": 63, "y": 129 },
                "width": 205,
                "height": 40
              },
              "average_confidence": 0.99
            }
          ]
        }
      ],
      "language": "en"
    }
  }
}

The output is truncated for this example.

Object detection (`object_detection`)

Detect objects in images and return their bounding boxes. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.objectDetection("https://r2public.jigsawstack.com/interfaze/examples/construction.png");

console.log(result);

Output

{
  "object": {
    "name": "object_detection",
    "result": {
      "detected_objects": [
        {
          "bounds": {
            "top_left": { "x": 630, "y": 139 },
            "top_right": { "x": 769, "y": 139 },
            "bottom_left": { "x": 630, "y": 225 },
            "bottom_right": { "x": 769, "y": 225 },
            "width": 139,
            "height": 86
          },
          "label": "crane"
        }
      ],
      "gui_elements": [
        {
          "type": "text",
          "bounds": {
            "top_left": { "x": 1140, "y": 722 },
            "top_right": { "x": 1232, "y": 722 },
            "bottom_left": { "x": 1140, "y": 752 },
            "bottom_right": { "x": 1232, "y": 752 },
            "width": 92,
            "height": 30
          },
          "interactivity": false,
          "content": "tower"
        }
      ]
    }
  }
}

GUI detection (`gui_detection`)

Detect interactive UI elements in screenshots — useful for computer-use and agent workflows. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.guiDetection("https://r2public.jigsawstack.com/interfaze/examples/computer_use.jpg");

console.log(result);

Output

{
  "object": {
    "name": "gui_detection",
    "result": {
      "gui_elements": [
        {
          "type": "button",
          "top_left_x": 1120,
          "top_left_y": 18,
          "bottom_right_x": 1192,
          "bottom_right_y": 44
        },
        {
          "type": "input",
          "top_left_x": 312,
          "top_left_y": 12,
          "bottom_right_x": 692,
          "bottom_right_y": 42
        },
        {
          "type": "link",
          "top_left_x": 72,
          "top_left_y": 64,
          "bottom_right_x": 116,
          "bottom_right_y": 88
        },
        {
          "type": "dropdown",
          "top_left_x": 720,
          "top_left_y": 64,
          "bottom_right_x": 820,
          "bottom_right_y": 90
        }
      ]
    }
  }
}

Web search (`web_search`)

Search the web and return ranked results with titles, descriptions and URLs. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.webSearch("GLP-1 research paper");

console.log(result);

Output

{
  "object": {
    "name": "web_search",
    "result": [
      {
        "title": "Glucagon-like peptide 1 (GLP-1) - PubMed",
        "description": "The glucagon-like peptide-1 (GLP-1) is a multifaceted hormone with broad pharmacological potential.",
        "content": "The glucagon-like peptide-1 (GLP-1) is a multifaceted hormone with broad pharmacological potential.",
        "url": "https://pubmed.ncbi.nlm.nih.gov/31767182/"
      },
      {
        "title": "Mapping the effectiveness and risks of GLP-1 receptor agonists - PubMed",
        "description": "Glucagon-like peptide 1 receptor agonists (GLP-1RAs) are increasingly being used to treat diabetes and obesity.",
        "content": "Glucagon-like peptide 1 receptor agonists (GLP-1RAs) are increasingly being used to treat diabetes and obesity.",
        "url": "https://pubmed.ncbi.nlm.nih.gov/39833406/"
      }
    ]
  }
}

Scraper (`scraper`)

Extract structured content from a URL. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.scrape("https://news.ycombinator.com");

console.log(result);

Output

{
  "object": {
    "name": "ai_scraper",
    "result": {
      "scraped_content": {
        "title": ["Google releases Gemma 4 open models", "Tailscale's new macOS home", "Cursor 3", "Artemis II's toilet is a moon mission milestone"],
        "points": ["962 points", "238 points", "221 points", "67 points"]
      },
      "scraped_elements": [
        {
          "selector": "#hnmain .hnname a",
          "results": [
            {
              "html": "Hacker News",
              "text": "Hacker News",
              "attributes": [{ "name": "href", "value": "news" }]
            }
          ],
          "key": "title"
        }
      ]
    }
  }
}

The output is truncated for this example.

Translate (`translate`)

Translate text between languages with context-aware accuracy. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

const result = await interfaze.tasks.translate(
	"The UK drinks about 100–160 million cups of tea every day, and 98% of tea drinkers add milk to their tea.",
	{ to: "French" }
);

console.log(result);

Output

{
  "object": {
    "name": "translate",
    "result": {
      "translated_text": "Le Royaume-Uni boit environ 100–160 millions de tasses de thé chaque jour, et 98 % des buveurs de thé ajoutent du lait à leur thé.",
      "source_language": "en",
      "target_language": "fr"
    }
  }
}

Forecast (`forecast`)

Forecast time series data by passing the historical values directly as text. Learn more.

Interfaze SDK

Vercel AI SDK

LangChain SDK

// for a hosted CSV, use interfaze.tasks.forecast(csvUrl, { periods: 7, unit: "days" })
const salesData = `date,sales_revenue
2024-06-01,18450
2024-06-02,16720
2024-06-03,9840
2024-06-04,10230
2024-06-05,11150
2024-06-06,12480
2024-06-07,15630
2024-06-08,19240
2024-06-09,17010
2024-06-10,9560
2024-06-11,10870
2024-06-12,11320
2024-06-13,12940
2024-06-14,16110`;

const response = await interfaze.chat.completions.create({
	task: "forecast",
	messages: [
		{
			role: "user",
			content: `Forecast the next 7 days of daily sales revenue for this retail store:\n${salesData}`,
		},
	],
});

const { result } = JSON.parse(response.choices[0]?.message.content ?? "{}");
console.log(result);

Output

{
  "object": {
    "name": "forecast",
    "result": {
      "predictions": [
        { "date": "2024-06-15 00:00:00", "value": 20330 },
        { "date": "2024-06-16 00:00:00", "value": 17650 },
        { "date": "2024-06-17 00:00:00", "value": 10120 },
        { "date": "2024-06-18 00:00:00", "value": 10640 },
        { "date": "2024-06-19 00:00:00", "value": 11890 },
        { "date": "2024-06-20 00:00:00", "value": 13260 },
        { "date": "2024-06-21 00:00:00", "value": 16780 }
      ]
    }
  }
}

Common issues faced

Only one <task> tag is parsed from the system message (the first match). One task can only be run at a time.
If a non-empty schema is provided alongside a <task> tag, it will result in 400 status code error.
The response is the raw task output, not a natural language summary — plan your downstream processing accordingly.

Precontext

Structured Outputs