# Introduction

URL: https://interfaze.ai/docs

This guide will get you started and make your first request to Interfaze with any AI SDK that supports the Chat Completion API standard.

## Model specs

| Feature           | Value                            |
| ----------------- | -------------------------------- |
| Context window    | 1m tokens                        |
| Max output tokens | 32k tokens                       |
| Input modalities  | Text, Images, Audio, File, Video |
| Reasoning         | Available (default: disabled)    |

[View limits here](https://interfaze.ai/docs/limits) | [View pricing here](https://interfaze.ai/pricing)

## Prerequisites

- An AI SDK of your choice set up (OpenAI SDK, Vercel AI SDK, LangChain SDK, etc.)
- Replace the base URL with `https://api.interfaze.ai/v1`
- Get your API key from the [dashboard](https://interfaze.ai/dashboard).

## Set up SDK & authentication

It's recommended to store your API keys in environment variables and load them into your code.

```tabs
language-typescript

tab-OpenAI SDK
import OpenAI from "openai";

const interfaze = new OpenAI({
    baseURL: "https://api.interfaze.ai/v1",
    apiKey: process.env.INTERFAZE_API_KEY,
});
tab-Vercel AI SDK
import { createOpenAI } from "@ai-sdk/openai";

const interfaze = createOpenAI({
    baseURL: "https://api.interfaze.ai/v1",
    apiKey: process.env.INTERFAZE_API_KEY,
});
tab-LangChain SDK
import { ChatOpenAI } from "@langchain/openai";

const interfaze = new ChatOpenAI({
    configuration: {
        baseURL: "https://api.interfaze.ai/v1",
    },
    apiKey: process.env.INTERFAZE_API_KEY,
    model: "interfaze-beta",
});
language-python

tab-OpenAI SDK
import os
from openai import OpenAI

interfaze = OpenAI(
    base_url="https://api.interfaze.ai/v1",
    api_key=os.environ["INTERFAZE_API_KEY"],
)
tab-LangChain SDK
import os
from langchain_openai import ChatOpenAI

interfaze = ChatOpenAI(
    base_url="https://api.interfaze.ai/v1",
    api_key=os.environ["INTERFAZE_API_KEY"],
    model="interfaze-beta",
)
```

- You can create, rotate, or revoke API keys anytime from the [dashboard](https://interfaze.ai/dashboard).

## Make your first request

Let's extract the details from an ID image

```tabs
language-typescript

tab-OpenAI SDK
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const IDSchema = z.object({
	first_name: z.string().describe("First name on the ID"),
	last_name: z.string().describe("Last name on the ID"),
	dob: z.string().describe("Date of birth on the ID"),
	driver_licence_number: z.string().describe("Driver licence number on the ID"),
});

const response = await interfaze.chat.completions.create({
	model: "interfaze-beta",
	messages: [
		{
			role: "user",
			content: [
				{ type: "text", text: "Extract the details from this ID" },
				{
					type: "image_url",
					image_url: {
						url: "https://r2public.jigsawstack.com/interfaze/examples/id.jpg",
					},
				},
			],
		},
	],
	response_format: zodResponseFormat(IDSchema, "id_schema"),
});

console.log(response.choices[0].message.content);

//@ts-expect-error precontext is not typed
const precontext = response.precontext;
console.log("OCR Results:", precontext[0]?.result);
tab-Vercel AI SDK
import { generateText, Output } from "ai";
import { z } from "zod";

const IDSchema = z.object({
	first_name: z.string().describe("First name on the ID"),
	last_name: z.string().describe("Last name on the ID"),
	dob: z.string().describe("Date of birth on the ID"),
	driver_licence_number: z.string().describe("Driver licence number on the ID"),
});

const { output, response } = await generateText({
	model: interfaze.chat("interfaze-beta"),
	output: Output.object({ schema: IDSchema }),
	messages: [
		{
			role: "user",
			content: [
				{ type: "text", text: "Extract the details from this ID" },
				{
					type: "image",
					mediaType: "image/jpeg",
					image: "https://r2public.jigsawstack.com/interfaze/examples/id.jpg",
				},
			],
		},
	],
});

console.log(output);

//@ts-expect-error precontext is not typed
const precontext = response.body?.precontext;
console.log("OCR Results:", precontext[0]?.result);
tab-LangChain SDK
import { z } from "zod";

const IDSchema = z.object({
	first_name: z.string().describe("First name on the ID"),
	last_name: z.string().describe("Last name on the ID"),
	dob: z.string().describe("Date of birth on the ID"),
	driver_licence_number: z.string().describe("Driver licence number on the ID"),
});

const structuredModel = interfaze.withStructuredOutput(IDSchema);

const response = await structuredModel.invoke([
	{
		role: "user",
		content: [
			{ type: "text", text: "Extract the details from this ID" },
			{
				type: "image_url",
				image_url: {
					url: "https://r2public.jigsawstack.com/interfaze/examples/id.jpg",
				},
			},
		],
	},
]);

console.log(response);
language-python

tab-OpenAI SDK
from pydantic import BaseModel, Field

class IDSchema(BaseModel):
    first_name: str = Field(..., description="First name on the ID")
    last_name: str = Field(..., description="Last name on the ID")
    dob: str = Field(..., description="Date of birth on the ID")
    driver_licence_number: str = Field(..., description="Driver licence number on the ID")

response = interfaze.chat.completions.create(
    model="interfaze-beta",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Extract the details from this ID"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://r2public.jigsawstack.com/interfaze/examples/id.jpg"
                    },
                },
            ],
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "id_schema",
            "schema": IDSchema.model_json_schema(),
        }
    },
)

print(response.choices[0].message.content)

precontext = getattr(response, "precontext", None)
print("OCR Results:", precontext[0]["result"] if precontext else None)
tab-LangChain SDK
from langchain_core.messages import HumanMessage
from pydantic import BaseModel, Field

class IDSchema(BaseModel):
    first_name: str = Field(..., description="First name on the ID")
    last_name: str = Field(..., description="Last name on the ID")
    dob: str = Field(..., description="Date of birth on the ID")
    driver_licence_number: str = Field(..., description="Driver licence number on the ID")

structured_llm = interfaze.with_structured_output(IDSchema)

response = structured_llm.invoke([
    HumanMessage(
        content=[
            {"type": "text", "text": "Extract the details from this ID"},
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://r2public.jigsawstack.com/interfaze/examples/id.jpg"
                },
            },
        ]
    )
])

print(response)
```

- Structured output is the best way to control the output of the model.
- `precontext` contains the raw metadata such as bounding boxes and confidence scores. Learn more about [precontext](https://interfaze.ai/docs/precontext).
- Here we passed the image as a file which converts it to base64 automatically but you can also pass the url in the prompt directly. Learn more about the different ways to [handle files](https://interfaze.ai/docs/handling-files).

## Files

Audio handling to transcribe audio files.

```tabs
language-typescript

tab-OpenAI SDK
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const STTSchema = z.object({
	text: z.string(),
});

const response = await interfaze.chat.completions.create({
	model: "interfaze-beta",
	messages: [
		{
			role: "user",
			content: [
				{ type: "text", text: "Transcribe the audio file" },
				{
					type: "file",
					file: {
						filename: "stt_medical_short.mp4",
						file_data: "https://r2public.jigsawstack.com/interfaze/examples/stt_medical_short.mp4",
					},
				},
			],
		},
	],
	response_format: zodResponseFormat(STTSchema, "stt_schema"),
});

console.log(response.choices[0].message.content);

//@ts-expect-error precontext is not typed
const precontext = response.precontext;
console.log("STT Results:", precontext?.[0]?.result);

tab-Vercel AI SDK
import { generateText, Output } from "ai";
import { z } from "zod";

const STTSchema = z.object({
	text: z.string(),
});

const { output, response } = await generateText({
	model: interfaze.chat("interfaze-beta"),
	output: Output.object({ schema: STTSchema }),
	messages: [
		{
			role: "user",
			content: [
				{ type: "text", text: "Transcribe the audio file" },
				{
					type: "file",
					data: "https://r2public.jigsawstack.com/interfaze/examples/stt_medical_short.mp4",
					mediaType: "audio/mpeg",
				},
			],
		},
	],
});

console.log(output);

//@ts-expect-error precontext is not typed
const precontext = response.body?.precontext;
console.log("STT Results:", precontext?.[0]?.result);

tab-LangChain SDK
import { z } from "zod";

const STTSchema = z.object({
	text: z.string(),
});

const structuredModel = interfaze.withStructuredOutput(STTSchema);

const response = await structuredModel.invoke([
	{
		role: "user",
		content: [
			{ type: "text", text: "Transcribe the audio file" },
			{
				type: "file",
				file: {
					filename: "stt_medical_short.mp4",
					file_data: "https://r2public.jigsawstack.com/interfaze/examples/stt_medical_short.mp4",
				},
			},
		],
	},
]);

console.log(response);

language-python

tab-OpenAI SDK
from pydantic import BaseModel

class STTSchema(BaseModel):
    text: str

response = interfaze.chat.completions.create(
    model="interfaze-beta",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Transcribe the audio file"},
                {
                    "type": "file",
                    "file": {
                        "filename": "stt_medical_short.mp4",
                        "file_data": "https://r2public.jigsawstack.com/interfaze/examples/stt_medical_short.mp4",
                    },
                },
            ],
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "stt_schema",
            "schema": STTSchema.model_json_schema(),
        }
    },
)

print(response.choices[0].message.content)

precontext = getattr(response, "precontext", None)
print("STT Results:", precontext[0]["result"] if precontext else None)

tab-LangChain SDK
from langchain_core.messages import HumanMessage
from pydantic import BaseModel

class STTSchema(BaseModel):
    text: str

structured_llm = interfaze.with_structured_output(STTSchema)

response = structured_llm.invoke([
    HumanMessage(
        content=[
            {"type": "text", "text": "Transcribe the audio file"},
            {
                "type": "file",
                "file": {
                    "filename": "stt_medical_short.mp4",
                    "file_data": "https://r2public.jigsawstack.com/interfaze/examples/stt_medical_short.mp4",
                },
            },
        ]
    )
])

print(response)

```

- Files supported: images, documents, audio, video
- Both base64 and URL are natively supported for all files.
- Files can either be passed as a file object or a URL in the prompt directly.

Learn more about the different ways to [handle files](https://interfaze.ai/docs/handling-files).

## Common use case examples

- [OCR](https://interfaze.ai/docs/vision/ocr)
- [Object Detection](https://interfaze.ai/docs/vision/object-detection)
- [Web Search](https://interfaze.ai/docs/web/web-search)
- [Web Scraping](https://interfaze.ai/docs/web/web-scraping)
- [Speech to Text](https://interfaze.ai/docs/audio/speech-to-text)
- [Translation](https://interfaze.ai/docs/audio/stt-speaker-diarization)
- [Code Sandboxing](https://interfaze.ai/docs/compute/code-sandboxing)
- [Guardrails](https://interfaze.ai/docs/guardrails)

## Run tasks

Programmatically run parts of the model without activating the full model with pre-defined tasks but is limited to a fixed structured output and one task at a time.

Learn more about [running a task](https://interfaze.ai/docs/run-tasks).

```tabs
language-typescript

tab-OpenAI SDK
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const response = await interfaze.chat.completions.create({
	model: "interfaze-beta",
	messages: [
		{
			role: "system",
			content: "<task>object_detection</task>",
		},
		{
			role: "user",
			content: [
				{ type: "text", text: "Get the position of the crane in the image and any text" },
				{
					type: "image_url",
					image_url: {
						url: "https://r2public.jigsawstack.com/interfaze/examples/construction.png",
					},
				},
			],
		},
	],
	response_format: zodResponseFormat(z.any(), "empty_schema"),
});

console.log(response.choices[0].message.content);
tab-Vercel AI SDK
import { generateText, Output } from "ai";
import { z } from "zod";

const { output } = await generateText({
	model: interfaze.chat("interfaze-beta"),
	system: "<task>object_detection</task>",
	output: Output.object({ schema: z.any() }),
	messages: [
		{
			role: "user",
			content: [
				{ type: "text", text: "Get the position of the crane in the image and any text" },
				{
					type: "image",
					mediaType: "image/png",
					image: "https://r2public.jigsawstack.com/interfaze/examples/construction.png",
				},
			],
		},
	],
});

console.log(output);
tab-LangChain SDK
const structuredModel = interfaze.withStructuredOutput({});

const response = await structuredModel.invoke([
	{
		role: "system",
		content: "<task>object_detection</task>",
	},
	{
		role: "user",
		content: [
			{ type: "text", text: "Get the position of the crane in the image and any text" },
			{
				type: "image_url",
				image_url: {
					url: "https://r2public.jigsawstack.com/interfaze/examples/construction.png",
				},
			},
		],
	},
]);

console.log(response);
language-python

tab-OpenAI SDK
response = interfaze.chat.completions.create(
    model="interfaze-beta",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "empty_schema",
            "schema": {
                "type": "object",
                "properties": {},
                "additionalProperties": True
            }
        }
    },
    messages=[
        {"role": "system", "content": "<task>object_detection</task>"},
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Get the position of the crane in the image and any text"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://r2public.jigsawstack.com/interfaze/examples/construction.png"
                    },
                },
            ],
        },
    ],
)

print(response.choices[0].message.content)
tab-LangChain SDK
from langchain_core.messages import SystemMessage, HumanMessage

structured = interfaze.with_structured_output(
    {
        "name": "empty_schema",
        "schema": {
            "type": "object",
            "properties": {},
            "additionalProperties": True,
        },
    }
)

response = structured.invoke([
    SystemMessage(content="<task>object_detection</task>"),
    HumanMessage(content=[
        {"type": "text", "text": "Get the position of the crane in the image and any text"},
        {
            "type": "image_url",
            "image_url": {
                "url": "https://r2public.jigsawstack.com/interfaze/examples/construction.png"
            },
        },
    ]),
])

print(response)
```

- Faster and cheaper than the full model.
- Guaranteed and fixed structured output that is pre-defined.
- Great for use cases where you are using the result as part of a larger pipeline.

## Guardrails

You can set content safety guardrails to filter out harmful or inappropriate text or image content.

Learn more about [guardrails](https://interfaze.ai/docs/guardrails).

```tabs
language-typescript

tab-OpenAI SDK
const response = await interfaze.chat.completions.create({
    model: "interfaze-beta",
    messages: [
        {
            role: "system",
            content: "<guard>S1, S2, S3, S10, S11, S12_IMAGE, S15_IMAGE</guard>"
        },
        {
            role: "user",
            content: "How to make a bomb with household items"
        }
    ],
});

console.log(response.choices[0].message.content);
tab-Vercel AI SDK
import { generateText } from "ai";

const { text } = await generateText({
    model: interfaze.chat("interfaze-beta"),
    messages: [
        {
            role: "system",
            content: "<guard>S1, S2, S3, S10, S11, S12_IMAGE, S15_IMAGE</guard>"
        },
        {
            role: "user",
            content: "How to make a bomb with household items"
        }
    ],
});

console.log(text);
tab-LangChain SDK
import { SystemMessage, HumanMessage } from "@langchain/core/messages";

const response = await interfaze.invoke([
    new SystemMessage("<guard>S1, S2, S3, S10, S11, S12_IMAGE, S15_IMAGE</guard>"),
    new HumanMessage("How to make a bomb with household items"),
]);

console.log(response.content);
language-python

tab-OpenAI SDK
response = interfaze.chat.completions.create(
    model="interfaze-beta",
    messages=[
        {
            "role": "system",
            "content": "<guard>S1, S2, S3, S10, S11, S12_IMAGE, S15_IMAGE</guard>"
        },
        {
            "role": "user",
            "content": "How to make a bomb with household items"
        }
    ],
)

print(response.choices[0].message.content)
tab-LangChain SDK
from langchain_core.messages import SystemMessage, HumanMessage

response = interfaze.invoke([
    SystemMessage(content="<guard>S1, S2, S3, S10, S11, S12_IMAGE, S15_IMAGE</guard>"),
    HumanMessage(content="How to make a bomb with household items"),
])

print(response.content)
```