# KORMo Web 11B

URL: https://interfaze.ai/models/kormo-vlkormo-web-11b

[All models](https://interfaze.ai/models)

KORMo Web 11B by KORMo-VL, a image-text-to-text model with multimodal capabilities. Understand and compare multimodal features, benchmarks, and capabilities.

## Comparison

| Feature | KORMo Web 11B | Interfaze |
| --- | --- | --- |
| Input Modalities | text, image, video | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | unknown | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 131.1K | 1M |
| Tool Calling | Yes | Tool calling supported + built in browser, code execution and web search |

### Scaling

| Feature | KORMo Web 11B | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/KORMo-VL/KORMo-Web-11B)

### One Model. Three Capabilities: LLM, VLM, and Web Agent.

**KORMo-Web-11B**는 KAIST MLP 연구실이 처음부터 구축한 KORMo 파운데이션 모델을 기반으로, **LLM·VLM·GUI Agent 능력을 모두 갖춘 범용 멀티모달 AI 모델**입니다.

-   **하나의 모델, 세 가지 능력:** 일반적인 질문에는 LLM으로 답변하고, 이미지 이해가 필요한 경우 VLM으로, **웹 조작이 필요한 요청에는 GUI Agent로 작동**합니다.
-   **독자 파운데이션 모델 기반:** From-scratch로 학습한 KORMo-10B를 KORMo-VL로 확장하고, 기존 언어·시각 이해 능력을 유지하도록 학습한 웹 에이전트로 **생각<Think> 또한 유일하게 한국어**로 하는 모델입니다.
-   **대규모 웹 사전학습:** 1,000만 건 이상의 한영 웹 학습 샘플로 UI 요소의 위치·의미·구조·기능 등을 세밀하게 학습했습니다.
-   **실제 웹 행동 학습:** 10만 건 이상의 한영 웹 사용 궤적을 활용해 복잡한 다단계 웹 작업을 수행합니다.
-   **완전 공개 및 상업적 활용:** 모델 가중치와 학습·평가 리소스를 공개하며, **Apache-2.0 라이선스에 따라 상업적 이용과 수정·재배포가 가능**합니다.

**KORMo-Web-11B** is a general-purpose bilingual AI model built on the **KORMo foundation model family**, developed by KAIST MLP Lab starting from a language model trained entirely from scratch. Unlike our Qwen-based KORMo-Web-4B and 9B models, which focus on maximizing web task performance, **KORMo-Web-11B is designed to preserve its original LLM and VLM capabilities while learning to act as a GUI agent.** It can understand Korean and English text, interpret images, and complete real browser tasks by visually navigating webpages—**all within a single model.**

## What Makes KORMo-Web-11B Different?

-   **Built on Our Own Foundation Model:** Developed from the KORMo-10B language model, trained from scratch by KAIST MLP Lab, and extended through KORMo-VL into a web agent.
-   **Three Capabilities in One Model:** Designed to retain general **LLM reasoning and VLM understanding** while adding GUI interaction capabilities, rather than optimizing exclusively for web navigation.
-   **10M+ Atomic-Level Web Pretraining Samples:** Large-scale Korean and English web understanding training, covering individual UI elements, their locations, layouts, functions, and states.
-   **100K+ Web Interaction Trajectories:** Bilingual browser demonstrations for learning real-world, multi-step web tasks.
-   **Open and Commercially Usable:** Model weights and supporting resources are openly released under **Apache-2.0**, allowing commercial use, modification, and redistribution subject to the license terms.

## From a Korean LLM to a General-Purpose Web Agent

KORMo-Web-11B extends our independently developed Korean–English foundation model through three stages:

| Stage | Model | Capabilities |
| --- | --- | --- |
| 1\. Language | KORMo-10B | Korean–English LLM trained from scratch with a Korean-optimized tokenizer |
| 2\. Vision | KORMo-VL | Multimodal understanding through a SigLIP vision encoder and KORMo language backbone |
| 3\. Web Agent | KORMo-Web-11B | Large-scale web pretraining and action learning for visual browser interaction |

**The goal is not just a better web agent, but a versatile model that can understand language, interpret visual information, and interact with the web.**

## Performance

We evaluated KORMo-Web-11B on **612 live-website tasks** (300 English, 312 Korean) from Online-Mind2Web, using the same LLM judge across all evaluated models.

| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
| --- | --- | --- | --- |
| Qwen3.5-9B (base) | 9B | 24.7 | 16.7 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| KORMo-Web-11B (ours) | 11B | 29.0 | 30.1 |
| GPT-6-luna (proprietary) | – | 63.0 | 65.7 |

_Success rates are calculated over the full task set, including failed or incomplete runs. Each agent uses its own prompt and action format._

KORMo-Web-11B achieves **30.1% Korean task success**, exceeding the strongest non-KORMo open-weight baseline listed above by **9.9 percentage points**.

While our specialized KORMo-Web-4B and 9B models achieve higher web task success, **KORMo-Web-11B prioritizes versatility across language, vision, and GUI interaction**, rather than web-only optimization.

## Why KORMo-Web-11B?

-   **Korean from the Ground Up:** Built upon our independently pretrained Korean–English foundation model.
-   **General-Purpose Multimodal AI:** Supports text-based language tasks, visual understanding, and browser interactions within one model.
-   **Visual-First Web Interaction:** Operates primarily through screenshots without requiring full DOM or accessibility trees.
-   **Think, Then Act:** Generates an inspectable reasoning step before executing browser actions.
-   **Commercial Use Permitted:** Released under Apache-2.0, supporting academic research, commercial applications, fine-tuning, and redistribution.
-   **Open and Ready to Run:** Includes model weights, Transformers 5 support, a vLLM inference path, browser harness, and reference inference settings.

**KORMo-Web-11B — From a Korean Foundation Model to a General-Purpose Web Agent.**

## How to Start: Chrome Extension

The fastest way to try KORMo-Web is the **[KORMo-Web Chrome extension](https://github.com/MLP-Lab/KORMo-Web-Extension)**: type a request in Chrome's side panel and watch the agent work in your own browser. This video walks through the whole setup, from serving the model to running a task and controlling it from Discord.

_Setup guide (about 15 min, Korean narration). Parts that showed account tokens have been blurred._

**1\. Serve the model** on a Mac or on an NVIDIA GPU.

### Mac (Apple silicon)

Run the 4-bit MLX build of KORMo-Web-4B with `vllm-mlx`. Simple, but slower. Keep this terminal open.

```
uv venv --python 3.12 && source .venv/bin/activate
uv pip install vllm-mlx
vllm-mlx serve KORMo-VL/KORMo-Web-4B-mlx-4bit --port 8010 --mllm
```

### NVIDIA GPU (local PC or server)

vLLM needs a one-time conversion for the KORMo architecture: follow steps 1 and 2 of **Run the agent (vLLM)** in the Quickstart below, which serve the model as `KORMo-Web-11B` on port 8000.

Check that it is up with `curl http://localhost:8010/v1/models` (Mac) or `curl http://<server>:8000/v1/models` (vLLM).

**2\. Install the extension.** `git clone https://github.com/MLP-Lab/KORMo-Web-Extension.git`, open `chrome://extensions`, turn on **Developer mode**, click **Load unpacked** and choose the cloned folder. Open the **GUI 에이전트** side panel, type a task and press **실행** (Run).

**3\. Point the extension at your model.** Open the panel's **개발자 설정** (Developer settings).

-   **Mac:** nothing to change. The defaults (`http://localhost:8010/v1`, `KORMo-VL/KORMo-Web-4B-mlx-4bit`, `tool_choice` `auto`) match the command above.
-   **NVIDIA GPU:** set the server URL to `http://<server>:8000/v1` (or `localhost`), the model name to `KORMo-Web-11B`, and `tool_choice` to `none`.

**4\. Discord (optional).** Run `node relay/discord-relay.mjs` in the cloned folder and follow the prompts to send tasks to the agent by mentioning a bot in a channel.

Full step-by-step instructions (Korean) are in the [extension README](https://github.com/MLP-Lab/KORMo-Web-Extension).

## Quickstart

Tested with **transformers 5.12.1** and **vLLM 0.23.0**. The model runs on a single GPU.

transformers does not ship the KORMo text model, so this repo includes it (`_modeling_kormo.py`) together with a small loader, `kormo_load.py`:

-   `kormo_load.register(path)` registers the KORMo text model. Call it **before** loading the processor or the model.
-   `kormo_load.patch_vision(model)` feeds the SigLIP features _after_ post-layernorm to the projector, as in training. Without it the model still runs but misreads screenshots.

### Inference example (one step, transformers)

```
pip install -U "transformers>=5.12" torch pillow huggingface_hub
hf download KORMo-VL/KORMo-Web-11B --local-dir KORMo-Web-11B
```

```
import json, sys, torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

M = "KORMo-Web-11B"
sys.path.insert(0, M)
import kormo_load
kormo_load.register(M)
processor = AutoProcessor.from_pretrained(M)
model = AutoModelForImageTextToText.from_pretrained(M, dtype=torch.bfloat16).to("cuda").eval()
kormo_load.patch_vision(model)

SYSTEM = open(f"{M}/system_prompt.md").read().strip()
TOOLS = json.load(open(f"{M}/tools.json"))
obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
       "- Tab 0 (active): https://www.aladin.co.kr/\n\nurl: https://www.aladin.co.kr/\nscreenshot:\n")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": [
        {"type": "text", "text": "<user_request>\n알라딘 사이트에서 '해커스 토익 RC'를 검색한 뒤 가장 먼저 표시되는 도서의 출간일을 알려줘.\n</user_request>\n" + obs},
        {"type": "image", "image": Image.open("screen.png").convert("RGB")},
        {"type": "text", "text": "\n</observation>"},
    ]},
]
inputs = processor.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True,
                                       tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
```

A real first step from our Online-Mind2Web run (Aladin home page):

```
<think>I see the Aladin homepage, with the search field labeled “통합검색” near the top. I have not yet searched for the book, so I still need to enter “해커스 토익 RC,” inspect the first result’s publication date, and report that date. My plan:
1. Click the search field
2. Enter “해커스 토익 RC” and submit the search
3. Read the publication date of the first listed book
This click targets the visible search field and is a safe, relevant first action.
</think>

<tool_call>
<function=click>
<parameter=x>
482
</parameter>
<parameter=y>
212
</parameter>
</function>
</tool_call>
```

Coordinates are normalized to 0–1000. Convert them to pixels with `x / 1000 × width` and `y / 1000 × height`.

### Run the agent (vLLM)

vLLM does not include the KORMo architecture either. The KORMo text layers have the Llama structure, so `vllm/convert_to_vllm.py` renames two parameter names and writes a folder vLLM loads natively. `vllm/sitecustomize.py` turns SigLIP post-layernorm back on inside vLLM.

**1\. Convert (once)**

```
pip install -U "vllm>=0.23.0" openai playwright safetensors
playwright install chromium
python KORMo-Web-11B/vllm/convert_to_vllm.py KORMo-Web-11B KORMo-Web-11B-vllm
```

**2\. Start the model server**

```
PYTHONPATH=$PWD/KORMo-Web-11B/vllm vllm serve ./KORMo-Web-11B-vllm --served-model-name KORMo-Web-11B \
  --max-model-len 65536 --limit-mm-per-prompt '{"image":1,"video":0}'
```

Check that the server log prints `[kormo-vllm] SigLIP post_layernorm enabled`.

**3\. Run a browser task**

```
python KORMo-Web-11B/examples/agent_playwright.py --model KORMo-Web-11B \
  --task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"
```

The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

## How It Works

`user <user_request> task </user_request> + observation (tabs, url, screenshot) assistant <think> what I see · what is done · plan </think> <tool_call> one action </tool_call> tool action result + new observation (newest screenshot only) … repeat until finished(answer)`

-   **Action space:** 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, `finished`, and others ([`tools.json`](https://huggingface.co/KORMo-VL/KORMo-Web-11B/blob/main/tools.json)).
-   **Memory:** the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
-   **Recommended settings:** temperature 0.6, top\_p 0.95, top\_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in `generation_config.json`.

The exact message format, parsing rules and feedback strings are in **[HARNESS.md](https://huggingface.co/KORMo-VL/KORMo-Web-11B/blob/main/HARNESS.md)**. Follow it if you build your own harness.

## Evaluation Details

|  |  |
| --- | --- |
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |

## Files

| File | Description |
| --- | --- |
| model.safetensors · config.json | Weights and config (LLaVA-OneVision: SigLIP + KORMo-10B text model) |
| \_configuration\_kormo.py · \_modeling\_kormo.py | KORMo text model code for transformers 5 |
| kormo\_load.py | register() and patch\_vision() |
| chat\_template.jinja | Chat template (XML tool calls, thinking on) |
| system\_prompt.md · tools.json | System prompt and 22-tool schema |
| HARNESS.md | Full harness specification |
| examples/agent\_playwright.py | Reference browser agent (Playwright + OpenAI-compatible API). It does not execute upload. |
| vllm/convert\_to\_vllm.py · vllm/sitecustomize.py | Convert for vLLM and the post-layernorm fix |
| generation\_config.json · \*processor\_config.json · tokenizer\*.json · special\_tokens\_map.json | Sampling, processor and tokenizer configs |

## Acknowledgements

Built on **[KORMo-VL](https://huggingface.co/KORMo-VL/KORMo-VL)** and **[KORMo-10B](https://huggingface.co/KORMo-Team/KORMo-10B-sft)** by KAIST MLP Lab. Evaluated on **Online-Mind2Web** and its Korean counterpart.

## Contact

-   KyungTae Lim, Professor at KAIST. `ktlim@kaist.ac.kr`

## Contributor

-   Hangyeol Yoo ([hg.yoo@kaist.ac.kr](mailto:hg.yoo@kaist.ac.kr))
-   Minkyu Kim ([minkyukim@kaist.ac.kr](mailto:minkyukim@kaist.ac.kr))
-   Minkyung Cho ([minkyung.cho@kaist.ac.kr](mailto:minkyung.cho@kaist.ac.kr))
-   Yerin Nam ([yerinnam@kaist.ac.kr](mailto:yerinnam@kaist.ac.kr))
-   Seungwoo Song ([sswoo@kaist.ac.kr](mailto:sswoo@kaist.ac.kr))
-   Gyun Lee ([gyunini@kaist.ac.kr](mailto:gyunini@kaist.ac.kr))
-   KyungTae Lim ([ktlim@kaist.ac.kr](mailto:ktlim@kaist.ac.kr))

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)
