# Qwythos 27B V1

URL: https://interfaze.ai/models/empero-aiqwythos-27b-v1

[All models](https://interfaze.ai/models)

Qwythos 27B V1 by empero-ai, a image-text-to-text model with multimodal capabilities. Understand and compare multimodal features, benchmarks, and capabilities.

## Comparison

| Feature | Qwythos 27B V1 | Interfaze |
| --- | --- | --- |
| Input Modalities | text, image | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | Yes | Yes |
| Language Support | 12 partial | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 1M | 1M |
| Tool Calling | Yes | Tool calling supported + built in browser, code execution and web search |

### Scaling

| Feature | Qwythos 27B V1 | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/empero-ai/Qwythos-27B-v1)

**Developed by [Empero](https://empero.org)**

**Qwythos-27B-v1** is an open-weight, full-parameter reasoning model: the larger sibling of [Qwythos-9B](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M), trained on the exact same curriculum and sized up on a Qwen3.5-27B base. It is a complete pre-RL checkpoint, post-trained through **SFT → DPO → ESFT**.

The point of this release is simple: **nothing was ablated to make it fit.** Qwythos retains the base checkpoint's native multi-token-prediction head, full vision tower, and 1,048,576-token context configuration—all intact and active.

-   **🧩 Nothing ablated** — native MTP, the full vision tower, and a 1,048,576-token YaRN context window are present together. Most community 27B fine-tunes give up at least one of them.
-   **🧠 The bigger sibling** — the same Qwythos-9B curriculum at 27B. In a closed-book organophosphate-poisoning evaluation, the 27B volunteered that physostigmine should be avoided, where the 9B needed a web search to get that fact right.
-   **🛠 Built for terminal trajectories** — held-out terminal/tool-session perplexity falls from **356.6** in the base to **2.76** after training, and the model transfers that format to an emulated Claude-Code session.
-   **🔧 Native function calling** — the packaged chat template carries Qwen3.5's full tool-use spec, so `tools=[...]` works out of the box with no wrapper and no tool-specific fine-tune.

> Qwythos is intentionally uncensored for technical and research use. It engages substantively with sensitive-but-legal cybersecurity, biomedical, and technical questions; use appropriate review and application-level controls for your deployment.

## Why this checkpoint

Qwythos-27B-v1 preserves the parts of the base that are often dropped during a community fine-tune:

-   **Native MTP is preserved.** The one-layer native MTP head and its 15 tensors were preserved verbatim; SFT did not touch them, and they were re-attached through the safetensors index. Runtimes that support it can use the head for self-speculative decoding.
-   **Vision is still here.** The Qwen3.5 vision tower is intact: 333 vision tensors, depth 27, hidden size 1152, and output hidden size 5120. Training was text-only and the vision parameters were frozen, so image behavior is inherited from the base rather than newly tuned.
-   **The full context configuration remains enabled.** The 262,144-token native context is extended with YaRN factor 4.0 to **1,048,576 tokens**. The configuration was verified numerically finite through position 1,048,575.

The larger model changes what can stay closed-book. On an organophosphate-poisoning prompt, Qwythos-27B correctly volunteered “Avoid physostigmine (it can worsen toxicity)” and gave correct atropine/pralidoxime dosing without tools. That specific fact required a web search in the 9B evaluation. It is a useful, concrete illustration of what the size-up buys—not a substitute for verification in clinical use.

## Agentic and terminal transfer

The strongest measured shift is target-format acquisition on held-out terminal/tool sessions: **356.6 → 2.76 perplexity** versus the base, evaluated on assistant tokens with identical inputs and sequences capped at 24,576 tokens. In a held-out emulated Claude-Code session for an authorized lab network assessment, Qwythos consumed injected `nmap` output, reasoned about scan trade-offs, and emitted the correct next shell-form tool call.

It also emits first-person `<think>...</think>` reasoning rather than the base model's tendency toward meta-planning outlines. Greedy long-form generations in the reviewed set were clean under the loop check, with no degenerate repetition.

## Native function calling

The packaged chat template implements Qwen3.5's tool-use spec in full. Pass `tools=[...]` to `apply_chat_template` and the model emits standard tool-call blocks — no wrapper, no tool-specific fine-tune:

`<tool_call> <function=NAME> <parameter=PARAM>value</parameter> </function> </tool_call>`

Combined with the terminal/agentic training above, this is the intended deployment path: give Qwythos real tools and let it verify its specifics rather than recalling them closed-book.

## Identity

The packaged chat template introduces the model as _"You are Qwythos, a model created by Empero AI. Only bring up your identity if the user asks."_ — it identifies itself only when asked.

## v1, before RL

This is **v1**, the pre-RL checkpoint. It has received full-parameter SFT, DPO, and ESFT, and is released as a usable model in its own right. An RL-trained v2 will follow soon. DPO and ESFT hyperparameters are not being published.

## Model details

| Property | Value |
| --- | --- |
| Base | Qwen/Qwen3.5-27B |
| Architecture | Dense Qwen3.5-27B; hybrid Gated-DeltaNet linear attention with a full-attention layer every fourth layer (3:1) |
| Text backbone | 64 hidden layers; hidden size 5120; 24 attention heads; 4 KV heads; vocabulary 248,320 |
| Modalities | Text + vision, with native MTP |
| Precision | bf16 |
| Context | 1,048,576 tokens: 262,144 native × YaRN factor 4.0 |
| Native MTP | One hidden layer (mtp\_num\_hidden\_layers: 1) |

## Training

Qwythos-27B was post-trained on the same curriculum family as Qwythos-9B — STEM and reasoning first, then the agentic target domain — scaled up to 27B.

The pipeline is **full-parameter SFT → DPO → ESFT**, run in bf16 with assistant-only loss on long, untruncated sequences. Qwythos-27B-v1 is the checkpoint at the end of that pipeline, before reinforcement learning.

Training data, per-stage hyperparameters, and the packing and kernel setup are not being published.

## How to use

The base is multimodal. For text-only inference, load it with `AutoModelForImageTextToText`:

```
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "empero-ai/Qwythos-27B-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype="bfloat16",
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Explain the trade-offs in this terminal plan and propose the next command.",
    }
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
    repetition_penalty=1.05,
    max_new_tokens=16384,
)
new_tokens = output[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))
```

Use a recent `transformers` with Qwen3.5 support and Gated-DeltaNet kernels (`flash-linear-attention` and a CUDA-matched `causal_conv1d`). Without them, the linear-attention layers fall back to slower PyTorch operations.

### With tools (function calling)

```
TOOLS = [
    {"type": "function", "function": {
        "name": "run_shell",
        "description": "Run a shell command and return combined stdout/stderr.",
        "parameters": {"type": "object",
                       "properties": {"command": {"type": "string"}},
                       "required": ["command"]}}},
    {"type": "function", "function": {
        "name": "web_search",
        "description": "Search the web for current facts and citations.",
        "parameters": {"type": "object",
                       "properties": {"query": {"type": "string"},
                                      "max_results": {"type": "integer"}},
                       "required": ["query"]}}},
]

prompt = tokenizer.apply_chat_template(
    messages, tools=TOOLS, tokenize=False, add_generation_prompt=True
)
```

### Serving at 1M context with vLLM

```
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve empero-ai/Qwythos-27B-v1 \
  --max-model-len 1010000
```

The static YaRN factor of 4.0 has a short-context quality trade-off. For roughly 512k context, use factor 2.0; if you will never exceed the 262,144-token native window, restore `rope_parameters.rope_type` to `"default"` from the included `config.json.pre_yarn` backup for maximum short-context fidelity.

### GGUF: llama.cpp, Ollama, and LM Studio

For GGUF releases, including llama.cpp, Ollama, and LM Studio workflows, see **[empero-ai/Qwythos-27B-v1-GGUF](https://huggingface.co/empero-ai/Qwythos-27B-v1-GGUF)**. The recommended default is `Qwythos-27B-Q4_K_M.gguf`; the MTP-enabled default is `Qwythos-27B-MTP-Q4_K_M.gguf`. Image use also requires `mmproj-Qwythos-27B-F16.gguf` alongside the selected text quant.

## Sampling recommendations

Qwythos-27B is a reasoning model and inherits Qwen3.5's thinking-mode behaviour. These are the shipped defaults in `generation_config.json` and the recommended starting point:

```
gen_kwargs = dict(
    do_sample=True,
    temperature=0.6,        # Qwen3.5 thinking-mode recommended
    top_p=0.95,
    top_k=20,
    repetition_penalty=1.05,
    max_new_tokens=16384,   # generous budget for the <think> block + final answer
)
```

| Parameter | Value |
| --- | --- |
| temperature | 0.6 (agentic / tool use) · 1.0 (open-ended or creative) |
| top\_p | 0.95 |
| top\_k | 20 |
| repetition\_penalty | 1.05 |
| max\_new\_tokens | 16,384+ |

-   **Agentic, harness, or tool use → temperature 0.6.** Tighter sampling keeps tool-call syntax and multi-step plans on the rails.
-   **Open-ended reasoning or creative work → temperature 1.0.** Loosen it up when you want the model to explore rather than commit early.
-   **Give it room.** Reasoning traces are long; budget **16,384+ `max_new_tokens`** so the `<think>` block and the final answer both fit.

Use `repetition_penalty=1.05` — a small deviation from Qwen's default of 1.0 that prevents rare non-terminating reasoning loops on long generations.

## Limitations

-   **Vision is inherited rather than tuned.** The vision tower is intact, but text-only training did not tune or evaluate image understanding.
-   **Long context has a trade-off.** Static YaRN at factor 4.0 slightly degrades short-context quality; reduce or remove the factor when the full range is unnecessary.
-   **Uncensored is a deployment responsibility.** The model engages with sensitive-but-legal technical content; provide your own policy, review, and access controls where appropriate.

## Stay in the loop

Sign up for the Empero newsletter at [empero.org](https://empero.org) for releases, evaluations, and research notes on Qwythos and future open-weight models.

## Support / Donate

If this model helped you, consider supporting the project:

-   **BTC:** `bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v`
-   **LTC:** `ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x`
-   **XMR:** `42Dbm5xg5Nq26fdyzfEU7KBnAJfhi7Cvz5J2ex5CzHXkfKuNEJzYCcmJ1GTbgjFZ5MBx72sdG1G9239Cd6rsZfv4QeDkYJY`

## Provenance & licensing

Qwythos-27B-v1 is derived from **Qwen3.5-27B**. The weights are released under **Apache-2.0**, inherited from the Qwen3.5-27B base, for research and experimentation as-is.

## Acknowledgements

-   Developed and released by [Empero](https://empero.org)
-   Base model: [Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) (Alibaba Qwen team)
-   Training: [TRL](https://github.com/huggingface/trl) + [Transformers](https://github.com/huggingface/transformers)
-   GGUF quantization: [llama.cpp](https://github.com/ggml-org/llama.cpp) (ggml-org)

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)