# Gemma 4 12B Coder Fable5 Composer2.5 V1 GGUF

URL: https://interfaze.ai/models/yuxinlu1gemma-4-12b-coder-fable5-composer25-v1-gguf

Gemma 4 12B Coder Fable5 Composer2.5 V1 GGUF by yuxinlu1, a text-generation model with multimodal capabilities. Understand and compare multimodal features, benchmarks, and capabilities.

## Comparison

| Feature | Gemma 4 12B Coder Fable5 Composer2.5 V1 GGUF | Interfaze |
| --- | --- | --- |
| Input Modalities | text, image, audio, video | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | 140 partial | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 131K | 1M |
| Tool Calling | Yes | Tool calling supported + built in browser, code execution and web search |

### Scaling

| Feature | Gemma 4 12B Coder Fable5 Composer2.5 V1 GGUF | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF)

### 🐣 Tiny footprint, big brain — a local **coding** model for _everyone_

> **No matter your GPU. No matter your RAM.** If you've got **~4.5 GB** of VRAM _or_ unified memory free, you can run your own private, offline coding assistant right now. 🚀 This is the **v1 / code edition** — distilled from **real chain-of-thought** so it _thinks through_ a problem before writing the solution. 🧠💻 All local, all yours, no API, no cloud.

### 🎯 What it is

A focused fine-tune of Gemma 4 12B on **verifiable Python coding** data — every training example's reasoning leads to code that **actually passed its tests**. The result reasons in the open (edge cases, complexity, approach) and then emits a clean, runnable solution. 💚

* * *

## 📚 Training data (the interesting part 🍳)

This is a **distillation** of two complementary chain-of-thought sources, both over verifiable Python coding tasks (algorithmic / function-level problems that come with deterministic tests):

-   **🥇 Main set — Composer 2.5 _real_ CoT.** Genuine, model-authored reasoning traces. The teacher solved each problem, its code was **run against the task's tests, and only the passing solutions were kept**. So the reasoning you're learning from leads to code that _actually works_.
-   **🥈 Aux set — Fable 5 (released today! 🎉).** A clever twist: we took the problems where **Composer 2.5 got it wrong** and handed them to **Fable 5** to _redo_ — re-deriving a fresh, self-consistent chain-of-thought and a correct solution, again **gated on passing the tests**. This recovers the hard cases the main teacher missed. These traces are **synthetic** (rationalized CoT), and are tagged separately so the two sources stay distinguishable.

The recipe: real CoT for the bulk of solid coverage, plus synthetic "second-attempt" CoT to patch the failures — both verified by execution before anything entered training. ✅

* * *

## 🗺️ Roadmap — v2 (if there's interest! 💚)

This is **v1**. If the likes / downloads add up, I'll ship a **v2** that **pushes for the benchmarks** 🏁.

### 📢 Update on v2 & the Fable 5 situation _(2026-06-14)_

Quick heads-up for everyone waiting on v2:

**Fable 5 access has been pulled.** The Fable 5 CoT data I managed to save beforehand is honestly a pretty small set — not enough on its own to act as the _primary_ signal for v2 without risking **overfitting**. So the plan is shifting:

-   **v2 will lean more heavily on Composer 2.5 verifiable CoT** as the backbone (the main, execution-verified source), and use the limited Fable 5 data carefully as a supplement rather than the core.
-   **If Fable 5 access doesn't come back within ~a week**, I'm considering bringing in **GLM-5.2** as an additional teacher. I just went through the benchmarks: per **BridgeMind's eval posted on X**, GLM-5.2 actually edges out Fable 5 on both the **BS** and **reasoning** leaderboards. I haven't tested it hands-on myself yet — my gut says it'll land _slightly below_ Fable 5 in practice, but likely **very close**.

Bottom line: v2 is still coming. I'd just rather take a little longer and ship something that **generalizes** than rush out an overfit model. Thanks for the patience and support 💚

⭐ **Like & download if you'd like to see v2** — that's the signal I'm watching!

* * *

## 📦 Pick your size (GGUF quants)

| Quant | Size | Vibe |
| --- | --- | --- |
| 🟢 Q2\_K | 4.5 GB | tiniest — runs almost anywhere |
| 🔵 Q4\_K\_M | 6.87 GB | the sweet spot 👌 (recommended) |
| 🟣 Q6\_K | 9.11 GB | near-lossless |
| ⚪ Q8\_0 | 11.8 GB | basically full quality |

* * *

## 🧮 "Will it fit?" — context length cheat-sheet

Rough estimates 🤓 (assumes `q8_0` KV cache + ~1.5 GB overhead; **use `q4_0` KV cache for ≈2× more context!**). Max context is **131K**. "—" = won't fit, pick a smaller quant. ✂️

| Your VRAM / unified mem | 🟢 Q2\_K (4.5G) | 🔵 Q4\_K\_M (6.87G) | 🟣 Q6\_K (9.11G) | ⚪ Q8\_0 (11.8G) |
| --- | --- | --- | --- | --- |
| 8 GB | ~16K ctx | tight (~2–4K) | — | — |
| 12 GB | ~48K | ~30K | ~12K | — |
| 16 GB | ~80K | ~64K | ~44K | ~22K |
| 24 GB | 131K (max) 🎉 | ~128K | ~110K | ~88K |
| 32 GB | 131K | 131K | 131K | 131K |

> 💡 Apple Silicon / integrated GPUs with **unified memory** count too — same numbers, just slower than a dGPU. 💡 Low on room? Drop a quant or switch KV cache to `q4_0` and your context roughly doubles.

* * *

## 🚀 How to run it (super easy)

### Option A — llama.cpp (recommended) 🦙

1.  Grab a quant above (e.g. `…-Q4_K_M.gguf`) and `llama-server` from [llama.cpp](https://github.com/ggml-org/llama.cpp).
    
    > ⚠️ Needs a **recent llama.cpp** (this is the `gemma4_unified` architecture — older builds won't load it).
    
2.  Run a server (Windows `.bat` shown — tweak `--port`, `--ctx-size` to taste):

```
@echo off
cd /d C:\llama.cpp
llama-server.exe ^
  -m C:\models\gemma4-coding-Q4_K_M.gguf ^
  --ctx-size 16384 ^
  --n-gpu-layers 99 ^
  --no-mmap ^
  -fa on ^
  --cache-type-k q8_0 --cache-type-v q8_0 ^
  --temp 1.0 --top-p 0.95 --top-k 64 ^
  --host 0.0.0.0 --port 18080
pause
```

3.  Open `http://localhost:18080` and chat. 🎉 (Tip: bump `--ctx-size` per the table; use `q4_0` KV for more.)

### Option B — one-click apps 🖱️

Works in **LM Studio**, **Jan**, **Ollama**, etc. — just import the GGUF, pick your quant, go. 🐾

### 🧠 Thinking mode

This model thinks in Gemma's native thought channel before answering — exactly how it was trained. Keep **`enable_thinking=true`** (the default chat template handles it). Recommended sampling: `temp 1.0, top_p 0.95, top_k 64`. For coding you can also go greedy (`temp 0`) for more deterministic solutions.

* * *

## ⚠️ Good to know

-   **Reduced refusals:** the training data is task-focused with no safety hedging, so this refuses less than the base model. It is **not** safety-aligned — add your own guardrails for production. Use responsibly. 🙏
-   Specialized for **Python / algorithmic** coding. Reasoning quality is strongest in that domain; general-knowledge facts/numbers should still be double-checked.
-   English-centric.

* * *

## 📚 Base & License

-   **Base model:** [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it). Subject to the **[Gemma Terms of Use](https://ai.google.dev/gemma/terms)** (derivatives must comply).
-   Personal/hobby project — shared as-is, no warranty. Have fun, and happy hacking! 🐾✨

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)