# Qwen3.8 27B Heretic Abliterated Uncensored GGUF

URL: https://interfaze.ai/models/0bserverxqwen38-27b-heretic-abliterated-uncensored-gguf

[All models](https://interfaze.ai/models)

Qwen3.8 27B Heretic Abliterated Uncensored GGUF by 0bserverx, a text-generation model with multimodal capabilities. Understand and compare multimodal features, benchmarks, and capabilities.

## Comparison

| Feature | Qwen3.8 27B Heretic Abliterated Uncensored GGUF | Interfaze |
| --- | --- | --- |
| Input Modalities | text, image, video | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | unknown | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | Yes | Yes |
| Context Input Size | 262.1K | 1M |
| Tool Calling | Yes | Tool calling supported + built in browser, code execution and web search |

### Scaling

| Feature | Qwen3.8 27B Heretic Abliterated Uncensored GGUF | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/0bserverx/Qwen3.8-27B-Heretic-Abliterated-Uncensored-GGUF)

**RVN** is a double-refined abliterated variant of **Qwen3.8-27B**, built on top of [`trohrbaugh/Qwen3.8-27B-heretic-ara`](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara) (an ARA abliteration by **Tim Rohrbaugh**) and further refined with **two additional full-weight ARA passes** targeting residual refusals. It retains very low behavioral damage (KL ≈ 0.0085) while reducing harmful-prompt refusals from 3/100 (source) to **0–1/100** in independent measurements.

> **Note on this repository's history.** This repo previously hosted the original `Qwen3.8-27B-Heretic-Q4_K_M.gguf` (single-quant release from the earlier `trohrbaugh/Qwen3.8-27B-heretic` source). That file is kept as **legacy** for download-count continuity and backward compatibility — it is the _older_ abliteration variant and is superseded by the RVN files below. Prefer the RVN quants for new deployments.

> **Not for all audiences.** This model has reduced safety guardrails by design. It is intended for adult audiences (18+) doing research, creative writing, roleplay, and uncensored generation. Certain guardrails are intentionally left in place; use responsibly and in accordance with your local laws.

> **Compatibility notice (2026-08-19):** the currently published RVN GGUFs are missing the embedded `tokenizer.chat_template` metadata. Plain chat may appear to work through llama.cpp's generic ChatML fallback, but Qwen-native tool/function calling and `chat_template_kwargs.enable_thinking` control are not reliable through that fallback. Until the same-name GGUF replacements are uploaded, download the official [`chat_template.jinja`](https://huggingface.co/0bserverx/Qwen3.8-27B-Heretic-Abliterated-Uncensored-GGUF/blob/main/chat_template.jinja) from this repository and launch llama.cpp with `--chat-template-file chat_template.jinja`. The file is copied byte-for-byte from `Qwen/Qwen3.8-27B` (8,952 bytes; SHA-256 `c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041`).

```
llama-server -m RVN-Q5_K_M.gguf --chat-template-file chat_template.jinja \
  -c 32768 -ngl 99
```

* * *

## What is ARA?

**ARA (Arbitrary-Rank Ablation)** is the abliteration technique implemented in [p-e-w/heretic](https://github.com/p-e-w/heretic). Traditional directional abliteration finds a single "refusal direction" in activation space and subtracts it — a one-shot, low-rank surgery that is simple but can leave residual refusals or damage unrelated behavior.

ARA instead treats abliteration as a **matrix optimization problem**. For every target module (attention out-projection and MLP down-projection), it collects activations on "good" prompts (harmless requests) and "bad" prompts (harmful requests), then uses an LBFGS optimizer to rewrite the module's weight matrix so that:

-   **Preserve:** outputs on good prompts change as little as possible (KL is kept low)
-   **Steer:** outputs on bad prompts are pulled toward the good-prompt output manifold (via k-nearest-neighbor distances), so harmful requests stop triggering the refusal circuitry
-   **Overcorrect:** outputs on bad prompts are additionally pushed _away_ from the original bad-prompt outputs, which helps overcome complex, multi-stage refusal mechanisms

Because the weight matrix is optimized directly (rather than subtracting a single direction), ARA is "arbitrary rank" — it can carve out a much richer refusal-removal subspace while keeping behavioral damage minimal.

## Why "Heretic" and "Abliterated"?

These two words describe two layers of the same process:

-   **Heretic** is the _tool_: the open-source implementation of ARA (and related abliteration methods) used to modify the model. Models produced with it are commonly labeled "heretic" in the community.
-   **Abliterated** is the _result_: the model's refusal behavior has been surgically removed. An abliterated model still knows everything the base model knows, but it no longer refuses to answer the categories that were steered away during the process.

So "Heretic Abliterated" means: _abliterated using the heretic toolset._ RVN goes one step further — it applies the ARA procedure **three times total**: once by the original author (`trohrbaugh`) to get from base Qwen3.8-27B to `-ara`, and twice more by us to get from `-ara` to RVN, squeezing out the last residual refusals.

## Special Thanks

This work would not exist without **Tim Rohrbaugh** (`trohrbaugh`), whose [`heretic-ara`](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara) ARA abliteration of Qwen3.8-27B (refusals 3/100, KL 0.0535) provided the foundation we refined into RVN. His upstream contributions to the heretic codebase — including the row-norm preservation feature and Qwen3.5 MoE/DeltaNet hybrid handling — are directly responsible for making DeltaNet-layer abliteration work at all. Thank you, Tim.

## Model Overview

| Property | Value |
| --- | --- |
| Base model | Qwen/Qwen3.8-27B |
| Abliteration source | trohrbaugh/Qwen3.8-27B-heretic-ara (ARA, KL 0.0535, refusals 3/100) |
| RVN refinement | 2-pass ARA on top of source → KL 0.0085, refusals 0–1/100 |
| Architecture | qwen3\_5\_text (Qwen3.8 family), Gated DeltaNet hybrid |
| Parameters | 27B total |
| Hidden size | 5120 |
| Layers | 64 (16 standard attention + 48 Gated DeltaNet linear attention) |
| Attention heads | 24 · KV heads 4 (GQA) · head\_dim 256 |
| Vocab | 248,320 |
| Context length | 262,144 (262K) |
| License | Apache-2.0 (retained from Qwen3.8-27B) |
| Format | GGUF (llama.cpp). Base files: MTP/NextN excluded. \*-mtp.gguf files ship the official Qwen3.8 MTP draft head embedded → see MTP Speculative Decoding |

## Why RVN?

`trohrbaugh/Qwen3.8-27B-heretic-ara` is already a strong ARA abliteration, but three harmful prompts still triggered refusals in our independent evaluation (racism website, malware, government database hacking). RVN applies **two additional full-weight ARA passes** using the same tight parameter set (start 26, end 56, preserve 0.9432, steer 0.0009, overcorrect 0.5038, neighbor 10), which:

-   Reduced refusals from **3/100 → 0–1/100** (the only remaining refusal is a chemical-weapon WMD prompt — one of the strongest safety-trained categories, and intentionally one of the guardrails we left in place)
-   Reduced KL damage from 0.0535 (source) to **0.0085** vs base — a ~6× improvement in behavioral preservation
-   Verified independently on two rented GPU machines with prefix-based (real-answer) refusal measurement

### Refusal evaluation (100 harmful-behaviors prompts, prefix-forced real answers)

| Model | Refusals | KL vs base |
| --- | --- | --- |
| Qwen3.8-27B (base) | ~99/100 | — |
| trohrbaugh -ara (source) | 3/100 | 0.0535 |
| RVN (this repo) | 0–1/100 | 0.0085 |

## Files & Quantization Spectrum

| File | Size (GB / GiB) | Notes |
| --- | --- | --- |
| RVN-F16.gguf | 53.81 / 50.11 | F16 reference (no NextN/MTP) |
| RVN-BF16.gguf | 53.81 / 50.11 | BF16 reference (no NextN/MTP) |
| RVN-Q8\_0.gguf | 28.60 / 26.63 | Max-quality 8-bit |
| RVN-Q6\_K.gguf | 22.08 / 20.57 | High-quality 6-bit |
| RVN-Q5\_K\_M.gguf | 19.23 / 17.91 | Balanced 5-bit |
| RVN-Q5\_K\_S.gguf | 18.68 / 17.40 | 5-bit small |
| RVN-Q4\_K\_M.gguf | 16.55 / 15.41 | Recommended 4-bit (24 GB VRAM) |
| Qwen3.8-27B-Heretic-Q4\_K\_M.gguf | 16.55 / 15.41 | Legacy (older abliteration variant, kept for download continuity) |
| RVN-IQ4\_NL.gguf | 15.89 / 14.80 | 4-bit non-linear — current file was produced without an imatrix; corrected replacement in progress |
| RVN-Q4\_K\_S.gguf | 15.59 / 14.52 | Small 4-bit |
| RVN-IQ4\_XS.gguf | 15.19 / 14.15 | 4-bit extra-small — current file was produced without an imatrix; corrected replacement in progress |
| RVN-Q3\_K\_L.gguf | 14.34 / 13.36 | Large 3-bit |
| RVN-Q3\_K\_M.gguf | 13.30 / 12.39 | Compact 3-bit |
| RVN-IQ3\_M.gguf | 12.58 / 11.72 | imatrix 3-bit — re-uploaded 2026-08-17 (previous file had corrupted tensor data: NaN/Inf scales + zeroed tensors from a bad quantize run; re-quantized from F16 with a fresh imatrix and verified — see note below) |
| RVN-IQ3\_S.gguf | 12.42 / 11.57 | imatrix 3-bit small |
| RVN-Q3\_K\_S.gguf | 12.07 / 11.24 | Compact 3-bit small |
| RVN-IQ3\_XS.gguf | 11.97 / 11.15 | imatrix 3-bit extra-small |
| RVN-IQ3\_XXS.gguf | 11.19 / 10.42 | imatrix 3-bit extra-extra-small |
| RVN-Q2\_K.gguf | 10.71 / 9.98 | 2-bit K-quant |
| RVN-Q2\_K\_S.gguf | 10.25 / 9.54 | 2-bit K-quant small |
| RVN-IQ2\_M.gguf | 10.00 / 9.32 | imatrix 2-bit |
| RVN-IQ2\_S.gguf | 9.36 / 8.72 | imatrix 2-bit small |
| RVN-IQ2\_XS.gguf | 9.09 / 8.47 | imatrix 2-bit extreme small |
| RVN-IQ2\_XXS.gguf | 8.43 / 7.85 | imatrix 2-bit (minimum) |
| RVN-IQ1\_M.gguf | 7.63 / 7.11 | imatrix 1-bit (experimental) |
| RVN-IQ1\_S.gguf | 7.15 / 6.66 | imatrix 1-bit (experimental) |

### 👁️ Vision-Protected (UD-style) Variants

The standard quants above use uniform quantization across all layers. For vision-heavy use (image understanding via the mmproj), the layers that first receive visual embeddings matter disproportionately — the "vision-language bridge". These `-vision` variants use Unsloth-style dynamic (UD) quantization: `token_embd`, `output`, and the **first 4 + last 4 transformer blocks** stay at **Q8\_0**, while the middle blocks are compressed to the target K-quant. Built with `llama-quantize --token-embedding-type q8_0 --output-tensor-type q8_0 --tensor-type-file` (106 bridge tensors overridden by regex; verified via tensor audit — 0 NaN/Inf, correct per-tensor types, 851 tensors).

| File | Size (GB / GiB) | Bridge / Middle |
| --- | --- | --- |
| RVN-Q5\_K\_M-vision.gguf | 21.02 / 19.57 | Q8\_0 bridge · Q5\_K\_M middle |
| RVN-Q4\_K\_M-vision.gguf | 18.75 / 17.46 | Q8\_0 bridge · Q4\_K\_M middle |
| RVN-Q3\_K\_M-vision.gguf | 16.19 / 15.08 | Q8\_0 bridge · Q3\_K\_M middle |

Usage — identical to the standard quants, just point at the `-vision` file:

```
llama-server -m RVN-Q5_K_M-vision.gguf --mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
  -c 32768 -ngl 99
```

### 📊 Perplexity (vs F16 reference)

Measured on an RTX PRO 6000 Blackwell (full GPU offload, llama.cpp master): `llama-perplexity`, tiny\_shakespeare corpus, ctx 2048.

| Model | PPL | Δ vs F16 |
| --- | --- | --- |
| RVN-F16.gguf | 4.5477 | — |
| RVN-Q5\_K\_M.gguf (standard) | 4.6493 | +2.23% |
| RVN-Q5\_K\_M-vision.gguf | 4.6497 | +2.24% |
| RVN-Q4\_K\_M-vision.gguf | 4.8751 | +7.20% |
| RVN-Q3\_K\_M-vision.gguf | 5.6490 | +24.2% |

Text perplexity is at parity with the standard quant (no regression from the Q8\_0 bridge), while the vision-critical bridge layers keep 8-bit precision. All `-vision` variants and the standard `Q5_K_M` passed image-description tests with the mmproj (accurate description, no hallucination on the test image).

> imatrix-based quants are produced from the same F16 with an activation importance matrix computed over wikitext-2-raw (original spectrum, **580 chunks**) or tiny\_shakespeare (2026-08-17 re-quant additions: `IQ3_M` fix + `IQ2_S`/`IQ3_XXS`/`IQ3_XS`/`IQ3_S`, **159 chunks**, llama-imatrix, `-ngl 99`). The currently published `IQ4_XS` and `IQ4_NL` files are exceptions: an independent GGUF-header audit and the retained production script confirm they were made without `--imatrix`. Their earlier imatrix labels were incorrect; corrected wikitext-imatrix replacements (including MTP twins) are being rebuilt. The `-vision` files are quantized with K-quant defaults (no imatrix) — the bridge protection is structural (Q8\_0 overrides), not imatrix-dependent.

## ⚡ MTP Speculative Decoding

Every quant ships a **`*-mtp.gguf` twin** with the official Qwen3.8 MTP draft head (q8\_0, ~1.8 GB) embedded. The main-model weights are byte-identical to the base file — the head is appended as an extra `blk.64.nextn.*` layer set (block\_count 65, `qwen35.nextn_predict_layers=1`). Abliteration never touched the draft head (it operates on main-model layers 26–56), so draft quality is unchanged, and speculative decoding is **output-equivalent**: same tokens, just faster.

**Usage (llama.cpp ≥ b10440, PR #22673):**

```
llama-server -m RVN-IQ3_M-mtp.gguf -c 32768 -ngl 99 \
  --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1
```

**Measured on 2× RTX PRO 6000 Blackwell (95 GB each, full GPU offload, llama.cpp b10472):**

| Quant | Normal (t/s) | \+ MTP (t/s) | Δ |
| --- | --- | --- | --- |
| Q6\_K | 61.6 | 126.2 | +105% |
| BF16 | 29.2 | 58.2 | +99% |
| Q8\_0 | 50.6 | 98.0 | +94% |
| IQ3\_S | 91.9 | 169.7 | +85% |
| IQ4\_XS | 83.4 | 152.0 | +82% |
| Q3\_K\_S | 84.5 | 153.6 | +82% |
| F16 | 29.4 | 52.9 | +80% |
| IQ3\_XS | 94.0 | 161.9 | +72% |
| Q3\_K\_L | 78.7 | 131.3 | +67% |
| IQ4\_NL | 80.8 | 138.0 | +71% |
| IQ2\_M | 106.2 | 175.9 | +66% |
| IQ2\_XS | 112.8 | 183.4 | +63% |
| Q4\_K\_M | 76.6 | 122.0 | +59% |
| Q4\_K\_S | 80.5 | 127.1 | +58% |
| IQ3\_M | 91.3 | 144.0 | +58% |
| Q3\_K\_M | 82.6 | 129.5 | +57% |
| IQ2\_XXS | 117.6 | 182.6 | +55% |
| Q2\_K | 98.2 | 150.2 | +53% |
| IQ3\_XXS | 98.5 | 138.5 | +41% |
| IQ2\_S | 111.5 | 155.3 | +39% |
| Q5\_K\_M | 68.1 | 93.8 | +38% |
| Q2\_K\_S | 105.5 | 145.1 | +38% |
| Q5\_K\_S | 70.7 | 91.1 | +29% |
| IQ1\_S | 127.0 | 131.7 | +3.7% |
| IQ1\_M | 119.5 | 47.6 | −60% ⚠️ |

Average: **+55% generation speed** (128-token continuation, `--spec-draft-n-max 2 --parallel 1`). **⚠️ `IQ1_M` is the exception: MTP makes it ~60% slower — use the base file for IQ1\_M.** `IQ1_S` gains almost nothing (+4%). All other quants gain +29% to +105%.

Community measurements on larger cards report **+33–145%** depending on GPU and context. Tips: `--spec-draft-n-max 2` is the sweet spot on 16–24 GB cards (3–4 on bigger/faster cards); pair with `--cache-type-k q4_0 --cache-type-v q4_0` for long context; `--spec-draft-p-min 0.60–0.75` helps on bandwidth-limited rigs.

## 👁️ Vision (image understanding)

Pair any `*-mtp.gguf` with the official vision projector for full image-text-to-text:

```
llama-server -m RVN-IQ3_M-mtp.gguf --mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
  -c 32768 -ngl 99 --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1
```

-   `mmproj-Qwen3.8-27B-Q8_0.gguf` (0.63 GB, Q8\_0) is the official Qwen3.8 vision projector from `ggml-org/Qwen3.8-27B-GGUF` (Apache-2.0). The vision tower is an image encoder only — ARA abliteration never touched it, so it pairs cleanly with every RVN quant.
-   Verified: `RVN-Q3_K_M-mtp` + this mmproj describes images correctly (MTP active).
-   Combo credit: [cfigueiroa/Qwen3.8-27B-RVN-vision-MTP](https://huggingface.co/cfigueiroa/Qwen3.8-27B-RVN-vision-MTP).

## Quant → GPU / Memory Guide

| GPU / Memory | Best quant(s) (full GPU load) | Effective ctx @ Q8\_0 KV |
| --- | --- | --- |
| 8 GB (RTX 3050, 4060 Laptop) | IQ1\_S, IQ1\_M; IQ2\_XXS partial offload only | ~2–4K |
| 12 GB (RTX 3060, 4070) | IQ2\_M, IQ2\_S, IQ2\_XS, Q2\_K\_S; IQ3\_XXS (tight) | ~8–16K |
| 16 GB (RTX 4080, 4090 Laptop, M3 Max) | IQ3\_M, IQ3\_S, Q3\_K\_M; IQ4\_XS/Q4\_K\_S/IQ4\_NL (tight ctx) | ~6–24K |
| 24 GB (RTX 3090, 4090, M4 Max) | Q5\_K\_M, Q5\_K\_S, Q6\_K, Q4\_K\_M; Q8\_0 partial | ~16–48K |
| 32 GB (RTX 5090, A6000) | Q8\_0, Q6\_K | ~24–64K |
| 64 GB+ (A100 80 GB, RTX PRO 6000, M3/M4 Ultra) | F16, BF16 | ~64–100K+ |

> Sizes in the file table are the **actual file sizes on the Hub** (decimal GB / GiB), pulled from repository metadata. Full GPU load means the whole quant fits in VRAM; quants whose file size exceeds your VRAM need partial offloading.
> 
> **2026-08-17 — RVN-IQ3\_M incident & fix:** the original `RVN-IQ3_M.gguf` generated only `/` characters on every backend (confirmed by the community and reproduced locally). A tensor-level audit showed corrupted quantization data — NaN/Inf block scales and fully zeroed tensors (e.g. `token_embd` had ~39.6M NaN values) — from a bad quantize run, not a llama.cpp regression (all other quants from the same F16 dequantize cleanly). The file was pulled, re-quantized from the F16 with a freshly computed imatrix, generation-tested (`"The capital of France is" → Paris`, 70+ t/s) and re-uploaded. New quants added the same day: `IQ2_S`, `IQ3_XXS`, `IQ3_XS`, `IQ3_S`, `Q3_K_L`, `Q5_K_S`.

**KV cache math (GQA, 4 KV heads, head\_dim 256):** `2 × 64 layers × 4 KV heads × 256 head_dim × 2 bytes = 256 KiB/token FP16` → 16K ctx ≈ 4.2 GB · 32K ctx ≈ 8.4 GB · 64K ctx ≈ 16.8 GB (Q8\_0 KV halves this). A 16 GB card running `Q3_K_M` (13.30 GB model) + 16K ctx Q8\_0 KV fits comfortably; `Q4_K_M` (16.55 GB) really needs a 24 GB card.

**Rule of thumb:** pick the largest quant that leaves ≥ 4 GB for KV cache + compute buffers. If you only need short replies, drop the quant one notch and get a bigger context; if you need long context, prioritize KV budget over quant size.

## Limitations & Responsible Use

-   **Reduced safety guardrails by design.** This model is not intended for use in applications requiring robust safety filtering, content moderation, or deployment to minors.
-   **Certain guardrails are intentionally left in place.** Abliteration targets refusal behavior on general harmful-prompt categories; a small set of hard safety-trained categories is deliberately not fully removed. Behavior may vary across domains and languages.
-   Not affiliated with or endorsed by Qwen/Alibaba or trohrbaugh.

## License & Attribution

-   Base model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) — Apache-2.0
-   Abliteration source: [`trohrbaugh/Qwen3.8-27B-heretic-ara`](https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara) — Apache-2.0
-   This repo: Apache-2.0. See [LICENSE](https://huggingface.co/0bserverx/Qwen3.8-27B-Heretic-Abliterated-Uncensored-GGUF/blob/main/LICENSE).

## Citation

```
@misc{rohrbaugh2026heretic,
  title={Qwen3.8-27B-heretic-ara: ARA Abliteration of Qwen3.8-27B},
  author={Rohrbaugh, Tim},
  year={2026},
  howpublished={\url{https://huggingface.co/trohrbaugh/Qwen3.8-27B-heretic-ara}}
}

@misc{rvn2026,
  title={RVN: Qwen3.8-27B Heretic Abliterated Uncensored},
  author={0bserverx},
  year={2026},
  howpublished={\url{https://huggingface.co/0bserverx/Qwen3.8-27B-Heretic-Abliterated-Uncensored-GGUF}}
}
```

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)