Edge0 145B NVFP4
Edge0 145B NVFP4 by Edge0, a text-generation model with multimodal capabilities. Understand and compare multimodal features, benchmarks, and capabilities.
Comparison
| Feature | Edge0 145B NVFP4 | Interfaze |
|---|---|---|
| Input Modalities | text, image | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | unknown | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 256K | 1M |
| Tool Calling | Yes | Tool calling supported + built in browser, code execution and web search |
Scaling
| Feature | Edge0 145B NVFP4 | Interfaze |
|---|---|---|
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |
View model card on Hugging Face
Rebuilt, not converted
Edge0 is not a relabelled Qwen3.8-Flash-Next, and not a re-quantization of it: we rebuilt the model's expert layer. In all 48 layers the routed MoE experts were replaced by a latent MoE — the token is projected down to a narrow latent width (ℓ = 1792 on the 145B, 1280 on the 120B), the experts run there, and the result is projected back to 2560. One expert is now a fixed 1.63 MB (145B) or 1.16 MB (120B) record instead of a full-width block. The architecture itself changed, so the model was post-trained afterwards on a long schedule over our own data; it is trained natively at 4 bits, not quantized after the fact.
That is what matters for a device. On-device inference runs out of memory capacity and memory bandwidth long before it runs out of compute, and the expert bytes a token pushes through set both decode speed and how much memory a machine needs:
| Qwen3.8-Flash-Next | Edge0-145B | Edge0-120B | |
|---|---|---|---|
| Routed expert parameters | 120.8 B | 84.6 B (−30%) | 60.4 B (−50%) |
| Expert parameters per token | 2.36 B | 1.65 B | 1.18 B |
| Expert pool on disk · NVFP4 builds | 63.2 GiB | 44.3 GiB | 31.6 GiB |
| Decode · 16 GiB VRAM + 20 GiB RAM | 18.5 tok/s | 36.8 tok/s | 62.4 tok/s |
On that budget, decode runs 2.0× Qwen's on the 145B and 3.4× on the 120B. Quality holds through the rebuild: the 145B is ahead on the agentic benchmarks — deepswe 1.1 64.6 vs 58.7, terminal-bench 2.1 86.5 vs 83.3 — and level elsewhere.
Capacity controls · VRAM and host RAM
All of these live in the pack's model.json, not on a command line: flags go in args, environment
variables in env, with paths relative to the pack directory. That is the same file the Edge0 Chat app
and edge0-infer read, so a change there applies to every launch.
"args": ["--expert-cache", "auto", "--kv-resident", "32768", "--vram-reserve-mib", "1024"],
"env": {"STRATA_LATENT_DENSE": "./latent_dense_q8.bin", "STRATA_LATENT_DENSE_FMT": "8",
"STRATA_LAT_FUSED_Q3K": "0"}VRAM
| Flag | Meaning | Default |
|---|---|---|
--vram-reserve-mib N | VRAM left free for the desktop, the driver and other processes; the engine sizes its caches from what remains. It means "leave this much free", not "give the engine this much", so the same number means different things on cards with different totals. Accepted but not listed by --help. | 700 |
--expert-cache N|auto | Expert slots kept in VRAM. auto derives the count from the free VRAM after the fixed footprint (skeleton projections ≈ 2.9 GiB, MTP draft ≈ 0.8 GiB, verify and draft-head buffers ≈ 0.45 GiB, plus the D/U device buffers). | 0 (none) |
--expert-cache-device1..3 N|auto | Use the second and third GPU as further expert-cache tiers. | off |
--kv-resident N | Keep N KV cells per QSA layer in VRAM and stream the K/V from pinned host RAM, which frees VRAM for experts. | 0 (all in VRAM) |
--mtp-q4 all|proj|head | 4-bit copies of the draft layer's projections and/or head, about 0.3–0.5 GiB less VRAM. | off |
--prefill CHUNK | Prompt chunk size; bigger chunks need bigger device buffers. auto uses the largest chunk whose buffers the expert cache can lend. | auto |
--spec N | The MTP drafter's verify window: how many tokens it proposes per check. A bigger window needs bigger verify buffers (about 180 MiB at 4) and pays off when every window has a fixed per-layer cost. The shipped packs use 4; the macOS setup picks 2, or 3 when the experts stream from SSD. Without an MTP layer there is no speculation. | 4 |
Host RAM
| Flag | Meaning | Default |
|---|---|---|
--memory-limit-mib N | The RAM this engine's container or cgroup may use, for when the container cannot see its own limit. | 0 (detect) |
--resident-budget-gib N | Cap on the host-RAM expert cache. It holds only the experts the VRAM cache does not hold, so the two sets are disjoint, and the value is clamped to the RAM actually available minus headroom. Requires a static --expert-profile. | 0 (off) |
--resident-experts | The same with the whole complement as the budget. | off |
--mmap-experts | Read experts through the pack file on demand instead of pinning them, so nothing is resident in RAM. | off |
--conversation-cache-mib N | Conversation cache. 0 turns it off. | adaptive |
--ple-io direct|mmap|ram | The PLE table on its SSD: direct reads it unbuffered and it never enters memory; ram locks the whole table in RAM. | direct |
--kv-resident N | The streamed K/V lives in pinned host RAM (1.55 GiB at 32768 cells). | — |
Community builds
Third-party conversions of this model. They are not made or endorsed by Edge0; this model's licence and the upstream terms still apply to them.
- infosave/Edge0-145B-cmf — a single-file CMF build for cortiq (Rust, CUDA and Vulkan, OpenAI-compatible). The routed experts are this NVFP4 build's weights copied bit for bit; the D/U projections and the shared expert are re-encoded to 8 bits and the router to 16. The author reports it running from 12 GB cards.