VibeVoice ASR BitNet
VibeVoice ASR BitNet by microsoft, a automatic-speech-recognition model with speech-to-text, multimodal capabilities. Understand and compare speech-to-text, multimodal features, benchmarks, and capabilities.
Comparison
| Feature | VibeVoice ASR BitNet | Interfaze |
|---|---|---|
| Input Modalities | audio, text | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | 51 partial | 162+ |
| Native Speech-to-Text | Yes | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 64K | 1M |
| Tool Calling | No | Tool calling supported + built in browser, code execution and web search |
Speech-to-Text Capabilities
| Feature | VibeVoice ASR BitNet | Interfaze |
|---|---|---|
| Time Stamping | No | Yes |
| Speaker Diarization | No | Yes |
| Long Audio Processing | Partial | Yes |
| Audio Processing Speed | 1hr of audio more than 5mins (varies depending on provider and quality output) | 1hr of audio under 30 seconds |
| Intent Recognition | No | Yes |
| Sentiment Analysis | No | Yes |
| Prompting Correction | No | Yes |
Scaling
| Feature | VibeVoice ASR BitNet | Interfaze |
|---|---|---|
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |
View model card on Hugging Face
VibeVoice-ASR-BitNet
VibeVoice-ASR-BitNet is a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs — no GPU required. Through heterogeneous quantization, the model is compressed from 4.62 GB to 1.58 GB while achieving 1.6–2.3× faster inference than Whisper.cpp with real-time capability (RTF < 1) on as few as 3 CPU threads.
➡️ Code: microsoft/VibeASR.cpp ➡️ Report: VibeVoice-ASR-BitNet Technical Report ➡️ Base Model: microsoft/VibeVoice-ASR
🔥 Key Features
- ⚡ Real-time on CPU — RTF < 1 with 3+ threads on commodity x86 (AVX2) and ARM (NEON) hardware
- 📦 Compact — 1.58 GB total (2.9× compression from FP16), fits in edge device memory
- 🌍 Multilingual — English, Chinese, French, Italian, Korean, Portuguese, Vietnamese, and more
- 🔧 Custom SIMD Kernels — Fused operators within the ggml framework for both ARM and x86 platforms
Quantization Strategy
| Component | FP16 | Quantized | Method | Compression |
|---|---|---|---|---|
| VAE Tokenizer | 1.31 GB | 0.65 GB | I8_S | 2.0× |
| LM Decoder | 3.32 GB | 0.92 GB | I2_S + Q6_K | 3.6× |
| Total | 4.62 GB | 1.58 GB | — | 2.9× |
Evaluation
Inference Speed
| Threads | 1 | 2 | 3 | 4 | 6 | 8 |
|---|---|---|---|---|---|---|
| RTF | 1.98 | 1.08 | 0.77 | 0.63 | 0.49 | 0.42 |
| vs. Whisper.cpp | 2.28× | 2.12× | 1.86× | 1.86× | 1.71× | 1.55× |
Benchmarked on AMD EPYC 7V13 (AVX2+FMA) with 20s audio. Bold = RTF < 1 (real-time).
Accuracy (WER%)
| Benchmark | VibeVoice-ASR-7B | VibeVoice-ASR-BitNet | Parakeet | Whisper | SenseVoice | FunASR |
|---|---|---|---|---|---|---|
| MLC-EN | 7.82 | 8.25 | 8.40 | 13.57 | 12.39 | 11.36 |
| MLC-FR | 16.03 | 17.41 | — | — | — | — |
| MLC-IT | 15.67 | 17.23 | — | — | — | — |
| MLC-KO | 9.83 | 11.15 | — | — | — | — |
| MLC-PT | 22.41 | 24.87 | — | — | — | — |
| MLC-VI | 20.15 | 22.38 | — | — | — | — |
| AISHELL4 | 19.83 | 27.45 | — | — | 22.52 | 20.41 |
| AMI-ihm | 17.42 | 21.36 | 21.92 | 27.07 | 30.81 | 32.07 |
| AMI-sdm | 24.18 | 25.87 | 26.33 | 36.92 | 48.11 | 40.17 |
| AliMeeting | 36.21 | 40.58 | — | — | 38.75 | 39.27 |
| Fleurs-en | 4.73 | 5.21 | 4.09 | 3.99 | 6.84 | 4.93 |
| Fleurs-zh | 7.92 | 8.35 | — | — | 5.56 | 7.00 |
| Libri-clean | 2.17 | 2.41 | 1.49 | 1.98 | 2.78 | 1.58 |
| Libri-other | 5.84 | 6.27 | 3.13 | 3.60 | 6.81 | 4.01 |
| VoxPopuli | 4.92 | 5.18 | 5.26 | 7.19 | 8.63 | 6.46 |
Model Files
| File | Size | Description |
|---|---|---|
vibeasr-vae-encoder-i8_s.gguf | 0.65 GB | VAE tokenizer, I8_S quantized (ready to use) |
vibeasr-lm-i2_s-embed-q6_k.gguf | 0.92 GB | LM decoder, I2_S quantized (ready to use) |
model-*.safetensors | 10.7 GB | Original SafeTensors (for conversion) |
License
This project is licensed under the MIT License.
Contact
This project was conducted by members of Microsoft Research. If you have suggestions, questions, or observe unexpected behavior, please contact us at VibeVoice@microsoft.com.