# MOSS Transcribe Diarize

URL: https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize

[All models](https://interfaze.ai/models)

MOSS Transcribe Diarize by OpenMOSS-Team, a audio-text-to-text model with speech-to-text, multimodal capabilities. Understand and compare speech-to-text, multimodal features, benchmarks, and capabilities.

## Comparison

| Feature | MOSS Transcribe Diarize | Interfaze |
| --- | --- | --- |
| Input Modalities | audio, video | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | 4 partial | 162+ |
| Native Speech-to-Text | Yes | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 128K | 1M |
| Tool Calling | No | Tool calling supported + built in browser, code execution and web search |

### Speech-to-Text Capabilities

| Feature | MOSS Transcribe Diarize | Interfaze |
| --- | --- | --- |
| Time Stamping | Partial | Yes |
| Speaker Diarization | Partial | Yes |
| Long Audio Processing | No | Yes |
| Audio Processing Speed | No | 1hr of audio under 30 seconds |
| Intent Recognition | No | Yes |
| Sentiment Analysis | Yes | Yes |
| Prompting Correction | Yes | Yes |

### Scaling

| Feature | MOSS Transcribe Diarize | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize)

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.

Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.

## News

-   2026-07-09: Released MOSS-Transcribe-Diarize 0.9B.

## Contents

-   [Introduction](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#introduction)
-   [Model Architecture](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#model-architecture)
-   [Evaluation](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#evaluation)
-   [Quickstart](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#quickstart)
    -   [Environment Setup](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#environment-setup)
    -   [Python Usage](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#python-usage)
    -   [Custom Prompt and Hotwords](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#custom-prompt-and-hotwords)
    -   [Serve with vLLM and SGLang](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#serve-with-vllm-and-sglang)
    -   [Subtitle Web App](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#subtitle-web-app)
-   [Output Format](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#output-format)
-   [More Information](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#more-information)
-   [License](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#license)
-   [Citation](https://interfaze.ai/models/openmoss-teammoss-transcribe-diarize#citation)

## Introduction

MOSS-Transcribe-Diarize 0.9B turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, it jointly performs speech transcription and speaker diarization, producing time-aligned text with consistent speaker labels.

The model is built for meetings, calls, podcasts, interviews, lectures, videos, and other long or messy multi-speaker recordings. It can also emit acoustic event annotations, giving downstream systems a richer view of what happened, who spoke, and when.

Core capabilities:

-   **Long-form transcription**: Converts long audio or video recordings into timestamped text.
-   **Speaker-aware diarization**: Assigns anonymous speaker labels such as `[S01]` and `[S02]` without a separate diarization pipeline.
-   **Promptable generation**: Supports custom transcription instructions, hotwords, and acoustic event annotations.

## Model Architecture

| Component | Specification |
| --- | --- |
| Text backbone | Qwen3-0.6B style causal decoder |
| Audio encoder | Whisper-Medium encoder configuration |
| Audio frontend | WhisperFeatureExtractor, 16 kHz, 80 mel bins, 30 s chunks |
| Audio-text bridge | 4x temporal merge + MLP adaptor |
| Fusion | Audio features replace <|audio\_pad|> embeddings via masked\_scatter |
| Output format | Compact \[start\]\[Sxx\]text\[end\] transcript with speaker tags such as \[S01\] |

This Hugging Face repository includes the custom Transformers remote code required to load the model with `trust_remote_code=True`.

## Evaluation

We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Error Rate (CER), concatenated minimum-permutation Character Error Rate (cpCER), and Delta-cp. Lower is better for all metrics. A dash (`-`) indicates that the result is unavailable.

## Quickstart

### Environment Setup

Use a clean Python environment. The model uses custom Transformers code, so load the model and processor with `trust_remote_code=True`.

```
conda create -n moss-transcribe-diarize python=3.12 -y
conda activate moss-transcribe-diarize

git clone https://github.com/OpenMOSS/MOSS-Transcribe-Diarize.git
cd MOSS-Transcribe-Diarize

pip install --index-url https://download.pytorch.org/whl/cu128 torch torchaudio
pip install -e .
```

The GitHub package provides helper utilities such as audio/video loading, transcription message construction, transcript parsing, CLI inference, and the subtitle web app. The model weights and remote-code model files are loaded from this Hugging Face repository.

### Python Usage

```
import torch
from transformers import AutoModelForCausalLM, AutoProcessor

from moss_transcribe_diarize import parse_transcript
from moss_transcribe_diarize.inference_utils import (
    build_transcription_messages,
    generate_transcription,
    resolve_device,
)

model_id = "OpenMOSS-Team/MOSS-Transcribe-Diarize"
audio_path = "audio.wav"

device = resolve_device("auto")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
).to(dtype=dtype).to(device).eval()

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)

messages = build_transcription_messages(audio_path)
result = generate_transcription(
    model,
    processor,
    messages,
    max_new_tokens=2048,
    do_sample=False,
    device=device,
    dtype=dtype,
)

print(result["text"])

for segment in parse_transcript(result["text"]):
    print(segment.start, segment.end, segment.speaker, segment.text)
```

The message flow follows the common Qwen multimodal pattern:

1.  `processor.apply_chat_template(messages, tokenize=False)` renders text with audio placeholders.
2.  The helper utilities load audio waveforms from the same messages.
3.  `processor(text=text, audio=audios)` computes Whisper input features and expands audio placeholders.
4.  `model.generate(...)` produces timestamped transcription and diarization text.

### Custom Prompt and Hotwords

The default prompt is optimized for timestamped transcription and speaker diarization:

```
请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。
```

To add hotwords, append a short hint to the default prompt:

```
请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。热词提示：热词1, 热词2, 热词3
```

More prompt recipes are available in the GitHub repository: [https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/main/examples/prompts.md](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/main/examples/prompts.md)

### Serve with vLLM and SGLang

MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.

```
uv pip install -U vllm \
  --torch-backend=auto \
  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
```

or:

```
uv pip install -U vllm \
  --torch-backend=auto \
  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130
```

```
vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code
```

```
curl http://localhost:8000/v1/audio/transcriptions \
  -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
  -F file=@"audio.wav" \
  -F response_format="json" \
  -F temperature="0"
```

The recommended way to serve MOSS-Transcribe-Diarize is [SGLang Omni](https://github.com/sgl-project/sglang-omni) through the OpenAI-compatible `/v1/audio/transcriptions` endpoint. Install `sglang-omni` by following the [installation guide](https://github.com/sgl-project/sglang-omni/blob/main/docs/get_started/installation.md), then download the model:

```
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize
```

Serve the model:

```
sgl-omni serve \
  --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --port 8000 \
  --max-running-requests 16 \
  --cuda-graph-max-bs 16 \
  --mem-fraction-static 0.80
```

Use `response_format=verbose_json` when you need parsed speaker segments. `json` returns the raw transcript text only.

```
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F file=@audio.wav \
  -F response_format=verbose_json
```

```
import requests

with open("audio.wav", "rb") as f:
    resp = requests.post(
        "http://localhost:8000/v1/audio/transcriptions",
        data={
            "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
            "response_format": "verbose_json",
        },
        files={"file": ("audio.wav", f, "audio/wav")},
        timeout=300,
    )

resp.raise_for_status()
payload = resp.json()
print(payload["text"])
for segment in payload.get("segments", []):
    print(f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}")
```

For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript:

```
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F file=@audio.wav \
  -F response_format=verbose_json \
  -F max_new_tokens=65536
```

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| file | file | required | Audio file uploaded as multipart form data |
| model | string | server default | Model identifier |
| language | string | unset | Optional language hint |
| response\_format | string | json | json, verbose\_json, or text |
| temperature | float | model default (0.0) | Sampling temperature |
| max\_new\_tokens | int | 5120 | Max generated tokens; raise for long audio, for example 65536 |
| prompt | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |

For benchmarking, performance numbers, and implementation details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md). The following single-H100 results are reported for short- and long-sequence multi-speaker ASR tasks.

`movies` short-sequence ASR:

| Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio\_s/s |
| --- | --- | --- | --- | --- |
| 1 | 2.57 | 0.388 | 0.0612 | 29.76 |
| 2 | 4.89 | 0.409 | 0.0659 | 56.55 |
| 4 | 6.62 | 0.513 | 0.0790 | 76.64 |
| 8 | 6.80 | 0.533 | 0.0810 | 78.70 |
| 16 | 7.08 | 0.659 | 0.0922 | 81.98 |

`aishell4_long` long-sequence ASR:

| Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio\_s/s |
| --- | --- | --- | --- | --- |
| 1 | 0.022 | 45.2 | 0.0197 | 50.64 |
| 2 | 0.032 | 60.7 | 0.0265 | 74.25 |
| 4 | 0.036 | 105.6 | 0.0461 | 81.64 |
| 8 | 0.040 | 172.6 | 0.0754 | 90.62 |
| 16 | 0.043 | 282.8 | 0.1237 | 98.83 |

### Subtitle Web App

The source package includes a local subtitle workflow for upload, review, subtitle export, and optional FFmpeg burn-in:

```
mtd-subtitle-web \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --host 127.0.0.1 \
  --port 7860
```

Open `http://127.0.0.1:7860`, upload an audio/video file, review the parsed subtitle segments, then download JSON/SRT/ASS or burn an MP4 if `ffmpeg` and `ffprobe` are available on `PATH`.

For batch processing:

```
mtd-subtitle /path/to/input.mp4 \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --out-dir runs/example \
  --render
```

## Output Format

The canonical output format is:

```
[start_time][Sxx]transcribed speech[end_time]
```

Example:

```
[0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready for evaluation[13.81][14.36][S01]Great, include the diarization results in the report[18.76]
```

In this format:

-   `start_time` and `end_time` are timestamps in seconds.
-   `[S01]`, `[S02]`, and similar labels are anonymous model-generated speaker labels.
-   Speaker labels are relative labels within the input audio and should not be interpreted as real speaker identities.

## More Information

-   **GitHub**: [https://github.com/OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
-   **MOSI.AI**: [https://mosi.cn](https://mosi.cn)
-   **OpenMOSS**: [https://www.open-moss.com](https://www.open-moss.com)

## License

MOSS-Transcribe-Diarize 0.9B is licensed under the Apache License 2.0.

## Citation

If you use MOSS-Transcribe-Diarize 0.9B, please cite the technical report:

```
@misc{moss_transcribe_diarize_2026,
  title={MOSS Transcribe Diarize Technical Report},
  author={{MOSI.AI}},
  year={2026},
  eprint={2601.01554},
  archivePrefix={arXiv},
  primaryClass={cs.SD},
  url={https://arxiv.org/abs/2601.01554}
}
```

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)