# Motif 3 Beta

URL: https://interfaze.ai/models/motif-technologiesmotif-3-beta

Motif 3 Beta by Motif-Technologies, a text-generation model. Understand and compare features, benchmarks, and capabilities.

## Comparison

| Feature | Motif 3 Beta | Interfaze |
| --- | --- | --- |
| Input Modalities | text | image, text, audio, video, document |
| Native OCR | No | Yes |
| Long Document Processing | No | Yes |
| Language Support | unknown | 162+ |
| Native Speech-to-Text | No | Yes |
| Native Object Detection | No | Yes |
| Guardrail Controls | No | Yes |
| Context Input Size | 262.1K | 1M |
| Tool Calling | No | Tool calling supported + built in browser, code execution and web search |

### Scaling

| Feature | Motif 3 Beta | Interfaze |
| --- | --- | --- |
| Scaling | Self-hosted/Provider-hosted with quantization | Unlimited |

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)

View model card on [Hugging Face](https://huggingface.co/Motif-Technologies/Motif-3-Beta)

> ⚠️ **Preview / beta checkpoint — not the final release.** This repository hosts an intermediate checkpoint of **Motif-3**. The **final checkpoint will be released soon.**

**Motif-3** is a large-scale Mixture-of-Experts (MoE) language model built from the ground up by [Motif Technologies](https://motiftech.io) following a fully in-house, proprietary design — not a re-parameterization of existing open-source architectures.

## Highlights

-   🧠 **~314B total parameters / ~13B active** per token (sparse MoE)
-   📏 **256K context length** (262,144 tokens), natively long-context
-   ⚡ Sparse routing: 384 experts with 8 activated per token, plus 1 shared expert
-   🌐 **Multilingual**, general-purpose

## Model details

|  |  |
| --- | --- |
| Model type | Mixture-of-Experts causal language model |
| Total parameters | ~314B |
| Active parameters | ~13B / token |
| Hidden size | 4096 |
| Layers | 53 |
| Routed experts | 384 (top-8) |
| Shared experts | 1 |
| Context length | 262,144 (256K) |
| Vocabulary | 220,160 |
| Tensor type | bfloat16 |

## Architecture

Motif-3 is a fully in-house design and introduces several custom components:

-   **Grouped Differential Latent Attention (GDLA)**
-   **Grouped PolyNorm activation**, applied per expert
-   **Modified mHC**

## Benchmarks

**Artificial Analysis Intelligence Index (AAII): 44**

See [Artificial Analysis](https://artificialanalysis.ai/) for details.

## Usage

> A dedicated **vLLM serving guide is coming soon.**

The model ships with custom modeling code, so load it with `trust_remote_code=True`:

```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Motif-Technologies/Motif-3-Beta"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Hello!"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```

## Access

This model is **openly available** — anyone can download the weights, no access request required.

## License

Permission is granted to use, modify, and redistribute this software for personal, educational, and non-commercial research purposes only.

Commercial use is prohibited without prior written permission from Motif Technologies.

* * *

© Motif Technologies. All rights reserved.

## Want more deterministic results?

[Try Interfaze](https://interfaze.ai/dashboard)[Read the Docs](https://interfaze.ai/docs)