Motif 3 Beta

Motif 3 Beta by Motif-Technologies, a text-generation model. Understand and compare features, benchmarks, and capabilities.

Comparison

Feature	Motif 3 Beta	Interfaze
Input Modalities	text	image, text, audio, video, document
Native OCR	No	Yes
Long Document Processing	No	Yes
Language Support	unknown	162+
Native Speech-to-Text	No	Yes
Native Object Detection	No	Yes
Guardrail Controls	No	Yes
Context Input Size	262.1K	1M
Tool Calling	No	Tool calling supported + built in browser, code execution and web search

Scaling

Feature	Motif 3 Beta	Interfaze
Scaling	Self-hosted/Provider-hosted with quantization	Unlimited

View model card on Hugging Face

⚠️ Preview / beta checkpoint — not the final release. This repository hosts an intermediate checkpoint of Motif-3. The final checkpoint will be released soon.

Motif-3 is a large-scale Mixture-of-Experts (MoE) language model built from the ground up by Motif Technologies following a fully in-house, proprietary design — not a re-parameterization of existing open-source architectures.

Highlights

🧠 ~314B total parameters / ~13B active per token (sparse MoE)
📏 256K context length (262,144 tokens), natively long-context
⚡ Sparse routing: 384 experts with 8 activated per token, plus 1 shared expert
🌐 Multilingual, general-purpose

Model details


Model type	Mixture-of-Experts causal language model
Total parameters	~314B
Active parameters	~13B / token
Hidden size	4096
Layers	53
Routed experts	384 (top-8)
Shared experts	1
Context length	262,144 (256K)
Vocabulary	220,160
Tensor type	bfloat16

Architecture

Motif-3 is a fully in-house design and introduces several custom components:

Grouped Differential Latent Attention (GDLA)
Grouped PolyNorm activation, applied per expert
Modified mHC

Benchmarks

Artificial Analysis Intelligence Index (AAII): 44

See Artificial Analysis for details.

Usage

A dedicated vLLM serving guide is coming soon.

The model ships with custom modeling code, so load it with trust_remote_code=True:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Motif-Technologies/Motif-3-Beta"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Hello!"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Access

This model is openly available — anyone can download the weights, no access request required.

License

Permission is granted to use, modify, and redistribute this software for personal, educational, and non-commercial research purposes only.

Commercial use is prohibited without prior written permission from Motif Technologies.