FastContext 1.0 4B SFT

1. Model Introduction

FastContext-1.0 is a lightweight repository-exploration subagent for LLM coding agents. Instead of letting a single model both explore the repository and solve the task, FastContext separates these two roles: it is invoked on demand by a main coding agent, issues parallel read-only tool calls (READ, GLOB, GREP), and returns compact file paths and line ranges as focused context.

Repository exploration is a major bottleneck in modern coding agents — locating relevant code consumes a large share of the token budget and pollutes the solver's context with irrelevant snippets. In our analysis of GPT-5.4 trajectories, reading and searching account for 56.2% of all tool-use turns and 46.5% of the main agent's total tokens. FastContext moves this work into a dedicated subagent so the main agent receives clean, grounded evidence rather than the long trail of exploratory reads and searches.

The model family spans 4B–30B parameters, bootstrapped from strong reference-model trajectories via supervised fine-tuning (SFT) and refined with task-grounded reinforcement learning (RL) for broad first-turn search, multi-turn evidence gathering, and precise citation generation.

• Backbones: Qwen3-4B-Instruct (4B explorer) and Qwen3-Coder-30B-A3B (30B explorer)
• Variants: FC-4B-SFT, FC-4B-RL (deployment targets), FC-30B-SFT (scaling reference)
• Context length: up to 262K tokens
• Paper: FastContext: Training Efficient Repository Explorer for Coding Agents
• Code & data: https://github.com/microsoft/fastcontext

How it works

Internally, FastContext runs an exploration loop:

1. Query understanding — translate the issue into search intents.
2. Parallel tool calling — issue multiple READ / GLOB / GREP calls in a single turn to cover complementary hypotheses.
3. Observation-driven refinement — use tool outputs to guide the next search turn.
4. Final citations — return a compact <final_answer> block of file paths and line ranges.

2. Evaluation Results

End-to-end performance (Mini-SWE-Agent)

Integrating FastContext into Mini-SWE-Agent improves end-to-end resolution rates by up to 5.5% while reducing main-agent token consumption by up to 60%, with only marginal overhead. Scores, tokens, and turns are measured on the main-agent trajectory; deltas are relative to w/o Explore for the same main agent.

Score / Tokens shown per cell. Best result per main-agent block in bold.

Highlights:

• FastContext improves end-to-end accuracy for every main agent and benchmark; the largest gains appear on SWE-bench Pro (e.g. GPT-5.4 +5.5, GLM-5.1 +5.0).
• The biggest token savings reach 60.3% (GPT-5.4 on SWE-QA).
• The compact 4B-RL explorer can outperform the larger 30B-SFT explorer — e.g. on GLM-5.1 SWE-bench Pro it reaches 22.5 vs. 20.0 while using fewer tokens.

3. Quick Start

Launch the model with an OpenAI-compatible server (e.g. SGLang). The example below serves the 4B explorer:

python3 -m sglang.launch_server \
    --model-path FastContext-1.0-4B-SFT \
    --tool-call-parser qwen \
    --context-length 262144 \
    --trust-remote-code \
    --dtype bfloat16 \
    --host 0.0.0.0 \
    --port 30000 \
    --tp-size 1 \
    --mem-fraction-static 0.8

FastContext exposes only three read-only tools to the model:

At each turn the explorer either issues one or more (parallel) tool calls or stops with a final <final_answer> evidence list. Wire FastContext into a coding agent (e.g. Mini-SWE-Agent) as an exploration subagent the main agent can invoke on demand.

4. Training Recipe

FastContext is trained in two stages:

• Supervised fine-tuning (SFT): The exploration traces, split into three sources matching the runtime behavior of the subagent — parallel_toolcalls (broad first-turn search), multiturn_traj (multi-turn evidence gathering), and linerange (precise citation generation).
• Reinforcement learning (RL): The model is rolled out as the actual subagent and optimized with GRPO using a deterministic reward combining file- and line-level F1, a bonus for bounded parallel exploration, and format penalties.

License

This project is licensed under the MIT License.