KwaiKAT Team Releases KAT-Coder-V2.5: An Agentic Code Model Trained on 100,000+ Verified Locations

The KwaiKAT team at Kuaishou has introduced KAT-Coder-V2.5. It’s a code model that’s trained to run inside real, executable repositories rather than a single-turn code release. The given model is available through StreamLake. An open source variant, KAT-Coder-V2.5-Dev, was released separately from Hugging Face under Apache-2.0.

AutoBuilder: environments that use target tests

The study puts the confirmed work as a triplet. It requires a precise job description, a usable repository, and a set of validation tests. A patch is only good if it passes all of them.

Jobs are mined from actual pull requests and commits, following the SWE bench pedigree. A compiled code modification provides a golden patch and the corresponding test modification provides a test patch. The raw problem text is discarded as specification. Instead, the definitions are refactored into three parts: the problem statement based on the golden patch, the requirements derived from the test patch, and the interface limitations derived from both. Check for clarity and discount anything that is confusing, incomplete, unclear, or internally inconsistent.

AutoBuilder handles the natural side. The build agent analyzes the repository and writes a configuration script that installs dependencies and runs tests from clean output. The authentication agent signs that script in an isolated sandbox.

The law of acceptance is the interesting part. Validation does not read exit codes or grep log patterns. It processes the output of a structured test framework, and accepts a site only when more than 90% expected tests are collected and the pass/fail results are reproducible for all runs. Failures are reported as systematic information for repeated maintenance.

Combining a preconfigured base environment, build system templates, and a retrievable library of prepackaged build recipes increased the build success rate from 16.5% to 57.2%. The result is more than 100,000 verified sites in 12 languages. Git history, commit metadata, and other executable traces are removed so that agents cannot read the reference solution from the repo.

Data Scaling Flywheel: sorting through the process

Sorting trajectories by final test success is misleading. Some pass-throughs rely on hard-coding, path-passing, or test-oriented shortcuts. Some of the failed runs contain key search, localization, and maintenance behaviors.

KwaiKAT addresses both directions. For near misses, process-level targeting techniques indicate what to test or verify without revealing the solution. That alone increases the pass rate of previously failed jobs to around 20%. Because the hinted trajectories contain information that is not available at the point of inference, the confirmed fragment is then corrected and the unhinted trajectory is reproduced from the original task context. Only samples that pass validation, show no leaks, and remain patch compatible are retained.

In passing runs, rule-based gates remove invalid, unstable, or exploitative trajectories. The scoring section then evaluates testing, localization, foresight, specification fidelity, repository principles, patch minimum, verification quality, discovery behavior, and reliability.

The third approach is directed at clustering overfitting. Tool names, conflicting conventions, output formats, and information templates are not randomized while functionality is maintained. Because validation is based on test results rather than tracking, a single function can be reused under multiple harness configurations. Practical bugs are also injected: missing dependencies, temporary command failures, truncated output, and noisy logs.

Check the pipe and numbers

Infrastructure failure has put rewards ahead of algorithmic limits

During the training of KAT-Coder-V2, slow reward curves were first suspected in the RL algorithm. However, the study revealed that ~16% of trajectories failed due to sandbox infrastructure problems rather than model policy, and boundary misalignment sometimes spilled observations by ~40 steps and corrupted rewards.

Three infrastructure reforms followed. First, the early release image policy reduced disk usage from 95% to 60%, reducing invalid releases caused by timeouts from 6–7% to less than 1%. Second, adjusting the environmental variables during the start of the remote sandbox stopped the system output that responded to the rewards in 6-7% of the samples, cutting errors less than 1%. Third, the Gateway Server bypassed the normal chat endpoints—which caused a 40% token drift at ~200-turn scale by re-applying_chat_template and re-tokening—and called /do directly to verify the alignment of the output token.

Together, these updates reduced the sandbox response error rate from about 16% to less than 2% and reduced the training crash by an order of magnitude.

Asymmetric PPO and three tier award

The research team chose PPO and GAE over non-critical sequencing methods because production harnesses divide periods into structurally different samples, making grouping difficult.

By using an asymmetric actor-critic, the Critic gets a special training context (rewards, tests, installations, patches, metadata, upcoming turns), while the Actor only sees the release state. The context of criticism and extrapolation is considered to be discarded.

The awards have three categories: Core Task Scores require that all tests fail_to_pass and fail_to_pass; Standard Behavior Limits penalize duplication, bad tool calls, and debugging artifacts; Failed Trajectory Incentives receive a file recovery file with F2 and provide partial test credit.

Five experts converged on Multi-Teacher On-Policy Distillation using reverse KL, non-policy initialization, and drift reduction in Prune-OPD.

Results

Under the integrated Claude Code harness, the KAT-Coder-V2.5 leads its panel in PinchBench with 94.9, beating the Opus 4.8 with 93.5. It ranks second in SWE-Bench Pro (65.2 vs 69.2) and KAT Code Bench internal (53.1 vs 57.3).

However, it lags in Terminal-Bench 2.1, finishing with 60.7 behind GLM-5.1 (61.8) and Opus 4.8 (84.6). In SciCode, it scores 50.3, which is the same as GLM-5.2.

Notably, KAT-Coder-V2.5-Dev with open weight is 35B-total / 3B-active MoE difference after training on Qwen3.6-35B-A3B using 127K SFT examples, then RL. Tested in a different in-house protocol, its results cannot be compared to the main flagship table.

Key Takeaways

• KwaiKAT treats agent codes as an infrastructure problem, not a model scale problem.
• AutoBuilder raised the success rate of building environments from 16.5% to 57.2%, generating 100,000+ verifiable sites in 12 languages.
• A sandbox test found ~16% of RL trajectories failed because of the sandbox, not the policy; corrections reduced that to less than 2%.
• KAT-Coder-V2.5 tops PinchBench at 94.9 and ranks second in SWE-Bench Pro at 65.2, behind Opus 4.8.
• Open source KAT-Coder-V2.5-Dev is a 35B-A3B MoE variant under Apache-2.0, with its own stop numbers.

Check out the Paper, Model of Weight on a Flat Face, and Product Page. All credit for this study goes to the researchers of this project.