Kimi K3 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

Summary

GPT-5.6 Sol edges Kimi K3 on single-shot quality, but Kimi wins on pass@k with k > 1 and costs 64% less per completed task. The two models succeed and fail in different ways, which makes routing between them the strongest play on the benchmark.

Kimi K3 vs GPT-5.6 Sol is close on DeepSWE pass@1: Sol leads 72.7% to 68.5%, a 4.2 point gap.

Give the models more attempts and Kimi K3 pulls ahead. It wins pass@2 (82.0 vs 81.0) and pass@4 (89.4% vs 85.8%).

Kimi K3 is far cheaper: \$4.65 per rollout vs \$8.37, and 2.8x more solved tasks per dollar.

GPT-5.6 Sol is the more reliable model: it solves more tasks on all 4/4 tries (61 vs 45).

The models diverge (0.46 correlation across which tasks they solve correctly and incorrectly) and fail differently, so a Kimi-first cascade that escalates to Sol covers 108 of 113 tasks and reaches about 85.6%.

Available now · US-hosted

Run Kimi K3 on Together AI

Full 1M context, automatic prefix caching, OpenAI-compatible API, served from US infrastructure.

Open the playground

DeepSWE · Head to Head

Kimi K3 vs GPT-5.6 Sol at a glance

Metric Kimi K3 GPT-5.6 Sol

DeepSWE pass@1 68.5% 72.7%

DeepSWE pass@2 82.0% 81.0%

DeepSWE pass@4 89.4% 85.8%

Coverage (solved at least once) 89.4% 85.8%

Reliability (4/4 pass rate) 76.6% 84.5%

Tasks solved four-for-four 45 61

Cost per rollout $4.65 $8.37

Solved tasks per $100 14.7 5.3

Median rol

…