How Much Better Can an AI Get Without a New Base Model?

Imagine a coding agent that knows the language, understands the repository, and can call every tool it needs. It still stops after the first plausible fix without checking the rest of the task.

Would a bigger model solve that?

Maybe. But another possibility is that the model needs better training in how to finish work with the capabilities it already has.

That is what caught my attention about Kimi K3.1 Pro, released on October 23, 2026. Moonshot describes it as a post-trained version of Kimi K3, using substantially more post-training compute. It is available through hosted products and APIs; the full weights are promised by November 4, so they are not publicly downloadable yet. Sources: Moonshot's release and the OpenRouter release record.

The useful question is not simply whether K3.1 Pro beats another model on a leaderboard.

How much better can a model become at using what it already learned?

Isn't training just making the model larger?#

Not necessarily. Model size and training effort are different knobs.

Pretraining builds broad capabilities by adjusting weights across a large data mixture. Post-training continues changing those weights to improve particular behaviors, such as following instructions, reasoning, or using tools. Neither step requires adding parameters.

Reinforcement learning adds a feedback loop: the system attempts tasks, receives signals about the results, and updates the model to make useful behavior more likely. For an agent, that can mean learning patterns across a whole sequence of actions rather than producing one attractive answer.

Moonshot's technical post describes joint reinforcement learning across hundreds of downstream domains, supported by work on sandboxes, rollout scheduling, long-context inference and environment-quality checks.

A rollout is an attempted solution trajectory. The sandbox gives that attempt somewhere to run. The environment needs a way to judge what happened.

That makes the training infrastructure part of the capability story. If the environment rewards superficial success, more attempts can reinforce the wrong habit.

What would better agent behavior actually look like?#

Consider an illustrative repository task:

Fix the invoice rounding bug, preserve the existing API, and add a regression test.

A weak attempt might edit the calculation, run one visible example and stop. A stronger attempt would inspect the rounding rule, check the API boundary, add a test that fails on the original behavior, and verify the fix.

Both agents might know the same arithmetic and programming syntax. The difference is how they use that knowledge across the workflow.

This is the part of post-training I find interesting: it can improve the policy for choosing the next action. That does not mean K3.1 Pro reliably completes this particular example; I haven't run it. It explains why an improvement need not come from a new base model.

Do independent results show that kind of improvement?#

Artificial Analysis's October 23 evaluation reports an Intelligence Index score of 55, up from 44 for Kimi K3. More specifically, its Terminal-Bench 4.0 result rises from 13% to 67%, and AutomationBench-AA rises from 58% to 73%.

The gains are uneven. In the same report, SciCode moves from 59% to 61%, while Humanity's Last Exam moves from 47% to 52%.

That pattern is more informative than one overall rank. It suggests a particularly large improvement on some agent workflows, without an equally large jump on every kind of problem.

It is still a comparison of complete model systems and evaluation settings. It does not isolate exactly how much improvement came from the RL recipe, data, reasoning budget or other changes.

What happens on a concrete data-analysis task?#

Plotly tested the standard model through OpenRouter on launch day. Its exam uses a synthetic wind-energy warehouse, with ordinary analysis questions, hidden patterns, data-quality problems and impossible requests.

These are Plotly's measurements in its own harness, not mine. The exam is private, which limits public reproduction.

The remaining failure is revealing. When comparing firmware groups, the model included days before firmware was installed in the other-firmware group. It produced a plausible statistical analysis while mishandling what belonged in the comparison.

That is exactly the kind of issue a polished answer can hide. Tool use helps, but choosing what the data means remains part of the job.

::strip{src=https://plotly.com/cms/img/fsV-iOIv7ZmV8lWp_data-analytics-benchmark-accuracy-vs-speed-kimi-k3-1-pro.png?auto=compress&fm=webp alt="Plotly's original accuracy-versus-latency chart, with Kimi K3.1 Pro near 98 percent accuracy and about 150 seconds median per task" caption="Original independent evaluation chart, credit Plotly, October 23, 2026. Results belong to PlotlyDataAnalyticsBench and its harness; the horizontal axis is logarithmic." size=standard}

Is more thinking the same as better training?#

No. These happen at different stages.

Post-training changes the model before deployment. Reasoning effort controls how it works through a request after deployment. Moonshot's reasoning documentation says K3.1 Pro always reasons, supports low, high and max, and defaults to max.

An Atomic Agent test makes that distinction practical. The developers used one Python package with five planted bugs, two missing features and 32 hidden tests. They ran each setup three times in Atomic Agent v0.6.9. Kimi used the direct API with temperature 1.

Both low and max effort passed all 32 hidden tests in all three runs. Low averaged 2.2 minutes, versus 4.9 minutes for max. Reported cost ranges were $0.16–$0.21 and $0.35–$0.46 per run respectively.

Those ranges include an unresolved discrepancy between token-based calculations and the account-balance change. And 96 passing test executions are not 96 independent coding tasks: they are 32 tests repeated over three runs.

For this small task, extra reasoning did not improve the measured pass result. It might matter on harder work. The lesson is to test the effort setting instead of assuming the default is the best operating point.

Does a lower token price guarantee cheaper work?#

An API quote prices units of activity. The application pays for the total activity it uses.

Moonshot's billing documentation distinguishes input, output and cache charges. In an agent loop, those costs accumulate across calls, including attempts that don't produce an accepted result.

A simple illustrative comparison: a model charging half as much per token can still cost more if it uses three times as many billed tokens. Real bills also depend on input/output mix and caching.

That is why I would record completed tasks, total bill, elapsed time and retries together. Token price alone cannot tell me whether the agent is economical.

What should we still be careful about?#

First, aggregate scores do not guarantee factual reliability. Artificial Analysis reports an AA-Omniscience hallucination rate of 44%, improved from K3's 53%. Its metric definition measures incorrect answers among non-correct responses, including partial answers and abstentions in the denominator. It is not the percentage of everyday answers that are false.

Second, the training account is Moonshot's explanation. The outside tests support useful capability changes, but they are not a controlled ablation of its training pipeline.

Third, API tests do not establish local-serving performance. Public weights are still pending as of October 24. Hardware, quantization and self-hosting behavior cannot be inferred from a hosted run.

I verified original written evaluations from Artificial Analysis, Plotly and Atomic Agent. I could not verify a detailed independent video test through accessible footage or a reliable transcript, so I have not treated creator clips as evidence.

So what would I test before switching an agent?#

I would use the same harness, tools and acceptance tests, then compare low and max effort on tasks that resemble the application's work. I would include failures that require recovery, incomplete data, and requests where stopping or asking a question is correct.

The benchmark improvement is a reason to investigate. The operating point still has to be chosen.

K3.1 Pro is a useful reminder that capacity, training and inference effort are three separate places to spend compute. Progress can come from teaching an existing base model to use its capabilities more effectively.

For an engineer, the next question is how much of that improvement survives inside a real workflow—and how much time and computation we should spend to get it.