Does an LLM Have to Write From Left to Right?

A developer asks an AI assistant for a complete HTML page. The answer is going to be long, and the assistant starts producing it from the beginning.

For many familiar language models, that is more than a display choice. The next token depends on the tokens already generated. Faster hardware helps, but the output still has a sequential dependency.

What if the model didn't have to construct the answer that way?

On October 19, 2026, Inception released Mercury 3, its latest diffusion language model. The company reports 1,068 output tokens per second on NVIDIA GPUs.

That number caught my attention. But the more interesting question is:

Can changing how a model produces text change which applications feel practical—and what does that speed number leave out?

Does an LLM actually have to write from left to right?#

An autoregressive model generates a continuation one token at a time. A token might be a word, part of a word, or punctuation. Each generated token becomes part of the context for the next prediction.

A diffusion language model approaches generation as a process of repeated refinement instead.

Inception's original Mercury technical report describes a model that starts with noisy output and improves multiple tokens in parallel through successive denoising steps. It still uses a Transformer neural network. “Diffusion” describes the training and generation approach, not the removal of attention or neural computation.

That report is from June 17, 2025, and covers Mercury Coder, not Mercury 3. It explains the family’s underlying idea; it is not a complete specification of the new model's internals.

My mental picture is a draft with several uncertain positions. Rather than permanently committing to the first word, then the second, the system repeatedly improves parts of the draft together.

That is an analogy, not a captured Mercury 3 generation trace.

But aren't those refinement steps still sequential?#

Yes. Parallel generation doesn't mean the whole answer appears correctly in one operation.

One refinement step still follows another. The opportunity is that a step can improve several token positions, rather than advancing the output by exactly one token.

Imagine filling a long form. One approach completes each field in order, using the preceding fields as context. Another repeatedly reviews several incomplete fields together until the form is ready. The second approach can expose more parallel work, but it still has to resolve uncertainty and maintain consistency.

This is why I wouldn't interpret “1,000 tokens per second” as “the model has stopped doing expensive work.”

Inception's earlier report also describes a proprietary serving engine and custom kernels. The generation method and the system implementing it both matter.

If text is refined in parallel, how does streaming work?#

This is where the API becomes interesting.

Inception's current streaming documentation says Mercury 3 normally streams blocks of refined text, rather than individual tokens. It also offers a diffusing mode that returns intermediate versions of the text.

There is an integration detail here that is easy to miss: in that mode, each chunk contains the text as it is being refined. The documented interface replaces the displayed text rather than simply appending every chunk.

Appending those snapshots could turn one evolving answer into several repeated answers.

That is a concrete example of why an API-compatible model can still deserve application-level testing. The endpoint can look familiar while the output's behavior changes.

Inception's official launch animation gives a visual impression of text becoming ordered. It is an illustration, not a recording of a measured inference run. I couldn't verify a controlled independent Mercury 3 video comparison in the sources checked.

So what actually changed with Mercury 3?#

The new release brings the diffusion approach into a more capable model, according to Inception's own evaluations. Its current model documentation lists a one-million-token context window, text input, tool calling, and structured outputs.

The launch comparison uses models labeled “medium” reasoning. Two coding rows show why the result deserves more than a winner label:

Source: Inception's original comparison chart. These are the company's results, not my measurements. A shared reasoning label doesn't establish an identical computation budget.

One benchmark favors Mercury; another favors Haiku. That is enough to make me cautious about replacing “different tradeoff” with “better at coding.”

What happens when someone outside Inception tries it?#

Two early reports make the picture more useful.

On October 20, Ellis Crosby at Spring Prompt reported tests through OpenRouter, served by Inception. In speed chess, with what the tester calls thinking switched off, Mercury 3 returned moves in a median 0.42 seconds.

With reasoning enabled, some moves took much longer; one reached 105 seconds after invalid replies. Across 20 shared playground tasks, the report gives Mercury a median completion time of 65 seconds, versus 44 for Haiku 5.5 and 47 for GPT-6 Luna.

Those are different tasks and measurements from an output-throughput benchmark. Hardware and quantization are not disclosed for the hosted service, and the article doesn't establish matched reasoning budgets across providers.

Also on October 20, Corey Noles at The Neuron reported eight API runs through OpenRouter.

His generated dungeon crawler passed a JavaScript syntax check and 2,000 randomized map-generation checks for the conditions he tested. That is encouraging, but those checks aren't 2,000 independent model generations or proof that every gameplay interaction works.

His constrained story missed some requirements. A fabricated-space-mission prompt was rejected, but the response introduced another factual error and used web searches.

Noles explicitly describes editorial spot checks rather than repeated, controlled benchmarking. Sampling settings and serving hardware are not fully specified. I haven't reproduced either tester's results.

How can fast generation still produce a slow answer?#

Because the number of tokens matters as much as the rate.

As a toy calculation, 10,000 generated tokens at 1,000 tokens per second still require about ten seconds of generation. That excludes input processing, network delays, tool execution, and retries. It is arithmetic, not a Mercury 3 latency prediction.

A model might generate quickly but spend more tokens exploring a problem. An agent might also make several model calls before doing anything visible.

The reasoning controls make this distinction explicit: Mercury 3 accepts instant, low, medium, and high. The documentation currently says the default is low.

I would record the actual setting in a test, rather than assume every reviewer means the same thing by “thinking on” or “off.”

Where would I try it first?#

I would start with a bounded feature whose output can be checked: generating a UI draft, transforming supplied text into a schema, or proposing a small code change with tests.

For the HTML-page example, my comparison would hold the prompt and requirements fixed, then record:

• Time until useful output appears and time until completion.
• Output and reasoning-token usage.
• Whether the page runs and satisfies the requested behavior.
• Retry count and total cost for an accepted result.

That last phrase matters. A cheap, quick answer that needs repeated repairs can lose its advantage.

I haven't run Mercury 3 myself. The evidence here consists of primary documentation, company-reported comparisons, two attributed firsthand reports, and my own engineering interpretation.

What interests me isn't a promise that every answer will become instantaneous. It's that left-to-right generation is not the only way to build a language model's output.

The question I would test next is:

Where does parallel refinement shorten the path to a correct result—not just produce the draft faster?