How Much Should a Transcription Model Be Allowed to Change?

Imagine turning a recorded deployment review into searchable notes.

Someone says, “Um, we need twenty five replicas—twenty five, not fifty.” A readable transcript could remove the filler and write the numbers as digits. But removing the correction would change something the team might need later.

Both operations can look like cleaning up speech. Only one is harmless in this example.

That is what interests me about NVIDIA's Nemotron 3.5 Transcribe, released on October 28, 2026. Its announcement describes a transcription model whose output convention can be selected through instructions for each request.

The useful question is bigger than whether it recognizes more words:

Can we tell a transcription system what to change without losing what the speaker actually meant?

Why would the same audio need different transcripts?#

A podcast editor might want captions without every hesitation. Someone studying an interview may need those hesitations preserved. A support tool needs an account number in a form another system can look up.

Those are different requirements, even when the recording is identical.

Here is an illustrative example, not output from a model I tested:

The second version is easier to scan. It still preserves the correction. A summary saying “Scale the deployment to 25 replicas” would be another transformation, with its own assumptions about whether this was a decision or a suggestion.

That distinction matters once transcripts feed search, reports or agents. The application needs to know which kind of text it is receiving.

What is NVIDIA putting inside this model?#

Nemotron 3.5 Transcribe connects a speech encoder to a language model, so the system receives both audio information and text instructions.

Its architecture description also includes a frozen speaker-diarization model. Diarization means identifying who spoke when. Speaker-activity information is injected into the speech representations before they reach the language model.

That gives it more to work with than a plain transcript handed to a separate text editor. A text-only cleanup stage cannot recover a word that the first stage never recorded. Access to audio creates an opportunity to resolve that ambiguity, although it does not guarantee a correct answer.

Word timestamps use an additional alignment step after text generation. They help connect output back to the recording; their presence alone does not prove every word was heard correctly.

What can we actually ask it to change?#

The model card's prompting guide distinguishes verbatim transcription, readable text and domain vocabulary.

Its readable-text preset removes fillers and accidental repetitions while retaining the remaining spoken wording. It is a narrower job than freely rewriting a meeting into polished prose.

Supplying expected names or technical terms is another useful control. In a conversation about a service called Kestrel, the spelling can matter as much as recognizing the surrounding sentence.

I would treat that vocabulary as a hint, then check whether it improves recognition without inserting terms that were never spoken. That is a proposed evaluation, not a failure I observed in this release.

Does instruction following show up in the evidence?#

NVIDIA's technical report, Table 11, reports 7.56% default versus 6.16% instructed on Preference-ASR's preference-aware word error rate.

This test accounts for requested conventions such as casing, number formatting and filler removal. The instructed condition supplies an example-specific style request. NVIDIA also notes that its generic prompt differs from the original benchmark paper's prompt.

These are vendor-reported results, rather than a test I reproduced. They support investigating the controls; they do not establish that every instruction will be followed or that every important detail will survive.

What happened in outside tests?#

Cekura's evaluation, by Tarush Agarwal and updated October 29, reports 2.28% overall word error rate, lowest among its 21 tested systems. It combines 1,000 public clips with eight private recordings from four phone conversations. The private recordings are separate speaker tracks, not eight independent calls.

Nemotron ranked fifth on the public set and second on the private set. Close private-set competitors have overlapping uncertainty ranges, so I would avoid declaring a decisive universal winner.

Chart: Cekura, CC BY 4.0; reproduced without alterations.

Artificial Analysis's current non-streaming table lists 3.4% AA-WER for Nemotron 3.5 Transcribe through DeepInfra, checked October 30. Its methodology uses roughly eight hours across three differently weighted datasets.

That number cannot be compared directly with Cekura's 2.28%. The recordings and aggregation differ. I could not establish Cekura's Nemotron-specific serving configuration from the linked public repository; underlying hardware and quantization were not disclosed for the Artificial Analysis endpoint either.

There is another detail worth noticing. Word error rate counts substituted, inserted and missing words against a reference transcript. But the text is often normalized before counting.

Cekura's scoring removes fillers and standardizes formatting. Artificial Analysis also normalizes fillers and equivalent number forms. That makes recognition comparisons more useful, but it means these headline scores do not test whether “um” was preserved or whether numbers followed a requested style.

Recognition accuracy and adherence to a transcript convention need separate checks.

Can we see what speaker attribution looks like?#

NVIDIA's launch post includes a film-clip demonstration. At 01 in the original video, several differently labelled caption lines are visible together, with highlighted words.

This illustrates the output presentation. It does not independently verify the words, speaker assignments or processing latency. I could not verify an independent video test of this exact release in the sources checked.

Where would I be careful before building with it?#

First, this is an offline model for recorded audio, not a streaming replacement for the listening path of a live voice agent. The model card supports recordings up to one hour and lists 34B total parameters, with about 4.3B active. Active computation is not the same as the amount of weights that must be stored.

Then I would test the actual product contract. For deployment reviews, that might mean:

• Did it preserve “not,” corrections, conditions and numbers?
• Did the requested cleanup change only the allowed wording?
• Are unusual names and speaker changes correct?
• Can a reviewer jump from a disputed statement back to the audio?

I would keep the original recording, prompt, model version and transcript convention together. A cleaned transcript can serve search while a more literal version supports review. Neither should silently become a confirmed action just because it reads fluently.

So what is the bigger engineering change?#

What I find useful here is making a previously hidden output convention explicit.

Once a model accepts instructions about how speech should become text, the application has to define what those instructions permit. “Make this readable” leaves much more room for interpretation than “remove fillers, preserve corrections and keep all remaining wording.”

A better transcript is useful when its transformations fit the job. The engineering work is making those transformations visible, testable and traceable to the recording.