Can an AI API Prove Which Model Answered You?

Imagine choosing a particular model for an application, running your evaluations, and sending production requests through a hosted API.

The responses look reasonable. The request says the right model name. But what evidence do you have that those responses came from the model you evaluated?

A provider's label is useful information. It is still something you have to trust.

That is the problem behind FlashProve, an open-source research prototype Anthropic announced on November 2, 2026. Its repository's initial public-release commit carries an October 30 timestamp. The work explores attaching cryptographic evidence to model computation without publishing the model's weights.

What interests me is the possibility of checking one very specific claim: does this output satisfy the promised computation?

The details of that promise matter quite a lot.

Why can't we just ask the model which model it is?#

Because the answer is another piece of generated text.

A system can produce the expected name without establishing anything about the weights behind it. Similarly, a good answer to a benchmark question tells us something about behavior, but does not uniquely identify the system that produced it.

Consider an illustrative document-processing service. Its developer evaluates model A, then buys access through a routing layer. If the routing layer uses model B, the output might remain plausible while the developer's evaluation no longer describes the deployed system.

This is a hypothetical example, not an incident alleged against a provider.

The desired check is about execution provenance. Whether the output is useful, factual or safe remains a separate question.

What would a cryptographic receipt actually contain?#

FlashProve's execution guide separates the job into commitment, proving and verification.

First, the provider publishes a commitment associated with its weights and configuration. Think of this as a sealed reference to the computation that later claims must be checked against.

The provider then produces a proof for the checked token positions. A client runs a verifier using the public information, the request and the output, without needing the weights.

The useful analogy is a receipt with a checkable mathematical claim attached. Unlike an ordinary receipt, the intended guarantee comes from a proof protocol rather than the seller's signature alone.

This uses zero-knowledge proof techniques. The goal is to make the computation checkable while protecting secret weights. It does not mean prompts are automatically hidden from the provider, and the released system has explicitly documented limits on weight hiding.

Does the commitment tell us whose model it is?#

This is an easy detail to miss.

The known-limits guide says a successful check does not identify whose weights were committed. A provider can commit to a different model.

So the client still needs a trusted association between a model identity and its commitment. Otherwise, it could carefully verify the wrong model.

For the document-service example, I would want to record the expected commitment and configuration alongside the evaluation version. Accepting whatever commitment arrives with today's response would leave that identity question unresolved.

Verification only helps when the thing being verified matches the requirement.

What did Anthropic actually measure?#

The published performance guide describes experiments using an integer model based on a public 4-bit conversion of Llama 4 Maverick, with 1,024-token windows.

These are author-reported measurements, not results I reproduced:

The crucial qualification is that the headline timed proof processes the prompt and checks one next output token. The rate counts prompt tokens. It is not a verified conversational output rate, and it excludes startup, commitment generation and subsequent verification.

The guide also describes whole-reply checks under a looser policy, with different conditions. Those should not be folded into the headline as though they were the same experiment.

The original cost table in the technical report, PDF page 41, preserves those distinctions. A rendered copy is included in the evidence archive.

Is proving still much more expensive than inference?#

Yes, in the reported comparison.

The technical report's Section 7.8 gives approximately 71× to 93× the GPU time of its stock-vLLM prefill baselines, normalized per GPU.

That baseline uses an FP8 checkpoint on eight H100s on other machines; FlashProve uses its integer-model setup. It is a comparison with an ordinary serving system, not an experiment holding arithmetic and hardware allocation identical.

Those numbers make this interesting research rather than an obvious feature to add to every chat request today. Proof bandwidth, batching delays and verification work also enter the practical cost.

I would assess whether provenance is valuable enough for a particular workload before treating throughput as the only adoption question.

Does a proof cover the model I already serve?#

Not automatically.

The report distinguishes its integer model from normal floating-point inference. Under an exact-match policy, the checked output must agree with that integer model's greedy choice: the token with the highest score.

The repository also supports a looser clipped-gap policy, which allows bounded differences in token scores. Its guide explicitly distinguishes an exact acceptance from an output that is merely within the permitted limits.

That changes the meaning of the receipt. “Matches this integer model exactly” and “meets this configured tolerance” are different claims.

An application needs to know which policy it expected, which token positions were checked, and how its text became token IDs. A green verification result cannot supply those missing expectations afterward.

If Lean checked the mathematics, is the software verified too?#

The project's formal-proof documentation draws this boundary explicitly.

Lean is a proof assistant used to check mathematical statements about the protocol under stated assumptions. The fast C++ verifier and CUDA prover are tested implementations; they are not themselves proved equivalent to the formal definitions.

That is meaningful progress, with an implementation gap still to inspect. The repository also labels itself an unmaintained research artifact.

For engineering purposes, I would distinguish the protocol's theorem, the actual verifier binary, the configuration and the surrounding API. Each contributes a different part of the trust story.

What happens when independent researchers try it?#

As of November 3, I could not verify an independent reproduction of this release's performance or an independent test video in the sources checked. I also did not find an original demonstration video I could inspect and timestamp reliably.

The released repository provides code, formal material and a stored example proof, which gives reviewers something concrete to examine. Its documentation says the small stored example uses synthetic weights. Passing that example would not reproduce the full-model result.

I have not built the verifier or run the model. The evidence here is the release, original documentation, report and my interpretation of their boundaries.

So what would I carry into an application design?#

I would separate three questions: which computation produced the tokens, whether the answer is correct, and whether the application is authorized to act on it.

FlashProve explores a way to make the first question checkable. It does not resolve the other two.

That distinction is what makes the work useful to think about. We can ask a hosted system for more precise evidence than a model-name field, while still being explicit about everything that evidence leaves unproved.