How it works

Nothing here trusts what a model says about itself. What cannot be faked cheaply is the serving stack's own bookkeeping: the model id on the response, the id named in a validation error, the tokenizer's count for a fixed string, the output cap the API enforces. Those decide the verdicts. A model's claims about itself are kept as quotes for you to read, and are scored in exactly one case — when they contradict the vendor being advertised.

• What the serving stack says about itselfNear-proof

  The model id on the response, an id named in a validation error, an upstream name in a header a gateway forgot to strip. A model cannot be prompted into faking these — they are written by the software that routed your request. Everything decisive here comes from this tier.

• Measurable behaviourReal but conditional

  Tokenizer counts for a fixed string, the output cap the API actually enforces, whether the same prompt returns the same text at temperature 0. Genuine evidence, but only where the published limits are exact and there is a reference to compare against.

• What the model says about itselfNever proof

  A reply of “I am Claude Opus 5” is one line of system prompt away for any model on earth. Self-reports are recorded verbatim as quotes for you to read, and scored only when they contradict the advertised vendor — the one direction a reseller would never prompt for.

1. 1
   Three identical runs at temperature 0The same five-item probe is sent three times with temperature 0 and a 512-token cap. Fixing the prompt makes the runs comparable to each other and to every other model on the same endpoint; three of them separate a stable backend from a load balancer spreading requests across different models.
2. 2
   The tokenizer is measured against its own cohortA fixed string mixing scripts, contractions and rare substrings is sent to count_tokens for each selected model. The score is the median of the cohort measured on this same endpoint — never a hardcoded number, because an absolute token count means nothing without a reference taken through the same gateway. Select at least two models to make this check scorable.
3. 3
   The output ceiling is probed, not assumedOne request asks for an absurd max_tokens (9,000,000). Anything that validates rejects it and usually states both the real cap and the resolved model id outright. When the error is terser, the ceiling is found by binary search over the caps real models publish.
4. 4
   A model id that does not exist is requestedAny honest API returns 404 or a validation error for an id it does not serve. A provider that answers not-a-real-model-9v9-20991231 with a normal completion is not routing on the model field at all — every id it advertises lands on the same backend.
5. 5
   The billed request is compared against the counted onecount_tokens tokenizes exactly the messages you send; the completion bills what the backend actually processed. A gateway that prepends a persona (“you are Opus 5, never say otherwise”) makes the second number larger, and the gap is the size of the injection.
6. 6
   The identity question is asked again, sidewaysA second request asks which company trained it, what its family and version number are, and asks it to complete “the underlying model serving this request is ___”. Phrasings a spoofing system prompt tends not to anticipate. Whatever comes back is quoted for you verbatim and never converted into a claim — if the answers name more than one vendor across runs, that inconsistency is itself reported as a finding.
7. 7
   Optionally, its “random” numbers are sampled and comparedWhen Deep fingerprint is switched on, each model is asked to “pick a random integer between 1 and 100” roughly forty times. LLMs are biased, model-specific random generators — a bias baked into the weights that a system prompt cannot rewrite. Each model’s distribution is compared, using Jensen–Shannon divergence, against the median of the cohort on this same endpoint. The cutoff is not a fixed number: it is learned from how much the cohort’s own members scatter, so a small sample cannot trip a false accusation.
8. 8
   Only contradictions are scoredEach check becomes a signal with a weight, skipped rather than guessed whenever the endpoint gave away too little. The verdict is then aggregated from the weights — a named substitution overrides everything, because a working model that is not the one sold is still not the one sold.

• Model the endpoint servedHard

  The id the serving stack itself reported, taken from the response model field, a validation error, or a header — in that order of trust. This is the closest thing to proof the tool can get. An endpoint that simply echoes back the id you requested is treated as having said nothing at all, not as a pass.

• Tokenizer fingerprintHard

  Models from one vendor generation share a tokenizer, so genuine siblings return the same count for the same text. More than 5% away from the cohort median on this endpoint means a different tokenizer, and so a different model family.

• Enforced output ceilingHard

  Scored only where the vendor rejects an over-cap max_tokens instead of clamping it — Anthropic today. Everywhere else the measured number is shown but never counted, because a clamping API returns the same answer whatever model is behind it.

• Arithmetic, letter counting, code generationHard

  Three questions with exactly one right answer each (17 × 24, the r’s in “strawberry”, the one-line Python sum of 1–100), chosen because they can be graded without a reference model. Downgraded to soft for small tiers and for ids not in the registry: a genuine Haiku really does miscount the r’s, and failing one for that would be a false accusation.

• Exact instruction followingSoft

  Whether every run replied with exactly ORANGE where the probe demanded that single word and nothing else.

• Response id shapeSoft

  Scored only where the vendor publishes a shape its own API emits — Anthropic’s msg_ prefix. A gateway that mints its own ids is not evidence either way, which is why this is weighed lightly.

• Run-to-run consistencySoft

  Three different answers to the same prompt at temperature 0 means the requests are not all reaching the same backend.

• Random-number distributionSoft

  Opt-in, and off by default because it costs about forty extra requests per model. A model’s bias when picking a “random” 1–100 lives in its weights, so it survives any system prompt. The sampled distribution is scored by Jensen–Shannon divergence against the cohort median, using a threshold learned from the cohort’s own spread. Soft on purpose — a distribution is suggestive, not a named substitution, and it needs 30+ parsed samples and a two-model cohort before it scores at all.

• Vendor it names for itselfSoft

  Recorded always, scored in one direction only. Naming the advertised vendor proves nothing — that is exactly what a reseller’s system prompt guarantees. Naming a different one is a real lead.

• Endpoint routes on the model nameContext

  A serious finding about the provider, and deliberately never scored against an individual model: an endpoint that ignores model names may still have answered this particular request with the right one.

• Hidden system promptContext

  The gap between billed input tokens and counted input tokens. A benign safety preamble has the same shape as a spoofing one, so this is shown, not scored — and a clean result is not proof either, since a gateway that padded its count_tokens by the same amount it injected would hide the gap entirely.

• Matches what was sold

  Every check the endpoint allowed is consistent with the model you asked for.

  Every scored check passed and none partially failed. Confidence rises with how many checks the endpoint allowed.

• Likely as advertised

  Mostly consistent with the advertised model, with something unexplained.

  At least 70% of the scored checks passed, with something left unexplained.

• Inconclusive

  The endpoint gave away too little to say either way.

  Nothing could be scored at all, or the pass ratio landed between 40% and 70%. Not a clean bill of health — the endpoint simply revealed too little.

• Likely not as advertised

  A verifiable contradiction with the advertised model.

  One hard check failed, or fewer than 40% of the scored checks passed. One verifiable contradiction is enough to doubt the label.

• Not the advertised model

  Several verifiable contradictions — this is not the model advertised.

  Two or more hard checks failed.

• Wrong model tier

  A real model from the right vendor, but a different tier than the one sold.

  The endpoint named a model from the advertised vendor but a different family. This overrides the pass ratio — passing behaviour cannot average away a named substitution.

• Different vendor entirely

  Answered by a different company's model than the id claims.

  The endpoint named a model belonging to another company. The highest-confidence verdict the tool issues, and the only one ranked above a wrong tier.

• Could not verify

  The probe could not complete, so nothing was established.

  No run returned any text, so nothing was established either way.

• A well-built proxy produces “inconclusive”, not “mismatch”

  A gateway that rewrites the response model field to the id you asked for, mints its own message ids, pads its count_tokens response, and forwards to a same-vendor sibling defeats every hard check at once. The result is Inconclusive, which is the honest answer — but read it as “nothing was proved”, never as “verified genuine”.

• Two checks need the endpoint to implement count_tokens

  When /v1/messages/count_tokens returns 404, both the tokenizer fingerprint and the hidden-prompt comparison are skipped. The tokenizer check also needs at least two models selected on the same endpoint, since the baseline is that cohort’s median.

• The output ceiling is only scorable for exact-cap vendors

  Anthropic rejects a max_tokens above the model’s cap; most other vendors clamp it, version their caps quietly, or differ per deployment. For those the measured number is reported and left unscored, so a ceiling mismatch cannot be used against them.

• The puzzles separate capability tiers, not vendors

  Any current flagship answers all three correctly, so passing them does not identify a vendor — a substituted sibling of comparable strength passes exactly as the advertised model does. They catch a cheap model behind an expensive label, not a same-tier swap.

• The random-number fingerprint is statistical, and only relative

  It compares each model against its cohort on the same endpoint, so it needs two or more models selected and 30+ parsed samples before it scores; a lone model, or a provider that serves one backend for every id, leaves it with nothing to contrast against. Because roughly forty samples is a small sample, it is scored softly against a threshold learned from the cohort’s own scatter rather than a fixed cutoff — it can corroborate a substitution but never establish one by itself.

• A pass is a statement about one moment

  Every verdict describes the requests that were actually sent, from one IP, at one time. A provider can route a percentage of traffic, or route differently under load or by customer. Re-run the check periodically rather than treating one clean result as permanent.

• The model table is hand-maintained

  Advertised context windows, output caps and vendor mappings come from a registry in this repository (version 2026.08.12). A model released after that date can be reported as an unrecognised id, which downgrades the puzzles to unscored and leaves nothing to compare a ceiling against.

Every run can be exported as a receipt: the verdict, every signal with its status and weight, the raw evidence behind it, and the model-table version that produced it, in Markdown with embedded JSON. It records the endpoint's host only — never your API key — so it can be attached to a support ticket or a dispute as-is. Your key is proxied server-side and is written to disk only if you explicitly ask this tool to generate a CLI config for you.