I Tried to Steal a Watermark Key Without Having It

Sixty documents. No key. No model. No detector API. The first lock fell. The second appeared to set a trap—and that was where the investigation nearly went wrong.

Incident brief

The attacker is not a cryptographer inside a model lab. It is an ordinary API customer—or anyone who can collect text that customers have already published.

The target is not necessarily the literal bytes of a secret key. That distinction matters. The operational target is the behavior induced by the key: enough knowledge of which token choices a detector rewards to reproduce the mark. If an attacker can clone that behavior, possession of the original key is optional.

The payoff is forgery.

Removing a watermark makes marked text resemble unmarked text. Forging one can make text the provider never generated resemble provider output. In a classroom, a newsroom, or an abuse investigation, that is the difference between destroying evidence and planting it.

Watermark stealing is not hypothetical. Jovanović, Staab, and Vechev demonstrated query-based stealing against multiple LLM watermark families, then used the approximated rules for spoofing and scrubbing; their ICML 2024 work reports attacks costing less than $50 in the tested setup.¹ My experiment is deliberately smaller: a transparent, stdlib-only reconstruction of the simplest leakage mechanism, with every secret known to the lab but hidden from the attack path.

The question: can published outputs leak enough of a watermark’s secret behavior to let an outsider manufacture a positive detection?

The scene

I built two toy targets inside WMTrace. Both are green-list watermarks. Before each token is sampled, a keyed function divides possible words into a favored green set and an unfavored red set. Generation biases the sampler toward green. Detection reconstructs the partition with the key and asks whether green words occur too often.

This family begins with the scheme introduced by Kirchenbauer and colleagues (KGW).² The detector’s simplified statistic is:

\[z = \frac{X-\gamma N}{\sqrt{N\gamma(1-\gamma)}}\]

where $N$ is the number of scored tokens, $X$ the green count, and $\gamma$ the null green fraction. Here $\gamma=0.25$.

The two locks differ in one detail:

Fixed-list / unigram: the same keyed green vocabulary is used at every position. Zhao and colleagues proposed this design for stronger robustness to edits.³
Contextual / KGW-style: the green set is recomputed from the previous two words. The word budget can be green after one prefix and red after another.

That detail changes the attack surface.

Attacker has	Attacker does not have
60 marked documents × 200 tokens	watermark key
a comparable clean reference corpus	detector or detector oracle
public scheme family and γ	model weights or logits
ordinary offline compute	generation API access during the attack

The 60 documents contain 12,000 marked tokens. The generator has a 128-word vocabulary and uses CRC32 as a reproducible toy pseudorandom function. This is a microscope slide, not a production model: small enough to inspect, large enough to expose the mechanism.

Reconnaissance: look for the fingerprint

Under a fixed list, a favored word is favored everywhere. If report is green, every marked document gives the sampler another opportunity to overproduce report. The key leaves a population-level fingerprint in plain sight.

The attack therefore contains no cryptanalysis:

Count each word in the marked corpus.
Count it in the clean corpus.
Rank words by relative-frequency lift.
Guess that the top 25% are green.

The target generator naturally uses the key to create the marked corpus. Once those documents are handed to the ranking stage, however, that stage receives only text. The key reappears afterward in a sealed evaluation function that scores guesses against ground truth; it is never used to rank candidates or serialized into the evidence bundle.

$ wmtrace attack steal --scheme unigram-toy --docs 60 --length 200
■ green-set estimation [E2] estimable
  scheme:    unigram-toy  (60 docs x 200 tokens)
  precision: 93.8%  chance: 25%  lift: 3.75x

Thirty of thirty-two guesses were correct for this key. The attacker had neither the key nor a detector response.

First breach

One successful key could be luck. I fixed twelve independent keys before the comparison and repeated the attack as the observed corpus grew.

Mean key recovery precision across twelve keys rises above 90 percent for the fixed unigram watermark while contextual recovery remains near the 25 percent chance baseline

Documents	Marked tokens	Fixed-list mean precision	Contextual mean precision
10	2,000	83.3%	16.7%
20	4,000	90.4%	23.3%
40	8,000	92.7%	25.8%
60	12,000	93.2%	23.2%
100	20,000	93.5%	23.7%

The fixed-list leak was visible after 2,000 tokens and plateaued above 90%. At 60 documents, the twelve-key range was 75–100%, compared with a 25% random baseline.

This is the apparent breakthrough in an intrusion report: persistence achieved, target behavior mapped, exploit repeatable. A fixed green list is a fingerprint smeared across every document it touches.

Then I attacked the second lock.

The second lock lies

The same frequency attack against the contextual scheme returned 9.4% precision on one run: only three correct guesses out of thirty-two.

That was more interesting than failure. Chance was 25%. A result at chance says there is no information. A result far below chance looks like an active anti-signal—as if the construction were steering the attacker away from the secret.

I had a mechanism ready. Frequent contexts inflate certain common words; those words dominate the ratio; context-dependent membership then makes their global labels unstable. Perhaps the attack reliably selected the wrong candidates.

It was elegant. It was security theater produced by a sample size of one.

The twist: the intruder was my narrative

Across the twelve preselected keys, contextual precision averaged 23.2%. The range was 9.4–34.4%. Some keys landed below chance and others above it. As the corpus grew from 4,000 to 20,000 marked tokens, the mean wandered around the 25% line instead of moving away from it.

The anti-signal disappeared because it had never existed.

Under the contextual scheme, budget is green only for a particular prefix. Across many varied prefixes, each word is green for roughly a γ fraction of its appearances regardless of the key. Marginal word frequencies therefore contain no stable, key-dependent label. The frequency attack sees sampling noise.

That is a narrower and stronger result than the exciting story:

not “the contextual key cannot be stolen”;
not “contextual schemes defeat spoofing”;
only “this first-order marginal attack found no recoverable signal.”

Jovanović and colleagues attack richer token and context behavior with learned models, demonstrating why that boundary matters.¹ Failure of one estimator is a bound on one attacker, not a security proof.

The false lead is part of the evidence. A single below-chance point invited a causal story; the key sweep falsified it. In a real investigation, that is the moment an analyst either reruns the experiment or ships a myth.

Weaponization: turn the estimate into a false positive

Recovery precision is a laboratory metric. The operational question is whether the estimate can cross the defender’s detector.

So I added the missing last stage. The attacker first recovers the top 32 words from 60 marked documents. A fresh 400-token document is then generated using only that estimated set. The generator receives no key and no detector feedback. After generation, the defender’s private detector scores the clean control and the forgery.

$ wmtrace attack steal --scheme unigram-toy --docs 60 --length 200 --forge
■ green-set estimation [E2] estimable
  precision: 93.8%  chance: 25%  lift: 3.75x

■ keyless forgery (private detector evaluation)
  clean:     z=0.46   no_scheme_evidence
  forged:    z=32.33  scheme_evidence  p=1.48e-198

The clean control scores z 0.46 and no scheme evidence, while the keyless forgery scores z 32.33 and scheme evidence with p 1.48 times 10 to the minus 198

The forgery also passed the detector’s safeguard against manufacturing evidence by repeating one token. Across the twelve fixed keys, 12 of 12 forgeries were accepted as scheme evidence; forged z-scores ranged from 21.7 to 34.6.

That is practical evidence of the security consequence: not merely “I guessed some words,” but “the private detector accepted a document produced without its key.” Related work calls this a spoofing attack, and has shown how spoofed or “piggybacked” watermarks can attach trusted-looking signals to harmful or false content.⁴

The result needs an equally visible limitation. This toy forger restricts itself to a 32-word estimated vocabulary, so its prose quality is poor. It proves detector compromise, not a production-quality impersonation. A capable attacker model can trade off fluency and watermark score; the ICML stealing work is the relevant evidence that this broader attack can be practical.¹

Why forgery is the worse failure

Removal and forgery are asymmetric.

Failure	Detector output	Immediate consequence
watermark removed	negative	marked text becomes indistinguishable from unmarked text
watermark forged	positive	unrelated text is made compatible with the victim’s keyed scheme

A negative result already has many explanations: human writing, an untested model, no watermark applied, too little text, or successful rewriting. It should never certify human authorship.

A forged positive is different. It can plant evidence. A p-value as small as $1.48\times10^{-198}$ does not rescue the attribution if the null model omits an adaptive attacker. The statistic says the text is extraordinarily unusual under an ordinary unmarked-text null; it does not say the provider authored the text.

This is why secret estimability belongs next to false-positive rate, quality, and robustness on every watermark security card.

The defender’s dilemma

The tempting engineering summary is:

Property	Fixed-list / unigram	Contextual / KGW-style
Isolated-edit synchronization	stronger	weaker; an edit changes later contexts
Marginal-frequency leakage in this lab	93.2% mean recovery	at chance
Keyless forgery demonstrated here	12/12 private detectors fooled	not tested

The fixed-list design was introduced for a real benefit: a substitution does not change the color of subsequent words.³ My separate weak-attack curve shows the contextual toy also retains evidence through moderate random editing, but loses signal as substitutions accumulate:

The toy watermark z-score declines under random word substitution and more slowly under cropping

Random substitution is not paraphrasing. DIPPER-style rewriting can replace the lexical surface while preserving much of the meaning and is a substantially stronger removal attack.⁵ “Watermarks in the Sand” gives conditional limits on simultaneously achieving broad applicability, low quality loss, and strong unremovability; it is a theorem under explicit assumptions, not a claim that every watermark is useless.⁶

The security lesson is not “always choose contextual.” It is:

Robustness to editing and resistance to secret estimation are separate objectives. Optimizing the first can quietly destroy the second.

The production stakes

Google’s SynthID-Text uses keyed, context-dependent tournament sampling rather than one permanent favored vocabulary. Its Nature paper reports a live experiment spanning nearly 20 million Gemini responses, with no statistically significant degradation in the measured user-feedback signal.⁷ That is important deployment evidence. It does not publish production keys or establish resistance to every stealing attack.

Anthropic has announced that future Claude models will use a version of SynthID-Text and states that detection requires its watermark key. It also says detection can establish likely Claude involvement but cannot distinguish “Claude wrote this” from “Claude heavily edited this.”⁸ That scope boundary becomes even more important under spoofing.

Nothing in this post measures Gemini or Claude. The targets are controlled toy schemes; CRC32 is not a cryptographic PRF; the vocabulary is 128 words rather than tens of thousands; and the reference corpus comes from the same toy generator. No detector should be named after a provider by guessing its key, tokenizer, or threshold.

But the production question is now concrete:

How much public output is required before an attacker can approximate enough of the production watermark to spoof its verifier?

A detection benchmark cannot answer that. It requires an estimability benchmark with an adaptive attacker, a held-out forgery stage, and private evaluation.

Evidence log and reproduction

Every number above is deterministic under the repository version and fixed seeds. The attack and detector evidence remain separate namespaces: estimating a key is not itself a watermark detection.

git clone https://github.com/gtesei/llm-watermark
cd llm-watermark
uv venv && . .venv/bin/activate
uv pip install -e ".[dev]"

# Single-key recovery
wmtrace attack steal --scheme unigram-toy --docs 60 --length 200

# Recovery followed by held-out keyless forgery
wmtrace attack steal --scheme unigram-toy --docs 60 --length 200 --forge

# Contextual negative control
wmtrace attack steal --scheme kgw-toy --docs 60 --length 200

# Editing and cropping controls
wmtrace attack robustness

# Reproduce the 12-key curve data (keys investigation-key-00 through -11)
python - <<'PY'
from statistics import mean
from wmtrace.attacks.stealing import estimate_green_set, forge_with_estimate
keys = [f"investigation-key-{i:02d}" for i in range(12)]
for n in (10, 20, 40, 60, 100):
    for scheme in ("unigram-toy", "kgw-toy"):
        row = [estimate_green_set(scheme, k, docs=n)["precision"] for k in keys]
        print(n, scheme, mean(row), min(row), max(row))
forged = [forge_with_estimate(k)["forged"] for k in keys]
print("accepted", sum(r["decision"] == "scheme_evidence" for r in forged), "of", len(keys))
print("forged z range", min(r["statistic"] for r in forged), max(r["statistic"] for r in forged))
PY

# Regression tests for the attack path and secret-handling boundary
pytest tests/test_attacks.py tests/test_robustness.py

What the experiment establishes

A fixed favored vocabulary leaks through aggregate token frequencies in this controlled setup.
The leak appears with a few thousand observed tokens and persists across keys.
The recovered behavior is sufficient to forge accepted positives against the private toy detector.
The same marginal attack remains at chance against the contextual toy.
Repetition across keys overturns the seductive below-chance false lead.

What it does not establish

Recovery of literal secret-key bytes.
A successful attack on any provider deployment.
Security of contextual schemes against richer estimators.
Fluent, production-quality forged prose from the toy 32-word forger.
Human authorship from any negative result.

Case closed, narrowly

The first target leaked because its secret behavior was reused at every token. The second seemed to retaliate, but the retaliation was noise. The real breach came when the recovered behavior crossed the private detector and turned an ordinary attacker into a forger.

The most important artifact is not the dramatic z-score. It is the sequence of controls around it: clean reference text, independent keys, a contextual negative control, a held-out forgery, and an explicit boundary between what the attacker sees and what the evaluator knows.

That is the real lesson from the investigation:

A watermark is not secure merely because detection works and removal is hard. It must also resist being learned from the evidence it leaves behind.

References

Nikola Jovanović, Robin Staab, and Martin Vechev, “Watermark Stealing in Large Language Models”, ICML 2024; code and attack overview. ↩ ↩² ↩³
John Kirchenbauer et al., “A Watermark for Large Language Models”, ICML 2023. ↩
Xuandong Zhao et al., “Provable Robust Watermarking for AI-Generated Text”, ICLR 2024. ↩ ↩²
Xiang Li et al., “No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices”, 2024. ↩
Kalpesh Krishna et al., “Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval Is an Effective Defense”, NeurIPS 2023. ↩
Xuandong Zhang et al., “Watermarks in the Sand: Impossibility of Strong Watermarking for Language Models”, ICML 2024. ↩
Pushmeet Kohli et al., “Scalable Watermarking for Identifying Large Language Model Outputs”, Nature 2024; official reference implementation. ↩
Anthropic, “Claude’s New Text Watermark”, accessed for this post. ↩