WMTrace

Forensic detection and benchmarking of LLM text watermarks

View the Project on GitHub gtesei/llm-watermark

Fast-DetectGPT, Binoculars, and rewrite robustness

See Installing local models for pinned Hugging Face download commands, expected storage, and bundle staging.

D1 Fast-DetectGPT

WMTrace implements the analytic conditional-curvature statistic from the MIT reference at commit 971b05202bac2bb504d60c0ac0812fea7a8f7c82:

(sum observed scoring-model log p - sum sampling-model expected log p)
/ sqrt(sum sampling-model variance of scoring-model log p)

A local bundle must contain aligned sampling and scoring model directories, checksummed by its manifest. Configure their relative paths as sampling_dir and scoring_dir. Token IDs and vocabulary shapes must match. The upstream demo probability and threshold are never reused.

D2 Binoculars

WMTrace implements the BSD-3-Clause reference formula at commit c8ae2f90d50ee696418bc71d8d9e5020e5f9d7b8:

mean performer token NLL
/ mean cross-entropy(observer distribution, performer distribution)

A local bundle uses observer_dir and performer_dir. Tokenizers and vocabulary shapes must match. The published Falcon thresholds are recorded as upstream context only and are never defaults for another pair.

Both adapters use exact continuation-token ownership across overlapping model windows, local-only Transformers loading, immutable checksums, empirical-null calibration, and language/domain/length gates. They emit E0 without assets, E1 for short text, E2 without a matching calibration, and E3 only at a valid local operating point.

A1-A3 robustness evaluators

wmtrace.attacks.transformations.RobustnessEvaluator supports:

A rewriter backend must return its immutable bundle ID, prompt ID, output, language metadata, intermediate translation where applicable, and a semantic similarity measurement. Quality gates enforce semantic similarity, number and named-entity retention, operation-specific length targets, and intermediate retention. Failed outputs are rejected and counted. Accepted and rejected records remain attack_evidence; transformations are never run by normal wmtrace scan.

No rewriter model is bundled. The web API reports A1-A3 as backend_required until a licensed local backend is staged.

Open publication gates

The adapters and evaluator contracts are implemented, but milestone release claims remain blocked until the repository publishes: