Attacks on Text Watermarks: Robustness in Practice

First, what a text watermark depends on

As covered in our SynthID deep dive, a statistical text watermark biases token selection using a key, and detection measures the accumulated signal across positions. Two facts drive every attack: the signal lives in the specific tokens chosen, and it scales with length. Change enough tokens, or shorten the text enough, and the detector loses its footing.

The chart is directional, drawn from the qualitative pattern reported across the literature rather than one benchmark, but the ranking is stable: the more you rewrite the token sequence, the less signal survives.

The attack catalog

Light editing. Fixing typos, adjusting punctuation, and reformatting change few tokens, so detection barely moves. This is why the watermark is called robust to casual tampering.

Synonym substitution. Swapping individual words shifts some token outcomes, but because the keyed partition re-seeds at each position, the overall statistic degrades only gradually. Manual light editing is a weak attack.

Truncation. Detection power scales with the number of scored positions. Halve the length and you roughly halve the evidence, which is why short passages are unreliable to begin with and cropping is a real attack.

Paraphrase. The strongest general attack. Re-generating the text, especially with a different model, rewrites the token sequence wholesale and washes most of the signal out. This is exactly what a humanizer does, and it is why we describe paraphrase as reducing signal without guaranteeing anything.

Translation round-trip. Translating out and back through another language forces a full re-tokenisation and tends to be as destructive as paraphrase, at the cost of meaning drift.

Mixing and splicing. Interleaving human written sentences dilutes the watermarked fraction, lowering the mean score below the detection threshold even when some watermarked text remains.

Why there is no key-free removal guarantee

Notice what none of these attacks require: the key. You do not need to know the partition to reduce the signal, you only need to change enough tokens. But the flip side is just as important. Without the key you cannot confirm you have removed the watermark, because you cannot run the real detector. You are reducing an invisible signal blind. That asymmetry, easy to weaken, impossible to verify from outside, is the honest core of watermark robustness, and it is why "guaranteed undetectable" is a claim no one can back.

Measuring an attack empirically

If you want to reason about robustness quantitatively rather than hand wave, the setup is straightforward in principle. Generate watermarked text, apply an attack, and compare the detector's score before and after at a fixed false positive rate. The metric that matters is the detection rate that survives, and the honest way to report it is as a curve across attack strengths, not a single number. The pseudo code below sketches the measurement loop; the real work is in having a calibrated detector, which, as we keep stressing, requires the key.

class="syn-keyword">def robustness(samples, attack, detect, threshold):
    survived = 0
    class="syn-keyword">for text class="syn-keyword">in samples:
        attacked = attack(text)          # paraphrase, truncate, etc.
        score = detect(attacked)         # needs the keyed detector
        class="syn-keyword">if score >= threshold:           # still flagged as watermarked
            survived += 1
    class="syn-keyword">return survived / len(samples)       # detection rate after attack

Run this across attacks and thresholds and you reconstruct the qualitative picture from the chart above: light edits barely move the rate, paraphrase collapses it.

Why the threshold is a policy choice

The detection threshold is not a technical constant; it is a value judgement encoded as a number. Set it low and you catch more watermarked text but flag more clean text as watermarked, a false positive. Set it high and you miss more evasions. There is no setting that is correct in the abstract, because the right trade depends on the cost of each error in your context. A plagiarism process and a spam filter should not use the same threshold, and a tool that hides this choice behind a single confident percentage is obscuring the most important decision in the whole pipeline.

Using this knowledge responsibly

Cataloging attacks is not the same as endorsing evasion. The reason to understand robustness is to build honest tools and to calibrate honest expectations, on both sides. If you build detection, this tells you where your confidence should drop. If you build removal, it tells you to describe your tool truthfully: paraphrase reduces signal, it does not guarantee removal, and you cannot even verify success without a key you do not have. The engineering is genuinely interesting, and the honesty is what makes it trustworthy.

What the defenders are doing

Watermark designers are not standing still. Production schemes tune the bias to stay robust while non-distortionary, and research on detection, including goodness-of-fit approaches, squeezes more power from the same signal. Regulation adds pressure to keep content marked. The realistic outcome is an equilibrium, covered in our policy piece on the arms race: casual content stays detectable, determined paraphrase still escapes, and length remains the watermark's best friend.

Practical implications for builders

If you build detection tooling, calibrate expectations to length and attack model, and never report a confident score you cannot defend. If you build removal or humanizing tooling, describe it honestly: paraphrase reduces signal, hidden-character and metadata stripping remove specific markers, and none of it is a guarantee. The engineering is interesting precisely because the limits are real, and a tool that respects them earns trust that an overpromising one burns.

A worked scenario

Suppose you have a watermarked essay and you want to understand, as an engineer, why different edits succeed or fail. Fix a typo and the token sequence barely changes, so a keyed detector still sees almost all its evidence, detection holds. Swap ten synonyms and a handful of positions flip, nudging the score down but rarely below threshold on a long text. Cut the essay to a third of its length and you remove two thirds of the evidence, which can drop a borderline case under the line. Run the whole thing through a different model and the token sequence is rewritten wholesale, so the signal largely disappears. Each outcome follows directly from the same rule: the signal is in the tokens and scales with length. Once you internalise that, no attack result is surprising.

Frequently asked questions

Can you remove a text watermark without the key? You can reduce it by changing enough tokens, mainly via paraphrase, but you cannot confirm removal without the key, because you cannot run the real detector.

Which attack is strongest? Full paraphrase, especially through a different model, because it rewrites the token sequence the signal depends on. Translation round trips are similarly destructive.

Are short texts safer from detection? Detection is unreliable on short texts because there are too few positions to accumulate a signal, which is also why truncation is an effective attack.

Keep reading: How SynthID works · Build a C2PA inspector · Text API reference