Skip to content
All projects

Undergraduate research · LSAMP research program · Texas A&M · Spring 2025

LLM watermark detection

Research on statistical watermarks for language models: bias generation toward a secret, pseudo-random “green list” of words, then detect the bias later with a hypothesis test.

Engineering

Outcome

An animated detector that shows how much evidence a watermark leaves, and how edits wash it out.

Topic

Watermarking language model output

Methods

Hard and soft green-list biasing, z-score detection

Based on

Kirchenbauer et al., 2023

Lab

Toy vocabulary, bigram model

RoleUndergraduate researcher.

How the watermark works

Before each word is generated, the previous word is hashed to seed a random split of the vocabulary into a green list and a red list. A hard watermark only picks green words. A soft watermark adds a bonus to green words, so it gives way when one word is clearly right, such as the second half of a name.

Detection

Anyone who knows the hashing rule can recount the green words without access to the model. Human text lands near the expected share, γ, and watermarked text lands well above it. A one-proportion z-test turns the gap into a p-value: z = (greens − γT) / √(T·γ(1−γ)).

What I studied

  • Hard versus soft biasing, and how the bias strength δ trades text quality for detectability
  • How many tokens are needed before detection is statistically meaningful
  • How edits and paraphrasing dilute the signal

About the demo

The detector runs on a teaching rebuild with a small vocabulary and a bigram model. It is not a real language model and not my research code. The method follows the published paper, and the human-written sample is President Kennedy's 1962 speech at Rice University.

Demo

Watermark detector

Watch every word get checked against a secret green list, and the z-score decide whether the text carries the watermark.

Loading demo…