Invisible Unicode in AI Text: A Practical Guide to Finding and Removing It

Text that comes out of a large language model rarely arrives as clean ASCII. Between the model itself, the chat interface, the clipboard and the Markdown renderer, a surprising number of non-obvious codepoints can end up in what looks like an ordinary paragraph. Some are cosmetic. Some are deliberate provenance signals. Some silently break your build. This guide explains what actually shows up, how to detect it, and the order in which to remove it without damaging legitimate content.

1. Zero-width and invisible formatting characters

The classic offenders are U+200B (zero width space), U+200C and U+200D (the joiners), U+2060 (word joiner) and U+FEFF (byte order mark used mid-string). They render as nothing at all, which makes them ideal carriers: a sequence of joiners and non-joiners can encode arbitrary binary data inside otherwise innocuous prose. Practically, they also break string equality checks, `grep`, CSV imports and database unique constraints — two strings that look identical will not match.

Detection is straightforward because the set is small and well defined. Removal is safe: outside of a handful of scripts that use ZWJ for ligature control (Arabic, Indic scripts, emoji sequences), deleting them changes nothing visible. If you work with emoji or non-Latin scripts, review the report before accepting the cleaned output.

2. Exotic spaces

U+00A0 (no-break space), U+202F (narrow no-break space), the en/em quad family (U+2000–U+200A) and U+3000 (ideographic space) all look like a normal space but are not one. They arrive from rich-text pasting, from typographic post-processing in chat UIs, and from models that were trained on typeset text. Normalising them to U+0020 is almost always correct for plain-text workflows, and almost always wrong for typeset documents where a no-break space was intentional — check context before bulk-replacing.

3. Bidirectional control characters

U+202A–U+202E and U+2066–U+2069 change the direction in which text is rendered. In a right-to-left document they are legitimate. In an English paragraph or a source file they are a red flag: the "Trojan Source" class of attacks uses exactly these codepoints to make code display differently from how a compiler reads it. Unless you are genuinely mixing scripts, strip them.

4. Unicode tag characters and variation selectors

The tag block, U+E0000 to U+E007F, mirrors ASCII in a completely invisible plane. It was deprecated for language tagging and now survives mainly in flag emoji sequences — and in steganography demos, where an entire hidden message can be appended to a sentence without a single visible pixel changing. Variation selectors (U+FE00–U+FE0F and the U+E0100 supplement block) can carry data the same way. Neither belongs in ordinary prose.

5. Homoglyphs (confusables)

A Cyrillic "а" (U+0430) is visually identical to a Latin "a" but is a different character entirely. Substituting a handful of homoglyphs across a document creates a durable, invisible fingerprint that survives copy-paste and reformatting. It also breaks search, spell-check and domain validation, which is why the same trick powers phishing URLs. Normalising the common Cyrillic and Greek look-alikes back to ASCII is safe for English-language text; it is destructive for genuinely multilingual text, so treat it as an opt-in step and read the report.

6. Typographic tells

Em dashes, curly quotes and the single-character ellipsis are not watermarks. They are stylistic habits that many models share, and they are the most commonly cited "AI writing" signal precisely because they are so visible. Normalising them to ASCII equivalents makes text look more hand-typed and improves compatibility with legacy systems, but it is a cosmetic choice, not a security measure.

7. Markdown and HTML provenance artefacts

Beyond the character level, tooling adds visible-in-source markers: HTML comments naming the assistant, data-* attributes, <meta name="generator"> tags, utm_source=chatgpt.com parameters appended to every outbound link, trailing "Generated with…" lines, and Co-authored-by: trailers in commit messages. These are easy to miss because they live in source rather than in rendered output.

8. What character cleaning cannot touch

Statistical watermarking — the SynthID-class approach — biases which tokens a model chooses, spreading a signal across word choice itself. There is no character to delete. The only thing that perturbs it is substantially rewriting the text, which is a different operation with different ethics. Similarly, C2PA manifests in images and XMP metadata in PDFs are binary-container concerns. Any tool claiming to strip these from plain text is misrepresenting what it does.

A safe order of operations

  1. Keep the original. Always.
  2. Run a detection pass first and read the report before replacing anything.
  3. Remove pure-invisible categories (zero-width, bidi, tag, variation selectors).
  4. Normalise spaces.
  5. Opt into homoglyph normalisation only for single-script text.
  6. Apply typographic normalisation last — it is taste, not hygiene.
  7. Diff the result against the original before shipping.

You can run every step above in your browser on the cleaner page, or read the frequently asked questions.