Unicode normalization and accent stripping explained

You paste a name like "José" into a search field, and it doesn't match another instance of "José" in your data—even though they look identical on screen. You export a list to a URL slug and get two different byte sequences from what should be the same word. The culprit is Unicode normalization, and understanding it is essential for text processing, data matching, and character encoding work.

What is Unicode normalization?

Unicode allows the same visual character to be represented in more than one way. The letter "é" can exist as a single precomposed character (U+00E9) or as a base letter "e" (U+0065) followed by a combining acute accent (U+0301). Both render identically on screen, but they have different byte sequences. Normalization converts these variants into a canonical form so they compare and sort consistently.

NFC vs NFD: the two common forms

The two normalization forms you'll encounter are:

• NFC (Canonical Decomposition, followed by Canonical Composition) — combines base characters with their accents into single precomposed characters. This is the W3C default and the most common form on the web.
• NFD (Canonical Decomposition) — breaks precomposed characters into their base character plus combining marks. Useful when you need to manipulate or inspect accents separately.

There are also NFKC and NFKD (compatibility variants) that go further, converting ligatures and style variants into their base equivalents—useful when you want "fi" to match "fi".

Why this matters in practice

String comparison depends on exact byte matching unless you normalize first. A database search for "café" in NFC form won't find "café" stored in NFD form. URLs containing accented characters can be interpreted differently by different systems. Sorting algorithms produce different orders if some entries are NFC and others NFD. Any text pipeline that combines data from multiple sources—API responses, user uploads, legacy databases—risks silent mismatches.

Normalizing and removing accents for slugs and search

Web addresses and database keys typically can't contain accented characters or combining marks. The standard approach is to normalize to NFC (or sometimes NFD, then remove the combining characters), then strip accents entirely using character decomposition. This converts "naïveté" to "naivete" for slug-safe output. Use normalize-unicode to see how both NFC and NFD look, or remove-accents to strip them in one step. For maximum compatibility, remove-non-ascii goes further and removes any character outside the ASCII range, leaving only a-z, A-Z, and basic punctuation.

Handling Unicode in your workflow

Before you search, compare, or sort text in an application:

1. Normalize all input to NFC (or NFD if you have a specific reason).
2. Store in that same form consistently.
3. Compare, search, and sort after normalization.

When building URLs or database identifiers, first normalize, then remove accents and non-ASCII characters. Understanding what each character actually is—its code point, combining marks, category—helps debug these issues. char-code-converter shows you the exact Unicode values and names behind each character.

Privacy-first text processing

All TextArray tools, including Unicode and accent handling, run entirely in your browser. Text never leaves your device, no accounts are required, and the site works offline after the initial load. You can safely paste sensitive data, experiment with normalization, and strip accents without uploading anything to a server. This is especially valuable when working with private names, addresses, or identifiers that need cleaning before use.

Getting started

Copy a string with accents into any of our text tools and see normalization and accent stripping in action. Experiment with NFC vs NFD, compare code points side by side, and build confidence in the characters you're working with. Once you understand how Unicode works under the hood, text matching and slug generation become predictable and reliable.