Unicode vs UTF-8 vs UTF-16: What Web Developers Actually Need to Know

Every web developer eventually hits an encoding bug — the garbled "é" where "é" should be, or the mysterious question mark that appears in a database column. The root cause is almost always a confusion between Unicode, UTF-8, and UTF-16. These three terms appear together so often they start to blur into one concept. They are not the same thing.

Unicode Is a Standard, Not an Encoding

Unicode is a character inventory — a giant numbered list that assigns a unique integer (called a code point) to every character in every writing system on Earth, plus emoji, mathematical symbols, and private-use areas.

The Euro sign € is code point U+20AC. The Latin letter "A" is U+0041. The pile-of-poo emoji 💩 is U+1F4A9. As of Unicode 15.1 (2023), the standard defines 149,813 assigned characters out of a possible 1,114,112 slots.

The important distinction: Unicode says nothing about how those integers are stored as bytes. That is what an encoding does. UTF-8 and UTF-16 are two different answers to the question "how do we write these numbers to disk or across a network?"

UTF-8: Variable-Width and Backward-Compatible

UTF-8 encodes each code point using 1 to 4 bytes depending on the code point's value:

| Code point range | Bytes | Example | |---|---|---| | U+0000–U+007F | 1 | "A" → 0x41 | | U+0080–U+07FF | 2 | "é" → 0xC3 0xA9 | | U+0800–U+FFFF | 3 | "€" → 0xE2 0x82 0xAC | | U+10000–U+10FFFF | 4 | "💩" → 0xF0 0x9F 0x92 0xA9 |

The single-byte range is identical to ASCII. That backward-compatibility is why UTF-8 won the web: existing ASCII documents are valid UTF-8 with no modification. According to W3Techs' 2024 web technology survey, 97.9% of websites declare UTF-8 as their character encoding — the remaining 2.1% are mostly legacy Latin-1 and Windows-1252 pages that predate the UTF-8 consensus.

Because the 1-byte slot is reserved entirely for ASCII (U+0000–U+007F), a UTF-8 multi-byte sequence can never contain a byte below 0x80. This makes it self-synchronizing: if you land in the middle of a stream, you can always find a safe character boundary by scanning forward for a byte that does not start with 10xxxxxx.

UTF-16: Two Bytes for Most Characters, Four for the Rest

UTF-16 takes a different approach: every character in the Basic Multilingual Plane (BMP, U+0000–U+FFFF) uses exactly 2 bytes. Characters above U+FFFF — which includes most emoji and many rarely-used CJK extension characters — need a 4-byte surrogate pair.

Compare the two encodings for a few specific characters:

Euro sign (U+20AC, inside the BMP):

• UTF-8: 0xE2 0x82 0xAC (3 bytes)
• UTF-16 LE: 0xAC 0x20 (2 bytes)

Grinning face emoji (U+1F600, outside the BMP):

• UTF-8: 0xF0 0x9F 0x98 0x80 (4 bytes)
• UTF-16 LE: 0xD8 0x3D 0xDE 0x00 (4 bytes — surrogate pair)

UTF-16 also needs a byte order mark (BOM) — U+FEFF at the start of a file — to tell readers whether the bytes are big-endian or little-endian. That BOM adds overhead and breaks naive text concatenation.

Where you encounter UTF-16 in practice: the Windows file system API, Java's String type, JavaScript's internal string representation, and legacy Microsoft Office XML formats. The Windows Notepad "Unicode" save option has historically produced UTF-16 LE with BOM, which is why .txt files saved on Windows can confuse Unix tools.

A Real Encoding Bug: Café Becomes Café

I reproduced this classic bug deliberately. I created a UTF-8 encoded file containing the word café and then read it with a Python script that assumed Windows-1252 (Latin-1) encoding:

Input file bytes (hex): 63 61 66 C3 A9

That is five bytes: c, a, f, then the two-byte UTF-8 sequence for "é" (C3 A9).

Output under incorrect Windows-1252 read: café

Windows-1252 treats each byte independently. 0xC3 maps to "Ã" and 0xA9 maps to "©". The bytes are untouched — only the interpretation changed, and two reasonable-looking characters replaced one intended one.

This class of bug is called mojibake (文字化け, Japanese for "character transformation"). It is entirely preventable by declaring encoding explicitly at every boundary: the HTTP Content-Type: text/html; charset=utf-8 header, the HTML <meta charset="utf-8"> tag, the database column collation, and the file write mode in your code.

When UTF-16 Breaks JavaScript String Length

JavaScript's .length property counts UTF-16 code units, not Unicode characters. For BMP characters, one code unit equals one character. For emoji and characters above U+FFFF, one character takes two code units — so .length returns 2.

"café".length    // 4 — correct
"😀".length      // 2 — misleading
[..."😀"].length // 1 — correct (spread iterates code points)

I ran this in Node.js 22.4 and confirmed both results. The spread operator and Array.from() use JavaScript's iterator protocol, which walks by Unicode code point rather than UTF-16 code unit. If you are building a character counter, a tweet validator, or a field-length check that faces international users, you need code-point iteration, not .length.

The same trap appears in older Java code where String.length() also returns the number of UTF-16 code units. A single emoji counts as 2. Java's codePointCount() method gives the correct Unicode character count.

Practical Tools for Inspecting Encoding in the Browser

Understanding the theory only gets you so far. When a string is misbehaving in production, you need a fast way to see the actual code points and byte sequences without spinning up a Python REPL.

The Unicode Character Inspector breaks any string into its individual code points, shows the Unicode name and block for each character, and displays the UTF-8 bytes in hex. Paste café and you instantly see U+0063 (LATIN SMALL LETTER C), U+0061, U+0066, and U+00E9 (LATIN SMALL LETTER E WITH ACUTE) with byte sequence 63 61 66 C3 A9 — exactly the bytes that cause mojibake when misread.

For counting how many bytes a string occupies in UTF-8 (as opposed to the .length value JavaScript gives you), the UTF-8 Byte Counter gives the precise byte size. This is directly useful when writing to fixed-width database columns, building APIs with byte-length limits, or sizing HTTP bodies against a content-length budget.

If your source contains Unicode escape sequences like € or \u{1F600} and you want to convert them to literal characters — or go the other direction — the Unicode Escape Converter handles both directions client-side with no server round-trip.

Three Lines That Summarize Everything

• Unicode assigns a number to every character. It does not define storage format.
• UTF-8 stores those numbers in 1–4 bytes per character, is 100% ASCII-compatible, and is what ~98% of the web uses.
• UTF-16 stores BMP characters in 2 bytes but needs surrogate pairs above U+FFFF; it dominates Windows APIs and Java runtimes.

When an encoding bug surfaces, check three places in sequence: the Content-Type header the server sends, the <meta charset> tag in the HTML, and the encoding declared on the database column or file stream. All three must agree on UTF-8, and most bugs disappear.

---

Made by Toolora · Updated 2026-06-28