The Complete Technical Guide to Fixing Mojibake & Encoding Errors
Few things in software development, data migration, and global web publishing are more alarming than seeing perfectly written Arabic text transform into a jumble of symbols like العربية, or European text corrupted with é and ’. This phenomenon is known worldwide in computer science as Mojibake (Japanese: 文字化け, literally "character mutation").
1. Why Does Text Get Garbled? The Byte-to-Glyph Mismatch
Computers do not store letters or emojis directly; they store raw bytes (numbers from 0 to 255). A character encoding is a dictionary that maps specific byte numbers to visual characters (glyphs).
UTF-8 Multi-Byte Encoding
Modern UTF-8 represents characters using 1 to 4 bytes. Standard English letters (A–Z) take 1 byte (0x41 = 'A'). Arabic letters and European accents take 2 bytes (e.g. Arabic Alif ا is 0xD8 0xA7). Smart quotes and emojis take 3 to 4 bytes.
Single-Byte Legacy Encodings
Legacy systems (Windows-1252, ISO-8859-1, Latin-1) assume every single byte is one character. When Windows-1252 receives the 2 bytes for Arabic Alif (0xD8 0xA7), it displays byte 0xD8 as Ø and byte 0xA7 as §, creating ا.
2. Common Mojibake Patterns & Their True Meanings
| Garbled Appearance | True Intended Text | Language / Symbol | Root Cause |
|---|---|---|---|
| ال | ال (Al-) | Arabic Definite Article | UTF-8 bytes (0xD8 0xA7 0xD9 0x84) read as Windows-1252 |
| مرØبا | مرحبا (Marhaban) | Arabic Greeting | UTF-8 bytes read as Windows-1252 |
| é, è, à | é, è, à | French / Spanish Accents | 2-byte UTF-8 read as ISO-8859-1 (0xC3 = Ã) |
| ’, “, †| ’, “, ” | Smart / Curly Quotes | 3-byte UTF-8 (0xE2 0x80 ...) read as Windows-1252 |
| — | — (Em Dash) | Punctuation Dash | 3-byte UTF-8 (0xE2 0x80 0x94) read as Windows-1252 |
| \u0645\u0631\u062d\u0628\u0627 | مرحبا | JSON Unicode Escapes | Unescaped Unicode literal code points from API output |
3. How to Prevent Mojibake in Databases, CSVs, and Web Servers
- Always specify UTF-8 in HTTP headers: Ensure your server sends
Content-Type: text/html; charset=utf-8. In PHP, putheader('Content-Type: text/html; charset=UTF-8');at the very top. - Set database connection collation to utf8mb4: In MySQL/MariaDB, use
utf8mb4_unicode_cirather than legacylatin1orutf8(which only supports 3 bytes and corrupts 4-byte emojis). - Include UTF-8 BOM for Microsoft Excel CSVs: Excel on Windows often defaults CSV imports to ANSI/Windows-1252 unless the file starts with the UTF-8 Byte Order Mark (
0xEF, 0xBB, 0xBF). - HTML Meta Tag: Always include
<meta charset="UTF-8">in the<head>section of every web page.
Was this tool helpful?
Comments
Loading comments...