Characters: ASCII and Unicode
Why Do Characters Need a Code?
Computers only store binary numbers. To store a letter like ‘A’ or a symbol like ‘&’, every possible character must be assigned a unique number, and that number is then stored in binary. The complete collection of characters a system can represent, together with the numeric codes assigned to each, is called a character set. Without an agreed character set, text sent from one computer to another would be garbled.
ASCII: The Original Standard
ASCII (American Standard Code for Information Interchange) was developed in the 1960s and became the dominant early standard. Standard ASCII uses 7 bits per character, giving \(2^7 = 128\) possible codes (0 to 127). These 128 codes cover:
- Upper-case letters A–Z (codes 65–90)
- Lower-case letters a–z (codes 97–122)
- Digits 0–9 (codes 48–57)
- Punctuation and special symbols (e.g. space = 32, ‘!’ = 33)
- Control characters (e.g. newline, carriage return, tab — used by printers and early terminals)
Three ASCII codes you must know: ‘A’ = 65, ‘a’ = 97, ‘0’ = 48. Note that upper-case letters always have lower codes than their lower-case equivalents — there is a gap of exactly 32 between ‘A’ (65) and ‘a’ (97). The digit ‘0’ is code 48, not 0.
Extended ASCII
As computers spread internationally, 128 codes proved insufficient. Extended ASCII uses 8 bits (one full byte), giving \(2^8 = 256\) codes (0 to 255). The extra 128 codes (128–255) were used for accented letters, currency symbols and graphics characters. Different manufacturers implemented the upper range differently, which caused compatibility problems — the same code could mean a different character on different systems.
To store the word “Cat” in memory using ASCII:
‘C’ = 67 = 01000011
‘a’ = 97 = 01100001
‘t’ = 116 = 01110100
Three bytes of storage are needed — one byte per character. The string “Cat” is stored as the sequence of binary codes 01000011 01100001 01110100.
Unicode: A Global Solution
Unicode was created to solve the limitations of ASCII. It assigns a unique code point to every character in every writing system in the world — Latin, Cyrillic, Arabic, Chinese, emoji and thousands more. Unicode currently defines over 140,000 characters (and the range can extend to over 1 million).
Unicode itself is a standard, not an encoding. The encoding — how the code points are stored as bytes — is specified separately. Three common encodings are:
- UTF-8: variable length, 1 to 4 bytes per character. ASCII characters use exactly 1 byte (backward compatible). The most widely used encoding on the web.
- UTF-16: variable length, 2 or 4 bytes per character. Used internally by Windows and Java.
- UTF-32: fixed 4 bytes per character. Simple but uses more storage than UTF-8 for mostly-ASCII text.
Unicode uses more bits per character than ASCII, which means Unicode files are larger. However, UTF-8 is backward compatible with ASCII: the first 128 Unicode code points match the ASCII codes exactly, so any pure ASCII text file is also a valid UTF-8 file. This is why UTF-8 replaced ASCII without breaking existing systems.
Comparing ASCII and Unicode
| Feature | ASCII (7-bit) | Unicode (UTF-8) |
|---|---|---|
| Number of characters | 128 (standard); 256 (extended 8-bit) | Over 140,000 (extendable to 1 million+) |
| Bits per character | 7 (or 8 for extended) | 8 to 32 (variable in UTF-8) |
| Languages supported | English and basic Western symbols only | All world languages including emoji |
| File size | Smaller for English text | Larger for non-ASCII scripts; same as ASCII for English in UTF-8 |
| Backward compatibility | N/A | UTF-8 fully compatible with ASCII for the first 128 codes |
The trade-off is clear: ASCII is compact but limited. Unicode is comprehensive but uses more storage for non-ASCII scripts. For modern software that must work in any language, Unicode is the only practical choice.
A key practical point: character encoding errors arise when a file saved in one encoding is opened assuming a different encoding. You have probably seen garbled symbols or question marks appear in a document — that is a mismatch between the encoding used to save and the encoding used to read.
Why Encoding Choice Matters
An encoding is only useful if both the program writing the file and the program reading it agree on which scheme is in use. A file saved in UTF-8 but opened as Windows-1252 will display garbled symbols where multi-byte characters used to be. This is called mojibake, and it remains one of the most common bugs in software that handles text from multiple sources. Modern web standards address this by requiring documents to declare their encoding in the HTTP header or, for HTML, in the document head, so the browser always knows which scheme to apply.
UTF-8 has become the dominant encoding on the web because it remains backward-compatible with ASCII while extending naturally to every script. A file containing only English text takes exactly the same number of bytes whether it is encoded in ASCII or in UTF-8, but the same encoding can also hold Greek, Japanese, emoji and historical scripts without further work. This combination of compatibility and reach is why nearly all new file formats, network protocols and source code now default to UTF-8 rather than ASCII alone.