Unravel › Java concepts › Strings and text
Characters, Unicode and encodings
A char is a 16-bit UTF-16 unit, not always a whole visible character.
Learn these first
At a glance
Letters are in a box of numbered tiles (Unicode).
- 1Character.isDigit(c), isLetter, isWhitespace, toUpperCase(c) classify and convert one charc - '0' turns the digit character '7' into the number 7.
- 2s.getBytes(StandardCharsets.UTF_8) and new String(bytes, StandardCharsets.UTF_8) convert between text and bytesThe no-argument versions use the platform default, which differs between machines.
- 3Characters outside the basic plane (many emoji) take two chars (a surrogate pair), so length() and charAt count UTF-16 unitscodePoints() and codePointCount count real characters.
- Typical bugs: "mojibake" (UTF-8 bytes read as Latin-1), a file written on Windows and read on Linux, and reverse() that splits an emoji in half
- Visually identical text can be different code points (é as one char or e plus an accent)
- Since Java 18 the default charset is UTF-8, but older versions use the OS setting (-Dfile.encoding)
import java.nio.charset.StandardCharsets;public class Digits { static int sumDigits(String s) { int total = 0; for (char c : s.toCharArray()) if (Character.isDigit(c)) total += c - '0'; return total; } static byte[] pack(String s) { return s.getBytes(StandardCharsets.UTF_8); }}- line 5 uses Character, the core of Characters, Unicode and encodings
- line 8 uses getBytes, the core of Characters, Unicode and encodings
Read the full explanation (about 1 min)
A char is a 16-bit UTF-16 unit, not always a whole visible character. Text becomes bytes only through an encoding (UTF-8, ISO-8859-1), and you should always name it.
A mental model
Letters are in a box of numbered tiles (Unicode). Writing a file means choosing a shorthand (UTF-8) for the numbers; reading with a different shorthand gives gibberish.
How it works
Character.isDigit(c), isLetter, isWhitespace, toUpperCase(c) classify and convert one char. c - '0' turns the digit character '7' into the number 7.
s.getBytes(StandardCharsets.UTF_8) and new String(bytes, StandardCharsets.UTF_8) convert between text and bytes. The no-argument versions use the platform default, which differs between machines.
Characters outside the basic plane (many emoji) take two chars (a surrogate pair), so length() and charAt count UTF-16 units. codePoints() and codePointCount count real characters.
Going deeper
Typical bugs: "mojibake" (UTF-8 bytes read as Latin-1), a file written on Windows and read on Linux, and reverse() that splits an emoji in half. Specify UTF-8 for files, HTTP and databases.
Visually identical text can be different code points (é as one char or e plus an accent). Use Normalizer.normalize(s, Form.NFC) before comparing user input.
Under the hood
Since Java 18 the default charset is UTF-8, but older versions use the OS setting (-Dfile.encoding). Internally strings may be stored as Latin-1 bytes when possible; the API always behaves as UTF-16 code units.
Common mistakes
- Calling
getBytes()without a charset. - Assuming
length()is the number of visible characters. - Casting an int to char for arithmetic and forgetting the numeric value of
'0'.
How Unravel recognises it
When you paste code, Unravel looks for these signs of characters, unicode and encodings. Spotting them yourself is a good habit when you read unfamiliar code:
- Types:
CharacterNormalizer - Method calls:
codePointAt()codePoints()getBytes()isLetter()isDigit()isLetterOrDigit()isWhitespace()isUpperCase()getNumericValue()
Part of a learning path
- Text, numbers and arrays · previous: Formatting text and numbers · next: Primitive types and floating point
Related concepts
Frequently asked questions
What is Characters, Unicode and encodings?
A char is a 16-bit UTF-16 unit, not always a whole visible character. Text becomes bytes only through an encoding (UTF-8, ISO-8859-1), and you should always name it.
What are the most common mistakes with Characters, Unicode and encodings?
Calling getBytes() without a charset. Assuming length() is the number of visible characters. Casting an int to char for arithmetic and forgetting the numeric value of '0'.
How can I see Characters, Unicode and encodings in real code?
Paste your own Java into Unravel and it will point out where characters, unicode and encodings appears, explain how it works at the detail level you choose, and draw the flow of the code around it.