BackendBench
Coffee

Unravel › Java concepts › Strings and text

Java language

Characters, Unicode and encodings

A char is a 16-bit UTF-16 unit, not always a whole visible character.

Learn these first

At a glance

Mental model

Letters are in a box of numbered tiles (Unicode).

What happens, in order
  1. 1Character.isDigit(c), isLetter, isWhitespace, toUpperCase(c) classify and convert one charc - '0' turns the digit character '7' into the number 7.
  2. 2s.getBytes(StandardCharsets.UTF_8) and new String(bytes, StandardCharsets.UTF_8) convert between text and bytesThe no-argument versions use the platform default, which differs between machines.
  3. 3Characters outside the basic plane (many emoji) take two chars (a surrogate pair), so length() and charAt count UTF-16 unitscodePoints() and codePointCount count real characters.
Remember
Watch out
Calling getBytes() without a charsetAssuming length() is the number of visible charactersCasting an int to char for arithmetic and forgetting the numeric value of '0'
Example (highlighted lines explain it)
import java.nio.charset.StandardCharsets;public class Digits {  static int sumDigits(String s) {    int total = 0;    for (char c : s.toCharArray()) if (Character.isDigit(c)) total += c - '0';    return total;  }  static byte[] pack(String s) { return s.getBytes(StandardCharsets.UTF_8); }}
  1. line 5 uses Character, the core of Characters, Unicode and encodings
  2. line 8 uses getBytes, the core of Characters, Unicode and encodings
Read the full explanation (about 1 min)

A char is a 16-bit UTF-16 unit, not always a whole visible character. Text becomes bytes only through an encoding (UTF-8, ISO-8859-1), and you should always name it.

A mental model

Letters are in a box of numbered tiles (Unicode). Writing a file means choosing a shorthand (UTF-8) for the numbers; reading with a different shorthand gives gibberish.

How it works

Character.isDigit(c), isLetter, isWhitespace, toUpperCase(c) classify and convert one char. c - '0' turns the digit character '7' into the number 7.

s.getBytes(StandardCharsets.UTF_8) and new String(bytes, StandardCharsets.UTF_8) convert between text and bytes. The no-argument versions use the platform default, which differs between machines.

Characters outside the basic plane (many emoji) take two chars (a surrogate pair), so length() and charAt count UTF-16 units. codePoints() and codePointCount count real characters.

Going deeper

Typical bugs: "mojibake" (UTF-8 bytes read as Latin-1), a file written on Windows and read on Linux, and reverse() that splits an emoji in half. Specify UTF-8 for files, HTTP and databases.

Visually identical text can be different code points (é as one char or e plus an accent). Use Normalizer.normalize(s, Form.NFC) before comparing user input.

Under the hood

Since Java 18 the default charset is UTF-8, but older versions use the OS setting (-Dfile.encoding). Internally strings may be stored as Latin-1 bytes when possible; the API always behaves as UTF-16 code units.

Common mistakes

  • Calling getBytes() without a charset.
  • Assuming length() is the number of visible characters.
  • Casting an int to char for arithmetic and forgetting the numeric value of '0'.

How Unravel recognises it

When you paste code, Unravel looks for these signs of characters, unicode and encodings. Spotting them yourself is a good habit when you read unfamiliar code:

Part of a learning path

Related concepts

Frequently asked questions

What is Characters, Unicode and encodings?

A char is a 16-bit UTF-16 unit, not always a whole visible character. Text becomes bytes only through an encoding (UTF-8, ISO-8859-1), and you should always name it.

What are the most common mistakes with Characters, Unicode and encodings?

Calling getBytes() without a charset. Assuming length() is the number of visible characters. Casting an int to char for arithmetic and forgetting the numeric value of '0'.

How can I see Characters, Unicode and encodings in real code?

Paste your own Java into Unravel and it will point out where characters, unicode and encodings appears, explain how it works at the detail level you choose, and draw the flow of the code around it.