Base32 Encoder and Decoder

A byte can hold any of 256 values. Most of the places people put data cannot.

Mail is the textbook case. The MIME specification of 1996 describes mail systems that limit a message to relatively short lines of seven-bit US-ASCII, and says this forces users to convert any non-textual data into seven-bit bytes representable as printable characters before sending it. Its answer was Base64, which uses a 65-character subset of ASCII and carries 6 bits in each printable character. The specification is frank about the bill: encoded data are consistently only about 33 percent larger than the original. For a machine talking to a machine, that is a fair bargain and it has lasted.

Base64 was never designed for people, though. Its alphabet mixes upper and lower case with special characters, and Douglas Crockford calls it difficult to type and difficult to pronounce. It contains a plus sign and a slash, and the standard that defines the family warns that the slash may be problematic in file names and URLs, and that both symbols are treated as word breaks by older search and index tools. Base32 exists for the places where those objections decide the matter.

Five bits at a time

The rule fits in a sentence. Take the input five bytes at a time, which is 40 bits, read those bits from the most significant end, cut them into eight groups of five, and replace each group with one of 32 characters. Because 32 is two to the fifth power, each character carries exactly 5 bits. RFC 4648 uses the letters A to Z for the values 0 to 25 and the digits 2 to 7 for 26 to 31, with a 33rd character, the equals sign, kept for padding.

The word foobar shows every part of it. Its six bytes are 66 6F 6F 62 61 72 in hexadecimal. The first five, written in binary, run 01100110 01101111 01101111 01100010 01100001. Regrouped in fives, the same forty bits read 01100 11001 10111 10110 11110 11000 10011 00001, which are the values 12, 25, 23, 22, 30, 24, 19 and 1. In the alphabet those are M, Z, X, W, 6, Y, T and B.

One byte is left over: 72, or 01110010. It gives one whole group, 01110, and three stray bits, 010. Two zero bits are added on the right to finish the second group, 01000, and the two values 14 and 8 become O and I. Six equals signs then fill the block to eight characters. The result is MZXW6YTBOI======, which is exactly what RFC 4648 lists among its test vectors. Six bytes became sixteen characters.

That is the cost, stated plainly. Five bytes always become eight characters, so Base32 output is 60 percent larger than its input before any padding, where Base64 turns three bytes into four characters. The same foobar is Zm9vYmFy in Base64: eight characters, half the length. A megabyte of data becomes 1.6 megabytes of Base32.

What the extra characters buy

RFC 4648 says what the design is for: Base32 represents arbitrary sequences of octets in a form that needs to be case insensitive but that need not be human readable. The alphabet does a little more than that sentence promises. The same document notes that the characters 0 and O are easily confused, as are 1, l and I, and observes that the Base32 alphabet contains neither zero nor one. A reader who sees a round shape in a Base32 string knows it is the letter.

The domain name system is one user. When DNSSEC needed to publish hashed names inside the DNS, RFC 5155 chose the second alphabet in RFC 4648, Base32 with the extended hex alphabet, which runs 0 to 9 and then A to V, and it reads those digits without regard to case. That alphabet has a property the first one lacks: encoded data keeps its sort order, so hashed names sort the same way as the hashes. The specification also drops the trailing padding.

Onion addresses make the same choice for the same reason. The Tor specification builds an address from a 32-byte public key, a 2-byte checksum and a 1-byte version, encodes those 35 bytes in Base32 and appends a suffix. Thirty-five is a multiple of five, so the result is 56 characters with nothing left over and no padding to argue about.

Base32 also carries the shared secret behind one-time sign-in codes. RFC 4226 requires a shared secret of at least 128 bits and recommends 160 bits. The otpauth URI format that authenticator apps read, written down in an Internet-Draft, says the secret must be encoded in Base32 and that padding should be omitted. A 160-bit secret is 20 bytes, which is exactly 32 Base32 characters. Sixteen characters hold only 80 bits, well short of the minimum.

None of this hides anything. Base32 is a spelling, not a lock, and anyone who can read the characters can recover the bytes. RFC 6238 asks that such keys be protected against unauthorized access and usage. A Base32 secret deserves the care a password gets.

Padding and the rules nobody reads

Padding is where implementations drift apart. RFC 4648 lists the only endings that can arise: a final group of two, four, five or seven characters, followed by six, four, three or one equals signs, or a full group of eight with none. It follows that a final group of one, three or six characters cannot occur at all. A decoder that meets one is looking at damaged input, not at an unusual encoding.

The same standard lets a specification that refers to it leave padding out, and many do. Without padding the length still tells the story, because n bytes always need eight n over five characters, rounded up. What is lost is a cheap check. The z-base-32 authors judged that check not worth the extra length, and said so.

A subtler rule concerns the stray bits. In foobar, the last character carries three real bits and two added ones. The standard says those added bits must be zero. If a decoder ignores them, then MZXW6YTBOJ====== decodes to the same six bytes as the correct string, because J and I differ only in bits that belong to no byte. RFC 4648 names the danger exactly: multiple encoded strings can then be decoded to the same binary data, and there is no canonical form to compare. It allows decoders to reject such input.

Two more requirements are routinely ignored. Implementations must reject characters outside the alphabet unless the referring specification says otherwise, partly because stray characters can serve as a covert channel. And they must not add line feeds unless told to. A strict decoder that refuses a pasted line break is following the text, not being difficult.

One name, several alphabets

Douglas Crockford’s Base 32 starts from different requirements: human readable, compact, error resistant and pronounceable, so that people can pass symbols to each other on a telephone. His alphabet is the ten digits followed by 22 letters. It leaves out I, L, O, U: the first three because they can be confused with 1 and 0, the last to avoid accidental obscenity. On decoding, both cases are accepted, i and l are read as 1 and o as 0, and hyphens are ignored.

Crockford also defines an optional check symbol. It encodes the number modulo 37, the least prime greater than 32, and needs five extra symbols for the values 32 to 36. It catches a wrong symbol and two symbols swapped. His page speaks of numbers, not byte streams, and the ULID identifier format uses it that way: 128 bits written as 26 characters, a string that could hold 130 bits, so the specification fixes the largest valid value.

The z-base-32 design goes further toward the human hand. Its authors drop 0, l, v and 2 as the characters most likely to be misread, write in lower case, and reorder the alphabet so that the characters that most often end a string are the easiest ones to read and write. It has no padding, and it can carry a bit count that is not a whole number of bytes. ZRTP uses that alphabet to turn 20 bits into the four characters two callers compare aloud.

Geohash uses a fourth arrangement: the digits and the lower case letters except a, i, l and o. There the characters encode a position on a grid, not arbitrary bytes, but the alphabet is one more string of 32 that looks like the others.

That resemblance is the practical trap. The ten characters MZXW6YTBOI are valid RFC 4648 and also valid Crockford, where the O and the I would be read as 0 and 1 and the bytes would differ. Nothing in the string says which was meant. A tool can rule alphabets out when it finds a character they lack. It cannot rule one in, and an honest one says when the evidence runs out.

So the method has clear limits. It does not compress. It does not conceal. Apart from Crockford’s check symbol, it does not notice a mistyped character that happens to be in the alphabet. What it does is narrow: it makes arbitrary bytes survive capitals lock, a font that draws zero like the letter O, a file system that folds case, and a voice on a bad line. The z-base-32 document adds the sensible caveat that anyone who needs to interoperate should use the standard encoding, at that time RFC 3548, which RFC 4648 later replaced.

Sixty percent is the price of that. For a key someone must type, it is cheap.