Arithmetic Coding Interval Visualizer

A whole message squeezed into one number.

  • Interval narrowing
  • Shortest code
  • 32-bit integer coder
A small clay table with a long soft clay rod split into rounded segments, a shorter rod nested inside one segment and a few loose rounded beads.

The guide

One number can carry a whole message, and it costs about what the message is worth

Arithmetic coding narrows an interval one symbol at a time and sends a single fraction. Its length tracks the information in the message, and the small excess has exact causes.

  • 7.381 bits information in the three-symbol example message
  • 8 bits shortest code whose whole block fits the interval
  • 7 bits a point-only code that also decodes the example
  • 23 bits abracadabra coded, against 22.444 bits of information
  • 26.758 bits information cost of abracadabra under an adaptive model
Read the full guide Fold the guide 6 min · 12 sources

Most compression schemes assign each symbol a code of its own and string the codes together. Arithmetic coding does something stranger. It treats the whole message as one fraction between 0 and 1, and the only thing it sends is enough digits of that fraction to pick it out.

The idea is easy to state. Start with the interval from 0 up to but not including 1. Lay the symbols along it in pieces whose lengths are their probabilities. To code a symbol, keep only its piece, and divide that piece in the same proportions for the next symbol. When the message ends, the interval is tiny, and any fraction inside it names the message.

One message, three narrowings

The standard textbook model has four symbols: a neutral one with probability 0.6, a positive one with 0.2, a negative one with 0.1 and an end-of-data symbol with 0.1. Call them N, P, G and E. They fill [0, 0.6), [0.6, 0.8), [0.8, 0.9) and [0.9, 1). The message is N, then G, then E.

The first symbol keeps [0, 0.6). Within it the second symbol G owns the part from 80 to 90 percent, which is [0.48, 0.54). Within that, E owns the last tenth, [0.534, 0.54). The final interval has width 0.006, which is 0.6 times 0.1 times 0.1. The width of the end result is always the product of the probabilities of the symbols, which is why likely messages leave wide intervals and cheap codes.

Which fraction should be sent? The page for the method gives 0.538, and it decodes correctly. The decoder starts with the same model and asks, step by step, where the number falls. It lands in N’s piece, then in G’s piece within that, then in E’s, and E ends the message. Replaying the model needs nothing more than the number.

The ends of the interval are not symmetric. The low end belongs to the interval and the high end does not. So 0.534 decodes as N, G, E, while 0.540 does not: with the length of three given, it decodes as N, E, N. Getting this wrong makes a coder that works until the one day a value lands exactly on an edge.

Why the length tracks the information

A message of probability p can be told apart from its neighbours with about minus log base 2 of p bits. For the example that is 0.737 plus 3.322 plus 3.322, or 7.381 bits. This is the information content, and it is the figure that Shannon’s source coding theorem makes the compression limit for a source with these probabilities. Arithmetic coding gets close to it, particularly for long messages, because it does not round each symbol to a whole number of bits.

A code has to be a whole number of bits, though, and it has to be a prefix code if codes are to follow each other. The safe way to build one is to find the shortest binary fraction whose whole block lies inside the interval. A block is every fraction that begins with those bits. For the example the shortest is 8 bits, 10001001, which is 0.53515625. That is 8.4 percent more than 7.381, the cost of the whole bits and the block.

A shorter code exists if you drop the prefix rule. The 7 bits 1000101 are 0.5390625, a point inside the interval, and it decodes to N, G, E when the decoder pads the number with zeros and the message ends itself. The page for the method states that an encoder would use at least 8 bits. That holds for the block rule, and the tool shows both numbers and names the rule of each.

Decimal is wasteful by comparison. Three decimal digits, 0.534 for example, carry 3 times log base 2 of 10, which is 9.966 bits. The step from digits to bits is one reason arithmetic coding is described in binary.

The excess is bounded. A block of 2 to the minus k wide fits inside an interval only when the interval is a little wider than the block, so the code can run up to about 2 bits over the information content, and a point-only code up to about 1. The test run in the tool on random messages of up to 400 symbols saw the block at most 1.92 bits over and the point at most 0.88.

There is a second trap in the end of the message. A number alone does not say how many symbols it holds. Leave the end symbol out of the example and the first two symbols, N and G, leave an interval of width 0.06, ten times wider and a cheaper code, but a decoder fed that code runs on and invents more symbols. The cost of the end symbol, here 3.322 bits, buys the right to stop. A stated length does the same job without spending probability.

A longer message, and the competition

Take abracadabra: eleven symbols, with a five times, b and r twice each, c and d once. The final width is 5 to the fifth times 2 to the fourth over 11 to the eleventh, about 1.75 times 10 to the minus 7. The information is 22.444 bits, or 2.0404 bits per symbol. The shortest prefix-free code has 23 bits, and a point-only code needs 22. A fixed three-bit code costs 33.

Huffman coding builds a code tree and gives each symbol a whole number of bits. For these counts it gives a one-bit code to a and three bits to each of b, c, d and r, so abracadabra costs 23 bits as well. The two methods tie here. Both reach the entropy when every probability is a power of one half, and otherwise arithmetic coding can do better, most of all when one symbol is very likely: a Huffman code cannot go below one bit per symbol, however skewed the source.

Now let the model learn. An adaptive model starts every symbol at count 1 and adds 1 each time a symbol is coded. For abracadabra the information cost under that model is 26.758 bits, and the code is 27 bits. That is 4.3 bits more than the model that was handed the true counts, the price of finding out what the counts were. An adaptive model needs no first pass over the data, which is why streaming coders favour it.

The adaptive model has a subtle rule that the tool follows exactly. The count of a symbol rises only after it has been coded, so the decoder, which sees the same symbols in the same order, always holds the same counts as the encoder at every step. If the encoder updated first, the two models would drift apart and the message would decode as noise.

What finite precision costs

Real coders cannot keep fractions with hundreds of digits. The range-coding variant divides a range of whole numbers instead, and a real implementation would use the full range of the machine integer. As the range narrows, its leading digits are settled and sent out early, and the range is stretched back up.

The tool’s 32-bit coder follows that scheme with three cases. If the range lies in the lower half, it emits a 0. In the upper half, it emits a 1. If it straddles the middle quarter, a bit is owed and counted as pending, and the next bit that is emitted settles it. At the end a closing bit is written. On abracadabra it produces exactly the 23 bits of the exact code.

It also costs something. On 300 seeded random strings, the integer code ran from 0.06 to 2.00 bits over the information content, within two bits of the ideal on every one. A skewed message tests the pending bits hardest: 3,000 symbols of probability 0.9999 take 2 bits, against an ideal of 0.43, and the decoder gets every symbol back. The totals are limited to 65,536 so the arithmetic fits.

What the visualizer will not do

It shows the idea with a model you give it, up to 200 symbols and 26 different ones. It does not mix contexts, use binary coders driven by probability tables, or write any real file format. The numbers are the cost of the model you chose, not of a file you have. Use it to see the interval shrink, then try the end symbol, the stated length and the adaptive model on the same text, and watch where the extra bits go.

Sources

Every source consulted for this guide. Links checked November 22, 2026.

  1. Arithmetic coding Wikipedia (CC BY-SA)
  2. Range coding Wikipedia (CC BY-SA)
  3. Huffman coding Wikipedia (CC BY-SA)
  4. Entropy (information theory) Wikipedia (CC BY-SA)
  5. Shannon's source coding theorem Wikipedia (CC BY-SA)
  6. Prefix code Wikipedia (CC BY-SA)
  7. Adaptive coding Wikipedia (CC BY-SA)
  8. Data compression Wikipedia (CC BY-SA)
  9. Lossless compression Wikipedia (CC BY-SA)
  10. Claude Shannon Wikipedia (CC BY-SA)
  11. BigInt MDN Web Docs (CC BY-SA)
  12. fractions: Rational numbers Python Software Foundation (PSF licence)