IDN Homograph Attack Detector

A reader looks at an address and sees letters. The machinery underneath never does. It handles a row of numbers, one per character, and only at the last moment does a font turn those numbers into shapes.

Most of the time the difference is of no interest to anyone. It matters when two different numbers are drawn with the same shape. Then two different names look like one name, and the advice to “check the address carefully” stops working, because the eye is being asked to tell apart things that were designed to be drawn identically. The only reliable reading is of the numbers themselves.

Why names left ASCII

For most of its history a host name was written in a small alphabet. A label, the part of a name between two dots, was made of ASCII letters, digits and the hyphen, could not begin or end with a hyphen, and could not exceed 63 octets. That was enough for English and for little else. ICANN describes the purpose of internationalized domain names plainly: they let people use domain names in local languages and scripts, built from characters such as Arabic, Chinese, Cyrillic or Devanagari.

The engineering problem was that the DNS itself could not be changed everywhere at once. The answer published in 2003 left it alone. Applications would accept names written in Unicode and convert each label, before any lookup, into a form that used only the characters host names already allowed. Every converted label starts with a fixed prefix so that software can recognize it. That prefix is xn--.

The rules were revised in 2010 by a set of documents known as IDNA2008, which moved the job of mapping what a person types, such as turning capitals into lower case, out of the protocol. Browsers fill that gap with Unicode’s compatibility processing, and the URL Standard that browsers follow says exactly which options to use. The detail matters later: what a person pastes and what the browser looks up are not always the same string.

Punycode in plain words

The encoding behind the prefix is called Punycode. It is short enough to follow by hand.

First, every ASCII character of the label is copied out in order. If there were any, a hyphen follows them. Then comes the clever part. Instead of writing down the remaining characters, Punycode writes down instructions for inserting them: a list of numbers, each of which says how far to count before the next character is put in, and where. Those numbers are written with 36 digits, the letters a to z for 0 to 25 and the digits 0 to 9 for 26 to 35, in a variable-length form that keeps small numbers short.

Take the word scope and replace its third letter with the Cyrillic small letter o, which is code point U+043E, or 1,086 in decimal. The four ASCII letters are copied first, giving scpe and then a hyphen. One character is left to insert. Counting starts at 128, the first value that is not ASCII, and a four-letter string offers five positions, so the number to encode is (1,086 - 128) x 5 + 2, where the 2 picks the third position. That is 4,792.

Written in Punycode’s digits, 4,792 comes out as 6, 5, d, and the whole label becomes xn--scpe-65d. The tool on this page shows the same result when its sample named “One swapped letter” is pressed, once from its own encoder and once from the browser’s. Its samples sit under the top-level name example, which was reserved in 1999 for documentation, so nothing real is imitated.

Now replace all five letters. The label is U+0455, U+0441, U+043E, U+0440, U+0435, every one of them Cyrillic. There is nothing ASCII to copy, so no hyphen appears, and the label is stored as xn--e1argc3h. On the screen both versions, and the plain ASCII word, read scope.

One letter, then every letter

The danger was described before the standard was finished, in a short paper titled “The Homograph Attack”. Dated to December 2001 and printed in Communications of the ACM in February 2002, it came from Evgeniy Gabrilovich and Alex Gontmakher of the Technion. They pointed out that the Russian letters that look like a, e, p and y are indistinguishable in writing from their English counterparts. To show it was practical they registered a copy of a large software company’s name in which two letters, c and o, were Russian.

They also saw the difficulty of the cure. Forbidding names that mix alphabets is the simplest fix, they wrote, but it blocks names that are certainly useful. Their other suggestions were to highlight suspicious names that mix letters within a single word, and to keep a map of identical letters and search it for collisions. Browsers later adopted versions of both ideas.

The mixed-script rule handles the single swapped letter. A label with a Cyrillic o among Latin letters uses two scripts, and Chrome’s documentation says Latin, Cyrillic and Greek cannot be mixed with each other; such a label is shown as xn--.

It does not handle the second case. In April 2017 Xudong Zheng published a name built entirely from Cyrillic letters which Chrome, Firefox and Opera displayed as the address of a well-known technology company. His account says he reported it on January 20, that a fix reached Chrome’s source on March 24 and shipped in Chrome 58, and that Firefox declined to change. One script, no mixture, no warning.

What the browser shows, and why browsers differ

Whether a name appears as letters or as xn-- is a decision each browser makes for itself. Chrome documents a list of tests, applied to each label separately. It shows the encoded form if the label holds characters outside the set Unicode allows in identifiers, if it mixes scripts beyond a few permitted combinations, or if it contains invisible characters. It does the same when every letter belongs to a set of Cyrillic letters that look like Latin ones while the top-level domain is not one associated with Cyrillic. It also compares a simplified form of the name against a list of popular domains. Only a label that passes everything is shown in Unicode.

Mozilla’s published algorithm is narrower. A label is shown in Unicode if all its characters come from one script, or from a short list of combinations such as Latin with Han, Hiragana and Katakana. The wiki page that describes it, last edited in April 2017, states that this system permits whole-script confusables and gives the reason: “it is up to registries to make sure that their customers cannot rip each other off.”

Two things follow. A name’s appearance is not a property of the name, since the same label can be letters in one browser and xn-- in another. And registries are part of the defence: ICANN’s implementation guidelines require all code points of a label to come from one script, with exceptions for languages whose writing needs several, and encourage registries to limit whole-script confusables.

Underneath the browser tests is a data file. Unicode publishes confusables.txt, which maps each character to a prototype that stands for everything drawn like it; version 18.0.0 holds 6,712 mappings. To compare two strings, each is decomposed, stripped of characters that are ignored by default, replaced character by character with prototypes, and decomposed again. The result is called a skeleton. Two strings with the same skeleton are confusable. The standard warns that a skeleton is not meant to be displayed, and the reason is easy to see: the table treats m and the pair rn as one shape.

Honest mixtures, hidden characters and the limits of looking

It would be easy to conclude that anything unusual is hostile. It is not. Japanese is normally written with Han, Hiragana and Katakana together, and the registry tables for Chinese, Japanese and Korean commonly include the Latin letters a to z. Unicode’s security report gives the example of a Russian site for XML documents, whose natural name joins a Latin abbreviation to a Cyrillic word. A name written wholly in Cyrillic is, for a very large number of readers, simply a word.

That is why a careful check separates three findings instead of issuing one alarm. A label that mixes scripts is one thing. A label whose every non-ASCII character has an ASCII lookalike, so that the whole reads as an ASCII name it is not, is another. A label in a single script that resembles nothing in ASCII is neither, and deserves no suspicion on those grounds.

Some characters draw nothing at all. The zero width joiner and non-joiner exist because Persian and several Indic scripts need them to spell correctly, and the standard permits them only in those contexts. Unicode notes that the context rules do not catch most cases where two names look alike because of them. Other invisible characters, the soft hyphen among them, are simply removed by the browser’s mapping, so a pasted link can differ from a familiar one as text and still lead to the same place.

What can a reader do? Type an address that matters, or open it from a bookmark, instead of following a link. Look at the xn-- form when one is offered: a name written only in ASCII never has one. Use a password manager, because, as Chrome’s documentation puts it, a password manager will not fill a password into a domain that is not exactly the right one. It compares numbers, not shapes.

Reading code points has limits, and they should be stated. It cannot say who registered a name or what waits at the other end. It cannot judge intent: the five Cyrillic letters above could be an imitation or a brand that happens to be spelled that way. The table is a list of shapes that someone judged alike, while real resemblance depends on the font in front of the reader. And a deception written in plain ASCII, a swapped digit or an extra word, contains no foreign letter to find.

The method answers one question well. Not “is this safe”, which no inspection of a name settles, but “is this the name I think I am reading”. For that, the letters are a picture and the numbers are the fact.