Reverse Text Generator

Reversing a string is the first exercise in a great many programming courses. Split the text into its characters, turn the list around, join it up again. Three steps, one line, done before the coffee cools. It works on the word “hello”. It works on most of what an English speaker types on an ordinary day. Then somebody feeds it a name with an accent, a flag or a family emoji, and the output contains question marks in boxes, an accent floating over the wrong letter and a flag of a different country.

Nothing went wrong with the reversing. The list was turned around perfectly. What went wrong was the first step, the innocent one: splitting the text into its characters. The program and the person disagreed about what a character is, and the program never asked.

Three things called a character

Unicode gives every character a number, called a code point, in a space that runs from 0 to 10FFFF in hexadecimal. That is 1,114,112 code points. Computers do not store those numbers directly. They store them in one of three encoding forms, built from units of 8, 16 or 32 bits, and a single code point may take more than one unit.

In UTF-8, the encoding of nearly every web page and text file, a code point takes from 1 to 4 bytes. In UTF-16, the form that JavaScript strings use, it takes one 16-bit unit or two. The length of a JavaScript string is a count of those units, not of anything a reader would recognise, which is why a single emoji so often reports a length of 2.

So there are already two candidates for “character”: the unit and the code point. Reverse a string unit by unit and every character that needed two units comes out as its two halves in the wrong order. Neither half means anything alone. The text is no longer valid, and whatever displays it can only draw a sign of surrender.

Reverse by code point instead and the halves stay together, which feels like the fix. It is not. An accented letter can be stored as a plain letter followed by a separate combining accent. A national flag is stored as two regional indicator characters, and the standard for emoji notes that one such character by itself is not a well-formed flag. A family is several people joined by an invisible joiner. A keycap is a digit, a selector and an enclosing mark.

The Unicode annex on text segmentation has a name for what the reader actually sees: the grapheme cluster, which it glosses as the user-perceived character. It is honest that this is a best-effort approximation, one that can be worked out by rule and without ambiguity. Those rules are what a correct reversal has to follow. Move whole clusters, and nothing breaks.

A sentence, counted four ways

Take this line: “Step on no pets (né 1987)”, with the accent typed as a separate combining mark, followed by a family of three, a Canadian flag and a keycap three, with single spaces between them.

A reader counts 31 characters. The same text is 39 code points, 44 UTF-16 units and 63 bytes of UTF-8. All four numbers are correct, and they answer four different questions. The segmentation annex makes the practical point: a form might limit a message to 300 characters as people count them and also to 3,000 bytes, because the first limit says nothing reliable about storage.

Now reverse it three ways. By cluster, all 31 characters arrive whole: the keycap first, then the flag, the family, the bracketed date and the palindrome. By code point, 5 of the 31 are destroyed. The accent has left its letter, the family has been rearranged, the keycap has come apart and the flag has changed country, because its two indicators now spell a different pair. By UTF-16 unit the same 5 are destroyed and 8 orphaned halves are left behind.

There is a quieter trap in the brackets. Turn “(né 1987)” around and the closing bracket leads. A reader expects brackets to face inwards, so a careful reversal swaps each one for its partner, using the pairs that Unicode publishes for mirroring. Quotation marks are a different matter. Which curly mark opens a quotation depends on the language, so the only safe choice is to leave them alone.

Even whole clusters can surprise. Moving them can bring together two neighbours that combine: a stray accent that began the text now follows a letter and lands on it. Nothing is lost, but the result no longer splits the way it was assembled, and reversing it again will not return the original. A careful tool counts those joins and reports them.

Text that already runs the other way

Hebrew and Arabic are written from right to left, and it is tempting to think they are stored backwards. They are not. Unicode prescribes logical order: text sits in memory in the order it is read, and the display works out the direction. In a browser, the order of characters in memory is simply not the order in which they are drawn.

That has a consequence for reversal. Turning around the stored order of a Hebrew word spells it backwards for its reader, exactly as it would for English. But in a line that mixes directions, the display lays out each run again by its own rules, so the result is not a mirror image of what was on screen before. Arabic letters change shape as well, since each joins to new neighbours.

The same machinery includes invisible control characters, and one of them is the cheapest way to make text look reversed: the right-to-left override, U+202E. It forces whatever follows to be treated as strongly right to left until a closing character or the end of the paragraph. The letters are stored untouched and drawn in the opposite direction.

The bidirectional annex itself says that overrides are to be avoided wherever possible, because of security concerns. The reason is easy to show. A catalogue of attack techniques describes a script file whose name contains the override, so that a file ending in .js is displayed as though it ended in .png. Unicode’s guidance on source code describes the same trick inside programs, where the order a reviewer reads can differ from the order a compiler receives. A reversing tool should therefore never slip such a character into its output unasked. Offered at all, it belongs behind a switch that is off, with its two extra characters counted and its risks stated.

Upside down is a table of strangers

Upside-down text looks like a rotation and is nothing of the kind. Plain text has no way to rotate a letter. What it has is a large inventory of other characters, some of which are an ordinary letter turned over and encoded as a symbol in its own right.

The character database names them plainly. There is a LATIN SMALL LETTER TURNED A, a TURNED AMPERSAND, an INVERTED QUESTION MARK. Search the names for every character on a keyboard and the result is modest: 31 of 95 printable keyboard characters have a counterpart named as their turned or inverted form. There is no turned b, no turned s, no turned capital E. Mirror writing fares worse, with just 11 letters and marks named as reversed, beyond the true mirrored pairs of brackets and mathematical signs.

Everything else in the familiar upside-down alphabets is a guess about shapes: a q standing in for a b, a letter borrowed from another script. Some guesses look fine in one typeface and wrong in the next. A table built only from names is smaller and less impressive, and every entry in it can be defended.

Either way the cost is the same. These are different characters. Accessibility guidance from one state government warns that characters which merely look like ordinary letters may not be interpreted the same way, or may be skipped entirely, by a screen reader. Search will not find the word. Spell-check will not know it. For a true flip of every character there is only one honest route, which is a picture, with a text alternative beside it.

The words that survive

A palindrome is the text that reversal cannot change. The idea is old: a Latin word square from the first century holds both word and sentence palindromes, and the English word for it dates from 1638. A poet coined an English sentence palindrome in 1614, with an ampersand doing some of the work. Finnish has an example in everyday use with 19 letters, saippuakivikauppias, a seller of soapstone.

Testing one raises every question above again. “Step on no pets” passes only if capitals are ignored, and a French palindrome only if accents are. Doing that properly means Unicode case folding, under which a German sharp s matches a double s, and normalization, which first splits an accented letter into letter and mark so the mark can be set aside. And it means comparing clusters, so that a flag counts once.

Finding the longest palindrome hidden inside a longer text sounds expensive, since every centre must be tried. A method found by Glenn Manacher in 1975 does it in time proportional to the length, by reusing what a palindrome already found says about the positions inside it.

None of this can settle what a reader will accept as a character in every language. Grapheme clusters are a default, and the rules still move: version 18.0 of the standard changed how certain Indic consonant clusters are held together, so two correct programs a year apart can disagree. Reversal is simple. Knowing what you are reversing is the whole problem.