How hard can it be? – Part 3: The answer was somebody else's

by Christian Rudolph

A recreational four-part summer series about trying to recover the settings of real wartime Enigma messages, eighty years after Bletchley Park had already read them.

At the end of the last part I had run every variant of the crib I could think of through a Bombe simulator and produced nothing at all. Not a wrong answer. Nothing. The loop logic simply declined to be satisfied, and declined without explanation, which is the least helpful way for anything to fail.

So I stopped attacking the message and started looking at the sheet it was printed on.

Reading the margins

The document is not a clean sheet of ciphertext. It is covered in marginalia – pencil, red crayon, scribbles, boxes, underlinings, the accumulated handling of a piece of paper that passed through several pairs of hands in a hurry. Somebody had written BKR across the top in red. Somebody else had added S.A Red, and Red II I/II, and Red D96. There is a boxed annotation near the foot of the page with a circled 33 beside it. There are ticks, crossings-out, a garbled group struck through and resent correctly on the line below, and a time – 1635 – at the bottom, as well as annotations from more or less gifted visitors.

I stared at all of this for some while. It plainly meant something. Every mark on that page was made by a person doing a job, and the job was breaking this message. It was, in principle, a complete account of what had happened to the thing I was trying to break.

In practice it meant nothing to me whatsoever. I do not have the working vocabulary of a 1940s intercept room. I could see that Red recurred and knew enough to know that it indicated a particular network within the Luftwaffe. But beyond that I was reading a language I did not speak. I made various attempts to decipher the marginalia, but not enough information emerged to really help me.

Having exhausted my own resources, I did the sensible thing several weeks later than I should have. I wrote to the Bletchley Park Trust.

I had two questions. What is this document, exactly? And – the one I actually cared about – was this message ever broken at the time? And if so: were the Enigma settings recorded?

The reply

I was absolutely delighted when, only a few days later, I received a response. And it was a great one, as will become clear, even though initially it did not seem that way, because the answer to the first question demolished the entire project in a single line.

The message was not from February 1941. It was from 1 November 1944.

The sheet is a reconstruction, assembled by the Trust to illustrate how real messages were processed, using genuine material from their collection. The traffic on it is real. The presentation is not a facsimile of a single original, and the date printed on the front does not belong to the message underneath it.

I want to be clear about how much this ruins. Every statistical run, every Bombe run, every variant crib, every hopeful evening: all of it conducted against 1941 key space, for a message sent in late 1944. It is not that I was searching badly. I was searching a different war.

The answer to my second question came as an attachment. I was sent scans of the original document – both sides. And glued to the reverse is a Typex decode tape: the plaintext, typed out in 1944 by whoever broke it.

So yes. It was broken, comprehensively, at the time. It had been sitting solved on the back of the thing I was attacking from the front.

Two consequences followed immediately. The first is that my crib was right. The tape confirmed the opening exactly as I had it, missing S and odd V and all – the faithful, uncorrected, slightly mangled German that an operator actually typed. Every doubt I had entertained about the crib in Part 2 was misplaced. It was never the problem.

The second is that the mysterious GitHub fragment finally made sense. Nobody guesses AUFKLAVRUNGRAUM ADRIA out of thin air. Whoever posted that crib had seen this tape – had almost certainly stood where I stood, turned the sheet over or been sent the same scans, and read the plaintext off the back. We had both been working from the same source without knowing it.

All of which sharpened the question rather than answering it. The crib was good. The alignment was good. The machine model was good. So why, in dozens of runs, had nothing ever stopped?

UKW-D

The 1944 date was the thread to pull. Once I started reading about late-war Luftwaffe practice rather than 1941 practice, the answer turned up quickly – and it had been printed on the front of the card the whole time. Red D96. That D.

From January 1944 the Luftwaffe began issuing a new reflector: Umkehrwalze D, a rewirable reflector. Where the standard UKW-B and UKW-C had fixed internal wiring, UKW-D could be rewired in the field. Its twenty-six contacts pair into thirteen connections – though, in a piece of engineering compromise, the plugs were bulky enough that only twelve would safely fit, so one pair, B–O, was left permanently wired and the keysheets listed only the other twelve. Those twelve were changed on a schedule, typically on the 1st, 11th, and 22nd of each month. On some keys, from August 1944, UKW-D was used exclusively.

That it was a Luftwaffe device is not an accident. Rewiring twelve pairs of contacts by hand in a cramped reflector is fiddly work, and Luftwaffe signals operators generally did it indoors, in sheltered and heated radio rooms or other reasonably sheltered locations. The Army, working under considerably rougher field conditions, used UKW-D on only one or two links. Whether the most sophisticated refinement of the Enigma system could be deployed at all came down, in the end, to whether the man operating it had somewhere sheltered and stable enough to do the rewiring.

This is catastrophic for an attacker. Every Bombe run I had made assumed UKW-B. When that produced nothing I tried UKW-C, got nothing again, and concluded there was something wrong with my crib. There was nothing wrong with my crib. The machine that encrypted this message had a reflector whose wiring I did not possess and could not derive, and no amount of correct crib, correct alignment, or correct anything else would ever produce a stop without it. I assume the person with the GitHub repository encountered the same problem.

I now knew exactly what I was missing, and it was thanks to the response by Bletchley Park Trust that I was able to develop this further. Not a technique, and not more computing power: thirteen pairs of wires. So I wrote to Frode Weierud, who runs Crypto Cellar Research and has spent decades recovering and publishing wartime cipher material, and asked whether the UKW-D wiring for the 1 November 1944 period was known.

Twenty-eight years of notes

He replied the same day, and rather overshot my question.

He recognised the message immediately. It is one that Tony Sale posted to the Bletchley Park mailing list in October 1998, and Weierud had broken it at the time, working with Philip Marks and Jim Gillogly. He still had his notes, and he very kindly sent them to me.

The complete daily key, which I had been trying to reconstruct from first principles for weeks:

• Rotor order (left–middle–right): IV – II – V
• Ring settings: G – M – Y
• Plugboard: HY PV RG XQ WF KC JB TM IS ND
• Reflector, UKW-D wiring: AW BO CU DM EV FN GR HS IX JT KY LQ PZ
• Indicator: WCB

I had asked for one field and been handed all of them, together with a number of other interesting documents.

He also supplied the detail that makes this historically interesting rather than merely difficult. The Luftwaffe's failure with UKW-D was procedural, not cryptographic: they swapped the reflector while leaving everything else about the day's key unchanged. Same rotor order, same ring settings, same plugboard as the UKW-B version of that day. Once Bletchley had both versions, the new reflector stopped being a wall and became an exercise. The machine was not defeated. The operating procedure was.

Running that key against the ciphertext, a search across all 17,576 rotor positions gave the correct starting position: CIN.

Which brings me back to the marginalia. That boxed annotation near the foot of the page, the one I could not read and had written off as an abbreviation of some kind, is – looking at my own photograph again with the answer in hand – almost certainly CIN, not, as I had assumed, C/N. It was the starting position. It had been sitting in that box, in a photograph on my phone, from the first day.

Before that becomes too neat a moral: it would not have helped. A rotor starting position is one field of a key. Without the date, the rotor order, the ring settings, the plugboard and above all the UKW-D wiring, CIN is a three-letter fact with nothing to attach itself to. I could have read the box perfectly on day one and been exactly where I was on day thirty.

The message

Here is what the traffic actually said, at 1550 hours on 1 November 1944, on the Luftwaffe Red network:

AUFKLAVRUNGRAUM ADRIA X EINS VIER X ZWO FUENF BIS EINS VIER X VIER NULL X UHR
WEGEN DOTORSTOERUNG UND SQLEQTWETTER ABGEBROQNN STOP WEITERE EINSAETZE
WEGEN WETTERLAGE NIQT DURQFUEHRBAR STOP UDASO X EINS ZWO

Reconnaissance in the area of the Adriatic, 14:25 to 14:40 hours, terminated due to engine trouble and bad weather. Further operations not feasible given the weather. Udaso 12.

A reconnaissance flight went up over the Adriatic, lasted fifteen minutes, and came home because the engine was playing up and the weather had closed in. That is the whole message. Nothing was decided by it. Someone had a bad afternoon.

The details are the best part. Q stands in for ch throughout, a standard Luftwaffe convention – SQLEQTWETTER is Schlechtwetter, NIQT is nicht, DURQFUEHRBAR is durchführbar. Numbers are spelled out, and zwei becomes ZWO so it cannot be confused with drei over a bad radio link. The operator dropped the S from AUFKLAERUNGSRAUM. And MOTORSTOERUNG – engine failure – was typed as DOTORSTOERUNG, because M and D sit next to each other on the keyboard and the man was in a hurry.

That typo went out over the air, was intercepted, was written down by hand in Buckinghamshire, was run against a Bombe, was decoded onto a Typex tape, was filed, survived the war, was put on display, was photographed by me, and cost me several weeks of evenings.

Putting it back together

Having the key means you can run the machine forwards as well as backwards. So I re-enciphered the plaintext – typos and all, no tidying – and checked it against the ciphertext on the card. Zero mismatches across all 170 surviving letters, which is the sort of result that makes you trust the whole chain retrospectively.

It also recovers what the card had lost. The fifth group on the document reads FG followed by damage; those three characters are simply gone. Re-encipherment fills them in. The group is FGZKG, and the missing letters are Z, K and G. Here is the complete message as it went out over the air on 1 November 1944:

XEOXO FXVBA ALGZH JKXKN FGZKG EJTEB YVJVE BSSXF FZCOB HFBQF
YVXJA PHXAV PPYGT CZXHS VVRXU PBTYI ZBVVH BMSSN RUKPH UUFDU
UWLMY WFJQN BANPP JMSLN SKVCR JQAJH LKSYA YIDYI IOKRT RDGEN
CPNRJ CBYVX QZCIW HKPVV FTF

Three characters of a wartime intercept, restored eighty years late. It is not much of a contribution to scholarship, but it is mine.

The experiment I could not run in Part 1

There is one question left over from the beginning of all this, and now – for the first time – I have what I need to answer it.

Part 1 ended with my statistical attack producing a gentle slope of near-misses and no clear winner. At the time I could not tell why. Was the method too weak? Was the message too short? Or was I, as it turns out, simply searching the wrong year with the wrong reflector? Those are very different diagnoses and I had no way to separate them.

With the plaintext in hand I can. Take the same message, encipher it with genuine 1941 Army settings – the key sheet from Part 1, day 2: rotors I–V–III, rings 16–07–02, ten plugboard pairs, a standard UKW-B reflector, no rewirable anything – and turn the original statistical attack loose on it. Same message, same length, same method. The only difference is that this time the cipher is the one I spent weeks assuming I was up against.

The result is more interesting than a clean yes or no.

It found the rotors. The correct wheel order came back ranked first out of all sixty possibilities. That is the hardest single part of the problem and the attack nailed it without a crib, without a hint, on statistics alone.

It could not find the rings. Ring settings came back as A–A–A against a true value of P–G–B, and once the rings are wrong everything downstream is wrong with them: starting position, plugboard, plaintext. The best candidate scored an index of coincidence of 0.0494, where real German sits above 0.065. Close enough to be clearly detecting something. Not close enough to read.

So the honest verdict on my original attack is neither "it was hopeless" nor "it would have worked but for the date". It is that the method was sound and underfed. The published threshold for crib-free statistical attacks of this kind sits somewhere around 200 to 250 characters. This message is 173. I was roughly fifty letters short of a fair fight, and no amount of correcting the year would have changed that.

Which is, I think, the most satisfying possible answer. Because the thing that closes the gap is precisely the thing Bletchley had and I did not: a crib. Given a known fragment of plaintext, the Bombe finds the ring settings mechanically – not by scoring likelihood but by testing consistency, stopping when the wiring cannot contradict itself. The statistical approach got me the rotors and then ran out of road. The Bombe's approach would have driven straight down it.

Rutherford's jibe – if your experiment needs statistics, do a better experiment – turns out not to be a rhetorical flourish here. It is a measurement. I ran the experiment that needs statistics, and it took me most of the way and then stopped. They built the machine that does not need them.

What it was actually about

I set out to break an Enigma message with a laptop, on the reasonable assumption that eighty years of progress must count for something. What actually broke it was a letter to an archivist and an email to Frode Weierud who had solved it in 1998 and kept his notes.

The computation was not useless – the machine worked, the Bombe logic was sound, the failures were honest failures rather than bugs. But every one of the three things standing in my way was a piece of information, not a piece of arithmetic: the true date, the reflector wiring, and the daily key. None of them was available by thinking harder. All of them existed, written down, in the keeping of people who had done the work and were glad to share it when asked.

I am not a cryptanalyst. I did not break this message, and it is worth saying plainly that at no point was I ever going to. What I did was walk, slowly and with a great deal of stumbling, along a path that people cut eighty years ago under conditions I can barely imagine – and in doing so I understood, for the first time, what they were actually up against. Not the cartoon version, where a clever man has an idea and the war shortens by two years. The real thing: the plugboard smearing every statistical result into mush; the beautiful sideways move of a machine that tests consistency instead of likelihood; the crib that has to be guessed, aligned, and trusted; the reflector that changes every ten or so days and takes the whole edifice down with it. Every one of those obstacles I hit personally, in the wrong order and for the wrong reasons, and each one taught me something no museum label could have.

That is a strange thing to get out of a photograph taken on a whim, on a day out, of a sheet of paper lying on a desk. I went looking for a puzzle.

Instead I found an archive.

The puzzle had already been solved. What remained was understanding how it had been solved, and why that had been so astonishing in the first place.

That, as it turns out, was the more interesting journey.

Similar and related posts:

• Part 1 of this series can be found here
• Part 2 of this series can be found here
• Part 4 of this series can be found here