The Development and Deployment of Colossus: A Machine that Changed the Course of World War II

In the early 1940s, a team of codebreakers at Bletchley Park, led by Alan Turing, were working on cracking the German Lorenz cipher. The process was laborious and time-consuming, requiring a great deal of effort and time to establish initial settings for each pair of wheels. This could take anywhere from 10 minutes to an hour, depending on the individual's familiarity with the settings. For those who didn't know the settings already, it could be even more daunting, taking around 2 hours to figure them out.

Frank Carter, a member of the Bletchley Park team, estimated that it took around 10 hours of Colossus time to establish the patterns of ones and zeroes on the wheels. This was a crucial step in deciphering the Lorenz cipher, which relied heavily on statistical analysis to identify biases and anomalies. The machine's ability to process large amounts of data quickly and efficiently made it an invaluable asset in the codebreaking efforts.

However, despite its impressive capabilities, Colossus had some limitations. It relied on statistical disparities between zeroes and ones, often referred to as "bias," to make predictions about the encrypted text. This bias was often caused by language structure itself, such as the prevalence of double letters like "zz" or "ll". The German cryptanalysts recognized this weakness in the Lorenz cipher but believed it would be difficult to overcome without building a machine.

The development and deployment of Colossus marked a significant turning point in the codebreaking efforts at Bletchley Park. The machine's electronic speeds were much faster than its electromechanical counterparts, allowing it to process data quickly and efficiently. This enabled the team to analyze large amounts of data and make predictions about the encrypted text with greater accuracy.

One of the key features of Colossus was its ability to identify patterns in data. For example, if a stream of bits showed a zero followed by zeros from 2 to 5, it could be an indication that the character had been generated using an exclusive OR operation on identical characters. This was an important insight, as it allowed the team to narrow down the possible explanations for the encrypted text.

The development of Colossus also highlights the importance of collaboration and communication in codebreaking efforts. The Bletchley Park team worked closely with other cryptanalysts and mathematicians to develop new techniques and strategies for deciphering the Lorenz cipher. Their work was often informal, with discussions taking place in pubs or around firesides.

The impact of Colossus on the course of World War II cannot be overstated. The machine's ability to process large amounts of data quickly and efficiently made it an invaluable asset in the codebreaking efforts at Bletchley Park. Its contributions played a significant role in helping the Allies crack the German Lorenz cipher, which had been used by the German military for many years.

In conclusion, Colossus was a machine that changed the course of World War II. Its development and deployment marked a significant turning point in the codebreaking efforts at Bletchley Park, enabling the team to make predictions about encrypted text with greater accuracy. The machine's electronic speeds and ability to process large amounts of data quickly made it an invaluable asset in the codebreaking efforts, playing a crucial role in helping the Allies win the war.

"WEBVTTKind: captionsLanguage: enyou've got this guy built up with his team of people or the team of people who was working with and they have cracked this code and why do they need computers where did Colossus commit colossus came in because the sheer amount of counting that you had to do was enormous he were basically in the end looking for the occurrences of dots or zeros in well 41 times 31 times five times whatever you know you might also afford to get the whole research section doing just one little bit of it but day in day out all you wanted to do was to look for patterns and count out the number of zeros within them and you need a computer it doesn't matter it's a special-purpose computer which Colossus was when they first realize this they tried to go back to enigma type technology oh we understand about relays and uni selectors let's build something electromechanical it was called his Robinson and for those of you in North America Heath Robinson is the direct equivalent of Rube Goldberg they were both cartoonists that drew impossibly complicated electromechanical machines and make cartoons out of them so there's this huge amount of stuff to be counted up but electro mechanically when they tried to do it it couldn't go fast enough it would take days and they tried to speed the machines up and they just went up in blue smoke and eventually I think Alan Turing had worked with Tommy flowers from Dulles Hill GPO general post office he'd worked with them for some aspects of the Enigma decode but as we know enigma electromechanical was just about okay but Turing said winder why not let's get in flowers for an opinion and Max Newman who was by then head of resistors and bring me will have a chat and flowers took one look at it and said you will never get it fast enough to do what you want electromechanical II forget it we've got to go electronic and use valve and of course there's a huge flower its are you off your head we all know valves I go back every few minutes they're not reliable I think I've said this before I'll say it again Tommy said to them I've been doing research on use of thermionic valves in telephone exchanges and I can tell you they can be remarkably reliable so long as you never turn them off and it's particularly the heaters on the cathodes if you bring those up to voltage very quickly so they instantly go red or white hot the filament will go back but if you bring them up very carefully some dull red to bright red and all that and then at the end of the day don't switch them all off lower the voltage and do that very very carefully you will minimize the number of thermionic valve blowouts you get and so basically the message was never ever turn them off and it'll be fine and in the end it was and the electronic speeds were just about enough but it's still talk a typical run on Colossus to discover initial settings on a pair of wheels might take 10 minutes something like that and you've got to do that for five different pairs so you know you're taking about an hour to work out settings if you didn't know them already standing oldest said you must never take more than two hours if you haven't got it sorted by then on the settings give up go to another message but then if you knew the settings but didn't know the wheel patterns that was a huge amount of effort was needed in fact Frank Carter reckons 10 hours of Colossus time to establish what the patterns of ones and zeroes were on the wheels now you've realized why they ended up with 10 colossi Bletchley Park they got a huge amount of work to do and he mustn't also run away the idea that Colossus could do absolutely everything he couldn't the great majority it could but it relied on this slight statistical disparity though almost more zeroes than ones and look for that what setting makes that happen but just occasionally a rogue message would come in where it just happened to be fifty/fifty and there wasn't a skew or a bias and then you have to throw that what do I say we'll come back to that later so it wasn't 100% but it was good enough to make a decisive difference to the war yes he seems a weird doesn't it that it's not 5050 between zeros and ones in a in a regime where we're doing exclusive ORS well what you've got to remember is if you exclusive or something with itself you get a bunch of zeroes but whatever it is you exclusive all those together if they're identical the exclusive or on a character basis will be five zeros and to make those show up at Bletchley they denoted it with a forward slash if you remember okay well that's all very well but that so how would that lead to a bias a scoot and sir hmm in many many languages not the least German and not the least English which as we must remember is a Germanic language you get doubled letters ok so Sean if I say to you I'm guessing the probability of Z in English is 1/100 what's the probability of getting two Zed's 1/100 times 1/100 well that would be the mathematical ones yeah yeah if they're all independent it will be one in 10,000 but they're not doubles red is far more common in even in English less alone in German than one in 10,000 really you know dazzl puzzle all these kind of things it's not massively common as a bigram but it's more common than the base probabilities would indicate double peas as well happy slapping flappy all these kind of things so character doubling was one of the vital components of saying that if you look on a certain stream and it's a zero and you look on the adjacent streams from 2 to 5 and it's a zero as well then it's an old character and that could have been generated by having one thing exclusive Ord with it's identical thing so on a bitstream basis they adapted that and said the reason we are seeing more zeroes is that if you slide these bitstreams over each other by one base and exclusive-or them you'll find that double letters occurrences lead eventually to more zeros coming out than ones because on the nature of exclusive or if something is the same as something else and you exclusive for it it gives zeros not ones so it's a bit rough and ready and hand wavy there's more to it than that but that is just one example of how the language structure itself can do you in and it's reported that the German Crypt analysts realized that this would be a weakness of the Lorenz cipher but they said we're not to worry it would need you to build a machine and I'll never be able to do that there'd be so much data it will kill him you'd need roomfuls of people and even within a month they wouldn't do it but what they didn't foresee was the advent of machines with electronic speeds not just electromechanical ones and that could just about get on top of it will the manipulating bits era become obsolete in a few years or basics will never change I think basics will always be there in some sense we will always have to worry about how much memory we have how much processor time we have we was being all informal is that hi Daveyou've got this guy built up with his team of people or the team of people who was working with and they have cracked this code and why do they need computers where did Colossus commit colossus came in because the sheer amount of counting that you had to do was enormous he were basically in the end looking for the occurrences of dots or zeros in well 41 times 31 times five times whatever you know you might also afford to get the whole research section doing just one little bit of it but day in day out all you wanted to do was to look for patterns and count out the number of zeros within them and you need a computer it doesn't matter it's a special-purpose computer which Colossus was when they first realize this they tried to go back to enigma type technology oh we understand about relays and uni selectors let's build something electromechanical it was called his Robinson and for those of you in North America Heath Robinson is the direct equivalent of Rube Goldberg they were both cartoonists that drew impossibly complicated electromechanical machines and make cartoons out of them so there's this huge amount of stuff to be counted up but electro mechanically when they tried to do it it couldn't go fast enough it would take days and they tried to speed the machines up and they just went up in blue smoke and eventually I think Alan Turing had worked with Tommy flowers from Dulles Hill GPO general post office he'd worked with them for some aspects of the Enigma decode but as we know enigma electromechanical was just about okay but Turing said winder why not let's get in flowers for an opinion and Max Newman who was by then head of resistors and bring me will have a chat and flowers took one look at it and said you will never get it fast enough to do what you want electromechanical II forget it we've got to go electronic and use valve and of course there's a huge flower its are you off your head we all know valves I go back every few minutes they're not reliable I think I've said this before I'll say it again Tommy said to them I've been doing research on use of thermionic valves in telephone exchanges and I can tell you they can be remarkably reliable so long as you never turn them off and it's particularly the heaters on the cathodes if you bring those up to voltage very quickly so they instantly go red or white hot the filament will go back but if you bring them up very carefully some dull red to bright red and all that and then at the end of the day don't switch them all off lower the voltage and do that very very carefully you will minimize the number of thermionic valve blowouts you get and so basically the message was never ever turn them off and it'll be fine and in the end it was and the electronic speeds were just about enough but it's still talk a typical run on Colossus to discover initial settings on a pair of wheels might take 10 minutes something like that and you've got to do that for five different pairs so you know you're taking about an hour to work out settings if you didn't know them already standing oldest said you must never take more than two hours if you haven't got it sorted by then on the settings give up go to another message but then if you knew the settings but didn't know the wheel patterns that was a huge amount of effort was needed in fact Frank Carter reckons 10 hours of Colossus time to establish what the patterns of ones and zeroes were on the wheels now you've realized why they ended up with 10 colossi Bletchley Park they got a huge amount of work to do and he mustn't also run away the idea that Colossus could do absolutely everything he couldn't the great majority it could but it relied on this slight statistical disparity though almost more zeroes than ones and look for that what setting makes that happen but just occasionally a rogue message would come in where it just happened to be fifty/fifty and there wasn't a skew or a bias and then you have to throw that what do I say we'll come back to that later so it wasn't 100% but it was good enough to make a decisive difference to the war yes he seems a weird doesn't it that it's not 5050 between zeros and ones in a in a regime where we're doing exclusive ORS well what you've got to remember is if you exclusive or something with itself you get a bunch of zeroes but whatever it is you exclusive all those together if they're identical the exclusive or on a character basis will be five zeros and to make those show up at Bletchley they denoted it with a forward slash if you remember okay well that's all very well but that so how would that lead to a bias a scoot and sir hmm in many many languages not the least German and not the least English which as we must remember is a Germanic language you get doubled letters ok so Sean if I say to you I'm guessing the probability of Z in English is 1/100 what's the probability of getting two Zed's 1/100 times 1/100 well that would be the mathematical ones yeah yeah if they're all independent it will be one in 10,000 but they're not doubles red is far more common in even in English less alone in German than one in 10,000 really you know dazzl puzzle all these kind of things it's not massively common as a bigram but it's more common than the base probabilities would indicate double peas as well happy slapping flappy all these kind of things so character doubling was one of the vital components of saying that if you look on a certain stream and it's a zero and you look on the adjacent streams from 2 to 5 and it's a zero as well then it's an old character and that could have been generated by having one thing exclusive Ord with it's identical thing so on a bitstream basis they adapted that and said the reason we are seeing more zeroes is that if you slide these bitstreams over each other by one base and exclusive-or them you'll find that double letters occurrences lead eventually to more zeros coming out than ones because on the nature of exclusive or if something is the same as something else and you exclusive for it it gives zeros not ones so it's a bit rough and ready and hand wavy there's more to it than that but that is just one example of how the language structure itself can do you in and it's reported that the German Crypt analysts realized that this would be a weakness of the Lorenz cipher but they said we're not to worry it would need you to build a machine and I'll never be able to do that there'd be so much data it will kill him you'd need roomfuls of people and even within a month they wouldn't do it but what they didn't foresee was the advent of machines with electronic speeds not just electromechanical ones and that could just about get on top of it will the manipulating bits era become obsolete in a few years or basics will never change I think basics will always be there in some sense we will always have to worry about how much memory we have how much processor time we have we was being all informal is that hi Dave\n"