The Evolution of Error Correction: A Historical Perspective

As a computer scientist, I have always been fascinated by the evolution of error correction techniques in computing. In this article, we will explore one such technique that has played a crucial role in ensuring reliable data transmission: single-error-correcting codes.

I remember spending hours pouring over lines of code, trying to understand how they worked. One particular example that stands out is when I was working with a coding system used for transmitting data over radio waves. The system consisted of three-bit codes, each representing one of the four values 0, 1, 2, or 3. To add some parity to the code, I would insert an extra bit at the end, making it a four-bit code. This was done to ensure that even if there were any errors in transmission, the receiving station could detect them.

The parity check bit is where things get interesting. In order to determine whether an error had occurred, the sending station would calculate the parity of the received message by counting the number of ones present. If the calculated parity matched the parity check bit at the end of the code, then it was assumed that no errors had occurred and the message could be decoded correctly. However, if the two didn't match, it was clear that an error had taken place.

Now, let's look at a specific example of how this system worked. Suppose I sent a message "0000 01" over the radio waves. If everything went according to plan, the receiving station would receive the same message. However, suppose that during transmission, the middle zero was distorted and became a one. This would result in the receiving station receiving "0011". The parity check bit at the end of the code is where things get interesting. Since there are two ones present in the received message (the first one and the last one), the calculated parity is 2. However, the parity check bit at the end of the code is a zero, which means that even though there's an error, the system still thinks it's a valid message.

As I continued to work with this coding system, I began to realize its limitations. Suppose that during transmission, not just one but two errors occurred. For example, suppose that both the middle and last zeros were distorted and became ones. This would result in the receiving station receiving "0011", which is a valid code according to our system. However, this means that the message was actually corrupted, as it now represents the value 3 instead of the intended value 2. The problem is that our system can't correct two errors; it can only detect one.

This brings us to the concept of distance in coding theory. A distance refers to the minimum number of bits that need to be changed in order for two different codes to be received by the receiving station. In this case, since we can detect one error but not correct it, our system has a distance of 1. This means that if you flip two bits, it changes into another good code. However, if you flip only one bit, it's still discernibly garbage and you know an error has occurred.

The development of single-error-correcting codes like this has been instrumental in ensuring reliable data transmission over radio waves and other communication channels. While there are limitations to these systems, they have played a crucial role in the advancement of computing and have paved the way for more complex coding techniques that can correct multiple errors at once.

"WEBVTTKind: captionsLanguage: enat Sacramento at your listening station you hear 010 and you say that can't be right what's the simplest thing to do to try and find out what the weather really is because you know that an error has occurred we've talked about the most compact way to transmit this information there are four possible patterns you can have two zeros you could have 0 1 1 Z and one one however we've got to be careful here because what we've done and referring back to what I've said previously about compression methods what I've done here is devised a code which has no redundancy at all we've talked about how to squeeze redundancy out of text files and pictures which is what compression is is getting rid of unnecessary information there's nothing unnecessary here every one of those bits is vital and you better hope they don't get lost or distorted on the way unfortunately they might get lost or distorted on the way so the next thing we have to say is these codes are beautiful but they rely on you having clean noisess communication across your telephone wire what would go wrong if that wasn't the case just occasionally one of these zero bits that's signified by a short Boop for a dot might get turned into a one because partway to Sacramento suddenly an electrical storm occurred and just hit a tel wire so what was received at Sacramento was not it was like that a longer second bit and the people oh that's 01 it's sunny in San Francisco no it isn't it's foggy so you can see very quickly there is no possibility for recovering from error in these if you get an error people will receive apparently a good pattern but it will be for the wrong weather condition how could you add something extra to these to give them a bit of protection against noisy telephone lines a simplest way one of the simplest it's been used in telecommunications and indeed within computers for years and years now you put in a parity check bit so here's a code with a check bit and all I'm going to do here I will take every 2bit code and turn it into a three bit code but the rule will be that in the overall three bit code I must have an even number of one bits so this is called an even parity system so here we go zero zero there are no ones in that so therefore at the right hand end here I'm going to add in the parity check bit and because we're on even parity it's a zero viewers of number file will have to forgive me I think there is a video on there questioning whether zero is an even number I'm sorry I'm just a computer scientist people have a delayed reaction people are not sure whether zero is an even number that's even as far as I'm concerned okay how about this one 01 well overall it's got to be even parity which means an even number of ones so my bit I add at the right hand end is a one equally here for clouding one and zero there it is two of them 101 rainy that's easy there an even number of ones already so I put a zero at the end now those codes with the check bit they help a lot in helping to detect whether an error has occurred just suppose that when this 0000 0 was sent it got distorted again an electrical disturbance just happen to chew up one of the zero bits lengthen it and make it look like 0 one0 if any of these got distorted a n turned to a one or a one turned to a zero it looks like a valid different weather State here it doesn't because look at the valid codes three Z's 011 1 01 1 1 0 I sent this and it arrived at the other end as 01 0 is this a code that in this set no it isn't if you try with these just changing any one of the bits and flipping it either a one to a zero or a zero to a one you end up with something that isn't in this list of good valid codes you could say well there's bad codes there's things that don't correspond to any of these and of course with three bits 2 to ^ 3 is eight so there's eight possible combinations what we've done here is split them up in into four good codes that are meaningful and four bad ones but the bad ones give you a protection against errors in transmission your listening station you hear 0 one0 and you say that can't be right what's the simplest thing to do to try and find out what the weather really is because you know that an error has occurred well the simplest thing is to send back to San Francisco a simple one bit message either a notter or one you either say acknowledge in other words I got it it's a good code or not acknowledge here I would send back a not acknowledge often called a knack for not acknowledge I send back a one bit which says not acknowledged got garbage and the sending end would say okay let's keep it simple just send it again not not not next time you hope it's going to get through without Distortion so this is the basis of this system in that it can detect that a single error has occurred the only trouble is that it can't tell you where the error occurred just consider the following I have got 0 one0 here by changing the middle zero into a one but it could have happened that it came from this one via Distortion it could have been that the bit that got distorted was not that one or that one but that one that trailing one bit might have got turned back into a zero there's an error but I don't know whether it came from this one or from this one so this kind of code then can detect one error but it can't correct it it can't tell you where it occurred suppose I had a even noisier line and that in one of these three bit codes you hope most of them are going to go through cleanly you hope that the ones that aren't clean will have at most one error in them what happens if I send one Z1 and there are two errors in it not just one the errors just so happen that it turns a long blip into a short one there and there that would then give you Z 0 Z two errors oh Calamity again this is received at the receiving station they say hey fantastic 0000 0 it's foggy in San Francisco no it isn't it's cloudy but it got two errors so it looks like it was foggy so the point here is it's resilient against one error it can detect it it can't correct it but against two errors there isn't resilience it distorts it into a good-looking code again so for this reason this system is called a distance to code because what it's saying is if you flip two bits it changes in into another good code if you flip one bit it's discernably garbage and you can say an errors occurred it's 80 megabytes and let me say again 80 megabytes not 80 gigabytes and it's in this huge package combination of Art and Science that goes into making a game is an extraordinary thingat Sacramento at your listening station you hear 010 and you say that can't be right what's the simplest thing to do to try and find out what the weather really is because you know that an error has occurred we've talked about the most compact way to transmit this information there are four possible patterns you can have two zeros you could have 0 1 1 Z and one one however we've got to be careful here because what we've done and referring back to what I've said previously about compression methods what I've done here is devised a code which has no redundancy at all we've talked about how to squeeze redundancy out of text files and pictures which is what compression is is getting rid of unnecessary information there's nothing unnecessary here every one of those bits is vital and you better hope they don't get lost or distorted on the way unfortunately they might get lost or distorted on the way so the next thing we have to say is these codes are beautiful but they rely on you having clean noisess communication across your telephone wire what would go wrong if that wasn't the case just occasionally one of these zero bits that's signified by a short Boop for a dot might get turned into a one because partway to Sacramento suddenly an electrical storm occurred and just hit a tel wire so what was received at Sacramento was not it was like that a longer second bit and the people oh that's 01 it's sunny in San Francisco no it isn't it's foggy so you can see very quickly there is no possibility for recovering from error in these if you get an error people will receive apparently a good pattern but it will be for the wrong weather condition how could you add something extra to these to give them a bit of protection against noisy telephone lines a simplest way one of the simplest it's been used in telecommunications and indeed within computers for years and years now you put in a parity check bit so here's a code with a check bit and all I'm going to do here I will take every 2bit code and turn it into a three bit code but the rule will be that in the overall three bit code I must have an even number of one bits so this is called an even parity system so here we go zero zero there are no ones in that so therefore at the right hand end here I'm going to add in the parity check bit and because we're on even parity it's a zero viewers of number file will have to forgive me I think there is a video on there questioning whether zero is an even number I'm sorry I'm just a computer scientist people have a delayed reaction people are not sure whether zero is an even number that's even as far as I'm concerned okay how about this one 01 well overall it's got to be even parity which means an even number of ones so my bit I add at the right hand end is a one equally here for clouding one and zero there it is two of them 101 rainy that's easy there an even number of ones already so I put a zero at the end now those codes with the check bit they help a lot in helping to detect whether an error has occurred just suppose that when this 0000 0 was sent it got distorted again an electrical disturbance just happen to chew up one of the zero bits lengthen it and make it look like 0 one0 if any of these got distorted a n turned to a one or a one turned to a zero it looks like a valid different weather State here it doesn't because look at the valid codes three Z's 011 1 01 1 1 0 I sent this and it arrived at the other end as 01 0 is this a code that in this set no it isn't if you try with these just changing any one of the bits and flipping it either a one to a zero or a zero to a one you end up with something that isn't in this list of good valid codes you could say well there's bad codes there's things that don't correspond to any of these and of course with three bits 2 to ^ 3 is eight so there's eight possible combinations what we've done here is split them up in into four good codes that are meaningful and four bad ones but the bad ones give you a protection against errors in transmission your listening station you hear 0 one0 and you say that can't be right what's the simplest thing to do to try and find out what the weather really is because you know that an error has occurred well the simplest thing is to send back to San Francisco a simple one bit message either a notter or one you either say acknowledge in other words I got it it's a good code or not acknowledge here I would send back a not acknowledge often called a knack for not acknowledge I send back a one bit which says not acknowledged got garbage and the sending end would say okay let's keep it simple just send it again not not not next time you hope it's going to get through without Distortion so this is the basis of this system in that it can detect that a single error has occurred the only trouble is that it can't tell you where the error occurred just consider the following I have got 0 one0 here by changing the middle zero into a one but it could have happened that it came from this one via Distortion it could have been that the bit that got distorted was not that one or that one but that one that trailing one bit might have got turned back into a zero there's an error but I don't know whether it came from this one or from this one so this kind of code then can detect one error but it can't correct it it can't tell you where it occurred suppose I had a even noisier line and that in one of these three bit codes you hope most of them are going to go through cleanly you hope that the ones that aren't clean will have at most one error in them what happens if I send one Z1 and there are two errors in it not just one the errors just so happen that it turns a long blip into a short one there and there that would then give you Z 0 Z two errors oh Calamity again this is received at the receiving station they say hey fantastic 0000 0 it's foggy in San Francisco no it isn't it's cloudy but it got two errors so it looks like it was foggy so the point here is it's resilient against one error it can detect it it can't correct it but against two errors there isn't resilience it distorts it into a good-looking code again so for this reason this system is called a distance to code because what it's saying is if you flip two bits it changes in into another good code if you flip one bit it's discernably garbage and you can say an errors occurred it's 80 megabytes and let me say again 80 megabytes not 80 gigabytes and it's in this huge package combination of Art and Science that goes into making a game is an extraordinary thing\n"