**Introduction to AWK: A Powerful Command Line Tool**

AWK is a powerful command line tool that allows users to process and manipulate text data in various ways. It is commonly used for data processing, filtering, and transforming large datasets. In this article, we will delve into the world of AWK and explore its capabilities, usage, and features.

**Basic Usage of AWK**

AWK stands for "Aho, Weinberger, and Kernighan," named after its creators. It is a natural extension of the C programming language and is widely used in various Unix-like operating systems, including Linux and macOS. To start using AWK, we need to create a text file that contains data we want to process. Let's assume we have a file called `test.txt` with the following content:

```

Hello World!

This is a test file.

It has multiple lines.

```

To use AWK, we open the single quotes and start our regular expression. We enter the mode we want to use, which is usually `s` for substitute, meaning we want to substitute the next word behind the forward slash with the word after the second forward slash. In this case, we want to replace the word "World" with "Universe". The `G` flag stands for globally, indicating that we want to do this substitution across the entire text file.

**Using AWK to Replace Text**

To perform the replacement, we type the following command:

```

awk '{print $1, " Universe"}' test.txt

```

Here, `$1` refers to the first word in each line of the input file. The `{ }` block defines the action to be taken on each line. In this case, we simply print the first word followed by " Universe". The output will be:

```

Hello Universe!

This is a test file.

It has multiple lines.

```

**Using AWK with Space Separators**

In our previous example, we used spaces as separators between words. However, in many cases, we need to use other characters as separators, such as commas or tab characters. To do this, we can add the `F` flag followed by the separator character. For instance, let's assume we have a CSV file called `test.csv` with the following content:

```

Name,Age,Country

John,25,USA

Alice,30,UK

Bob,35,Canada

```

To use AWK to split this CSV file into separate lines based on commas, we type the following command:

```bash

awk '{print $0}' test.csv | awk -F, '{print $1 " " $2 " " $3}' -

```

Here, the first `awk` command uses the `-f` flag followed by the separator character, comma. The second `awk` command splits each line into three separate fields using spaces as separators.

**Using AWK with Pipes**

AWK can also be used in conjunction with pipes to process multiple commands in a single pipeline. Let's assume we have a file called `test.txt` with the following content:

```

Hello World!

This is a test file.

It has multiple lines.

```

To use AWK and pipe commands together, we type the following command:

```bash

echo "Just get this word: colon" | awk '{print $1}' | awk -F: '{if ($1 == 5) print $2}'

```

Here, `echo` outputs the string "Just get this word: colon". The first `awk` command uses the single quote to split the input into separate words. The second `awk` command uses the `-F:` flag followed by a colon to split each line into two fields using spaces as separators.

**Using sed instead of AWK**

sed (short for "stream editor") is another powerful command line tool used for text manipulation and processing. While both AWK and sed can be used for similar tasks, they have different syntax and use cases. To replace values in a text file using sed, we type the following command:

```bash

sed 's/fly/grasshopper/' test.txt

```

Here, `s` stands for "substitute," and `fly` is replaced with `grasshopper`.

**Using set instead of AWK or sed**

set (short for "search and replace") is another powerful command line tool used for text manipulation and processing. While both AWK and sed can be used for similar tasks, they have different syntax and use cases. To replace values in a text file using set, we type the following command:

```bash

set -i test.txt

```

Here, `I` stands for "ignore case," which means that the replacement will not be case-sensitive.

In conclusion, AWK is a powerful command line tool used for data processing, filtering, and transforming large datasets. Its basic usage involves creating a text file, opening single quotes, entering the mode we want to use (usually `s` for substitute), and defining the action to be taken on each line using curly brackets. We can also use AWK with pipes to process multiple commands in a single pipeline.

While both AWK and sed can be used for similar tasks, they have different syntax and use cases. set is another powerful command line tool used for text manipulation and processing, often replacing values in a text file without the need for quotes or curly brackets.

"WEBVTTKind: captionsLanguage: enWelcome to this crash course onbest scripting. Throughout thiscourse Herbert will teach you awide range of topics, includingbasic commands, writing yourfirst bash script, working withvariables, mastering controlstructures, and diving intopowerful text manipulation toolslike awk and sed, Get ready tounlock the full potential ofbash scripting and enhance yourproductivity by automatingtasks, streamlining processes,and making your workflow moreefficient. Hello, everyone, andwelcome to this introductioncourse to bash shell scripting.This course is aimed at anyonewho is new to shell scriptingpeople who want to expand theirknowledge or for those who wantto refresh their knowledge. Myname is Herbert and I will beyour instructor for this course.I've been in the IT sector as asystem engineer since 2009. I'vedealt with multiple systemsranging from Windows Server toLinux servers, and in the lastfew years, I startedspecializing in automation anddevelopment. I've been doingYouTube content creation since2017. Up until now, and I'vealready created a course onLinux, which might be a goodstarting point. If you'recompletely new to Linux, I wantto learn more about the basicsof the operating system itself.Now what is bash, bash is shortfor Bourne again, shell, itreplaced the Bourne shell in theglue slash Linux project, whichwas the default shell for Unixoperating systems. A shell is auser interface for ease of use,it makes it easy for the user tomanage the operating systemwithout knowing all of the innerworkings and complexity of theoperating system itself.Now why would you learn bash?Well, the reason you want tolearn Bash is because it'scurrently the most popular shellscripting language for the Linuxoperating system. Bash has beenaround forever it has been usedsince the early days of Linux,and it has stood the test oftime ever since. It's alsoincluded in Mac OS and inwindows when you're using theWindows subsystem for Linux.Now, why would you not use bash?Well, even though Bash isconsidered a programminglanguage, it obviously lackssome features. For example, theability to do object orientedprogramming is not available inbash. Python would obviously bethe preferred choice here ifyou're looking to create moreadvanced scripts that allowobject oriented programming.Speaking of Python, bash is alsomore complex in its syntax. Whencomparing Python syntax to bashone could say that Bash is a lotharder to read and write. Notonly Python is often a betterchoice for more advancedscripting. Newer tools likeAnsible also make it a loteasier for you to managemultiple systems, which is a lotharder to do with bash. Ofcourse, Ansible in Python areoften overkill for simplescripts and using one does notexclude the other. Bash is oftenused in tandem with tools likeAnsible, or integrated in Pythonscripts. This is exactly thereason why you should learnbash, just knowing the basicscould mean a big difference in alot of situations. Sometimes,the old trusty tools are stillthe most valuable. What will youneed for this course, I aimedthis course at the standardWindows user. So I built thecourse around the windowssubsystem for Linux, aka WsL.Since we will be only usingcommand line tools and no GUIWsL seems like the obviouschoice. And I also have a videoexplaining how to install WsL onWindows. If you're running atrue Linux distro, even better.If you're using a Mac, you canfollow along in your terminalwindow. But first make sure thatyou have bash set as yourpreferred shell. smalldisclaimer basic Linux knowledgeis required. If you'recompletely new to Linux, or evennew to bash, I would suggest youat least follow along with theLinux guide that I have on mychannel is going to give you agood basic understanding of howthe Linux operating systemworks.Before we start with actualscripting, let's first do somebasic commands in the terminal.Let's look at an example of twovery simple commands to displaytext in the command line. Thefirst one is called Echo, whichwill display the text you passit as an argument. We will writethe command echo and then wepass it some text like so wejust type in Echo Hello. And sowe can detect this by looking atEcho as the command and hello asthe argument. We will learn moreabout arguments later in thiscourse. In this case, we talkabout a positional argument.Now we can see the output thatsays hello, which is the word wegave it as an argument.Let's try and see how anothercommand the cat command comparesto this. The cat commanddisplays the contents of a file.But right now we don't have one.So let's first create a filewith our text editor Vim. Inthis course, we're always goingto be using vim because it's agood text editor that comes withalmost all Linux distributions.Plus, it's actually not thathard to use if you get the hangof it. I myself have writtenmultiple scripts in bash.In Python, with Vim, so it'sdefinitely a good text editorunless you want to start writingsome serious scripts orsoftware. In that case, a decentGUI text editor like VS code,atom, or Sublime, or betteroptions.Let's create a new file withthem by typing vim text file.txt, and let's write somethingin it. To write something inVim, press the I button, and youwill enter the insert mode.From this mode, you can startwriting text. Let's write helloworld in this file, now we pressthe Escape button. And you willsee that the insert mode is nowgone, we have exited the insertmode and we're back into commandmode. And once you're back intothe regular command mode, youcan press the colon W to writethe file. And this will makesure the changes we made areactually done and saved to thedisk. And then we can press Q toexit the file. Also, to makelife simpler, we can go intoVim, make some changes andcombine the colon W and call onQ to write colon wq. This willautomatically write the file andquit vim at the same time.If you made any changes youdon't want to save, you canignore the changes by typingcolon Q, exclamation mark, whichwill quit vim while ignoring anychanges made.Now that we created our file,let's try to print the contentsof the file with the catcommand. This command will printout whatever's in the firstpositional argument, in our caseis the name of the file we justcreated. So we just type in cattxt file a TXT, and we shouldsee the contents of the fileprinted here. Okay, great. Wecan now print our own text andalso text files to the terminal.This may not sound like much,but it's something that you willuse a lot when writing bashscripts.So now finally, we are going towrite a shell script with theknowledge we just gathered inthe previous section. Start offby first opening your favoritetext editor, and named the textfile shell test.sh, like so. Vimspace, shell test.sh.Now that we have our text editoropened, let's try another modeof writing text in Vim, theAppend mode, instead of pressingi On your keyboard, press the Abutton. This will append yourtext rather than inserted. Withthe insert mode, you write thecharacter before the cursor withthe Append mode, you write thecharacter after the cursor. Nowin this case, it doesn't matterof course, because it's a newfile, but I thought I wouldmention it anyways, sincesometimes it can come in handy.Let's not start with our shellscript. In Vim, write echo HelloWorld, and write and quit thefile by pressing the insert orappend mode, writing echo HelloWorld, and pressing the Escapebutton and then writing colonwq. Now let's see if we canactually find the file in hereusing the ls command, which willlist all the files in ourcurrent directory. As you cansee, the file is there and wecan verify the location bylooking at the terminals currentlocation over here, which saysthe tilde symbol and the tildesymbol indicates that we are inour current home directory. Bytyping P WD. We can verify thatwe are indeed in the homedirectory. pwd stands for printworking directory. And this waywe can print the directory thatwe are currently working in,which is indeed our homedirectory.Now let's run our file byproviding the command line withthe command line interpreter.Which is bash space, shelltest.sh. This will output thecommand we entered in the shellscript using the bashinterpreter. But when we type inEcho dollar sign shell, we canverify that we are actuallyrunning in a bash shell. Soshould we really have to specifyit? Well, the short answer isyes. Just because we are runningin a bash shell, that does notmean that Linux is going toassume every file we want to runis also in bash. We shouldalways provide an interpreterfor our shell scripts to use thecorrect type of shell.That does not mean however, thatwe shouldn't make it easier forourselves. We can actuallyprovide the command lineinterpreter in the shell scriptitself by typing it at the topof our script. In our case, wehave to provide the full path tothe shell interpreter. Let'scopy our output from the echodollar sign shell command anduse it in our shell script.Copy the output by selecting itand using either Ctrl C or rightclicking the selection either itshould work.Now let's go back into our shellscript and let's enter theinsert mode. Since we want toenter all this information inthe beginning of the file, pressI am type in pound signexclamation mark and then pastein the output of the echoDollar Sign shell command. Andwhat we do here is we providethe shebang. So the pound signexclamation mark is a way oftelling the shell script whichinterpreter to use, we follow upthe shebang by providing thefull path to the shellinterpreter. And this way, theshell knows which type of shellinterpreter it needs to use, wecan now run this file doingperiod forward slash shelltest.sh. But we will see thatpermission is denied and we needto give the file permission tobe executed. This is because inLinux, we need to give filespermission to act as anexecutable file. Let's have alook at this when we type in LSminus l, which is a long formatof the ls command, which isanother type of argument we calla flag. So the minus L here iscalled a flag we give as anargument to the ls command,we're actually asking the lscommand to provide the longformat, which will display somemore information about the filesand directories, we can see inour permissions that we do nothave executable permissions yet,if you want to learn more aboutthe basics of file permissions,you should look at my Linux forbeginners tutorial.Let's give our file executablepermissions by typing in ch, moD or smart u plus x space shelltest.sh. We want to provide uplus x because this will giveonly the owner user permissionto execute the file rather thanthe entire group or anybody onthe system.It's good to have thisdiscipline baked into your head,although on a developmentenvironment, it doesn't reallymatter very much.Okay, now we can finally run ourshell script by typing in periodforward slash shell test.sh. Andwe will see the output on theterminal. Now that we finallyhave the basics out of the way,we can actually start writingsome bash scripts.To understand a little bitbetter what variables do inbash, have a look at thisscript. So in this script, youhave a classic example of a copycommand where we copy somethingfrom to a location underneaththat we have the same locationpaths twice typed outcompletely. Now, this issomething that you want to avoidas much as you can. Underneaththat we have a better optionwhere we actually define alocation path in a variablecalled my location from now thisis quite a long variable name,you could actually just definesomething like loc underscorefrom, which would be betterto make it a little bit shorter.But it's just you know, it'sjust for demonstration purposeshow this would work. So youcould understand if you wouldhave a very long path here, wecould have that very long filelocation stored in a variable.And then you could actuallyreuse that variable over andover again, like we did in theexample below with that. Solet's now have a look at how wecan actually use this in anexample. So we can either typeour name like this first nameequals, let's say Herbert. Andthat's how we can actually usethese variables by typing likeEcho Hello, and then typing thevariable like so. So that's howit would work like in line or inthe command line itself. But wecould actually also just write ascript to demonstrate a littlebit better how variables work.So just type in Vim, hello,there.sh.And then we'll do the insertmode with the shebang forwardslash bin forward slash bash.And then over here, we'll defineour first name equals pervert oryou know your name. And thenlast name, equals lindemans,which is my last name. And thatwill just do like this echo.Hello, dollar sign first name.And then we'll do dollar signlast name.And then we'll exits with wq.And now, the only thing we needto do is Srimad u plus x. Hellothere.sh. And now we can run it.And now we can see that our nameis printed out here. Just likethat.So now we actually type thisstatically. But we can actuallyalso ask the user for input,like so. So let's open a newshell script with Vim.Interactive shell.sh will do thesame insert, they'll do theshebang bin bash. And first ofall, we'll need to ask the userlike, What is your name? So whatis your name? What is your firstname? Let's ask. And then we'lldo a read statement. I will readthe first name. I will do thesame with what is your last nameand I will read last name and Iwill echo HelloOh,dollar sign first name,space dollar sign last name.And then we'll write and quit.And then we'll do the classicsmart u plus x. And theninteractive shell.sh. I will runit and now it's going to promptus for our first name, or lastname, and then it will justprint out our first and lastname back. Sowe could have the terminaldisplay our own name, oranything else provided in apositional argument as well. Butfirst, what is a positionalargument? A positional argumentis exactly what it sounds like,we place the argument at acertain position behind thecommand or script we want torun. The positional argument canbe in position, 123, and soforth. The positions are alwaysseparated by a space. Whenworking with positionalarguments, you expect argumentsto be in a certain position thatis separated by a space andcounting from one position zerois reserved for the shellitself.Now let's have a look at one ofthese positional arguments.Create a new script by typingvim pulse ru.sh. And enter theinsert mode again, we will havethe script taking our first andlast name and position one andtwo and have it take these wordsas positional arguments. Let'sjust write our shebang and writeour one line of code like so.shebang, Bin bash, Echo, hello.Dollar sign $1 sign two. Nowsave the file and close Vim,change our permissions. Andlet's run the script again withthe positional arguments dotforward slash POS argue that sh,herbert lindemans, or whateveryour name is here. And as youcan see, we get the same outputhere with only one line of code.One of the most frequently usedfeatures in Bash is piping, itmakes it very easy to perform aspecific action on the output ofthe command. Let's say theoutput of a command is very longand you wish to filter somethingout. Piping is going to give youthat option. Now how does pipingwork? Let's have a look at anexample.Let's say we want to look at aspecific directory. But we justwant to filter out certain filesor subdirectories. We could forexample, use a command likethis. So we type in LS minus Lforward slash our directory. Inthis case, we're using the bindirectory at forward slash userforward slash bin and then spaceand then we type the pipingsymbol SPACE grep, which issomething that we use to filterout specific words and then wetype in bash. And we'll see theoutput is just the bash binariesthat we can see here. Ratherthan the entire bin directory.In this command, we forward theoutput of the ls command intothe next command with the pipesymbol. This symbol represents apipe and this is what we callpiping. We literally sent theoutput from a command before thepipe symbol to the commandbehind the pipe symbol. grep isjust an example there are manymore commands that have verypowerful potential when usedwith piping.When sending output to a file,we have to use different kindsof symbols than the pipe symbol.Instead, we use the greater thanand double greater than symbols.These symbols will be used tosend the output from a commandto a file. This can have manypotential use cases, the firstthing that comes to mind islogging something from yourscript to a log file. And thisis something that you'lldefinitely use in real lifescenarios. Let's see how we canuse these symbols. First, wecatch the output of a simpleecho command. Start by typing inEcho hello world greater thanhello dot exe. This prints helloworld into a file called hellodot txt. Notice that we put thecommand before the greater thansymbol and the name of the textfile after the greater thansymbol. Now we can see if thiswork by typing in cat hello dottxt and we should see the outputof the echo command in here. Nowlet's try something differenthere typing echo Good day toyou. Greater than hello dot txt.And let's see what's the hellodot txt file by typing in cathello dot txt. But what is this,the file was overwritten. Wedon't have our hello worldanymore. This is because thegreater than symbol alwaysoverwrites any existing file.What we need to have is a properway to append text rather thanoverwrite it. This is where thedouble greater than symbol comesin. This will append any outputto our destination file ratherthan overwrite it. Let's try itout. But first let's remove thatoriginal file with RM spacehello dot txt. Then we use Echohello world double greater thansymbol hello dot txtWe check if the contents areindeed correct with cat hellodot exe. And indeed, the text isthere. And once we verifiedthis, we sent another output tothe file using the doublegreater than symbol and type inEcho Good day to you doublegreater than hello dot exe. Andwe once again verify this outputwith cat hello dot exe. And weshould see both outputs here.Now imagine using this incombination with timestamps tosee which command ran at whichtime and also some errorhandling. We're starting tocollect the tools we need towrite real script, we can alsofeed input into a command byreversing the greater thansymbol, we can use the lesserthan double lesser than or eventriple lesser than symbol to getinput from a file, multiplelines of text or a single stringof text. Let's first look at thelesser than operator, which willget input from a txt file. Wewill use the word count commandas an example here, because it'sone of those commands that has ause case for these types ofoperators. Let's type in WCminus w hello dot exe to get theword count. But what happenshere, we get a little more thanwe asked for as an output. Thewc command, by default alsoshows the filename, which is notwhat we want. If we just want toget the number of words, we needto feed the wc command and inputfrom the file with the lesserthan operator rather than thefile itself as a positionalargument. This can be done likeso WC minus w space lesser thanspace hello dot txt, and thisway, we will redirect the fileto the wc command. Rather thanpassing it as a positionalargument to the command, we willnow get to the second way offeeding data to a command withthe double lesser than operator.This is a great way to supplymultiple lines of text to acommand. When we write thedouble lesser than operator, wewill immediately follow it upwith a word that will open andclose the text we want to input.Basically, we tell the commandline Hey, wait for this word,and then capture everything Iwrote in between the first andsecond occurrence. Often peoplewill write e o f, but you canwrite whatever you want. Let'shave a look at how this wouldwork. So we type in cat space,double lesser than symbol elf.Now we see that the command linewaits for input. So we can keepwriting lines of text here. Andlet's write some text creating anew line with the enter keywe'll type in I will enter writesome enter text here, enter. Andnow we close it with the firstword we supply it and we typeagain e o f. And now we shouldsee the text that we wrote herebetween the two e o fstatements. Now last but notleast, we can also supply singlestrings of text to the commandline. This will be done with youguessed it the triple lesserthan operator. Let's have a lookat another example of the wccommand we use. By default, wcwill either read a file or acommand output, but not actualstrings supplied as positionalarguments. We should feed thecommand either an entire file orthe output of a file with one ofthe lesser than operatorvariations. Let's see how we canprovide a string to this wccommand. So we typing w c minusw again, space, triple lesserthan symbol, and then we openwith double quotes Hello, theirword count. And then we closeagain with double quotes and weshould see the word countprinted here in the terminal.This is how we can feed a stringinto a command and take notethat the string needs to be inbetween double quotes otherwiseit will not work.In bash, we have a built incommand called test that willtake in a couple of argumentsand show you if the expressionis true or not. For example, wecan ask a terminal to show uswhether or not a string of textis equal to another. We canwrite test followed by theexpression, but I prefer usingsquare brackets. Let's see howthat's done. So we just writeopen square brackets space. Andthis is important because youneed the space in between Hellospace, equal sign space againHello, followed by another spaceand enclosing the squarebrackets. And this will show nooutput but when we do echodollar sign question mark. Weshould see output zero herebecause the first string of textis equal to the second and thatwill give us a return value of$0 sign question mark returnsthe value of the exit code orthe last executed command. Wewill learn more about exit codeslater. For now it's enough toknow that exit code zero meansthat the command was executedwithout any issues. Anotherexample of this would becomparing numbers with eachother. For example, againWe open the square bracketsspace one equals zero space, weclose the square brackets, andthen we do echo dollar signquestion mark. And this shows usthat the values are not the samebecause we get a return codeone. But we could also use adifferent operator here to makesure that the values areactually numerical. We could usethe minus EQ operator for this.And we do this with openingsquare brackets, one minus EQ,one closing square brackets,Echo dollar sign question mark.This also shows us the samereturn value exit code zero,which means that the values doequate, though this will throwan error if you would usealphabetical characters insteadof numerical ones.Now for some more powerfulstuff, we can actually use thesetest expressions and perform anaction on it. Let's have a lookat this more closely with anactual script. Let's create alogin script which will notreally log us in but it willshow us what is possible withthese if Elif else statements.Create a new script called ifElif else.sh.Type in the shebang.Do F open the square brackets,dollar sign, open curly braces,one comma comma close curlybraces and I'll explain to youwhat this means in just a secondequals your username, close thesquare brackets, semicolon,then, and then we follow up whatwe actually want to do here. Sowe want to echo Oh, you're theboss here. Welcome.Then start a new line. lf opensquare brackets again, dollarsign, curly braces, one commacomma curly braces equals help.Close the square brackets,semicolon, then echo, just enteryour user name data. Else. Echo.I don't know who you are, butyou're not the boss of me. Andthen close it with FY. Now letme explain to you what we didhere. So this script will takein a positional argument, whichis a great way to test how theif statements will work.We start our if statements withan F followed by the testexpression. Take note of thesquare brackets used in previoussection, we test if the firstpositional argument is equal tothe value we provide. Take noteof the double comma and thecurly braces. This is called aparameter expansion. And it willallow us to ignore upper andlower cases when comparing thetwo values. We then end the testoperator with a semicolon. Andwe'll follow it up by then. Andthen beginning a new line. Andon this line, we type what weactually want to do with settest expression. In this case,we just print something out tothe console telling the user thescript has recognized the username and that he is indeed theboss of the system. We can alsohave a script check for multipletest expressions. We can definethis with an elif statement.This stands for else if beingthat else if the previousstatement wasn't true test ifthis is true. Again, start a newline. We defined a testexpression that will test if thepositional argument is help. Andif it is, we display a messagethat shows us instructions onhow to use the script. Being aneasy script we want to statethat it isn't so hard to use.Maybe a little rude, but it'sall in good fun. Then if none ofthese statements are true,finally we have the elsestatement. This will perform anaction if none of the if or Elifstatements are true. We candisplay a message saying thatthe user name was not detectedand that the system will not bebossed around by someone thatdoes not have the correctusername.Finally, to close our ifstatements we ended with FYI.This closes the entire block ofif Elif else statements and isrequired.If Elif else statements areuseful, but you won't use themvery often if you want to checkfor multiple values. For thesekinds of things. We use casestatements. First of all, whatis a case statement? Well, acase statement performsdifferent actions depending onwhich case is true. This meansthat we can define multipleoptions, and if the optionmatches the case, we willperform an action bound to thatoption. Let's have a look at howthis works in a real script.Let's try to recreate ourprevious exercise with a casestatement instead of the if Elifelse statements, create a newfile with vim login.sh and typein the shebang. And then westart our case statement bytyping in case space. Dollarsign and thenie variable expansion with thecurly brackets, one comma,comma, space in and over here,we're going to do somethingspecial, we're going to definemultiple options. Now this issomething that you could also dowith if statements. But I'm justintroducing it here with thecase statements. Since I thoughtit would be interesting to seehow it works. So we type in ourusername, space, and then thepipe symbol. And over here, thisis actually like a separator formultiple options, rather thanthe pipe symbol that we sawearlier, to pipe things throughtwo different commands. In acase statement, it serves adifferent function over here, itserves as a separator formultiple options. So user name,space, pipe, symbol, space, andthen another user name, forexample, the administrator, andthen we close the bracket. Andthen we define what we want tosay or do here. So you want tosay something like, Oh, you'rethe boss here, welcome, again,like we did in the previousexercise, and then we close thisoption with double semicolons.Then we define another optionhere, for example, help like wedid in the previous exercisewill copy the same echo commandthat we close that option withdouble semi colons. And thenwhat we do here is we definethis star symbol or asterisk orwhatever you want to call it.This is the catch all optionthat we're going to define. Andthe catch all option is like thedefault option, when none of theoptions are true, or when nooptions are given. And then wedefine Hello there, but you'renot the boss of me, please entera valid username, then we closethat option with doublesemicolons. And then close thatwith E sec. So it's case justreversed. So again, like I saidbefore, notice that we have twooptions in one line using thepipe symbol, again, and when weuse the pipe symbol in one ofthe case statement options, wecan use it to have multiplevalues to test for, we defineour option for help. also definea catch all or default option.And this is equal to like theelse statement like we sawearlier, meaning that if none ofthe options are equal to theinput, we will perform aspecific action and now we canrun our scripts and get theresults that we expect.Before we get into looping, weneed to first extend ourknowledge about variables, wecan actually assign multiplevalues to one variable collectedin a list. We call these kindsof variable lists arrays. Let'ssee how we can actually createthis in the command line. In thecommand line type my first listequals open brackets one, spacetwo, space three, space four,space five, close the brackets.Take note of how we start thearray with open brackets, use aspace as a separator for eachitem, and close the array withthe closing bracket. Now let'ssee what the output is when wetry to print this array. Justtype in Echo, and then just likewe would with any othervariable, space, dollar sign myfirst list and now we will seethat the output is just thefirst element. And this is ofcourse not what we want. To echothe entire list, we need tospecify it like this echo,dollar sign, open curly braces,my first list square brackets,apt symbol, closing squarebrackets, close the curlybraces, and this will print outeverything in the list. But wecan also define which item wewant to print like this echospace, dollar sign curly braces,my first list square brackets,and then over here we'll justtype in zero, which is theindex, close the square bracketsclose the curly braces, and thiswill print out the first item inthe array because we define theindex in between the squarebrackets. And because in bash,the first item in the arrayalways has index zero countingupwards we will see the firstitem in the array. But let's saywe want to do something elsewith all these items in thearray. We can do this with a forloop.Now that we have created ourarray, we want to loop over theitems in it and do somethingusing a for loop. In ourexample, we will count thelength of each word in the arrayby piping the item output usingecho to the wc commanddon'tGet to specify the minus n flagin our echo command, becauseotherwise it will also count thenewline characters.type in the command line forspace, item space in dollar signcurly braces, my first list,square brackets at squarebrackets, curly braces, semicolon space, do space, echospace minus n space, dollar signitem, space pipe symbol SPACE,wc space minus c semi colonspace done.Take note of how we write thefor loop here, we first definethe variable item, which willrepresent each item in the arrayduring the loop. Then we definethe iterable, which is ourarray. And what shall weactually define it in our forloop, we encapsulate it in ourcurly brackets and add thesquare brackets, add squarebrackets after it to make surewe loop over the entire array.Then we follow it up with a semicolon space, then we type in doand then we type in what weactually want to perform here.So we'll type in do echo minusn. As I said the minus n is justa flag for the echo command toignore all the newlinecharacters and then space dollarsign item, because the dollarsign item will represent one ofthe items that we are currentlylooping over space and then thepipe symbol, WC minus c. Andthen we do again semicolon, andthen we'd have been done.To understand why we needfunctions in a bash script, weshould first consider this. Insome scenarios, we have a lot ofrepeated code, we might want todo a specific set of commands ina certain order. Or we want torun that through a set of ifelse statements multiple times.For these scenarios, we createfunctions. These functions arelike little programs within yourscript that you can run fromwithin your script. It makes itso that you can make your codereusable very powerful stuff.Because we can save so much timeand lines of code. Let's writeour first function in bash.Let's open up a new file calledfirst function that sh i willtype in the shebang and then anew line we'll type in the nameof the function and we'll justcall this show uptime. And thenwe open and close the brackets.And then we open the curlybrackets will type in up equalsdollar sign open brackets uptime minus p pipe cut minus Cfor minus and then to close thebrackets. We'll type in sinceequals dollar sign open bracketsuptime minus s, close thebrackets and again, if I'm goinga little bit too fast for you,you can pause the video and I'mgonna go over what we actuallydid here, I'm going to dissecteverything we did here in just asecond. But then a new lineagain cat and then we'll use thedouble lesser than symbol elf, Iwill type a new line I will justcreate something pretty here. Afew dashes, I will type in thismachine has been up for dollarsign curly braces up curlybraces, it has been runningsince dollar sign curly bracketssince curly braces that a newline of then we do a new line ofdashes. Then we end ourstatement with elf, close thecurly brackets and then we justtype in show uptime. So let'sdissect this again. First, wedefine the name of the functionfollowed by open and closebrackets followed by the curlybrackets. To define what ourfunction does, we enter all thethings that needs to do inbetween the curly brackets. Inour case, we will show theuptime of the machine. Watch howhe catched the output of acommand into a variable usingthe dollar sign in bracketsnotation, then we do this againin the next variable. We defineda variable in the same way. Butwe use different flags for theuptime command and store thatoutput in the since variable andwe can now create a nice outputusing these variables using thecat command and the redirectionwe learned earlier in the coursewe end by closing the curlybrackets, and then we call thefunction by just typing itanywhere in the script. As longas it's typed under the functionit's fine, because we need tomake sure that the functionfirst is defined.When we define a variable in ourfunction, it's available to theentire script by default, thismight seem okay, but it canactually cause trouble when youwrite bigger scripts. We want todefine the variables inside ourfunctions as local variables sothat they are only available toour function and not to theentire script. This way, we willavoid accidentally overwritingglobal variables, which mightcause issues when we are usingfunctions that share thesevariable names. Let's have alook at what happens when wedon't use local variables. Solet's define our variables namedup equalsBefore since equals function,and then we'll just print thoseout before we define thefunction. So we'll just echo upand echo since. And then now wewill define the show uptimefunction. Again, we will leaveit the way it is, and then echoup an echo since, and let's runit and in this example, we cansee that the variables areindeed globally available, whichis not what we wanted. We'll seealso that they are globallyavailable, but we have actuallyoverwritten the variables thatwere defined before thefunction. And then the functionwas actually reassigning thosevariables because they are notassigned as local variables.Let's fix that. So the onlything we need to do to fix itreally is just add local beforewe define the variable in ourfunction, so instead of justwriting up, and since equals thecommands that we want to use,before the up, we'll just enterlocal space. And then the samegoes for since we'll definelocal space. And when we runthis, we can now see that ourvariables are available to thescope of our function, but theglobal variable is notoverwritten.Just like our entire bash scriptcan have a positional argument.So can our function, let's havea look at an example. Create anew script called function passargue.sh. And we'll write theshebang. And that will open ourfunction with the function name,open the brackets, open thecurly brackets, and then we'llwrite echo Hello, dollar signone.And then close the curlybrackets. And then we'll writethe function name space. Andthen whatever you want to writehere, we'll write our own name.So we can write out whatever setof characters we want. But inthis case, we wrote our ownname, we could even go as far asto pass positional arguments ofthe entire script to theposition or argument of thefunction. And of course,functions can also take multiplepositional arguments. So lots ofstuff is possible here.Let's open up the previous file.Underneath the echo statement,we'll type an if statement. Andthen we'll say if dollar signcurly braces, one comma, commaequals and then your user name,which in my case is Herbert toclose the brackets semicolon,then return zero, else, we willreturn one. And that will closethe if statement. Then also overhere, we'll remove this staticname and change it to apositional argument one, andthat also, if dollar signquestion mark equals one, thenclose square brackets,semicolon, then echo and then itwill write someone unknowncalled the function and thenwe'll close the if statements orwrite and quit. And let's nowtry to run this with tests. Forexample. It'll say hello test,but then you'll see that it alsosays that someone unknownrequested the function.Aw K is one of the most usefultools in the bash arsenal, weuse aw K to filter filecontents, or the output of acommand in such a way that wecan print out the most essentialparts and get the output the waywe like it. So use aw K, we caneither filter parts of a file,or we can filter parts of acommand output by piping thatoutput into a Wk let's have alook at how we can filterthrough a simple text file witha Wk we first create a test filewhich contains three wordsseparated by a space once wecreated that file, we can put aWk to work type in a Wk singlequote, open the curly bracketsprint dollar sign one, closecurly brackets, single quotesspace the name of our test file,we will see that the dollar signone acts as a placeholder. Forthe first item in the text file.We could also get the secondword that comes after the firstword always separated by spaceby typing in a Wk single quotescurly braces, print dollar signto close curly braces singlequote and again our test filenow that we get the idea of aw Kin its default mode, which isusing a space character as aseparator, we can move on tochanging it into othercharacters such as a car, let'screate a CSV file and add threevalues in it again separated bya comma this time now we can usea Wk again to split the valuesin the CSV file by adding theminus capital F flag andspecifying the split characterafterwards like this aw k minuscapital F comma space open thesingle quotes open the curlybraces print dollar sign oneclose the curly braces singlequote test CSV dot CSV and nowwe will see that the output isalso the same as in our previousexamples, but we will see thataw K actually used the comma asa separator character. And lastbut not least, we can also pipecommands into a Wk like so typein Echo just get this word colonhello and then we pipe into a WkI willprint out number five, becausethis is the fifth word separatedby a space and then we'll seethat the word hello will beprinted out. Or we can alsosplit this at the colon symbollike so. Echo, just get thisword colon Hello, pipe into aWK. And then we specify theminus f, we follow it up with acolon and then print dollar signto and that will still have tocut out the first characterwhich is a space and that willget the same results as well.And that's just the beginning.Of course, there's a lot morepossible with a WK. So if youwant to practice you can tryusing it more often in yourscripts. It really pays off toknow this well because you'll doa lot of output filtering andreformatting in bash.There will come a time where youwant to change certain values intext files, and that's where sedor set comes in. Set is acommand line tool that willallow you to modify values in atxt file using regularexpressions. Let's have a lookat an example on how to use asEdie are set to replace valuesin a text file. Create a textfile called Set test dot txt andlet's write down some text here.Now we will replace the word flywith grasshopper just fordemonstration purposes. So wetype in set single quotes sforward slash fly forward slashgrasshopper forward slash Gsingle quotes and then the nameof our txt file, which is settest dot txt. Now the structureof this command is a little bitdaunting, but let's break itdown. First, we start off withthe set command, we open thesingle quotes and start ourregular expression. First weenter the mode we want to use,which is as for substitute,meaning we want to substitutethe next word behind the forwardslash with the word after thesecond forward slash, which isthe first one is fly. And thenthe second one is grasshopperthat after the last, we providethe G which means that we wantto do this globally. G standsfor globally and globally meansthat we want to do this acrossthe entire text file, whichmeans that we want to changeevery occurrence, we close itwith another single quote and weput the text file we want tochange behind the entirecommand. Let's say we want tokeep the original in a backupfile. We can do this by addingthe minus II flag to the commandlike so. So we just do the samething we just do set minus i dotoriginal and then we enter theexact same command we justprovided earlier, we can seehere that we use the minus IIflag, but we don't put a spacebehind it. Instead, we write theentire suffix behind the minusII flag. This will create abackup file which contains theoriginal content of the file wejust changed with the setcommand but the original filewill have the changes we desiredjust like with aw k this is abasic introduction to set andthere will be so much more to itbut as with all things in it,you need to discover this byyourself by trying it out asmuch as you can. Well thisbrings us to the end of thecourse\n"