Web based School
Red Hat Linux rhl26
26
gawk
The awk programming language was created by the three people who gave their last-name initials to the language: Alfred Aho, Peter Weinberger, and Brian Kernighan. The gawk program included with Linux is the GNU implementation of that programming
language.
The awk language is more than just a programming language; it is an almost indispensable tool for many system administrators and UNIX programmers. The language itself is easy to learn, easy to master, and amazingly flexible. Once you get the hang of
using awk, you'll be surprised how often you can use it for routine tasks on your system.
To help you understand gawk, I will follow a simple order of introducing the elements of the programming language, as well as showing good examples. You are encouraged, or course, to experiment as the chapter progresses.
I can't cover all the different aspects and features of gawk in this chapter, but we will look at the basics of the language and show you enough, hopefully, to get your curiosity working.
What Is the awk Language?
awk is designed to be an easy-to-use programming language that lets you work with information either stored in files or piped to it. The main strengths of awk are its abilities to do the following:
- Display some or all the contents of a file, selecting rows, columns, or fields as necessary.
- Analyze text for frequency of words, occurrences, and so on.
- Prepare formatted output reports based on information in a file.
- Filter text in a very powerful manner.
- Perform calculations with numeric information from a file.
awk isn't difficult to learn. In many ways, awk is the ideal first programming language because of its simple rules, basic formatting, and standard usage. Experienced programmers will find awk refreshingly easy to use.
Files, Records, and Fields
Usually, gawk works with data stored in files. Often this is numeric data, but gawk can work with character information, too. If data is not stored in a file, it is supplied to gawk through a pipe or other form of redirection. Only ASCII files (text
files) can be properly handled with gawk. Although it does have the ability to work with binary files, the results are often unpredictable. Since most information on a Linux system is stored in ASCII, this isn't a problem.
As a simple example of a file that gawk works with, consider a telephone directory. It is composed of many entries, all with the same format: last name, first name, address, telephone number. The entire telephone directory is a database of sorts,
although without a sophisticated search routine. Indeed, the telephone directory relies on a pure alphabetical order to enable users to search for the data they need.
Each line in the telephone directory is a complete set of data on its own and is called a record. For example, the entry in the telephone directory for "Smith, John," which includes his address and telephone number, is a record.
Each piece of information in the record—the last name, the first name, the address, and the telephone number—is called a field. For the gawk language, the field is a single piece of information. A record, then, is a number of fields that
pertain to a single item. A set of records makes up a file.
In most cases, fields are separated by a character that is used only to separate fields, such as a space, a tab, a colon, or some other special symbol. This character is called a field separator. A good example is the file /etc/passwd, which looks like
this:
tparker:t36s62hsh:501:101:Tim Parker:/home/tparker:/bin/bash etreijs:2ys639dj3h:502:101:Ed Treijs:/home/etreijs:/bin/tcsh ychow:1h27sj:503:101:Yvonne Chow:/home/ychow:/bin/bash
If you look carefully at the file, you will see that it uses a colon as the field separator. Each line in the /etc/passwd file has seven fields: the user name, the password, the user ID, the group ID, a comment field, the home directory, and the startup
shell. Each field is separated by a colon. Colons exist only to separate fields. A program looking for the sixth field in any line needs only count five colons across (because the first field doesn't have a colon before it).
That's where we find a problem with the gawk definition of fields as they pertain to the telephone directory example. Consider the following lines from a telephone directory:
Smith, John 13 Wilson St. 555-1283 Smith, John 2736 Artside Dr, Apt 123 555-2736 Smith, John 125 Westmount Cr 555-1726
We "know" there are four fields here: the last name, the first name, the address, and the telephone number. But gawk doesn't see it that way. The telephone guide uses the space character as a field separator, so on the first line it sees
"Smith" as the first field, "John" as the second, "13" as the third, "Wilson" as the fourth, and so on. As far as gawk is concerned, the first line when using a space character as a field separator has six fields.
The second line has eight fields.
When working with a programming language, you must consider data the way the language will see it. Remember that programming languages take things literally.
To make sense of the telephone directory the way we want to handle it, we have to find another way of structuring the data so that there is a field separator between the sections. For example, the following uses the slash character as the field
separator:
Smith/John/13 Wilson St./555-1283 Smith/John/2736 Artside Dr, Apt 123/555-2736 Smith/John/125 Westmount Cr/555-1726
By default, gawk uses blank characters (spaces or tabs) as field separators unless instructed to use another character. If gawk is using spaces, it doesn't matter how many are in a row; they are treated as a single block for purposes of finding fields.
Naturally, there is a way to override this behavior, too.
Pattern-Action Pairs
The gawk language has a particular format for almost all instructions. Each command is composed of two parts: a pattern and a corresponding action. Whenever the pattern is matched, gawk executes the action that matches that pattern.
Pattern-action pairs can be thought of in more common terms to show how they work. Consider instructing someone how to get to the post office. You might say, "Go to the end of the street and turn right. At the stop sign, turn left. At the end of
the street, go right." You have created three pattern-action pairs with these instructions:
end of street: turn right stop sign: turn left end of street: turn right
When these patterns are met, the corresponding action is taken. You wouldn't turn right before you reached the end of the street, and you don't turn right until you get to the end of the street, so the pattern must be matched precisely for the action to
be performed. This is a bit simplistic, but it gives you the basic idea.
With gawk, the patterns to be matched are enclosed in a pair of slashes, and the actions are in a pair of curly braces:
/pattern1/{action1}
/pattern2/{action2}
/pattern3/{action3}
This format makes it quite easy to tell where the pattern starts and ends, and when the action starts and ends. All gawk programs are sets of these pattern-action pairs, one after the other. Remember these pattern-action pairs are working on text files,
so a typical set of patterns might be matching a set of strings, and the actions might be to print out parts of the line that matched.
Suppose there isn't a pattern? In that case, the pattern matches every time and the action is executed every time. If there is no action, gawk copies the entire line that matched without change.
Here are some simple examples. The gawk command
gawk '/tparker/' /etc/passwd
will look for each line in the /etc/passwd file that contains the pattern tparker and display it (there is no action, only a pattern). The output from the command will be the one line in the /etc/passwd file that contains the string tparker. If there is
more than one line in the file with that pattern, they all will be displayed. In this case, gawk is acting exactly like the grep utility!
This example shows you two important things about gawk: It can be invoked from the command line by giving it the pattern-action pair to work with and a filename, and it likes to have single quotes around the pattern-action pair in order to differentiate
them from the filename.
The gawk language is literal in its matching. The string cat will match any lines with cat in them, whether the word "cat" by itself or part of another word such as "concatenate." To be exact, put spaces on either side of the word.
Also, case is important. We'll see how to expand the matching in the section "Metacharacters" a little later in the chapter.
Jumping ahead slightly, we can introduce a gawk command. The command
gawk '{print $3}' file2.data
has only an action, so it performs that action on every line in the file file2.data. The action is print $3, which tells gawk to print the third field of every line. The default field separator, a space, is used to tell where fields begin and end. If we
had tried the same command on the /etc/passwd file, nothing would have been displayed because the field separator used in that file is the colon.
We can combine the two commands to show a complete pattern-action pair:
gawk '/UNIX/{print $2}' file2.data
This command will search file2.data line by line, looking for the string UNIX. If it finds UNIX, it prints the second column of that line (record).
The quotes around the entire pattern-action pair are very important and should not be left off. Without them, the command might not execute properly. Make sure the quotes match (don't use a single quote at the beginning and a double quote at the end).
You can combine more than one pattern-action pair in a command. For example, the command
gawk '/scandal/{print $1} /rumor/{print $2}' gossip_file
scans gossip_file for all occurrences of the pattern "scandal" and prints the first column, and then starts at the top again and searches for the pattern "rumor" and prints the second column. The scan starts at the top of the file
each time there is a new pattern-action pair.
Simple Patterns
As you might have figured out, gawk numbers all of the fields in a record. The first field is $1, the second is $2, and so on. The entire record is called $0. As a short form, gawk allows you to ignore the $0 in simple commands, so the instructions
gawk '/tparker/{print $0}' /etc/passwd
gawk '/tparker/{print}' /etc/passwd
gawk '/tparker/' /etcpasswd
result in the same output (the latter one because no action causes the entire line to be printed).
Sometimes you want to do more than match a simple character string. The gawk language has many powerful features, but I'll just introduce a few at the moment. We can, for example, make a comparison of a field with a value. The command
gawk '$2 == "foo" {print $3}' testfile
instructs gawk to compare the second string ($2) of each record in testfile and check to see if it is equal to the string foo. If it is, gawk prints the third column ($3).
This command demonstrates a few important points. First, there are no slashes around the pattern because we are not matching a pattern but are evaluating something. Slashes are used only for character matches. Second, the == sign means "is equal
to." We must use two equal signs, because the single equal sign is used for assignment of values, as you will see shortly. Finally, we put double quotations around foo because we want gawk to interpret it literally. Only strings of characters that are
to be literally interpreted must be quoted in this manner.
Don't confuse the quotes used for literal characters with those used to surround the pattern-action pair on the command line. If you use the same quote marks for both, gawk will be unable to process the command properly.
Comparisons and Arithmetic
An essential component of any programming language is the ability to compare two strings or numbers and evaluate whether they are equal or different. The gawk program has several comparisons, including ==, which you just saw in an example. Table 26.1
shows the important comparisons.
| Comparison | Description |
| == | Equal to |
| != | Not equal to |
| > | Greater than |
| < | Less than |
| >= | Greater than or equal to |
| <= | Less than or equal to |
These are probably familiar to you from arithmetic and other programming languages you may have seen. From this, you can surmise that the command
gawk '$4 > 100' testfile
will display every line in testfile in which the value in the fourth column is greater than 100.
All of the normal arithmetic commands are available, including add, subtract, multiply, and divide. There are also more advanced functions such as exponentials and remainders (also called modulus). Table 26.2 shows the basic arithmetic operations that
gawk supports.
| Operator | Description | Example |
| + | Addition | 2+6 |
| - | Subtraction | 6-3 |
| * | Multiplication | 2*5 |
| / | Division | 8/4 |
| ^ | Exponentiation | 3^2 (=9) |
| % | Remainder | 9%4 (=1) |
You can combine column numbers and math, too. For example, the action
{print $3/2}
divides the number in the third column by 2.
There is also a set of arithmetic functions for trigonometry and generating random numbers. See Table 26.3.
| Function | Description |
| sqrt(x) | Square root of x |
| sin(x) | Sine of x (in radians) |
| cos(x) | Cosine of x (in radians) |
| atan2(x,y) | Arctangent of x/y |
| log(x) | Natural logarithm of x |
| exp(x) | The constant e to the power x |
| int(x) | Integer part of x |
| rand() | Random number between 0 and 1 |
| srand(x) | Set x as seed for rand() |
The order of operations is important to gawk, as it is to regular arithmetic. The rules gawk follows are the same as with arithmetic: all multiplications, divisions, and remainders are performed before additions and subtractions. For example,
the command
{print $1+$2*$3}
multiplies column two by column three and then adds the result to column one. If you wanted to force the addition first, you would have to use parentheses:
{print ($1+$2)*$3}
Because these are the same rules you have heard about since grade school, they shouldn't cause you any confusion. Remember, if in doubt, put parentheses in the proper places to force the operations.
Strings and Numbers
If you've used any other programming language, these concepts will be familiar to you. If you are new to programming, you will probably find them obvious, but you'd be surprised how many people get things hopelessly muddled by using strings when they
should have used numbers.
A string is a set of characters that are to be interpreted literally by gawk. Strings are surrounded by quotation marks. Numbers are not surrounded by quotation marks and are treated as real values.
For example, the command
gawk '$1 != "Tim" {print}' testfile
will print any line in testfile that doesn't have the word Tim in the first column. If we had left out the quotation marks around Tim, gawk wouldn't have processed the command properly. The command
gawk '$1 == "50" {print}' testfile
will display any line that has the string 50 in it. It does not attempt to see if the value stored in the first column is different than 50; it just does a character check. The string 50 is not equal to the number 50 as far as gawk is concerned.
Formatting Output
We've seen how to do simple actions in the commands we've already discussed, but you can do several things in an action. For example, the command
gawk '$1 != "Tim" {print $1, $5, $6, $2}' testfile
will print the first, fifth, sixth, and second columns of testfile for every line that doesn't have the first column equal to "Tim". You can place as many of these columns as you want in a print
command.
Indeed, you can place strings in a print command, too, such as in this command
gawk '$1 != "Tim" {print "The entry for ", $1, "is not Tim. ", $2}' testfile
which will print the strings and the columns as shown. Each section of the print command is separated by a comma. There are also spaces at the end of the strings to ensure there is a space between the string and the value of the column that is printed.
You can use additional formatting instructions to make gawk format the output properly. These instructions are borrowed from the C language, and they use the command printf (print formatted) instead of print.
The printf command uses a placeholder scheme, but the gawk language knows how to format the entry because of the placeholder and looks later in the command line to find out what to put there. An example will help make this obvious:
{printf "%5s likes this language\n", $2}
The %5s part of the line instructs gawk how to format the string, in this case using five string characters. The value to place in this position is given at the end of the line as the second column. The \n at the end of the quoted section is a newline
character. If the second column of a four-line file held names, printf would format the output like this:
Tim likes this language Geoff likes this language Mike likes this language Joe likes this language
You will notice that the %5s format means to right-justify the column entry. This prevents awkward spacing.
The gawk language supports several format placeholders. They are shown in Table 26.4.
| Placeholder | Description |
| c | If a string, the first character of the string; if an integer, the character that matches the first value |
| d | An integer |
| e | A floating-point number in scientific notation |
| f | A floating-point number in conventional notation |
| g | A floating-point number in either scientific or conventional notation, whichever is shorter |
| o | An unsigned integer in octal format |
| s | A string |
| x | An unsigned integer in hexadecimal format |
Whenever you use one of the format characters, you can place a number before the character to show how many digits or characters are to be used. Therefore, the format 6d would have six digits of an integer. Many formats can be on a line, but
each must have a value at the end of the line, as in this example
{printf "%5s works for %5s and earns %2d an hour", $1, $2, $3}
Here, the first string is the first column, the second string is the second column, and the third set of digits is from the third column in a file. The output would be something like:
Joe works for Mike and earns 12 an hour
A few little tricks are useful. As you saw in an earlier example, strings are right-justified, so the command
{printf "%5s likes this language\n", $2}
results in the output
Tim likes this language Geoff likes this language Mike likes this language Joe likes this language
To left-justify the names, place a minus sign in the format statement:
{printf "%-5s likes this language\n", $2}
This will result in the output
Tim likes this language Geoff likes this language Mike likes this language Joe likes this language
Notice that the name is justified on the left instead of on the right.
When dealing with numbers, you can specify the precision to be used, so that the command
{printf "%5s earns $%.2f an hour", $3, $6}
will use the string in column three and put five characters from it in the first placeholder, and then take the value in the sixth column and place it in the second placeholder with two digits after the decimal point. The output of the command would be
like this:
Joe earns $12.17 an hour
The dollar sign was inside the quotation marks in the printf command, and was not generated by the system. It has no special meaning inside the quotation marks. If you want to limit the number of digits to the right of the period, you can do that too.
The command
{printf "%5s earns $%6.2f an hour", $3, $6}
will put six digits before the period and two after.
Finally, we can impose some formatting on the output lines themselves. In an earlier example, you saw the use of \n to add a newline character. These are called escape codes, because the backslash is interpreted by gawk to mean something different than
a backslash. Table 26.5 shows the important escape codes that gawk supports.
| Code | Description |
| \a | Bell |
| \b | Backspace |
| \f | Formfeed |
| \n | Newline |
| \r | Carriage return |
| \t | Tab |
| \v | Vertical tab |
| \ooo | Octal character ooo |
| \xdd | Hexadecimal character dd |
| \c | Any character c |
You can, for example, escape a quotation mark by using the sequence \", which will place a quotation mark in the string without interpreting it to mean something special. For example:
{printf "I said \"Hello\" and he said "\Hello\"."
Awkward-looking, perhaps, but necessary to avoid problems. You'll see lots more escape characters used in examples later in this chapter.
Changing Field Separators
As I mentioned earlier, the default field separator is always a whitespace character (spaces or tabs). This is not often convenient, as we found with the /etc/passwd file. You can change the field separator on the gawk command line by using the -F
option followed by the separator you want to use:
gawk -F":" '/tparker/{print}' /etc/passwd
This command changes the field separator to a colon and searches the etc/passwd file for the lines containing the string tparker. The new field separator is put in quotation marks to avoid any confusion. Also, the -F option (it must be a capital F) is
before the first quote character enclosing the pattern-action pair. If it came after, it wouldn't be applied.
Metacharacters
Earlier I mentioned that gawk is particular about its pattern-matching habits. The string cat will match anything with the three letters on the line. Sometimes you want to be more exact in the matching. If you only want to match the word "cat"
but not "concatenate," you should put spaces on either side of the pattern:
/ cat / {print}
What about matching different cases? That's where the or instruction, represented by a vertical bar, comes in. For example
/ cat | CAT / {print}
will match "cat" or "CAT" on a line. However, what about "Cat?" That's where we also need to specify options within a pattern. With gawk, we use square brackets for this. To match any combination of "cat" in
upper- or lowercase, we must write the pattern like this:
/ [Cc][Aa][Tt] / {print}
This can get pretty awkward, but it's seldom necessary. To match just "Cat" and "cat," for example, we would use the pattern
/ [Cc]at / {print}
A useful matching operator is the tilde (~). This is used when you want to look for a match in a particular field in a record. For example, the pattern
$5 ~ /tparker/
will match any records where the fifth field is tparker. It is similar to the == operator. The matching operator can be negated, so
$5 !~ /tparker/
will find any record where the fifth field is not equal to tparker.
A few characters (called metacharacters) have special meaning to gawk. Many of these metacharacters will be familiar to shell users, because they are carried over from UNIX shells. The metacharacters shown in Table 26.6 can be used in gawk patterns.
| Metacharacter | Meaning | Example | Meaning of Example |
| ~ | The beginning of the field | $3 ~ /^b/ | Matches if the third field starts with b |
| $ | The end of the field | $3 ~ /b$/ | Matches if the third field ends with b |
| . | Matches any single character | $3 ~ /i.m/ | Matches any record that has a third field value of i, another character, and then m |
| | | Or | /cat|CAT/ | Matches cat or CAT |
| * | Zero or more repetitions of a character | /UNI*X/ | Matches UNX, UNIX, UNIIX, UNIIIX, and so on |
| + | One or more repetitions of a character | /UNI+X/ | Matches UNIX, UNIIX, and so on, but not UNX |
| \{a,b\} | The number of repetitions between a and b (both integers) | /UNI\{1,3\}X | Matches only UNIX, UNIIX, and UNIIIX |
| ? | Zero or one repetitions of a string | /UNI?X/ | Matches UNX and UNIX only |
| [] | Range of characters | /I[BDG]M/ | Matches IBM, IDM, and IGM |
| [^] | Not in the set | /I[^DE]M/ | Matches all three character sets starting with I and ending in M, except IDM and IEM |
Some of these metacharacters are used frequently. You will see some examples later in this chapter.
Calling gawk Programs
Running pattern-action pairs one or two at a time from the command line would be pretty difficult (and time consuming), so gawk allows you to store pattern-action pairs in a file. A gawk program (called a script) is a set of pattern-action pairs stored
in an ASCII file. For example, this could be the contents of a valid gawk script:
/tparker/{print $6}
$2 != "foo" {print}
The first line would look for tparker and print the sixth column, and the second line would start at the top of the file again and look for second columns that don't match the string "foo", then display the entire line. When you are writing a
script, you don't need to worry about the quotation marks around the pattern-action pairs as you did on the command line, because the new command to execute this script makes it obvious where the pattern-action pairs start and end.
After you have saved all of the pattern-action pairs in a program, they are called by gawk with the -f option on the command line:
gawk -f script filename
This command causes gawk to read all of the pattern-action pairs from the file script and process them against the file called filename. This is how most gawk programs are written. Don't confuse the -f and -F options!
If you want to specify a different field separator on the command line (they can be specified in the script, but use a special format you'll see later), the -F option must follow the -f option:
gawk -f script -F":" filename
If you want to process more than one file using the script, just append the names of the files:
gawk -f script filename1 filename2 filename3 ...
By default, all output from the gawk command is displayed on the screen. You could redirect it to a file with the usual Linux redirection commands:
gawk -f script filename > save_file
There is another way of specifying the output file from within the script, but we'll come back to that in a moment.
BEGIN and END
Two special patterns supported by gawk are useful when writing scripts. The BEGIN pattern is used to indicate any actions that should take place before gawk starts processing a file. This is usually used to initialize values, set parameters such as
field separators, and so on. The END pattern is used to execute any instructions after the file has been completely processed. Typically, this can be for summaries or completion notices.
Any instructions following the BEGIN and END patterns are enclosed in curly braces to identify which instructions are part of both patterns. Both BEGIN and END must appear in capitals. Here's a simple example of a gawk script that uses BEGIN and END,
albeit only for sending a message to the terminal:
BEGIN { print "Starting the process the file" }
$1 == "UNIX" {print}
$2 > 10 {printf "This line has a value of %d", $2}
END { print "Finished processing the file. Bye!"}
In this script, a message is initially printed out, and each line that has the word UNIX in the first column is echoed to the screen. Next, the file is processed again to look for any line with the second column greater than 10, and the message is
generated with its current value. Finally, the END pattern prints out a message that the program is finished.
Variables
If you have used any programming language before, you know that a variable is a storage location for a value. Each variable has a name and an associated value, which may change.
With gawk, you assign a variable a value using the assignment operator, =:
var1 = 10
This assigns the value 10 (numeric, not string) to the variable var1. With gawk, you don't have to declare variable types before you use them as you must with most other languages. This makes it easy to work with variables in gawk.
Don't confuse the assignment operator, =, which assigns a value, with the comparison operator, ==, which compares two values. This is a common error that takes a little practice to overcome.
The gawk language lets you use variables within actions, so the pattern-action pair
$1 == "Plastic" { count = count + 1 }
checks to see if the first column is equal to the string "Plastic", and if it is, increments the value of count by one. Somewhere above this line we should set a preliminary value for the variable count (usually in the BEGIN section), or we
will be adding one to something that isn't a recognizable number.
Actually, gawk assigns all variables a value of zero when they are first used, so you don't really have to define the value before you use it. It is, however, good programming practice to initialize the variable anyway.
Here's a more complete example:
BEGIN { count = 0 }
$5 == "UNIX" { count = count + 1 }
END { printf "%d occurrences of UNIX were found", count }
In the BEGIN section, the variable count is set to zero. Then, the gawk pattern-action pair is processed, with every occurrence of "UNIX" adding one to the value of count. After the entire file has been processed, the END statement displays
the total number.
Variables can be used in combination with columns and values, so all of the following statements are legal:
count = count + $6 count = $5 - 8 count = $5 + var1
Variables can also be part of a pattern. The following are all valid as pattern-action pairs:
$2 > max_value {print "Max value exceeded by ", $2 - max_value}
$4 - var1 < min_value {print "Illegal value of ", $4}
Two special operators are used with variables to increment and decrement by one, because these are common operations. Both of these special operators are borrowed from the C language:
| count++ | increments count by one |
| count— | decrements count by one |
Built-In Variables
The gawk language has a few built-in variables that are used to represent things such as the total number of records processed. These are useful when you want to get totals. Table 26.7 shows the important built-in variables.
| Variable | Description |
| NR | The number of records read so far |
| FNR | The number of records read from the current file |
| FILENAME | The name of the input file |
| FS | Field separator (default is whitespace) |
| RS | Record separator (default is newline) |
| OFMT | Output format for numbers (default is %g) |
| OFS | Output field separator |
| ORS | Output record separator |
| NF | The number of fields in the current record |
The NR and FNR values are the same if you are processing only one file, but if you are doing more than one file, NR is a running total of all files, while FNR is the total for the current file only.
The FS variable is useful, because it controls the input file's field separator. To use the colon for the /etc/passwd file, for example, you would use the command
FS=":"
in the script, usually as part of the BEGIN pattern.
You can use these built-in variables as you would any other. For example, the command
NF <= 5 {print "Not enough fields in the record"}
gives you a way to check the number of fields in the file you are processing and generate an error message if the values are incorrect.
Control Structures
Enough of the details have been covered to allow us to start doing some real gawk programming. Although we have not covered all of gawk's pattern and action considerations, we have seen all the important material. Now we can look at writing control
structures.
If you have any programming experience at all, or have tried some shell script writing, many of these control structures will appear familiar. If you haven't done any programming, common sense should help, as gawk is cleanly laid out without weird
syntax. Follow the examples and try a few test programs of your own.
Incidentally, gawk enables you to place comments anywhere in your scripts, as long as the comment starts with a # sign. You should use comments to indicate what is going on in your scripts if it is not immediately obvious.
The if Statement
The if statement is used to allow gawk to test some condition and, if it is true, execute a set of commands. The general syntax for the if statement is
if (expression) {commands} else {commands}
The expression is always evaluated to see if it is true or false. No other value is calculated for the if expression. Here's a simple if script:
# a simple if loop
(if ($1 == 0){
print "This cell has a value of zero"
}
else {
printf "The value is %d\n", $1
})
You will notice that I used the curly braces to lay out the program in a readable manner. Of course, this could all have been typed on one line and gawk would have understood it, but writing in a nicely formatted manner makes it easier to understand
what is going on, and debugging the program becomes much easier if the need arises.
In this simple script, we test the first column to see if the value is zero. If it is, a message to that effect is printed. If not, the printf statement prints the value of the column.
The flow of the if statement is quite simple to follow. There can be several commands in each part, as long as the curly braces mark the start and end. There is no need to have an else section. It can be left out entirely, if desired. For example, this
is a complete and valid gawk script:
(if ($1 == 0){
print "This cell has a value of zero"
})
The gawk language, to be compatible with other programming languages, allows a special format of the if statement when a simple comparison is being conducted. This quick-and-dirty if structure is harder to read for novices, and I don't recommend it if
you are new to the language. For example, here's the if statement written the proper way:
# a nicely formatted if loop
(if ($1 > $2){
print "The first column is larger"
}
else {
print "The second column is larger"
})
Here's the quick-and-dirty method:
# if syntax from hell
$1 > $2{
print "The first column is larger"
}
{print "The second column is larger")
You will notice that the keywords if and else are left off. The general structure is retained: expression, true commands, and false commands. However, this is much less readable if you do not know that it is a if statement! Not all versions of gawk will
allow this method of using if, so don't be too surprised if it doesn't work. Besides, you should be using the more verbose method of writing if statements for readability's sake.
The while Loop
The while statement allows a set of commands to be repeated as long as some condition is true. The condition is evaluated each time the program loops. The general format of the gawk while loop is
while (expression){
commands
}
For example, the while loop can be used in a program that calculates the value of an investment over several years (the formula for the calculation is value=amount(1+interest_rate)^years):
# interest calculation computes compound interest
# inputs from a file are the amount, interest_rate, and years
{var = 1
while (var <= $3) {
printf("%f\n", $1*(1+$2)^var)
var++
}
}
You can see in this script that we initialize the variable var to 1 before entering the while loop. If we hadn't done this, gawk would have assigned a value of zero. The values for the three variables we use are read from the input file. The
autoincrement command is used to add one to var each time the line is executed.
The for Loop
The for loop is commonly used when you want to initialize a value and then ignore it. The syntax of the gawk for loop is
for (initialization; expression; increment) {
command
}
The initialization is executed only once and then ignored, the expression is evaluated each time the loop executes, and the increment is executed each time the loop is executed. Usually the increment is a counter of some type, but it can be any
collection of valid commands. Here's an example of a for loop, which is the same basic program as shown for the while loop:
# interest calculation computes compound interest
# inputs from a file are the amount, interest_rate, and years
{for (var=1; var <= $3; var++) {
printf("%f\n", $1*(1+$2)^var)
}
}
In this case, var is initialized when the for loop starts. The expression is evaluated, and if true, the loop runs. Then the value of var is incremented and the expression is tested again.
The format of the for loop might look strange if you haven't encountered programming languages before, but it is the same as the for loop used in C, for example.
next and exit
The next instruction tells gawk to process the next record in the file, regardless of what it was doing. For example, in this script:
{ command1
command2
command3
next
command4
}
as soon as the next statement is read, gawk moves to the next record in the file and starts at the top of the current script block (given by the curly brace). In this example, command4 will never be executed because the next statement moves back up to
command1 each time.
The next statement is usually used inside an if loop, where you may want execution to return to the start of the script if some condition is met.
The exit statement makes gawk behave as though it has reached the end of the file, and it then executes any END patterns (if any exist). This is a useful method of aborting processing if there was an error in the file.
Arrays
The gawk language supports arrays and enables you to access any element in the array easily. No special initialization is necessary with an array, because gawk treats it like any other variable. The general format for declaring arrays is
var[num]=value
As an example, consider the following script that reads an input file and generates an output file with the lines reversed in order:
# reverse lines in a file
{line[NR] = $0 } # remember each line
END {var=NR # output lines in reverse order
while (var > 0){
print line[var]
var—
}
}
In this simple program (try and do the same task in any other programming language to see how efficient gawk is!), we used the NR (number of records) built-in variable. After reading each line into the array line[], we simply start at the last record
and print them again, stepping down through the array each time. We don't have to declare the array or do anything special with it, which is one of the powerful features of gawk.
Summary
We've only scratched the surface of gawk's abilities, but you might have noticed that it is a relatively easy language to work with and places no special demands on the programmer. That's one of the reasons gawk is so often used for quick programs. It
is ideal, for example, for writing a quick script to count the total size of all the files in a directory. In the C language, this would take many lines, but it can be done in less than a dozen lines in gawk.
If you are a system administrator or simply a power user, you will find that gawk is a great complement to all the other tools you have available, especially because it can accept input from a pipe or redirection. For more information on gawk, check the
man pages or one of the few awk guides that are available.
Popular Tutorials
-
MS Access
1109
-
C++
1222
-
HTML
584
-
JavaScript
616
-
Vbscript
873
-
Oracle
473
-
VC++
875
-
SQL
2959
-
XML
514
-
Java
814
-
Perl
455
-
Linux
451
