AWK Arrays Explained with 5 Practical Examples

Awk programming language supports arrays. As part of our on-going awk examples series, we have seen awk user defined variables and awk built-in variables. Arrays are an extension of variables. Arrays are variable that hold more than one value. Similar to variables, arrays also has names. In some programming languages, arrays has to be declared, so that memory will be allocated for the arrays. Also, array indexes are typically integer, like array[1],array[2] etc.,

Awk Associative Array

Awk supports only associative array. Associative arrays are like traditional arrays except they uses strings as their indexes rather than numbers. When using an associative array, you can mimic traditional array by using numeric string as index.

Syntax:
arrayname[string]=value

In the above awk syntax:

arrayname is the name of the array.

string is the index of an array.

value is any value assigning to the element of the array.

Accessing elements of the AWK array

If you want to access a particular element in an array, you can access through its index — arrayname[index], which gives you the value assigned in that index.

If you want to access all the array elements, you can use a loop to go through all the indexes of an array as shown below.

Syntax:
for (var in arrayname)
actions

In the above awk syntax:

var is any variable name

in is a keyword

arrayname is the name of the array.

actions are list of statements to be performed. If you want to perform more than one action, it has to be enclosed within braces.

This loop executes list of actions for each different value which was used as an index in array with the variable var set to that index.

Removing an element from the AWK array

If you want to remove an element in a particular index of an array, use awk delete statement. Once you deleted an element from an awk array, you can no longer obtain that value.

Syntax:
delete arrayname[index];

The loop command below removes all elements from an array. There is no single statement to remove all the elements from an array. You have to go through the loop and delete each array element using awk delete statement.

for (var in array)
delete array[var]

5 Practical Awk Array Examples

All the examples given below uses the Iplogs.txt file shown below. This sample text file contains list of ip address requested by the gateway server. This sample Iplogs.txt file contains data in the following format:

Third field ($3) is an ip address. This is used as an index of an array called Ip.

For each line, it increments the value of the corresponding ip address index.

Finally in the END section, all the index will be the list of unique IP address and its corresponding values are the occurrence count.

Example 2. List all the IP address and calculate how many sites it accessed

The last field in the Iplogs.txt is the number of sites each IP address accessed on a particular date and time. The below script generates the report which has list of IP address and how many times it requested gateway and total number of sites it accessed.

It has two arrays. The index for both the arrays are same — which is the IP address (third field).

The first array named “Ip” has list of unique IP address and its occurrence count. The second array called “count” has the IP address as an index and its value will be the last field (number of sites), so whenever the IP address comes it just keeps on adding the last field.

In the END section, it goes through all the IP address and prints the Ip address and access count from the array called Ip and number of sites from the array count.

Example 5. Remove duplicate and nonconsecutive lines using awk

Awk reads every line from the file “temp”, and using “in” operator it checks if the current line exist in the array “a”.

If it does not exist, it stores and prints the current line.

Recommended Reading

Sed and Awk 101 Hacks, by Ramesh Natarajan. I spend several hours a day on UNIX / Linux environment dealing with text files (data, config, and log files). I use Sed and Awk for all my my text manipulation work. Based on my Sed and Awk experience, I’ve written Sed and Awk 101 Hacks eBook that contains 101 practical examples on various advanced features of Sed and Awk that will enhance your UNIX / Linux life. Even if you’ve been using Sed and Awk for several years and have not read this book, please do yourself a favor and read this book. You’ll be amazed with the capabilities of Sed and Awk utilities.

Looking at several of these examples here I would refer to these arrays more as MAPS. The difference in my mind is that the indexes are not necessarily numerically consecutive in the awk cases…. they are more like the keys used in C++ style maps which can take any value as needed to suit the data set.

Interesting functionality though, I will be sure to use it at some point I’m sure.

So in fact you have left out two of the IP4 address bytes to make this work. That’s made them non-unique anyway, so you are now counting the wrong thing.

(2) You are still starting the print with “var, “. Var is an undefined variable — it appears nowhere else in the code. It happens awk treats that situation as a null (empty) value, but it’s poor practice to do it anyway.

(3) Your problem is not the dots, it is the quoting.

Your code should make the IP address a string, and then it will work on the original data.

Because you present the IP address without quotes, awk will try to figure text stuff as a variable name, and numeric stuff as an integer or floating-point, and anything else as an expression, and it will choke on it, or do something unexpected.

About The Geek Stuff

My name is Ramesh Natarajan. I will be posting instruction guides, how-to, troubleshooting tips and tricks on Linux, database, hardware, security and web. My focus is to write articles that will either teach you or help you resolve a problem. Read more about Ramesh Natarajan and the blog.

Contact Us

Email Me :
Use this Contact Form to get in touch me with your comments, questions or suggestions about this site. You can also simply drop me a line to say hello!.