Unix Text Processing Commands: Unix Filters with Examples

In this tutorial, we will learn about filters and then work with various filter commands. Filters are commands that read input from stdin and write output to stdout.

By default, when using a shell terminal, the stdin is from the keyboard, and the stdout is to the terminal. Mechanisms to change the stdin and stdout will be covered in the next tutorials.

This Unix Text Processing Commands Tutorial is divided into 3 parts:

Unix Filters

Unix Pipes

More Filter Commands like awk and sed

Unix Video #19:

Text Processing in Unix

Unix provides a number of powerful commands to process texts in different ways. These text processing commands are often implemented as filters.

Filters are commands that always read their input from ‘stdin’ and write their output to ‘stdout’. Users can use file redirection and ‘pipes’ to setup ‘stdin’ and ‘stdout’ as per their need. Pipes are used to direct the ‘stdout’ stream of one command to the ‘stdin’ stream of the next command.

Some standard filter commands are described below. These commands may also take an input file as a parameter, but by default when the file is not specified, they operate as filter commands.

Unix Filter Commands

grep: Find lines in stdin that match a pattern and print them to stdout.

sort: Sort the lines in stdin, and print the result to stdout.

uniq: Read from stdin and print unique (that are different from the adjacent line) to stdout.

cat: Read lines from stdin (and more files), and concatenate them to stdout.

more: Read lines from stdin, and provide a paginated view to stdout.

cut: Cut specified byte, character or field from each line of stdin and print to stdout.

paste: Read lines from stdin (and more files), and paste them together line-by-line to stdout.

head: Read the first few lines from stdin (and more files) and print them to stdout.

tail: Read the last few lines from stdin (and more files) and print them to stdout.

wc: Read from stdin, and print the number of newlines, words, and bytes to stdout.

tr: Translate or delete characters read from stdin and print to stdout.

Next, let’s work through some of these commands in detail:

Command

grep - It is a command for pattern searching in a file and prints those lines containing that specified pattern. If the file name is not mentioned, grep searches in stdin.

Common Syntax

$ grep [option] pattern [filename …]

Example

$ grep ‘[A-M]’ file1 Prints those lines which contains capital letters in the range of A to M

Command

wc - It is a command to count the number of lines, words and characters in a file

Common Syntax

$ wc [OPTION] ….[FILE]

Example

$ cat file1 Hello How do you do $ wc file1 2 5 20 file1

No of lines-2 No of words-5 No of characters(bytes)-20

Command

more - This command is used to display the page one screen at a time

Common Syntax

more [options] file…

Example

$ls -l | more Will display long listing of files and directories one screen at a time

Command

paste: this command is used to paste the contents of two files.

Common Syntax:

paste [OPTION] ….[FILE]….

Example:

paste file1 file2 This command will combine the contents of file1 and file2

Hope you enjoyed this tutorial. Check out our upcoming tutorial to explore more on Text Processing in Unix with Unix Pipes.