6 Answers
6

If you just use a pipe, it receives data on STDIN (the standard input stream) as a raw pile of data that it can sort through one line at a time. However some programs don't accept their commands on standard in, they expect it to be spelled out in the arguments to the command. For example touch takes a file name as a parameter on the command line like so: touch file1.txt.

If you have a program that outputs filenames on standard out and want to use them as arguments to touch, you have to use xargs which reads the STDIN stream data and converts each line into space separated arguments to the command.

These two things are equivalent:

# touch file1.txt
# echo file1.txt | xargs touch

Don't use xargs unless you know exactly what it's doing and why it's needed. It's quite often the case that there is a better way to do the job than using xargs to force the conversion. The conversion process is also fraught with potential pitfalls like escaping and word expansion etc.

The warning feels a little string to me. Of the two common options to get a stream onto a command line (xargs and $(...)), xargs is far safer than command substitution. And I cannot recall ever coming across a legitimate filename with a newline in it. Aren't the escaping and word expansion pitfalls issues with command substitution, not xargs?
– camhNov 19 '11 at 22:31

6

@camh: They're potential pitfalls with both. In the shell, you have to worry about filenames getting split on spaces, tabs, and newlines. In xargs, you only have to worry about newlines. In xargs, if you're output is formatted properly, you can split words/filenames on the NUL character instead (xargs -0), which is useful in conjunction with find -print0.
– Ken BloomNov 20 '11 at 1:22

Does xargs call the program via the shell with space separated args, or does it actually construct the argument list internally (eg. for use with execv/execp)?
– detlyNov 20 '11 at 9:55

1

It constructs it internally and uses execvp, so it's safe. Also, GNU xargs (as used on Linux and a few others) lets you specify newline as your delimiter with -d \n, although BSD xargs (OSX et al) does not appear to support this option.
– fluffyNov 21 '11 at 0:20

Good to note that the -0 argument to xargs makes it consider the NULL character to be the input item delimiter. find -print0 output NULL-delimited items. This is great practice for filenames that may contain spaces, quotes, or other special characters.
– Dan DascalescuJan 6 at 9:02

The -n 1 argument will make xargs turn each line into a command of its own. The sed -i "s/color/colour/g" command will replace all occurrences of color with colour for the specified file.

Note that this only works if you don't have any spaces in your paths. If you do, you should use null terminated paths as input to xargs by passing the -0 flag. An example usage would be:

$ git ls-files -z "*.tex" | xargs -0 -n 1 sed -i "s/color/colour/g"

Which does the same as what we described above, but also works if one of the paths has a space in it.

This works with any command that produces filenames as output such as find or locate. If you do happen to use it in a git repository with a lot of files though, it might be more efficient to use it with git grep -l instead of git ls-files, like so:

\ls | grep Cases | less lets you browse the list of file names produced by ls and grep. It doesn't matter that they happen to be file names, they're just some text.

\ls | grep Cases | xargs less lets you browse the files whose names are produced by the first part of the command. xargs takes a list of file names as input and a command on its command line, and runs the command with the file names on its command line.

When considering using xargs, keep in mind that it expects input formatted in a strange way: whitespace-delimited, with \, ' and " used for quoting (in an unusual way, because \ isn't special inside quotes). Only use xargs if you your file names don't contain whitespace or \'".

@Gilles:xargs has the -0, --null option to get around the spaces issue (it's highly likely I learnt that from you :), so I assume that you are referring to a no-options xarg call, but I'm puzzled by your reference to the quotes. Do you have a link or an example regarding that? .. (ps. | xargs less is a handy "trick" +1.. thanks..
– Peter.ODec 1 '11 at 20:59

In your example you don't need to use xargs at all since find will do exactly and safely what you want to do.

Exactly what you want using find is:

find -maxdepth 1 -name '*Cases*' -exec touch {} +

In this example -maxdepth 1 means only search in the current directory, don't descend into any subdirectories; by default find will look in all subdirectories (which is often what you want) unless you constraint it with maxdepth. The {} is the name of the file that will get substituted in its place and the + is one of two end-of-command markers, the other being ;. The difference between them is that ; means exec the command on each file one at a time, whereas + means exec the command on all the files at once. Note, however, that your shell will probably try to interpret ; itself, so you will need to escape it with either \; or ';'. Yes, find has a number of little annoyances like this, but its power more than makes up for it.

Both find and xargs are tricky to learn at first. To help you learn xargs try using the -p or --interactive option which will show you the command it is about to execute and prompt you whether or not you want to run it.

Similarly with find you can use -ok in place of -exec to prompt you whether or not you want to run the command.

There are times, though, when find won't be able to do everything you want and that is where xargs comes in. The -exec command will only accept one instance of {} appearing, so if you would get an error with find -type f -exec cp {} {}.bak \; so you could instead do it like so: find -type f -print0 | xargs -0 -l1 -IX cp X X.bak

Also, I mentioned that find safely does what you want because when you are dealing with files you are going to encounter spaces and other characters that will cause problems with xargs unless you use the -0 or --null option along with something that generates input items terminated by a null character instead of whitespace.

@Wildcard filenames with spaces or chars such as ' or " can be problematic, whereas find will handle those cases without a problem.
– aculichDec 4 '16 at 18:33

Yes, I know. See my answer to the linked question. I probably should have rephrased that question to a statement in the above comment, or added the phrase "See the question..." in front of it. :D
– WildcardDec 4 '16 at 21:23

xargs (along with find, sort, du, uniq, perl and a few others) accepts a command-line switch to say "STDIN has a list of files, separated by a NUL (0x00) byte". This makes it easy to handle filenames with spaces and other funny characters in them. Filenames don't contain NULs.