Work the Shell - Dealing with Spaces in Filenames

In the good old days when UNIX was young, no one ever would have thought of
putting a space in a filename. It simply wasn't done—just as you'd never
do that on a DOS or Windows system. Filenames were short, succinct and
well-formed, like HW43.DOC.

Most of the Linux command-line utilities and the shells themselves have been
designed based on the premise that a space delimits a field value rather than
being an acceptable component of a filename. If you've done any scripting,
you already know this. Spaces in filenames can cause great trouble in shell
scripts!
Here's a simple example:

Yes, to maximize trouble, I have a filename that includes quotes as well as a
space. Don't get me started on having an escape character or non-printable
character in the name though. It's doable, but I'd rename it as soon as
possible.

Not all the filenames above have an “a” in them, so let's see what
happens when the fragmentary script is run in this directory:

The shell can deal with these filenames if they're simple enough, and the for
loop for name in *a* yields three filenames, not six, but somewhere
along your scripting journey, you inevitably will slam into the problem of
embedded spaces.

The most common error is to forget to quote filenames when you use them
elsewhere in the script, of course. As an example, let's work on a script
that replaces spaces in filenames with underscores.

As a discipline, it's always good to quote filenames you reference in any
context to ensure that when the shell passes them to the command as arguments,
the filenames with embedded spaces are handled properly.

This isn't a universal solution, however, because if you're using
subshells and pipes, it can be pretty darn hard for quotes to survive multiple
steps.

One path to travel is to set IFS, the internal field separator, in the
shell to something other than a space, as explained in the Bash man page:

IFS: The Internal Field Separator that is used for word splitting after
expansion and to split lines into words with the read built-in command. The
default value is “<space><tab><new-line>”.

That's useful for “read”, particularly if you're reading lines
of text and want to have a different field separator (think flat-file text
database files), but it still doesn't really solve our filename problem.

One thing I've used in the past, although it's a sloppy, crude solution,
is to start by changing spaces to some unlikely sequence of characters, run
through all the processing, and change them back at the last second. For
example:

Dave Taylor has been hacking shell scripts for over thirty years. Really.
He's the author of the popular "Wicked Cool Shell Scripts" and
can be found on Twitter as @DaveTaylor and more generally at
www.DaveTaylorOnline.com.

for filename in *; do aux=$filename/; LS[${#LS[@]}]=${aux%/}; done
...
rm "${LS[24]}", as example

The catch here is to avoid the conversions taken by the shell when you access to the content of a variable, the trick is to append to the variable a character, trying to make the shell guess that it's a string, and then you can gain access without infamous conversions, but at later you ougth to remove the char in order to use it as a filename.

Dave, did you mean '-print0' as opposed to '-print'? While both kind of make sense, when paired with 'xargs -0' I think the former makes *more* sense.

With respect to spaces, I usually don't bother dealing with them unless I *know* that they'll be there (e.g. I'm manipulating data that non-geek customers access via a Samba share and put plenty of weird characters in).

One of the other solutions I've considered, but not really had to use yet, is to switch to Perl or Python for these particular use-cases, given they handle escaping of spaces somewhat better.

It's something I'd rather not do given I'm more familiar with Bash than either of the above, but it may be the lease bad solution, even if I do have to dig up my text books to work out how to do something that would be trivial in Bash.