Bash Co-Processes

One of the new features in bash 4.0 is the coproc statement.
The coproc statement allows you to create a co-process that
is connected to the invoking shell via two pipes: one to send
input to the co-process and one to get output from the co-process.

The first use that I found for this I discovered while trying to do
logging and using exec redirections.
The goal was to allow you to optionally start writing all of a script's
output to a log file once the script had already begun
(e.g. due to a --log command line option).

The main problem with logging output after the script has already started
is that the script may have been invoked with the output already redirected
(to a file or to a pipe).
If we change where the output goes when the output has already been
redirected then we will not be executing the command as intended by the user.

Here, if the script's stdout is not connected to the terminal,
we create a named pipe (a pipe that exists in the file-system) using mknod
and setup a trap to delete it on exit.
Then we start tee in the background reading from the named pipe and writing to the log file.
Remember that tee is also writing anything that it reads on its stdin
to its stdout.
Also remember that tee's stdout is also the same as the script's stdout
(our main script, the one that invokes tee) so the output from tee's stdout
is going to go wherever our stdout is currently going
(i.e. to the user's redirection or pipeline that was specified on the command line).
So at this point we have tee's output going where it needs to go:
into the redirection/pipeline specified by the user.

In the case that our standard output is going to the terminal then
we just use exec to redirect our output to the desired log file, as before.
If our output is not going to the terminal then we use coproc to
run tee as a co-process and redirect our output to
tee's input and redirect tee's output to where our output was originally going.

Running tee using the coproc statement is essentially the same as running
tee in the background (e.g. teelog&), the main difference is that bash
runs tee with both its input and output connected to pipes.
Bash puts the file descriptors for those pipes into an array named
COPROC (by default):

COPROC[0] is the file descriptor for a pipe that is connected
to the standard output of the co-process

COPROC[1] is connected to the standard input of the co-process.

Note that these pipes are created before any redirections are done
in the command.

Focusing on the part where the original script's output is not
connected to the terminal.
The following line duplicates our standard output on file descriptor 7.

exec 7>&1

Then we start tee with its output redirected to file descriptor 7.

coproc tee log 1>&7

So tee will now write whatever it reads on its standard input
to the file named log and to file descriptor 7, which is
our original standard out.

Now we close file descriptor 7 with (remember that tee still has
the "file" that's open on 7 opened as its standard output) with:

exec 7>&-

Since we've closed 7 we can reuse it, so we move the pipe that's
connected to tee's input to 7 with:

And finally, we close the pipe connected to tee's output, since
we don't have any need for it, with:

eval"exec ${COPROC[0]}>&-"

The eval here is required here because otherwise bash thinks
the value of ${COPROC[0]} is a command name.
On the other hand, it's not required in the statement above
(exec7>&${COPROC[1]}-), because in that one bash can recognize
that "7" is the start of a file descriptor action and not a command.

Also note the commented command:

#ls -la /proc/$$/fd

This is useful for seeing the files that are open by the current process.

We now have achieved the desired effect: our standard output is going into tee.
Tee is "logging" it to our log file and writing it to the pipe or
file that our output was originally going to.

As of yet I haven't come up with any other uses for co-processes,
at least ones that aren't contrived.
See the bash man page for more about co-processes.

Comment viewing options

It is a valid syntax and it works .Its called process substitution.Here you redirect the standard output to the file descriptor created by bash from which tee reads from, The standard output of tee is not being redirected so in the end tee writes to the standard output and the log file.

One important note is that process substitution is not enabled if bash is running in posix mode (in which case you will get a syntax error). Add a set +o posix at the start of your script to turn off posix mode.

To be honest, to me the traditional first example looks very much simpler and more understandable.
I find it quite difficult to really understand what the advantage of coproc really is, but after all, i am not doing this kind of stuff a lot.

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.