Here are the answers to questions about signal handling and taking care of global data when writing multi-threaded programs.

Perhaps the two most common questions I'm
asked about multi-threaded programming (after “what is
multi-threaded programming?” and “why would you want to do it?”)
concern how to handle signals, and how to handle cases where two
concurrent threads use a common function that makes use of global
data, and yet the two threads need thread-specific data from that
function. By definition, global data includes static local
variables which are in truth a kind of global variable. In this
article I'll explain how these questions can be dealt with in C
programs using one of the POSIX (or almost POSIX) multi-threading
packages available for Linux. I live in hope of the day when the
most common question I'm asked about multi-threaded programming is,
“Can we give you lots of money to write this simple multi-threaded
application, please?” Hey—I can dream, can't I?

All the examples in this article make use of POSIX compliant
functionality. To the best of my knowledge at the time I write
this, there are no fully POSIX-compliant multi-threading libraries
available for Linux. Which of the available libraries is best is
something of a subjective issue. I use Xavier Leroy's LinuxThreads
package, and the code fragments and examples were tested using
version 0.5 of this library. This package can be obtained from
http://pauillac.inria.fr/~xleroy/linuxthreads. Christopher
Provenzano has a good user-level library, although the signal
handling doesn't yet match the spec, and there were still a number
of serious bugs the last time I used it. (These bugs, I believe,
are being worked on.) Other library implementations are also
available. Information on these and other packages can be found in
the comp.programming.threads newsgroup and (to give a less than
exhaustive list):

http://www.mit.edu:8001/people/proven/pthreads.html

http://www.aa.net/~mtp/PCthreads.html

ftp://ftp.cs.fsu.edu/pub/PART/PTHREADS

Thread-specific data

As I implied above, I use the term “global data” for any
data which persists beyond normal scoping rules, such as static
local variables. Given a piece of code like:

void foo(void)
{
static int i = 1;
printf( "%d\n", i );
i = 2;
}

the first call to this function will print the value 1, and
all subsequent calls will print the value 2, because the variable
i and its value persist from one invocation of
the function to the next, rather than disappearing in a puff of
smoke as a “normal” local variable would. This, at least as far
as POSIX threads are concerned, is global data.

It is commonly said (I've said it myself) that using global
data is a bad practice. Whether or not this is true, it is only a
rule of thumb. Certainly there are situations where using global
data can avoid creating artificial circumstances. The previous
article (Linux Journal Issue 34) explained how
threads can share global data with careful use of mutual exclusion
(mutex) functions to prevent one thread from
accessing an item of global data while another thread is changing
its value. In this article I will look at a different type of
problem, using a real example from a recent project of mine.

Consider the case of a virtual reality system where a client
makes several network socket connections to a server. Different
types and priorities of data go down different sockets. High
priority data, such as information about objects immediately in the
field of view of the client, is sent down one socket. Lower
priority data such as texture information, background sounds, or
information about objects which are out of the current field of
view, is sent down another socket to be processed whenever the
client has available time. The server could create a collection of
new threads every time a new client connects to the server,
designating one thread for each of the sockets to be used to talk
to each of the clients. Every one of these threads could use the
same function to send a lump of data (not a technical term) to the
client. The data to be sent details of the client it is to be sent
to, the priority and type of data to be sent could all be held in
global variables, and yet each thread will make use of different
values. So how do we do it?

As a trivial example, suppose the only global data which our
lump-sending function needs to use is an integer that indicates the
priority of the data. In a non-threaded version, we might have a
global integer called priority used as in
Listing 1.

In the multi-threaded version we don't have a global integer,
instead we have a global key to the integer. It is through the key
that the data can be accessed by means of a number of
functions:

pthread_key_create() to prepare
the key for use

pthread_setspecific() to set a
value to thread-specific data

pthread_getspecific() to
retrieve the current value

pthread_key_create() is called once,
generally before any of the threads which are going to use the key
have been created. pthread_getspecific() and
pthread_setspecific() never return an error if
the key that is used as an argument has not been created. The
result of using them on a key which has not been created is
undefined. Something will happen, but it could vary from system to
system, and it can't be caught simply by using good error handling.
This is an excellent source of bugs for the unwary. So our
multi-threaded version might look like
Listing 2.

There are a few things to note here:

The implementation of POSIX threads can limit the
number of keys a process may use. The standard states that this
number must be at least 128. The number available in any
implementation can be found by looking at the macro
PTHREAD_KEYS_MAX. According to this macro,
LinuxThreads currently allows 128 keys.

The function
pthread_key_delete() can be used to dispose of
keys that are no longer needed. Keys, like all “normal” data
items, vanish when the process exits, so why bother deleting them?
Think of key handling as being similar to file handling. An
unsophisticated program need not close any files that it has
opened, as they will be automatically closed when the program
exits. But since there is a limit to the number of files a program
can have open at one time, the best policy is to close files not
currently being used so that the limit is not exceeded. This policy
also works well for key handling, as you may be limited in the
number of thread-specific data keys a process may have.

pthread_getspecific() and
pthread_setspecific() access thread-specific
data as void* pointers. This ability can be used
directly (as in Listing 2), if the data item to be accessed can be
cast as type void*, e.g., an
int in most, but not necessarily all,
implementations. However, if you want your code to be portable or
if you need to access larger data objects, then each thread must
allocate sufficient memory for the data object, and store the
pointer to the object in the thread-specific data rather than
storing the data itself.

If you allocate some memory (using the standard
function malloc(), for instance) for your
thread-specific data, and the thread exits at some point, what
happens to the allocated memory? Nothing happens, so it leaks, and
this is bad. This is the situation where the extra parameter in the
pthread_key_create() function comes into use.
This parameter allows you to specify a function to call when a
thread exits, and you use that function to free up any memory that
has been allocated. To prevent a waste of CPU time, this destructor
function is called only in the case where a thread has made use of
that particular key. There's little point in tidying up for a
thread that has nothing to be tidied. When a thread exits because
it called one of the functions exit(),
_exit() or abort(), the
destructor function is not called. Also, note that
pthread_key_delete() does not cause any
destructors to be called, that using a key that has been deleted
doesn't have a defined behavior, and that
pthread_getspecific() and
pthread_setspecific() don't return any error
indications. Tidy up your keys carefully. One day you'll be glad
you did. So a better version of our code is
Listing 3.

Some of this code might look a little strange at first sight.
Using pthread_getspecific() to store a thread
specific value? The idea is to get the memory location this thread
is to use, and then the thread specific value is stored
there.

Even if global data is anathema to you, you might still have
good use for thread-specific data. In particular, you might need to
write a multi-threaded version of some existing library code that
is also going to be used in a non-threaded program. A good simple
example is making a version of the standard C libraries fit for use
by multi-threaded programs. That friend of all C programmers,
errno, is a global variable that is commonly set
by library functions to indicate what went wrong during a function
call. If two threads call functions which both set errno to
different values, at least one of the threads is going to get the
wrong information. This is solved by having thread-specific data
areas for errno, rather than one global variable
used by all threads.

A race occurs if two or more signals arrive in similar timeframes - there is nothing to prevent "handled_signal" from being overwritten prior to another thread being scheduled and told about the first delivered signal.

signals SIGSTOP and SIGSEGV cannot be caught or ignored. I am not sure about SIGFPE....according to POSIX ignoring SIGFPE results in undefined behavior....I have tried to catch SIGFPE on my Linux system...but programme terminated with "Floating point exception"

SIGSEGV and SIGFPE can certainly be caught; in fact, I have intentionally generated and caught both in the same program, to sandbox pseudo-simulated code. The SIGSEGV signal is thrown not only when a "truly invalid" memory access is made, but when permissions are incorrect on the desired page; it is possible to change the permissions on the page from within the handler, and return to the original code (this is what my handler did).

From the sigaction man page:

signum specifies the signal and can be any valid signal except SIGKILL and SIGSTOP.

Regarding Listing 3. I believe not all malloc() implementations are thread-safe (in fact, very few of them actually are); so I'll assume that calling malloc inside a thread without a mutex is a typo.

My real question is. The key destructor in pthread_key_create(), from where is it called? The main thread or the same thread that is terminating? Is not clear anywhere. Perhaps I should check the POSIX specification but it is also difficult to find.