sbcl-devel

The x86 part of Dan's pthread branch is merged with a few fixes.
It's a bit over 40k so here is a link instead:
http://retes.hu/~mega/pthreads.diff
What do pthreads bring:
+ errno is no longer shared
+ interrupt-thread pinning problem gone
=2D signal handling semantics change
=46ixes:
+ fixed recursive get on session-lock that happened when a gc interrupting
get-foreground reaped a thread
+ fixed sigint handling: removed broken (by pthread signal handling semanti=
cs)=20
sigint enable/disable machinery in favor of sigint-%break looking up the=20
foreground thread and interrupting it, which is itself racy :-(.
+ numerous fixes for interrupt-thread
Other:
* removed suspend-thread, resume-thread
* destroy-thread is now equivalent to terminate-thread. Maybe a pthread_can=
cel=20
based implementation is possible.
TODO:
* signal handling audit: pthreads and signals don't mesh well, handlers are=
=20
per process not per thread
* proper thread objects
* threads should start with (most?) signals blocked and enable them when the
lisp side setup is done
* there is a race in thread exit: the thread marks itself dead an if gc hits
before it really exits than it will have no stack =3D> either someone else
marks it dead or the thread control stack should be managed by pthreads
* restore errno in signal handlers
* think about session semantics, quit, terminate-session
* x86-64
* gcc4
Only x86 with :sb-thread needs pthreads (but the non threaded x86 version i=
s=20
built with -lpthread, too (scream if it is problem)), non-threaded builds u=
se=20
pids for thread ids, kill instead of pthread_kill and sigprocmask instead o=
f=20
pthread_sigmask.
=46eed back,
G=E1bor
PS: I found a memory fault that turned out to be present in the current cvs=
,=20
too. On x86, if threads are enabled with QSHOW and QSHOW_SIGNALS then a sta=
ck
exhaustion in the startup thread segfaults. I haven't been able to track it
down and since it's giving me much gray hair, here is what happens:
1. the control stack guard page is hit
2. handle_guard_page_triggered unprotects it and arranges for a return to
lisp (calling function control-stack-exhausted-error)
3. c-s-e-e does its thing and sooner or later it dies on
undefined_alien_address in handle_guard_page_triggered
I guess it's the alien stack setup. I know that QSHOW_SIGNALS is unsafe, bu=
t=20
why only the startup thread and only when built with threads?

The freeze is almost over and it's time for a wreckless attempt at introduc=
ing=20
thread objects:
http://retes.hu/~mega/thread-objects.diff
Here is another one for slime to work with it:
http://retes.hu/~mega/slime-threads.diff
It is fairly big, so here is some narrative. I seek comments on the public=
=20
interface, advice on the stucturing=20
({early-,,target-,target-uni,target-multi}thread.lisp) and the cold init=20
mods.
Executive summary
=2D----------------
* sb-thread:*current-thread*
#<SB-THREAD:THREAD 0 "initial thread">
* (sb-thread:make-thread (lambda ()) :name "good thread")
#<SB-THREAD:THREAD 1 "good thread">
* (sb-thread:thread-alive-p *)
NIL
The public interface changes
=2D---------------------------
* proper thread objects instead of thread ids
* (MAKE-THREAD FN &KEY NAME) =3D> THREAD
* (THREAD-ID THREAD) =3D> INTEGER is different from pid and pthread_t,
the initial thread has an id of 0, then next thread has an id of 1,
=2E..
* (THREAD-NAME THREAD): threads have names (useful for debugging, logging)
* (THREAD-ALIVE-P THREAD)
* *CURRENT-THREAD* special is bound in each thread
* (LIST-ALL-THREADS) returns a list of all active threads
* (FIND-THREAD THREAD-ID) =3D> THREAD or NIL
Notes
=2D----
* thread-init moved earlier in cold-init and reinit
* the lisp side does not ever use os_thread_t (it was problematic due
to pthread_t being opaque) but struct thread *
* threads are reaped (i.e. the thread is pthread_joined and struct
thread* is freed) by the thread object's finalizer. This makes it easy to=20
implement resetting threads. Running threads are kept in=20
sb-thread::*all-threads*.
* threads block all blockable signals when going down:
interrupt-thread and others cannot catch it at an inappropriate
moment, for instance calling quit outside the catch %end-of-the-world
* target-thread.lisp renamed target-multithread.lisp,
target-thread.lisp now contains the generic thread support
* new file early-thread.lisp: define *current-thread*
* removed thread state STOPPING that was only used for assertions and
complicated matters
* renumbered thread states
Done since the previous TODO
=2D---------------------------
* proper thread objects
* threads should start with (most?) signals blocked and enable them
when the lisp side setup is done
* there is a race in thread exit: the thread marks itself dead if
gc hits before it really exits than it will have no stack
Remaining TODO
=2D-------------
* are these useful: return value, join, recycling of threads?
* signal handling audit: pthreads and signals don't mesh well,
handlers are per process not per thread (setitimer is going to lose)
* restore errno in signal handlers
* think about session semantics, quit, terminate-session, sigint
* x86-64
* gcc4
Cheers, G=E1bor

On 24. Jun 2005, at 17:20, G=E1bor Melis wrote:
[clever things]
> * (FIND-THREAD THREAD-ID) =3D> THREAD or NIL
>
Do thread IDs overflow into bignums or wrap around? And in the =20
latter case, what about the highly-unlikely-so-bound-to-happen case =20
of getting back another thread with the same ID? IMAO, find-thread =20
should operate on thread names (or be removed entirely), since thread =20=
names are chosen by the user so the burden of keeping them unique =20
rests on him as well.
Rudi

Quoting Rudi Schlatte (rudi@...):
> Do thread IDs overflow into bignums or wrap around? And in the
> latter case, what about the highly-unlikely-so-bound-to-happen case
> of getting back another thread with the same ID?
IDs would be guaranteed to be unique, I assume...
> IMAO, find-thread should operate on thread names (or be removed
> entirely), since thread names are chosen by the user so the burden of
> keeping them unique rests on him as well.
... in contrast to thread names. I do not think thread names can or
should be unique, because there is no one "user" of threads. It should
be possible to run different unrelated applications in the same Lisp
image that all use threads, and those would not know how to coordinate
their thread names. Names should, of course, be descriptive enough that
human users can tell them apart when looking a the thread list, but
requiring uniqueness goes a bit far IMHO.
d.

On Friday 24 June 2005 17:38, Rudi Schlatte wrote:
> On 24. Jun 2005, at 17:20, G=C3=A1bor Melis wrote:
> [clever things]
>
> > * (FIND-THREAD THREAD-ID) =3D> THREAD or NIL
>
> Do thread IDs overflow into bignums or wrap around?
Overflow into bignums.
> IMAO, find-thread
> should operate on thread names (or be removed entirely), since thread
> names are chosen by the user so the burden of keeping them unique
> rests on him as well.
There can be several users starting threads so distributing the responsibil=
ity=20
to keep them unique is not easy. However, I do not very much care for the i=
ds=20
either as they duplicate object identity, but they can be externalized=20
easily.
On the practical side, if the idea of ids is killed, SLIME has to maintain =
an=20
id->thread mapping, likely as a weak hashtable, which doesn't sound half as=
=20
bad as decorating it with plists.
>
> Rudi

On Friday 24 June 2005 17:38, Rudi Schlatte wrote:
> On 24. Jun 2005, at 17:20, G=C3=A1bor Melis wrote:
> [clever things]
>
> > * (FIND-THREAD THREAD-ID) =3D> THREAD or NIL
>
> Do thread IDs overflow into bignums or wrap around? And in the
> latter case, what about the highly-unlikely-so-bound-to-happen case
> of getting back another thread with the same ID? IMAO, find-thread
> should operate on thread names (or be removed entirely), since thread
> names are chosen by the user so the burden of keeping them unique
> rests on him as well.
OK, time to retreat. Thread ids may not be a good idea after all. The deniz=
ens=20
of #lisp pointed out the error of my ways, and while I put up a valiant=20
defense they were numerous and higher level, too.
=46ind-thread and thread-id shall be removed, threads will be printed by=20
print-unreadable-object with :identity t. I've implemented the id/thread=20
mapping in SLIME but it ain't pretty. If only weak hashtables worked.
>
> Rudi
G=C3=A1bor

Hi,
Quoting G?bor Melis (mega@...):
> The public interface changes
> ----------------------------
>
> * proper thread objects instead of thread ids
>
> * (MAKE-THREAD FN &KEY NAME) => THREAD
one advantage of thread objects could be that they make it easy to
associate user-defined information with threads (beyond just their
name). The traditional way have been plists in the thread objects, I
think, which is pretty fishy in general and not thread-safe in
particular. Perhaps an alternative could be to document the thread
object (structure-)class, so that users could define subclasses and
instantiate those by specifying the class name when calling MAKE-THREAD?
> * threads are reaped (i.e. the thread is pthread_joined and struct
> thread* is freed) by the thread object's finalizer. This makes it easy to
> implement resetting threads. Running threads are kept in
> sb-thread::*all-threads*.
Isn't that dangerous? (How often does gencgc do full GCs?)
> * are these useful: return value, join, recycling of threads?
"join": If you mean whether joining a thread should be part of the
documented API, then yes, I could use that.
d.

On Friday 24 June 2005 18:01, David Lichteblau wrote:
> one advantage of thread objects could be that they make it easy to
> associate user-defined information with threads (beyond just their
> name). The traditional way have been plists in the thread objects, I
> think, which is pretty fishy in general and not thread-safe in
> particular. Perhaps an alternative could be to document the thread
> object (structure-)class, so that users could define subclasses and
> instantiate those by specifying the class name when calling MAKE-THREAD?
Yes, that would solve the problem of attaching info to a thread _if_ the=20
creator of that thread foresees the need. Or maybe the thread can be massag=
ed=20
into a suitable class later by interested parties as well.
> > * threads are reaped (i.e. the thread is pthread_joined and struct
> > thread* is freed) by the thread object's finalizer. This makes it easy
> > to implement resetting threads. Running threads are kept in
> > sb-thread::*all-threads*.
>
> Isn't that dangerous? (How often does gencgc do full GCs?)
Dangerous as in not reclaiming memory soon enough? The danger I can see her=
e=20
comes from the fact that create-thread maps memory itself without going=20
through the allocator, thus the gc never knows about it. Creating threads i=
n=20
a loop will soon get you an error but gc is not triggered.
> > * are these useful: return value, join, recycling of threads?
>
> "join": If you mean whether joining a thread should be part of the
> documented API, then yes, I could use that.
I imagine something like:
* (sb-thread:join-thread (sb-thread:make-thread (lambda () 42)))
;; waits until the thread exits and returns its return value
42
> d.
G=E1bor