onelesd has asked for the
wisdom of the Perl Monks concerning the following question:

I have a script in which I use threads; use LWP; to make many simultaneous HTTPS connections using LWP::UserAgent. If there is more than one thread making a request at the same time, threads will crash and/or perl will segfault. Most of the time in each thread is spent making the HTTPS request, so locking while the request is made defeats my purpose for using threads and is not an option.

I traced down the problem to Crypt::SSLeay not being thread-safe, which LWP uses to make SSL requests. Here's the bug report. There is also a very old thread here at perlmonks without a solution.

How can I work around this?

$ perl -v
This is perl 5, version 12, subversion 3 (v5.12.3) built for x86_64-li+nux-gnu-thread-multi

Coro doesn't scale across multiple processors and that would hurt performance, but I suppose I could move the HTTPS request from the child threads into a Coro thread in the parent, but I'd need some way to send data from the children to the Coro thread.

If you cannot fix the problem and need bi-di communications with the kids, then you could do worse than to use a queue talking to threads that run lwp-requests via piped-opens. The https sessions run in separate processes, but you fetch the data back into the parent process where you can coordinate between them.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

I use Thread::Queue; to async {$Qin->enqueue($_) while (<>);}; and the child threads loop and block while (my $line = $Qin->dequeue) {...}. I am not familiar with POE or AnyEvent - is there a similar pattern using those modules?

I am not familiar with C, and the patch is for an old version. Here is the patch, if anyone in the know cares to comment. There are a couple of notes about the patch in the bug report which I don't quite understand: working only with pthreads and a possible problem with one part of the patch ("casting a 'pthread_t' to an 'unsigned long'".

I seriously doubt that will help in this case. The non-theadsafe part is most likely to be in one of the XS components or their underlying C libraries and will likely manifest themselves regardless of whether the Perlish code is used then cloned into the threads or required into each thread separately.

Working around module non-thread-safety by requireing only really works if you then only make use of the module from a single thread, which wouldn't achieve the OPs goal.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.