1.1.1a: crash in CRYPTO_THREAD_lock_free

1.1.1a: crash in CRYPTO_THREAD_lock_free

I'm trying OpenSSL 1.1.1a on FreeBSD 11.2-RELEASE-p4 and got the following
crash in one of my test programs (I compiled OpenSSL with -g after the
first time this happened to get at least some debug info):

Since my program doesn't use pthreads, I compiled OpenSSL with no-threads
and the crash doesn't happen (surprise...).
Is this a bug in my application or in OpenSSL? I'm not sure how to debug
this any further (without going into the details of pthreads on that OS).
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

The first thing to note is that the function tkaes a *pointer* to
a pthread_rwlock_t, but the thing pointed to is itself a pointer.
It has three "magic" values, and otherwise points to allocated
storage, freed on line 127 (matching your stack trace).

So you'd need to look at (**lock) not (*lock) to see the underlying
structure, but the address may be invalid. I don't know how it
came to be 0x800000000 (2^35)... I seems that something zeroed
the low 32 bits of the pointer.

More interesting that the lock might be the content of the EVP_PKEY
in frame #5:

Re: 1.1.1a: crash in CRYPTO_THREAD_lock_free

Thanks for the reply, it helped me adding some more debugging
statements to various places to track down the problem:
it is due to a change in TLS session handling in TLSv1.3.
It seems there are multiple SSL_SESSION structures for a single SSL
connection (SMTP session). The callback installed using
SSL_CTX_sess_set_new_cb() was called twice for the same SSL connection
and the code was written to handle only one callback per connection.
This resulted in a "use after free" situation. Sorry for the false
alarm.

Re: 1.1.1a: crash in CRYPTO_THREAD_lock_free

> On Nov 28, 2018, at 6:31 PM, Claus Assmann <[hidden email]> wrote:
>
> Thanks for the reply, it helped me adding some more debugging
> statements to various places to track down the problem:
> it is due to a change in TLS session handling in TLSv1.3.

Thanks for following up, much appreciated.

> It seems there are multiple SSL_SESSION structures for a single SSL
> connection (SMTP session). The callback installed using
> SSL_CTX_sess_set_new_cb() was called twice for the same SSL connection
> and the code was written to handle only one callback per connection.

Yes, by default the OpenSSL library now issues two session tickets
for each full (non-resumed handshake), and a new session ticket
after each resumption.

Recent versions of the Postfix SMTP server, when linked against OpenSSL
1.1.1 ask the library to issue just one session ticket after a full
handshake:

and none on resumption if the current ticket is still valid (existing
code from Postfix 2.11, which implements session ticket key sharing
and rotation for a pool of Postfix SMTP servers on a single host):

Linkability of sessions is not in my view a concern for SMTP, and SMTP
clients either don't cache session tickets at all, or cache at most one,
so issuing two initially and avoiding re-use is largely wasteful.

> This resulted in a "use after free" situation. Sorry for the false
> alarm.

Makes sense. I did look briefly for potential issues in the library,
that matched your stack trace (freeing the peer's DH public key), but
did not find any smoking gun. I might however add belt-and-suspenders
safety in one code path were I think that the current safe behaviour
could prove fragile as the code evolves. So it has not been entirely
unproductive.