Do you mean the server, running 1.0.1h on Win7, produced this error message,
or some client talking *to* such a server produced the error?
In either case, what is in the error output or log of the opposite peer?

If you try to connect s_client to the server, or the client to s_server,
respectively,
does it work or what error info does it give you?

RE: SSL alert number 51

Unlikely I am being MitM'ed -- the connection is over a VPN. (Why TLS when
there is already a VPN in place? I am testing TLS software and the VPN is a
fact of life and my only client to server link.

> Do you mean the server, running 1.0.1h on Win7, produced this error
message, or some client talking *to* such a server produced the error?

Statement was kind of ambiguous, wasn't it? The server, which is OpenSSL
1.0.1h 5 Jun 2014, produced this message, when the client attempted to
connect.

The client is application software that uses the IBM GSK crypto library on
z/OS. The error message at the client end is Error code 9 returned from GSK
function gsk_secure_socket_init(): Cryptographic processing error. It is my
code that produces that exact message, but the 9 comes back from the
indicated method and the text comes from a system function, gsk_strerror(9).
The documentation says

9 Cryptographic processing error.
Explanation: An error is detected by a cryptographic
function. This error may also occur if key sizes that are
non-FIPS are used during an SSL handshake while
operating in FIPS mode.
User response: If the error occurred while executing
in FIPS mode, check that only FIPS key sizes are used.
Collect a System SSL trace containing the error and
then contact your service representative.

I can connect between the client and the server using the set of parameters
under test. They negotiate TLSV1.1 and what you call DHE-RSA-AES256-SHA and
GSK calls Cipher Suite 39 - SSL V3.0 AES SHA-1(ephemeral Diffie-Hellman)
RSA. It works provided I do not turn on FIPS 140-2 mode. If I turn on FIPS
140-2 mode with rc = gsk_fips_state_set(GSK_FIPS_STATE_ON); and use
otherwise identical parameters then this error occurs. (Cipher Suite 39 is a
valid FIPS 140-2 cipher suite, according to the IBM GSK documentation.)

I don't think that an s_client test would be terribly informative, seeing as
I can connect with the actual client software.

Do you mean the server, running 1.0.1h on Win7, produced this error message,
or some client talking *to* such a server produced the error?
In either case, what is in the error output or log of the opposite peer?

If you try to connect s_client to the server, or the client to s_server,
respectively, does it work or what error info does it give you?

Re: SSL alert number 51

On 19/11/14 22:57, Charles Mills wrote:

> Dave -
>
> Thanks much.
>
>> Either there's a bug somewhere or you are being attacked (MitM'ed).
>
> Unlikely I am being MitM'ed -- the connection is over a VPN. (Why TLS when
> there is already a VPN in place? I am testing TLS software and the VPN is a
> fact of life and my only client to server link.
>
>> Do you mean the server, running 1.0.1h on Win7, produced this error
> message, or some client talking *to* such a server produced the error?
>
> Statement was kind of ambiguous, wasn't it? The server, which is OpenSSL
> 1.0.1h 5 Jun 2014, produced this message, when the client attempted to
> connect.
>
> The client is application software that uses the IBM GSK crypto library on
> z/OS. The error message at the client end is Error code 9 returned from GSK
> function gsk_secure_socket_init(): Cryptographic processing error. It is my
> code that produces that exact message, but the 9 comes back from the
> indicated method and the text comes from a system function, gsk_strerror(9).
> The documentation says
>
> 9 Cryptographic processing error.
> Explanation: An error is detected by a cryptographic
> function. This error may also occur if key sizes that are
> non-FIPS are used during an SSL handshake while
> operating in FIPS mode.

My guess is that this last sentence is the cause of your problem.

> User response: If the error occurred while executing
> in FIPS mode, check that only FIPS key sizes are used.
> Collect a System SSL trace containing the error and
> then contact your service representative.
>
> I can connect between the client and the server using the set of parameters
> under test. They negotiate TLSV1.1 and what you call DHE-RSA-AES256-SHA and

FIPS 140-2 places restrictions on the size of the RSA key that you can
use. I'm not a FIPS 140-2 expert but I believe you have to be compliant
with the various other FIPS standards including FIPS 186-4(?):

"This Standard specifies three choices for the length of the modulus
(i.e.,nlen): 1024, 2048 and 3072 bits. Federal Government entities shall
generate digital signatures using one or more of these choices."

So how big is your RSA key on the server? Are you able to post the
certificate?

Re: SSL alert number 51

On Wed, Nov 19, 2014, Matt Caswell wrote:

>
>
> On 19/11/14 22:57, Charles Mills wrote:
>
> > User response: If the error occurred while executing
> > in FIPS mode, check that only FIPS key sizes are used.
> > Collect a System SSL trace containing the error and
> > then contact your service representative.
> >
> > I can connect between the client and the server using the set of parameters
> > under test. They negotiate TLSV1.1 and what you call DHE-RSA-AES256-SHA and
>
> FIPS 140-2 places restrictions on the size of the RSA key that you can
> use. I'm not a FIPS 140-2 expert but I believe you have to be compliant
> with the various other FIPS standards including FIPS 186-4(?):
>
> "This Standard specifies three choices for the length of the modulus
> (i.e.,nlen): 1024, 2048 and 3072 bits. Federal Government entities shall
> generate digital signatures using one or more of these choices."
>
> So how big is your RSA key on the server? Are you able to post the
> certificate?
>

>
>
> On 19/11/14 22:57, Charles Mills wrote:
>
> > User response: If the error occurred while executing in FIPS mode,
> > check that only FIPS key sizes are used.
> > Collect a System SSL trace containing the error and then contact
> > your service representative.
> >
> > I can connect between the client and the server using the set of
> > parameters under test. They negotiate TLSV1.1 and what you call
> > DHE-RSA-AES256-SHA and
>
> FIPS 140-2 places restrictions on the size of the RSA key that you can
> use. I'm not a FIPS 140-2 expert but I believe you have to be
> compliant with the various other FIPS standards including FIPS 186-4(?):
>
> "This Standard specifies three choices for the length of the modulus
> (i.e.,nlen): 1024, 2048 and 3072 bits. Federal Government entities
> shall generate digital signatures using one or more of these choices."
>
> So how big is your RSA key on the server? Are you able to post the
> certificate?
>

RE: SSL alert number 51

To be perfectly clear, the server is not OpenSSL itself but application code
that calls OpenSSL. The code is stable and in production and, as I said,
works if I do *not* turn on FIPS on the client. I could trace through the
calls if necessary.

Also, I will be out of the office all day Thursday so this is probably my
last reply for ~36 hours.

>
>
> On 19/11/14 22:57, Charles Mills wrote:
>
> > User response: If the error occurred while executing in FIPS mode,
> > check that only FIPS key sizes are used.
> > Collect a System SSL trace containing the error and then contact
> > your service representative.
> >
> > I can connect between the client and the server using the set of
> > parameters under test. They negotiate TLSV1.1 and what you call
> > DHE-RSA-AES256-SHA and
>
> FIPS 140-2 places restrictions on the size of the RSA key that you can
> use. I'm not a FIPS 140-2 expert but I believe you have to be
> compliant with the various other FIPS standards including FIPS 186-4(?):
>
> "This Standard specifies three choices for the length of the modulus
> (i.e.,nlen): 1024, 2048 and 3072 bits. Federal Government entities
> shall generate digital signatures using one or more of these choices."
>
> So how big is your RSA key on the server? Are you able to post the
> certificate?
>

> Dave -
>
> Thanks much.
>
>> Either there's a bug somewhere or you are being attacked (MitM'ed).
>
> Unlikely I am being MitM'ed -- the connection is over a VPN. (Why TLS
> when there is already a VPN in place? I am testing TLS software and
> the VPN is a fact of life and my only client to server link.
>
>> Do you mean the server, running 1.0.1h on Win7, produced this error
> message, or some client talking *to* such a server produced the error?
>
> Statement was kind of ambiguous, wasn't it? The server, which is
> OpenSSL 1.0.1h 5 Jun 2014, produced this message, when the client
> attempted to connect.
>
> The client is application software that uses the IBM GSK crypto
> library on z/OS. The error message at the client end is Error code 9
> returned from GSK function gsk_secure_socket_init(): Cryptographic
> processing error. It is my code that produces that exact message, but
> the 9 comes back from the indicated method and the text comes from a

system function, gsk_strerror(9).
> The documentation says
>
> 9 Cryptographic processing error.
> Explanation: An error is detected by a cryptographic function. This
> error may also occur if key sizes that are non-FIPS are used during an
> SSL handshake while operating in FIPS mode.

My guess is that this last sentence is the cause of your problem.

> User response: If the error occurred while executing in FIPS mode,
> check that only FIPS key sizes are used.
> Collect a System SSL trace containing the error and then contact your
> service representative.
>
> I can connect between the client and the server using the set of
> parameters under test. They negotiate TLSV1.1 and what you call
> DHE-RSA-AES256-SHA and

FIPS 140-2 places restrictions on the size of the RSA key that you can use.
I'm not a FIPS 140-2 expert but I believe you have to be compliant with the
various other FIPS standards including FIPS 186-4(?):

"This Standard specifies three choices for the length of the modulus
(i.e.,nlen): 1024, 2048 and 3072 bits. Federal Government entities shall
generate digital signatures using one or more of these choices."

So how big is your RSA key on the server? Are you able to post the
certificate?

Re: SSL alert number 51

On Fri, Nov 21, 2014, Matt Caswell wrote:

>
>
> On 21/11/14 14:43, Charles Mills wrote:
> > I posted the certificates. What's next?
> >
> > Charles
>
> The key sizes look ok to me. As I said I'm no FIPS expert, but this page
> http://wiki.openssl.org/index.php/FIPS_mode_and_TLS>
> says the following:
> "The RSA key in the certificate has to be of suitable size (2048 bits
> minimum) as do all other keys in the chain and none of the CAs can sign
> using SHA1."
>
> But your certificates say:
> Signature Algorithm: sha1WithRSAEncryption
>
> So I'm wondering if that is the problem? Failing that you may need to
> approach IBM since the alert is being generated from their code.
>

The rules changed some time after the release of the current OpenSSL FIPS
module.

In the original version SHA1 was allowed and keysizes for RSA, DH had to be
1024 bits minimum.

The new version disallows SHA1 for digital signatures and has the 2048 bit
limit. Depending on the interpretation it might effectively ban anything
other than TLS 1.2 (previous versions of TLS use MD5+SHA1 for RSA signatures).

RE: SSL alert number 51

Thanks. I guess I may have to open a problem with IBM. The IBM documentation
clearly lists a number of "cipher suites" (at they call them) that use SHA1
(including the one we (IBM+OpenSSL) default to as being FIPS 140-2
compliant.

GSK appears to only support SHA1 and MD5, and MD4 is pretty clearly not FIP
140-2 compliant.

Hmm. I had this note partly composed when Dr. Henson's reply came in. I am
thoroughly mystified.

Re: SSL alert number 51

On Fri, Nov 21, 2014, Charles Mills wrote:

> Thanks. I guess I may have to open a problem with IBM. The IBM documentation
> clearly lists a number of "cipher suites" (at they call them) that use SHA1
> (including the one we (IBM+OpenSSL) default to as being FIPS 140-2
> compliant.
>
> GSK appears to only support SHA1 and MD5, and MD4 is pretty clearly not FIP
> 140-2 compliant.
>
> Hmm. I had this note partly composed when Dr. Henson's reply came in. I am
> thoroughly mystified.
>

Could try to connect your client to OpenSSL's s_server utility with the -state
(or for 1.0.2 -trace)? If we can find out what message is triggering that
error it might give some hints.

RE: SSL alert number 51

Thanks guys for all of the clues! I got it working! Long story. Wow, FIPS is
a moving target. I re-did my root CA with SHA 256, and my server
certificate. I had to move my testing from z/OS V1R13 to z/OS V2R1 --
*apparently* V1R13 does not support TLS V1.2 which as you intimated at some
point may be required for things that FIPS requires. (A corollary would seem
to be that z/OS V1R13 does not support current FIPS requirements but don't
quote me on that.)

> Thanks. I guess I may have to open a problem with IBM. The IBM
> documentation clearly lists a number of "cipher suites" (at they call
> them) that use SHA1 (including the one we (IBM+OpenSSL) default to as
> being FIPS 140-2 compliant.
>
> GSK appears to only support SHA1 and MD5, and MD4 is pretty clearly
> not FIP
> 140-2 compliant.
>
> Hmm. I had this note partly composed when Dr. Henson's reply came in.
> I am thoroughly mystified.
>

Could try to connect your client to OpenSSL's s_server utility with the
-state (or for 1.0.2 -trace)? If we can find out what message is triggering
that error it might give some hints.

Thanks guys for all of the clues! I got it working! Long story. Wow, FIPS is
a moving target. I re-did my root CA with SHA 256, and my server
certificate. I had to move my testing from z/OS V1R13 to z/OS V2R1 --
*apparently* V1R13 does not support TLS V1.2 which as you intimated at some
point may be required for things that FIPS requires. (A corollary would seem
to be that z/OS V1R13 does not support current FIPS requirements but don't
quote me on that.)

RE: SSL alert number 51

> Thanks. I guess I may have to open a problem with IBM. The IBM
> documentation
> clearly lists a number of "cipher suites" (at they call them) that use
SHA1
> (including the one we (IBM+OpenSSL) default to as being FIPS 140-2
> compliant.
>
"cipher suite(s)" is the official term in the TLS standards,
mostly two words but sometimes hyphenated or run together,
so not surprisingly most implementations use it or a variant.
The "SHA" at the end a suite name defined before TLS1.2
is actually SHA1 used within HMAC for integrity check.
(HMAC is a generic MAC-from-hash construction.)
The new suites defined in or after TLS1.2 use SHA256 or SHA384
for HMAC, or are authenticated-encryption with *no* HMAC,
although they still vary the hash used in the PRF for key derivation.

> GSK appears to only support SHA1 and MD5, and MD4 is pretty clearly not
> FIP 140-2 compliant.
>
(That's a typo. SSL/TLS never used MD4, or MD2. It did use RC4 and RC2.)

Not quite, the picture is more nuanced. Although if you *can*
go to TLS1.2 and a SHA256 or SHA384 suite that is Best Practice.

800-131A (Jan 2011) "codified" in 800-57 part1 rev3 (July 2013)
prohibits SHA1 *for signature and hash-only* (which are assumed
subject to collision attack) after 2013. It is still allowed for HMAC
and some other uses that protect against collision. (Even after 2030
when 3TDEA, SHA-224, IFC&FFC 2048, and ECC 224 are scheduled
to go away, although they may well re-think before then.)

In particular, draft 800-52 rev1 (Jan 2013) allows the TLS1.0&1.1
PRF (key derivation) with SHA1-xor-MD5; MD5 is not Approved
at all but this construction doesn't rely on it and SHA1 *for KDF*
is okay. However TLS1.0 is disallowed for another reason.

Similarly in non-FIPS situations the two (HMAC-)MD5 suites
that are not SSLv2-only and not export-weakened are still
mostly considered acceptable, though at the same time
certs *signed* with MD5 are not, and certs signed with SHA1
won't be within a year or two. Not that this really matters,
since you practically always have a better option.