2 answers

There are a few recurring cases where I see this error being thrown, as well as others that could cause this error. Full debug logs will give a better idea of what happened leading up to this error. The packet capture will show what is happening at the network level.

In order to troubleshoot further, you would likely need to collect the following:

Debug level logs
An error report from the default domain
A packet capture

One common case is when a client opens a connection but then aborts the connection before completing the handshake. This would most likely be accompanied by another log message such as "Request processing failed: Connection terminated before request headers read because of the connection error occurs, from URL:" or a connection hangup error.

Another case where I typically see a lot of these handshake errors is when there is an external load balancer sending TCP health checks to ports on DataPower which expect SSL connections. This is another example of this error which does not necessarily indicate any issue. Another tell tale sign here is you will see these errors on a regular basis from a few similar IPs, IE every 2 minutes from X.X.X.1 and X.X.X.2.

Another possible scenario would be if there was some cert issue on either DataPower or the client I would expect to see a handshake error. A lot of times we would need to consider the full context to understand why a specific error was thrown. This is why typically we want full debug logs + pcaps to verify the behavior at the network level too.