I am setting up a server that will need to authenticate with two way authentication to another server that I do not manage. The other server is denying my connection, and their support simply states this:

"You have our cert, we have your cert. When you send the handshake over we acknowledge it and trust your cert, then return it for you to acknowledge and trust our cert. As long as you have our cert loaded in your keystore correctly, you should trust our cert and then the transaction can take place."

At least tell us the cert type, how you added it and so on. Have you captured a tcp dump?What is the service name that is communicating with the remote side? Have you enabled debugging for that service?