Make sure you have a correct IDPEndpoints entry in the SAML20 section in your service provider web.config.

You'll get that error if you're missing an entry for the IdP you're authenticating with, because it uses the IDPEndpoints entry for the certificate validation implementation, and it makes a non-optimale default decision if it can't properly resolve the IdP you're authenticating with.

Post the SAML20 section from you're SP web.config and I'll confirm if it's the case.