When the validateRequest method of a JASPIC SAM is called in Tomcat 9, the "javax.security.auth.message.MessagePolicy.isMandatory" key in the MessageInfo map is erroneously always set to true.
This happens in org.apache.catalina.authenticator.AuthenticatorBase.getJaspicState via the following code:
new MessageInfoImpl(request.getRequest(), response.getResponse(), true);
The "true" param becomes the "authMandatory" value in the MessageInfo map:
map.put(IS_MANDATORY, Boolean.toString(authMandatory));
However, according to section 3.8.1.1 of the JASPIC 1.1 spec this should only be true if the target resource is protected. To be more exact when:
"... the resource identified by the HttpServletRequest is covered by a Servlet auth- constraint, or in a JSR 115 compatible runtime, if the corresponding WebResourcePermission is NOT granted to an unauthenticated caller."
So while the SAM should always be called (whether authentication is required or not), "javax.security.auth.message.MessagePolicy.isMandatory" should only be set to true when authentication is actually required (which incidentally, is also the case when HttpServletRequest#authenticate is called).

Thanks a lot for the quick fix!
ps.
I added a CI target for Tomcat to the Java EE samples project (https://github.com/javaee-samples/javaee7-samples.git)
After a git clone the JASPIC tests can be started using:
mvn clean install -pl "test-utils,util" -am
cd jaspic
mvn clean test -P tomcat-ci-managed
Tomcat versions from the Apache public and staging repositories can be tested. It currently defaults to version 8.5.6, but another version can be used via the -Dtomcat.version option. E.g.
mvn clean test -P tomcat-co-managed -Dtomcat.version=8.5.5
Note that for Tomcat the integration tests with EJB, JSF, JACC, etc are automatically skipped.
The staged Tomcat 8.5.6 has a perfect test result btw, all tests pass. Very impressive!

This is ASF Bugzilla: the Apache Software Foundation bug system. In case
of problems with the functioning of ASF Bugzilla, please contact
bugzilla-admin@apache.org.
Please Note: this e-mail address is only for reporting problems
with ASF Bugzilla. Mail about any other subject will be silently
ignored.