Not directly shown in the table, but one of the biggest changes is that Liberty 9 beta 2016.5 no longer needs JASPIC to be activated in a problematic and proprietary way. Historically Liberty was one of the most problematic servers in this area. It originally required all users and groups to be made known to a Liberty specific artefact called a user registry (a kind of identity store). This requirement was later relaxed such that only groups had to be defined, but since group to role mapping was also mandatory, this stil meant defining the same group at least 3 times if no actual group mapping was needed.

In Liberty 9 beta 2016.5 all this overhead is now completely gone, and just like GlassFish and WebLogic, JASPIC just works without any activation whatsoever (with a caveat for GlassFish that by default it does require group to role mapping). One thing to watch out for is that if you do need group to role mapping, you seemingly can't just use Liberty's role mapping file (ibm-application-bnd.xml), but also need to define the above mentioned user registry again. All other servers (that feature a role mapper) allow the mapping of groups set by a JASPIC SAM via only a single XML file. When asked about this, the Liberty lead mentioned that this is due to backwards compatibility, which does make sense. It's also not such a bad thing; if you want to use the proprietary group to role mapping feature it's only a relatively small extra step to also activate a proprietary artefact such as the user registry.

As far as JASPIC functionality goes, Liberty keeps failing the CDI related tests and the JACC ones. The JACC ones are a clear spec violation (confirmed by the JACC and JASPIC spec lead), but since it concerns a requirement that was never enforced (namely, JACC has to be available and used by default) it's likely far from trivial to just fix this. Liberty also fails a test where the authenticated user is changed during the same HTTP session. This however clashes with Liberty's protection against a session fixation attack, which for instance happens when a user logs out and without resetting the http session (specifically changing its ID) a new user logs in again. This protection can be disabled by putting <httpSession securityIntegrationEnabled="false"/&gt in server.xml.

As mentioned above, TomEE is new to JASPIC scene, but what an entrance it makes! In it's very first release it passed all JASPIC tests, safe the JACC ones. TomEE (as not a full Java EE 7 certified server) is however not required to support JACC at all, let alone out of the box. Since TomEE does actually support JACC it may be possible to have it pass these tests as well. Right from the start TomEE also doesn't require any explicit activation of JASPIC. It just works out of the box.

JBoss/WildFly, while still having a very good JASPIC implementation fell a tiny bit behind. It still doesn't pass all JACC tests, even though JACC is in fact available by default (a bug prevents roles to be propagated correctly). Additionally, a new bug was found that prevents roles being propagated to EJB beans when the remember session feature is used. JBoss also still requires the dummy-like marker file (jboss-web.xml) in an archive to activate JASPIC. Even though such marker file is only a small nuisance compared to having to modify the server itself, it does make JBoss the sole server now that requires such activation at all (but as mentioned above, GlassFish/Payara requires a proprietary group to role mapping file to be present).

Payara also suffered from a newly discovered bug, this one involving the combination of the remember session feature and a custom principal. This was however quickly fixed in the 4.1.1.163 branch and Payara remains one of the strongest JASPIC implementations. There's essentially only one minor failure involving a not too common case of including a JSF resource from within a SAM.

Conclusion

7 years after portable authentication was introduced in Java EE 6 and 3 years after its revision in Java EE 7, JASPIC implementations are now finally getting really compliant. There's a residu of bugs in the various implementations that still need to be fixed, and a few inconveniences to take away in some servers (JBoss' jboss-web.xml and GlassFish/Payara's glassfish-web.xml), but overal things are starting to look very good.

The one big exception remains WebLogic 12.2.1, which currently doesn't work at all after a major bug was introduced in that version that prevents basic authentication to work correctly (effectively a blocker bug that makes running all other tests rather moot)

9 comments:

Basically yes. Both remember session failures are causes by the EJB propagation bug (one tests a string/name based principal, the other a custom principal). Likewise the two JACC tests are also variants that fail because the groups/roles aren't being propagated to the Subject that JACC uses.

The 2 JSF failures are an exception though, and are very likely because of Mojarra (it doesn't initialise correctly when called as an include).

Arjan, Nice article and comparisons. Just for context, how many of your tests overlap with the TCKs for JASPIC and JACC? Or, are you considering TCK compliance table-stakes? (I would assume not since TomEE isn't considered Java EE compliant yet.)

Also, by any chance have you run your set of tests against traditional WebSphere App Server v9? Since you referenced the legacy WebLogic server, I was curious if you had considered the original WebSphere server. My guess is that traditional WebSphere will have a few more "failures" than the Liberty profile, but it would help complete your comparison. I guess the same type of comment would apply to JBoss EAP... So, maybe you just want to stick to the more "flexible" implementations?

For Java EE 7 (JASPIC 1.1) this is likely very close to 0. The fact of the matter is that almost every one of those tests at some point failed on some certified server. The TCK for JASPIC is likely to be very small and doesn't test a lot. I've been in contact with the JASPIC TCK engineer, and there will likely be some improvements at least for the Java EE 8 TCK.

>Also, by any chance have you run your set of tests against traditional WebSphere App Server v9?

Nope, I haven't done so. I'm mostly on OS X, so I would need to set up some Windows virtual machine, but then traditional WebSphere is always quite a pain to download and install. I did run the earliest version of the tests manually against WebSphere though (8.0 at the time, I think). Also, the Java EE 7 samples tests are (of course) testing for Java EE 7, and WebSphere classic has only been out in a Java EE 7 version for a short period (I don't think it had betas). But if I find some time I'll try to add it next time. Thanks for the hint!

As for JBoss EAP, just didn't came around to testing it this time ;) It was included in the tests here: http://arjan-tijms.omnifaces.org/2015/11/the-state-of-portable-authentication.html

JBoss EAP is likely not that different from the WildFly version that was released just before it; those two are much more identical than e.g. WebSphere and Liberty are.

For the next instalment I also wanted to add WebLogic 12.2.1.1. I skipped 12.2.1 here because of the mentioned big regression, not because of it being inflexible ;)

The tests do check for the authenticated identity *not* being remembered, and indeed, none of them other than the explicit register session tests *should* create an http session (and even those are not mandated to create an http session, although doing that is very likely)

Did you see any server in any case other than "register session" creating a session?