Principales respuestas

Java SDK authentication - native_auth library doesn't load

Pregunta

I have written some code using the TFS Java SDK to talk to TFS version control. The code works in isolation in my automated tests, but when I deploy it into my application it fails to authenticate. I would appreciate any help you can give me in debugging
the problem.

Both cases are running under Java 1.6 on Linux.

I have turned on debug logging in both cases, which allows me to see the details of native library loading and the HTTP conversation.

Respuestas

I've written a wrapper around your SDK which runs in its own classloader. That solves the problem and it's now working fine.

This is a practical solution for me at the moment because I'm only using a very small subset of your API. But for a more complex use of the library it would be extremely painful to have marshal data across the interface between classloaders.

I see that both HttpClient 3 and 4 support a plugin mechanism for authentication. Is there any reason why your NTLMv2 client implementation can't be rewritten as a plugin? In fact, I see that HttpClient 4 supports NTLMv2 and Kerberos natively.

Todas las respuestas

The log messages are a bit misleading in this case. JavaNTLM doesn't require the native_auth library (NTLM is implemented in pure Java), but we always try to load that library so other mechanisms can be used if the system libraries are present (Kerberos,
for example). NTLM authentication is always available, regardless of whether libnative_auth.so can be loaded.

Are you using the same credentials for your tests vs. deployment? The final 401 in the failed case could point to a misspelled password, or perhaps TFS permissions don't allow this account access to the project collection. You might check the
server's event log for details.

Generally, you really want the native libraries to load for your SDK applications. Certain version control features won't work correctly if they don't. Make sure you're defining the "com.microsoft.tfs.jni.native.base-directory" system property
to point where the redistributable native libraries are located. The readme.txt files that come with the SDK samples show an example of this.

I am using the same credentials (and the same URL and project) in both cases. The native libraries are loading fine as far as I can see from other logs.

My understanding of NTLM is that the second message that the client sends to the server (the response to the challenge) should be considerably longer than the earlier messages in the handshake. I see this in the successful case, but in the failing case the
response is far shorter

The logs on the server show an error "TF53008: The authentication type is not supported."

Which Java runtime are you using on the deployment platform? Linux distributions often install GNU GCJ by default, which historically did not support the cryptography libraries that TEE needs, and is not supported. Oracle's runtime is recommended
and supported.

It might be worth pointing out that, while I'm developing this stuff, the deployment platform is actually the same phyisical box as the development platform. The only difference is that there are a whole bunch more libraries loaded and maybe some different
environment variables.

We ship a modified version of Apache Commons HTTPClient 3, which includes NTLMv2 and Kerberos (SPNEGO) authentication mechanisms. (The stock version of HTTPClient includes only NTLMv1 compatibility, which is unsuitable for authentication.)

However, we still include the package as "org.apache.commons.httpclient" so that SDK users can swap ours out their own HTTPClient mechanism for whatever reason.

Is it possible that your deployment platform also includes org.apache.commons.httpclient?

You might try putting the TFS SDK JARs on the classpath before the other HttpClient contributor. I don't know how your application is deployed, but some application containers let you use different classloaders for each component, to keep their dependencies
isolated. That might also work, but I don't have any specific advice in that area.

I've tested using the TFS SDK to provide the Apache Commons HTTPClient to the whole system. That works fine, confirming that this is indeed the problem.

I am not terribly happy with the idea of using the TFS SDK as a source of the HTTPClient to the whole system, particularly as you've modified it. The stuff I'm working on is a small corner, integrating TFS with a large, existing system. Apart from anything
else, it would make us reliant on you to propagate any security patches to that library.

I will investigate the possibility of running the TFS integration inside an isolated classloader.

Have you had any luck contributing your changes back to the core Apache project?

It's good to know that we've found the cause. I wouldn't recommend replacing your existing implementation with our HTTPClient. While it's undergone thorough testing in our product, we will not support it as a general replacement for Apache Commons
HTTPClient and do not test it as such.

I believe that a classloading strategy will suffice -- please do let us know the results of your testing.

As for contributing back: while I'd love to see more packages adopt
a proper NTLM2 implementation, we've decided not to contribute our code back to HttpClient for three major reasons: first, we've removed NTLMv1 as an authentication option in HTTPClient in order to meet our security standards and while we absolutely
feel that this is an appropriate decision for our product, we do not feel that we should make this decision for all library users in general. Second, HttpClient 3 is being replaced by HttpComponents HttpClient 4 and while we could target that
for patching, it's simply not a priority for us to move to HttpClient 4 at the moment. Finally, our Kerberos implementation relies heavily on native method calls using JNI, as does our NTLM implementation on Windows. We do not feel that making
HttpClient depend on native libraries for authentication is appropriate for a general-purpose library.

In any case, we're looking at this more for our next release to see how we can minimize your pain here. We simply hadn't considered some of the inventive ways people would want to use our Java SDK.

I've written a wrapper around your SDK which runs in its own classloader. That solves the problem and it's now working fine.

This is a practical solution for me at the moment because I'm only using a very small subset of your API. But for a more complex use of the library it would be extremely painful to have marshal data across the interface between classloaders.

I see that both HttpClient 3 and 4 support a plugin mechanism for authentication. Is there any reason why your NTLMv2 client implementation can't be rewritten as a plugin? In fact, I see that HttpClient 4 supports NTLMv2 and Kerberos natively.

We'll consider releasing our authentication code in the future. It currently contains some other changes to support special authentication scenarios required by TFS (in addition to the NTLM/Negotiate providers). We also took a different approach
for Kerberos integration, using the OS's GSS libraries directly instead of Java GSS, so the user doesn't have to configure Java GSS during installation.

I've written a wrapper around your SDK which runs in its own classloader. That solves the problem and it's now working fine.

This is a practical solution for me at the moment because I'm only using a very small subset of your API. But for a more complex use of the library it would be extremely painful to have marshal data across the interface between classloaders.

I think I'm seeing the same problem as you; my code runs fine in isolated tests, but authentication to TFS fails when deployed inside a larger application (which includes Apache Commons HttpClient on its classpath).

Can you please share some information on how you've implemented the wrapper around the SDK? I also only need a small subset of the API; retrieve a list of team project collections and projects, and submit work items.

Edit: Never mind, after delving into Classloader documentation I already crafted my own solution. My code was already abstracted by an interface, so I now simply instantiate my code using a custom classloader like so:
return (<iface>)new ParentLastURLClassLoader(<impl-jar>).loadClass(<impl>).newInstance();

Can you please help me with the code to do NTML authentication using TFS JAVA SDK. I have used all the available methods but it seems realm is always BASIC and not NTLM. Following is the link to my thread: