I am currently working on a solution to retrieve the document metadata
from the registry using the document UniqueId. I am behind a proxy. I
guess am not able to send a query to the registry using the following code
because of the exception. Does anybody know a solution how to fix this
problem because i set the proxy correctly and the transaction iti-14 works
also for this registry except the query. I did not find anything about
such a problem. It seems that invokeStoredQuery throws this exception.
Indeed, i guess its the sending part inside the invokeStoredQuery method.

It looks from the stack trace that this request made it through all the
OHF code and out. We'd have to see your logs to be sure (you can attach
here or in a Bugzilla).

Does running one of the JUNITS in OHF for GetDocumentsQuery work by
itself with this registry from your environment?? Maybe try setting
your proxy settings in the runtime for your program (in case order is a
problem here). A final thought, is that since OHf code is administering
the connection for you to the registry URL, it may not permit proxy (or
we don't expose it in a way for you to make this switch).

Another note, unrelated to your question, you don't need the line:

Config.start(true);

This is only used if you want to work (backwards compatible) with XDS
registry implementations prior to 2007 ... where patient id's required
an assigning authority name in addition to the OID and ISO type. It's
not documented too well ... so another thing to add to our list :-) The
most recent set of test code is in the
src_tests/org.eclipse.ohf.ihe.xds.consumer.test.mesa package. Here --
the Test configuration and the base tests (ConsumerMesaTest.java and
B_ConsumerMesaTest.java) provide more up to date examples.

You've uncovered an issue that I'm not 100% sure how to fix and/or
workaround. IF you could provide the use scenario, that'll help us
determine a fix.

OHF's XDS component uses Apache Axis2 to handle the XDS Web service
transaction. Apache Axis2 uses the Apache Commons HTTPClient component
to handle the HTTP Web service. HTTPClient handles proxies without a
problem. Ordinarily the fix would be to uncover the static /
threadlocal API into HTTPClient that exposes how to set the HTTP proxy's
hostname, port, username, and password. This is well-documented.

The complicating factor is that, in OHF, we manage the socket connection
top to bottom. That is, we give Axis2 / HTTP Client the java.io.Socket
instance that it uses for the Web service connection. The principal
reason is because of the complicated requirements for mutual TLS under
IHE specifications. Now, we also do the same thing for non-TLS (plain
HTTP) connections, but mainly for consistency. This breaks the ability
to use HTTP proxies in OHF. We could easily revert this for plain HTTP,
but it is impossible for us to allow HTTPClient to create the socket for
TLS connections.

My question for you is... is this simply a development use case (e.g.
HTTP connections in your organization are proxied?) or something more
larger-scale? Also, are you expecting to use "IHE Node Authentication"
(e.g. mutual TLS) through this proxy as well? If so, could you maybe
help me better understand how that would work. These questions should
help us to find a fix.

Thanks for the report - this is definitely an important gap that needs
to be addressed.

Actually, i did some more tests on this problem. After hours of research
it seems that that the problem seems to be on a lower level. Normally you
create an instance of the B_Consumer or Consumer object which is then able
to create a valid query request. The creation of this message seems to be
ok, except of the sending part. The B_Consumer::invokeStoredQuery contains
a send call which fails. So after going deeper you can see that the lower
level AbstractXDSSOAPClient::send method fails on the part where it tries
to send the message. So after going deeper it seems that the
AbstractOHFSOAPSender fails on a specific method call. The problem
sections seems to be the "configureTransportProtocol" method on the part
where the getSocket method of the AtnaAgent is invoked. Thats the point
where an exception is thrown. ... Infact, the endpoit adress seems to be
valid and the chunked parameter is configured. Actually, it seems to be
mysterious problem.

A possible workaround...
It seems that a working workaround would be to write transaction like the
iti-14 (where we set the proxy in the same way) to straight forward the
well formated query to the registry. So the best way is to replace only
the sending call of the B_Consumer which has then the effect that this
transaction works properly.

By the way, it seems that the proxy never gets reached by the typical
sending call of the B_Consumer class.

Yep, what you found in AtnaAgent.getSocket() is indeed how we control
the socket construction. This allows us to manage, at runtime, the
certificates used in TLS authentication on a per-domain basis. However,
it's not a requirement for non-TLS connections, it's only there for
consistency.

I'm going to experiment with a couple things and let you know the results.

Thanks!
-Matt

Jason wrote:
> Hi Matt
>
> First, thanks for your answer!
>
> Actually, i did some more tests on this problem. After hours of research
> it seems that that the problem seems to be on a lower level. Normally
> you create an instance of the B_Consumer or Consumer object which is
> then able to create a valid query request. The creation of this message
> seems to be ok, except of the sending part. The
> B_Consumer::invokeStoredQuery contains a send call which fails. So after
> going deeper you can see that the lower level
> AbstractXDSSOAPClient::send method fails on the part where it tries to
> send the message. So after going deeper it seems that the
> AbstractOHFSOAPSender fails on a specific method call. The problem
> sections seems to be the "configureTransportProtocol" method on the part
> where the getSocket method of the AtnaAgent is invoked. Thats the point
> where an exception is thrown. ... Infact, the endpoit adress seems to be
> valid and the chunked parameter is configured. Actually, it seems to be
> mysterious problem.
>
> A possible workaround...
> It seems that a working workaround would be to write transaction like
> the iti-14 (where we set the proxy in the same way) to straight forward
> the well formated query to the registry. So the best way is to replace
> only the sending call of the B_Consumer which has then the effect that
> this transaction works properly.
> By the way, it seems that the proxy never gets reached by the typical
> sending call of the B_Consumer class.
> The whole test id done without mutual TLS
>
> Hope this helps to find the problem
> Greetings jason
>

If this works, I'm going to have you try a couple more things and then
we'll install a permanent fix for both non-TLS and TLS connections
(although, again, I'm not sure you would ever proxy a TLS connection).

Thanks,
-Matt

Matthew Davis wrote:
> Jason,
>
> Yep, what you found in AtnaAgent.getSocket() is indeed how we control
> the socket construction. This allows us to manage, at runtime, the
> certificates used in TLS authentication on a per-domain basis. However,
> it's not a requirement for non-TLS connections, it's only there for
> consistency.
>
> I'm going to experiment with a couple things and let you know the results.
>
> Thanks!
> -Matt
>
>
> Jason wrote:
>> Hi Matt
>>
>> First, thanks for your answer!
>>
>> Actually, i did some more tests on this problem. After hours of
>> research it seems that that the problem seems to be on a lower level.
>> Normally you create an instance of the B_Consumer or Consumer object
>> which is then able to create a valid query request. The creation of
>> this message seems to be ok, except of the sending part. The
>> B_Consumer::invokeStoredQuery contains a send call which fails. So
>> after going deeper you can see that the lower level
>> AbstractXDSSOAPClient::send method fails on the part where it tries to
>> send the message. So after going deeper it seems that the
>> AbstractOHFSOAPSender fails on a specific method call. The problem
>> sections seems to be the "configureTransportProtocol" method on the
>> part where the getSocket method of the AtnaAgent is invoked. Thats the
>> point where an exception is thrown. ... Infact, the endpoit adress
>> seems to be valid and the chunked parameter is configured. Actually,
>> it seems to be mysterious problem.
>>
>> A possible workaround...
>> It seems that a working workaround would be to write transaction like
>> the iti-14 (where we set the proxy in the same way) to straight
>> forward the well formated query to the registry. So the best way is to
>> replace only the sending call of the B_Consumer which has then the
>> effect that this transaction works properly.
>> By the way, it seems that the proxy never gets reached by the typical
>> sending call of the B_Consumer class.
>> The whole test id done without mutual TLS
>>
>> Hope this helps to find the problem
>> Greetings jason
>>

1) My first thought was, that the proxy is is no capable of the current
http chunked setting. So i tried both .. disabled and enabled.

2) Reading the proxy access and error protocol after sending the message.
Well, it seems that the proxy never gets reached using the original
sourcecode. The funny thing is that the proxy settings are correct set
because they also in use for the ITI-14(own) and ITI-15 transaction. I
have also debuged the proxy settings at the line 194 and seems to be ok.

Actually i have already found a workaround which seems to work by just
replacing the sending part of the B_Consumer Class with my own sending
class. With this solution everything seems to work properly with the
original NIST registry.
My own sender class now uses the sendReceive method of the
org.apache.axis2.client.ServiceClient package with the settings
options.setProperty(AddressingConstants.DISABLE_ADDRESSING_F OR_OUT_MESSAGES,
true); and
options.setProperty(HTTPConstants.CHUNKED, false);

The HTTP Client should then revert to creating its own socket for the
HTTP request and obey proxy settings.

-Matt

Jason wrote:
> Hi Matt
>
> Line 194-196 in the AbstractOHFSOAPSender is actually the TLS part which
> currently never gets reached because of the Non-TLS tests. So, after
> debuging only the code in the else part is reached.
> if (isDoingTLS()) {
> 194-196: sProtocol = new Protocol("https", (new
> OHFSocketFactoryWrapper(agent.getSocket(mEndpoint, true))), 443);
> Protocol.registerProtocol("https", sProtocol);
> } else {
> ... here we go:
>
> Problem Section: Socket s1 = agent.getSocket(mEndpoint, false);
> OHFSocketFactoryWrapper ofw = new OHFSocketFactoryWrapper(s1);
> sProtocol = new Protocol("http", ofw, 80);
> Protocol.registerProtocol("http", sProtocol);
> }
>
> Consequently i did also another tests on it.
>
> 1) My first thought was, that the proxy is is no capable of the current
> http chunked setting. So i tried both .. disabled and enabled.
> 2) Reading the proxy access and error protocol after sending the
> message. Well, it seems that the proxy never gets reached using the
> original sourcecode. The funny thing is that the proxy settings are
> correct set because they also in use for the ITI-14(own) and ITI-15
> transaction. I have also debuged the proxy settings at the line 194 and
> seems to be ok.
>
> Actually i have already found a workaround which seems to work by just
> replacing the sending part of the B_Consumer Class with my own sending
> class. With this solution everything seems to work properly with the
> original NIST registry.
> My own sender class now uses the sendReceive method of the
> org.apache.axis2.client.ServiceClient package with the settings
> options.setProperty(AddressingConstants.DISABLE_ADDRESSING_F OR_OUT_MESSAGES,
> true); and
> options.setProperty(HTTPConstants.CHUNKED, false);
>
> All tests are done as non-TLS
>
> Thanks for helping
> Jason
>

It looks from the stack trace that this request made it through all the
OHF code and out. We'd have to see your logs to be sure (you can attach
here or in a Bugzilla).

Does running one of the JUNITS in OHF for GetDocumentsQuery work by
itself with this registry from your environment?? Maybe try setting
your proxy settings in the runtime for your program (in case order is a
problem here). A final thought, is that since OHf code is administering
the connection for you to the registry URL, it may not permit proxy (or
we don't expose it in a way for you to make this switch).

Another note, unrelated to your question, you don't need the line:

Config.start(true);

This is only used if you want to work (backwards compatible) with XDS
registry implementations prior to 2007 ... where patient id's required
an assigning authority name in addition to the OID and ISO type. It's
not documented too well ... so another thing to add to our list :-) The
most recent set of test code is in the
src_tests/org.eclipse.ohf.ihe.xds.consumer.test.mesa package. Here --
the Test configuration and the base tests (ConsumerMesaTest.java and
B_ConsumerMesaTest.java) provide more up to date examples.

You've uncovered an issue that I'm not 100% sure how to fix and/or
workaround. IF you could provide the use scenario, that'll help us
determine a fix.

OHF's XDS component uses Apache Axis2 to handle the XDS Web service
transaction. Apache Axis2 uses the Apache Commons HTTPClient component
to handle the HTTP Web service. HTTPClient handles proxies without a
problem. Ordinarily the fix would be to uncover the static /
threadlocal API into HTTPClient that exposes how to set the HTTP proxy's
hostname, port, username, and password. This is well-documented.

The complicating factor is that, in OHF, we manage the socket connection
top to bottom. That is, we give Axis2 / HTTP Client the java.io.Socket
instance that it uses for the Web service connection. The principal
reason is because of the complicated requirements for mutual TLS under
IHE specifications. Now, we also do the same thing for non-TLS (plain
HTTP) connections, but mainly for consistency. This breaks the ability
to use HTTP proxies in OHF. We could easily revert this for plain HTTP,
but it is impossible for us to allow HTTPClient to create the socket for
TLS connections.

My question for you is... is this simply a development use case (e.g.
HTTP connections in your organization are proxied?) or something more
larger-scale? Also, are you expecting to use "IHE Node Authentication"
(e.g. mutual TLS) through this proxy as well? If so, could you maybe
help me better understand how that would work. These questions should
help us to find a fix.

Thanks for the report - this is definitely an important gap that needs
to be addressed.

Actually, i did some more tests on this problem. After hours of research
it seems that that the problem seems to be on a lower level. Normally you
create an instance of the B_Consumer or Consumer object which is then able
to create a valid query request. The creation of this message seems to be
ok, except of the sending part. The B_Consumer::invokeStoredQuery contains
a send call which fails. So after going deeper you can see that the lower
level AbstractXDSSOAPClient::send method fails on the part where it tries
to send the message. So after going deeper it seems that the
AbstractOHFSOAPSender fails on a specific method call. The problem
sections seems to be the "configureTransportProtocol" method on the part
where the getSocket method of the AtnaAgent is invoked. Thats the point
where an exception is thrown. ... Infact, the endpoit adress seems to be
valid and the chunked parameter is configured. Actually, it seems to be
mysterious problem.

A possible workaround...
It seems that a working workaround would be to write transaction like the
iti-14 (where we set the proxy in the same way) to straight forward the
well formated query to the registry. So the best way is to replace only
the sending call of the B_Consumer which has then the effect that this
transaction works properly.

By the way, it seems that the proxy never gets reached by the typical
sending call of the B_Consumer class.

Yep, what you found in AtnaAgent.getSocket() is indeed how we control
the socket construction. This allows us to manage, at runtime, the
certificates used in TLS authentication on a per-domain basis. However,
it's not a requirement for non-TLS connections, it's only there for
consistency.

I'm going to experiment with a couple things and let you know the results.

Thanks!
-Matt

Jason wrote:
> Hi Matt
>
> First, thanks for your answer!
>
> Actually, i did some more tests on this problem. After hours of research
> it seems that that the problem seems to be on a lower level. Normally
> you create an instance of the B_Consumer or Consumer object which is
> then able to create a valid query request. The creation of this message
> seems to be ok, except of the sending part. The
> B_Consumer::invokeStoredQuery contains a send call which fails. So after
> going deeper you can see that the lower level
> AbstractXDSSOAPClient::send method fails on the part where it tries to
> send the message. So after going deeper it seems that the
> AbstractOHFSOAPSender fails on a specific method call. The problem
> sections seems to be the "configureTransportProtocol" method on the part
> where the getSocket method of the AtnaAgent is invoked. Thats the point
> where an exception is thrown. ... Infact, the endpoit adress seems to be
> valid and the chunked parameter is configured. Actually, it seems to be
> mysterious problem.
>
> A possible workaround...
> It seems that a working workaround would be to write transaction like
> the iti-14 (where we set the proxy in the same way) to straight forward
> the well formated query to the registry. So the best way is to replace
> only the sending call of the B_Consumer which has then the effect that
> this transaction works properly.
> By the way, it seems that the proxy never gets reached by the typical
> sending call of the B_Consumer class.
> The whole test id done without mutual TLS
>
> Hope this helps to find the problem
> Greetings jason
>

If this works, I'm going to have you try a couple more things and then
we'll install a permanent fix for both non-TLS and TLS connections
(although, again, I'm not sure you would ever proxy a TLS connection).

Thanks,
-Matt

Matthew Davis wrote:
> Jason,
>
> Yep, what you found in AtnaAgent.getSocket() is indeed how we control
> the socket construction. This allows us to manage, at runtime, the
> certificates used in TLS authentication on a per-domain basis. However,
> it's not a requirement for non-TLS connections, it's only there for
> consistency.
>
> I'm going to experiment with a couple things and let you know the results.
>
> Thanks!
> -Matt
>
>
> Jason wrote:
>> Hi Matt
>>
>> First, thanks for your answer!
>>
>> Actually, i did some more tests on this problem. After hours of
>> research it seems that that the problem seems to be on a lower level.
>> Normally you create an instance of the B_Consumer or Consumer object
>> which is then able to create a valid query request. The creation of
>> this message seems to be ok, except of the sending part. The
>> B_Consumer::invokeStoredQuery contains a send call which fails. So
>> after going deeper you can see that the lower level
>> AbstractXDSSOAPClient::send method fails on the part where it tries to
>> send the message. So after going deeper it seems that the
>> AbstractOHFSOAPSender fails on a specific method call. The problem
>> sections seems to be the "configureTransportProtocol" method on the
>> part where the getSocket method of the AtnaAgent is invoked. Thats the
>> point where an exception is thrown. ... Infact, the endpoit adress
>> seems to be valid and the chunked parameter is configured. Actually,
>> it seems to be mysterious problem.
>>
>> A possible workaround...
>> It seems that a working workaround would be to write transaction like
>> the iti-14 (where we set the proxy in the same way) to straight
>> forward the well formated query to the registry. So the best way is to
>> replace only the sending call of the B_Consumer which has then the
>> effect that this transaction works properly.
>> By the way, it seems that the proxy never gets reached by the typical
>> sending call of the B_Consumer class.
>> The whole test id done without mutual TLS
>>
>> Hope this helps to find the problem
>> Greetings jason
>>

1) My first thought was, that the proxy is is no capable of the current
http chunked setting. So i tried both .. disabled and enabled.

2) Reading the proxy access and error protocol after sending the message.
Well, it seems that the proxy never gets reached using the original
sourcecode. The funny thing is that the proxy settings are correct set
because they also in use for the ITI-14(own) and ITI-15 transaction. I
have also debuged the proxy settings at the line 194 and seems to be ok.

Actually i have already found a workaround which seems to work by just
replacing the sending part of the B_Consumer Class with my own sending
class. With this solution everything seems to work properly with the
original NIST registry.
My own sender class now uses the sendReceive method of the
org.apache.axis2.client.ServiceClient package with the settings
options.setProperty(AddressingConstants.DISABLE_ADDRESSING_F OR_OUT_MESSAGES,
true); and
options.setProperty(HTTPConstants.CHUNKED, false);

The HTTP Client should then revert to creating its own socket for the
HTTP request and obey proxy settings.

-Matt

Jason wrote:
> Hi Matt
>
> Line 194-196 in the AbstractOHFSOAPSender is actually the TLS part which
> currently never gets reached because of the Non-TLS tests. So, after
> debuging only the code in the else part is reached.
> if (isDoingTLS()) {
> 194-196: sProtocol = new Protocol("https", (new
> OHFSocketFactoryWrapper(agent.getSocket(mEndpoint, true))), 443);
> Protocol.registerProtocol("https", sProtocol);
> } else {
> ... here we go:
>
> Problem Section: Socket s1 = agent.getSocket(mEndpoint, false);
> OHFSocketFactoryWrapper ofw = new OHFSocketFactoryWrapper(s1);
> sProtocol = new Protocol("http", ofw, 80);
> Protocol.registerProtocol("http", sProtocol);
> }
>
> Consequently i did also another tests on it.
>
> 1) My first thought was, that the proxy is is no capable of the current
> http chunked setting. So i tried both .. disabled and enabled.
> 2) Reading the proxy access and error protocol after sending the
> message. Well, it seems that the proxy never gets reached using the
> original sourcecode. The funny thing is that the proxy settings are
> correct set because they also in use for the ITI-14(own) and ITI-15
> transaction. I have also debuged the proxy settings at the line 194 and
> seems to be ok.
>
> Actually i have already found a workaround which seems to work by just
> replacing the sending part of the B_Consumer Class with my own sending
> class. With this solution everything seems to work properly with the
> original NIST registry.
> My own sender class now uses the sendReceive method of the
> org.apache.axis2.client.ServiceClient package with the settings
> options.setProperty(AddressingConstants.DISABLE_ADDRESSING_F OR_OUT_MESSAGES,
> true); and
> options.setProperty(HTTPConstants.CHUNKED, false);
>
> All tests are done as non-TLS
>
> Thanks for helping
> Jason
>