What I was expecting is if I make an invocation from the server to the client as is currently the case for delivering messages to clients, then the http transport is clever enough to realise it is only allowed to make invocations from client to server, so uses some kind of polling mechanism to get messages from server to client.

This would all be insulated from the user of remoting (us) so we wouldn't have to change any code.

If this is is not the case, then it's going to be a lot of grief to change how messaging uses the remoting api to get it to work.

"timfox" wrote:One of the selling points of remoting was that we could use the remoting API in a consistent way irrespective of the transport which was plugged in.

This clearly appears not to be the case with the HTTP transport.

As far as I can see, "push callbacks" are currently being done with the socket transport, but they're not Remoting "push callbacks". If I remember correctly, somewhere on the server side there is an explicit call to org.jboss.remoting.Client.invoker(). There shouldn't be any distinction in the way the socket and http transports handle this behavior.

"timfox" wrote:What I was expecting is if I make an invocation from the server to the client as is currently the case for delivering messages to clients, then the http transport is clever enough to realise it is only allowed to make invocations from client to server, so uses some kind of polling mechanism to get messages from server to client.

Here's the call on the server side that calls the callback server on the client side. It's in ServerConsumerEndpoint.Deliver.run():

1. The messaging client needs to connect to the server over HTTP2. We need it implemented in such a way that there's no socket listening on the client, but the client can still receive messages asynchronously.3. When the change is completed, all current Messaging functional tests, stress tests and smoke test must pass.

We have:

- Remoting- Current Messaging code base

Do whatever you think it's necessary, in your work area, to make the above requirements a reality.

Ron, you can use one of the Client.addListener() methods that supports uni-directional callback polling (which will simulate push callbacks by using internal polling for uni-directional transports). Can use the same method when using bi-directional transport (for true push callbacks on same socket connection), so there is no code change needed when switching transports (i.e. from http to multiplex). The javadoc explains which addListener() methods would be best fit (see http://labs.jboss.com/portal/jbossremoting/docs/api/org/jboss/remoting/Client.html). These methods were added in May per request from the messaging team (http://jira.jboss.com/jira/browse/JBREM-363). Can also take a look at org.jboss.test.remoting.callback.push.unidirectional.CallbackPollTestClient to see it in use. For the uni-directional transports, probably the only config that you will want to consider is callbackPollPeriod (org.jboss.remoting.callback.CallbackPoller.CALLBACK_POLL_PERIOD), for setting how often client should poll for callback messages internally.

As for type of streams being passed to marshalling layer, I would still be willing to change the remoting code (socket invoker in particular) so that marshalling layer is never passed an object stream but just the raw stream. However, this seems to be the opposite of what asked for in http://jira.jboss.com/jira/browse/JBREM-371.

As a first approximation, replacing the current Messaging push callbacks with Remoting's polling simulation of push callbacks is pretty easy. On the client side, CallbackManager is transformed from a ServerInvocationHandler to an InvokerCallbackHandler. When the Remoting poller pulls callbacks from the server, it passes them to CallbackManager, which proceeds as before.

On the server side, instead of passing around the remoting callback Client, which was being used to make explicit invocations that were processed by CallbackManager, the ServerInvokerCallbackHandler from which the callback Client was obtained is passed around. Instead of calling Client.invoker(), ServerConsumerEndpoint.Deliverer calls ServerInvokerCallbackHandler.handleCallback(), which, since it is configured for pull callbacks, just stores the callback with its load of messages. Eventually the client side poller will retrieve them and pass them to CallbackManager.

To some extent, these changes work. E.g., simple tests like HttpExample and QueueExample work. However, some of the more complex tests, for example, MessageConsumerTest.redel8(), fail. It looks like CallbackManager is trying to deliver the messages to a Consumer that has already closed.

"tim_fox" wrote:... when you close a consumer, closing() first gets called on the ServerConsumerEndpoint, this causes stop() to be called, which waits for any deliveries of messages to the client which might be in progress to complete first.

Once that is done, the actual client consumer is closed and removed from the callback manager.

Looking a the code again, I noticed something I missed the first time around. When MessageCallbackHandler gets a message, it does one of two things:

1. If a thread is blocked in receive(), it wakes up the thread and returns. 2. If a MessageListener is registered, it queues up a Runnable to call onMessage().

Now, 1 is irrelevant if the MessageConsumer has called close(), so I thought the point was to make sure that a MessageConsumer didn't close while a listener was in onMessage(). But it looks like that could still happen. Is that ok? Just curious.

If a session executor's thread is currently executing onMessage() and close() is called on the consumer from a different thread, the closing thread is put on wait until the listener code execution completes.

Take a look at MessasgeCallbackHandler.waitForOnMessageToComplete() (the method body, and also, its invocation point)

I'm copying this from the Remoting forum in case nobody has a watch on it.

"timfox" wrote:

Messaging has no requirement to get acknowledgements that messages have been delivered to the client side.

There may be no formal messaging requirement for acknowledgements, but it seems to derive from the current implementation. Currently, messaging delivers messages from the server to the client by the synchronous org.jboss.remoting.Client.invoke() method, and ServerConsumerEndpoint.stop() waits for the completion of the call to invoke():

this.executor.execute(new Deliverer());
//Now wait for it to execute
Future result = new Future();
this.executor.execute(new Waiter(result));
result.getResult();

Now, if you want to have the client side poll for messages, then the completion of the Deliverer only means that the messages have been handed over to Remoting. My understanding is that you want all messages that have left ServerConsumerEndpoint to be delivered to MessageCallbackHandler before concluding ServerConsumerEndpoint.stop(), and the only way I can see to get that information is to get an acknowledgement from the client side. Let me know if I've misinterpreted something.

"timfox" wrote:

(The HTTP transport might use acknowledgments internally to manage it's buffer -although I am not sure that is the best way to do it- but that's an implementation detail of the HTTP transport).

My understanding is that this issue isn't specific to the HTTP transport. Messaging's current use of the socket transport involves a callback org.jboss.remoting.Client on the server side contacting a ServerSocket on the client side. I thought you wanted to avoid the client side ServerSocket in all cases. Or does this restriction apply only to the HTTP transport?

I've run into another issue. When memory is running short, the server side callback handler has the ability to persist callbacks to secondary storage. Currently, there is one true persisting callback store, which I guess predates jboss serialization, since it uses java serialization. Therefore, it can't handle some Messaging callbacks. Extending it to use jboss serialization shouldn't be a problem. However, Tom mentioned that, at Tim's request, he implemented a BlockingCallbackStore, which simply blocks the thread passing calllbacks to Remoting.

So I'm wondering what the Messaging philosophy here. Do you want to have the option to configure a true persisting callback store?