In this article, we’ll see this in practice as we implement the Remote Procedure Call (RPC) pattern in RabbitMQ. This is a fancy way of saying request/response, except that it all happens asynchronously! That’s right. No blocking.

The RabbitMQ.Client NuGet package is necessary to make this code work. The client is written using an asynchronous Main() method, which requires at least C# 7.1 to compile.

RabbitMQ RPC Overview

You can think of RPC as request/response communication. We have a client asking a server to process some input and return the output in its response. However, this all happens asynchronously. The client sends the request on a request queue and forgets about it, rather than waiting for the response. Eventually, the server will (hopefully) process the request and send a response message back on a response queue.

The request and response can be matched on the client side by attaching a CorellationId to both the request and the response.

In this context, we don’t really talk about publishers and consumers, as is typical when talking about messaging frameworks. That’s because in order to make this work, both the client and the server must have both a publisher and a consumer.

Client: Main Program

For our client application, we’ll have the following main program code. We will implement an RpcClient that will hide the request/response plumbing behind a simple Task that we then await:

The program continuously asks for input, and sends that input as the request message. The server will process this message and return a response. Note that we are using the ScopedConsoleColour class from my “Scope Bound Resource Management in C#” article to colour certain sections of the output. Here is a taste of what it will look like:

While this console application only allows us to send one request at a time, the underlying approach is really powerful with APIs that can concurrently serve a multitude of clients. It is asynchronous and can scale pretty well, yet the consuming code sees none of the underlying complexity.

Client: Request Sending

The heart of this abstraction is the RpcClient class. In the constructor, we set up the typical plumbing: create a connection, channel, queues, and a consumer.

Here, we are generating GUID to use as a CorrelationId, and we are adding an entry in the dictionary for this request. This dictionary maps the CorrelationId to a corresponding TaskCompletionSource. When the response arrives, it will set the result on this TaskCompletionSource, which enables the underlying task to complete. We return this underlying task, and that’s what the main program awaits. The main program will not be able to continue until the response is received.

In this method, we are also calling a private Publish() method, which takes care of the details of publishing to the request queue on RabbitMQ:

While this publishing code is for the most part pretty standard, we are using two particular properties that are especially suited for the RPC pattern. The first is CorrelationId, where we store the CorrelationId we generated earlier, and which the server will copy and send back as part of the response, enabling this whole orchestration. The second is the ReplyTo property, which is used to indicate to the server on which queue it should send the response. We don’t need it for this simple example since we are always using the same response queue, but this property enables the server to dynamically route responses where they are needed.

Server

The request eventually reaches a server which has a consumer waiting on the request queue. Its Main() method is mostly plumbing that enables this consumer to work:

In this example, the server’s job is to reverse whatever messages it receives. Thus, each response will contain the same message as in the corresponding request, but backwards. This reversal code is taken from this Stack Overflow answer. Although trivial to implement, this serves as a reminder that there’s no need to reinvent the wheel if somebody already implemented the same thing (and quite well, at that) before you.

The response is sent back on the queue specified in the ReplyTo property of the request message. The response is also given the same CorrelationId as the request; that way the client will know that this response is for that particular request.

Client: Response Handling

When the response arrives, the client’s own consumer event handler will run to process it:

The client extracts the CorrelationId from the response, and uses it to get the TaskCompletionSource for the corresponding request. If the TaskCompletionSource is found, then its result is set to the content of the response. This causes the underlying task to complete, and thus the caller awaiting that task will be able to resume and work with the result.

If the TaskCompletionSource is not found, then we ignore the response, and there is a reason for this:

“You may ask, why should we ignore unknown messages in the callback queue, rather than failing with an error? It’s due to a possibility of a race condition on the server side. Although unlikely, it is possible that the RPC server will die just after sending us the answer, but before sending an acknowledgment message for the request. If that happens, the restarted RPC server will process the request again. That’s why on the client we must handle the duplicate responses gracefully, and the RPC should ideally be idempotent.” — RabbitMQ RPC tutorial

Demo

If we run both the client and server, we can enter messages in the client, one by one. The client publishes each message on the request queue and waits for the response, at which point it allows the main program to continue by setting the result of that request’s TaskCompletionSource.

Summary

A TaskCompletionSource has an underlying Task that can represent a pending request. By giving each request an ID, you can keep track of it as the corresponding response should carry the same ID. A mapping between request IDs and TaskCompletionSource can easily be kept in a dictionary. When a response arrives, its corresponding entry in the dictionary can be found, and the Task can be completed. Any client code awaiting this Task may then resume.

In this article, we’ll learn how to use TaskCompletionSource. It’s one of those tools which you will rarely need to use, but when you do, you’ll be glad that you knew about it. Let’s dive right into it.

TaskCompletionSource is just a wrapper for a Task, giving you control over its completion. Thus, a TaskCompletionSource<bool> will contain a Task<bool>, and you can set the bool result based on your own logic.

Here, we are using TaskCompletionSource as a synchronization mechanism. Our main thread spawns off an operation and waits for its result, using the Task in the TaskCompletionSource. Even if the operation is not Task-based, it can set the result of the Task in the TaskCompletionSource, allowing the main thread to resume its execution.

Let’s add some diagnostic code so that we can understand what’s going on from the output:

As you can see, the main thread waited until tcs.SetResult(true) was called; this triggered completion of the TaskCompletionSource’s underlying task (which the main thread was awaiting), and allowed the main thread to resume.

Aside from SetResult(), TaskCompletionSource offers methods to cancel a task or fault it with an exception. There are also safe Try...() equivalents:

SDK Example

One scenario where I found TaskCompletionSource to be extremely well-suited is when you are given a third-party SDK which exposes events. Imagine this: you submit an order via an SDK method, and it gives you an ID for that order, but not the result. The SDK goes off and does what it has to do to perhaps talk to an external service and have the order processed. When this eventually happens, the SDK fires an event to notify the calling application on whether the order was placed successfully.

Notice how MockSdk‘s SubmitOrder does not return any form of Task, and we can’t await it. This doesn’t necessarily mean that it’s blocking; it might be using another form of asynchrony such as the Asynchronous Programming Model or a messaging framework with a request-response fashion (such as RPC over RabbitMQ).

At the end of the day, this is still asynchrony, and we can use TaskCompletionSource to build a Task-based Asynchronous Pattern abstraction over it (allowing the application to simply await the call).

We then add a dictionary, which allows us to relate each OrderId to its corresponding TaskCompletionSource. Using a ConcurrentDictionary instead of a normal Dictionary helps deal with multithreading scenarios without needing to lock:

This method calls the underlying SubmitOrder() in the SDK, and uses the returned OrderId to add a new TaskCompletionSource in the dictionary. The TaskCompletionSource’s underlying Task is returned, so that the application can await it.

When the SDK fires a completion event, the proxy will remove the TaskCompletionSource from the pending order and set its result. The application awaiting the underlying task will resume and take a decision depending on the logic.

We can test this with the following program code in a console application: