Another TCP echo server using IOCP

Introduction

This tip shows a very simple non-blocking TCP echo server using Windows Input/Output Completion Ports, capable of handling any number of clients simultaneously.

Background

IOCPs are generally considered the best (some would say the only) way of writing high-performance, highly-scalable network servers in Windows. But there are no simple examples of their use readily available in the internet.

This tip intends to fulfill this gap, by providing a very simple C-language IOCP server example. There are no classes to learn, no other Windows API to worry about, no thread stuff to complicate matters, just the two functions relating to IOCPs: CreateIoCompletionPort and GetQueuedCompletionStatus. Also, of course, the usual sockets API are used, in the Winsock asynchronous flavor: AcceptEx, listen, bind, WSARecv and WSASend.

Description

The general flow of control of the echo server is described in the steps below:

That's it! Well, there are all those pesky error situations that have to be handled, and there's also some code to track the connections, but all in all, the above pseudo-code sums up most of the program code. One thing to note though is that all operations must be "overlapped" (a kind of asynchronous I/O operation in Windows).

Also, there's a small client program (not using IOCPs) that can be used to test the echo server.

Conclusion

IOCPs are easy to use, and the program shipped with this tip intends to show how to do it.

Share

About the Author

I've been developing software for retail and banking automation in C/C++ for many years now. In the old days I even did some COBOL programming, and some assembly for the 8080.

My experience ranges from low level software such as interface code for serial devices for DOS and Windows (bar code scanners, printers, cash dispensers, etc) and goes to writing end user applications for POS terminals and bank ATMs. In between I've done a great deal of TCP/IP programming using the basic Berkeley sockets interface, which is my main interest nowadays.

Comments and Discussions

Firstly, a five star for your simple and useful sample to help in understanding IOCP.

If I am not mistaken, i didnt see any worker thread creation that is to be associated with the completion port. Besides, worker thread that spawned will block on IO completion port until request or completion packet kicks in, right? or the creation of IO Completion port with the concurrency value of 0 will automatically create thread based on processor counts that will serve the completion packets that are passed in to the queue?

If the latter is the case, then what purpose is the worker thread spawned for?? for non IO operation??As for specific business logic that is to implemented on the received data, for example, the server will act as the calculator that will return back the calculated output based on the numbers or data sent by the client; where can we implement it without making all the IO threads that serve IO completion port block?I am trying to get the big pictures on how this is working.

This example is not multi-threaded. There's no explicit thread creation, all the logic is being handled by the main thread, in the function run (and the other functions called by run). Even though it's not multi-threaded this little program can deal with many concurrent connections at the same time.

This event-driven design is not so much as common as explicitly (or even implicitly) multi-threaded designs, but it's used (as far as I know) in some high profile software like nginx and the libuv library behind node.js.

Anyway, this program is meant only to show in a simple and clear way the mechanics of using IOCP, though the way it uses IOCP is by no means the only or the best. This program is not intended as an example of how to write TCP servers in a general way.

In the current code next read is not called until we got write_completed.However, with a real high performance server we should allow next start_reading on the same socket that we sent to start_writing even before we got write_completed.

For sure we'll need additional SocketState instances for supporting this.Assuming I will maintain them all, is this legal?

"In the current code next read is not called until we got write_completed.However, with a real high performance server we should allow next start_reading on the same socket that we sent to start_writing even before we got write_completed."

Yes. I've got lots of code like that, with concurrent send and receive asynchronous operations on the same socket.

"For sure we'll need additional SocketState instances for supporting this.Assuming I will maintain them all, is this legal?"

For each asynchronous operation using IOCP we have to have an OVERLAPPED structure. That's how it works. I think my SocketState structure should encapsulate the OVERLAPPED structure, that's how it's always done. The way it is now, I'm using SocketState in place of the IOCP completion key. Although it works in my example, that's highly unusual.

Anyway you have to have in mind that this example is meant just to explain how IOCP works in a simple and understandable way. It's not meant to be used as a model for a real application.

you are right!I just found the following in MSDN (both for WSARecv and for WSASend):

If this function is completed in an overlapped manner, it is the Winsock service provider's responsibility to capture the WSABUF structures before returning from this call. This enables applications to build stack-based WSABUF arrays pointed to by the lpBuffers parameter.

I had already seen that, but I think that the MSDN article is a little confusing. Anyway, like I said elsewhere, at my workplace we have code using WSABUF variables on the stack, and don't have issues with that.