Introduction to Win32 Named Pipes (C++)

There are times when it’s extremely useful to be able to pass some data between different programs running on the same system. For example, you might have multiple programs forming part of the same package, and they need to share some important information or work together to process something.

There are several ways to do it, but my choice in a recent task was to use the named pipes functionality of the Win32 API (working in C++). Note that pipes on other operating systems are a little different, so not all of this information is portable.

What is a pipe?

The “pipe” metaphor is fairly simple. Just like a literal pipe can carry water from one place to another, the metaphorical pipe carries data from one program to another. However, unlike most literal pipes around your home, the metaphorical pipes can easily support two-way flow.

In practical terms, the pipe is accessed very much like a file. Some of the behaviour is a little different though (in particular, it is more like a client-server architecture), and there are various other commands to be aware of. It is especially important to learn where things can go wrong, and what error codes to look out for.

A simple example

Here’s a quick overview of the steps required to create and use a simple named pipe to send data from a server program to a client program.

I’ve included full source code for each program at the bottom of this article. This is a very simple example though, and there’s lots more you can do with pipes on Win32. Take a look at the MSDN article on Named Pipe Operations for more information on other useful functions.

Pipe names

You can name Win32 pipes almost anything you like, as long as they start with the prefix “\\\\.pipe\\”. In practice it becomes “\\.pipe”, since you have to escape backslashes in C++ strings. Everything after that in the name is up to you, so long as you don’t use backslashes though, and don’t exceed 256 characters in total.

Read/write modes

There are two main modes of read/write operation available on pipes: byte stream, and message. The difference is fairly small, but can be very significant depending on your application.

Message mode simply makes a distinction between each set of data sent down the pipe. If a program sends 50 bytes, then 100 bytes, then 40 bytes, the receiving program will receive it in these separate blocks (and will therefore need to read the pipe at least 3 times to receive everything).

On the other hand, byte stream mode lets all the sent data flow continuously. In our example of 50, 100, then 40 bytes, the client could happily receive everything in a single 190-byte chunk. Which mode you choose depends on what your programs need to do.

Overlapped pipe IO

By default, pipe operations in Win32 are synchronous, or blocking. That means your program (or specifically the thread which handles the pipe operations) will need to wait for each operation to complete before it can continue. This can seem frustrating, but it makes programming much simpler. When one of the pipe functions returns, it means you know it has either been successful or it has failed.

Using overlapped pipe IO means that pipe operations can process in the background while your program continues to do other things (including running other pipe operations in some cases). This can be very helpful, but it means you have to keep track of which operations are in progress, and monitor them for completion.

An alternative to overlapped operation is to run synchronous pipe operations in a separate thread. If your pipe IO needs are fairly simple then this may be a simpler option.

Buffered input/output

When calling “CreateNamedPipe(..)” you can choose to specify buffer sizes for outbound and inbound data. These can be very helpful for program performance, particularly in synchronous operation. If your buffer size is 0 (which is entirely valid) then every byte of data must be read from the other end of the pipe before the write operation can be completed.

However, if a buffer is specified then a certain amount of data can linger in the pipe before it gets read. This can allow the sending program to carry on with other tasks without needing to use overlapped pipe IO.

The “Hello Pipe World” pipe example

I’ve included source code below for the server and client programs. I recommend using Visual C++ (any version should be fine), and creating each one as a Win32 console application. I’ve programmed them to cope with Unicode, since that’s the direction Windows programs seem to be going these days.

Important note: When running the programs, run the server first! The client program fails if the pipe is not available.

I'm a software engineer at Vertual Ltd., working on virtual reality medical training systems for radiotherapy. I work mainly with C/C++, using OpenGL for graphics. However, I also have experience with C#, Java, Python, PHP, SQL, JavaScript, and HTML/CSS.
I have a BSc Hons. (1st class) in Computer Games Technology, and am currently writing-up a PhD thesis in educational technology. In my free time, I'm a Karate Nerd, keen musician, electronics hobbyist, and all-round geek.

46 comments on “Introduction to Win32 Named Pipes (C++)”

Nice tutorial, but I would be interested its overlapped version too;-) I’m writing a tracing program for my other programs and it tends to skip some of the sent messages and I can’t find the source of the problem. I’ve written it based on tutorial on msdn but there’s no client version part (I used simple synchronous write, so I can only hope that’s fine)

Thanks. A lot of the basic calls for overlapped mode are pretty much the same. Unfortunately it’s harder to illustrate neatly though, as the structure can depend heavily on the rest of your program, but I’ll keep it in mind for future posts.

Losing messages can be caused by several issues with pipes, such as a connection temporarily being broken, or the pipe being busy. The best thing I can recommend is making a log file of any failed sends, along with the error code that was reported by GetLastError().

You can convert the error code to text using FormatMessage(). Details are on MSDN:

I created a win32 console app several years ago using code similar to what demonstrated above. It was developed on XP and has been successfully running in production for several years. I now have to make this code available for the windows 7 64 bit platform. I’ve compiled the code on the 64 bit platform, but here’s the problem. I can successfully create the server-side end of the pipe, connect to it, and set it up for reading and writing, but when the code for the client-side executes, i.e., CreateFile(..), it fails because it cannot find the file specified…meaning it cannot find the pipe created via CreateNamedPipe. Any ideas?

Hi Marcus. I haven’t tried using pipes in a 64-bit executable, but I’ll try to give it a shot later today or tomorrow.

In the meantime, this may be a really obvious question, but did you change the code to handle pointers as 64-bit instead of 32-bit? That may or may not be necessary, depending on how you implemented it.

Your comment about 64 bit pointers caused me to review the old code. The code to create the client pipe is using JNI and the only explicit pointer being used is to get the pipe name (sent from my Java code). I’ve checked the value of pipeName before calling CreateFile, and it is same name that was used in my call to CreateNamedPipe. As soon as I get access to the test machine again, I’ll hard code the pipe name in CreateFile to see what happens. I know for a fact that the server end of the pipe is successfully created, connected to, and waiting for the client end to connect.

Do you know what error code is being reported by CreateFile()? It’s not always helpful, but it might shed some light on the problem.

Also, have you tried specifying FILE_ATTRIBUTE_NORMAL for the file flags/attributes parameter? It’s the second last argument in the CreateFile() call, and I don’t think it’s technically supposed to be 0.

The error message is “The system cannot find the file specified.” and error code 2 is returned. I hard-coded the value of the pipe name in the client code, and I received the following error:
The filename, directory name, or volume label syntax is incorrect.” and error code 123 was returned. Changing the flags/attributes parameter resulted in the same error message/error code = 123.

It looks like the problem is that your struct is only holding a pointer to the address data. A pointer only indicates the location in memory that the data is stored, and you usually can’t pass pointers from process to another. (You certainly can’t pass them from one system to another.)

Instead, you have to copy the data itself. The simplest solution is to store the address data in the same way as you store the name — i.e. using a fixed-size array of chars.

I’m afraid passing the pointer itself can’t work, because one system can’t access the other system’s memory over the network. Instead, you’ll need to pass the data which is being pointed-to. That means you’ll need to pass the address data separately from the rest of the struct.

Assuming the address data is a null-terminated string, you could do something like this:

On the client side, when it expects the address data, it should keep reading bytes one-by-one until it hits the null character, and then stop. That will be the end of the address data. The alternative would be to pass the length of the string down the pipe first so the client knows how many bytes to expect.

Hi,
I am trying to have 2 separate server threads to read and write on the same pipe handle. It doesn’t seem to work at all. My reading thread comes back with a random error every time the writing threads writes into the pipe. Introducing a Critical Section creates a deadlock. How do I do this correctly? My server application is supposed to constantly write into the pipe and occasionally read as the client will only write back once in a blue moon.
Thanks

Hi! Very good article – I’ve learnt a lot from your example. Now I have a bit of a problem with my pipes:
I have program (actually it’s an DLL) that is gathering data from game using SDK made by the developers – this is my client.
Then I have server that sending that data somewhere else via sockets. Server receives data from client using pipe (or I rather should say received). All of this has been working well, but after some changes in sended structure suddenly stopped working that way.
I know that client is logging data, but have problems with sending it. Client connects properly, server accepts connection and then… nothing works like it should. Log shows that client sends 0 bytes on every attempt.
Here is some code:
Structure that I’m trying to pass through pipe:

Thanks for your pages, it’s very helpful. One question I have is similar to a preceding post and something which is glossed over on the MS examples. Some words about data type compatibility would be useful. For example: The examples all show bytes/chars/strings being transmitted through pipes. If I want to send integers/floats and so on, can this be done? Must everything be converted to bytes first? If so ( I think I can read about casts) how do I know how to re-convert to the correct data type at the receiver? Do I need to make some kind of protocol around the pipe?

Pipes don’t really care about data types. They just send whatever raw binary data you give them. It’s often good to use strings though, as it lets your program handle all of the data formatting, without worrying about binary compatibility between programs/computers (issues like padding and endianness… they’re best avoided if possible!).

Your sender will need to convert all your data into a text format, and your receiver will need to convert it back. If you’re using C++ (rather than C), I recommend using string streams to achieve that.

The first thing to do is include the header file, which is:

#include

In this example, let’s say we want to transmit 2 integers and a float. This code will write the numbers into a string stream, with spaces between them:

int num1 = 17, num2 = 24;
float num3 = 14.5f;

std::wstringstream strm;
strm < < num1 << ' ' << num2 << ' ' << num3;

The strm object now contains the text: "17 24 14.5". You can send it over the pipe like this:

By default, the string stream skips over spaces when you’re reading data out, so you don’t need to include them here. (It’s important to include them when sending though, so it knows where one number stops and the next one starts).

As a quick note, if you’re not working with Unicode enabled in your program, you’ll need to use std::stringstream (instead of std::wstringstream).

Obviously, all of that relies on your receiver knowing what kind of data is arriving. A typical approach is to include ‘header data’ at the start of every transmission. It will be the same format every time, and will include information about what is being sent. You could use an integer as a kind of message identifier. For example, you could say message ID 19 contains the current time expressed as 3 integers (hours, minutes, seconds). The possibilities are totally up to you.

I hope that makes sense and is helpful. String streams are extremely versatile and effective once you get used to them!

That is very helpful, and thanks for replying. I’m using ‘C’ (Dev-C++ IDE), so I still have a couple of questions as I dont think I will be able to use streams (although alternative suggestions are always welcome). I have just about given up on my assignment anyways (!), here’s what I have been doing so far to explain my situation.

My task is to transmit from process to process using a named pipe. My pipes are created ok but I seem to get exception errors because of the way I am bringing data into the write to pipe command (sorry, the source code is on my home PC so working from memory here).

My code uses a structure(actually, a pointer to a struct) to move data around from function to function, so what I have done is put the functions that process data into another process. All I need to do is send the data structure down the pipe from process A to process B and I am done (easy , right…).

So, I am using memcpy to read from the structure to a buffer before sending the buffer to the write file command. Unfort I get exception errors. If I change memcpy to use * or & I sometimes get no exception but only 5 bytes written instead which seems to me maybe I am sending an address instead of memory contents.
I realise this is not really a named pipe issue but I’d appreciate any thoughts – but I also realise it is hard to debug invisible code !

OK, it sounds like you should be able to send raw binary data quite safely in that case. The most important question is whether or not your structure contains any pointers? If it just has plain old data (e.g. ints, floats, or static arrays[]) then there’s no need to copy it all to an intermediate buffer first. You can send it down the pipe directly. For example:

There are a couple of key things to watch out for. Firstly, myData is already a pointer, so all you need to do is cast the pointer’s type (don’t use (&myData) or you’ll get a pointer to your pointer instead!). Similarly, make sure you don’t do sizeof(myData), because that will only get the size of the pointer, and not the size of the original structure.

Unfortunately, if your structure contains pointers to other data (including strings declared as char*) then it’s more complicated. If that’s the case, I’d definitely recommend refactoring it if possible.

I’m trying to understand the security implications of using these pipes. Obviously, there are situations where you might want to restrict access to only the owner of the pipe. I searched around and found this page:

I followed the links on that page to the links on more pages, to the links on still more pages, etc. Right now I have 22 MSDN pages open in Firefox, and still can’t understand how to control access to a named pipe. I finally came to a dead end with a function named ConvertStringSecurityDescriptorToSecurityDescriptor() but it doesn’t seem to be supported in mingw32.

Surely there must be an easier way around this? (Wishing I could just say “chmod 600 ThePipe”

I’ve never actually used security options with named pipes. I think it’s just a case of creating an appropriate descriptor, and passing it to CreateNamedPipe() on the server side. As far as I can tell, the client side call to CreateFile() doesn’t need one (it will pick-up whatever the server has specified).

Hi Peter, just to let you know you need to change the code sections (the #include lines), html doesn’t display which header files are supposed to be included thanks to the angle brackets (at least not in my browser, chrome v25.0.1364.97). I managed to find sstream in the source from the comments, but in the code sections I can’t find the names even by looking at the page source.

Thanks very much for pointing that out, mtosiv. I had to edit it recently to fix something else, and WordPress must have filtered those lines out thinking they were incorrect HTML tags. It should hopefully be better now.

Hello Peter, and thank you for your example. I’m working on a solution that is using named pipes between C# and C++. I have this working. The server is C# and the client is C++. My problem is that I can not always re-connect to the service. The client needs to be able to connect, disconnect, connect, disconnect, … . The first connection is no problem, but sometimes the re-connect is failing with ERROR_FILE_NOT_FOUND on the client.

The problem seems to be in the Server. I have a while loop that creates the pipe, connects to the pipe, and then hands the handle off to a thread, that takes care of the receive.(The pipe is created with PIPE_ACCESS_INBOUND |FILE_FLAG_OVERLAPPED, PIPE_READMODE_BYTE, and PIPE_UNLIMITED_INSTANCES). Then the loop starts back at the top, creating a new pipe instance, and blocks on the connect.

I do a check that the handle is valid before moving on to the connect, and I’m also calling GetLastError() regardless of the state of the handle. The first pipe instance indicates that there is no error, but the second instance the error is ERROR_ALREADY_EXISTS. The behavior from this point on is different on different computers. On my dev machine, the client is able to reconnect almost every time, but on the test machine, reconnect almost always fails. Any insight would be very helpful.

Hi TaterTot. If I understand correctly, your main thread is trying to create another named pipe while your worker thread is still using the previous pipe. If the pipes have the same name then that would explain the problem. You’d need your main thread to wait until the worker thread confirms that the previous pipe has been closed.

Hi Karla. I’m not sure I can offer you much help with that. It looks like you’re using interprocess pipes on *nix, which is somewhat different to named pipes on Win32. My guess would be that you actually want to setup a pipe from the “ls” process to “more”, and then another from “more” to stdout.

Hi Peter,
First of all, I would like to thank you for this excellent tutorial; short and to-the-point.

Here, I pointing out some problems that I faced while implementing the programs on my machine using Visual Studio C++ under Windows 7 64-bit, and the solutions that I found so that others can benefit in case they run into the same problems.

When implemented as is the server and client programs have two errors:
1- Build fails as the program requires
#include “stdafx.h”;
so add this as the first line of code to both the client and the server programs to get them to build successfully. (I know this is trivial, but you never know

Hi Ahir. If both processes are on the same system, I would normally expect pipes to be a lot faster. This is because sockets require quite a lot of overhead in the transport layer and protocol etc. The actual data transfer rate depends on various factors, but I would expect 200 MB to take only a few seconds via local pipes.

hello peter. i have created an application in which each of client and server has it’s own thread for receive data and method for sending data. but i can send normal char buffer[] but i have problem with char* buffer[]. is there any difference in writefile() and readfile() for sending data? i am using this function like…
WriteFile(Pipe,sendData,byteToWrite,&WrittenByte,NULL);. is it true?

About

Avid Insight is my personal blog about various software, programming, electronics, and occasionally academic things. I also have a few past projects linked in the navigation menu above, so feel free to look around!