Memory Stream Multiplexer–write and read from many threads simultaneously

MemoryStreamMultiplexer is a MemoryStream like buffer manager where one thread can write and many threads can read from it simultaneously. It supports blocking reads, so that reader threads can call .Read() and wait for some data to be written. Handy for loading data in one thread that is consumed c

Introduction

Here’s an implementation of MemoryStream like buffer manager where
one thread can write and many threads can read simultaneously. Each reading
thread gets its own reader and can read from the shared stream on its own
without blocking write operation or other parallel read operations. It supports
blocking Read call so that reader threads can call Read(…) and wait
until some data is available, exactly the same way you would expect a
Stream to behave. You can use this to read content from network or
file in one thread and then get it read by one or more threads simultaneously. Readers do not block writing. As a result, both read and write happens concurrently.
Handy for building http proxy where you are downloading a certain file and you
have multiple clients asking for the same file at the same time. You can
download it in one thread and let one or more client threads read from the same buffer
exactly at the same time. You can also use this to read same file on disk from
multiple clients at the same time. You can also use this to implement a server
side cache where the same buffer is read by multiple clients at the same time.

How does it work

First you create a MemoryStreamMultiplexer object that holds the
shared buffer. It has a Write(…) method to write
byte[] to the shared buffer. Then you call GetReader()
to get a MemoryStreamReader created that can read the content from
the shared buffer. You can call GetReader() from a different thread
so that you can read and write simultaneously. Whenever you call
Write(…) it signals all the MemoryStreamReader
instances that content is now available to read. The readers that were
waiting on a Read(…) call gets the signal and reads from the shared
buffer.

It maintains a list of ManualResetEvent that is used to signal
the reader. Each reader gets two ManualResetEvent passed to it. One
to signal whenever a Write() happens, so that it can unblock the
Read call made by the reader threads and let them process the newly
available content. The other one to signal the Readers that writing has finished
so that it can stop expecting more content from the buffer.

Next is the MemoryStreamReader where most of the complicated
code lies.

I have tested it thoroughly on a Quad Core PC to make sure parallel reads really
happen and no thread overlaps on each other. I made sure the number of locks
hold are also minimal. You can see parallel Write and Read happening from the
Console output:

The above console output shows you that both read and write happening concurrently.

Performance testing the library

Here’s the Visual Studio 2010 Profiler report. It shows the most expensive code
is GetReader only and there’s no other function that comes anywhere
close to it. This is a good indication that the implementation is good enough.

Even in the GetReader function, the most expensive line of code is
creating the MemoryStreamReader:

When you do the Concurrency analysis to see which thread is doing what, it shows
that the reader threads read available content as soon as the writer thread
writes to the shared buffer. There’s no delay in reader threads getting the
signal and reading the recently added content.

The green bars on the threads show that as soon as the writer thread (Thread
7784) signals (the yellow bars), the reader threads execute and pickup the data.
There’s just one thread 8524 which seems to struggle picking up the signal for
some reason. But rest of the threads pickup the signal and read the recently
added bytes within 0.02ms on average.

Comments and Discussions

Read from and Write to the same buffer simultaneously has been a problem since forever!
Most Operating Systems have an accessible implementation of some sort for this issue.
In Windows and for .NET it is called ReaderWriterLock (System.Threading.ReaderWriterLock[^]). Also, ReaderWriterLockSlim (System.Threading.ReaderWriterLockSlim[^]) can be used.
To me your code is trying to do what ReaderWriterLockSlim does. Please feel free to correct me if I am wrong
I have not done benchmarking on these against your code but I have used them all the time with no performance issue so far.