If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register or Login
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

[RESOLVED] Questions about a sample code on shared memory.

Hi,

I attached a sample code of shared memory that I downloaded from somewhere. However there are few things about it:

1. Producer must be started first, o.w. consumer gives an error: "shm_open: No such file or directory". -- I want to be able to start the processes in any order.
2. If Producer restarts, then Consumer must be restarted as well. -- That's very inconvenient, if my producer goes down then I'd have to restart Consumer as well.
3. Consumer takes 100% of CPU time of one core. -- I think that's because Consumer is polling the buffer, can it be resolved using semaphores?

Re: Questions about a sample code on shared memory.

>> ... that I downloaded from somewhere.
That code is garbage. Don't even try to learn something from it.

This looks better for learning: https://github.com/risingape/producer-consumer
It's a double-buffer technique instead of a ring-buffer. Once you understand how to synchronize with semaphores, applying it to a ring-buffer approach isn't too difficult (there are examples of that on the inter-web too).

The producer pushes items on the queue using push() and if the consumer calls pop() it will sleep waiting for an item to arrive. The try_pop() method returns immediately with or without an item depending on whether there was one.

Re: Questions about a sample code on shared memory.

"shared memory" does not require an order of who starts first, although if you want a consumer to start before a producer, this may involve additional synchronisation and code to properly initialise the shared memory.

It appears your code sample isn't about shared memory, but about using a memorymapped file to achieve more or less the same thing.
Again, this can be changed to consumer first, but it may require additional synchronisation to have the consumer "start up" the file mapping (possibly creating the file) if it doesn't already exist.

Re: Questions about a sample code on shared memory.

@OReubens, that code sample is about using shared memory without locking so to reduce latency. I am looking at the codeplug's sample now.

@razzle, thanks for the code snippet, I'll re-use it somewhere. However, in this particular case I essentially have two applications - two executables that do not share any code, not sure if I can use container.

Re: Questions about a sample code on shared memory.

@codeplug - these are some financial data and there are a lot of them. I am using FIFO right now (with your help too btw ) but I found it to be somewhat too slow.

So it's an event-driven application with Data Adapter (producer) that either reads data from the internet over sockets in real-time or it reads stored data from a file / database. I have Data Analyzer (consumer) that processes those data. Right now producer sends serialized data to consumer over FIFO. It works alright when producer reads the data from the Internet but when producer reads the data from the database then FIFO becomes a bottleneck. Note, that my consumer cannot go directly into a file / database to read the data, the data must go from producer to consumer - that's the requirement.

To replace FIFO I am considering to use shared memory. I've also just started looking at the Observer Pattern.

Re: Questions about a sample code on shared memory.

@OReubens, that code sample is about using shared memory without locking so to reduce latency. I am looking at the codeplug's sample now.

Now you're confusing a lock free algorithm
with shared memory
and with memory mapping.

if you really have to, you can implement a lock-free single linked list in shared memory which is slightly more complex as a regular implementation as you need to account for the fact that the list may be located at a different memory location in each process. I'm not aware of any other containers that simultaneously work lock-free and in shared memory.
There's a couple "gotacha's" with it though.

The recommended way it to stick with regular synchronisation if you need shared memory.

Re: Questions about a sample code on shared memory.

@OReubens - so I have a producer and a consumer, the point is to send the data as fast as possible. Since synchronization takes time, a person who developed the code that I posted, used polling instead of semaphores.

Thx about linked list, is it possible to have a lock-free linked list in a memory of a process instead of a shared memory? Can I use it to exchange data between threads in a lock-free manner?