Synchronous concurrency

Synchronous concurrency
You are encouraged to solve this task according to the task description, using any language you may know.

The goal of this task is to create two concurrent activities ("Threads" or "Tasks", not processes.) that share data synchronously. Your language may provide syntax or libraries to perform concurrency. Different languages provide different implementations of concurrency, often with different names. Some languages use the term threads, others use the term tasks, while others use co-processes. This task should not be implemented using fork, spawn, or the Linux/UNIX/Win32 pipe command, as communication should be between threads, not processes.

One of the concurrent units will read from a file named "input.txt" and send the contents of that file, one line at a time, to the other concurrent unit, which will print the line it receives to standard output. The printing unit must count the number of lines it prints. After the concurrent unit reading the file sends its last line to the printing unit, the reading unit will request the number of lines printed by the printing unit. The reading unit will then print the number of lines printed by the printing unit.

This task requires two-way communication between the concurrent units. All concurrent units must cleanly terminate at the end of the program.

This Ada example starts by creating a package defining a single instance of a printing task. Ada requires packages to be separated into two parts. The package specification defines the interface to all public members of the package.

Note that the task body contains an accept block for each entry defined in the task specification. When some other task calls an entry in the Printer task the communication between the tasks is synchronized.

This example uses an infinite loop in the printer task. There is no way to know ahead of time how many lines the printer task will need to print. Each iteration through the loop causes the task to execute a selective accept. That means that it can either accept a call on the Put entry, or it can accept a call on the Get_Count entry. The terminate option is execute only when the program contains no more tasks that can call the entries in the Printer task. If no task has called either entry the Printer task will wait for a task to call one of the entries, or for the terminate option to apply.

The next file contains the main procedure for this program. The main or entry-point procedure for a program always runs in the environment task. For this program, the environment task is takes on the role of the file reading concurrent unit while the Printer task takes on the role of the printing concurrent unit.

with Synchronous_Concurrent; use Synchronous_Concurrent;with Ada.Text_Io; use Ada.Text_Io;

In this example only the environment task can call the entries on the Printer task. When the environment task completes the terminate option of the Printer task applies, terminating the Printer task. The environment task completes right after printing the number of lines sent through the Printer task. Because of the terminate option, the Printer task terminates just after the environment task prints the count.

// This is a BCPL implementation of the Rosettacode synchronous// concurrency test using BCPL coroutines and a coroutine implementation// of a Occum-style channels.// BCPL is freely available from www.cl.cam.ac.uk/users/mr10

IF tracing DO writef("outfn: Received zero, so sent count=%n back to inco*n", linecount)

cowait(linecount)}

// The following functions are a implementation of Occum-style channels// using coroutines.

// The first coroutine to request a transfer through a channel becomes// suspended and the second causes the data to be transfers and then allows// both coroutines to resume (in some order). The channel word is either// zero or points to a suspended (read or write) cocoutine.

/* * These are global variables of this process. All cothreads of this * process will share these global variables. */cothread_t reader;cothread_t printer;struct{char*buf;/* Not a C string. No terminating '\0'. */size_t len;/* Length of line in buffer. */size_t cap;/* Maximum capacity of buffer. */} line;size_t count;/* Number of lines printed. */

/* * The reader cothread reads every line of an input file, passes each * line to the printer cothread, and reports the number of lines. */voidreader_entry(void){ FILE *input;size_t newcap;int c, eof, eol;char*newbuf;

/* * The printer cothread starts the reader cothread, prints every line * line from the reader cothread, and counts the number of lines. */intmain(){ reader = co_create(4096, reader_entry); printer = co_active(); count =0;

The writer executes as an agent on a thread from a thread pool. The state of the agent is the count of written lines,
initialized to 0. The reader will send the writer calls to the write-line function, which the agent will execute asynchronously.
The state argument is the agent's state at the start of the call, and the last expression becomes the agent's new state.

(use '[clojure.java.io :as io])

(def writer (agent 0))

(defn write-line [state line](println line)(inc state))

The reader executes on the main thread. It passes each line to the writer -- the send call returns immediately, waits until the writer has finished writing all the lines, gets the line count & prints it, and terminates the writer.

(defun message (recipient name&rest message)(with-lock-held ((lock-of recipient));; it would have been better to implement tail-consing or a LIFO(setf(mailbox-of recipient)(nconc(mailbox-of recipient)(list(list* name message))))(condition-notify (condition-of recipient))) message)

(defun writer (stream reader);; that would work better with ITERATE(loop with line-count =0do(receive-message((|here's a line for you| line)(write-line line stream)(incf line-count))(|looks like i've got no more lines|(return))(|how many lines?|(message reader '|line count| line-count)))))

This task uses two processes, reader/writer, synchronized by a semaphore S and its queue of messages. The reader sends write or reader-count-please messages. The writer responds with ack or count messages.

This code will read lines from the file on one thread, and print them to the console on one or more other
threads from the ThreadPool, using a MailboxProcessor for lock-free communication between threads and tracking the line count without the use of mutable state.

The following Haskell code uses simple MVars for thread communication. While the GHC libraries for concurrency give quite a wide design space for thread communication, I felt that the following was fairly reasonable.

For those who are unaware of MVars, they are essentially mutable cells which may be empty or hold a single value, and which have the following important properties:

takeMVar will get the contents of an MVar when it is full, emptying it.

takeMVar will block if the MVar is empty, until it has been filled by another thread.

putMVar will fill an empty MVar with a given value.

putMVar will block until the MVar is empty if it is full.

So MVars are essentially bounded channels which hold a maximum of one element at a time.

The code below defines various signals in terms of takeMVar and putMVar and then passes those to the parts of the code which should be permitted to use them. Note that this way, it is impossible for the reader process to take the current line, for example.

The reader simply reads the file lazily, applying putLine to each of the lines in turn, which blocks until the writer has taken the line. It then signals that it is finished with putEOF, and then takes the count and prints it.

// No need to start a third thread for the reader, just use this threadBufferedReader br =newBufferedReader(newFileReader("input.txt"));String line;while((line = br.readLine())!=null) queue.put(line); br.close(); queue.put(EOF); writerThread.join();// AtomicLong is not needed here due to memory barrier created by thread join, but still need a mutable long since lineCount must be final to access it from an anonymous classSystem.out.println("Line count: "+ lineCount.get());return;}}

The following example can be found in the Logtalk distribution and is used here with permission. The "input.txt" file contains the terms "a(0)..a(9)", one per line. Works when using SWI-Prolog, XSB, or YAP as the backend compiler.

The reader is a simple loop. It starts by opening a file, then reads lines from that file until there is nothing left to read. After each line, it sends Some v on channel lines_dest, where v is the contents of the line. Once there are no lines anymore,
exception End_of_file is raised and we send None on that same channel. After that, it's just the matter of waiting for one message on count_source, closing the file and printing the result:

The printer is also a simple loop. It keeps receiving messages on lines_source. If a message has structure Some v, then v is a line, print it and increment the counter. Otherwise, the message has structure None, which means that we're done, just send the number of lines on count_dest:

Oforth implements concurrency with tasks and communication between tasks using channels. Here, we create 2 channels (one for print, one to retrieve the number of lines).

When the file has been read and sent to channel chPrint, this channel is closed. All tasks waiting for an object in this channel are released and receive null. Then printing task returns number of printed lines into chCount channel for the main task.

-- add an item to the work queue. This is a-- guarded method, which means this is a synchronized access::method addItem guardedexposequeue actionPendingusearg item-- add the item to the queuequeue~queue(item)-- indicate there's something new. This is a condition variable-- that any will wake up any thread that's waiting on access. They'll-- be able to get access once we exit actionPending = .true

-- another method for coordinating access with the other thread. This indicates-- it is time to shut down::method stop guardedexpose actionPending stopped-- indicate this has been stopped and also flip the condition variable to-- wake up any waiters stopped = .true actionPending = .true

-- read the next item off of the queue. .nil indicates we've reached-- the last item on the queue. This is also a guarded method, but we'll use-- the GUARD ON instruction to wait for work if the queue is currently empty::method nextItemexposequeue stopped actionPending-- we might need to loop a little to get an itemdoforever-- if there's something on the queue, pull the front item and returnif\queue~isEmpty thenreturnqueue~pull-- if the other thread says it is done sending is stuff, time to shut downif stopped thenreturn.nil-- nothing on the queue, not stopped yet, so release the guard and wait until-- there's something pending to work on.guardonwhen actionPendingend

-- one half of the synchronization effort. This will read lines and-- add them to the work queue. The thread will terminate once we hit end-of-file::class filereader::method init-- accept a generic stream...the data source need not be a fileuseargstream, queue

-- the other end of this. This class will read lines from a work queue-- and write it to a stream::class filewriter::method init-- accept a generic stream...the data source need not be a fileuseargstream, queue

Concurrent units are established by use of the feed operator, which works much like an in-process object pipe. Since the final feed goes to a variable declaration that belongs to the outer thread, it serves as a backchannel from the printer thread. In this case the outer thread signals the desire for a line count by terminating the pipe to the printing thread.
(Note: soon these will be implemented with real threads in Perl 6, but this is currently emulated with coroutine semantics, aka lazy lists.)

procedure countlines()object line linecount = 0 while 1 do enter_cs(qlock) if length(queue)=0 then leave_cs(qlock)-- sleep(0.1) else line = queue[1] queue = queue[2..$] leave_cs(qlock) if atom(line) then exit end if-- ?line linecount += 1 end if end while exit_thread(0)end procedure

The following two tasks communicate via UDP, so in fact they don't need to run
within the same process and not even the same machine. "input.txt" would rather
be a device (like a named pipe or socket) than a plain file.

The above program has no concurrency, because the printer is a block { |line| print line; count += 1 } that can print only one line. (The reader calls block more than one time.) After the refactoring, the printer will print all lines, and the program will jump between the reader and the printer. For the refactoring, Fibers might be best, Continuations work if our Ruby is too old for Fibers, and Threads are another alternative.

Fibers. Ruby 1.9 gives Fiber to us. Fibers provide concurrency. Each fiber has its own stack (to hold nested function calls). A pair of fibers has a boss-vassal relationship: the boss fiber calls vassal.resume to jump to the vassal, and the vassal fiber calls Fiber.yield to jump to the boss.

We create a fiber as the reader, and we use the current fiber as the printer.

Our reader is a generator, because reader.resume generates a line, and Fiber.yield passes the generated value.

This generator allows Fiber.yield inside nested function calls (because a fiber has its own stack). We nested three calls: we have Fiber.yield inside our line block, inside IO.foreach, inside our fiber block.

Because the reader and printer are in the same process, they can share the count variable. An alternate way is to use reader.resume(count) to pass the count, so Fiber.yield would return the count.

If IO.foreach raises an IO error, then the reader dies, and reader.resume raises the same error in the printer. This is what we want. If we run this program with no input.txt file, then we see the error.

Continuations. Ruby 1.8 gives Continuation to us. (Ruby 1.9 still gives Continuation if we require 'continuation' from the standard library.) The trick is that you can continue a function call after you leave it. Continuations can provide concurrency. The problem is that continuations make spaghetti code with very confusing control flow.

The above program uses continuations almost like fibers. The reader continues the printer, and the printer continues the reader.

The first call to reader[c] uses Proc#[] to start the reader; but later calls to reader[c] use Continuation#[] to continue the reader.

The reader and printer share the same stack. The control flow when IO.foreach raises an IO error is very strange. The reader dies, and the original call to Proc#[] raises the same error in the printer.

Threads. Both Ruby 1.8 and Ruby 1.9 give Thread to us. Threads provide preemptive concurrency. The scheduler preempts threads and switches between threads, seemingly at random. Threads seem worse than continuations, because threads have unpredictable control flow, but we can use a Queue to restore some order. We use Thread with Queue.

We create a thread as the reader, and use the current thread as the writer.

If a thread tries to pop an empty queue, then the thread waits until some other thread queues something.

The queue of lines can become long; the worst case allows the reader to read the entire file before the printer pops the first line! If you wanted to prevent a long queue, a SizedQueue.new(5) would hold only 5 elements.

If IO.reader raises an IO error, then the reader dies. The writer would deadlock on an empty queue after the reader dies. To prevent this deadlock, the reader ensures to queue a final nil before it dies. The writer uses this nil to break its loop and call reader.join. If the reader dies with an IO error, then reader.join raises the same error.

Using delimited-continuation-based obtain and yield-from to simulate co-routines, wrapped in some OOP. A thread base class is derived into consumer and producer, both of which provide run methods. The consumer has a counter also, and producer holds a reference to a consumer.

When the objects are instantiated, their co-routines auto-start, thanks to the :postinit hook.

To get things going, we resume the producer via pro.(resume), because we started that in a suspended state. This is actually not necessary; if we remove the suspended t from the new expression which instantiates the producer, we can remove this line. However, this means that the body of the let doesn't receive control. The producer finishes producing and then the pro variable is bound, and the final (put-line ...) expression evaluates. Starting the producer suspended lets us insert some logic prior to dispatching the producer. We implicitly start the consumer, though, because it immediately suspends to wait for an item, saving its context in a continuation and relinquishing control.

This task has been flagged for clarification. Code on this page in its current state may be flagged incorrect once this task has been clarified. See this page's Talk page for discussion.

the main process is the one started by cat file.
The subshell created by >(...) is the secondary process
the main pipeline waits for the secondary process to finish,
collects and prints the count. This falls a-foul of the requirement
that fork should not be used as every thing in pipes is done using
forks.