OR 1=1

To demonstrate the different async options in .NET, I wrote the following samples that call the async read methods of the .NET FileStream. The following APM, TPL, and Rx code samples complete in roughly the same amount of time. I am biased to Rx, so, it’s the most elegent code sample of them all.

A counter, buffer, and waithandle are created. An action delegate calls the beginRead delegate, and registers an anonymous delegate as the callback. In this callback, the EndRead method is called which returns the bytesRead. If it’s zero, the waithandle is signaled and the method exits, otherwise, a count is increased (and checked against a constant to write progress), and calls itself to loop.

An action delegate invokes the task and hooks in a continuation delegate to handle the completion. In the continuation if the result is 0 the waithandle is set and the method exists. Otherwise the count in increased (and checked against a constant to write progress), and calls itself to loop.

A infinite sequence of 0 is created, the value 0 is ignored by the SelectMany operator by invoking the observable factory function, constrain the sequence to only observe when bytesRead > 0, buffer the results into a sequence of IList<int> and we block with the ForEach operator to display the progress to the console.

Thanks to David Fowler for recomending the idea to compare the differences between the different aync options in .NET!

Another IObservable<> extension method I needed was Split – just like string.Split (), but generically for any array and an IObservable<T[]>. I started first with a generic IEnumberable<T> Split<T> ():

Now that I can split an array, I need to an IObservable<T[]> Split() as well:

///<summary>/// Splits the arrays from the source observable by the separator///</summary>///<typeparam name="TSource">type of the source array</typeparam>///<param name="source">source array</param>///<param name="separator">the separator(s)</param>///<returns>An observable containing the same number of items (if the /// separator(s) are not found) or additional items by spliting the incoming/// arrays</returns>publicstaticIObservable<TSource[]> Split<TSource> (thisIObservable<TSource[]> source, params TSource[][] separator) {

///<summary>/// Splits the arrays from the source observable by the delimiter///</summary>///<typeparam name="TSource">type of the source array</typeparam>///<param name="source">source array</param>///<param name="separator">the delimiter</param>///<returns>An observable containing the same number of items (if the /// delimiter is not found) or additional items by spliting the incoming/// arrays</returns>publicstaticIObservable<TSource[]> Split<TSource> (thisIObservable<TSource[]> source, params TSource[] separator) {

Rx includes the AsyncRead () method that reads a stream from the beginning returning an IObservable<byte[]>. This method does the same, but in reverse. The public method overloads accept a stream and optionally a bufferSize and startingPosition.

Using IEnumerable<IObservable<object>>, yield, Observable.FromAsyncPattern (), and Observable.Iterate (), we’re able to read a stream in blocks (of bufferSize) from the end to the front. The public method calls Iterate () which passes an observer instance to the following private method:

Chris was concerned that the initial Rx implementation was inefficiently reading an entire 5 GB text file throwing everything away except for the last 10 lines. To optimize I/O, I’ve modified the code to read the file in reverse. Here’s a new version that tails a 5 GB text file in less than 150 ms:

Starting from the end of the file, reading chunks backwards, reversing the byte[] chunks, delimiting by \r and \n (creating smaller byte[] results), take the first 10, reversing them back, converting to string (via ASCII encoding), and then sending to standard output. In this version we are reading from disk only the information required to tail the file – no unnecessary disk IOs. Unfortunately some of these extension methods didn’t exist, so I had to create three to build this: