My name is Bartosz Adamczewski and this blog is all about programming on the NET platform and beyond. I'm focusing on two languages C# and F#, but also I'm planning to throw some SQL to the mix, and maybe Java + Android platform, and C++ as cherry on top.

Pages

Tuesday, June 19, 2012

Fibers

Note: Due to enormous length of this article I decided to split it into parts.

When working with applications that utilize lots of threads and those threads arent particularry long running we can stumble upon a problem that we may lose most of the processing time for thread context switches, so it would be desirable to do all of the processing on a limited number of threads, usualy this can be done using a queue of delegates that each thread will process, but sometimes we cannot have that and still we would want to spawn units of work that behave like threads, enter Fibers.

A fiber logically is a unit of execution that behaves exacly like a thread but it's executing in the thread itself so one thread can "simulate" the execution of multiple threads, in most circumstancess fibers are not very beneficial as the cost of running then can be the same as threads and sometimes even higher, but this is not always true.

Fibers in actuality are different then threads as by design the user controls where and when to switch the context, and this must be done explicitly otherwise the context will never switch and other fibers will be stuck, manual switiching has some advantages thouh as the user can have a grater knowlede how to switch more effectivly.

Unmanaged Fibers

.NET hasn't got fiber functionality, but Windows so we can eaisy do a wrapper class arround the unmanaged fiber api like so:

This solution is very problematic as it requires from the user a code that keeps track of the Fiber Id's as well as Switching and destroying them by explicityly passing the main fiber Id. This is not very desirable except sittuations where we want a very precise control over fibers but in such cases we should stick to unamanaged code all the way. To fix the sittuation a much better way is to create a fiber scheduler that automatically manages context switches, proper disposal and serves as the main point of dispatching communication, this can be very simple round robin queue or a more complex prority based scheduler, but for the sake of this example we will implement the simplest scheduler.

In actuality the Scheduler class is doing much more than it supposed to as this class spawns the main fiber on a separate thread and manages the deletes of child fibers, but for the sake of this example all of those things are contained under a single class. This implementation makes a big assumption that all fibers will switch to the primary fiber and that all fibers will be serviced by one thread only, which is not the best in therms of raw performance as switching to other fibers would be much faster, and a separate thread would be spawned if the number of fibers on a thread would exceed a certain threshold, still performance wise this solution will do just fine in most cases as it will be faster then standard thread spawning to solve a concurrent problem (more complex and better fiber scheduler could be posted in another article though).

To prove that it actually works let's set up some sample code together:

privateconstint numThreads = 5;

publicstaticvoid F()

{

string s = string.Empty;

for (int i = 0; i < 100; i++)

{

s = i.ToString();

Console.WriteLine("F: " + s);

FiberScheduler.Yield();

}

}

publicstaticvoid F2()

{

string s = string.Empty;

for ( int i = 0; i < 100; i++ )

{

s = i.ToString( );

Console.WriteLine("F2:" + s);

FiberScheduler.Yield();

}

}

publicstaticvoid Main( string[] args )

{

Fiber[] fibers = newFiber[numThreads];

for ( int i = 0; i < numThreads; i++ )

{

if (i % 2 == 0)

fibers[i] = newFiber(F);

else

fibers[i] = newFiber(F2);

}

for ( int i = 0; i < numThreads; i++ )

{

fibers [i].Run( );

}

}

The downside of creating unamanaged fiber api over .NET code is non trival error handling. Ask yourself this, what will happen when the code will throw an unmanaged exception in some point? It can bring the entire process down. The next potential problem is that freeing resources in Fibers needs to be performed in the main fiber thread, thus the code for fiber deletion sits in the scheduler, attempt's to do it from another thread results in MemoryAccessViolation (at least that was my experience and that's what MSDN sorta states). The final problem with such wrapper on fiber API is that a fiber must run in a context of an unmanaged thread and .NET threads aren't simple unamanged wrappers as one managed thread can run on a single thread or many, moreover the GC collection thread can interrupt other threads, this may lead to very nasty or unexpected behaviors so while the solution works and it's usable it can also lead to some unexpected behaviors ( I have yet to confirm that ).

Managed Fibers

Instead we can create our own Fiber system using .NET, the main advantage is that the code is managed therefore we have grater control over the flow and error handling. To be able to create such system in a managed language without any low level api we eiter need to be able to delegate a switch upon some code execution in a method that will be executed in the context of a fiber, or we can use the yeild generators which actually are very simillar to fibers in a sence that they do most of the work like switching and keeping track of the execution context.

We will create the fiber model based arround yeild generators as it's easier to manage as using some context switch delegates to wrapp arrount existing code in a method can obscure the code quite a bit instead it's more desirable to slightly change the methods to be fiber friendly ( although the first option can also be exposed with yeilds to get best of both worlds ).

/// Sets the amount of fibers that can run on a single scheduling thread (5 by default).

///</summary>

///<param name="number">number of fibers.</param>

publicstaticvoid SetFibersPerThreadThreshold(int value)

{

lock (locker)

{

fiberThreadCnt = value;

}

}

///<summary>

/// Gets the amount of fibers that can run on a single scheduling thread (5 by default).

///</summary>

///<returns></returns>

publicstaticint GetFibersPerThreadThreshold()

{

return fiberThreadCnt;

}

}

///<summary>

/// Represents a context switch status of a fiber.

///</summary>

publicenumContextStatus

{

Switch,

Wait

}

///<summary>

/// Represents a Fiber Status upon making a context switch.

///</summary>

publicclassFiberStatus

{

///<summary>

/// Gets the fiber context switch status.

///</summary>

publicContextStatus Status { get; internalset; }

private FiberStatus(ContextStatus status)

{

this.Status = status;

}

///<summary>

/// Invokes a context switch to another fiber.

///</summary>

///<returns>Fiber Status.</returns>

publicstaticFiberStatus Yield()

{

returnnewFiberStatus(ContextStatus.Switch);

}

///<summary>

/// Invokes a context switch with a full cycle wait.

///</summary>

///<remarks>

/// This is usefull for controling the priority of certain

/// fiber tasks.

///</remarks>

///<returns></returns>

publicstaticFiberStatus Wait()

{

returnnewFiberStatus(ContextStatus.Wait);

}

}

///<summary>

/// Represents a Fiber that's a lightweigh isolated unit of concurent

/// execution.

///</summary>

publicclassFiber

{

internalIEnumerator<FiberStatus> FiberContext;

privatestaticint internalId = 0;

///<summary>

/// Gets the unique fiber Id.

///</summary>

publicint Id { get; privateset; }

///<summary>

/// Initializes a new instance of a Fiber.

///</summary>

///<param name="fiber">fiber funcion.</param>

public Fiber(Func<IEnumerable<FiberStatus>> fiber)

{

Id = ++internalId;

this.FiberContext = fiber().GetEnumerator();

}

///<summary>

/// Schedules fiber for execution.

///</summary>

publicvoid Run()

{

FiberScheduler.Schedule(this);

}

}

The managed implementation does not make use of multiple threads so it's using a mix of fiber code and thread code. When a single thread that is a primary fiber reaches a certain threshold of fibers the scheduler creates a new thread and starts to assign new fibers there, this is useful as a single thread running huge number of fibers will get slow, thus it is advised to run only five to ten fibers on a single thread (this depends largely on the case that we are in). The managed implementation introduces a second fiber state as opposed to simply Yield which is a context switch, now fibers can also wait, this may come in handy in situations where some fiber can be delayed a little in order to speed some other critical fiber tasks that are running.

Performance

In order to prove that fibers are a handy thing that has potential benefits we need to do some simple performance tests and compare them to running the same code on threads. We will not be testing it against a .NET thread pool as it uses some advanced techniques that this solution isn't so the performance will be the same or a bit slower (not much however), in part 2 of this article FiberPool will be made (not confirmed yet) and then those two pools will be compared. I will not compare the performance of the umanaged solution just yet their correctness in therms of CLR needs to be concluded first.

In order to compare finished subroutines I decided to implement a simple class called countdown, now before you will say that identical solution is present in .NET 4.0 let me stop you there, the code is written in 3.5 so custom implementation was necessary.

The test code assumes that some tasks will execute longer then others, as you can see we are testing it on a fairly large number of concurrent tasks as for low concurrency systems you can very easily get away with threads (there is little sense to use fibers if the maximum concurrent load is below NumOfCores * (2 | 4), I do speak from my personal experience though). Please do keep in mind that fibers actually can and will be slower then threads in certain class of problems like very few long lived tasks, or huge number of very short tasks ( yes they will probably be slower ), so this test does not conclude that fibers are best in every given scenario rather then that it concludes that in normal concurrency circumstances (mid lived tasks) they will be faster.

The speeds are as follows:

Summary

Fibers can be a very handy tool for solving concurrency performance issues that are based around expensive thread context switches. This article is the first part of the whole series on fibers as there's much more to topics to be touched like work stealing fibers, and fiber resource locking.

Full source code will be available soon, (i will update this article and put a link here).

4 comments:

A threadpool is not made to decrease the number or cost of context switches between threads, it's to decrease the cost of creating and starting new threads, as well as distributing work (4.0 work stealing) so this is a totally different concept.

Fibers can be considered lightweight threads, they are light in a sense that they actually do in code context switches so they don't have to go to kernel scheduler, which is a lot less expensive, their start-up time is also faster. So Fibers cannot be compared to a thread pool, a fiber can only be compared to a thread, you would need to create a FiberPool and that you could compare to a threadpool (which will be implemented in another part about fibers).

About Me

I'm a software consultant working day to day on financial systems. I specialize in high frequency concurrent systems that incorporate fragmentation free analysis and software design.
For speaking engagements or other opportunities contact me at: bartoszadamczewski(AT)gmail.com
http://about.me/badamczewski