I'm currently looking in to using Mono to support user scripting of a
virtual world.

The hope is that Mono's JIT compilation will provide a large
performance boost over the current interpreted system and that,
ultimately, it will be possible to use any language that compiles to
the CLR to script the world.

So, I'm looking to develop a compiler which compiles our current
simple, proceedural scripting language to CIL. I have lex and yacc
files for the language and it's very simple, but any pointers to
resources for writing compilers for the CLR would be appreciated.

The major concern I have at the moment is ensuring that the system
supports the required level of concurrency. Currently the interpreter
runs each script for a few milliseconds each frame, checking the
elapsed time after each bytecode and switching to a new script once a
quanta has elapsed. Even this simple approach allows 1000s of scripts
to run concurrently.

The question is, how to approach concurrency when the scripts aren't
interpreted, but JIT compiled? Using 1 thread per script is out of the
question, and even using the .NET thread pool will only allow as many
scripts to run concurrently as there are threads in the pool.

My current thinking is to inject code in to the CIL output which
implements simple, cooperative concurrency in a similar way to the C#
yield statement. Each function maintains its local state in an object
rather than on the stack and can return an "incomplete" value to the
caller, signalling that it should be called again at some time in the
future to continue processing where it left off. Effectively the
system turns any function that may take considerable time in to
semi-coroutine. Does this sound like a good plan? How would I go about
implementing it using a compiler which targets the CLR?