I'm a C# developer having worked with .Net since it was in beta. Before that I mainly worked in C and C++. I have been developing commercial software for more than 20 years. I also mess around with microprocessors, but that's just for fun. I live near Cambridge, England and work from home in my 'silicon shed'.

Blogroll

I've recently been messing about with Microsoft's Azure platform; and I already think it's brilliant. I didn't expect the support for Linux Virtual Machines to be quite so good. Literally with just a few clicks I created a Linux box on the cloud (I went for Ubuntu), I can't really see it getting much easier. I suppose I should have known when I recently saw that Microsoft's CEO Satya Nadella had presented this:

Anyway, the first thing that I did with Azure was test my Linux VM out by copying over my C webserver code and compiling it. Everything went perfectly. But I quickly realised that it would be useful to be able to leave my code running in another terminal session when I disconnected from the console. That's where I tried out GNU Screen which allowed me to start my program in a session and leave it running after I had disconnected. GNU Screen is a nice simple solution, which means that I can keep my code simple. I like this :-) And at the same time I noticed a couple of bugs in the webserver code which I had not spotted before, so I was able to fix them too.

It's great to think that I can write some C code on my MacBook; cross-compile and test it on a small router running OpenWrt, then move it to the Azure platform without any worries. This is one reason why I still like programming in C (and Linux).

Next I decided to copy my H2D2 server across to Azure (of course). Again, everything worked perfectly. So I'm currently testing H2D2 on the cloud, let's see how stable it is. I also took the opportunity to tidy up the html, to make it look more presentable as well:

Originally, I started with the most basic type of VM on offer, with a shared core. But then I did an upgrade to an 8-core machine, which gave a noticeable performance boost (as you'd expect). It was nice to see that Azure can upgrade your VM really easily, yet again it was all done with a couple of clicks (and a reboot of the VM). But now, to save CPU I've gone back to the shared core again which will be less of a drain on my account.

Since I wrote this blog post,
I have been looking for a better demo program to show what I can do with
my H2D2 programming language. I needed something more interesting than
"hello world" ... and which didn't involve drawing the Mandelbrot set.

I started trawling through the Rosetta Code
website for inspiration and came across this
sphere drawing program
in C. I guess it's a simple ray tracing algorithm really. It seemed like a
reasonable candidate.

So I set about porting it from C to H2D2, which was an interesting experience.
Interesting because H2D2 is not very forgiving when you make mistakes - since I have
not (yet) made the compiler error messages very user friendly. So it makes you reason
things out very carefully in your head, rather than rely on the compiler to spot
problems with your code.

I have managed to get it to work though, and the resulting H2D2 example program
shows a few of the language features (like a function call and arrays for example).
However, it also highlights a few things I should work on, like creating a
syntactic shortcut for assigning values to an array. Anyway, I can now create
output like this:

Although these programs are very trivial, it shows that I can now do H2D2 programming from my browser. It's the server part which compiles the source into bytecode and then executes the program in timeslices. So the actual code is running inside the server and we're just seeing the output in the browser (alongside the source code of course). Since I recorded this in a coffee shop before work this morning, the H2D2 server was running locally inside Xcode on my Mac, for convenience. But I would be able to do the same thing even if the server was running elsewhere, like on my TP-Link router.

Considering that the whole H2D2 language is written in C, I think that's pretty good going. It seems reasonably fast and I don't even think I have the compiler optimisations turned on. Obviously there is a little bit of JavaScript doing the HTTP posts between the browser and the server. But I really do need to write some better code examples, which should be things that don't involve the Mandelbrot set for a change...

UPDATE: one of the "better code examples" I mention above can now be seen here.

So recently, several of the different things I have been messing about with have come together. This was always in the back of my mind,
but I didn't really know how long it was going to take. Recently, I have been working on:

building my own webserver (dweb)

cross-compiling for Linux on a TP-Link router (with OpenWrt)

...and my own experimental programming language (H2D2)

Now I have bean able to bring all that together, what it means is that I can run
H2D2 programs on my little TP-Link router. At the moment, I'm doing it through
jQuery Ajax calls. So the programs are sent to the router with an HTTP POST from an ajax request, the router runs the program remotely and then returns the output back to the browser. Nice.

Here is a screenshot showing it all in action:

Woo-hoo... it works! Oh, and apologies for the partial ASCII Mandelbrot,
I really can’t help it.
The next experiment will probably be to do what H2D2 was actually designed for,
namely to run a program partially on the router and then finish off the same
program instance on some other platform. Then, I should be able to bounce a
single program between the router, a Raspberry Pi and a desktop, which will
be a neat trick.

I remember there was a time I was very happy because H2D2 was running my Mandelbrot test program in about 650 milliseconds (whereas the original proof of concept, DALIS, took nearly 12 seconds to do the same thing).

Since moving over to my new MacBook and compiling the H2D2 source using Xcode, just look at the result now:

Whoa! 72 milliseconds! Now that's impressive. This new MacBook is pretty fast, but I'm sure the Apple LLVM compiler is also playing its part in achieving that level of speed. I have done some optimisation work in H2D2 recently, but I don't think I can take the credit this time.

It really pays to do this type of 'system level' programming in C (plus I enjoy it of course). Right, I'm off to work on parsing string expressions now...

OK, first of all, I've bought a new laptop, and it's a Mac - a 13" MacBook Pro to be precise, the one with the Intel Core i7 processor, 8Gb of RAM and 750Gb hard disk. After having great success with running things as VMs at home, I've decided to do the same on my laptop. But the old Toshiba T130 was just not powerful enough to do that (it seemed like quite a powerful laptop when I bought it, but that was over 4 years ago I think). Besides, the Toshiba was feeling quite slow anyway.

But I came to realise that if I was going to run all my stuff as a VM, then I didn't need Windows as the host OS. And getting a new shiny Apple piece of kit seemed like a pretty good choice. I've virtualised the old (Windows) laptop and now it's running happily as a VM in VirtualBox on my new MacBook. Pretty seamless really.

But, I then went and downloaded Xcode (Apple's IDE) and quickly realised that it will happily support plain old C programs. So I copied the H2D2 codebase from Pelles C on Windows into Xcode on the Mac. It compiled and ran first time - even I was surprised. But that is what is supposed to happen when you write portable C code - you can run it natively on just about any platform.

So, I'm going to carry on developing H2D2 in Xcode for a while and see how that goes. My initial impression is good, I'm still learning the keyboard shortcuts … but then I'm still getting used to the new Mac keyboard as well. I was surprised to realise that there is no hash (#) key, at least not on the UK Mac keyboard. Maybe twitter isn't as popular amongst Apple users as I had thought. Or maybe that's just because I think of Steven Fry when I think of a typical Mac user :-)

Anyway, I'm still finding my way round the IDE at the moment. It's probably too soon to draw any final conclusions. Maybe I should buy a book on Xcode...

So progress on my programming language, still codenamed H2D2 continues on
background threads :-) I've not given up, but I haven't made many posts
on the blog about it recently. It was probably getting boring anyway,
and recent work has been tidying things up and writing unit tests - which
does not make for interesting reading.

One thing that I wanted to try, was to compress the H2D2 bytecode so that
it doesn't take as much time to serialise a running program between machines.
I believe that Java JAR files are zipped, so I thought that I'd do something
similar with H2D2. But I also wanted something that I could write myself
that didn't take an age. Since I'm doing this for fun, it's nice to
start with a blank page and go from there. So anyway, my web wanderings
brought me to here.
That page talks about Run Length Encoding - something that I implemented
once myself back in the 1980s (or maybe the early 90s) - not that I have the
source code anymore. But I read about the RLE PackBits algorithm and thought
it might be a good way to go. Not a massive processing overhead and
reasonably simple to implement. I chose not to look at the example souce
code provided ... and just to write it from scratch myself. It sounded
like fun.

It took me about an hour whilst drinking my morning coffee to write
the compression code, and an hour or so in the evening to decompress the
data back to the original values (I also wrote some unit tests to check
for any obvious problems I could see). It seems to work OK: when I
compress H2D2 bytecode I'm getting about a 30% reduction. It's not as
good as zip compression obviously, but it will do for now.

I'm sure my code won't win any prizes for optimisation or style, but
at least it works. I guess it might also come in handy if you were trying
to squash some data on a microprocessor without needing a massive
library or lots of processing power. So here is an example C program:

Obviously this compression will only shrink files which have
runs of the same repeated byte value in them. Otherwise it may
increase the file size. You can try it from the command line and
pass /c as the first argument to compress a file and then use /u
to decompress the file back. The next two arguments need to be
the names of the input and output files respectively. This example
doesn't have any error trapping, so if the file fails to open, or
something like that it will go bang. But that would not be hard to add.

Of course, in my H2D2 code, I'm using the same algorithm, but
I've adapted the code to work on blocks of memory rather than on
files. But in H2D2 I have made a VFILE struct to work alongside the C
FILE pointers and I've added some functions that work the same as the C
IO functions but taking my VFILE pointer instead. This means that I can
easily swap code to operate either on disk or in memory...

I said before that I might try and do some unit testing in C, rather than use the features in Visual Studio.
It would mean that for H2D2 I can do everything inside Pelles C and won't need to have two IDEs running.
But I've never done any unit testing in C before (at least if I have it was so long ago I've forgotten).
So I needed to go off and find some means of doing that.

So I found the libtap library which is a
unit testing framework for C programs. Well, it is something that implements the Test Anything Protocol
which you can read about here. I'm no
expert on the Test Anything Protocol, this is the first time I've used it, but for my purposes it looks
like it will do fine. As long as I can run code and check for expected results and have the Passes and Failures
shown in some sort of output then I'll be happy... But for the moment
it looks ideal. I've had to make some minor modifications to get it working in my project
(in Pelles C for Windows) but it was not very much effort really. I've just made my own version of
tap.c and included it in my project. I had to include an implementation of vasprintf() which I
found
here
rather than rolling my own, but that seems to work quite well for me.

I found libtap after searching for C unit testing on
stack overflow
in case you're wondering how I found it in the first place.

What I've ended up with is a #define which tells the compiler which version of
main() to include. One version is my usual code and the other runs my test suite. I haven't
began porting my actual H2D2 unit tests back over from C# yet, but I do have a trivial example running just to
check that I can run some unit tests. So this this is all I've got so far:

I've also set it up so that the content of tap.c is not included when I compile normally, so this means
that my code isn't bloated with the test framework when I'm not using it.

I've resisted the temptation to mess around with libtap too much, I've only made enough changes to
get it working in my project. Thanks need to go to Nik Clayton, who wrote the library and to whoever
River Tarnell-7 is for posting his vasprintf() implementation. Thanks! In case it
helps any other people using Pelles C who would like to add units tests to their project, here are the files:

I'll probably try this out on the Raspberry Pi soon, because I think it may be useful to have
a lightweight unit testing framework for GCC on the Raspberry Pi too. I'll post an update if anything comes of
that idea.

Since my H2D2 project has quite a lot of code in it now, I've been using .Net interop as a quick way to write some unit tests for the H2D2 programming language.

I have written a C# unit test project in Visual Studio which tests much of the existing H2D2 language syntax. This means that I can add new features and quickly check that I've not broken any of the existing code. So I can very easily make sure that programs which used to run continue to work as expected.

Recently, I've been making H2D2 support function calls, and so it's been handy to write tests for various different types of function call (passing parameters, reading return values, allowing functions that don't return a value, etc). The unit tests have made it much easier to ensure that everything works after I've made changes.

Since I've written more unit tests in C# than anything else, I thought it would be the quickest way to get my tests up and running. And because I had already compiled H2D2 into a windows dll (the unmanaged kind), it is not very hard to run H2D2 programs from .Net - even easier because I had already written some basic wrapper code for all the interop stuff. It's not much harder than putting some H2D2 source code into a string and calling a Run() method. Obviously all this has been done 'just for fun', since it's only one of my hobbyist projects anyway.

But when I mentioned to one of my colleagues that I had spent some of my free time writing unit tests for fun he said that I was clearly stark raving mad. There's a possibility he's correct.

However, when I think of how I'd feel if I broke some of my existing H2D2 code and had to go back and unpick where I went wrong, I reckon that these unit tests are really less about having fun and more about preventing a severe loss of fun if I ended up scratching my head and trying to figure out what I'd done to make something break.

So I hope that's a suitable explanation for why I've been writing unit tests in my free time. It's the best excuse I've got.

Although ultimately, I may end up looking for a unit testing framework for C so that I only need one IDE loaded to run my tests... The .Net interop solution may just be a stepping stone.

I've been busy lately adding the concept of function calls to my H2D2 programming language. It hasn't
been as hard as I thought, but there has been a reasonable amount of code to write. I have added the notion of
scope as well, so that when a function returns, any local variables go out of scope and the H2D2
virtual machine can free the memory allocated to them. Currently, functions can return a value
(although it's possible in H2D2 to return nothing from a function too) ... but I can't yet pass parameters
to functions, that's on my TODO list, I'll work on that next.

So here is the current test program that I'm using to see if it all works:

Simple stuff, but it demonstrates calling the "test()" function like a statement (ignoring the return value)
and also assigns its return value to a variable too. I have also done some work allowing this type of code to be
written out as labelled bracket notation. Essentially I'm treating each function like a seperate program, starting
with its own root node, and then each call to that function simply points to the correct root node. This is how the above program looks if I render the labelled bracket notation generated from the bytecode:

Currently, when I parse the source code, I simply scan for function definitions and emit the code for them
and then go back to a 'bootstrap' node which is the programs entry point. The starting point of the source code is
pointed to from the 'bootstrap' node. It seems to work fine, and it means that programs always execute from node zero (which I'm making sure is always the bootstrap node).

In other news, I compiled H2D2 as a Windows dll, and used Interoperability to call it from C#, this worked a treat and was extremely fast. With the dll version, any output that the H2D2 program creates is returned as a pointer to a string,
so after the program has been run the C# code needs to make an additional call the dll and tell it to free the string containing the output. But, my mandelbrot test program runs in about 200ms this way, since it doesn't have to muck about with writing to the screen. Once the C# program has all the output it can write it in one go rather than a character at
a time, which is much quicker.

Now that I have installed Visual Studio 2012 I might be able to hide H2D2 behind a websocket interface. This should allow for a 'chatty' interface to H2D2 where you can ask it to run programs and get the results returned. This is also on my TODO list :-)