Archive

This post is a look into a new library that I’m writing that’s intended to make life easier for WPF developers working with Intel RealSense devices. As many of you may know, I’ve been involved with the RealSense platform for a couple of years now (back from when it was called the Perceptual Computing SDK). When I develop samples with it, I tend to use WPF as my default development experience, and the idea of hooking up the Natural User Interface capabilities of RealSense devices with the NUI power of WPF in an easy to use package is just too good to resist. On top of this, I still strongly believe in WPF and it will take a lot to remove me from developing desktop applications with it because it is just so powerful.

To this end, I have started developing a library called RealSenseLight that will enable WPF developers to easily leverage the power of RealSense without having to worry about the implementation details. While it”s primarily aimed at WPF developers, the functionality available will be usable from other C# (Windows Desktop) applications, so hooking into Console applications will certainly be possible.

One of the many decisions I’ve taken is to allow configuration of features to be set via Fluent interface, so it’s possible to do things like this:

ViewModels will be able to hook into RealSense using convenient interfaces that abstract the underlying implementations. There’s no need to call Enable… to enable a RealSense capability. The simple fact of integrating a concrete implementation means that the feature is automatically available. The following example demonstrates what an IoC resolved implementation looks like:

The library will provide the ability to do things such as pause/resume individual RealSense capabilities, identify and choose from the relevant RealSense compatible devices, although this does require identifying up front, what the different aspects are you’re interested in because it uses these to evaluate the devices that meet these capabilities.

I’m still fleshing out what the whole interface will look like, so all of the features haven’t been determined yet, but I will keep posting my designs and a link to the repo once I have it in a state where it’s ready for an initial commit.

Long time readers will have realised that I have been spending a lot of time with the technology that was formally known as Perceptual Computing (PerC). You may also know that this technology is now known as RealSense and that it will be rolling out to a device near you soon. What you might not know is that I’m currently writing a course on this technology for Pluralsight. As part of writing this course, I’ve been creating a few little wrapper utilities that will make your life easier when developing apps with the SDK.

In this post, I’m going to show you a handy little method for working with API methods. Pretty much every RealSense API method returns a status code to indicate whether or not it was successful. Now, it can get pretty tedious writing code that looks like this:

As you can imagine, the more calls you make, the more status checks you have to do. Well, I like to log information about what I’m attempting to do and what I have successfully managed to do, so this simple method really helps to write information about the methods being invoked, and to throw an exception if things go wrong.

Okay, that’s possibly a bit too melodramatic a title, but that’s almost what it felt like when my Ultrabook decided to pack in and shuffle off to the great gig in the sky. This post will contain no screenshots for reasons that will soon become abundantly clear.

Some context first, I’ve been using the Ultrabook pretty much continuously since I got it. It was my default, go-to, day to day development box. Yup, that’s a lot of different ways of saying that I was using the Ultrabook very heavily. So heavily, in fact, that I was using it as the workhorse for developing my Synxthasia project for Intel ; a Theramin(ish) type of music application which translated gestures in 3D space into sound and visuals – it even took your picture at intervals and imposed that into the visuals; I was particularly proud of the fact that it gave you the ability to alter the “shape” of the sound to simulate a wah effect just by using your mouth. The SDP really is an excellent device for development tasks like this.

So, about a week before I was due to submit this to Intel; with the app heavily in the polishing stages, the Ultrabook died on me – taking 8 days of code that hadn’t been committed back to source control (I have no excuses really as I should have been committing this to source control as I went along). Remember that I said that it had the habit of giving the low battery warning and then switching off, well this is exactly what happened. Anyhoo, I plugged it in and started it back up – all was fine and dandy here, but I noticed that Windows was reporting that there was no battery present. Well, I couldn’t leave the Ultrabook permanently plugged in and I needed to go to a client site – I unplugged the unit and set off. When I tried to power it back on later on, it refused to start – all that happened was the power button LED flashed blue and that was it.

I got in touch with contacts in Intel and they provided some first line support options which included opening the unit and disconnecting the batteries. Unfortunately, this hasn’t worked – the unit still has all the vitality of the Norwegian Blue. Intel has offered to replace the device but as I have been away from the physical unit for a while, I’ve been unable to take them up on their kind offer. Once I get back home, I will avail myself of their help because the device itself really is that good.

What has become clear to me over the last year or so, is just how much Intel cares about the feedback it gets. I cannot stress enough how they have listened to reviewers and developers like myself, and how they have sought to incorporate that feedback into their products. As a developer, it’s a genuine pleasure for me to deal with them as they really do seem to understand what my pain points are, and they provide a heck of a lot of features to help me get past those problems. Why do I say this? Well, I was lucky enough to have an SDP from last year to use and while it was a good little device, there were one or two niggles that really got to me as I used it – the biggest problem, of course, being the keyboard. The first SDP just didn’t have a great keyboard. The second issue was that the first SDP also felt flimsy – I always felt that opening the screen was a little fragile and that I could do damage if I opened it too vigorously. Well, not this time round – the updated version of the SDP has a fantastic keyboard, and feels as robust as any laptop/Ultrabook that I’ve used, and all this in a slimmer device. You see, that’s what I mean by Intel listening. I spend a lot of time at the keyboard and I need to feel that it’s responsive – and this unit certainly is. The thing is, Intel aren’t selling these units – they are giving them away for developers to try out for free. It would be perfectly understandable if they cut corners to save costs, but it’s patently apparent that they haven’t – they really have tried to give us a commercial level device. Yes, I’ve been unlucky that the Haswell died on me, but given the care from Intel this hasn’t tarnished my opinion of it.

So, does the death of the device put me off the Haswell? Of course it doesn’t. For the period I have been using it, it has been a superb workhorse. It combines good looks with performance with great battery life. It’s been absolutely outstanding and I wouldn’t hesitate to recommend it. Needless to say, I’m delighted with the help and support that I’ve had from Intel – it certainly hasn’t been any fault of theirs that I’ve been unable to return the device for them to replace. I’d also like to thank Rick Puckett at Intel for his help and patience, as well as Carrie Davis-Sydor and Iman Saad at DeveloperMedia for helping hook me up with the unit in the first place and their help when the Haswell shuffled off the mortal coil.

Well, since my initial review of the Ultrabook, I’ve been using it for pretty much all my computing needs. From writing C++ and C# code, through standard surfing and the likes, through to heavier duty 3D work, and the Haswell just keeps on going.

First of all, let me just say that the keyboard on this generation of Intel SDP is a whole lot better than the one that shipped with last years SDP. This one actually feels solid and responsive, and it features a nice little backlight. Intel really has upped the game for what are effectively demo units here. My only, admittedly minor, niggle is the fact that the keyboard is designed for the US market. When I change it to the UK settings, the \ key is missing.

Writing code using this Ultrabook is a pleasant experience. The screen resolution is good, and the screen itself is bright and very pleasant to use. Unfortunately, the 4GB memory means that there’s sometimes a slow down when compiling a C++ application from scratch using Visual Studio 2012. Once you’re past the first compilation though, the compile process is responsive enough, so this isn’t a huge issue. As I’m a heavy user of Visual Studio and WPF, I’m pleased to see that the responsiveness of the WPF designer window is nice and snappy – there’s no hold up in the system making me wish I was using a different machine. Even the dodgy upper case menus in Visual Studio don’t distract me while I’m looking at this glorious screen.

One thing I’ve tried to do is keep the settings of the machine as default as possible (except for the keyboard layout – that’s’ a compromise too far). This has allowed me to test the claims that the Haswell gives me long battery life. Well, it’s true – using it for a mixture of DirectX development, 3D work and WPF development has allowed me to get just over 7 hours on a single charge. It’s not quite the potential 12 hours, but my day to day usage is pretty atypical, so I wouldn’t be unduly worried about the battery life if I were looking to use this on a really long journey. The reason that I mentioned the defaults is because Windows 8 suddenly tells me that I’ve got about 5% battery life left, and then it shuts down – there’s not enough time for me to get the unit plugged in then.

Now Pete – surely it can’t be all sweetness and light, can it? Where are the niggles that must exist here, or are you playing nicey-nice with Intel here? Well, there is one thing that really gets me annoyed and that’s the Trackpad. If I click on the left hand side of the Trackpad (anywhere on the left), I can trigger a click – and that’s exactly what I’d expect. Clicking or tapping on the right hand side doesn’t trigger anything. This has caused me quite a bit of frustration. It’s my issue to deal with though, as the Trackpad is a multi touch unit so it’s something I’m going to have to work on.

You may wonder what applications I typically have running on the Ultrabook, to see how this compares to what you would use. Well, on a day to day basis, I typically have up to 4 instances of Visual Studio 2012 running simultaneously. I also have a couple of instances of Expression Blend running, as well as Chrome, with a minimum of 10 to 12 fairly heavy web pages in them (if you’re really interested, these pages are Facebook, Code Project – several instances covering different forums and articles,, GMail, Twitter, The Daily Telegraph and nufcblog.org). I also usually have Word, Excel, Photoshop, Huda and Cinema 4D running, along with Intel’s Perceptual Computing SDK and Creative Gesture camera. As you can see, that’s a lot of heavy duty processing going on there, and this machine copes with them admirably.

Okay, to whet your appetite – I’ve been using the Ultrabook, off the charger, just editing documents and web pages for the last two hours, and I still have 91% battery life left. That’s absolutely incredible.

The reveal

Bonk, bonk bonk bonk bong….

That’s the sound that greeted me when I opened the box from Intel™ featuring a developer preview Haswell Ultrabook™, and there’s no geek in the world who wouldn’t want to repeat that sound, so after opening and closing the case for ten minutes or so, I decided to actually get the Ultrabook™ out of the box.

Okay, this is the Ultrabook™ packaging (outside of the cardboard it was shipped in). Little did I know the simple joy I was to receive upon opening this box.

I’ve opened up the packaging, and this is the sight that greets me. I’m keen to start the unit, but I want to see what other goodies the box has inside.

Is that a carry case for the Ultrabook™? Nope, it’s just packing material to keep the unit from getting scratched.

Ahh, that’s more like it. Cables and a USB drive (containing the Windows reinstallation media). The reflection in this image is from the plastic covering that protects this layer.

Okay, the plastic is off and I can see the quick start guide as well as the power supply and USB drive. That PSU is tiny. Consider me seriously impressed.

What have we here? Ah, Mini HDMI to Full, to VGA and a USB to Ethernet. Thoughtful of them to supply this.

Well, there’s the keyboard. Nice.

After starting the machine up, it’s a simple matter to configure Windows™ 8. Windows really suits a machine like this.

Day to day

Well, the first thing to do with a machine like this is to take it out for a drive. In my case, this means installing Visual Studio Ultimate 2012. As you might expect, I have several computers, so I have a lot of reference there on how long this actually takes to install. On my old I7 with a spindle hard drive, it took up to an hour to install. The question then, is how long would it take to install on this machine with the same feature set. Start to finish, the whole process took 8 minutes. Yup, you read that right, 8 minutes start to finish. I couldn’t ask for a better initial impression than that.

The good

This unit is fast. Oh, it’s so very fast. Okay, the boot up time is a tad slower than my I7 Lenovo™ Yoga 13, (by less than half a second), but once you actually get going, the machine is very responsive. While it’s a fairly entry level I5, it is more than capable of coping with my day to day development tasks. Opening up a XAML page in Visual Studio 2012 and showing it with both the designer and code view open, the code view updated Intellisense as I typed, and the designer refreshed itself snappily – those of you who develop XAML based apps know how slow this can be – it was an absolute joy to behold.

The unit is running Windows 8 Pro (Build 9200), and Windows works well on it. The screen is responsive to the touch, and Windows feels snappy and responsive.

The machine is light. Maybe it’s not Mac Air light, but it’s heading in the right direction, and it’s thin. If you were going to use this only on your desktop, this wouldn’t matter but if, like me, you travel a lot, you’ll really appreciate them.

That battery life. Oh boy, 6 hours spent doing a lot of fairly intensive development and the battery still has plenty of life to give yet. This really has hit the sweetspot with regards to providing me with portability. I can easily see myself whiling away journeys using this.

The niggles

This is going to sound churlish considering I’ve received this unit as a “favour”, but I do wish it had more than 4GB of RAM and a bigger hard drive than it has (a 128GB SSD).

The shift keys are on the large side, but I have no doubt I will soon get used to them.

There’s a lot of real estate lost around the screen, but I’ve no doubt that commercial units will take care of this, using all the available space.

It’s hard to put my finger on what the exact problem is, but the keys don’t feel quite right to me. There’s a bit of play to them that doesn’t make them feel that stable. Obviously, as this is not a commercial Ultrabook™, I’m not unduly worried – I would expect a commercial Ultrabook™ to have a more solid feel to the keys.

So, where are we at?

I’ve been asked to write the reviews as unbiased as I possibly can be. This review is my initial impression only. My next review will cover using this machine on a daily basis. As I do a lot of Ultrabook™ related work as part of my daily development routine, this will be my primary device, so I’ll let you know how this compares.

This is a copy of the post I made on the Intel site here. For the duration of the contest, I am posting a weekly blog digest of my progress with using the Perceptual Computing items. This weeks post shows how Huda has evolved from the application that was created at the end of the third week. My fellow competitors are planning to build some amazing applications, so I’d suggest that you head on over to the main Ultimate Coder site and follow along with their progress as well. I’d like to take this opportunity to wish them good luck.

Executive summary

The judges will be pleased to know that this blog entry is a lot less verbose than previous ones. This isn’t because there’s not a lot to talk about, but rather because I’m concentrating on writing a lot of code this week, and I’ve hit a real roadblock with regards to the gesture camera. Code that was working perfectly well last week has suddenly stopped working. This means that I’ve got some serious debugging to do here to find out why the gesture code is no longer displaying the cursor on the screen – this doesn’t happen all the time, but it’s happened often enough for me to be concerned that there’s an underlying problem there that needs to be sorted out.

As part of my remit is to help coders learn to write code for the Perceptual SDK, I will be continuing writing about the coding process, so I apologise to the judges in advance – there’s nothing I can do to alleviate the detail of explanation. If they wish to skip ahead, the summary talks about where we are today and what my plans are for next week, and here are some screen shots. Unlike my previous blogs, I’m not going to be breaking this one down on a day by day basis. There’s a lot that needs to be done, and it doesn’t break down neatly into days.

For those who want to know the result, don’t look away now. I found and fixed the issue with the gesture code. It’s all a matter of patient debugging and narrowing down areas where there can possibly be problems.

I’ve been wondering how best to train people in how to use Huda, and this week I’ve hit on the solution. I’m going to let Huda teach you. By the end of the week, I added in voice synthesis, and I think I can use this to help provide context. The type of thing I have in mind is, say you bring up the filters, Huda will tell you that a swipe left or right will cycle through the filters, and the thumbs up will add the filter to the currently selected image. The beauty is, now that I have the code in place to do the voice synthesis, I can easily add this type of context.

Voice recognition is an area of the Perceptual SDK that has a long way to go. It just isn’t accurate enough and, for that reason, I’m dropping support for it from Huda. I had the commands “Open Filter” and “Close Filter” in the application, and no matter what I tried, it kept turning Open Filter into Close Filter. Yes, it recognises a few words, but given that the accuracy is so hit and miss, I can’t leave it in there. So, I apologise to you right now – voice control is out. I had to take this decision this week because there are certain things I was using it for that I have to find other ways to accomplish these things and I really don’t have the time to keep mucking around with this.

If you don’t want to read the rest of the post, this is what Huda looks like now:

Woe to the interface

It’s going to be a busy week this week. The more I play with the interface, the more I find it clunky and unpolished, and this is incredibly frustrating. While I know that I’m a long way from polishing the interface, I can’t help but think that I’m missing a trick or two here. The biggest thing that I have issues with is the concept of showing the folders and selecting the images. So, the first thing I’m going to do is to unify the two parts; this should make it obvious that the picture selection and folder selection actually belong together. Part of my thinking here is that I want it to be apparent to anyone who’s used a photo editing application before that this is still a photo editing application. While it’s fine to play about with an interface, there still has to be a certain level of familiarity; there has to be a feeling of comfort in using the application otherwise the user will end up being overwhelmed and will give up on the application. While the contest is about working with Perceptual computing and features of the ultrabook, it’s important that I don’t lose sight of the fact that this must be an application that people can use.

Once I’ve got this in place, I want to address the selection of “things to do to the photograph”. By this, I mean that I want to put some form of toolbar into place to trigger options. Don’t worry that I’m throwing sections of the UI together – I’m not. What I will be doing here is giving the user obvious trigger points – I can use this to bridge the gap between traditional photo editing applications and Huda. There has to be some familiarity for the user here, so I’ll be providing that today.

You may think that this is going to take a lot of work, and in other languages you may well be right, but I’m using MVVM and WPF. As the different parts of the interface are loosely coupled controls, it’s a simple matter for me to move the controls into other containers, and they will still work. The only change I’m going to make is to keep the images showing regardless of whether or not the user has opened a folder.

With the basics of the “toolbar” in place, I now have something that feels more natural for using gestures with. I’m going to hook the filter selection into both touch and gestures, and do something a little bit fancy for displaying the list of filters. What I’ll do is move the filters into a carousel, and use the swipe left/swipe right to move backwards and forwards through the list.

I’m concerned about the speed of loading of various items when thingsare selected, so I will be moving things out onto background threads.

The interface is really starting to firm up now. It’s starting to feel that little bit more polished to me, but there is more that I can do with it.

Don’t worry that all the filters have the same name and appearance right now. This is purely a visual that will be changed soon – I need to come up with some good iconography for these. It’s strangely satisfying swiping your hand left and right to rotate through this list. My final action on this end will be to introduce a gesture that actually applies the filter. The thumbs up seems oddly appropriate to me.

The big reveal

After a frenzy of activity on the actual interface, I want to address one of the central concepts of Huda; namely, the ability to filter the image, and not touch the underlying image. This is where I give up the secret of Huda – it’s all a massive con. If you remember, I promised that the underlying image would not be touched. I’ve had people get in touch with all sorts of wild theories on how I was going to do this – from the person who thought I was going to use steganography to hide this information in the picture to someone who thought I was going to convert the file into a custom format and put something in place to fool image editors into thinking that the type was still the original file format.

If I were to tell you that the solution to this problem was incredibly straightforward, and blindingly obvious, would this surprise you? Well, here it is. Here’s how it’s done…. Did you catch that? Yes, it’s all smoke and mirrors. What Huda does is actually not modify the original image at all – instead, we maintain an index of all the files that we’ve edited in Huda, along with the details of all the edits that have been performed, and these can be played again and again and again. In other words, when you open a file up in Huda, it goes to the index first to see if you’ve previously opened it. If you have, it gets the image transformations and reapplies them there and then. It’s as simple as that. Of course, this does rely on you not moving photos around, but I do have some plans in place for a post contest version of Huda to take care of that as well. Anyone for hashing?

Transformers – Robert’s in disguise

A judgely warning – this section is going to be intense. I’m sorry, but there’s no way around the fact that we need to write some code, and there needs to be some explanation of it. It’s going to be quite dry, so I won’t blame you if you want to skip to the bit at the end.

As a first pass for the transformations that I want to apply, I decided that I needed to declare a common interface that all my transformation classes would apply. It’s a very simple interface, but as I want all my transformations to be serializable, I’ve made sure that it implements ISerializable by default

At this point it’s merely a matter of creating some simple transformations that use this functionality. Being the nice chap that I am, and to give Nicole some coding kudos that she can drop into casual conversation, I’ll show you all the classes I’m using for transformation. First of all, here’s the image crop routine:

There’s a lot of code in here that you’ll see in other classes, so I’ll just explain it once and you should easily be able to follow how I’m using it in other locations. The first thing to notice is that I’ve marked this class as Serializable – you need to do that if you want .NET to do it’s mojo when you save things. You’ll see that there are two things inside the class that say CropImage; these are the constructors that are used to create the object. The second one is the special constructor, and is the reason that I had IImageTransform implemenent ISerializable. .NET knows, when it sees this interface, that you want to read and write out the values in the class yourself. There are many, many reasons that I want to do this, but the main reason is because of what happens if you just let the runtime try and figure this stuff out itself – it’s not very performant because of the way it has to map things together. By taking control of this myself, I make the job easy. If you see how I use info.GetInt32 (there are many other methods for reading other types of data), it’s apparent what type of data the property is – we must always make sure we get it right, otherwise unpredictable and nasty things can happen.

At the bottom of the code, there’s a matching method called GetObjectData that just writes the items to the serialization stream. It’s important to make sure that you use the same names for the items in the constructor that you did in this method. The property names don’t have to match these names, but the names themselves do.

The Version currently does nothing, but if I have to add features to any of these classes, I can use this to work out which properties should be present, so the saving and reading of the data carries on seamlessly. It’s always a good idea to design your serialization with this firmly in mind.

The Left, Top, Width and Height properties must be set before we attempt to call Transform. If we forget to set them, all that will happen is that the rectangle that’s created will be 0 pixels in size, and we wouldn’t want a zero pixel crop.

The Transform method is where the “clever” stuff actually happens. I bet you thought there would be more to it. All it does is create a rectangle based on the size and location set in the properties, and then it uses the inbuilt CroppedBitmap class to actually perform the crop.

See! I told you that was easy. Next, I’ll show you what the FlipImage transformation looks like:

I won’t explain this code, as it’s largely the same as the CropImage code. If you’re interested why I didn’t put things like the Version and Transform into an abstract base class, please let me know and I’ll cover attempt to answer it in the comments. At this point, the judges are screaming for mercy and begging me to stop, so I’ll try not go too far into philosophical architectural debates.

The ResizeImage transformation is an interesting one (well, interesting if you’re into that type of thing), and the Transform code is more complex than other tranformations. To save you having to fight with this yourself, here it is

I have been asked how the gesture hookup behaves. Well, imagine you were tracking mouse moves to see which items were under the cursor, you might want to create a Blend Behavior that looked something like this:

With only a minimal amount of change, that’s exactly the Behavior that you hook up to a treeview.

And there you have it – code to do some basic transformations. The other filters that I apply will all follow this pattern, so it’s an incredibly trivial thing for me to put in place – and, more importantly, this provides the core functionality for me to save out the filters along with details of the files they apply to. All of a sudden, this application is starting to come together.

This weeks music

Deep Purple – Purpendicular

Elbow – Build A Rocket Boys

David Garrett – Rock Symphonies

Crash Test Dummies – God Shuffled His Feet

Nickelback – All The Right Reasons

Jethro Tull – Aqualung Live

Creedance Clearwater Revival – Creedance Clearwater Revival

Deep Purple – The Battle Rages On

Caitlin – The Show Must Go On

Status update

I’ve reached the end of the week and it’s worth taking stock of where I’ve reached. The harsh reality right now is that I won’t be delivering a fully-featured photo editing application at the end. Before the alarm bells start ringing, this is exactly what I expected. The aim for this competition is not to release the next Photoshop at the end of week 7. This would be a physical impossibility, given how I’m incorporating none Photoshop features. I am confident, however, that I will deliver what I was aiming at from the very start – a none destructive gesture and touch driven photo editing application. For this reason, I am going to only going to incorporate a limited number of filters. Don’t worry though, if you want to use Huda, I will continue developing it once the contest has finished. Your investment in using it will not be wasted.

Right now, Huda is limited to only editing a single image at a time. If I have time, before the end of the contest, I will give Huda the ability to edit multiple photos. Over the next week, I’m going to be tackling the reordering of items in the list, fleshing out that toolbar and also having the interface behave a little bit differently when it’s in tablet or desktop mode. Right now, the design has all been based on a tablet experience – but let’s see what we can do for the desktop users. Oh, and I should also have the first draft of the voice context in place as well.

This is a copy of the post I made on the Intel site here. For the duration of the contest, I am posting a weekly blog digest of my progress with using the Perceptual Computing items. This weeks post shows how Huda has evolved from the application that was created at the end of the first week. My fellow competitors are planning to build some amazing applications, so I’d suggest that you head on over to the main Ultimate Coder site and follow along with their progress as well. I’d like to take this opportunity to wish them good luck.

Week 2

Well, it’s week 2 now and development is carrying on apace. I’ve managed to get a fair bit into Huda in the first week, so the question has to be asked “Will Peter manage to maintain this awesome pace?” We all know that was the question you wanted to ask, but were far too nice to ask, so we’ll take it as read, shall we?

Monday

Today, I’m going to be working on adding some filters to the pictures that I’m opening. In the short term, all I’m going to do is hardcode the application to apply blur and Sepia filters to any picture that opens. This isn’t a huge task, and it’s a great way to get something in place that I can use to style the open filters window.

As I want to use filters from many places, I’m going to create a central location for all my filter code. I’ll treat this as a service and use something that’s known rather nerdily as Service Location to get a single reference to my filter code where it’s needed. Ultimately, this is going to be the master model for the whole approach of managing and manipulating the filters, including the “save” method. One thing I want to do with this approach is get away from the concept of the user having to save work. It’s no longer 1990, so why should the user have to remember to save their work. This is probably going to be the most controversial area for those who are used to traditional desktop applications but, given that we are building the next generation of application for people who are used to working with tablets and phones, I think this is an anachronism that we can dispense with quite happily.

Right, that’s a central filter model in place, so all I need to do is hook up the photo selection code and *bang* we have our filters being applied in place. As that was so easy to do, why don’t we get some feedback on the photograph as well? Let’s have the RGB histograms and display them to the user. We’ll put them at the bottom of the display, and add some animation to them so that we can start to get some feedback from them. Again, because I’m working in WPF, and I’m using the MVVM design pattern, the actual screens and animation are incredibly trivial to implement.

As an aside, here, you may be wondering what I’m using in terms of external libraries other than the Perceptual SDK. While I developed my own MVVM framework, I decided for this implementation to go with my good friend Laurent Bugnion’s MVVM Light, as there’s a good chance that the WPFers who end up downloading and playing with the source will have had exposure to that. I contemplated using Code Project uber developer Sacha Barber’s excellentCinch implementation, but the fact that Laurent had converted MVVM Light over to .NET 4.5 was the winning vote for me here. For the actual photo manipulation, I’ve gone with AForge.NET. This is a superb library that does all that I need, and so much more. Again, I could have gone with home rolled code, but the benefit of doing so is far outweighed by the tight timescales.

Tuesday

It’s Tuesday, now, and Intel have arranged a Google hangout for today. This is our opportunity to speak directly to the judges and to give feedback on where we are, and what’s going on. I’ve left the office earlier than I normally would, so as to be in plenty of time for the hangout. All I need is a link from our contacts and I am good to go.

Okay, the hangout has started and I’ve still had no link – nothing. I can see the other parties, but I can’t get in. I’ll fire off an email and hope that someone opens it to let me in. Robert and Wendy are indredibly clued in, so I’m hopeful that I should be able to join soon.

The hangout has ended, and I’m feeling pretty despondent now. Nothing came through, so I haven’t had a chance to talk things through with the others, or to talk to the judges. I’d hoped that they would be online, as well, so we could get some feedback from them. Still, this did give me time to noodle around on a guitar for an hour or so – something that I haven’t had anytime for since the competition started – so that was good. For those that are interested, I was practicing the Steve Morse trick of alternate picking arpeggios that you would normally sweep pick; talk about a challenge. (Note – I was playing this on an Ibanez Prestige RG550XX – a very smooth guitar to play on).

Anyway, I don’t have time to let this get in the way of the coding. As all the other competitors noted, time is tight. Time is so tight that at least one team has allocated a project manager to help them prioritise and manage this as a deliverable. I can’t do this unfortunately, but at least I won’t have to fill in status reports on my progress other than through the blog. One of my fellow solo coders, Lee, made some bold claims in his blog posting, and what I’ve seen from his blog video shows that his bold claims are backed up by bold ability – he’s definitely one to watch here; his project really could revolutionise online webinars and skype like activities.

Todays code is doing some animation on the histogram views, and then hooking that animation up to the gesture camera. This is where things start to get serious. There are, effectively, two approaches I can take here. The first approach is to take the coordinates I’m getting back from the camera and use the VisualTreeHelper.HitTest method in WPF to identify the topmost window that the coordinates match with. This is fairly trivial to implement, but it’s not the method I’m going to take for several reasons, the top ones being:

This returns the topmost item, regardless of where the mouse is. In other words, it will return the top item of the window, even if I’m not over something that I could be interested in. This means I’ll get continuous reports of things like the position being over a layout Grid or Border.

The hit test is slow. What it would be doing, on every frame from the camera, is looking through the VisualTree for the topmost item at that point. That’s an expensive computational operation.

Once I’ve identified that the topmost item is actually something I’m interested in, the code would have to trigger some operation there. This means that I would have to put knowledge of the views I’m showing into the code behind the main window. This is something that I want to avoid at all costs, because this hard coupling can lead to problems down the line.

The approach I’m taking is for each control that I want to interact with the gesture camera to subscribe to the gesture events coming out of the gesture camera. What this means to the code is that each control has to hook into the same gesture camera instance – I’ve created what’s known as a singleton in my Perceptual SDK that serves up a single instance of each perceptual pipeline that I’m interested in. In real terms, this currently intersperses the gesture logic in a few locations, because I want the code that’s displaying the finger tracking image handled in the main window, but the code that’s actually working out whether or not the finger is over the control needs to be in the control. Today, I’m just going to get this working in one control, but the principle should allow me to then generalise this out into a functionality you can subscribe to if you want to have this handled for you.

As I’ve added an animation to the the histogram view, this seems to be a great place to start. What I’ll do here is have the animation triggered when the gesture moves into the control, and have a second animation triggered when the gesture moves out of the control. I now have enough of a requirement to start coding. The first thing I have to do here is to hook into the gesture event inside the Loaded event for the control; this is done here so that we don’t end up having problems with the order that views are started meaning that we try to hook into the event when the pipeline hasn’t been initialised.

Okay, now that I’ve got that event in place, I can work out whether or not the X and Y coordinates fall inside my control. Should be simple – I just need to translate the coordinates of my control into screen coordinates and as I’m just working with rectangular shapes, it’s a simple rectangle test. So, that’s what I code – half a dozen lines of boilerplate code and a call to trigger the animation.

Time to run the application. Okay, I’m waving my hand and the blue spot is following along nicely. Right, what happens when I move my hand over the control I want to animate? Absolutely nothing at all. I’ll put a breakpoint in the event handler just to ensure the event is firing. Yup. I hit the breakpoint no problem, but no matter what I do, the code doesn’t trigger the animation. My rectangle bounds checking is completely failing.

A cup of coffee later, and the answer hits me, and it’s blindingly obvious when it does. I’m returning the raw gesture camera coordinates to Huda, and I’m converting it into display coordinates when I’m displaying the blue blob. What I completely failed to do, however, was remember to perform the same transformation in the user control. In order to test my theory, I’ll just copy the transformation code in to the control and test it there. This isn’t going to be a long term fix, it’s just to prove I’m right with my thinking. Good programming practice indicates that you shouldn’t repeat yourself (we even give this the rather snappy acronym of DRY – or Don’t Repeat Yourself), so I’ll move this code somewhere logical, and just let it be handled once. The logical thing for me to do is to actually return the transformed coordinates, so that’s what I’ll do.

Run the application again and perform the same tests. Success. Now I have the histogram showing when I move my hand over it, and disappearing when I move my hand away from the control. Job done, and time to turn this into a control that my user controls can inherit from. When I do this, I’m going to create GestureEnter and GestureLeave events that follow a similar pattern that WPF developers are familiar with for items such as Mouse and Touch events.

One thing that I didn’t really get time to do last week was to start adding touch support in, beyond the standard “press the screen to open a photo”. That’s something I’m going to start remedying now. I’m just about to add the code for zooming and panning the photo based on touch. Once I’ve got that in place, I think I’ll call it a night. I’ve decided that I’m going to add pan and zoom to the photograph based on touch events; the same logic I use here will apply to gestures, so I will encapsulate this logic and just call it where I need to. WPF makes this astonishingly simple, so the code I will be using looks a lot like this:

Now, some people think that MVVM requires you to remove all code from the code behind. Far be it for me to say that they are wrong but, they are wrong. MVVM is about removing the code that doesn’t belong in the view, from the view. In other words, as the image panning and zooming relates purely to the view, it’s perfectly fine to put the logic into the code behind. So, let’s take advantage of the ManipulationDelta and ManipulationStarting events which give us the ability to apply that ol’ pan and zoom mojo. It goes something like this:

I’m not going to dissect this code too much but, suffice it to say the ScaleAt code is responsible for zooming the photo and the Translate code is responsible for panning it. Note that I could have easily added rotation if I’d wanted to, but free style rotation isn’t something I’m planning here.

Wednesday

Well, it’s a new day and one of the features I want to be able to do is to retrieve the item that a finger is pointed at, based on it’s X/Y coordinates. As WPF doesn’t provide this method by default, it’s something that I’ll have to write myself. Fortunately, this isn’t that complicated a task and the following code should do nicely.

Thursday

Today I’ve really been motoring, and managed to pull slightly ahead of my schedule, but I’ve also hit a bit of a gesture road block. So, I’ve updated the gesture code to retrieve multiple finger positions, as well as to retrieve a single node. This actually works really well from a code point of view, and it’s great to see my “fingers” moving around on the screen. The problem is that one finger “wobbles” less than a hand, plus the item selection actually works better based on a single finger. At least I’ve got options now – and the gesture code is working well – plus, I’ve managed to hook single selection of a list item based on my finger position. This allows the gesture code to mimic the touch code, something I’m going to explore more in week 3. My gesture selection code has now grown to this:

I’ve hooked into the various events from here to handle the type of interactions we are used to seeing from mouse and touch. Again, there’s a bit of repeated code in here, but I’ll be converting this into WPF behaviors which you will be able to use in your own projects. I’ve said it before, I love writing Blend Behaviors – they really make your life easy. As a personal plea to Microsoft; if you want to get people to write for WinRT, give us the features we’re used to in XAML right now. The fact that I can’t use Blend Behaviors in Windows 8 “Modern” applications is one more barrier in the way of me actually writing for the app store.

Friday

Following the big strides forwards I made with the gesture code yesterday, I’m going to have an easy one today and hook up the filter code. A lot of the time here will just be putting the plumbing in place to actually display the filters and select them. Initially, I’m only going to offer a few filters – I’ll add more next week. The key thing here is to prove that I can do this easily. I’m not spending too much time on the UI for the filters right now, but I will be adding to it in the near future. The point is that, while the interface is quite spartan right now, for a seasoned WPF developer, actually beefing it up isn’t a major task. The key thing I have in place is that all the main interface areas are solid black, with a glowy border around them. Once I’ve added the filters in and hooked them up, I think I’ll post a picture or two to show the before and after filter behaviour of Huda.

This screenshot is taken immediately after the very posh sheep photo was loaded:

If you look carefully at the bottom of the image, you’ll see the very faint Histogram view that I was talking about earlier. I’ve made this part of the interface transparent – gesture over it, or touch it and it becomes opaque. I’ll be doing something similar with other operations as well – the photo should be the view point, not the chrome such as the available filters.

I’ll apply the channel filter and blur like this:

Well, they have applied quite nicely. I’ll add other filters and the ability to set their values next week.

This weeks music

Deep Purple – Purpendicular

AC/DC – Back in Black

Andrea Bocelli – Sacred Arias

Muse – Black Holes and Revelations

Chickenfoot = Chickenfoot III

Jethro Tull – Aqualung and Thick As A Brick

End of week 2

Well here we are. Another week of developing and blogging. Where do I think I’m at right now? In some ways, I’m ahead of schedule but in other ways I feel I’m behind schedule. Oh, the code will be finished on time, and the application will do what I set out to do, but when I look at what my competitors are doing I can’t help feeling that I could do so much more with this. The big problem, of course, is that it’s easy to get carried away with “cool” features but if they don’t add anything to the application, then they are worthless. While the things that the other teams are doing make sense in the problem domains they are using, some of these features really don’t make sense in a photo editor. Perhaps I could have been more ambitious with what I wanted to deliver at the start, but overall I’m happy with what I’m going to deliver and what is coming up.

As a corollary to this, have you checked out what my fellow contestants are planning? Please do so and comment on their posts. They are all doing some amazing things, and I would take it as a personal favour if you would take the time to offer your encouragement to them.

By the end of week 3, I fully intend to have the whole central premise linked in. If you remember, my statement for Huda was that edits should be none-destructive. Well, they aren’t, but they aren’t saved anywhere right now. That’s what I need to hook in next week. The edits must be saved away, and the user must be able to edit them. If I haven’t delivered this, then I’ve failed to deliver on the promise of the application. Fortunately, I already know how I want to go about the save and it doesn’t touch on too much code that’s already in place, so it should be a nice and natural upgrade.

While I’ve thanked my fellow competitors, I haven’t taken the time to thank the judges and to our support team, (primarily the wonderful Bob Duffy and Wendy Boswell; what does the X stand for in your name?) There’s a real sense of camaraderie amongst the competitors, more so than I would ever have expected in a contest. We are all prepared to give our time and our thoughts to help the other teams – ultimately, we were all picked because of our willingness to share with the wider audience so there’s a natural symbiosis of us sharing with each other. Personalities are starting to come to the fore, and that is wonderful. Anyway, back to the judges (sorry for the digression). Come Wednesday, I hit refresh on the browser so many times waiting for their comments. Surprisingly enough, for those who know me, this is not an ego issue. I’m genuinely interested to read what the judges have to say. This is partly because I want to see that they think the application is still worth keeping their interest (and yes Sascha, I know precisely why Ctrl-Z is so important in photo apps given the number of disastrous edits I’ve made in the past). The main reason, though, is that I want to see that I’m writing the blog post at the write level. As someone who is used to writing articles that explain code listings, I want to make sure that I don’t make you nod off if you aren’t a coder, but at the same time I have a duty of care towards my fellow developers to try and educate them. So, to the judges, as well as Bob and Wendy; I thank you for your words and support.

You may have noticed that I haven’t posted any Youtube videos this week. There’s a simple reason for that. Right now, Huda is going through rapid changes any day. It’s hard to choose a point that I want to say “yes, stopping and videoing at this point makes sense” because I know that if I wait until the following day, the changes to the application at that point would make as much, or even more, sense. My gut feeling, right now, is that the right time to make the next video is when we can start reordering the filters.

One thing that was picked up on by the judges was my posting the music I was listening to. You may be wondering why I did that. There was a reason for it, and it wasn’t just “a bit of daft” on my part. Right now, this contest is a large part of my life. I’m devoting a lot of time to it, this isn’t a complaint as I love writing code, but you can go just the slightest bit insane if you don’t have some background noise. I can’t work with the TV on, but I can code to music, so I thought it might be interesting for the none coders to know how at least one coder hangs on to that last shred of what he laughingly calls his sanity. Last week, I didn’t manage to pick up my guitar so I am grateful that I managed to get 40 minutes or so to just noodle around this week. My down time is devoted to my family, including our newest addition Harvey, and this is the time I don’t even look at a computer. Expect to see more of Harvey over the next few weeks.

Harvey in all his glory.

So, thanks for reading this. I hope you reached this point without nodding off, and the magic word for this week is Wibble. We’ll see which judges manage to get it into their posts – that’ll see who’s paying attention. Until next week, bye.