Friday, December 20, 2013

Windows and Windows Phone 8 and its WinRT / WinPRT component model provide a nice way to leverage good ol' native C/C++ libraries in all WinMD language projections by wrapping them in a WinRT C++/Cx component. Such a native WinRT component only exposes WinRT types and can therefore be consumed through the WinMD projections also in C# or WinJS.
Often those good ol' native libraries provide events or callback functions for asynchronous calls. This means one would also have to provide a WinRT wrapper for those events and callbacks as well, although the WinRT has the nice IAsyncAction and IAsyncOperation constructs together with the Parallel Patterns Library (PPL) which make the WinRT component way better to use.
This post describes how to plug both worlds together to turn your good ol' C++ code into rose colored C++/Cx consumable through WinMD.

StartAsyncOperation([this](HRESULT error) -> HRESULT
{
// Do whatever here after the async is done...
// Notify the calling code through an event or another callback
// that the async operation has finished.
return S_OK;
});

It's worth mentioning this will also work with C function pointers and you can use a C++ 11 lambda for it as long as the lambda does not capture any variable. Stateless lambdas will just degrade to function pointers.

Way better API for the custom WinRT component if you ask me!
The trick to achieve that is the usage of the PPL's create_async, create_task and especially the task_completion_event which provides a nice way to make that callback-based model work with the proper WinRT modern async Task-based way.

Windows WinRT 8.1 brought the support of the RenderTargetBitmap and the ability to render the visual tree of the UI into a bitmap. WriteableBitmapEx makes it even easier in the latest version and you might end up in a situation where you want to save a WriteableBitmap to a file or to a stream in general. For that the raw pixel array of the WriteableBitmap needs to be encoded into a common format like JPEG, PNG, etc.

It's all pretty straightforward actually: The SaveWriteableBitmapAsJpeg creates a file in the user's pictures library and passes the file's stream and an BitmapEncoder ID on to the EncodeWriteableBitmap method which is doing the actual work. EncodeWriteableBitmap first copies the WriteableBitmap pixel buffer into a byte array, then creates the BitmapEncoder based on the file's stream and the desired BitmapEncoder ID, then sets the pixels and flushes all into the stream.
The BitmapEncoder class supports a couple of BitmapEncoder IDs out of the box like JPEG, PNG and more.

Monday, December 2, 2013

The lack of WriteableBitmap.Render in Windows WinRT 8.0 was quite an issue for many apps, but fortunately did WinRT 8.1 (re-)introduce the RenderTargetBitmap class which provides the functionality of rendering the visual tree of the UI to a bitmap. The new update of WriteableBitmapEx makes it even easier to use with a WriteableBitmap by introducing the FromPixelBuffer method.

You can download the binaries here or via the NuGet package. Thepackages contain the WriteableBitmapEx binaries. As usual all samples and the source code can be found in the repository.

How to use
The below code snippet shows how a part of the UI can be rendered into a WriteableBitmap via RenderTargetBitmap, how it can be modified and then finally presented back into the UI by using an Image control.

The Windows Phone File and Uri associations are very useful for implementing custom cross-app functionality like Pictures Lab and other apps provide. Windows Phone also provides uri schemes for many built-in apps and settings which can be very handy.
The Windows Phone 8 update GDR3 introduced a new setting to lock the screen rotation. This setting is quite useful especially when reading in bed or for providing on-screen QR codes to scanners.
This useful screen rotation lock is unfortunately hidden in the settings and it takes a few taps to access it. Therefore I've written a tiny free app called Screen Rotation Lock. It provides quick access to the screen rotation settings through a pinned secondary tile.

The relevant code in the MainPage is rather straight forward but there are some details which are worth sharing and which apply to any shortcut uri navigation:

The first if statement tests if the navigation comes from the secondary tile with the associated key UriQueryStringKeyPage. If it's a back navigation, the user was actually forwarded already to the setting and tapped back. In that case we don't want the user to land in our app, but rather go back to the start screen where she/he comes from, so we close our app. If it's not a back navigation the forward to the setting takes place using the Launcher API and the LaunchUriAsync method with the right uri for screen rotation.
The last if statement tests the version of the device and shows an error message if the device does not have GDR3 installed. This is done as last statement since it's actually only needed when the user wants to pin the secondary tile shortcut and opens the app itself.

Thursday, July 4, 2013

Windows Phone 8 introduced a nice new feature called File and Uri associations. This feature makes it possible to register an app for certain file extensions and Uri protocols. A couple of apps already support it and Nokia has a nice site which lists a couple.

Pictures Lab recently added support for that as well and uses it to call the app Instagraph. This means with the help of that app Pictures Lab now provides posting / sharing on Instagram. But that's not all, Pictures Lab also supports inbound Uri associations, so you can pass an image to Pictures Lab through the mechanism of an Uri association pretty easily.

How it works
In order to call Pictures Lab from an app this scheme has to be used:

fileNameinthemedialib:This is the filename of the image in the Media library / Pictures Hub. It does not matter if the file extension like .jpg is provided or not.

ClientId: This is an optional parameter. If it's provided then it should be the GUID that also matches the store product Id. It's not used in Pictures Lab now, but might be useful in the future to know the source of the protocol call and one could also use it to build a link like that: http://www.windowsphone.com/s?appid=ClientId

or to just launch Pictures Lab without passing an image:pictureslab:launch

Note, it's not necessary to UrlEncode the Uri, the Windows Phone platform handles it automatically.
If the user has Pictures Lab not installed and the above Uri is called, the Windows Phone OS is also smart enough to show the user a dialog which asks to search for an app in the Store that can handle the protocol. This will bring up Pictures Lab.

Conclusion
The concept of Uri associations is like the little brother of Windows 8's Share Charm, but it's still quite powerful and gives us app developers a nice way to combine our apps and make it a seamless UX. It's a win for the user and therefore it could be a win-win situation for the involved apps.

Future
For a next version I'm also thinking about a callback scenario. The user story would be: A user opens app A and wants to edit an image, this will open Pictures Lab, when the user is done with editing, Pictures Lab will call app A again. The user is back in app A with the edited image.
This callback mechanism would probably use a Callback parameter in the scheme above where app A can provide the UrlEncoded callback Uri with a placeholder for the edited pictureName. Something like that:

images:edit?Name=MyInputImageName.jpg&Callback=myapp%3Apost%3FName%3D%7B0%7DThe Url decoded part of the Callback: myapp:post?Name={0} This callback format string will be used in Pictures Lab to call app A and pass the edited picture back.

Makes sense?
Please let me know your thoughts and if that would be useful. Feel free to comment or reach out to me through any channel.

Tuesday, March 19, 2013

Natural User Interfaces are a trend since a few years and the development of new hardware technology for affordable prices helps to push that movement into the consumer market. The Kinect is one example, the soon-to-be-available Leap Motion is another one or the announced MYO device.

I'm one of the lucky people who applied and actually got a Leap Motion developer device before its release. I also have a Kinect for Windows. Both devices use computer vision with various sensors combined with artificial intelligence like machine learning for the object recognition. The Kinect for Windows recognizes skeletons with joints in a distance of 80 - 400 cm or 40 - 300 cm (Near Mode) which means the Kinect is perfect for the mid range distance but not for close PC interaction. The Leap Motion on the other hand is more for the close range and detects hands, fingers and pen-like objects very precisely. That's why I wanted to combine them and build a proof of concept hybrid solution that leverages both Kinect and Leap Motion to get the best of both worlds (close and mid distance).

In the video below I demo my proof of concept which is based on the Kinect for Windows SDK 1.7 and its Interactions and Controls components, plus the Leap Motion developer device and its beta SDK.
By the way, the Kinect for Windows SDK 1.7 is now publicly available and the bits are nothing less than awesome.

In case you are wondering about the infrared interference of the Kinect and Leap Motion. There's some, but I actually didn't notice any serious effect on the recognition capabilities of the Kinect nor the Leap.
Below is a picture of the Leap Motion as seen through the Kinect's infrared stream. The Leap Motion has three light sources but those are not very bright.

Some might say the Leap Motion is a competitor to the Kinect for Windows, I see it more like a nice addition to our developer toolset and the beginning of a story: Kinect ♥ Leap Motion.