This blog attempts to be a collection of how-to examples in the Microsoft software stack - things that may take forever to find out, especially for the beginner. I see it as my way to return something to the Microsoft community in exchange for what I learned from it.

29 October 2016

Intro

No, I am not drunk, nor did I smoke some peculiar produce our capital Amsterdam is famous for, and I did not bump my head a bit too hard on my car while getting out of it either. I was just curious how far the U of UWP would carry me. And it turns out, a lot further than I thought. It’s actually possible to run a HoloLens app on a Raspberry PI2. When I saw it happen, I actually had a bit trouble believing my own eyes, but it is possible indeed. With surprisingly little work. Although I added a little extra work to make it more fun.

Parts required & base material

I used a Raspberry PI2 running the latest Windows 10 IoT Core Insider’s build and a NexDock. Mainly because I have one, it’s handy and it looks cool. Be aware that to connect the NexDock keyboard you will need a Bluetooth dongle for your RP2 – or you will need use a RP3 which has it on board.

This post builds upon my (as of the time of this writing still incomplete) series about the Hololens Aircraft tracker, but that’s only because that’s a cool app to demo the idea on. It is no part of the series. I made a separate branch of the app at the end of the 6th episode. So this is like an interlude.

Some brute-force demolishing to get things to compile

Basically it’s as simple as setting the build target to ARM:

The only problem is, if you do that and then rebuild the project, it will complain about this

For the very simple reason this dll, which is part of the HoloToolkit, is not available for ARM. Hell, HoloLens runs x86 only, so why should it be available. No-one ever anticipated using a dolt peculiar person like me trying to run it on a bloody Raspberry PI2.

There is a rather crude way of fixing it. We are not using SpatialUnderstanding anyway, most certainly not on the Raspberry PI2. So I got rid of the plugin that Visual Studio complained about, by going to this folder in Unity:

And hitting delete. Rebuild project in Unity, try to compile it in Visual Studio. Alas. Still no success

But this is a different plugin dll – PlaneFinding. Also something we don’t use. Now this is a bit confusing, as there are three folder containing a Planefinding.dll. Maybe that’s an error in the HoloToolkit.

Whatever. Let’s get rid of the whole plugins folder under SpatialMappping. Once again, rebuild in Unity, compile in Visual Studio. And wouldn’t you know it…

The sweet smell of success. Now I want you to understand this is no way to go about to make a HoloLens project compatible with a Raspberry PI2. This is using a mallet to hit very hard on a square peg to make it go through a round hole. I have demolished two components you might want to use in the future version of your HoloLens app. But this is not serious development – this is hacking for fun’s sake to prove a point. That is why I have made a separate branch ;).

Controlling the app’s viewpoint

When you run the app on the HoloLens this is no issue at all. If you want to see the airport and it’s planes from closer up or from a different angle, you just move your head, walk to the object of your interest – or around it. If if runs on a screen, things are a bit different. So I created this little behaviour (with “ou” indeed, which suggests the good folks at Unity have been educated in The Queen’s English) that more or less replicates the key mappings of the HoloLens emulator:

It’s crude, ugly, the result is a bit stuttering – but it does the job.

Drag this behaviour on top of the camera (or for what matters, anything at all). You will be able to control the camera’s standpoint via what I have been told is the the standard PC gamer’s WASD and arrow key mappings. PageUp/PageDown will move the camera standpoint up and down, ESC will bring you back to the original viewpoint when the app started, and using SHIFT will make things go faster.

Deploy and run

Deploy the app to your Raspberry PI2 or 3 using Visual Studio – and select your PI as remote machine. Use either “Release” or “Master” build configuration. The latter should - in theory – go faster, but takes much longer (as in very much longer) to compile and deploy. Also, if you choose “Master”, the app does not always start on my PI, it’s sometimes only deployed – so you have to get it going via the device’s Device Portal. This may have something to do with me running an Insider’s build.

Either way – if you have WiFi either built-in or via dongle, that’s fine, but unless you particularly like waiting, connect your PI2/3 to a wired connection while deploying. I also observed issues when the app runs while only having WiFi connectivity – data comes in pretty slow, it can take minutes before the first airplanes appear, while on wired it takes like 30 seconds, max. Apparently my 2.4Ghz network (which the PIs use) is not that strong compared to the 5G all the other devices in the house employ.

And it works. Here are some pictures and a video of the app in action. The performance is not stellar (big surprise here, matching a $35 device against a $3000 device that comes straight out of Star Trek), but still – it works pretty reasonable.

Conclusion

Looks like the U in Universal Windows Platform is pretty universal indeed. Microsoft weren’t talking BS rubbish about this promise. This app can also be deployed to PCs (I learned that by accidentally choosing “Local Machine” as a target) and I don’t doubt it will run on phones and even XBox One, although I think I would have to ponder a little about the way to control the viewpoint on those devices as they don’t have a keyboard. Yet, an impressive result.

26 October 2016

Intro

This episode is nearly codeless, it’s all Unity and general geeky GIS stuff. We are going to cut out a piece of OpenStreetMap to depict the location of Schiphol. We are also putting a real 3D ATC tower on the map. If you look closely, it has only a very superficial liking to that of the actual Schiphol ATC tower, but those are just minor details ;)

Getting and positioning the map

Getting it was easy enough. As I wrote, I went about to cut out a piece of rectangular OpenStreetMap that contains Schiphol airport and, incidentally and certainly not coincidentally, the Wortell offices. We will deal with that in the final episode of the series.

Now positioning that map is a little tricky, as I have set the center Unity’s coordinate system to the center of Schiphol according to Google Maps. But the center of Schiphol, according to this nifty site which also displays coordinates, is found at a location that is about 25% from the bottom of the image and a little to the left.

The way I solved this was with a little bit of knowledge of geo coordinates and an advanced technique I learned in my years in Geo Information Systems, which can be described as ‘fiddling until it fits’ ;)

First order of business is to try and find out as closely as you can the lat/lon coordinates of the upper left and the lower right corner of the image. I also used latlong.net for that. As far as I could see, top/left is (52.368739, 4.706697) and bottom right (52.285193, 4.811196), while the center of Schiphol is (52.307687, 4.767424)

Now we need to go back to a precursor to this series in which I explained how to convert lat/lon coordinates to Unity’s X/Y/Z coordinates using a tangential plane. We can convert these coordinates using this code

So we now know the map in Unity is 0.474885850951468 (east-west) by 0.619819392903478 (north-south) meters. We will need this knowledge soon. But first, drag your map cut-out into your App/Textures folder in Unity:

Then proceed to create a material based upon this texture. In the Materials folder, right-click, hit Create/Material and call the material “SchipholMaterial” (or whatever it is you fancy). Then drag your Schiphol texture on top of the “Albedo map” square

Next, click the white rectangle to the right of the Albedo square. That pops up a color picker – set the color to B3B3B3FF to make the Schiphol map a bit less overly bright.

Then, in HologramCollection, create a 3D Plane object. Initially this will look like this:

What is important to wrap your head around is that at scale 1:1, a plane’s size is 10x10 of these blocks which is 10x10 meters in HoloLens. We need to reduce that size so that it’s about 0.62 by 0.475 meters as that is 1:15000 scale. As 10m = scale 1, we need to scale the plane by 0.0475 for X, and 0.062 for Z (leave Y to 1).

Now drag your SchipholMaterial below the “Add Component” button on your Plane. Yes! Unity shows or Schiphol map, with the right dimensions, the right size…

… upside down. Life – and Unity is - full of these little surprises. I have no idea why this is necessary, but if you go to the transform section of the object and hit “180” in the Rotation Y box,

… that will fix it. Unfortunately, we are still not done, and I will show you why. Add a Sphere to the HologramCollection, size it 0.015 in all directions, give it the Trail Line as material. You will notice a blue dot in the middle of the image if you look from above. That’s 0,0,0 according to Unity.

(I toned down the light a little to make this better visible). Unfortunately, as I stated before, 0,0,0 should be on another place, quite a bit down and to the right, near the “Schiphol” label. If we continue to use this, airplanes taking off won’t be properly aligned with the runways :(.

Summarize both X values, and divide the result by two. That is half of the difference in distance to what should be the X-center. This is about –0.038

Summarize both Y values, and divide that result by two as well. That is half of the difference in distance to what should be the Z-center (yeah, gets me every time too – what you tend to think of as Y from your Math class, is Z in Unity). This is about 0.143

Add those values to X and Z position in the Plane’s transform:

And boom. Compare that to the Google maps marker coordinate and that seems to fit pretty well.

Not an exact fit, but that’s because we manually determined the corner’s lat/lon positions. But we are not talking zoning permits here (lengthy legal battles have been fought over mere centimeters on those in my lovely country) so it will do for this app.

I left the sphere in the project as a reference but disabled it, as we don’t need it anymore but you can enable it again and see it indeed shows up at the center of the map.

So. Now we have a properly sized and positioned map. There is a tiny detail missing.

Adding an ATC tower

Once again we go to CGTrader and browse around for a tower. Unfortunately the iconic Schiphol ATC tower is not available (paid or free) so I settled for the Hong Kong ATC tower that bears some superficial resemblance to it. Unfortunately there is some building in the attached to the actual tower in model - that I neither need or want. I followed this rather crude work flow to deal with that:

I downloaded the model fbx model zip file

Unpacked it, and renamed 3d-model.fbx to tower.fbx

I created a folder “Tower” in my Assets folder in the Unity editor

I dragged the tower.fbx into that folder

And then I dragged the resulting tower prefab on the HologramCollection. It will be very large.

Set its position to far far away, like 5000,5000,5000 to be out of the way of what we have already created.

Double click on “Tower” to get in into view. I got about this:

Rotate around it till you get about this:

Now what we want to get rid off, is the building next to the tower. I don’t know if I am legally entitled to change a model, so I limit myself to hiding parts of it. This works as follows.

First, make sure this button is selected.

Select the model by clicking on the building. Not the tower. This will select the whole model nonetheless

Click the building again. This will only select the building

Go to the inspector. Uncheck the checkbox all the way to the top left

That will get rid of most of the unwanted stuff:

See? Rinse and repeat until all the stuff you don’t want is invisible. Remember: click once selects the whole model, click twice to select only what you want. Hidden too much? CTRL-Z is your friend. It’s a bit of a hassle, especially the small floating bits just right of the tower, but in the end you should be left with just this:

When you are done, scale the tower to 0.001,0.001,0.001 and put it’s location to 0,0,0. Select the Main Camera, then GameObject/Align View to Selected and you will see Schiphol again, with a tiny ATC tower standing on top of it. If you look at it almost dead from above, you will see that it’s a bit off to the right and the bottom, according to this aerial picture from Google Maps that ironically enough blocks the view of the actual tower itself with a label – so I marked it with a red circle

As it is the wrong tower – and as it’s not to scale, I did not bother with calculating it exact position but dragged it around a bit until it looked like it was in the same spot, which was –0.0283 for X and 0.0316 for Z. Fiddling a bit around with the view spot so I could see the tower’s base hitting the ground made it look like this:

And that’s fine with me. I’ll call it a day.

Conclusion

I have shown you how to orientate, scale and properly position (geographical) maps and objects on those maps in Unity, using some GIS knowledge, calculus, common sense and fiddling around. This makes your app look just that little extra pretty, plus it bases it in ‘real space’ which makes it extra cool.

Code – well, more like assets and config, can be found here. And I’d like to say hi to my fellow MVP Tom Verhoeff and his trainee Ryan, who seem to like this tutorial so I hope I made both their days ;)

17 October 2016

Intro

In the video in the first post of this series you can see the airplane models leaving a kind of trails in the air, while they move and rotate in a smooth fashion - in stead of merely hopping from place to place. The first is accomplished by adding a TrailRender to the AircraftHolder prefab – the second by using an awesome Unity plugin called iTween.

Adding the trail

Before you can add the trail, it must have a material from which it’s made. This the color of the line. I will freely admit materials are still a bit vague to me, and after quite some fiddling I came to the following settings:

Shader: HoloToolkit/Fast

Albedo: hex color 120067FF

Metallic: 1

Emission: hex color 021F61

This leads to a kind of metallic light blue to dark blue trail line (depending on where the ‘directional light’) is shining, which is quite satisfactory. Feel free to mess around with the parameters. I did ;).

Then go to App/Scripts and open the AircraftHolder prefab. Click on “Add Component”, then select “Trail Render”. Use the following workflow/ setting:

Expand “Materials”. Drag the Trail Line Material that we just created on top of the “Element 0” field

Set start width to 0.003

Set end width to 1e-05.

This means the trail will be visible for 120 seconds, so the tail of the trail line will be at the place where the airplane was two minutes ago. Older segments are automatically deleted. Also, near the airplane the trail will be 0.003 meter wide, petering out to 0.00001 meter (1e-05) at the very end. And sure enough:

Unity takes care of the drawing, erasing, and making-smaller-toward-the-end part. The only thing we have to do is move the airplane. Which is what we were doing already anyway. It once again almost feels like cheating.

Introducing and obtaining iTween

In a blog post about an earlier project, the CubeBouncer, I talked about using coroutines to achieve smooth animations. In the mean time I have stumbled upon iTween, an awesome Unity plugin created by one Bob Berkebile, that makes creating animations of all kinds a lot easier. Although some concepts are a bit odd – particularly the the use of the Hash object to pass in an arbitrary number of parameters into methods – once you wrap your head around it, it’s very easy to use. Oh. And it’s free too.

The easiest way to obtain it, is via the Unity Asset Store (Window/Asset Store or CTRL-9). Then enter iTween as search term. Click the iTween Logo, click import, and then the following dialog, as displayed to the right (below this paragraph), pops up. I usually deselect everything but the actual plugin, but if you would like to see samples and the Readme, feel free to leave everything checked.

When the process is finished, your project tab should contain the iTween plugin like below

Once you have the iTween plugin installed, build the project, and move over to Visual Studio.

Using iTween for moving and rotating

So we head back to the AircraftController’s SetLocationOrientation method. We are going to change that a little:

So if the first set of data (or the initial move to a location) as has not been applied yet – that is, this is a newly created airplane – just do the same as we did before – plonk the airplane at it’s location with the proper rotation and attitude, and give it a size that makes sense. But if this aircraft already exists – move it smoothly to it’s new location in 7 seconds, using local position. And rotate it too, if needed, in the same time. And that is allyou need for a smooth transition. Here you see the Hash in action. iTween has a few overloads for both Move and Rotate where you don’t need it, but in effect, when you want to do something only a little advanced – like using local space – say hello to Mr. Hash.

You can of course rant against these ‘stringly typed’ variables and talk about type safety and stuff like that – but this is how it works, and it’s perfectly usable, albeit a bit odd for those who are fans of tight architecture and clean code. But then again, HoloLens is a pretty out-of-the-ordinary device anyway. When in Rome, act like Romans. ;)

Making the text fade in and out

Now the only thing we are missing is that nice effect where the text fades out just before the airplane moves – and appears again – with the newly updated flight data. This prevents the text from flashing and also gives a nice indication whether or not we are receiving updates for an aircraft (you may have noticed by now that aircraft that have landed tend to hang around for a while just above the airport before disappearing).

So when the first move is indeed done, it will fade the label’s alpha channel to 0 in half a second. When that is done, call the original SetLocationOrientation. So this is a callback. What is a very important iTween convention to understand – default the callback will be called on the object you are operating on – and that is the label. Not the airplane. Therefore I added an “oncompletetarget” entry with the current gameObject as value – so iTween knows it should call SetLocationOrientation and use my gameObject as a target, not the label.

Oh, in the else - if the airplane was just created and does not have an initial location - just move it to it’s initial location without using iTween.

Then we move to SetNewFlightData and change the call to SetLocationOrientation to StartSetLocationOrientation:

This fades the label’s alpha channel (back) to 1, in 0.5 second, after a delay of 0.2 seconds. I found these values after experimenting. Feel free to play with times and stuff.

Conclusion

I have shown you how to add a trail line to an airplane, and how to use iTween for smooth transition effects. Once the basics of your HoloLens app stand, it’s quite easy to make just a little more effort and get it smooth and cool. Using a powerful tool like iTween makes it almost child’s play IMHO, once you get your head around how to using it.

15 October 2016

Time to rock and roll

All right my friends, it’s time to show the heart of the matter – how to read data from an Azure Mobile App Services data service and make the airplanes appear in the right places.

Adding the JSON data object scripts

As the projects are partially regenerated, and have a total different .NET baseline, I have not found a good way to share classes as binaries between full .NET code and Unity without getting in all kinds of trouble, so I choose the easy way out: I took

TrackType.cs

Coordinate.cs

Flight.cs

from the FlightDataService\DataObjects, moved them the App/Scripts/DataObjects folder in the Unity project. Then of course the service does not compile anymore, but I added the moved file as link. Then at least we have once source of truth as far as these classes are concerned.

Then we need a simple class to hold a list of flights with a timestamp (add that to the DataObjects folder as well):

Adding a DataService to read data

I opted to add the DataService as a Singleton that can be accessed in the app. But we have a bit of an issue here - Unity runs on Mono, e.g. .NET 3.something, and does not have things like packages for reading App services, or support async and Tasks and stuff. But remember – a HoloLens app is a two stage rocket. Unity does not generate an app for the HoloLens, but an UWP project, and that UWP project can use all the goodness from all the latest stuff. Meet the simple way to close the gap - my new friend UNITY_UWP.

Code between #if UNITY_UWP – #endif blocks is invisible to Unity – but will be accessible in the UWP HoloLens app that is generated by Unity. You will see Unity swallows it – no problems at all. There is only this tiny thing – you cannot add NuGet packages to Unity. So we have to do that in the UWP app, in Visual Studio. But… the UWP app is generated from Unity by File/Build and overwritten. The trick is to know not the whole app is overwritten, but only part of it. What I did was the following:

Generate the app in a subfolder “App” of the Unity project. This should be <your_root>\HoloATC_Demo\AMS HoloATC Demo\App

Open the solution in HoloATC_Demo\AMS HoloATC Demo\App (not the one in HoloATC_Demo\AMS HoloATC Demo)

Go to the project Assembly-Csharp

Add the NuGet Package Microsoft.Azure.Mobile.Client

Find the project.json file that belong to this project. You will find it’s not under AMS HoloATC Demo\App at all, but in AMS HoloATC Demo\UWP\Assembly-CSharp\project.json

This is the only file that has been changed with respect to the generated code. You will have to add this to source control. Even if Git claims it’s an ignored file.

After that – if you checkout the source in a different location or on a different machine, you will only have to regenerate the solution in the right place (the App folder), and revert this specific file. You will need to do this, or else the UWP app won’t compile.

Adding the AircraftLoader to create and update aircraft

I have created one class that is actually responsible for actually getting the aircraft data from the DataService – and one that is responsible for creating, updating and deleting aircraft. The updating – moving to a new position – is handled by the AircraftController – basically all the AircraftLoader says to the airplane is ‘here is new data for you – handle it’

Remember – it’s always best to start in Unity. So create an AircraftLoader and an AircraftController script in Assets/App/Scripts. Then proceed to double-click on AircraftLoader. This will open a Visual Studio instance. Be aware – this is not the project your will be running. This is the other solution, the one that is in the project root – I call this the ‘Edit’ solution. First, we are going to make sure the “usins” of the class are properly organized – that is, in a way that doesn’t make Unity go belly-up:

using System;
using UnityEngine;
using System.Collections.Generic;
using System.Linq;
using FlightServices.Data;
#if UNITY_UWP
using System.Threading.Tasks;
using Windows.Web.Http;
using Newtonsoft.Json;
#endif

Then we are going to add a whole lot of fields. And yes, some are public. These are things that can be set from the Unity editor. I have talked about that before.

The Aircraft GameObject field will be used to drag our AircraftHolder onto – so this behaviour knows which game object to create

TopLevelName is the name of the parent object to which all the aircraft are added to. This “HologramCollection” by default

_aircrafts is a dictionary of aircraft game objects with their id as key, so we can update/delete existing aircraft game objects based upon the data coming in

waitTime – minimal time to load new data

_lastUpdate – the time the last update of aircraft was completed

_receivedData – a queue with data coming from the service. I use this pattern all the time. I queue up data from the service, then read it in the Update method that Unity calls automatically 60 times a second. Don’t ever try to update game objects from .NET callback methods or events – you might regret it, or just get plain crashes – kind of like happens in UWP XAML apps, where you need the Dispatcher to take care of that

_topLevelObject – to store the game object with the TopLevelName in it. I have been told the method I use to find an object by name is quite heavy on performance – so better do it once and retain the result, right.

First it finds out if it’s necessary to download new data, but downloading and adding happens asynchronously. Then, if there’s any data in the queue, it first makes a list of all the id’s in the newly received flights.

Aircrafts that are in the aircraft game object dictionary (with the flight id as key) but are no longer in the list of flights, can be deleted (they have landed or moved out of Dutch airspace)

Aircraft that appear in that dictionary and in the list of flights need to be updated – possibly moved to a new position

Aircraft that do not have an entry in the game object dictionary are new, and need to be created.

It’s not that hard, see ;). The methods for creating, updating and deleting aircrafts are not that hard either.

The CreateAircraft is the most interesting – it instantiates the aircraft game object, makes the HoloGramCollection it’s parent, passes the actual flight data to the object, and adds it to the dictionary of aircraft game objects using the flight id as the key. Oh, and it also sets the scale to 0, making the plane effectively invisible. This is because, without setting a location on instantiation, the aircraft will appear on 0,0,0 before – where in future episodes we will place the center of Schiphol, very near the tower, and that’s not a place where aircraft belong.

UpdateAircraft just sends flight data to the game object, and DeleteAircraft – well, deletes it. There is only this matter of SetNewFlightData:

But that has the actual method that moves the flight data to the AircraftController commented out, because that does not even exist. What is worse, the Aircraft does not even have to component. So let’s head over back to the Unity Editor.

Wiring up some stuff in Unity

First of all, we drag the DataService and AircraftLoader Script from the App/Scripts folder on top of the HologramCollection’s inspector page. Then we drag AircraftHolder prefab from App/Prefabs on top of the AircraftLoader’s “Aircraft” property. Net result should be this:

Please make sure the Data Url property of the Data Service indeed points to the place where your data service as created in the first post is published.

Then go to the App/Prefabs folder, open the AircraftHolder prefab itself, and drag the (still default) AircraftController on top of the AircraftHolder’s inspector page.

Save the scene, then rebuild the app with File/Build settings etc. Go back to Visual Studio once the building is finished.

Creating the AircraftController

Having programmed OO since the start of this century, I tend to put control where I logically think it belongs, so rather than programming the ‘flight logic’ into a class that loads and translates data too (the AircraftLoader) I opted for putting it into a separate class that would be part of the game object. I tend to conceptualize these combined game objects and components as OO objects - although that is not completely right, it helps me think about it. The start is simple enough

_speed and _heading keep the last speed and heading. The stream of data sometimes misses a beat and provides no speed and/or heading – I prefer then to display the latest data, in stead of heaving the aircraft suddenly rotate to the North (heading 0) and back again when the next set of data is correct again

_text keeps a reference to the text mesh so I don’t have to look it up every update - potentially 60 times a second

the _initComplete boolean is a trick I use regularly to prevent other routines using variables like _text before the are initialized. Remember, the Update loop is called independently of what you do. It may look a bit overdone now with only one initialization statement, but believe me – there will be more.

I like to write the code at high level as almost self-explanatory. First we determine if the aircraft needs to be moved, then we ingest the data, and extract speed and heading. Then, when the aircraft needs to be moved, change it’s location, if not, just update the label. The ExtractSpeedAndHeading is pretty straightforward. There is only the thing with heading - that sometimes comes in a negative value, and although Unity3D has no problem with that in positioning the aircraft, I think it looks ugly in the label. So I make sure it's always positive.

Just some clever formatting to prevent empty postfixes like km/h and degrees in the label. By the way – stick to ye olde string.Format and don’t be tempted to use C# 6 string interpolation or you will be sorry (unless you put it between “#if UNITY_UWP – #endif). Anyway, on to the next routine, that actually does all the aircraft manipulation:

It sets the text too, then set’s the location based on the flight’s location, and set rotation based upon the heading of the aircraft and whether it’s going up or down. Notice I use local position, heading, and scale. This means all those things are relative to the position, rotation and scale of the containing GameObject – HologramCollection. This has the advantage that I can move, rotate and scale the containing object and in one go everything that is in it follows suit. So you don’t have to do all calculations for that – Unity takes care of that form me. I don’t use it in this app just yet, but the actual app already has some experimental code to do that (although that code is not yet in the store version).

Also, note the fact the airplane gets it’s size here (when it’s created it’s 0,0,0). I have the feeling the model is actually a 1:1 scale representation of the actual aircraft – the first time I saw it with the HoloLens I could not find it at first, then turned around and had a “Glimly Glider experience” - a giant aircraft silently swooping down on me from what looked only 10-15 meters. I decide to scale it down to 0.0015 of it’s original size so it appears to be about 10-15cm in a HoloLens – a size more beneficial for getting an good overview, not to mention the blood pressure and heart rate of the average user ;).

I found that the data, although it provides information about whether the aircraft is actually ascending or descending, that data is not always correct. So I decided to calculate that myself, based upon the difference between the current location and an older location. Now since an aircraft usually descends a lot slower than it takes off (which is very fortunate for the passenger’s – or at least my – peace of mind) this method basically returns –20 when the aircraft is going up, and 10 when it’s going down. And then we get some beautiful Unity3D math again – or more accurately, methods that prevent you from having to use all kinds of advanced 3D math:

For the heading we have to rotate around the axis that is going up (and down, too) from the center of the aircraft – this what they call yaw in aviation

For the vertical angle – that makes it look whether the aircraft is going up or down – we have to rotate around the axis that goes to the right (and left – so basically over the wings) from the center of the aircraft. In aviation, this is called pitch

You use summarize Quaternion.AngleAxis(angle, Vector3.<the axis you desire>).eulerAngles over multiple axes to get the combined rotation of an object and assign that in one go. Hence you see in SetLocationOrientation() the statement "transform.localEulerAngles = GetNewRotation();"

The final things

Go to AircraftLoader 's SetNewFlightData and uncomment the line

//controller.SetNewFlightData(flight);

Because we have implemented that in the previous section. Then there’s is the method LocationEquals in Coordinate, that we used in AircraftController but that was not implemented yet ;)

And if you run this in a HoloLens or the Emulator, you will see aircraft!

You will notice they won't move through the air but jump from position to position. They also don't show their track, there is no Schiphol Airport map, no ATC tower, no church, no gaze cursor - and you cannot select anything yet - but that is because this post is long enough as it is. The base is here. 3D visualization of a JSON stream. What is left, is basically making things more slick :)

Conclusion

In hindsight I might better have splitted this episode in two blog post still, but I hope you have made it to the end. I feel this blog post is the heart of the matter – reading a data stream and turning it into a 3D model – making ‘dry records’ come to life, almost literally. I also showed you some key concepts about positioning and rotating stuff in 3D.

08 October 2016

Intro

In this post we are going to create a game object holding the aircraft and a label showing aircraft data that always looks at the user. We are not going to worry about scaling yet – we will handle that from code, in the next blog post.

First some housekeeping

Just as in the CubeBouncer project, we are going to create some folders in the Assets first to keep our custom assets nicely separated. Create a folder App, and in this folder the following subfolders:

Audio

Materials

Prefabs

Scripts

Textures

UWPAssets

Creating the Aircraft Holder and including the aircraft

Inside the HologramCollection, create an empty GameObject called AircraftHolder. Then drag into that object the A320 object form the A320 folder. A rather large A320 appears in the Scene, as indicated in the previous blog post. Just ignore this for now.Open the A320 object (in the hierarchy, not the object in the A320 folder), disable the animator, and make sure X,Y,Z position and rotation are all 0,0,0 and scale 1,1,1:

At this point I have no idea what the Animator is supposed to do, but we don’t need it – I think.

Create a label

In this section we create the label above the airplane that displays flight information, like flight number, aircraft type, speed, altitude, etc. Right-click the AicraftHolder and create a “3D Text” Object. Call this “Label”. That label now sits at the bottom of the airplane, where apparently it’s origin is.

Open the label in the inspector, set the Y position to 15 and the Y rotation to 270. That places the text 15 meters above the airplane, placing it parallel above the fuselage, making the default text “Hello World” readable from the right side of the airplane

The go down to the “Text Mesh”. Change the following things:

Anchor to lower center

Change font size to 80

Change the color to red

Now hit “Add component”, and add a Mesh Collider. Select the “Convex” checkbox:

Of course, it’s not very practical if the airplane’s data is only readable from the left side. So to close off this article, we actually going to write code – by adding a simple script that keeps the text readable from every angle. Go to the Assets/App/Scripts folder, right-click it and hit “Create Script”. Call the script “LookAtCamera”. Double-click it – that will open Visual Studio – and change the update method to this:

Save the file, go back at Unity. Drag the new script on top of the Label inside the AircraftHolder game obejct To see this actually works: hit the Unity “Play” button

Now the fun thing is – as long as you are in play mode, changes in settings are not stored so this is great for fooling around. Initially your game screen looks like this – you are looking from under the plane right up to the front landing gear.

Select the AircraftHolder in the hierarchy, then change Z to 500.

The plane flicks forward… and the text is still readable although we are not looking form the left, but the back. Change rotation X to –20 and rotatation Y to –30:

The plane is rotated, we are now looking more or less form the right, but the text is still pointing straight to you, and is readable. Mission accomplished. Not bad for two lines of code!

Exit Play mode. Go to the label once more, and remove the text “Hello World”. The label seems to disappear, but it’s still there, it’s now just empty. It will be filled by code.

And finally…

Select the Prefab folder in Assets/App, and drag the AircraftHolder into it. After you have done it, remove it from the HologramCollection. And save the Scene.

Net result:

The reason why I am making the AircraftHolder in stead of adding components directly to the A320 model – which is perfectly possible – is that by using this way I can easily switch out the actual airplane model, while all logic and additional components are preserved. If you find a different (more pretty) aircraft 3D model this way saves you work.

Feedback, comments and tokens of appreciation

If you spot things that are incorrect, or if you don't understand what I mean, please drop a comment on the offending article and I will help you ASAP. You can e-mail me at joostvanschaik at outlook dot com or contact me via twitter.

If you find the information on this blog useful (and apparently some 600+ people per day do on average) please let me know as well, that encourages me to keep doing this. Or do tell others - that made me an MVP; who knows what more it might bring ;-P

Disclaimer and legal stuff

Although I take great care in providing quality samples, all postings, articles and/or files on this site are provided "AS IS" with no warranties, and confer no rights. The views expressed on this blog are strictly my own and do not necessarily reflect the views of my employer, or anyone else on the planet for that matter.

I usually make original content, sometimes building upon other people's work. Sometimes I explain things that can be found elsewhere because I felt what I read was not clear enough for my limited mind so I explain it the way it finally clicked with me. In all cases I take great pains to be sure to link to people or articles who deserve the credit. If you think I have shortchanged you on the credits please let me know.

Please note, I do not work for Microsoft and while I proudly wear the title of "Microsoft Most Valueable Professional", my opinions, files offered, etc. do not represent, are approved of, endorsed by or paid for by Microsoft. The only power behind it is me and my sometimes runaway passion for parts of Microsoft's technology.