Dotnet by Examplehttp://dotnetbyexample.blogspot.com/<b>Motto: "Talk is cheap, show me the code!" </b><br>
<br>
This blog attempts to be a collection of how-to examples in the Microsoft software stack - things that may take forever to find out, especially for the beginner. I see it as my way to return something to the Microsoft community in exchange for what I learned from it.ennoreply@blogger.com (Joost van Schaik)Mon, 25 Sep 2017 02:38:40 PDTBloggertag:blogger.com,1999:blog-5295746446529817470287125Creating a 3D topographical map in your HoloLens / Windows MR app with the Bing Maps Elevation APIhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/GbCI396-sUI/creating-3d-topographical-map-in-your.htmlHoloLensMappingUnity3DUWPWindows Mixed Realitynoreply@blogger.com (Joost van Schaik)Wed, 02 Aug 2017 03:03:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-8911723110347363840<h2>Intro</h2><p>All right, my friends. It's time for the real deal, the blog post I have been anticipating to write since I first published the 3D version of <a href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ" target="_blank">Walk the World</a> like two months ago. It took me a while to extract the minimal understandable code from my rather convoluted app. Everyone who has ever embarked on a trip of 'exploratory programming' (meaning you have an idea what you want to do, but only some vague clues how) knows how you end up with a repo (and code) full of failed experiments, side tracks, etc. So I had to clean that up a little first. Also, my app does a lot more than just show the map, and those features would obscure the general idea. As a bonus - after creating this blog post I finally actually understand myself how and most importantly <em>why</em> the app works :).</p><p>So, without further ado, I am going to show you how to display a 3D map in your HoloLens or Windows MR headset. Just like in Walk the World. I will build upon <a href="http://dotnetbyexample.blogspot.com/2017/07/create-geographical-map-on-floor-in.html" target="_blank">my previous post, in which I showed you how to make a flat slippy map</a>. This time, we are going 3D.</p><h2>The general idea</h2><p>As you can read in <a href="http://dotnetbyexample.blogspot.com/2017/07/create-geographical-map-on-floor-in.html" target="_blank">the previous post</a>, I 'paste' the actual map tiles - mere images - on a Unity3D Plane. A Plane is a so-called mesh that exists out of a grid of 11x11 points, that form the vertices. If I somehow would be able to ascertain actual elevation on those locations, I can move those points up and down and actually get a 3D map. The tile itself will be stretched up and down.The idea of manipulating the insides of a mesh, which turns out to be very simple, is explained by the awesome <a href="https://twitter.com/rickbarraza" target="_blank">Rick Bazarra</a> in the <a href="https://www.youtube.com/watch?v=a-quaXuEdhU&amp;list=PL64OnOdZ_3NF4iv53e8eG7t1erC1PgKx6" target="_blank">first episode of his must-see "Unity Strikes Back" explanatory video series</a> on YouTube, a follow-up to his <a href="https://channel9.msdn.com/Series/UnityCreativeCoding" target="_blank">Creative Coding with Unity series</a> on Channel 9, that I consider a standard starting point for everyone who wants to get off the ground with Unity3D.</p><p>So where do we get those elevations? Enter the awesome Microsoft service called the <a href="https://msdn.microsoft.com/en-us/library/jj158961.aspx" target="_blank">Bing Maps Elevation API</a>. It seems to be built-to-order for this task. Your first order of business - <a href="https://www.bingmapsportal.com/" target="_blank">get yourself a Basic Bing Maps key</a>. </p><h2>Adding some geo-intelligence to the tile</h2><p>The Bing Maps Elevation API documentation describes an endpoint <a href="https://msdn.microsoft.com/en-us/library/jj158961.aspx" target="_blank">GetElevations</a> that allows you to get altitudes in several ways. One of them is a grid of altitudes in a bounding box. That is what we want - our tiles are square. The documentation says the bounding box should be specified as follows:</p><p><strong>"A bounding box defined as a set of WGS84 latitudes and longitudes in the following order: <br>
south latitude, west longitude, north latitude, east longitude"</strong></p><p>If you envision a tile positioned so that north is up, we are required to calculate the geographical location of the top-right and bottom-left of the tile. The Open Street Maps Wiki provides <a href="http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#C.23" target="_blank">code for the north-west corner of the tile</a>, i.e. top <em>left. </em>I translated the code to C#...</p><pre style="font-size: 11px;">//http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#C.23
private WorldCoordinate GetNorthWestLocation(int tileX, int tileY, int zoomLevel)
{
var p = new WorldCoordinate();
var n = Math.Pow(2.0, zoomLevel);
p.Lon = (float)(tileX / n * 360.0 - 180.0);
var latRad = Math.Atan(Math.Sinh(Math.PI * (1 - 2 * tileY / n)));
p.Lat = (float) (latRad * 180.0 / Math.PI);
return p;
}</pre><p><a href="https://lh3.googleusercontent.com/-VU4tXizvfr4/WYGjbftVWNI/AAAAAAAAPK4/lotZoDHjz7grl7jeIcF5KjqNbCGh96XSACHMYCw/s1600-h/image%255B5%255D"><img width="124" height="122" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-3dLtZb2cRAI/WYGjb3AUXaI/AAAAAAAAPK8/xptSYRko00Q0JTHF-RlDLNLqgNedSnuFQCHMYCw/image_thumb%255B3%255D?imgmax=800" border="0"></a>... and it works fine, but we need the <em>south west</em> and the <em>north east</em> points. Well, if you consider how the tiles are stacked, you can easily see how the north east point is the north-west point of the tile <em>right</em> of our current tile, and the south-west point is the north west point of the tile <em>below</em> our tile. Therefore we can use the north-west points of those adjacent tiles to find the values we actually need - like this:</p><pre style="font-size: 11px;">public WorldCoordinate GetNorthEast()
{
return GetNorthWestLocation(X+1, Y, ZoomLevel);
}
public WorldCoordinate GetSouthWest()
{
return GetNorthWestLocation(X, Y+1, ZoomLevel);
}
</pre><p>That was easy, right?</p><h2>Size matters</h2><p>Although Microsoft is a USA company, it fortunately has an international orientation so the Bing Maps Elevation API returns no yards, feet, inches, miles, furlongs, stadia, or any other <a href="http://www.zmescience.com/other/map-of-countries-officially-not-using-the-metric-system/" target="_blank">deprecated distance unit</a> – it returns plain old meters. Which is very fortunate, as the Windows Mixed Reality distance unit is – oh joy – meters too. But it returns elevation in <em>real world values</em>, and while it might be fun to show Kilimanjaro in real height, it will be a bit too big to fit in my room (or any room, for what matters). Open Street Map is shown at a definite scale per zoom level – and as a GIS guy, I like to be the height correctly scaled - to get for this real life feeling. Once again, referring to the Open Street Map Wiki – there is a nice table that shows <a href="http://wiki.openstreetmap.org/wiki/Zoom_levels" target="_blank">how many meters a pixel is at any given zoom level</a>. We will need the size per <em>tile </em>(which is 256 pixels, as I explained in the previous post), so we add the following code that will give you a scale factor for the available zoom levels for Open Street Map:</p><pre style="font-size: 11px;">//http://wiki.openstreetmap.org/wiki/Zoom_levels
private static readonly float[] _zoomScales =
{
156412f, 78206f, 39103f, 19551f, 9776f, 4888f, 2444f,
1222f, 610.984f, 305.492f, 152.746f, 76.373f, 38.187f,
19.093f, 9.547f, 4.773f, 2.387f, 1.193f, 0.596f, 0.298f
};
private const int MapPixelSize = 256;
public float ScaleFactor
{
get { return _zoomScales[ZoomLevel] * MapPixelSize; }
}</pre><h2>Creating the request</h2><p>Now we move to MapTile. We add the following code to download the Bing Maps Elevation API values</p><pre style="font-size: 10px;">private string _mapToken = "your-map-token-here";
public bool IsDownloading { get; private set; }
private WWW _downloader;
private void StartLoadElevationDataFromWeb()
{
if (_tileData == null)
{
return;
}
var northEast = _tileData.GetNorthEast();
var southWest = _tileData.GetSouthWest();
var urlData = string.Format(
"http://dev.virtualearth.net/REST/v1/Elevation/Bounds?bounds={0},{1},{2},{3}&amp;rows=11&amp;cols=11&amp;key={4}",
southWest.Lat, southWest.Lon, northEast.Lat, northEast.Lon, _mapToken);
_downloader = new WWW(urlData);
IsDownloading = true;
}</pre><p>This simply queries the TileInfo structure for the new methods we have just created. Notice it then builds the URL, containing the bounds, the hard coded 11x11 points that are in a Unity Plane, and the key. Then it calls a piece of Unity3D code called “WWW”&nbsp; which is a sort of HttpClient named by someone with a <em>lot</em> of fantasy (NOT). And that’s it. We add a call to the existing SetTileData method like this:</p><pre style="font-size: 12px;">public void SetTileData(TileInfo tiledata, bool forceReload = false)
{
if (_tileData == null || !_tileData.Equals(tiledata) || forceReload)
{
TileData = tiledata;
<strong><font color="#ff0000">StartLoadElevationDataFromWeb();</font></strong>
}
}</pre><p>so that whenever tile data is supplied, it does not only initiate the downloading of the tile, but also the downloading of the 3D data.</p><h2>Processing the 3D data</h2><p>Next up is a method ProcessElevationDataFromWeb, that is called from Update (so about 60 times a second). In this method we check if a the MapTile is downloading – and if it’s ready, we process the data</p><pre style="font-size: 12px;">protected override void OnUpdate()
{
ProcessElevationDataFromWeb();
}
private void ProcessElevationDataFromWeb()
{
if (TileData == null || _downloader == null)
{
return;
}
if (IsDownloading &amp;&amp; _downloader.isDone)
{
IsDownloading = false;
var elevationData = JsonUtility.FromJson&lt;ElevationResult&gt;(_downloader.text);
if (elevationData == null)
{
return;
}
ApplyElevationData(elevationData);
}
}</pre><p>An ElevationResult is a class to deserialize a result from a call to the Bing Maps Elevation API in. I entered the result of a manual call in <a href="http://json2csharp.com/" target="_blank">Json2CSharp</a> and got a class structure back – only I changed all properties into public fields so the rather <strike>stupid</strike> limited Unity JsonUtility, that does not seem to understand the concept of properties, can handle it. I also initialized lists in the objects from the constructors. It’s not very interesting but if you want a look go <a href="https://github.com/LocalJoost/SlippyMapDemo/blob/3D/Assets/App/Scripts/ElevationResult.cs" target="_blank">here in the demo project.</a><font color="#000000"></font></p><h2>Applying the 3D data.</h2><p>So now it’s time to actually move the mesh points up an down. Mostly using code I stole from Rick Bazarra, with a few adaptions from me:</p><pre style="font-size: 11px;">private void ApplyElevationData(ElevationResult elevationData)
{
var threeDScale = TileData.ScaleFactor;
var resource = elevationData.resourceSets[0].resources[0];
var verts = new List&lt;Vector3&gt;();
var mesh = GetComponent&lt;MeshFilter&gt;().mesh;
for (var i = 0; i &lt; mesh.vertexCount; i++)
{
var newPos = mesh.vertices[i];
newPos.y = resource.elevations[i] / threeDScale;
verts.Add(newPos);
}
RebuildMesh(mesh, verts);
}
private void RebuildMesh(Mesh mesh, List&lt;Vector3&gt; verts)
{
mesh.SetVertices(verts);
mesh.RecalculateNormals();
mesh.RecalculateBounds();
DestroyImmediate(gameObject.GetComponent&lt;MeshCollider&gt;());
var meshCollider = gameObject.AddComponent&lt;MeshCollider&gt;();
meshCollider.sharedMesh = mesh;
}</pre><p>First we get the scale factor – that’s simply the value by which elevation data must be divided to make it match the current zoom level. Next, we get the elevation data itself, that is two levels down in de ElevationData. And then we go modify the elevation of the mesh points to match those of the elevation we got. For some reason - and that's why I said it looks like the Bing Maps Elevation API looks to be like built-to-order for this task - the points come in at exactly the right order for Unity to process in the mesh. </p><p>As I learned from Rick, you cannot modify the points of a mesh, you have to <em>replace</em> them. So we loop through the mesh points and fill a list with points that have their y – so the vertical direction – changed to a scaled value of the elevation. Then we call RebuildMesh, that simply replaces the entire mesh with new vertices, does some recalculation and rebuilds the collider, so your gaze cursor will actually play nice with the new mesh. I also noticed that it you don’t do the recalculate stuff, you will end up looking partly through tiles. I am sure people with a deeper understanding of Unity3D will understand why. I just found out that it needs to be done.</p><p>Don't press play yet! There a few tiny things left to do, to make the result look good. </p><h2>Setting the right location and material</h2><p>First of all, the map is kind of shiny, which was more or less okay-ish for the flat map, but if you turn the map into 3D you will get this over bright effect. So open up the project in Unity, create a new material “MapMaterial” and apply the properties as displayed here below left. The color of the material should be #BABABAFF. See left image. When it is done, drag it on top of the MapTile (see right image). </p><p><a href="https://lh3.googleusercontent.com/-UjGGhFj0LPQ/WYHilfQuPfI/AAAAAAAAPLM/a3nDHuPMbfwj6KbnBcBvuL5fo5BP2iVsQCHMYCw/s1600-h/image%255B17%255D"><img width="300" height="468" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-5R6qkw82gV4/WYHil6PaJtI/AAAAAAAAPLQ/FC2214zEma4QwnyFfiisht-5YrzieXFEACHMYCw/image_thumb%255B11%255D?imgmax=800" border="0"></a><a href="https://lh3.googleusercontent.com/-1JnscMuDaUc/WYK7Y4TzZEI/AAAAAAAAPMg/A9NWh54IT0oNGsFnJ0XqEHBmSLnuQOGlACHMYCw/s1600-h/image12%255B2%255D"><img width="300" height="444" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-HBqL8F7AYrA/WYK7Zfc9qbI/AAAAAAAAPMk/zJJHQfVywGY3Bhfnn-DZAnmiLAHK7UtQACHMYCw/image12_thumb%255B1%255D?imgmax=800" border="0"></a></p><p>Then, the app is still looking at Redmond. While that’s an awesome place, there isn’t much spectacular to see as far as geography is concerned. So we mosey over to the MapBuilder script. There we change the zoom level to 14, the Latitude to 46.78403 and the Longitude to -121.7543</p><p><a href="https://lh3.googleusercontent.com/-OptUWt6NHo4/WYHinQwycxI/AAAAAAAAPLc/GtIFb1prxgAwCOg1FzK4m9vuu0GXSxMKgCHMYCw/s1600-h/image%255B4%255D"><img width="650" height="287" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-CrSmFC-DMwA/WYHin-keZdI/AAAAAAAAPLg/G9o4rQEndsQj6kmaEvi3S6jCYoeIn_UHACHMYCw/image_thumb%255B2%255D?imgmax=800" border="0"></a></p><p>It's a little east and quite a bit more south from Redmond. In fact, when you press play, you will see a landmark that is very familiar if you live anywhere near the Seattle area or visited it:</p><p><a href="https://lh3.googleusercontent.com/-TVd8--50duU/WYH6S9iDJSI/AAAAAAAAPL0/sMyPao0uqdoQ0BC0aB-unbaxYyqyE-A7gCHMYCw/s1600-h/image%255B5%255D"><img width="654" height="547" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-83TUh9Cz-_0/WYH6TdMfDVI/AAAAAAAAPL4/-EBznxT6VSQyF0ma4JY-iIZJJoGHIahBQCHMYCw/image_thumb%255B2%255D?imgmax=800" border="0"></a></p>
<p>famous Mount Rainier, the volcano sitting about 100 km from Seattle, very prominently visible from aircraft - weather permitting. To get this view, I had to fiddle with the view control keys a little after pressing play - if you press play initially you will see Rainier from a lot more close up.</p><p align="left">And that, my friends, is how you make a 3D map in your HoloLens. Of <em>almost</em> <em>any place in the world</em>. Want to see Kilimanjaro? Change Latitude to -3.21508 and Longitude to 37.37316. Press play. </p><p align="left"><a href="https://lh3.googleusercontent.com/-K8LXVA8_F_Y/WYH6UrC7xxI/AAAAAAAAPL8/nUeHreXUtYc5KHx5c9W-BM4m8yp7SLKyACHMYCw/s1600-h/image%255B10%255D"><img width="650" height="544" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-h8BqeMh7_PA/WYH6VrZr9OI/AAAAAAAAPMA/IZe7QxZ9xTsxrYPl8C7vXRZxZUNOoeqsQCHMYCw/image_thumb%255B5%255D?imgmax=800" border="0"></a></p><p>Niagra falls? Latitude 43.07306, Longitude -79.07561 and change zoom level to 17. Rotate the view forward a little with your mouse and pull back. You have to look <em>down</em>. But then, here you go.</p><p><a href="https://lh3.googleusercontent.com/-PRQWQ2TdwE4/WYH6WMBxLCI/AAAAAAAAPME/JN58YHrl5TsrrBGi1d9nUgBYgjAYTBUhgCHMYCw/s1600-h/image%255B15%255D"><img width="650" height="544" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-J6zK11IlCVw/WYH6XMTVlTI/AAAAAAAAPMI/7l1Nbz9FBAk3PvUaALQ3UurlOeU_BT2JwCHMYCw/image_thumb%255B8%255D?imgmax=800" border="0"></a></p><p>GIS is cool, 3D GIS is ultra-cool! All that is left is to generate the UWP app and deploy it into your HoloLens or view it in your new <a href="https://developer.microsoft.com/en-us/windows/mixed-reality" target="_blank">Windows Mixed Reality</a> device.</p><h2>Caveat emptor</h2><p>Awesome right?&nbsp; Now there are a few things to consider. In my <a href="http://dotnetbyexample.blogspot.com/2017/07/create-geographical-map-on-floor-in.html" target="_blank">previous post</a> I said this app was a bandwidth hog, as it downloads 169 tiles <em>per map</em>. In addition, it now also fires 169 requests per map to the Bing Maps Elevation API. For. Every. Single. Map. Every time. Apart from the bandwidth and power consequences, there's another thing to consider. If you go to <a href="https://www.microsoft.com/maps/create-a-bing-maps-key.aspx" target="_blank">this page</a> and click "Basic Key", you will see something like this:</p><p><a href="https://lh3.googleusercontent.com/-1g4q2kb-Hn4/WYH6Xp_R_MI/AAAAAAAAPMM/9vORsj6urU4u3Ptwx23mmtxPxA7gTBuRACHMYCw/s1600-h/image%255B20%255D"><img width="650" height="317" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-JZvno63Mipo/WYH6YDMR1DI/AAAAAAAAPMQ/1DJdspmcWUgy6N6uh9d38zM1MbBH7X2JwCHMYCw/image_thumb%255B11%255D?imgmax=800" border="0"></a></p><p>What is boils down to is - if your app is anywhere near successful, it will eat your allotted request limit <em>very fast</em>, you will get a mail from the Bing Team kindly informing you of this (been there, got that) - and then suddenly you will have a 2D app again. You will have to buy and enterprise key and those are <em>not</em> cheap. So I advise you to do some caching - both in the app and if possible on an Azure service. I employed a Redis cache to that extent.</p><p>Furthermore, I explained I calculate the north east and south west points of a tile using the north west points of the tiles left of and below the current tile. If those tiles are not present, because you are at the edge of the map, I have no idea what will happen - but presumably it won't work as expected. You can run into this when you are zoomed out sufficiently in the very south or east of the map. But then you are either at the <a href="https://en.wikipedia.org/wiki/International_Date_Line" target="_blank">International Date Line</a> (that runs from the North Pole to the South Pole exactly on the side of Earth that is exactly opposite of the <a href="https://en.wikipedia.org/wiki/Prime_meridian_(Greenwich)" target="_blank">Greenwich Meridian</a>) or at Antarctica. On the first spot, there’s mostly ocean (why else do you think they’ve put it there) and thus no (visible) geography to speak of. As far as Antarctica goes, you’ll hit another limitation, for it clearly says in the Bing Maps Elevation API documentation:</p><p><strong>"There is a limitation in that latitude coordinates outside of the range of -85 and 85 are not supported."</strong><p>So beware. Stay away from the Earth's edges. Your app might fall off :).<h2>Some assembly required, batteries not included</h2><p>Indeed, it does not look exactly <a href="https://www.youtube.com/watch?v=elbesPvRUXc&amp;t" target="_blank">like in the videos I showed</a>. Walk the World employs a different map tile sets (plural indeed), and there's also all kinds of other stuff my app does - like sticking the map to the floor so that even at high elevations you have a nice overview, reverse geocoding so you can click on the map to see what's there, tracking the user's moves so it can make a new map appear where the user is walking off it&nbsp; - connecting to the old one, zoom in/out centered on the user's location in the map, showing a map of the physical surroundings... there's a lot of math in there. I only showed you the basics. If you need a HoloDeveloper who knows and understand GIS to the core, you know who to contact now :)<h2>Conclusion</h2><p>Once you know the basics, it's actually pretty easy to create a 3D scaled map of about anywhere in the world, that is - anywhere where the Bing Maps Elevation API is supported. The 3D stuff is actually the <em>easy</em> part - knowing how to <a href="http://dotnetbyexample.blogspot.com/2017/07/create-geographical-map-on-floor-in.html" target="_blank">calculate tiles and build a slippy map</a> is harder. But in the end, it is always easy when you know what to do. Like I said, I think GIS is a premier field in which Mixed Reality will shine. I am ready for it: I hope you are too. Innovate or die - there are certainly companies I know that could take that advice and get moving.<p>Get the <a href="https://github.com/LocalJoost/SlippyMapDemo/tree/3D" target="_blank">demo project</a> and get inspired!<h2>Credits</h2><p>Thanks to <a href="https://twitter.com/rschu" target="_blank">René Schulte</a>, the wise man from 51.050409, 13.737262 ;)&nbsp; for pointing me to the Bing Maps Elevation API. And of course to <a href="https://twitter.com/rickbarraza" target="_blank">Rick Bazarra</a>, who inspired me so often and actually provided some crucial code in his YouTube training video series.<img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/GbCI396-sUI" height="1" width="1" alt=""/>2017-08-03T07:59:20.348+02:001http://dotnetbyexample.blogspot.com/2017/08/creating-3d-topographical-map-in-your.htmlCreating a geographical map on the floor in your Hololens / Windows MR apphttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/rKFaF5sN9uU/create-geographical-map-on-floor-in.htmlHoloLensMappingUWPWindows Mixed Realitynoreply@blogger.com (Joost van Schaik)Sat, 22 Jul 2017 05:22:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-3198924645477729823<h2>Intro</h2><p>If you make an awesome app like <a title="Walk the World" href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ">Walk the World</a>, of course you are not going to spill the beans on how you built it, right? Well – wrong, because I think sharing knowledge is the basis of any successful technical community, because I like doing it, and last but not least – I feel it is one the primary things that makes an MVP an MVP. I can’t show you <em>all</em> the details, if only because the app is humongous and is badly in need of some overhaul / refactoring, but I <em>can</em> show you some basic principles that will allow you to make your own map. And that’s exactly what I am going to do.</p><h2>‘Slippy map’?</h2><p>Getting on my GIS hobby horse here :) Gather around the fire ye youngsters and let this old GIS buff tell you all about it. ;)</p><p>Slippy maps are maps made out of pre rendered square images that together form a continuous looking map. Well-known examples are Bing Maps and Google Maps, as well as the open source map <a href="http://www.openstreetmap.org" target="_blank">Open Street Maps</a> – that we use for this example, The images are usually 256x256 pixels. Slippy maps have a fixed number of discrete zoom levels. For every zoom level there is a number of tiles. Zoom level 0 is typically 1 image showing the whole world. Zoom level 1 is 4 tiles each showing 1/4th of the world. Zoom level 2 is 16 tiles each showing 1/16th of the world. You get the idea. Generally speaking a zoom level has 2<sup>zoomlevel</sup> x 2<sup>zoomlevel</sup> tiles. </p><p>You can see the number of tiles (and the amount of data servers need to store those) go up <em>very</em> quickly. Open Street Maps’ maximum zoom level is 19 – which all by itself is 274.9 <em>billion</em> tiles, and that is <em>on top of all the other levels</em>. Google Map’s maximum zoom level is 21 at some places and I think Bing Maps goes even further. The amount of tiles of level 21 alone would be 4398046511104, a little short of 5 <em>trillion. </em>And that is not all - they even have <em>multiple layers</em> – map, terrain, satellite. Trillions upon trillions of tiles - that is even too much to swallow for Microsoft and Google, which is why they vary the maximum zoom level depending on the location – in the middle of the ocean there’s considerable less zoom levels available ;). But still: you need really insane amounts of storage and bandwidth to serve a slippy map of the whole world with some amount of detail, which is why you have so little parties actually meeting this challenge – mostly Google, Microsoft, and Open Street Maps. </p><p>Anyway - in a slippy map tiles are defined in rows and columns per zoom level. The takeaway from this is to realize that a map tile can be identified by three parameters and three parameters only: X, Y and zoom level. And those need to be converted into a unique URL per tile. The way these tiles are exactly organized depends on the actual map supplier. And if your starting point is a Lat/Lon coordinate, you will have to do some extra math. Basically all you need to know you can find <a href="http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames" target="_blank">here at the Open Streep Maps wiki</a>. But I am going to show in more detail anyway.</p><h2>Setting up the project</h2><p>For this project we will use <a href="https://unity3d.com/unity/whats-new/unity-2017.1.0" target="_blank">Unity 2017.1.0.f3</a>. Do not forget to install the Windows Store (.NET) Target Support as well. If you are finished, clone my basic setup from <a href="http://dotnetbyexample.blogspot.nl/2017/06/setting-up-hololens-project-with.html">Setting up a HoloLens project with the HoloToolkit - June 2017 edition</a>. Rename the folder to “SlippyMapDemo”, then</p><ul><li>
Open the project in Unity
<li>Open the Build Settings window (CTRL+B)<li>Make sure it looks like this:</li></ul><p><a href="https://lh3.googleusercontent.com/-Mg7x9EPv8JI/WWzpWAGveCI/AAAAAAAAPII/KPOlf3L95jgjaVZjucZInwmw6dNu3YQAgCHMYCw/s1600-h/image%255B9%255D"><img width="350" height="337" title="image" style="margin: 0px 0px 0px 21px; border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-oGUwXwwdhGk/WWzpWa6OAvI/AAAAAAAAPIM/5bA9SkufW3UikfGMmffgju33cM2szKQiQCHMYCw/image_thumb%255B7%255D?imgmax=800" border="0"></a>&nbsp; </p><ul><li>Hit the "Player Settings..." button
<li>Change "June2017HoloLensSetup" in "SlippyMapDemo" where ever you see it. Initially you will see only one place, but please expand the "Icon" and "Publishing Settings" panels as well, there are more boxes to fill in.&nbsp; </li></ul><h2>Upgrading the <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a></h2><p><a href="https://lh3.googleusercontent.com/-4YQXSfZBUKw/WWzpWySNyLI/AAAAAAAAPIQ/BP5r8dyBWeQjaz3QvkBGgy9Yk9tzep4EQCHMYCw/s1600-h/image%255B17%255D"><img width="240" height="131" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-igIvWqdmDK8/WWzpXTEihZI/AAAAAAAAPIU/vWmXqZYCFZQyZ2ubsHQUYuT8uk9ySwDAQCHMYCw/image_thumb%255B10%255D?imgmax=800" border="0"></a>To make a working app on the new Unity version, we will need a new HoloToolkit. So delete everything from the Assets/Holotoolkit <em>but do it from Unity, </em>as this will leave the .gitignore in place.</p><p>Select all items, press delete. Then, go to <a title="http://holotoolkit.download/" href="http://holotoolkit.download/">http://holotoolkit.download/</a> and download the latest HoloToolkit. If you click “Open” while downloading, Unity will automatically start to import it. I would once again suggest de-selecting Holotoolkit-Tests as they are not useful for an application project. </p><h2>Some basic stuff first</h2><p>We will need to have a simple class that can hold a Lat/Lon coordinate. UWP supplies those, but that means we cannot test in the editor and, well, we don’t need all what they can do. So we start off with this extremely simple class:</p><pre style="font-size: 11px;">public class WorldCoordinate
{
public float Lon { get; set; }
public float Lat { get; set; }
public override string ToString()
{
return string.Format("lat={0},lon={1}", Lat, Lon);
}
}
</pre><h2>Lat/Lon to tile</h2><p>A map has a location – latitude and longitude, so that is our starting point. So we need to have a way to get the tile on which the desired location is. A tile is defined by X, Y and zoom level, remember? We start like this:</p><pre style="font-size: 11px;">using System;
using UnityEngine;
public class TileInfo : IEquatable&lt;TileInfo&gt;
{
public float MapTileSize { get; private set; }
public TileInfo(WorldCoordinate centerLocation, int zoom, float mapTileSize)
{
SetStandardValues(mapTileSize);
var latrad = centerLocation.Lat * Mathf.Deg2Rad;
var n = Math.Pow(2, zoom);
X = (int)((centerLocation.Lon + 180.0)/360.0*n);
Y = (int)((1.0 - Mathf.Log(Mathf.Tan(latrad) + 1 / Mathf.Cos(latrad)) / Mathf.PI) / 2.0 * n);
ZoomLevel = zoom;
}
private void SetStandardValues(float mapTileSize)
{
MapTileSize = mapTileSize;
}
public int X { get; set; }
public int Y { get; set; }
public int ZoomLevel { get; private set; }
}</pre><p>Via a simple constructor X and Y are calculated from latitude, longitude and zoomlevel, The MapTileSize is the apparent physical size of the map tile - that we will need in the future (it will be 0.5 meters, as we will see later). </p><p>Wow. That seems like some very high brow GIS calculation, that only someone like me understands, right? ;)Maybe, maybe not, but you find this formula on the Open Street Map wiki, more specifically, <a href="http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Tile_numbers_to_lon..2Flat._2" target="_blank">here</a>. </p><p>So now we have the center tile, and to calculate the tiles next to it, we simply need another constructor </p><pre style="font-size: 11px;">public TileInfo(int x, int y, int zoom, float mapTileSize)
{
SetStandardValues(mapTileSize);
X = x;
Y = y;
ZoomLevel = zoom;
}</pre><p>in a simple loop, as we also will see later, we can find the tiles next, above and below it and we can define a MapBuilder class that can loop over this information, and make map tiles <em>grid</em>. BTW, this class also has some equality logic, but that’s not very exiting in this context, so you can look that up in the <a href="https://github.com/LocalJoost/SlippyMapDemo" target="_blank">demo project.</a></p><h2>Tile to URL</h2><p>As I have explained, a tile can be identified by three numbers: X, Y and zoom level but to actually show the tile, you need to convert that to a URL. So I defined this interface to make the tile retrieval map system agnostic: </p><pre style="font-size: 11px;">public interface IMapUrlBuilder
{
string GetTileUrl(TileInfo tileInfo);
}</pre><p>and this single class implementing it for Open Street Map:</p><pre style="font-size: 11px;">using UnityEngine;
public class OpenStreetMapTileBuilder : IMapUrlBuilder
{
private static readonly string[] TilePathPrefixes = { "a", "b", "c" };
public string GetTileUrl(TileInfo tileInfo)
{
return string.Format("http://{0}.tile.openstreetmap.org/{1}/{2}/{3}.png",
TilePathPrefixes[Mathf.Abs(tileInfo.X) % 3],
tileInfo.ZoomLevel, tileInfo.X, tileInfo.Y);
}
}</pre><p>this is not new code. Regular readers (or better – long time readers) of this blog <a href="http://dotnetbyexample.blogspot.com/2010/10/showing-open-source-maps-on-windows.html" target="_blank">may have seen it as early as 2010, when I showed how to do this for Window Phone 7</a>. </p><h2>Some (very little) re-use</h2><p>Basically we are going to download images from the web again, using the URL that the IMapUrlBuilder can calculate from the TileData. Downloading an showing images in a HoloLens app – been there, done that, and it fact, that’s what made the idea of Walk the World popup up in my mind in the first place. So I am going to reuse the <a href="http://dotnetbyexample.blogspot.com/2017/02/a-behaviour-for-dynamically-loading-and.html" target="_blank">DynamicTextureDownloader from this post</a>. In the <a href="https://github.com/LocalJoost/SlippyMapDemo" target="_blank">demo project</a>, it sits in the HolotoolkitExtensions. We make a simple child class:</p><pre style="font-size: 11px;">using HoloToolkitExtensions.RemoteAssets;
using System.Collections;
using UnityEngine;
public class MapTile : DynamicTextureDownloader
{
public IMapUrlBuilder MapBuilder { get; set; }
private TileInfo _tileData;
public MapTile()
{
MapBuilder = MapBuilder != null ? MapBuilder : new OpenStreetMapTileBuilder();
}
public void SetTileData(TileInfo tiledata, bool forceReload = false)
{
if (_tileData == null || !_tileData.Equals(tiledata) || forceReload)
{
TileData = tiledata;
}
}
public TileInfo TileData
{
get { return _tileData; }
private set
{
_tileData = value;
ImageUrl = MapBuilder.GetTileUrl(_tileData);
}
}
}
</pre><p>So what happens – if you set TileData using SetTileInfo, it will ask the MapBuilder to calculate the tile image URL, the result will be assigned to the parent class’ ImageUrl property, and the image will automatically be drawn as a texture on the Plane it’s supposed to be added to.</p><h2>Creating the MapTile prefab</h2><p>In HologramCollection, create a Plane and call it MapTile. Change it’s X and Z scale to 0.05 so the 10x10 meter plane will show as 0.5 meters indeed. Then add the MapTile <em>script</em> to it as a component. Finally, drag the MapTile <em>Plane </em>(with attached script) from the HologramCollection to the Prefabs folder in Assets.</p><p><a href="https://lh3.googleusercontent.com/-rKYvHZ0u2RU/WXMbAIbHpXI/AAAAAAAAPJA/q_uQuCxwO0UVBhDP1_51Jps8W-EfNFV4ACHMYCw/s1600-h/image%255B6%255D"><img width="600" height="362" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-Fp7awVSM6kM/WXMbA85kcLI/AAAAAAAAPJE/97kGiJNtCqwALQSrV8fv-XBMOTSA-s2yACHMYCw/image_thumb%255B3%255D?imgmax=800" border="0"></a></p><p>If you are done, remove the MapTile from the HologramCollection in the Hierarchy.</p><h2>A first test</h2><p>To the HologramCollection we add another empty element called “Map”. Set it’s Y position to –1.5, so the map will appear well below our viewpoint. To that Map game object that we add a first version of our MapBuilder script:</p><pre style="font-size: 11px;">using UnityEngine;
public class MapBuilder : MonoBehaviour
{
public int ZoomLevel = 12;
public float MapTileSize = 0.5f;
public float Latitude = 47.642567f;
public float Longitude = -122.136919f;
public GameObject MapTilePrefab;
void Start()
{
ShowMap();
}
public void ShowMap()
{
var mapTile = Instantiate(MapTilePrefab, transform);
var tile = mapTile.GetComponent&lt;MapTile&gt;();
tile.SetTileData(
new TileInfo(
new WorldCoordinate{ Lat = Latitude, Lon = Longitude },
ZoomLevel, MapTileSize)
);
}
}</pre><p>This simple script basically creates one tile based upon the information you have provided. Since it does nothing with location, it will appear at 0,0,0 in the Map, which itself is at 1.5 meters below your viewpoint. If you were to run the result in the HoloLens, a 0.5x0.5m map tile map should appear right around your feet.</p><p>Anyway, drag the prefab we created in the previous step on the Map Tile Prefab property of the Map Tile Builder …</p><p><a href="https://lh3.googleusercontent.com/-VzDRdfw4CG8/WXMlnY04gTI/AAAAAAAAPJU/FxSftS0Sj6Mc_AkJWZw8FGAMpE093KP9gCHMYCw/s1600-h/image%255B14%255D"><img width="600" height="360" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-XOGjK5sbanQ/WXMln-twxmI/AAAAAAAAPJY/h7HDi43sdzo_0rTmnu_ejngoNlH7uCGtwCHMYCw/image_thumb%255B7%255D?imgmax=800" border="0"></a></p><p>… and press the Unity Play button.You will see nothing at all, but if you rotate the Hololens Camera 90 degrees over X (basically looking down) <em>while being in play mode</em> a map tile will appear, showing Redmond.</p><p><a href="https://lh3.googleusercontent.com/-cJ_3EAyXtaY/WXMlojqbGGI/AAAAAAAAPJc/7IMY8Wu5yBk6QAA_cBKsYsHA2kXxvTJiQCHMYCw/s1600-h/image%255B24%255D"><img width="650" height="401" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-rQobz51hp6g/WXMlpHgrYkI/AAAAAAAAPJg/xbzZv9mWhEcjtXyamIfEnVg7IoovN5O-ACHMYCw/image_thumb%255B13%255D?imgmax=800" border="0"></a></p><p>Only it will be upside down, thanks to the default way Unity handles Planes - something I still don't understand the reason for. <em>Exit play mode</em>, select the Map game object and change it’s Y rotation to 180, hit play mode again and rotate the camera once again 90 over X.</p><p><a href="https://lh3.googleusercontent.com/-dfchj8uDJT0/WXMlp4M8bCI/AAAAAAAAPJk/Uez7nyBYSRYcc1lUutkSXaR_gZ8dou-2wCHMYCw/s1600-h/image%255B31%255D"><img width="340" height="183" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-xjWlLI6OmT4/WXMlqTcYEEI/AAAAAAAAPJo/ZXfMX1o4apU81uCp_-nud7mbNsb1v0gPgCHMYCw/image_thumb%255B18%255D?imgmax=800" border="0"></a></p><p>That’s more like it.</p><h2>The final step</h2><p>Yes, there is only one step left. In stead of making one tile, let’s make a <em>grid</em> of tiles.</p><p>We add three more properties to the MapBuilder script:</p><pre style="font-size: 11px;">public float MapSize = 12;
private TileInfo _centerTile;
private List&lt;MapTile&gt; _mapTiles;</pre><p>And then here’s the body of MapBuilder v2</p><pre style="font-size: 11px;">void Start()
{
_mapTiles = new List&lt;MapTile&gt;();
ShowMap();
}
public void ShowMap()
{
_centerTile = new TileInfo(new WorldCoordinate { Lat = Latitude, Lon = Longitude },
ZoomLevel, MapTileSize);
LoadTiles();
}
private void LoadTiles(bool forceReload = false)
{
var size = (int)(MapSize / 2);
var tileIndex = 0;
for (var x = -size; x &lt;= size; x++)
{
for (var y = -size; y &lt;= size; y++)
{
var tile = GetOrCreateTile(x, y, tileIndex++);
tile.SetTileData(<br> new TileInfo(_centerTile.X - x, _centerTile.Y + y, ZoomLevel, MapTileSize),
forceReload);
tile.gameObject.name = string.Format("({0},{1}) - {2},{3}", x, y, tile.TileData.X,
tile.TileData.Y);
}
}
}
private MapTile GetOrCreateTile(int x, int y, int i)
{
if (_mapTiles.Any() &amp;&amp; _mapTiles.Count &gt; i)
{
return _mapTiles[i];
}
var mapTile = Instantiate(MapTilePrefab, transform);
mapTile.transform.localPosition = new Vector3(MapTileSize * x - MapTileSize / 2, 0,
MapTileSize * y + MapTileSize / 2);
mapTile.transform.localRotation = Quaternion.identity;
var tile = mapTile.GetComponent&lt;MapTile&gt;();
_mapTiles.Add(tile);
return tile;
}</pre><p>In ShowMap we first calculate the center tile’s data, and then in LoadTiles we simply loop over a square matrix from –(MapSize/ 2) to +(MapSize/2) and yes indeed, if you define a MapSize of 12 you will actually get a map of 13x13 tiles because there has to be a center tile. If there is a center tile, the resulting matrix by definition has an uneven number of tiles.</p><p>In LoadTiles you see TileInfo’s second constructor in action: just X, Y, and Zoomlevel. Once you have the first center tile, calculating adjacent tiles is extremely easy. GetOrCreateTiles does the actual instantiation and <em>positioning in space</em> of the tiles – that’s what we need the MapTileSize for. Note, that for an extra performance gain (and a prevention of memory leaks), it actually keeps the instantiated game objects in memory once they are created, so if you call ShowMap from code after you have changed one of the MapBuilder’s parameters, it will re-use the existing tiles in stead of generating new ones. LoadTiles itself also creates a name for the MapTile but that’s only for debugging / educational purposes – that way you can see which tiles are actually downloaded in the Hierachy while being in Unity Play Mode.</p><p>If you deploy this into a HoloLens and look down you will see a map of 6.5x6.5 meters flowing over the floor. </p><p><a href="https://lh3.googleusercontent.com/-cuWEX8R_tqE/WXNDjM1H09I/AAAAAAAAPJ4/GhOC8NulghwTp3wjcPZYWR4AID5eEqS-gCHMYCw/s1600-h/image%255B38%255D"><img width="504" height="400" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-icjiCGkISSg/WXNDjgemUDI/AAAAAAAAPJ8/3eaholJ2YDoqVQIHYzvro15aEjuCxNfuwCHMYCw/image_thumb%255B23%255D?imgmax=800" border="0"></a></p><p>To make this completely visible I had to move the camera up over 20 meters ;) but you get the drift. </p><h2>Some words of warning</h2><ul><li>The formula in TileInfo calculates a tile from Latitude and Longitude. That only guarantees that location will be on that tile but it doesn’t say anything about <em>where on the tile it is</em>. You can see Redmond on the center tile, but it’s not quite in the center of that center tile. It may have well been op the top left. The more you zoom out, the more this will be a factor.</li><li>This sample shows Open Street Map and Open Street Map only. You will need to build IMapUrlBuilder implementations yourself if you want to use other map providers. Regular readers of this blog know this very easy to do. But please be aware of the TOS of map providers.</li><li>Be aware that the MapBuilder with map setting of 12 downloads 13x13=169 tiles from the internet. <em>On every call. </em>This app is quite the bandwidth hog - and probably a power hog as well.</li></ul><h2>Conclusion and some final thoughts</h2><p>Building basic maps in Unity for use in your HoloLens / Windows Mixed Reality apps is actually pretty easy. I will admit I don’t understand all the fine details of the math either, but I do know how slippy maps are supposed to work, and once I had translated the formula to C#, the rest was actually not that hard, as I hope to have shown.</p><p>Almost all data somehow has a relation with a location, and being able to generate a birds’ eye environment in which you can collaborate with you peers, will greatly enhance productivity and correct interpretation of data. Especially if you add 3D geography and/or buildings to it. It looks like reality, it shows you what’s going on, and you less and less have to interpret 2D data into 3D data. We are 3D creatures, and it’s high time our data jumps off the screen to join us in 3D space. </p><p>I personally think GIS and geo-apps are one of the premier fields in which AR/VR/MR will shine – and this will kick off an awesome revolution in the GIS world IMHO. Watch this space. More to come!</p><p>Demo project <a href="https://github.com/LocalJoost/SlippyMapDemo" target="_blank">here</a>. Enjoy!</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/rKFaF5sN9uU" height="1" width="1" alt=""/>2017-08-02T11:29:28.923+02:002http://dotnetbyexample.blogspot.com/2017/07/create-geographical-map-on-floor-in.htmlStyles in Xamarin Forms don't work properly in UWP .NET Native - here is how to fix ithttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/R7DVGVb4EFk/styles-in-xamarin-forms-dont-work.htmlUWPXamarin Formsnoreply@blogger.com (Joost van Schaik)Sat, 15 Jul 2017 04:15:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-131842386141738117<h2>Intro</h2><p>Xamarin Forms is awesome. If you have learned XAML from WPF, Silverlight, Windows Phone, Universal Windows Apps or UWP, you can jump right in using the XAML you know (or at least something that looks remarkably familiar) and start to make apps that will run cross platform on iOS, Android and UWP. So potentially your app cannot only run on phones but also on XBox, HoloLens and PCs. </p><h2>OnPlatform FTW!</h2><p>One of the coolest thing is the OnPlatform construct. For instance, you can have something this:</p><pre style="font-size: 11px;">&lt;Style TargetType="Label" x:Key="OtherTextStyle" &gt;
&lt;Setter Property="FontSize"&gt;
&lt;OnPlatform x:Key="FontSize" x:TypeArguments="x:Double" &gt;
&lt;On Platform="Windows" Value="100"&gt;&lt;/On&gt;
&lt;On Platform="Android" Value="30"&gt;&lt;/On&gt;
&lt;On Platform="iOS" Value="30"&gt;&lt;/On&gt;
&lt;/OnPlatform&gt;
&lt;/Setter&gt;
&lt;/Style</pre><p>This indicates the label that has this style applied to it, should have a font size of 100 on Windows, 30 on Android, and 30 on iOS. In the <a href="https://github.com/LocalJoost/UWPStyleIssue" target="_blank">demo project</a> I have defined some styles in the App.xaml, and the net result is that is looks like this on Android (left), iOS(right) and Windows (below).</p><p><a href="https://lh3.googleusercontent.com/-D6yV-_ow35o/WWn5FtfwAmI/AAAAAAAAPHE/rdZ_V2YQj-w0puGeSwx9tsv9MFhJ-0MVgCHMYCw/s1600-h/image%255B8%255D"><img width="200" height="333" title="image" style="margin: 0px 0px 0px 39px; border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-3LgkRegYy8w/WWn5GBDCI2I/AAAAAAAAPHI/WKXI_MuKRig5COHc-ktatwtKB6Okq4RGQCHMYCw/image_thumb%255B4%255D?imgmax=800" border="0"></a><a href="https://lh3.googleusercontent.com/-PMf2_fiP1dc/WWn5Ga5QcQI/AAAAAAAAPHM/aM3WGVCCr6wdYZxHMVsbfdFgZ0oTP7U8gCHMYCw/s1600-h/image%255B11%255D"><img width="182" height="333" title="image" align="left" style="border: 0px currentcolor; border-image: none; float: left; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-uf2mCjBtFhk/WWn5G45cRaI/AAAAAAAAPHQ/4RZGo7kjg-QSmJJ3UoF07lYltAaXb3hvwCHMYCw/image_thumb%255B7%255D?imgmax=800" border="0"></a></p><p><a href="https://lh3.googleusercontent.com/-xV2YtWnI_20/WWn5IkqJpUI/AAAAAAAAPHU/53hc68P4J9YwDyYPIrwYXAEyfuRU2nYhgCHMYCw/s1600-h/image%255B63%255D"><img width="442" height="287" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-BYvi7StPscg/WWn5JJ5O6yI/AAAAAAAAPHY/k_u-WJacCtkoS-PrXGWYhvL2Bvc090SxwCHMYCw/image_thumb%255B55%255D?imgmax=800" border="0"></a></p><p>The result is not necessarily very beautiful, but if you look in the MainPage.xaml you will see everything has a style and no values are hard coded. You can also see that although the Android and iOS apps are mobile apps and the Windows app is essentially an app running on a tablet or a PC (the demarcation line between these is becoming hazier with the day) it will still work out using OnPlatform.</p><p> I have used various constructs. Apart from the inline construct as I showed above, there's also this one</p><pre style="font-size: 11px;">&lt;OnPlatform x:Key="ImageSize" x:TypeArguments="x:Double" &gt;
&lt;On Platform="Windows" Value="150"&gt;&lt;/On&gt;
&lt;On Platform="Android" Value="100"&gt;&lt;/On&gt;
&lt;On Platform="iOS" Value="90"&gt;&lt;/On&gt;
&lt;/OnPlatform&gt;
&lt;Style TargetType="Image" x:Key="ImageStyle" &gt;
&lt;Setter Property="HeightRequest" Value="{StaticResource ImageSize}" /&gt;
&lt;Setter Property="WidthRequest" Value="{StaticResource ImageSize}" /&gt;
&lt;Setter Property="VerticalOptions" Value="Center" /&gt;
&lt;Setter Property="HorizontalOptions" Value="Center" /&gt;
&lt;/Style</pre><p>A construct I would very much recommend, as it enables you to re-use the ImageSize value for other things, for instance the height of button, in another style. You can also use these doubles directly in Xaml, like I did with SomeOtherTextFontSize in the last label in MainPage.xaml</p><pre style="font-size: 11px;">&lt;?xml version="1.0" encoding="utf-8" ?&gt;
&lt;ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
x:Class="UWPStyleIssue.MainPage"&gt;
&lt;Grid VerticalOptions="Center" HorizontalOptions="Center" &gt;
&lt;Grid.RowDefinitions&gt;
&lt;RowDefinition Height="*"&gt;&lt;/RowDefinition&gt;
&lt;RowDefinition Height="*"&gt;&lt;/RowDefinition&gt;
&lt;RowDefinition Height="*"&gt;&lt;/RowDefinition&gt;
&lt;RowDefinition Height="*"&gt;&lt;/RowDefinition&gt;
&lt;/Grid.RowDefinitions&gt;
&lt;Image Grid.Row="0"
Source=
"https://media.licdn.com/mpr/mpr/shrinknp_400_400/[abbreviated]jpg"
Style="{StaticResource ImageStyle}"&gt;&lt;/Image&gt;
&lt;Label Text="Welcome to Xamarin Forms!" Grid.Row="1"<br> Style="{StaticResource TextStyle}"/&gt;
&lt;Label Text="Yet another line" Grid.Row="2" Style="{StaticResource OtherTextStyle}"/&gt;
&lt;Label Text="Last Line" Grid.Row="3" <font color="#ff0000"><strong>FontSize="{StaticResource SomeOtherTextFontSize}"</strong></font>/&gt;
&lt;/Grid&gt;
&lt;/ContentPage<br></pre><p>
Although I do not recommend this practice - styles are much cleaner - sometimes needs must and this can be handy. <p>I can hear you think by now: "your point please, kind sir?" (or most likely something less friendly). Well... it works great on Android, as you have seen. It also works great on iOS. And yes, on Windows too...<p><h2>OnPlatform WTF?</h2><p>... until you think "let's get this puppy into the Windows Store". As every Windows Developer knows, if you compile for the Store, you compile for Release, which kicks off the .NET Native toolchain. This is very easy to spot as the compilation process takes <em>much</em> longer. The result is not Intermediary Language (IL),&nbsp; but binary code - an exe - which makes UWP apps so much faster than their predecessors. Unfortunately, it also means the release build is an entirely different beast than a debug build, which can have some unexpected side effects. In our application, if you run the Release build, you will end up with this.</p><p><a href="https://lh3.googleusercontent.com/-lc1Og0bbJVQ/WWn5Jb9zrJI/AAAAAAAAPHc/CWd1SualVy8lBpnTljdA2US3NovJMQ8AACHMYCw/s1600-h/image%255B67%255D"><img width="446" height="291" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-DoH3ubhWx78/WWn5J-krFaI/AAAAAAAAPHg/txNxXQUR7v8FdNUKK4zoK0jJ8ZSNRbu7QCHMYCw/image_thumb%255B57%255D?imgmax=800" border="0"></a></p><p>That is quite some 'side effect'. No margin to pull the first text up, no font size (just default), no image... WTF indeed.</p><h2>Analysis</h2><p>Unfortunately I had some issues with another library (<a href="https://github.com/luberda-molinet/FFImageLoading" target="_blank">FFImageLoading</a>) which took me on the wrong track for quite a while, but after <a href="https://github.com/luberda-molinet/FFImageLoading/pull/651" target="_blank">I had fixed that</a> I noticed that when I changed the styles from Onplatform to hard coded values the styling started to work again - even in .NET Native. So if I did this</p><pre style="font-size: 11px;"><strong><font color="#ff0000">&lt;x:Double x:Key="ImageSize"&gt;150&lt;/x:Double&gt;</font></strong>
&lt;!--&lt;OnPlatform x:Key="ImageSize" x:TypeArguments="x:Double" &gt;
&lt;On Platform="Windows" Value="150"&gt;&lt;/On&gt;
&lt;On Platform="Android" Value="100"&gt;&lt;/On&gt;
&lt;On Platform="iOS" Value="90"&gt;&lt;/On&gt;
&lt;/OnPlatform&gt;--&gt;</pre><p>at least my image showed up again:<p><a href="https://lh3.googleusercontent.com/-jPquFIYlEdE/WWn5KIhYUXI/AAAAAAAAPHk/yXtKgY2EJiAqGV2tCnYubj_2O54InsDKQCHMYCw/s1600-h/image%255B78%255D"><img width="442" height="287" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-UQthNUzfDYA/WWn5KjjfcUI/AAAAAAAAPHo/Uirg6sWQLr0iT5wrnWXX9uxb81Gua6QggCHMYCw/image_thumb%255B64%255D?imgmax=800" border="0"></a></p><p>With a deadline looming and an ginormous style sheet in my app I really had no time to make a branch with separate styles for Windows. We had to go to the store and we had to go <em>now</em>. Time for a cunning plan. I came up with this:</p><h2>A solution/workaround/hack/fix ... sort of</h2><p>So it works when the styles do contain <em>direct</em> values, <em>not</em> OnPlatform, right... ? If you look at App.xaml.cs in the portable project you will see a line in the constructor that's usually not there, and it's commented out</p><pre style="font-size: 11px;">public App()
{
InitializeComponent();
<strong><font color="#ff0000">//this.FixUWPStyling();</font></strong>
MainPage = new UWPStyleIssue.MainPage();
}</pre><p>If you remove the slashes and run the app again in Release....</p><p><a href="https://lh3.googleusercontent.com/-iGCcPHn6Xnw/WWn5LJYPIlI/AAAAAAAAPHs/9yplkHy0U7gKvoqh6fEGNQcTX_C10jQkQCHMYCw/s1600-h/image%255B71%255D"><img width="446" height="291" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-GKwqBN5KUG8/WWn5LZnerAI/AAAAAAAAPHw/nsAGh0WvJFsg1RoJADf0uUHcvLmQS_q-wCHMYCw/image_thumb%255B59%255D?imgmax=800" border="0"></a></p><p>magic happens. All styles seem to work again. This is because of an extension method that's in the file ApplicationExtensions, that you will find in the Portable project in de Extensions folder</p><pre style="font-size: 11px;">public static void FixUWPStyling(this Application app)
{
if (Device.RuntimePlatform == Device.Windows)
{
app.ConvertAllOnPlatformToExplict();
app.ConvertAllOnDoubleToPlainDouble();
}
}</pre><p>The first method, ConvertAllOnPlatformToExplict, does the following:</p><ul><li>Loop trough all the styles</li><li>Loop through all the setters in a style</li><li>Check if the setters 's property name is either "HeightRequest", "WidthRequest", or "FontSize"</li><li>If so, extract the Windows value from the OnPlatform struct</li><li>Set the setter's value to a plain double with as value the extracted Windows value</li></ul><p>It's crude, it requires about everything to be in OnPlatform, but it does the trick. I am not going to write it all out here, it's not very great code, and you can see it all <a href="https://github.com/LocalJoost/UWPStyleIssue/blob/master/UWPStyleIssue/UWPStyleIssue/Extensions/ApplicationExtensions.cs" target="_blank">on GitHub</a> anyway.</p><p>Then, for good measure it calls ConvertAllOnDoubleToPlainDouble, which loops trough the all the doubles, like</p><pre style="font-size: 11px;">&lt;OnPlatform x:Key="ImageSize" x:TypeArguments="x:Double" &gt;...&lt;/OnPlatform&gt;
</pre><p>It extracts the Windows value, removes the OnPlatform from the resource dictionary and adds a new plain double with the Windows style only to the resource dictionary. For some reason, replacement is not possible.</p><h2>Conclusion</h2><p>There is apparently a bug in the Xamarin Forms .NET Native UWP tooling, which causes OnPlatform values being totally ignored. With my dirty little trick, you can at least get your styles to work without having to rewrite the whole shebang for Windows or have a separate style file for it. Note this does not fix <em>everything</em>, if you have other value types (like GridHeights) you will need to add your own conversion to ConvertAllOnPlatformToExplict&nbsp; What I have given you was enough to fix my problems, but not all potential issues that may arise from this bug. </p><p>I hope this drives the Xamarin for UWP adoption forward, and I also hopes this helps the good folks in Redmond fix the bug. I've pretty much identified <em>what</em> goes wrong, now they 'only' have to take are of the <em>how</em> ;)</p><p>Demo project with fix can be found <a href="https://github.com/LocalJoost/UWPStyleIssue" target="_blank">here</a>.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/R7DVGVb4EFk" height="1" width="1" alt=""/>2017-07-17T09:55:05.227+02:000http://dotnetbyexample.blogspot.com/2017/07/styles-in-xamarin-forms-dont-work.htmlBuilding a dynamic floating clickable menu for HoloLens/Windows MRhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/9ceRvs3B9qE/building-dynamic-floating-clickable.htmlHoloLensUnity3DUWPWindows Mixed Realitynoreply@blogger.com (Joost van Schaik)Wed, 12 Jul 2017 04:44:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-4560823261355412318<h2>Intro</h2><p>In the various XAML-based platforms (WPF, UWP, Xamarin) that were created by or are now part of Microsoft we have the great capability to perform databinding and templating - essentially saying to for instance a list 'this is my data, this is how a single item should look, good luck with it' and the UI kind of creates itself. This we don't quite have in Unity projects for <a href="https://developer.microsoft.com/en-us/windows/mixed-reality" target="_blank">Windows Mixed Reality</a>. But still I gave it my best shot when I created a dynamic floating menu for my app <a href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ" target="_blank">Walk the World</a> (only the first few seconds are relevant, the rest is just showing off a view of Machu Picchu)</p><p><iframe width="650" height="365" src="https://www.youtube.com/embed/5jkn9aEIB8Y" frameborder="0" allowfullscreen=""></iframe></p><h2>Starting point</h2><p>We actually start <a href="https://dotnetbyexample.blogspot.com/2017/07/building-floating-hololens-info-screen-2.html" target="_blank">using the end result of my previous post</a>, as I don't really like to do things twice. So copy that project to another folder, or make a branch, whatever. I called the renamed folder FloatingDynamicMenuDemo. Then proceed as follows:</p><ul><li>Delete FloatingScreenDemo.* from the project's root</li><li>Empty the App sub folder - just leave the .gitignore</li><li>Open the project in Unity</li><li>Open the Build Settings window (CTRL+B) </li><li>Hit the "Player Settings..." button</li><li>Change "FloatingScreenDemo" in "FloatingDynamicMenuDemo" whereever you see it. Initially you will see only one place, but please expand the "Icon" and "Publishing Settings" panels as well, there are more boxes to fill in.</li><li>Rename the HelpHolder to MenuHolder</li><li>Remove the Help Text Controller from the HelpHolder. </li><li>Change the text in the 3DTextPrefab from the Lorum ipsum to "Select a place too see" </li><li>Change the text's Y position from 0.84 to 0.23 so it will end up at the top of the 'screen' </li></ul><p>So now we have a workspace with most of the stuff we need already in it. Time to fill in the gaps.</p><h2>Building a Menu Item part 1 - graphics</h2><p><a href="https://lh3.googleusercontent.com/-KtpyrlwB9m4/WV5A_34LVCI/AAAAAAAAPDE/N5gZOtDhg5YWhBM1eTCxzA6a9D0Q3e4xQCHMYCw/s1600-h/image%255B4%255D"><img width="300" height="377" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-yN1_rYfBnXI/WV5BAWsDdAI/AAAAAAAAPDI/nAJk5AXd_cwVT6fsXVdXFNUpBXOf40tbgCHMYCw/image_thumb%255B2%255D?imgmax=800" border="0"></a>So, think templating. We first need to have a template before we can instantiate it. But the only thing I can instantiate are <em>game objects</em>. So... we need to make one... a combination of graphics and code. That sounds like - a prefab indeed!</p><p>First, we will make a material for the menu items, as this will be easier for debugging. Go to the App/Materials folder, find HelpScreenMaterial, hit CTRL-D, and rename HelpScreenMaterial 1 to MenuItemMaterial. Then, change its color to a kind of green, for instance 00B476FF. Also, change the rendering mode to "Opaque". This is so we can easily see the plane. </p><p>Inside the MenuHolder we make a new empty game object. I called it - d'oh - MenuItem. Inside that MenuItem, we first make a 3DTextPrefab, then a Plane. The plane will be of course humongous again, and very white. So first drag the green MenuItemMaterial on it. Then change it's X Rotation to 270 so it will be upright again. Then you have to experiment a little with the X and Z scale until it is more or less the same width as your blue Plane, and a little over 1 line of text height, as showed to the left.The values I got were X = 0.065 and Z = 0.004 but this depends of course on the font size you take. Make sure there is some extra padding between the left and right edges of the green Plane and the blue Plane.</p><p><a href="https://lh3.googleusercontent.com/-NZpmioW6l-o/WV5BA7FmjbI/AAAAAAAAPDM/ZaC3Q8Rb9cAyuhV3cET4Lr067Dr6qqTgACHMYCw/s1600-h/image%255B22%255D"><img width="300" height="185" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-PbkwsV0lZm0/WV5BBL5FeeI/AAAAAAAAPDQ/rjhDTAMvRPgCa8LPazPf52a20m65umiLgCHMYCw/image_thumb%255B14%255D?imgmax=800" border="0"></a>As you can see in the top panel, the text and the menu pane are invisible - they are only visible when looked upon dead right from the camera in the game view. This is because they basically are at the same distance as the screen. So we need to set -0.02 to the Z of the green Plane - so it appears in front of the blue screen - and -0.04 to the Z of the 3DTextPrefab so it will appear in front of the green Plane, and you will see the effect in the Scene pane as well now. </p><p>Since this is a Menu, we want the text to appear from the left. The Anchor is now middle center and it's Alignment Center, and that is not desirable. So we have to set Alignment to Left and Anchor to Middle Left, and then we drag the text prefab to the left till the edge of the green plane. I found an X position value of -0.32.</p><p>Now create a folder "Prefabs" in your App folder in the Assets pane, and drag the MenuItem object from there. This will create a Prefab. The text MenuItem in the Hierarchy will turn blue. </p><p><a href="https://lh3.googleusercontent.com/-38nySy7Em2E/WV5BBiyTD1I/AAAAAAAAPDU/q_AjUeKxGQYd8FOXx7QWjt8z_hd3JjWwwCHMYCw/s1600-h/image%255B63%255D"><img width="336" height="438" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-HIMaYyLV8LI/WV5BBzSaS4I/AAAAAAAAPDY/0RnjXZGOpE8SMtcpAtz-DTaEnqs79RETgCHMYCw/image_thumb%255B51%255D?imgmax=800" border="0"></a></p><p>You can now safely delete the MenuItem from the Hierarchy. Mind you, the Hierarchy. Make sure it stays in Prefabs.</p><h2>Building a Menu Item part 2 - code</h2><p>Our 'menu' needs some general data structure helper. So we start with an interface for that:</p><pre style="font-size: 11px;">public interface IMenuItemData
{
object SelectMessageObject { get; set; }
string Title { get; set; }
int MenuId { get; set; }
}</pre><p>And a default implementation:</p><pre style="font-size: 11px;">public class MenuItemData : IMenuItemData
{
public object SelectMessageObject { get; set; }
public string Title { get; set; }
public int MenuId { get; set; }
}
</pre><p>The SelectedMessageObject is the payload - the actual data. The Title contains the text we want to have displayed on the menu, and the MenuId we need so we can distinguish select events coming from multiple menus, should your application have such. For the distribution of events we once again use the <a href="https://dotnetbyexample.blogspot.com/2017/04/using-messenger-to-communicate-between.html" target="_blank">Messenger</a> that I introduced before (and have used extensively ever since).</p><p>To send a selected object around we need a message class:</p><pre style="font-size: 11px;">public class MenuSelectedMessage
{
public IMenuItemData MenuItem { get; set; }
}</pre><p>And then we only need to add this simple MenuItemController, a behaviour that handles when the MenuItem is tapped:</p><pre style="font-size: 11px;">using HoloToolkit.Unity.InputModule;
using HoloToolkitExtensions.Messaging;
using UnityEngine;
public class MenuItemController : MonoBehaviour, IInputClickHandler
{
private TextMesh _textMesh;
private IMenuItemData _menuItemData;
public IMenuItemData MenuItemData
{
get { return _menuItemData; }
set
{
if (_menuItemData == value)
{
return;
}
_menuItemData = value;
_textMesh = GetComponentInChildren&lt;TextMesh&gt;();
if (_menuItemData != null &amp;&amp; _textMesh != null)
{
_textMesh.text = _menuItemData.Title;
}
}
}
public void OnInputClicked(InputClickedEventData eventData)
{
if (MenuItemData != null)
{
Messenger.Instance.Broadcast(<br> new MenuSelectedMessage {MenuItem = MenuItemData});
PlayConfirmationSound();
}
}
private AudioSource _audioSource;
private void PlayConfirmationSound()
{
if (_audioSource == null)
{
_audioSource = GetComponent&lt;AudioSource&gt;();
}
if (_audioSource != null)
{
_audioSource.Play();
}
}
}
</pre><p>There is a property MenuItemData that accepts an IMenuItemData. If you set it, it will retain the value in a private field but also shows the value of Title in a TextMesh component. This behaviour is also an IInputClickHandler, so if the user taps this, the OnInputClicked method is called. Essentially all it does, is sending off it's MenuItemData object - that was used to fill the text with a value - to the Messenger. And it tries to play a sound. You should decide for yourself if you want that.</p><p>So all we have to to is add this behaviour MenuItem prefab, as <em>this</em> is the thing we are going to click on. That way, if you click next to the text but at the correct height (the menu 'row'), it's still selected. So select MenuItem, hit the "Add Component" button and add the Menu Item Controller.</p><p><a href="https://lh3.googleusercontent.com/-Hd6f3r6geKI/WV_BuMRJ9cI/AAAAAAAAPEA/VB22nLicvysdHphdibX-bA7hNXBxjAXJgCHMYCw/s1600-h/image%255B11%255D"><img width="600" height="230" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/--fUWezCf5ig/WV_BuevHuaI/AAAAAAAAPEE/AUhsslBZh90Mrh-VL3uhlRMj6mt62TJbQCHMYCw/image_thumb%255B6%255D?imgmax=800" border="0"></a></p><p>Now if you like, you can add an with AudioSource with a special sound that signifies the selection of a menu. As I have stated before, immediate (audio) feedback is very important in immersive applications. I have done not so. I usually let the receiver of a MenuSelectedMessage do the notification sound.</p><h2>Building the menu itself</h2><p>This is done by a surprisingly small and simple behavior. All it does is instantiate a number of game object on a certain positions.</p><pre style="font-size: 11px;">using System.Collections.Generic;
using System.Linq;
using UnityEngine;public class MenuBuilder :MonoBehaviour
{
public float MaxNumber = 10;
public float TopMargin = 0.1f;
public float MenuItemSize = 0.1f;
private List&lt;GameObject&gt; _createdMenuItems = new List&lt;GameObject&gt;();
public MenuBuilder()
{
_menuItems = new List&lt;IMenuItemData&gt;();
}
public GameObject MenuItem;
private IList&lt;IMenuItemData&gt; _menuItems;
public IList&lt;IMenuItemData&gt; MenuItems
{
get { return _menuItems; }
set
{
_menuItems = value;
BuildMenuItems();
}
}
private void BuildMenuItems()
{
foreach (var menuItem in _createdMenuItems)
{
DestroyImmediate(menuItem);
}
if (_menuItems == null || !_menuItems.Any())
{
return;
}
for (var index = 0; index &lt; MenuItems.Count; index++)
{
var newMenuItem = MenuItems[index];
var newGameObject = Instantiate(MenuItem, gameObject.transform);
newGameObject.transform.localPosition -=
new Vector3(0,(MenuItemSize * index) - TopMargin, 0);
var controller = newGameObject.GetComponent&lt;MenuItemController&gt;();
controller.MenuItemData = newMenuItem;
_createdMenuItems.Add(newGameObject);
}
}
}
</pre><p>All the important work happens in BuildMenuItems. Any existing items are destroyed first, then we simply loop through the list of menu items - these are IMenuItemData objects. Then a game object provided in MenuItem is instantiated inside the current game object, and it's vertical position is calculated and set. Then it gets the MenuItemController from the instantiated game object - it just assumes it must be there - and puts the newMenuItem in it - so the MenuItem will show the associated text.</p><p>So now add the MenuBuilder to the HelpHolder. Then, from prefabs, drag the MenuItem prefab onto the Menu Item property. Net result:</p><p><a href="https://lh3.googleusercontent.com/-d4zodqeM58Y/WV_Buw4yaPI/AAAAAAAAPEI/_h4pzxKb-4QSDP1GIUZYLx_RmXvBNxFPQCHMYCw/s1600-h/image%255B5%255D"><img width="400" height="117" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-mg92iYsxG1A/WV_BvHVAGZI/AAAAAAAAPEM/uwz-S3lF6B0jsM5bsVZAHHXlKfMS8KM3ACHMYCw/image_thumb%255B2%255D?imgmax=800" border="0"></a></p><p>Now let's add an initialization behaviour to actually make stuff appear in the menu. This behavior has some hard coded data in it, but you can imagine this coming from some Azure data source</p><pre style="font-size: 11px;">using System.Collections.Generic;
using UnityEngine;
public class VistaMenuController : MonoBehaviour
{
void Start()
{
var builder = GetComponent&lt;MenuBuilder&gt;();
IList&lt;IMenuItemData&gt; list = new List&lt;IMenuItemData&gt;();
list.Add(new MenuItemData
{
Title = "Mount Everest, Nepal",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(27.91282f, 86.94221f)
});
list.Add(new MenuItemData
{
Title = "Kilomanjaro, Tanzania",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(-3.21508f, 37.37316f)
});
list.Add(new MenuItemData
{
Title = "Mount Rainier, Washington, USA",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(46.76566f, -121.7554f)
});
list.Add(new MenuItemData
{
Title = "Niagra falls (from Canada)",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(43.07306f, -79.07561f)
});
list.Add(new MenuItemData
{
Title = "Mount Robson, British Columbia, Canada",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(53.061809f, -119.168358f)
});
list.Add(new MenuItemData
{
Title = "Athabasca Glacier, Alberta, Canada",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(52.18406f, -117.257f)
});
list.Add(new MenuItemData
{
Title = "Etna, Sicily, Italy",
MenuId = 1,
SelectMessageObject = new WorldCoordinate(37.67865f, 14.9964f)
});
builder.MenuItems = list;
}
}
</pre><font color="#ff0000"></font><p>This comes straight from Walk the World - these 7 of it's 10 vistas with a location to look from it. Add this behaviour to the Help Holder as well. Now it's time to run the code and see our menu for the very first time!</p><h2><a href="https://lh3.googleusercontent.com/-XIUEZaofoe0/WWYL8W7H9rI/AAAAAAAAPF4/HC5FcbO4fUwXp5uPvRzMZc4C3USYprWQgCHMYCw/s1600-h/image271%255B2%255D"><img width="186" height="240" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-MQWoCiAInhE/WWYL8ypbc4I/AAAAAAAAPF8/Rd8iup6a4jcx4YNQZ55uKDlSIFka1rvsgCHMYCw/image271_thumb%255B1%255D?imgmax=800" border="0"></a>Some tweaking and fiddling</h2><p>If you press the play button in Unity, you will get something like displayed to the left. A former British colleague would say something among the lines of "<a href="https://verybritishproblems.teemill.co.uk/product/bloody-hell-t-shirt/" target="_blank">It's not quite what I had in mind</a>". But we can fix this, fortunately.</p><p>In the Menu Item Builder that you have added to HelpHolder, there's two more properties:</p><p><a href="https://lh3.googleusercontent.com/-lC12v0OH2WQ/WV_BweUmCxI/AAAAAAAAPEY/RK-2Bn6gPCA8LfEW0MUqeVa9bUw7rkFLgCHMYCw/s1600-h/image%255B26%255D"><img width="398" height="76" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-6MwYN8MCkwo/WV_Bw-F0OhI/AAAAAAAAPEc/_s-5Z3hAcnk217z17u1h6nbnNE0S2mXAQCHMYCw/image_thumb%255B13%255D?imgmax=800" border="0"></a></p><p>The first one is the relative location where the first item should appear, and the second the size allotted for each menu. Clearly the first MenuItem is placed too low. The only way to really get this done is by trial an error. The higher you make Top Margin, the higher up the first item moves. A value of 0.18 gives about this and that seems about right:</p><p><a href="https://lh3.googleusercontent.com/-XAPbpL7Q98Y/WV_BxSvcroI/AAAAAAAAPEg/eyI27lbGc4EYpZCjVzKq-dGonfZqNfw0wCHMYCw/s1600-h/image%255B34%255D"><img width="304" height="232" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-NkAXPSvMGak/WV_Bx6vSCcI/AAAAAAAAPEk/y0KHU4VB-6o5fQPOKTSjqsGnRXmyCbn_QCHMYCw/image_thumb%255B17%255D?imgmax=800" border="0"></a></p><p>And 0.041 for Menu Item Size gives this:</p><p><a href="https://lh3.googleusercontent.com/-PK5qLnKRYZs/WV_ByLDyPFI/AAAAAAAAPEo/fqxpxvKNVLAsZC6utQmXqJhilN3qYcdYwCHMYCw/s1600-h/image%255B35%255D"><img width="304" height="220" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-VGwIQ6jGN7I/WV_BysABH6I/AAAAAAAAPEs/L_pWV0MaDE851zeOdU356O6oC_MLGBi1wCHMYCw/image_thumb%255B18%255D?imgmax=800" border="0"></a></p><p>Which is just what you want - a tiny little space between the menu items. Like I said, just trial and error.</p><h2>Testing if its works</h2><p>Once again, a bit lame: a simple behaviour to listen to the menu selection messages:</p><pre style="font-size: 11px;">using HoloToolkitExtensions.Messaging;
public class MenuListener : MonoBehaviour
{
// Use this for initialization
void Start()
{
Messenger.Instance.AddListener&lt;MenuSelectedMessage&gt;(ProcessMenuMessage);
}
private void ProcessMenuMessage(MenuSelectedMessage msg)
{
if (msg.MenuItem.MenuId == 1 )
{
Debug.Log("Taking you to " + msg.MenuItem.Title);
Debug.Log(msg.MenuItem.SelectMessageObject.ToString());
}
}
}
</pre><p>Add this behaviour to the Managers object, click play, and sure enough if you click menu items, you will see in Unity's debug console:</p><p><a href="https://lh3.googleusercontent.com/-Su4tpan-COg/WWJffYcePxI/AAAAAAAAPFM/ZCt1bvRJLnw5588uNj7IxADy-ncwM7CnwCHMYCw/s1600-h/image%255B7%255D"><img width="350" height="260" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-qNfOnBbD8H0/WWJff3rreNI/AAAAAAAAPFQ/OSoKiRN9azEI6wGhUURJGiLIj_ij9F0AQCHMYCw/image_thumb%255B4%255D?imgmax=800" border="0"></a></p><p>Yes, I know, that's a lame demo - <em>you</em> connect something to the message that actually does something. Speak out the name. Have a dancing popup. The point is that it works and the messages get out when you click :)</p><h2>Some final look &amp; feel bits</h2><p>Yah! We have a more or less working menu but it looks kind of ugly and not everything works - the close button, for instance. Let's fix the look &amp; feel first. We needed the greenish background of the menu item to properly space and align the items, but now we do not need it anymore. So go to the MenuItemMaterial. Select a new <a href="https://lh3.googleusercontent.com/-ftWiu677OvM/WWYL9Yj_kbI/AAAAAAAAPGA/KiS42tTEpggEWglc0dYhqd8g9pqOkMLmgCHMYCw/s1600-h/image%255B23%255D"><img width="350" height="115" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-9vn9hlMI-Hc/WWYL99awtyI/AAAAAAAAPGE/AxIeGeVDdGU863sMHsPcHqrdNa7amEjhQCHMYCw/image_thumb%255B14%255D?imgmax=800" border="0"></a>shader: under <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a>, you will find "Vertex Lit Configurable Transparent".</p><p>Then go all the way down, to "Other" and set "Cull" to front. That way, the front part of the plane - the green strips will be invisible - but still hittable. </p><p>If you press play, the menu should now look like this:</p><p><a href="https://lh3.googleusercontent.com/-dhghZON6FAI/WWYOG1Pm5XI/AAAAAAAAPGQ/Xwz2X797trU1iGVZz4y3fQUpIW3nOGgWwCHMYCw/s1600-h/image%255B27%255D"><img width="304" height="220" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-voc1BQLF0x0/WWYOHDXw5lI/AAAAAAAAPGU/3gCYe2X3fV4vMNXd1RkNgMsW1_5eBor_QCHMYCw/image_thumb%255B16%255D?imgmax=800" border="0"></a></p><h2>Getting the button to work</h2><p>As stated above, the button is not working - and for a very simple reason: in my <a href="https://dotnetbyexample.blogspot.com/2017/07/building-floating-hololens-info-screen-2.html" target="_blank">previous post</a> I showed that it looks for a component in it's parent that is a BaseTextScreenController (or a child class of that). There are none. </p><p>So let's go back to the VistaMenuController again. The top says</p><pre style="font-size: 11px;">public class VistaMenuController : MonoBehaviour</pre><p>Let's change that into </p><pre style="font-size: 11px;">public class VistaMenuController : BaseTextScreenController</pre><p>You will need to add "using HoloToolkitExtensions.Animation;" to top to get this to work. You will also need to change</p><pre style="font-size: 11px;">void Start()<br>{</pre><p>into </p><pre style="font-size: 11px;">public override void Start()<br>{<br> base.Start();</pre><p>If you now hit "Play" in Unity you will end up with this</p><p><a href="https://lh3.googleusercontent.com/-5q5SD7vXjOc/WWYL-M4wfYI/AAAAAAAAPGI/Jshs8kNpz_kUICNu_u6tcuyuGbS2jD_yQCHMYCw/s1600-h/image%255B14%255D"><img width="304" height="249" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-wAgcsVQ6CT4/WWYL-cfxyHI/AAAAAAAAPGM/cqzzt4dJP7MDy6ch_dKEcYZszK0ezbEmgCHMYCw/image_thumb%255B7%255D?imgmax=800" border="0"></a></p><p>Right. Nothing at all :). This is because the base Start method (which is the Start method of BaseTextScreenController) actually <em>hides</em> the menu, on the premises that you don't want to see the menu initially. So we have to have a way to make it visible. Fortunately, that's very easy. We will just re-use the ShowHelpMessage from the previous post again to make this work. Go back one more time to the VistaMenuController "Start" method, and add one more statement:</p><pre style="font-size: 11px;">public override void Start()<br>{<br> base.Start();<br> <strong> <font color="#ff0000">Messenger.Instance.AddListener&lt;ShowHelpMessage&gt;(m =&gt; Show());</font></strong></pre><p>If you now press play, you will still see nothing. But if you yell "Show help" to your computer (or press "0" - zero) the menu pops up and comes into view. With, I might add, the for my apps now iconic "pling" sound. And if you click the button, the menu will disappear with the equally iconic "clonk" .</p><h2>Some concluding remarks</h2><p>Of course, this is still pretty primitive. With the current font size and menu item size, stuff will be happily rendered outside of the actual menu screen if your texts are too long or you have more than 7 menu items. That is because the screen is just a floating backdrop. Scrolling for more items? Nope. Dynamic or manual resizing? Nope. But it is a start, and I have used it with great success. </p><p>Let me know if this was valuable to you, and what you used it for. <a href="https://github.com/LocalJoost/FloatingDynamicMenuDemo" target="_blank">Full demo project at GitHub</a>, as always.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/9ceRvs3B9qE" height="1" width="1" alt=""/>2017-07-13T07:36:26.982+02:003http://dotnetbyexample.blogspot.com/2017/07/building-dynamic-floating-clickable.htmlBuilding a floating HoloLens 'info screen' - 2: adding the C# behaviourshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/jq_OZb0F3rg/building-floating-hololens-info-screen-2.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 01 Jul 2017 09:51:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-3303129793364780709<h2>Intro</h2><p>In the <a href="http://dotnetbyexample.blogspot.com/2017/06/building-floating-hololens-info-screen-1.html" target="_blank">first installment of this 2-part blog post</a> we have created the UI of the info screen, now we are going to build the dynamics.</p><ul><li>First we will make the app recognize a speech command</li><li>Then we will add the dynamics to make the screen appear and disappear when we want.</li><li>Then we will make the close button work</li><li>As a finishing touch we will add some spatial sound.</li></ul><h2>Adding a speech command – the newest new way</h2><p>For the second or maybe even the third time since I have started using the <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a>, the was the way speech commands are supposed to work has changed. The keyword manager is now obsolete, you now have to use SpeechInputSource and SpeechInputHandler. </p><p>First, we add <a href="http://dotnetbyexample.blogspot.com/2017/04/using-messenger-to-communicate-between.html" target="_blank">a Messenger, as already described in this blog post</a>, to the Managers game object. It sits in Assets/HoloToolkitExtensions/Scripts/Messaging.</p><p>Then, since we are good boy scouts that like to keep things organized, a create a folder “Scripts” under “Assets/App”. In “Scripts” we add a “Messages” folder, and in that we create the following highly complicated message class ;) </p><pre>public class ShowHelpMessage
{
}</pre><p>In Scripts we create the SpeechCommandExectutor, which is simply this:</p><pre>using HoloToolkitExtensions.Messaging;
using UnityEngine;
public class SpeechCommandExecutor : MonoBehaviour
{
public void OpenHelpScreen()
{
Messenger.Instance.Broadcast(new ShowHelpMessage());
}
}</pre><p>Add this SpeechCommandExecutor to the Managers game object. Also add a SpeechInputSource script from the HoloToolkit, click they tiny plus-button on the right and add “show help” as keyword:</p><p><a href="https://lh3.googleusercontent.com/-_0iC9OG6-M4/WVE522Gq2RI/AAAAAAAAO78/D2kv5-sxgz0odqJ_u1CTRsJ1v4OKWNyVACHMYCw/s1600-h/image%255B54%255D"><img width="300" height="98" title="image" align="left" style="border: 0px currentcolor; border-image: none; float: left; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-uIh8659Jvtw/WVE53eFpA4I/AAAAAAAAO8A/YR3xxUCCZr07rLiv7te2D2wIDvg-VA96wCHMYCw/image_thumb%255B26%255D?imgmax=800" border="0"></a><a href="https://lh3.googleusercontent.com/-y-ww-aIUUhk/WVE53w5brPI/AAAAAAAAO8E/l9m02IUbtRQIdyIZQYgq4DkPKXxSg9IawCHMYCw/s1600-h/image%255B35%255D"><img width="300" height="84" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-0fybz6l9gls/WVE54dXHm0I/AAAAAAAAO8I/ouxkqWTYuBcZAqwpJK5YGZHRThjBA5s0gCHMYCw/image_thumb%255B15%255D?imgmax=800" border="0"><br></a></p><p><a href="https://lh3.googleusercontent.com/-FpCED4cuUZk/WVE54n8NU4I/AAAAAAAAO8M/LVp9tTi9tjQfESTlgFsm7e9JqJEma8FLgCHMYCw/s1600-h/image%255B21%255D"><br></a></p><p><br></p><p><br></p><p>Also, select a key in “key shortcut”. Although they Unity3D editor supports voice commands, you can now also use a code to test the flow. And believe me – your colleagues will thank you for that. Although lots of <em>my</em> colleagues are now quite used to me talking to devices and gesturing in empty air, repeatedly shouting at a computer because it was not possible to determine if there’s a bug in the code or the computer just did not hear you… is still kind of frowned upon.</p><p>Anyway. To connect the SpeechCommandExecutor to the SpeechInputSource we need a SpeechInputHandler. That is also in the HoloToolkit. So drag it out of there into the Managers objects. Once again you have to click a very tiny plus-button:</p><p><a href="https://lh3.googleusercontent.com/-llDoMS5wRnM/WVE55JUcSNI/AAAAAAAAO8Q/_uEm4xumkSM3Ib44laMt6aqc4_bJn5SaQCHMYCw/s1600-h/image%255B53%255D"><img width="300" height="74" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-nmqFLVqWf6w/WVE55nWU9oI/AAAAAAAAO8U/x6tyaQGb5SMzg61WNo38O1rGeeZrKaGZgCHMYCw/image_thumb%255B25%255D?imgmax=800" border="0"></a></p><p>And then the work flow is a follows</p><p><a href="https://lh3.googleusercontent.com/-mV550eSXx4o/WVE56CSAvjI/AAAAAAAAO8Y/NZ0f-xjrLrYdSuKbiujppJfKQvUcDmYqQCHMYCw/s1600-h/image%255B67%255D"><img width="650" height="341" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-ZKPPpEwI8Xo/WVE56aTMNOI/AAAAAAAAO8c/qOtWJOfsl6o5BuAo05f2EIKmTKtCjzK5gCHMYCw/image_thumb%255B35%255D?imgmax=800" border="0"></a></p><ol><li>Check the “Is Global Listener” checkbox (that is there because of a pull request by Yours Truly)</li><li>Select the plus-button under “Responses”</li><li>Select “Show help” from the keyword drop down</li><li>Drag the Managers object from the Hierachy to the box under “Runtime only”</li><li>Change “Runtime only” to “Editor and Runtime”</li><li>Select “SpeechCommandExecutor” and then “OpenHelpScreen” from the right dropdown.</li></ol><p>To test you have done everything ok:</p><p>In Assets/App/Scripts, double-click SpeechCommandExecutor. </p><p><a href="https://lh3.googleusercontent.com/-F7_tfmigxjc/WVE56wGtIYI/AAAAAAAAO8g/MtVAV7dh6jQJUib4P0JW09npcmU_P0Q0ACHMYCw/s1600-h/image%255B71%255D"><img width="304" height="148" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-5hMrPaPuM1c/WVE57P7MAvI/AAAAAAAAO8k/xXUMn24KJrsmjpJ0MyYaocOlgIOaRvs1wCHMYCw/image_thumb%255B37%255D?imgmax=800" border="0"></a></p><p>This will open Visual Studio, on the SpeechCommandExecutor. Set a breakpoint on </p><pre>Messenger.Instance.Broadcast(new ShowHelpMessage());</pre><p>Hit F5, and return to Unity3D. Click the play button, and press “0”, or shout “Show help” if you think that’s funny (on my machine, speech recognition in the editor does not work on most occasions, thus I am very happy with the keys options). </p><p>If you have wired up everything correctly, the breakpoint should be hit. Stop Visual Studio and leave Unity Play Mode again. This part is done.</p><h2>Making the screen follow your gaze</h2><p>Another script from my HoloToolkitExtensions, that I already mentioned in some form, is MoveByGaze. It looks like this:</p><pre style="font-size: 10px;">using UnityEngine;
using HoloToolkit.Unity.InputModule;
using HoloToolkitExtensions.SpatialMapping;
using HoloToolkitExtensions.Utilities;
namespace HoloToolkitExtensions.Animation
{
public class MoveByGaze : MonoBehaviour
{
public float MaxDistance = 2f;
public float DistanceTrigger = 0.2f;
public float Speed = 1.0f;
private float _startTime;
private float _delay = 0.5f;
private bool _isJustEnabled;
private Vector3 _lastMoveToLocation;
public BaseRayStabilizer Stabilizer = null;
public BaseSpatialMappingCollisionDetector CollisonDetector;
// Use this for initialization
void Start()
{
_startTime = Time.time + _delay;
_isJustEnabled = true;
if (CollisonDetector == null)
{
CollisonDetector = new DefaultMappingCollisionDetector();
}
}
void OnEnable()
{
_isJustEnabled = true;
}
// Update is called once per frame
void Update()
{
if ( _isBusy || _startTime &gt; Time.time)
return;
var newPos = LookingDirectionHelpers.GetPostionInLookingDirection(2.0f,
GazeManager.Instance.Stabilizer);
if ((newPos - _lastMoveToLocation).magnitude &gt; DistanceTrigger || _isJustEnabled)
{
_isJustEnabled = false;
var maxDelta = CollisonDetector.GetMaxDelta(newPos - transform.position);
if (maxDelta != Vector3.zero)
{
_isBusy = true;
newPos = transform.position + maxDelta;
LeanTween.moveLocal(gameObject, transform.position + maxDelta,
2.0f * maxDelta.magnitude / Speed).setEaseInOutSine().setOnComplete(MovingDone);
_lastMoveToLocation = newPos;
}
}
}
private void MovingDone()
{
_isBusy = false;
}
private bool _isBusy;
}
}
</pre><p>This is an updated, LeanTween (in stead of iTween) based version of a thing I already described before <a href="http://dotnetbyexample.blogspot.com/2017/01/dragging-holograms-with-gaze-and.html" target="_blank">in this post</a> so I won’t go over it in detail. You will find it in the Animation folder of the HoloToolkitExtensions in <a href="https://github.com/LocalJoost/FloatingScreenDemo/" target="_blank">the demo projec</a>t. It uses helper classes BaseSpatialMappingCollisionDetector, DefaultMappingCollisionDetector and SpatialMappingCollisionDetector that are also described in the same post – these are in the HoloToolkitExtensions/SpatialMapping folder of <a href="https://github.com/LocalJoost/FloatingScreenDemo/" target="_blank">the demo project</a>.</p><p>The short workflow, for if you don’t want to go back to that article:</p><ul><li>Add a SpatialMappingCollisionDetector to the Plane in the HelpHolder</li><li>Add a MoveByGaze to the HelpHolder itself</li><li>Drag the InputManager on top of the “Stabilizer” field in the MoveByGaze script</li><li>Drag the Plane on top of the “Collision Detector” field</li></ul><p>The result should look like this</p><p><a href="https://lh3.googleusercontent.com/-gSwKrWfU8uE/WVNQeboZ04I/AAAAAAAAO9s/zOspYbonweYNclmSCn-Uewm77_pKbLoXACHMYCw/s1600-h/image%255B7%255D"><img width="400" height="132" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-X1eIqxaewU4/WVNQfPvI1dI/AAAAAAAAO9w/rClDj8q3uL8oZPwerwfi-eEdUauNFzrGgCHMYCw/image_thumb%255B4%255D?imgmax=800" border="0"></a></p><p>I would suggest updating “Speed” to 2.5 because although the screen moves nice and fluid, the default value is a bit slow for my taste. If you now press the Play Button in Unity, you will see the screen already following the gaze cursor if you move around with the mouse or the keyboard.</p><iframe width="650" height="365" src="https://www.youtube.com/embed/dur5C8eyfwk" frameborder="0" allowfullscreen=""></iframe><p>The only thing is, it is not always aligned to the camera. For that, we have the LookAtCamera script I already wrote about <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-3creating.html" target="_blank">in October in part 3 of the HoloLens airplane tracker app</a>, but I will show it here anyway:</p><pre>using UnityEngine;
namespace HoloToolkitExtensions.Animation
{
public class LookatCamera : MonoBehaviour
{
public float RotateAngle = 180f;
void Update()
{
gameObject.transform.LookAt(Camera.main.transform);
gameObject.transform.Rotate(Vector3.up, RotateAngle);
}
}
}
</pre><p>because it’s so small. The only change between this and the earlier version is that you know can set the the rotate angle in the editor ;). Drag it on top of the HelpHolder now the screen will always face the user after moving to a place right in front of it. </p><h2>Fading in/out the help screen</h2><p>In the first video you can see the screen fades nicely in on the voice command, and out when it’s clicked. The actual fading is done by no less than three classes, two of whom are inside the HoloToolkitExtensions. First is this simple FadeInOutController, that is actually usable all by itself:</p><pre>using UnityEngine;
namespace HoloToolkitExtensions.Animation
{
public class FadeInOutController : MonoBehaviour
{
public float FadeTime = 0.5f;
protected bool IsVisible { get; private set; }<br>
private bool _isBusy;
public virtual void Start()
{
Fade(false, 0);
}
private void Fade(bool fadeIn, float time)
{
if (!_isBusy)
{
_isBusy = true;
LeanTween.alpha(gameObject, fadeIn ? 1 : 0, time).setOnComplete(()
=&gt; _isBusy = false);
}
}
public virtual void Show()
{
IsVisible = true;
Fade(true, FadeTime);
}
public virtual void Hide()
{
IsVisible = false;
Fade(false, FadeTime);
}
}
}
</pre><p>So this is a pretty simple behaviour that fades the current gameobject in or out, in a configurable timespan, and it makes sure it will not get interrupted while doing the fade. Also – notice it initially fades the gamobject out in zero time, so initially any gameobject with this behavior will be invisible</p><p>Next up is BaseTextScreenController, that is a child class of FadeInOutController:</p><pre>using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
namespace HoloToolkitExtensions.Animation
{
public class BaseTextScreenController : FadeInOutController
{
private List&lt;MonoBehaviour&gt; _allOtherBehaviours;
// Use this for initialization
public override void Start()
{
base.Start();
_allOtherBehaviours = GetAllOtherBehaviours();
SetComponentStatus(false);
}
public override void Show()
{
if (IsVisible)
{
return;
}
SetComponentStatus(true);
var a = GetComponent&lt;AudioSource&gt;();
if (a != null)
{
a.Play();
}
base.Show();
}
public override void Hide()
{
if (!IsVisible)
{
return;
}
base.Hide();
StartCoroutine(WaitAndDeactivate());
}
IEnumerator WaitAndDeactivate()
{
yield return new WaitForSeconds(0.5f);
SetComponentStatus(false);
}
}
}
</pre><p>So this override, on start, gathers all other behaviors, then de-activates components (this will be explained below). When Show is called, it first activates the the components, then tries to play a sound, then calls the base Show to unfade the control. If Hide is called, it first calls the base fade, then after a short wait starts to de-activate all components again. </p><p>So what is the deal with this? The other two missing routines are like this:</p><pre>private List&lt;MonoBehaviour&gt; GetAllOtherBehaviours()
{
var result = new List&lt;Component&gt;();
GetComponents(result);
var behaviors = result.OfType&lt;MonoBehaviour&gt;().Where(p =&gt; p != this).ToList();
GetComponentsInChildren(result);
behaviors.AddRange(result.OfType&lt;MonoBehaviour&gt;());
return behaviors;
}
private void SetComponentStatus(bool active)
{
foreach (var c in _allOtherBehaviours)
{
c.enabled = active;
}
for (var i = 0; i &lt; transform.childCount; i++)
{
transform.GetChild(i).transform.gameObject.SetActive(active);
}
}
</pre><p>As you can see, the first method simply finds all behaviors in the gameobject – the screen - <em>and its immediate children, </em>except for this behavior. If you supply “false” for “active”, it will first disable all behaviours (except the current one), and then it will set all child gameobjects to inactive. The point of this is that we have a lot of things happening in this screen. It’s following your gaze, checking for collisions, it’s spinning a button, and it’s waiting for clicks – all in vain as the screen is invisible. So this setup makes the whole screen dormant, disables all behaviors except the current one – and also can bring it back ‘to life’ again by supplying ‘true’. The important part is to do the right order (first the behaviours, then the gameobjects). It’s also important to gather the behaviours at the start, because once gameobjects are deactivated, you can’t get to their behaviors anymore. </p><p>The final class does nearly nothing – but this is the only app-specific class</p><pre>using HoloToolkitExtensions.Animation;
using HoloToolkitExtensions.Messaging;
public class HelpTextController : BaseTextScreenController
{
public override void Start()
{
base.Start();
Messenger.Instance.AddListener&lt;ShowHelpMessage&gt;(ShowHelp);
}
private void ShowHelp(ShowHelpMessage arg1)
{
Show();
}
}</pre><p>Basically the only thing this does is make sure the Show method is called when a ShowHelpMessage is received. If you drag this HelpTextController on top of the HelpHolder and press the Unity play button, you see an empty screen in stead of the help screen. But if you press 0 or yell “show help” the screen will pop up.</p><h2>Closing the screen by a button tap</h2><p>So now the screen is initially invisible, it appears on a speech command – now how do we get rid of it again? With this very simple script the circle is closed:</p><pre>using HoloToolkit.Unity.InputModule;
using UnityEngine;
namespace HoloToolkitExtensions.Animation
{
public class CloseButton : MonoBehaviour, IInputClickHandler
{
private void Start()
{ }
void Awake()
{
gameObject.SetActive(true);
}
public void OnInputClicked(InputClickedEventData eventData)
{
var h = gameObject.GetComponentInParent&lt;BaseTextScreenController&gt;();
if (h != null)
{
h.Hide();
var a = gameObject.GetComponent&lt;AudioSource&gt;();
if (a != null)
{
a.Play();
}
}
}
}
}</pre>
<p>This a standard HoloToolkit IInputClickHandler – when the user clicks, it tries to find a BaseTextScreenController in the parent and calls the Hide method, effectively fading out the screen. And it tries to play a sound, too.</p><h2>Some finishing audio touches</h2><p>Two behaviours – the CloseButton and the BaseTextScreenController – try to play sound when they are activated. As I have stated multiple times before, having immediate audio feedback when a HoloLens <a href="https://lh3.googleusercontent.com/-H3bGFBDw_Kw/WVZBd68bsDI/AAAAAAAAO-Q/My6UzaRYDpw12_gx2_51-p6Q2cXs4GCYwCHMYCw/s1600-h/image%255B6%255D"><img width="300" height="358" title="image" align="right" style="border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-EW0TckU-dVY/WVZBeZpRbmI/AAAAAAAAO-U/a8Vorn0NEvQH7IVMXTWfRRB8sfhh47UGgCHMYCw/image_thumb%255B3%255D?imgmax=800" border="0"></a>‘understands’ a user initiated action is vital, especially when that action’s execution may take some time. At no point you want the user to have a ‘huh it’s not doing anything’ feeling.</p><p>In <a href="https://github.com/LocalJoost/FloatingScreenDemo/" target="_blank">the demo project</a> I have included two audio files I use quite a lot – “Click” and “Ready”. “Click” should be added to the Sphere in HelpHolder. That is easily done by dragging it onto the Sphere from App/Scripts/Audio onto the Sphere. That will automatically create an AudioSource. </p><p>Important are the following settings:</p><ul><li>Check the “Spatialize” checkbox</li><li>Uncheck the “Play on awake checkbox</li><li>Move the “Spatial Blend” slider all the way to the right</li><li>In the 3D sound settings section, set “Volume Roloff” to “Custom Rolloff” </li></ul><p>Finally, drag “Ready” on top of the HelpHolder itself, where it will be picked up by the HelpTextController (which is a child class of BaseTextScreenController ) and apply the same settings. Although you might consider not using spatial sound here, because it’s not a sound that is particularly attached to a location – it’s a general confirmation sound</p><h2>Conclusion</h2><p>To be honest, a 2d-ish help screen feels a bit like a stopgap. You can also try to have a kind of video of audio message showing/telling the user about the options that are available. Ultimately you can think of an intelligent virtual assistant that teach you the intricacies of an immersive app. With the advent of ‘intelligent’ bots and stuff like LUIS it might actually become possible to have an app help you through it’s own functionality by having a simple questions-and-answers like conversation with it. I had quite an interesting discussion about this subject at Unity Unite Europe last Wednesday. But then again, since Roman times we have pointed people in right directions or conveyed commercial messages by using traffic signs and street signs – essentially 2D signs in a 3D world as well. Sometimes we used painted murals, or even statue like things. KISS sometimes just works. </p><p>The completed demo project <a href="https://github.com/LocalJoost/FloatingScreenDemo/" target="_blank">can be downloaded here.</a></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/jq_OZb0F3rg" height="1" width="1" alt=""/>2017-07-03T08:13:28.065+02:000http://dotnetbyexample.blogspot.com/2017/07/building-floating-hololens-info-screen-2.htmlA HoloLens helper class to get a position dead ahead of the user–on a physical object or at a max distancehttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/-aEHvro2VhM/a-hololens-helper-class-to-get-position.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Tue, 27 Jun 2017 04:45:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-4912135930703015673<p>This is a little tidbit I use in a lot of apps, for instance in the <a href="http://dotnetbyexample.blogspot.com/2016/07/hololens-cubebouncer-application-part-1.html" target="_blank">CubeBouncer</a> and for floating info screens in my other apps. Basically I want to know a position dead ahead of the user, at a certain maximum distance, or closer by if the user is looking at a physical object that is closer by. Think of an invisible ray coming out of the HoloLens – I want to have a point where it strikes a physical object – and if there is no such thing within a certain maximum distance, I want a point along that ray at a maximum distance.</p><p>I made a little helper class for that, and it’s called LookingDirectionHelpers</p>
<pre style="font-size: 10px;">using HoloToolkit.Unity.InputModule;
using HoloToolkit.Unity.SpatialMapping;
using UnityEngine;
namespace HoloToolkitExtensions.Utilities
{
public static class LookingDirectionHelpers
{
public static Vector3 GetPostionInLookingDirection(float maxDistance = 2,
BaseRayStabilizer stabilizer = null )
{
RaycastHit hitInfo;
var headReady = stabilizer != null
? stabilizer.StableRay
: new Ray(Camera.main.transform.position, Camera.main.transform.forward);
if (SpatialMappingManager.Instance != null &amp;&amp;
Physics.Raycast(headReady, out hitInfo, maxDistance,<br> SpatialMappingManager.Instance.LayerMask))
{
return hitInfo.point;
}
return CalculatePositionDeadAhead(maxDistance);
}
public static Vector3 CalculatePositionDeadAhead(float distance = 2,
BaseRayStabilizer stabilizer = null)
{
return (stabilizer != null
? stabilizer.StableRay.origin + stabilizer.StableRay.direction
: Camera.main.transform.position + Camera.main.transform.forward)
.normalized * distance;
}
}
}
</pre><p>Although it’s a small thing, it actually does quite a lot. If you call GetPostionInLookingDirection</p><ul><li>It first tries to determine a so-called head ray. This can either come from the stabilizer or be directly calculated from the location and angle of the camera (that is, your head). I would recommend feeding it a stabilizer, as that makes for getting a much more reliable location.</li><li>If it actually finds a hit, it returns that point</li><li>If it does not, it uses CalculatePositionDeadAhead to, well, calculate a position dead ahead. It takes once again either the stabilizer head ray or calculates one from the camera, then normalizes it (i.e. makes it’s length 1) and then multiplies it with the desired distance. This effectively gives a point <em>distance</em> meters right before the user’s eyes.</li></ul><p>This script requires the presence of a SpatialMappingManager prefab, for that’s the only way to find out which Unity layer contains the spatial mesh. If you want to call this script using a stabilizer (and I think you should), the InputManager should be present as well, as that will create a GazeManager singleton, which contains the stabilizer. So you can call this helper class like this:</p><pre style="font-size: 10px;">var pos = LookingDirectionHelpers.GetPostionInLookingDirection(2.0f, GazeManager.Instance.Stabilizer);</pre><p>And that will return you a point at either a Spatial Mesh, or at 2 meters distance from the user.</p><p>Although and the code involved is not very voluminous, I have gone all the way and made and – albeit pretty lame – minimalist demo for it <a href="https://github.com/LocalJoost/LookingDirectionDemo" target="_blank">that you can download here</a>. This is based upon a <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a> based setup <a href="http://dotnetbyexample.blogspot.com/2017/06/setting-up-hololens-project-with.html" target="_blank">as described here</a>. All it does it, when you move around, is print the location onto the debug log. So this will only work in the Unity editor, on the HoloLens emulator, or on an app running in debug mode on an actual HoloLens – but you won’t see much happening on the device itself, just in debug windows on your screen. </p><p><a href="https://lh3.googleusercontent.com/-osT5e9avZ0A/WVJFVYXPf8I/AAAAAAAAO9U/a6_b4OJV-kgV6aySMZVqWB_ULZw1rCioACHMYCw/s1600-h/image%255B3%255D"><img width="644" height="419" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-JgHMb0S27qY/WVJFV3yJVlI/AAAAAAAAO9Y/iUCpxGL38mk0uN2z8rn-yUAMIG3iTo2fgCHMYCw/image_thumb%255B1%255D?imgmax=800" border="0"></a></p><p>But it shows how it works and can be used, and that is the point. Later in this blog we will see a better application for this.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/-aEHvro2VhM" height="1" width="1" alt=""/>2017-06-27T17:18:52.866+02:000http://dotnetbyexample.blogspot.com/2017/06/a-hololens-helper-class-to-get-position.htmlBuilding a floating HoloLens 'info screen' - 1: making the Unity assetshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/cO4a2Ir_O5U/building-floating-hololens-info-screen-1.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Mon, 26 Jun 2017 05:43:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-5614149573458129380<h2>Intro</h2><p>Those who have seen my HoloLens apps (most notably <a href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ" target="_blank">Walk the World</a>) have noticed I tend to use floating "info screens", especially for help screens. My apps are mostly voice command driven as I don't like to have floating controls that are in view all of the time. They stress the fact that you are in an virtual environment, and that degrades the actual immersive experience, IMHO. So I go as much for gestures and voice as possible.</p><p>But where there are no visual clues for functionality, there's also lack of discoverability. So I tend to include a voice command "help" or "show help" that brings up a simple floating screen that shows what the app can do. </p><iframe width="650" height="365" src="https://www.youtube.com/embed/l37QxsyYuxU" frameborder="0" allowfullscreen=""></iframe><p>A few important things that you might not see right away:</p><ul><li>The screen follows your gaze</li><li>The screen tries to move away from you to be readable, but will stop moving if it get's pushed against an obstacle. So it won't disappear into another hologram or a physical object, like the wall or a floor. Or at least it tries to. I must admit it does not always works perfectly.</li><li>Notice that at first it will appears like 1 meter before you and move into view, next time it will appears where you last left it and then move into view.</li></ul><p>In a two-part post I will describe how I have created such a screen. </p><ul><li>The first part will handle building the actual visual structure in Unity (and one little behaviour)</li><li>The second part describes the code for all other Unity behaviours.</li></ul><p>I am going to assume you know a bit about Unity but not too much, so there's going to be lot of images.</p><h2>Setting up a base project</h2><p><a href="https://lh3.googleusercontent.com/-gvCUlE_yVBE/WU5uzmAClJI/AAAAAAAAO3o/Mtz02jlnug0P1ruoBa4DALB_56Sh-OxMgCHMYCw/s1600-h/image%255B8%255D"><img width="200" height="123" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-VbwPdYNI3go/WU5u0Pp25gI/AAAAAAAAO3s/naBtd6fT5JkThd5l8m1_DB5dJCNo8WK5wCHMYCw/image_thumb%255B4%255D?imgmax=800" border="0"></a>That's easy, as I described that <a href="http://dotnetbyexample.blogspot.com/2017/06/setting-up-hololens-project-with.html" target="_blank">in my previous blog post</a>. Just make sure you download a <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a> from June 16, 2017, or later. This includes <a href="https://github.com/Microsoft/HoloToolkit-Unity/pull/721" target="_blank">this pull request by yours truly</a> that we will need in this app. And while you are importing stuff, also import <a href="http://dentedpixel.com/LeanTweenDocumentation/classes/LeanTween.html" target="_blank">LeanTween</a> from the Unity Asset Store (hit CTRL-9 to open it immediately without having to hunt it down in the menu). When doing so, make sure you deselect everything but the Plugisn checkbox.</p><h2>The basic setup of an info screen</h2><p>My info screens basically exist out of three simple components:</p><ul><li>A background, usually a Plane</li><li>Text displayed by a 3DTextPrefab</li><li>Some kind of 'button' to close the screen. I tend to use a rotating Sphere for that, <a href="http://dotnetbyexample.blogspot.com/2017/02/a-generic-toggle-component-for-hololens.html" target="_blank">a step towards true 3D UIs that I already explored in this post</a>.</li></ul><p>So let's build those!</p><h2>Background plane</h2><p>Inside the HologramCollection that we inherited <a href="/http://dotnetbyexample.blogspot.com/2017/06/setting-up-hololens-project-with.html" target="_blank">from my previous post</a> we will first make an empty game object that I called "HelpHolder" as this will be a help screen, but you can call it anything you like. To make designing a little easier, set it's Z position to 1, else it will be sitting over the camera, which is always on 0,0,0 in a HoloLens app. That kind of obscures the view.</p><p><a href="https://lh3.googleusercontent.com/-AzPBpPd7d0A/WU5u0rLYbWI/AAAAAAAAO3w/smhJ6dS_vgglwgptS2dGNvGncK2sVo9JACHMYCw/s1600-h/image%255B43%255D"><img width="433" height="180" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-wXO61Z17Ftc/WU5u1AAzF-I/AAAAAAAAO30/fPJ45KPhO7ExP_RgRM7s90ZGnBJVxAKngCHMYCw/image_thumb%255B29%255D?imgmax=800" border="0"></a></p><p>Inside that HelpHolder we first make a common Plane. This gives the standard, way too big 10x10m horizontal square. Change it's rotation to 270 and change X and Z scale to 0.07m (changing the Y scale makes no sense as a Plane essentially has no Y dimension).</p><p><a href="https://lh3.googleusercontent.com/-5iH1N95Yt2M/WU5u1Q82tMI/AAAAAAAAO34/4pqweoJpW3AMlo16S5nxiL9NEPirdHOSgCHMYCw/s1600-h/image%255B44%255D"><img width="431" height="140" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-U13QukquQ1Q/WU5u1y22-wI/AAAAAAAAO38/Ps_g8Cgvv74Ve9FksQvqecbDrJqUSLqWQCHMYCw/image_thumb%255B30%255D?imgmax=800" border="0"></a></p><p>Double click the Plane in the HelpHolder - this will make your scene zoom in. Now use that hand button and left top the scene screen and the CTRL key to rotate around to you get to see the white side of the Plane (the other side is translucent). Notice the HoloLens cursor ;)</p><p><a href="https://lh3.googleusercontent.com/-OwgY8aEdGhk/WU5u2Ygs7yI/AAAAAAAAO4A/ITtLjpv8IJUin6lKC2wawyv3QR1aKWLpQCHMYCw/s1600-h/image%255B35%255D"><img width="640" height="441" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-yz0B8jOmihg/WU5u27OdPqI/AAAAAAAAO4E/fB3axC6blrMqTQE9KJ2pIpFgvSEq-wDIACHMYCw/image_thumb%255B23%255D?imgmax=800" border="0"></a></p><p>Now a white background for a text doesn't look good for me, I find it too bright. So we are going to make a material to make it look better.</p><p>To keep things organized, we first create an "App" folder in "Assets", and within that a "Materials" folder. In that Materials folder we create a HelpScreenMaterial Material</p><p><a href="https://lh3.googleusercontent.com/-TU74lEHpaVo/WU5u3F1W6BI/AAAAAAAAO4I/ToH2-38FtkcEfDC-VSk6YAJyS997mDLBQCHMYCw/s1600-h/image%255B64%255D"><img width="368" height="209" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-now3PTztnA8/WU5u3xxpAjI/AAAAAAAAO4M/czTsG-7dQR0rY2Hu0flw33XSspCj4JKoQCHMYCw/image_thumb%255B42%255D?imgmax=800" border="0"></a></p><h2>Setting some color and reflection</h2><p>Now over on the right side:</p><ul><li>Set "Shader" to "HoloToolkit/StandardFast"</li><li>Set "Rendering Mode" to "Transparent"</li><li>Set "Color" to a background color you fancy, I took a kind of sky blue (#0080FFFF)</li><li>Move the "Smoothness" slider all the way to the left - we don't want any reflections or stuff from this 'screen'</li></ul><p><a href="https://lh3.googleusercontent.com/-_zqaAjqEgKM/WU5u4NbIyFI/AAAAAAAAO4Q/LphukvXLSikgp-qOcV6PknHFi-aKZqV8wCHMYCw/s1600-h/image%255B52%255D"><img width="443" height="199" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-Ie6cbR6LsZg/WU5u4mBLYaI/AAAAAAAAO4U/7i92r-kmQO8wsZJFK7bxtCCk5kM2687nACHMYCw/image_thumb%255B34%255D?imgmax=800" border="0"></a></p><p>Now you only have to drag the material on your plane and it will turn blueish.</p><p><a href="https://lh3.googleusercontent.com/-utdl9G-oBvY/WU5u5LtaAJI/AAAAAAAAO4Y/wn-gcf_wejI7nYi4P2rv3BwrNg1Ya5KfgCHMYCw/s1600-h/image%255B63%255D"><img width="640" height="310" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-UIjOA56fSfQ/WU5u5qFvPRI/AAAAAAAAO4c/9QRLFh_mF6QQsxzMTdfdMb7BQixVySJbACHMYCw/image_thumb%255B41%255D?imgmax=800" border="0"></a></p><p>Rather standard Unity stuff this, but I thought it nice to point it out for beginners.</p><h2>Changing the collider</h2><p>A Collider is a component that determines how a game object collides with other objects. When you use primitive objects, those are of the same type as actual shape you use, although the names are not always the same. So a Sphere typically has a Sphere Collider, but a Cube has a Box Collider. There is no Plane collider, as a Plane is a generic Mesh - so it uses a Mesh Collider. And here we run into an issue, because a Plane typically has one side and it looks the Mesh Collider has that as well - and if not, it does not prevent the help window from getting pushed though the floor or a wall. As I found out&nbsp; making this demo :D. </p><p>So select the Plane, hit the Add Component button at the bottom and add a Box Collider.</p><p>Then</p><ul><li>Unselect the checkmark next to "Mesh Collider". This will disable the old Mesh Collider. A game object may have only one active Collider so we want to get rid of this. You can also delete it if you want using the dropdown that appears if you click the gear icon all the way to the right.</li><li>Put "0.02" in the Y filed in the "Size" Section. This will make the Collider as big as the blue plane, and 2 cm thick.</li></ul><p><a href="https://lh3.googleusercontent.com/-wwiFJzgcgNI/WVEBg5N6HwI/AAAAAAAAO6s/7Kf2vgQjJM0c0mydZNBppIgKNIFouJY5wCHMYCw/s1600-h/image%255B8%255D"><img width="436" height="480" title="image" align="left" style="border: 0px currentcolor; border-image: none; float: left; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-9MNwcTEB-5I/WVEBhTkaiPI/AAAAAAAAO6w/v9oElm4ZBzUSdP4q8mMWDxGqf0BlYKIQQCHMYCw/image_thumb%255B4%255D?imgmax=800" border="0"></a>What may seem confusing is that the Collider claims it's 10 x10 meters in size. That is true, but it is also scaled to 0.07 in X direction, and 0.050535134 in Z direction. If you remember the default size of a Plane is 10x10, this makes the screen about 70 cm wide and 50 cm height, which looks like the size you saw on the video. A Plane has no thickness, so if the scale of Y is set to 1, the colliders width will be the actual size in the Y field. </p><p><br></p><p><br></p><p><br></p><p><br></p><p><br></p><p><br></p><p><br></p><p><br></p><p><br></p><a href="https://lh3.googleusercontent.com/-ggszfHNkzJ8/WVEBhmtCeOI/AAAAAAAAO60/N0mXyJWv9NM1oqRYD3rsAlm02VK7W2aYQCHMYCw/s1600-h/image%255B12%255D"><img width="126" height="240" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-09K4G2bskgo/WVEBiCqMTaI/AAAAAAAAO64/rR08kqzl2SYnNvc92-iOC0DNo_hKB4eAwCHMYCw/image_thumb%255B6%255D?imgmax=800" border="0"></a>If you look at the screen on edge, you can see the green indicator lines showing the outline of the Collider:<p><br></p><p><br></p><p><br></p><p><br></p><h2>Adding text</h2><p>Find the 3DTextPrefab and drag it onto the HelpHolder:</p><p><a href="https://lh3.googleusercontent.com/-N07nbSPKWtg/WU5u6TDV5VI/AAAAAAAAO4g/JPlsSd9hqIENnly7crBSR2XhR-oMqGbWgCHMYCw/s1600-h/image%255B73%255D"><img width="640" height="309" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-J4xRSyFc8yI/WU5u617mI9I/AAAAAAAAO4k/7QEKnq-Ba5Iau_qRYsWUGWIlKjmVHqCmQCHMYCw/image_thumb%255B47%255D?imgmax=800" border="0"></a></p><p>It should end up <em>under</em> the Plane. Zoom a little in on the plane to see the text clearly.</p><p>Now change the text into the help text you want (I took some Lorum Ipsum) and change some of the text settings:</p><p><a href="https://lh3.googleusercontent.com/-5Y1Cd4ulyGM/WVEGB0XUxJI/AAAAAAAAO7k/nCXEvnkDNzAJmQR4AQBFiHwXcL_a7xTmwCHMYCw/s1600-h/image74%255B1%255D"><img width="640" height="259" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-x4FOsi1SANI/WVEGCuiMCDI/AAAAAAAAO7o/PnoJKRQW35ISESluTvOvv8ya50DZlHVMACHMYCw/image74_thumb?imgmax=800" border="0"></a></p><ul><li>Change Y to 0.133 (this will move the text towards the top of the 'screen', making room for the button later on).</li><li>Change Z to -0.02 (this will move the text to 2cm before the 'screen', this will prevent rendering issues later on</li><li>Change "Alignment" to left</li></ul><p>I wish to stress the Y value hugely depends on the size of your text, the the font size, and the size of your&nbsp; 'help screen'. To get it right, it requires some fiddling around (as we will see later on).</p><h2>Building the button - part 1</h2><p><a href="https://lh3.googleusercontent.com/-YOmzWnsE-TU/WU5u8yN9N3I/AAAAAAAAO4w/X-gOeXrtgaQMyrVkOy6xEJB59t0tm5JywCHMYCw/s1600-h/image%255B85%255D"><img width="240" height="108" title="image" align="right" style="border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-KpmoK8YsIjo/WU5u9XGZdjI/AAAAAAAAO40/wYxMacpGDlsISgNsrc0NQfmkXksqPwuPQCHMYCw/image_thumb%255B53%255D?imgmax=800" border="0"></a>Right click the HelpHolder and add a Sphere. This will - like almost everything is initially - way too large. To change it's scale to 0.08 in all three dimensions. Then change it's Z-value to -0.045 (this will put the button in front of the 'screen' and also change the Z value to -0.01</p><p>This results in the following and now you can see where the fiddling starts, because that screen is too big for the text and the button it not quite where we want it</p><p><a href="https://lh3.googleusercontent.com/-iumRp8FCRzo/WU5u9zuh31I/AAAAAAAAO44/xBiBLoVqqC8JeEnwLZHGBzXjxqq71edGgCHMYCw/s1600-h/image%255B98%255D"><img width="300" height="290" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-7kcRkTO7r4Q/WU5u-fgeaNI/AAAAAAAAO48/r-oCtj0wtHIF7X8YfCRa_T0MpqWNbdTogCHMYCw/image_thumb%255B60%255D?imgmax=800" border="0"></a></p><h2>Some in-between fiddling around </h2><p><a href="https://lh3.googleusercontent.com/-7OxElupZljo/WU5u-qTyRfI/AAAAAAAAO5A/7roEwRDB53oz0wWL0-1ozNUmE-YCjH5jgCHMYCw/s1600-h/image%255B97%255D"><img width="178" height="55" title="image" align="left" style="border: 0px currentcolor; border-image: none; float: left; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-Pi-t1mtQzv8/WU5u_MlADQI/AAAAAAAAO5E/abG6YeM8FrQHoMWXSl2GzFbH81WxXeD_QCHMYCw/image_thumb%255B59%255D?imgmax=800" border="0"></a>With these two buttons you can very easily move (left) or resize (right) objects. Select the Plane, then select the desired function. </p><p><br></p><p><a href="https://lh3.googleusercontent.com/-n1H7AG1_N5c/WVEBivgpHQI/AAAAAAAAO68/mPlqi064bhwdDtwVgUGI1Ybgl1zfAKFhQCHMYCw/s1600-h/image106%255B1%255D"><img width="300" height="216" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-mHKUYJEYe04/WVEBjOA925I/AAAAAAAAO7A/6JYn0Ka-RmIzjf19C-aIyxl0vUVY_rETwCHMYCw/image106_thumb?imgmax=800" border="0"></a><a href="https://lh3.googleusercontent.com/-ll6ZxaR-jO4/WVEBjQDrRyI/AAAAAAAAO7E/ZkdCiNnUEgQ61yYoAhqwlmWDfvm3ZYgcgCHMYCw/s1600-h/image102%255B1%255D"><img width="300" height="220" title="image" style="margin: 0px 0px 0px 22px; border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-pXTfJMDZp4U/WVEBkG8lkuI/AAAAAAAAO7I/f2fom0uI2SUg75GpjzYrCbcXrmYAxBwTgCHMYCw/image102_thumb?imgmax=800" border="0"></a></p><p>By dragging the blue block in the left image you can change the screen size in vertical direction, the red block will do so in horizontal. With the yellow arrow (right image) you can move the plane upward until it is where you like it.</p><p>In my Inspector pane on the right it said this when I was done:</p><p><a href="https://lh3.googleusercontent.com/-JNp3yA70Ezg/WVEBkfwnikI/AAAAAAAAO7M/1b8mqoPfvy0A2Njr0k9h_BEw9MkfqSxTgCHMYCw/s1600-h/image111%255B2%255D"><img width="350" height="113" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/--Leo0ShDYik/WVEBloTQE3I/AAAAAAAAO7Q/bs4nf-Y8REMBby4hOzgTDBSbJL7is7NFACHMYCw/image111_thumb%255B1%255D?imgmax=800" border="0"></a></p><p>But... now all of or stuff is <em>quite</em> off center as far as the HelpHolder, the root object, is concerned. It's center point is pretty low on our screen, which means the screen it too high.</p><p><a href="https://lh3.googleusercontent.com/-HQBnReFo7Gw/WU5vCeb4sxI/AAAAAAAAO5g/XB7Vx5MPs94EF2BjIlwvQ2AYwQNzj8INgCHMYCw/s1600-h/image%255B131%255D"><img width="640" height="275" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-5tXuixjC4gg/WU5vC78XzUI/AAAAAAAAO5k/W_F9_KrNhBU1iX23R4m37g2z3t6vagyWQCHMYCw/image_thumb%255B78%255D?imgmax=800" border="0"></a></p><p>This can be fixed by selecting all three denizens of HelpHolder (using the CTRL button), select the Move button again, grab the yellow arrow and move the whole combination downward until the read arrow is more or less on the horizon.</p><p><a href="https://lh3.googleusercontent.com/-3yuy3Tt7KNk/WU5vDRqZXQI/AAAAAAAAO5o/aI3Hb98yfbAu4yP4hDN5MNU-5qeKgi9WgCHMYCw/s1600-h/image%255B130%255D"><img width="640" height="298" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-T2LWfIto9LY/WU5vD7mwpfI/AAAAAAAAO5s/1qojsFX-3UME385Z1lDJOsDBQPKLjhjAwCHMYCw/image_thumb%255B77%255D?imgmax=800" border="0"></a></p><p>It does not have to be a super precise hit, as long as it's not so much off center as it first was.</p><h2>Building the button - part 2</h2><p>A white sphere is not a button, so add some that makes sure you can click it. I think a real designer might have to say something about it, but I have found that a red texture with a large OK text on it works - in the sense that I never had to explain to anyone that it's something you can air tap and that will act like something of a button. So I created this awesome :D picture, created a "Texture" folder under app and put it there</p><p><a href="https://lh3.googleusercontent.com/-qsDAmszqkU8/WU5vEV02yOI/AAAAAAAAO5w/eZYTy_2PT2sEYam5upn-Yb8cZKFxntAVACHMYCw/s1600-h/image%255B169%255D"><img width="146" height="194" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-MPWSDW1o4S0/WU5vE1MfLJI/AAAAAAAAO50/UpElOJnmfS0aWil058XF07JklI-H_y-JACHMYCw/image_thumb%255B112%255D?imgmax=800" border="0"></a><a href="https://lh3.googleusercontent.com/-HtL4JO_uFiU/WU5vFbCbuZI/AAAAAAAAO54/o6cuvz5De88A2p_rpq93zIZO45WWcoI1QCHMYCw/s1600-h/image%255B168%255D"><img width="357" height="194" title="image" style="margin: 0px 0px 0px 19px; border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-CKAQQjLFe2M/WU5vFz2w-KI/AAAAAAAAO58/wLinEAkmuRYFvBU0GtzraSPaeWgxrhvdQCHMYCw/image_thumb%255B111%255D?imgmax=800" border="0"></a></p><p>It's odd shape will become clear soon.</p><p>First, create a "DoneMaterial" in Materials. Then:</p><ul><li>Set "Shader" once again to "HoloToolkit/StandardFast"</li><li>Set "Rending Mode" to "Fade" (<em>not</em> Transparent)</li><li>Select the "Textures" folder in the App/Assets folder, and drag the "Done" texture on top of the little square left of the "Albedo" text</li><li>Change the X value of "Emissions/Tiling" to 3. This will morph three images on the Sphere, repeating them horizontally.</li></ul><p><a href="https://lh3.googleusercontent.com/-ngP-Xm-OCcs/WU5vGf6oyII/AAAAAAAAO6A/RuzygNNEVgAY8vwKNHpgR5T-nA32EjeyACHMYCw/s1600-h/image%255B183%255D"><img width="588" height="480" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-S3jDRBHNxSk/WU5vG8OIcII/AAAAAAAAO6E/F6m8X_tGBSUwZdJSJt2ws9ZPp-k99ZRmgCHMYCw/image_thumb%255B120%255D?imgmax=800" border="0"></a></p><p>If you have done everything correct, you will see the material looks like above.</p><p>Now drag the DoneMaterial from the Materials folder onto the Sphere in the Hierarchy</p><p><a href="https://lh3.googleusercontent.com/-ZlOhDSfu3vU/WU5vHcVwuII/AAAAAAAAO6I/tlPLjaPNYrcxf7KMfeJrqi3hhehkNdfkwCHMYCw/s1600-h/image%255B184%255D"><img width="640" height="269" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-gJvXjfpBJFU/WU5vH7IpTsI/AAAAAAAAO6M/0inFxKEMoFgxyC_Gp8g1pD9w1vv3-_bdACHMYCw/image_thumb%255B121%255D?imgmax=800" border="0"></a></p><p>And the button on the screen looks already familiar :). I left the default Shader settings as it's a bit smooth, so it looks like it reflects a little light adding to it's 3D appearance.</p><h2>Turn off shadows</h2><p>This is a thing trick I learned from <a href="https://www.linkedin.com/in/dvroegop/" target="_blank">Dennis Vroegop</a>, long time Microsoft MVP, my unofficial MVP 'mentor' who learned me to deal with suddenly being a Microsoft MVP too way back in 201,1 and long time expert on Natural User Interface. The trick is this: unless your app really <em>really</em> uses them for some reasons, turn off "receive shadows" and "cast shadows" for the renderers of all three objects. As you can see the actual real light sources in your room through HoloLens, shadows will appear on the wrong places anyway and only give cause for confusion at best - at worse they will 'break the spell'. As a bonus, the Unity engine won't need to calculate the shadows so you will save some performance as well.</p><p>So select all three objects (use the CTRL key for that like in any other normal Windows program) and turn this off in one go:</p><p><a href="https://lh3.googleusercontent.com/-dSyY2kCR4Mc/WVEBmNGG2RI/AAAAAAAAO7U/E4P7fUlUVkEDjVctFXMKuRtYp0fgs26jQCHMYCw/s1600-h/image%255B24%255D"><img width="521" height="322" title="image" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-1RIWm3LeBGE/WVEBmWrSylI/AAAAAAAAO7Y/ffgXSnIY5S4gqIqeexASaJ5IyrUzp-AowCHMYCw/image_thumb%255B14%255D?imgmax=800" border="0"></a></p><h2>A little code for dessert</h2><p>This is part of my HoloToolkitExtensions libary, that I one day will publish in full, but, well, time. It is called HorizontalSpinner, and basically is the grandchild of the SimpleRotator that briefly passed by (without explaination) in <a href="http://dotnetbyexample.blogspot.com/2017/02/a-generic-toggle-component-for-hololens.html" target="_blank">my post about a generic toggle component for HoloLens</a>. It uses LeanTween and looks like this:</p><pre style="font-size: 12px;">using UnityEngine;
namespace HoloToolkitExtensions.Animation
{
public class HorizontalSpinner: MonoBehaviour
{
public float SpinTime = 2.5f;
public bool AbsoluteAxis = false;
private void Start()
{
if (AbsoluteAxis)
{
LeanTween.rotateAround(gameObject, Vector3.up, 360 / 3.0f,
SpinTime / 3.0f).setLoopClamp();
}
else
{
var rotation = Quaternion.AngleAxis(360f / 3.0f, Vector3.up).eulerAngles;
LeanTween.rotateLocal(gameObject, rotation, SpinTime / 3.0f).setLoopClamp();
}
}
private void OnEnable()
{
LeanTween.resume(gameObject);
}
private void OnDisable()
{
LeanTween.pause(gameObject);
}
}
}
</pre>
<p>Default this spins an object around for 120° in local space. Since our the tiling of our 'button' is 3 in X direction, this will look like the button spins around in 2.5 seconds, while in reality, it will jump back to it's original position. But as you can see if you press the Unity Play button, it looks like the button is rotating around endlessly. LeanTween is doing most of the work: it rotates the game object around and the setLoopClamp at the end makes it a loop. No need for coroutines, lerps, and heaven knows what.</p><p>I am using local space because I want the button to perpendicular to the user's view at any given moment, but since the screen moves around with the user's gaze and gaze angle, it needs to be in the local space of the HelpHolder.</p><p>The HorizontalSpinner is in HoloToolkitExtensions/Scripts/Animations. Simply drag it on top of the Sphere, then change values as you like, although I would recommend not changing the Absolute Axis setting.</p><h2>Conclusion</h2><p>We have built the visual parts of a help screen but with very little functionality or interaction. It's actually not hard to do, if you know what you are doing. I hope I have helped you getting a bit more feeling for that.</p><p>In the next installment, which will hardly contain images, we will see WAY more code.</p><p>The project so far can be <a href="https://github.com/LocalJoost/FloatingScreenDemo/tree/blog1" target="_blank">found at GitHub</a>, as always.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/cO4a2Ir_O5U" height="1" width="1" alt=""/>2017-07-17T16:58:17.518+02:000http://dotnetbyexample.blogspot.com/2017/06/building-floating-hololens-info-screen-1.htmlSetting up a HoloLens project with the HoloToolkit - June 2017 editionhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/qckhV_uA4wE/setting-up-hololens-project-with.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Wed, 14 Jun 2017 10:05:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-6113362511946992481<h2>Preface with lame apologies</h2><p>My apologies for the longest blogging hiatus in the history of this blog, I was <em>very</em> busy with things both HoloLens and not-HoloLens related. Chief culprit though was my HoloLens app "<a href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ" target="_blank">Walk the World</a>", whose development completely captured and absorbed me. Passion for tech sometimes does that with me ;). Then came some work-related issues, but now I finally am up to speed again. </p><h2>Intro</h2><p>A consequence of the hiatus is a lot of change (the only thing that changes faster than your average JavaScript-framework-of-the-month is the <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a>) so any HoloLens how-to blog post I'd start with I would have to start with 'how to set up a project'. So I make a little separate post of that subject, thus I can refer to it often. Until the HoloToolkit changes again ;)</p><h2>File/new project and Getting what you need</h2><p>First, start a new Unity Project. I have called mine June2017HoloLensSetup.</p><p>Then, you open a browser and go to <a href="http://withinrafael.com/"><strong>Rafael Rivera</strong></a>'s&nbsp; <a href="http://holotoolkit.download/" target="_blank">HoloToolkit download site</a> and you click the top HoloToolkit-somenumber-unitypackage. At the time of this writing the top looks like this:</p><p><a href="https://lh3.googleusercontent.com/-pwe7kH2xcdA/WUFsclugvSI/AAAAAAAAO00/lTVlCYbnheIM1RiTz67DZwtnDn15UTiMgCHMYCw/s1600-h/image%255B3%255D"><img width="644" height="238" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-2dxuhcdIn9Y/WUFsdZCHTEI/AAAAAAAAO04/fP5CSIfecUUW-L4JR37xJodEYeR8hTY0gCHMYCw/image_thumb%255B1%255D?imgmax=800" border="0"></a>.&nbsp; </p><p>Downlaod and double click the resulting .unitypackage file - Unity will automatically import it into your project. This will give you a window with a <em>lot</em> of checkboxes:</p><p><a href="https://lh3.googleusercontent.com/-l55KPcvu9qg/WUFseLVqYOI/AAAAAAAAO08/oy74JcI9620-Et-nZJUzDZ38IED1Ho5UgCHMYCw/s1600-h/image%255B7%255D"><img width="639" height="484" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-42i8Wf2BD_A/WUFse1Fj2UI/AAAAAAAAO1A/6dSUMFJpcM8bj5_wUcs9wilvFd69Iw8WACHMYCw/image_thumb%255B3%255D?imgmax=800" border="0"></a></p><p>This looks a bit intimidating, but just collapse the HoloToolkit node in the list, that will reveal another node "HoloToolkit-test" that you don't need, and you can de-select it<a href="https://lh3.googleusercontent.com/-nc-l9bSz65A/WUFsfXVQk1I/AAAAAAAAO1E/FIIz92uNdRUASdPr1pWNsfzfJyyO8EI5wCHMYCw/s1600-h/image%255B11%255D"><img width="644" height="156" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-1HsbUOEx8_U/WUFsgCTS6uI/AAAAAAAAO1I/t8CyFWjsMZod3qvMmvBw7uOJBPD4WGUHwCHMYCw/image_thumb%255B5%255D?imgmax=800" border="0"></a></p><p>And then hit the Import-button. Then very soon, Unity will show a popup:</p><p><a href="https://lh3.googleusercontent.com/-fjOmvZlf2ME/WUFsgnpk1FI/AAAAAAAAO1M/ETLD8LokrwYxA85FHPs4bt_2XHzpWWAYgCHMYCw/s1600-h/image%255B21%255D"><img width="354" height="140" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-tmF9qiLC4c0/WUFshGgouFI/AAAAAAAAO1Q/ogBPUVsct5cZVek_LEMRfkqwzRoI_Y9wwCHMYCw/image_thumb%255B11%255D?imgmax=800" border="0"></a></p><p>Select "Force Text Serialization". This is very important if you want to be able to easily upgrade the HoloToolkit and be able to check it in into your source control of choice. </p><p>The HoloToolkit will now be imported. Well, that was not so hard, wasn't it? This is time to get some coffee as Unity will churn a little on this. </p><h2>Setting up the camera</h2><p>HoloLens apps need some settings applied to the camera, and the HoloToolkit folks have taken care of that. So first of all, you delete the Main Camera from your Hierarchy - just select it and delete it.</p><p><a href="https://lh3.googleusercontent.com/-_XknnztcGrc/WUFsh1SkkiI/AAAAAAAAO1U/HCWaZ7zwm0AB2dzPeNHNUrv5qUrIU4SmQCHMYCw/s1600-h/image%255B17%255D"><img width="354" height="75" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-Lha7s3jMzms/WUFsidIEOWI/AAAAAAAAO1Y/mUW-ZNCx8YUacNriamM9g03uSsnGBPQYQCHMYCw/image_thumb%255B9%255D?imgmax=800" border="0"></a></p><p>Then, find the HoloLensCamera by entering "Camera" into the search box indicated with a red ellipse, and drag the new Camera into your hierarchy</p><p><a href="https://lh3.googleusercontent.com/-SoKUFg3vaoM/WUFsiyvuz6I/AAAAAAAAO1c/3leb3QfoM2A4gksHc9VLcrNyEUGf6MWRACHMYCw/s1600-h/image%255B25%255D"><img width="354" height="252" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-eFiD99lKF8E/WUFsjruqZ-I/AAAAAAAAO1g/ibl2CFlOmBAn6Up1S6rj_CYnYg0tW4xtACHMYCw/image_thumb%255B13%255D?imgmax=800" border="0"></a>&nbsp;</p><p>This is also a good moment to save your scene. Hit CTRL+S (or click File/Save Scenes), enter an name for your scene (I usually take "Main") and you are done with the this part.</p><h2>Setting a whole lot of properties - the easy way</h2><p>Now you may not have noticed it, but after importing the HoloToolkit you got an extra extra menu options.</p><p><a href="https://lh3.googleusercontent.com/-uiNhZWlhIdI/WUFskPjS66I/AAAAAAAAO1k/Jh94tWvEyfsJv8Z00IQaRJXrbTFwJCzbwCHMYCw/s1600-h/image%255B30%255D"><img width="454" height="137" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-bI3Zc4BTAdk/WUFsk-F6a0I/AAAAAAAAO1o/Y3M5EbuquUo4XuX8TbFtsnE8e1GhiQagQCHMYCw/image_thumb%255B16%255D?imgmax=800" border="0"></a></p><p>This was added because you have to set a whole lot of properties necessary for or beneficial to a HoloLens projects -&nbsp; scattered all over Unity, and good luck to you determining what you forgot if something doesn't work and where you have to correct that. Those were the bad old days (like a year ago ;) ). Now it's easy. </p><p>Clicking "Apply HoloLens Scene Settings" gives you this popup. Make sure everything is selected, then hit Apply. Always hit apply!</p><p><a href="https://lh3.googleusercontent.com/-SWv0RVWxHXw/WUFslvVMy8I/AAAAAAAAO1s/fQqf3gyqIcIKEqBGALghI51OBxlAxxnhACHMYCw/s1600-h/image%255B35%255D"><img width="304" height="230" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-zuKLd1Cy5qE/WUFsmLyXpAI/AAAAAAAAO1w/cQZBWLg2b38n5lN775L77iTzXAUWxVDDQCHMYCw/image_thumb%255B19%255D?imgmax=800" border="0"></a></p><p>Next up is "Apply HoloLens Project Settings" </p><p><a href="https://lh3.googleusercontent.com/-DKq_QqsjW4M/WUFsmi3OxNI/AAAAAAAAO10/qQRUMXnvCpAr14nD_KoAqIgUOgB7MkOEACHMYCw/s1600-h/image%255B39%255D"><img width="304" height="248" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-5fNRONNMz0s/WUFsnPTlKMI/AAAAAAAAO14/aKy4KTMbOogd_aZ1lPuOakAGsFaBvaVNgCHMYCw/image_thumb%255B21%255D?imgmax=800" border="0"></a></p><p>Same procedure: make sure everything is selected, hit apply. Unity asks if you want to reload the project - click Yes and wait for the project to reload. Finally, "Apply HoloLens Capability Settings"</p><p><a href="https://lh3.googleusercontent.com/-9Qs4SGFeHb0/WUFsnySXuKI/AAAAAAAAO18/T-e3vEstF1oWHYk8yoj9L95ENer8fLsegCHMYCw/s1600-h/image%255B43%255D"><img width="304" height="246" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-aHfr4ufwdfg/WUFsocxtWWI/AAAAAAAAO2A/2LdaYueE_jgsg_qVYOt2i15phwyk0yY-QCHMYCw/image_thumb%255B23%255D?imgmax=800" border="0"></a></p><p>This is what I always select - my HoloLens apps all tend to use voice commands, all use Spatial Perception (that is, after all, HoloLens' biggest USP) and most of them download some data from the internet. I never use the camera. Make your selections according to your necessary capabilities and hit apply.</p><h2>Sound should be Spatial!</h2><p>For some reason this is not part of these nice HoloToolkit setup screens - but if you use <a href="https://developer.microsoft.com/en-us/windows/mixed-reality/spatial_sound" target="_blank">spatial sound</a> in your apps (and you should!) there is an extra step. You select "Edit/Project Settings/Audio" like this</p><p><a href="https://lh3.googleusercontent.com/-qerIBr2jEXo/WUFso5Oa3VI/AAAAAAAAO2E/_ydEakz2YKkjNUvokinDI6RkZ7N_AHqhgCHMYCw/s1600-h/image%255B47%255D"><img width="354" height="227" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/--IyN-CdPYAQ/WUFspnMYlYI/AAAAAAAAO2I/Cva95cJ1EWQDZnkUNe6GWEY_GK_2GUdYgCHMYCw/image_thumb%255B25%255D?imgmax=800" border="0"></a></p><p>"Project settings is almost all the way down in a <em>long</em> Edit menu. Now on the <em>right</em> side of the editor, all the way on the top, a new menu has appeared:</p><p><a href="https://lh3.googleusercontent.com/-1KgEvVMQaHQ/WUFsqN8m65I/AAAAAAAAO2M/a2_gSxGdEksr52OLgrBeznjUsaOZJvXEgCHMYCw/s1600-h/image%255B53%255D"><img width="454" height="229" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-vS03U8PCKiU/WUFsq5fByGI/AAAAAAAAO2Q/jeZ_vuZwh8IrEbg-V2w0POvqYLcSFL8FQCHMYCw/image_thumb%255B29%255D?imgmax=800" border="0"></a></p><p>The Spatializer Plugin is default set to "none", which will give you the default Unity stereo sound - nice, but that's sub-optimal usage of a $3K (or $5K) HoloLens - we want real spatial sound. So set it to MS HRTF Spatializer. This is, by the way, a very important and often overlooked part of immersive experiences. Human beings are 3D aware not only by vision, but also by sound. Think for instance of how you move around in a city, you hear cars coming to you even if they are outside of your vision (and that has more than once saved my life). In your house, there is the hum of the air conditioner, the bubbling of the fish tank filter, the voice of your partner - you don't have to see any one of those, but still you can tell where they are. It all contributes to your 3D awareness - and so does spatial sound contribute heavily to an immersive holographic experience.</p><p>So far for this soapbox moment ;).</p><h2>Some organizing </h2><p>It tend to create two empty gameobjects - HologramCollection and Managers. The HologramCollection will actually hold Holograms that are part of the application, the Managers object is like a container for global functions, object and standard stuff.</p><p><a href="https://lh3.googleusercontent.com/-vUZIjIj2gQc/WUFsrZTE9bI/AAAAAAAAO2U/IcPEDWBHHsMjjms8X2T0Volg1gTDZ2hFACHMYCw/s1600-h/image%255B57%255D"><img width="204" height="290" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-KAX1Ekx_Alw/WUFssHOv6vI/AAAAAAAAO2Y/q25PEm33gXQTAdcoJzxdPYbBxYFVrX77wCHMYCw/image_thumb%255B31%255D?imgmax=800" border="0"></a></p><h2>Some standard stuff to always add</h2><p>What (I think) you will always need is an InputManager (the thing that acts as a kind of hub for all input, be it gestures, speech or whatever), SpatialMapping (duh) and some kind of cursor. I usually take the object that is just called "Cursor", there's also "DefaultCursor" and "CursorWithFeedback". </p><p><a href="https://lh3.googleusercontent.com/-2YlIbjbmfvw/WUFssgrk_9I/AAAAAAAAO2c/wHDKKZx-0ug9w6nJfjn2HRMaebaP3ocewCHMYCw/s1600-h/image%255B60%255D"><img width="212" height="200" title="image" style="margin: 0px; display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-DmwcZNiOZAM/WUFstAsd5dI/AAAAAAAAO2g/wYwCNcNBLqQJ1EJNyE8m8ito3PrQ1J78ACHMYCw/image_thumb%255B32%255D?imgmax=800" border="0"></a></p><p>The easiest way to find them is once again entering their name in the search box. Now this can sometimes give cause for some confusion. For instance, if you enter "Cursor" in the search box you will get a lot of hits.</p><p><a href="https://lh3.googleusercontent.com/-MMmZDkG8ovk/WUFstq-gO5I/AAAAAAAAO2k/VgUf96VV6awDSoXeJaA492vsDcuxhFfiACHMYCw/s1600-h/image%255B68%255D"><img width="404" height="438" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-c5X-GQ7qGiM/WUFsuWrw6NI/AAAAAAAAO2o/vul_qCrFx2sqhLO-RRiCg8mZq9lUFYzBgCHMYCw/image_thumb%255B36%255D?imgmax=800" border="0"></a></p><p>Only choose the so-called <em>prefabs</em>, the things that get the simple sky blue cube next to it. The same goes, by the way, for InputManager and SpatialMapping. Always take the prefabs.</p><p>When you are done adding SpatialMapping, find the property "Surface Material" and change it to "Occlusion" by clicking the little circle button behind it. Then, select it from the "Select Material" window that pops up</p><p><a href="https://lh3.googleusercontent.com/-lpKfAwz9zfM/WU5IEQQa2MI/AAAAAAAAO3U/5SZ4vgovqyYQRwp5buwGs06yldMhjlADACHMYCw/s1600-h/image%255B4%255D"><img width="644" height="432" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-NXHqcqlUpT8/WU5IE8B0WqI/AAAAAAAAO3Y/LPVupg67A-Y0AntK22m5pYIq0XgoQ8xCQCHMYCw/image_thumb%255B1%255D?imgmax=800" border="0"></a></p><h2>Finally some stubbornness</h2><p>The current settings make that stuff is clipped when you are within 85cm of a Hologram. There are good technical and performance reasons for it, yet I am a stubborn *** ** * ***** so I tend to make this 15 cm. <a href="https://lh3.googleusercontent.com/-fLICSCdNVtg/WUFsvC2F4SI/AAAAAAAAO2s/H5qMtYI0tdgjgpXb_P26fq-GXVAY1n55ACHMYCw/s1600-h/image%255B72%255D"><img width="644" height="261" title="image" style="display: inline; background-image: none;" alt="image" src="https://lh3.googleusercontent.com/-8NsEDeKccVs/WUFsv4ro3CI/AAAAAAAAO2w/d1xOkup88aw_XREmgnW2bgkN6iPVHJk6gCHMYCw/image_thumb%255B38%255D?imgmax=800" border="0"></a></p><p>Simply because I think it's cool to get that close to a Hologram.</p><h2>A final word on source control</h2><p>Assuming you use Git, making a good .gitignore is pretty hard. Especially the HoloToolkit contains stuff you would expect not to reside in a source repository, there's all kinds of dlls in it and folders called x86 and x64, typically folders that are part of a build path and should usually be ignored. I've been bitten so many times by just that little piece of the HoloToolkit, especially when upgrading. Therefore a hard-won piece of advice: if you value your sanity, put a .gitignore file in Assets\HoloToolkit containing just these lines:</p><p><strong>!*.*<br>
!*</strong></p><p>That should always add everything, I think. </p><p>You can download the (empty) project for reference, <em>with</em> my .gitignore files (there are no less than three) <a href="https://github.com/LocalJoost/June2017HoloLensSetup" target="_blank">here</a>.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/qckhV_uA4wE" height="1" width="1" alt=""/>2017-06-24T13:09:04.494+02:002http://dotnetbyexample.blogspot.com/2017/06/setting-up-hololens-project-with.htmlUsing a messenger to communicate between objects in HoloLens appshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/xeJtY7rbjmM/using-messenger-to-communicate-between.htmlArchitectureHoloLensUnity3Dnoreply@blogger.com (Joost van Schaik)Wed, 12 Apr 2017 10:30:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-6575715529260379672<h2>Intro</h2> <p>I know I am past the ‘this has to work one way or the other’ stage of a new development environment when I start spinning off reusable pieces of software. I know that I am really getting comfortable when I am start thinking about architecture and build architectural components.</p> <p>Yes my friends, I am going to talk architecture today.</p> <h2>Unity3D-spaghetti – I need a messenger</h2> <p>Coming from the clean and well-fleshed out of UWP XAML development using MVVM (more specifically <a href="http://www.mvvmlight.net/" target="_blank">MVVMLight</a>), Unity3D can be a bit overwhelming. Apart from the obvious – the 3D stuff itself - there is no such thing as data binding, there is no templating (not sure how this would translate to a 3D environment anyway) and in samples (including some of my own) components communicate by either getting references to each other by looking in parent or child objects and calling methods in those components. This is a method that breaks as soon as 3D object hierarchies change and it’s very easy to make spaghetti code of epic proportions. Plus, it hard links classes. Especially speech commands come in just ‘somewhere’ and need to go ‘somewhere else’. How lovely it would be to have a kind of messenger. Like the one in MVVMLight. There is a kind of messaging in Unity, but in involves sending messages up or down the 3D object hierarchy. No way to other branches in that big tree of objects without a lot of hoopla. And to make things worse, you need to call methods by (string) name. A very brittle arrangement.</p> <h2>Good artist steal…</h2> <p>I will be honest up front – most of the code in the <a href="https://github.com/LocalJoost/CubeBouncer2/blob/master/Assets/HoloToolkitExtensions/Scripts/Messaging/Messenger.cs" target="_blank">Messenger</a> class that I show here is stolen. <a href="http://wiki.unity3d.com/index.php?title=CSharpMessenger_Extended" target="_blank">From here</a>, to be precisely. But although it solves one problem – it creates a generic messenger – it still uses strings for event names. So I adapted it quite heavily to use typed parameters, and now – in usage – it very much feels like the MVVMLight messenger. I also made it a <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a> Singleton. I am not going to type out all the details – have a look in the code if you feel inclined to do so. This article concentrates on <em>using</em> it.</p> <p>So basically, you simply drag this thing anywhere in your object hierarchy – I tend to have a special empty 3D object “Managers” for that in the scene – and then you have the following simple interface:</p> <ul> <li>To subscribe to a message of MyMessageType, simply write code like this</li></ul><pre>Messenger.Instance.AddListener&lt;MyMessage&gt;(ProcessMyMessage);
private void ProcessMyMessage(MyMessage msg)
{
//Do something
}</pre>
<ul>
<li>To be notified to a message of MyMessageType, simply call</li></ul><pre> Messenger.Instance.Broadcast(new MyMessage());</pre>
<ul>
<li>To stop being notified of MyMessageType, call</li></ul><pre>Messenger.Instance.RemoveListener&lt;MyMessage&gt;(ProcessMyMessage);</pre>
<h2>Example setup usage</h2>
<p><a href="https://lh3.googleusercontent.com/-Ro2xKy1aYkM/WO5kG4rtQ-I/AAAAAAAAOx4/zxtMHbKjx6c/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 8px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-6s6joGqlNiU/WO5kHWyNxPI/AAAAAAAAOx8/FNQl-7iroS4/image_thumb%25255B5%25255D.png?imgmax=800" width="254" align="left" height="484"></a>I have revisited my good old <a href="http://dotnetbyexample.blogspot.nl/2016/07/hololens-cubebouncer-application-part-1.html" target="_blank">CubeBouncer</a>, the very first HoloLens app I ever made and wrote about (although I never published it as such) that basically uses everything a HoloLens can do: it uses gaze, gestures, speech recognition, spatial awareness, interaction of Holograms with reality, occlusion, and spatial sound. Looking back at it now it looks a bit clumsy, which is partially because of my vastly increased experience with Unity3D and HoloLens, but also because of the progress of the <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a>. But anyway, <a href="https://github.com/LocalJoost/CubeBouncer2" target="_blank">I rewrote it using the new HoloToolkit and using the Messenger class</a> as a working demo of the Messenger.</p>
<p>In the Managers object that I use to group, well, manager-like scripts and objects, I have placed the a number of components that basically control the whole app. You see the messenger, a ‘Speech Command Handler’ and a standard HoloToolkit Keyword manager. This is a enormous improvement over building keyword recognizing script manually, as I did in <a href="http://dotnetbyexample.blogspot.nl/2016/07/hololens-cubebouncer-application-part-4.html" target="_blank">part 4</a> of the original CubeBouncer series. In case you need info on how the Keyword Manager works, see <a href="http://dotnetbyexample.blogspot.nl/2017/01/manipulating-holograms-move-scale.html" target="_blank">this post on moving objects by gestures</a> where it plays a supporting role.</p>
<p>Note, by the way, that I also assigned a keyboard key to all speech commands. This enables to test quickly within the Unity3D editor without actually speaking, thus preventing distracting (or getting funny looks and/or remarks) from your colleagues ;).</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>The SpeechCommandHandler class is <em>really</em> simple</p><pre>using CubeBouncer.Messages;
using UnityEngine;
using HoloToolkitExtensions.Messaging;
namespace CubeBouncer
{
public class SpeechCommandHandler : MonoBehaviour
{
public void CreateNewGrid()
{
Messenger.Instance.Broadcast(new CreateNewGridMessage());
}
public void Drop(bool all)
{
Messenger.Instance.Broadcast(new DropMessage { All = all });
}
public void Revert(bool all)
{
Messenger.Instance.Broadcast(new RevertMessage { All = all });
}
}
}</pre>
<p>It basically forwards all speech commands as messages, for anyone who is interested. Notice now, as well, that in the Keyword Manager both “drop” and “drop all” call the same method, but if you you look at the image above you will see a checkbox that is only selected for ‘drop all’. This is pretty neat, the editor that goes with this component automatically generates UI components for target method parameters.</p>
<p>Indeed, very similar to how it's done in MVVMLight</p>
<h2>Example of consuming messages</h2>
<p><a href="https://lh3.googleusercontent.com/-8nCXFTxqBDs/WO5kH9GLfRI/AAAAAAAAOyA/zGjJHnUN4m0/s1600-h/image%25255B15%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-x-nyQLtx0Jc/WO5kIez3K1I/AAAAAAAAOyE/Eb1-Er6TBhU/image_thumb%25255B7%25255D.png?imgmax=800" width="504" height="225"></a></p>
<p>Now the CubeManager, the thing that creates and manages cubes (it was called “MainStarter” in the original CubeBouncer) is sitting in the HologramCollection object. This is for no other reason than to prove the point that the location of the consumer in the object hierarchy doesn’t matter. This is (now) the only consumer of messages. It's start method goes like this.</p><pre>void Start()
{
_distanceMeasured = false;
_lastInitTime = Time.time;
_audioSource = GetComponent&lt;AudioSource&gt;();
Messenger.Instance.AddListener&lt;CreateNewGridMessage&gt;(p=&gt; CreateNewGrid());
Messenger.Instance.AddListener&lt;DropMessage&gt;( ProcessDropMessage);
Messenger.Instance.AddListener&lt;RevertMessage&gt;(ProcessRevertMessage);
}</pre>
<p>It subscribes to three types of messages. To process those messages, you can either used a Lambda expression or just a regular method, as shown above. </p>
<p>The processing of the message is like this:</p><pre>public void CreateNewGrid()
{
foreach (var c in _cubes)
{
Destroy(c);
}
_cubes.Clear();
_distanceMeasured = false;
_lastInitTime = Time.time;
}
private void ProcessDropMessage(DropMessage msg)
{
if(msg.All)
{
DropAll();
}
else
{
var lookedAt = GetLookedAtObject();
if( lookedAt != null)
{
lookedAt.Drop();
}
}
}
private void ProcessRevertMessage(RevertMessage msg)
{
if (msg.All)
{
RevertAll();
}
else
{
var lookedAt = GetLookedAtObject();
if (lookedAt != null)
{
lookedAt.Revert(true);
}
}
}</pre>
<p>For Drop and Revert, if the “All” property of the message is set, all cubes are dropped (or reverted) and that’s it, the rest works as before. Well kind of – for the actual revert method I now used two <a href="http://dentedpixel.com/LeanTweenDocumentation/classes/LeanTween.html" target="_blank">LeanTween</a> calls to move the Cube back to it’s original location – the actual code shrank from two methods of about 42 lines together to one 17 line method – that actually has an extra check in it. So as an aside – please use <a href="http://itween.pixelplacement.com/gettingstarted.php" target="_blank">iTween</a>, LeanTween or whatever for animation. Don’t write them yourself. Laziness is a virtue ;).</p>
<h2>Conclusion</h2>
<p>I will admit it’s a bit contrived example, but the speech recognition is now a thing on it’s own and it’s up to any listener to act on it – or not. My newest application “<a href="https://www.microsoft.com/store/productid/9P6SVQQCP2SQ" target="_blank">Walk the World</a>” uses the Messenger quite a bit more extensively and components all over the app communicate via that Messenger to receive voice commands, show help screen, and detect the fact the user has moved too far from the center and the map should be reloaded. These components do not need to have hard links to each other, they just put their observations on the Messenger and other components can choose to act. This makes re-using components for application assembly a lot easier. Kind of like in the UWP world.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/xeJtY7rbjmM" height="1" width="1" alt=""/>2017-04-12T19:35:26.561+02:000http://dotnetbyexample.blogspot.com/2017/04/using-messenger-to-communicate-between.htmlA ‘roller blind’ animation component for HoloLens applicationshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/aXKEsSh5VHY/a-roller-blind-animation-component-for.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 01 Apr 2017 00:25:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-9116838076475802872<h2>Intro</h2> <p>This is a cool little tidbit that I wrote in the cause of a project that required 2D images to be shown in a 3D context. Not everyone has 3D models of everything, and sometimes you just have a schematic drawing, a picture, or whatever 2D thing you want to see on a ‘screen’. That does not excuse you from making a good user experience, and I made this little tidbit to give just that little extra pizzazz to a boring ole’ 2D image in a HoloLens. So you click on a 3D device, out comes a schematic drawing. So it’s 2D in a 3D context, not 2D per se. </p> <h2>Say what ?</h2> <p>It basically pulls an image ‘down’ like a roller blind is being expanded. Typically you ‘hang’ this below a ceiling or the object the image is connected to/needs to clarify. Without much further ado, let’s just show what I mean</p><iframe height="365" src="https://www.youtube.com/embed/yYv2R0z3y-c" frameborder="0" width="650" allowfullscreen></iframe> <p>Nice, eh? I have the feeling my study is becoming quite a household scene by now for the regular readers of this blog ;).</p> <h2>Setting the stage</h2> <p>Being the lazy b*st*rd that I am, I just made a branch of <a href="http://dotnetbyexample.blogspot.com/2017/02/a-generic-toggle-component-for-hololens.html" target="_blank">my previous post</a>, deleted the floating 3D objects, implemented the roller blind as a Togglable, and used the 3D ‘button’ already present in the project as a Toggler. So now I have something to click on and start the animation. I also reused my <a href="http://dotnetbyexample.blogspot.com/2017/02/a-behaviour-for-dynamically-loading-and.html" target="_blank">DynamicTextureDownloader from the post before that</a> to show this image of daffodils in my front garden because that what less work than actually making a texture. Did I mention already I can be pretty lazy at times?</p> <h2>Unity setup</h2> <p><a href="https://lh3.googleusercontent.com/-H0IEcPs8cdU/WNpPKl4U0YI/AAAAAAAAOww/o_NNaxuPQhM/s1600-h/image%25255B6%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 8px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-pl6QvMTYqRY/WNpPLEonRpI/AAAAAAAAOw0/6r_CiVBaDpM/image_thumb%25255B1%25255D.png?imgmax=800" width="230" align="left" height="244"></a>What we have, initially, is just the button and a floating plane. There are some important settings to its rotation and it’s scaling. The rotation is because we want use see the image head-on. This important, as a Plane has only one side – if you look from it from behind you look right trough it.</p> <p>The default position of a plane is horizontal, like a flat area. So in order so see it head-on, we first need to rotate in 90⁰ over x (that will put it upright) and then 180⁰ over z to see the ‘front’ side (that used to be the top). Don’t try to be a clever geometrist and say “Hey, I can just rotate it 270⁰ and then I will look at the front side as well”. Although you are technically right, the picture will appear upside down. So unless you are prepared to edit all your textures to compensate for that, follow the easy path I’d say. The picture left shows the result, and the picture below it how it’s done.</p> <p>&nbsp;</p> <p><a href="https://lh3.googleusercontent.com/-eJ88DpFMKoU/WNpPL3lAqXI/AAAAAAAAOw4/BAd2MLSS8T4/s1600-h/image%25255B10%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 8px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-TGnN_tKm3gc/WNpPMZVEo0I/AAAAAAAAOw8/e5fLZm1mn58/image_thumb%25255B3%25255D.png?imgmax=800" width="230" align="left" height="307"></a></p> <p>So to the Plane, called RollerBlind, we add two components. First the DynamicTextureDownLoader. Set it’s Image Url to <a title="http://www.schaikweb.net/dotnetbyexample/daffodils.jpg" href="http://www.schaikweb.net/dotnetbyexample/daffodils.jpg">http://www.schaikweb.net/dotnetbyexample/daffodils.jpg</a>, which is a nice 1920x1080 picture of dwarf daffodils on the edge of my front garden</p> <p><img src="http://www.schaikweb.net/dotnetbyexample/daffodils.jpg" width="354" height="210"></p> <p>(yeah, I was a bit pessimistic about the ‘return rate’ and the buggers turned out to be multi headed too – so I am aware it’s a <em>bit</em> overdone for the space). Important is <em>not</em> to check the “Resize Plane” checkbox here as that will totally mess up the animation. You have to manually make sure the x and z(!!) sizes match up. So as the image is 1920*1080, horizontal size = 1.78 x vertical size. As the horizontal size is 0.15, so vertical size should be 0.15 / 1.78 = 0.084375. Be aware that a standard plane’s size – at scale 1 = is 10x10m, so this make the resulting picture appear as about 150 by 84 cm. I will never understand why standard shapes in Unity3D have different default sizes at scale 1 – for instance a cube =1x1x1m, a capsule roughly 1x2x1m, and a plane 10x10m – but I am sure there’s logic in that. Although I still fail to see it. But I digress.</p> <p>I stuck the RollerBlind into a “Holder” and used that to position the whole thing around. I place it 1.5 meters from the camera (same distance as the rotating button) and 70cm below it. Go convert that to feet if you must ;)</p> <p><a href="https://lh3.googleusercontent.com/-jBPBIjAndEY/WNpPNKjFoFI/AAAAAAAAOxA/fcu-hinprlk/s1600-h/image%25255B14%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-dyjm-ht_iVU/WNpPNvpAx2I/AAAAAAAAOxE/Hb3WbWQ46xQ/image_thumb%25255B5%25255D.png?imgmax=800" width="456" height="181"></a></p> <p>The only thing missing is the RollerBlindAnimator itself</p> <h2>Code! Finally!</h2> <p>We start with the first part - basically all state date and the stuff that collects the initial data</p><pre>using HoloToolkit.Unity.InputModule;
using UnityEngine;
namespace HoloToolkitExtensions
{
public class RollerBlindAnimatior : Togglable
{
public float PlayTime = 1.5f;
private AudioSource _selectSound;
private float _foldedOutScale;
private float _size;
private bool _isBusy = false;
public bool IsOpen { get; private set; }
public void Start()
{
_selectSound = GetComponent&lt;AudioSource&gt;();
_foldedOutScale = gameObject.transform.localScale.z;
var startBounds = gameObject.GetComponent&lt;Renderer&gt;().bounds;
_size = startBounds.size.y;
AnimateObject(gameObject, 0, 0);
}
}
}</pre>
<p>_selectSound is a placeholder for a sound to be played when this thing is being toggled, but the button is already taking care of that, so that’s not used here. Now the roller blind is going to be animated over what appears to be y, but since it’s rotated over x, that should now be the z-axis. So we collect the initial z scale. We also collect the objects apparent y–size. That we get from the bounds of the renderer, that apparently gives back it’s value in absolute values, not taking rotation into account. And then it quickly ‘closes’ the blind so it’s primed for use.</p>
<h2>Why do we need to know this size and center stuff?</h2>
<p>The issue, my friends, is that a Plane’s origin is at it’s <em>center</em>. So if you start shrinking the scale of the z-axis, the plane does not collapse to the top or bottom, but indeed - to it’s center. So rather than a roller blind going up, we get the effect of an old tube CRT TV being turned off (who is old enough to remember that? I am) – the picture collapses to a line in the middle. In order to compensate for that, for every decrease of scale by <em>n,</em> we need to move the whole thing<em> 0.5*n</em> <strong>up</strong>.</p>
<p>And that is exactly what AnimateObject does:</p><pre>private void AnimateObject(GameObject objectModel, float targetScale, float timeSpan)
{
_isBusy = true;
var moveDelta = (targetScale == 0.0f ? _size : -_size) / 2f;
LeanTween.moveLocal(objectModel,
new Vector3(objectModel.transform.localPosition.x,
objectModel.transform.localPosition.y + moveDelta,
objectModel.transform.localPosition.z), timeSpan);
LeanTween.scale(objectModel,
new Vector3(objectModel.transform.localScale.x,
objectModel.transform.localScale.y,
targetScale), timeSpan).setOnComplete(() =&gt; _isBusy = false);
}
</pre>
<p>As you can see I have taken a liking to LeanTween over iTween, as I find it a bit easier to use – no hashes but method chaining, that supports IntelliSense so I don’t have to remember that much names (did I mention I was lazy already?). </p>
<p>The last thing missing is the Toggle methods that you can override in Togglable. That’s not very special and only mentioned here for the sake of completeness</p><pre>public override void Toggle()
{
if (_isBusy)
{
return;
}
AnimateObject(gameObject, !IsOpen ? _foldedOutScale : 0, PlayTime);
if (_selectSound != null)
{
_selectSound.Play();
}
IsOpen = !IsOpen;
}</pre>
<h2>Two final things</h2>
<p>We need to tell the toggle button that it needs to toggle the roller blind when it’s tapped.So we set it’s Togglers Size value to 1 and drag the RollerBlind object from the hierachy to the Element 0 field.</p>
<p><a href="https://lh3.googleusercontent.com/-kGGuDSqaD5g/WNphSDcMhGI/AAAAAAAAOxU/BSwK8i4K0hI/s1600-h/image%25255B19%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-klYXhqAH0dc/WNphSvQjPfI/AAAAAAAAOxY/TECds-33woI/image_thumb%25255B8%25255D.png?imgmax=800" width="407" height="344"></a></p>
<p>And the very final thing: this app accesses the internet. It downloads the daffodil image after all. Do not forget to set the ‘internet client’ capability. I did. And spent an interesting time cursing my computer before the penny dropped. Sigh.</p>
<h2>Concluding words</h2>
<p>I hope I have added once again a fun new tool to your toolbox to make HoloLens experiences just a bit better. I notice I get a bit of a feeling for this – past the ‘OMG how am I ever going to make this work’, now into spinning of reusable components and pieces of architecture. As I said, I was too lazy to set up a proper repo, so I’ve put this <a href="https://github.com/LocalJoost/ToggleDemo/tree/rollerblind" target="_blank">in a branch</a> of the code belonging to <a href="http://dotnetbyexample.blogspot.com/2017/02/a-generic-toggle-component-for-hololens.html" target="_blank">previous blog post</a>. Enjoy!</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/aXKEsSh5VHY" height="1" width="1" alt=""/>2017-04-12T19:34:45.574+02:000http://dotnetbyexample.blogspot.com/2017/04/a-roller-blind-animation-component-for.htmlA generic toggle component for HoloLens appshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/su2FSkyGD0o/a-generic-toggle-component-for-hololens.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Tue, 21 Feb 2017 10:36:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-2061431843314432013<h2>Intro</h2> <p>The following scenario is one I have seen a lot of times – the user taps on a UI element, and then it and/or a couple of elements need to fade out, disappear, whatever. I suppose every developer has felt this itch that occurs when you basically make something the same the second time around, and you feel there will be a third and a fourth time coming up. Time for spinning up a new reusable component. Meet Toggler and it’s friend, Togglable.</p> <h2>The Toggler</h2> <p>This is a simple script that you can attach to any object that will function as a toggle – a ‘button’ if you like. It’s so simple and concise I just write the whole thing in one go: <pre style="font-size: 12px">using System;
using System.Collections.Generic;
using HoloToolkit.Unity.InputModule;
using UnityEngine;
namespace HoloToolkitExtensions
{
public class Toggler : MonoBehaviour, IInputClickHandler
{
private AudioSource _selectSound;
public List&lt;Togglable&gt; Toggles = new List&lt;Togglable&gt;();
public virtual void Start()
{
_selectSound = GetComponent&lt;AudioSource&gt;();
}
public virtual void OnInputClicked(InputClickedEventData eventData)
{
foreach (var toggle in Toggles)
{
toggle.Toggle();
}
if (_selectSound != null)
{
_selectSound.Play();
}
}
}
}
</pre>
<p>This thing has a list of Togglable. When it’s clicked, it calls the method “Toggle” on all Togglable objects in the list, and optionally plays a feedback sound to confirm the toggle has been clicked.</p>
<h2>The Togglable </h2>
<p>This is almost embarrassingly simple.</p><pre style="font-size: 12px">using UnityEngine;
namespace HoloToolkitExtensions
{
public abstract class Togglable : MonoBehaviour
{
public abstract void Toggle();
}
}</pre>
<p>and in itself completely uninteresting. What is interesting though is that you can use this base class to implement behaviours that actually do something useful (which is the point of bas classes, usually. D’oh). I will give a few examples.</p>
<h2>A toggleable that ‘just disappears’ </h2>
<p>Also not very complicated, although there’s a bit more to it than you would think<br></p><pre style="font-size: 12px">namespace HoloToolkitExtensions
{
public class ActiveTogglable : Togglable
{
public bool IsActive = true;
public virtual void Start()
{
gameObject.SetActive(IsActive);
}
public override void Toggle()
{
IsActive = !IsActive;
gameObject.SetActive(IsActive);
}
public virtual void Update()
{
// This code to make sure the logic still works in someone
// set the IsActive field directly
if (IsActive != gameObject.activeSelf)
{
gameObject.SetActive(IsActive);
}
}
}
}
</pre>
<p>To if Toggle is called, SetActive is called with either true or false and it will make the gameobject that it’s attached to flash in and out of existence. </p>
<h2>A toggleable that fades in or out</h2>
<p>This is a bit more work, but with the use of LeanTween animating opacity is pretty easy:</p><pre style="font-size: 12px">using UnityEngine;
namespace HoloToolkitExtensions
{
public class FadeTogglable : Togglable
{
public bool IsActive = true;
public float RunningTime = 1.5f;
private bool _isBusy = false;
private Material _gameObjectMaterial;
public virtual void Start()
{
Animate(0.0f);
_gameObjectMaterial = gameObject.GetComponent&lt;Renderer&gt;().material;
}
public override void Toggle()
{
IsActive = !IsActive;
Animate(RunningTime);
}
public virtual void Update()
{
// This code to make sure the logic still works in someone
// set the IsActive field directly
if (_isBusy)
{
return;
}
if (IsActive != (_gameObjectMaterial.color.a == 1.0f))
{
Animate(RunningTime);
}
}
private void Animate(float timeSpan)
{
_isBusy = true;
LeanTween.alpha(gameObject,
IsActive ? 1f : 0f, timeSpan).setOnComplete(() =&gt; _isBusy = false);
}
}
}
</pre>
<p>Initially it animates to the initial state in 0 seconds (i.e. instantly), and when the Toggle is called it animates in the normal running time from totally opaque to transparent – or the other way around.</p>
<p>There is a little caveat here – the object that needs to fade out then needs to use a material that actually supports transparency. So, for instance:</p><a href="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image_thumb.png" width="418" height="154"></a>
<p>So what is the point of all this?</p>
<p>I have created a little sample application to demonstrate the point. There is one ‘button’ – a rotating blue sphere with red ellipses on it, and four elements that need to be toggled when the button is clicked – two cubes that simply need to wink out, and two capsules that need to fade in and out:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image_thumb_3.png" width="507" height="293"></a></p>
<p>You drag the ActiveTogglable on both cubes, and FadeTogglable on both capsules. In fact, I did it a little bit different: I made prefab of both cube and capsule and dragged two instances on the scene. Force of habit. But in the end it does not matter. What does matter is that, once you have dragged a Toggle script on top of the sphere, you can now simply connect the Toggle and the Toggleables in the Unity editor, like this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-generic-toggle-component-for-HoloLens-_FF34/image_thumb_4.png" width="647" height="304"></a></p>
<p>Which makes it pretty darn powerful and reusable I’d say – and extendable, since nothing keeps you from implementing your own Toggleables.</p>
<p>The result in action looks like this:
<p><iframe height="365" src="https://www.youtube.com/embed/1R6uUHetYp8" frameborder="0" width="650" allowfullscreen=""></iframe></p>
<h2>Why not an interface in stead of a superclass?</h2>
<p>Yeah, that’s what I thought too. But you just try – components that can me dragged on top of each other need to be just that – components. So everything you drag needs to be a component at minimum, but you want the concrete class to be behaviours. So – you <em>have </em>to use a base class that’s a behaviour too. Welcome to the wondrous world of Unity, where nothing is what it seems – or what you think it is supposed to be ;)</p>
<h2>Concluding remarks and some thoughts about 3D interfaces</h2>Remember how Apple designed skeuomorphic user interfaces, that for instance required you to take a book out of a bookshelf? For young people, who never may have held much physical books, that’s about as absurd as the floppy disk icon for save – that is still widely used. But it worked in the real world, so we took that to the digital 2D world, even when it did no longer make sense. Microsoft took the lead with what was then called ‘Metro’ for the ‘digital native’ float design. Now buttons no longer mimic 3D (radio buttons) and heaven knows what.
<p>We are now in the 2007 of 3D UI design. No-one has any idea how to implement true 3D ‘user interfaces’, and there is no standard at all. So we tend to fall back on what worked – 2D design elements or 3D design elements that resemble 3D objects – like 3D ‘light switch buttons’ attached to some ‘wall’. Guilty as changed – my <a href="ms-windows-store://pdp/?ProductId=9NBLGGH52SZP" target="_blank">HoloLens app for Schiphol</a> has a 2D ‘help screen’&nbsp; complete with button.</p>
<p>With my little rotation globe I am trying to find a way to ‘3D digital native design’, although I am not a designer at all. But I am convinced the future is somewhere in that direction. We need a ‘digital design language’ for Mixed Reality. Maybe it’s rotating globes. Maybe it’s something else. But I am sure as hell about what it’s <em>not – </em>and that is floating 2D or 3D buttons or ‘devices’ resembling physical machinery. </p>
<p>Code, as per my trademark, can be found <a href="https://github.com/LocalJoost/ToggleDemo" target="_blank">here</a>. </p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/su2FSkyGD0o" height="1" width="1" alt=""/>2017-02-22T07:45:14.858+01:002http://dotnetbyexample.blogspot.com/2017/02/a-generic-toggle-component-for-hololens.htmlA behaviour for dynamically loading and applying image textures in HoloLens appshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/Tqev5KWZCoI/a-behaviour-for-dynamically-loading-and.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 11 Feb 2017 08:38:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-2805756850048207769<h2>Intro</h2> <p>After two nearly code-less posts it’s time for something more code-heavy, although it’s still out of my ordinary mode of operation: it’s fairly short and not much code. So rest assured, not the code equivalent of “War and Peace”, as usual ;)</p> <p>For both a customer app and one of my own projects I needed to be able to download images from an external source to use as texture on a Plane (this is a flat object with essentially only width and height). Now that’s not that hard – on the Unity scripting reference there’s a <a href="https://docs.unity3d.com/ScriptReference/WWW-texture.html" target="_blank">clear example how to do that</a>. But for my own project I need to make sure I could also change the image (so reload an image on a plane that already had loaded a texture before) and I must also be able to make sure the image was not distorted by width/height ratio differences between the Plane and the image. That required a radical different approach. </p> <h2>Enough talk: code!</h2> <p>The behaviour itself is rather small and simple, even if I say so myself. It starts as follows:</p><pre style="font-size: 12px">using UnityEngine;
public class DynamicTextureDownloader : MonoBehaviour
{
public string ImageUrl;
public bool ResizePlane;
private WWW _imageLoader = null;
private string _previousImageUrl = null;
private bool _appliedToTexture = false;
private Vector3 _originalScale;
void Start()
{
_originalScale = transform.localScale;
}
void Update()
{
CheckLoadImage();
}
}</pre>
<p>The ImageUrl is property you can either set from code or the editor and points to the location of the desired image on the web, ResizePlane (default false) determines whether or not you want the Plane to resize to fit the width/height ratio of the image. You may not always <em>want</em> that, as the <em>center</em> of the Plane stays in place. For instance, if the Plane’s top is aligned with something else. If the resizing makes the Plane’s height decrease, that may ruin your experience.</p>
<p>The other first three privates are status variables, the last one is the original scale of the plane before we started messing with it. We need to retain that, as can’t trust that scale once we start messing with it. I have seen the Plane become smaller and smaller when I alternated between portrait and landscape pictures.</p>
<p>The crux is the CheckLoadImage method:</p><pre style="font-size: 12px">private void CheckLoadImage()
{
// No image requested
if (string.IsNullOrEmpty(ImageUrl))
{
return;
}
// New image set - reset status vars and start loading new image
if (_previousImageUrl != ImageUrl)
{
_previousImageUrl = ImageUrl;
_appliedToTexture = false;
_imageLoader = new WWW(ImageUrl);
}
if (_imageLoader.isDone &amp;&amp; !_appliedToTexture)
{
// Apparently an image was loading and is now done. Get the texture and apply
_appliedToTexture = true;
Destroy(GetComponent&lt;Renderer&gt;().material.mainTexture);
GetComponent&lt;Renderer&gt;().material.mainTexture = _imageLoader.texture;
Destroy(_imageLoader.texture);
if (ResizePlane)
{
DoResizePlane();
};
}
}</pre>
<p>This might seem mightily odd if you are .NET developer, but that’s because of the nature of Unity. Keep in mind this method is called from Update, so it’s called 60 times per second. The flow is simple:</p>
<ul>
<li>If ImageUrl is null, just forget it
<li>If an ImageUrl is set <em>and </em>it is a new one, reset the two status variables and make a new WWW object. You can see this as a kind of WebClient. Key to know it’s async, and it has a <em>done</em> property, that only gets true when it’s downloading. So while it’s downloading, the next part is skipped
<li>If, however the WWW object is done, we will need to apply it to the texture, but only if we did not do so before. So then we actually apply it.</li></ul>
<p>So after the image is applied, the first if clause is false, because we have an ImageUrl. The second one is false, because the last loaded url is equal to the current one. And finally, the last if clause is false because the texture is applied. So although it’s called 60 times a second, it essentially does nothing. Until you change the ImageUrl.</p>
<p>An important note – you see that I first destroy the existing Render’s texture, then load the WWW’s texture into the renderer, and then destroy the WWWs texture again. If you are using a lot of these objects in one project <em>and</em> have them change image regularly, Unity’s garbage collection process cannot keep up and on a real device (i.e. a HoloLens) you will run out of memory soon. The nasty thing is this won’t happen soon in the editor or an emulator. This is why you always need to test on a real device. And this is also why I had to update this post later ;)</p>
<h2>Resizing in correct width/height ratio</h2>
<p>Finally the resizing, that’s not very hard it turns out. As long a you keep in mind the Plane’s ‘natural posture’&nbsp; is ‘flat on the ground’, so what you tend to think of a X is indeed X, but what you tend to think of as Y, is in fact Z in the 3D world. </p><pre style="font-size: 12px">private void DoResizePlane()
{
// Keep the longest edge at the same length
if (_imageLoader.texture.width &lt; _imageLoader.texture.height)
{
transform.localScale = new Vector3(
_originalScale.z * _imageLoader.texture.width / _imageLoader.texture.height,
_originalScale.y, _originalScale.z);
}
else
{
transform.localScale = new Vector3(
_originalScale.x, _originalScale.y,
_originalScale.x * _imageLoader.texture.height / _imageLoader.texture.width);
}
}</pre>
<p>It also turns out a loaded texture comes handily with it’s own size attributes, which makes it pretty easy do to the resize. </p>
<h2>Sample app</h2>
<p>I made this really trivial HoloLens app that shows two (initially empty) Planes floating in the air. I have given them different with/height ratios on purpose (in fact they mirror each other):</p>
<p><a href="https://lh3.googleusercontent.com/-wvhlzsiQoPM/WJ8-DE4H6qI/AAAAAAAAOtA/sBS-kUCD8k8/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-vVTm44B3xX0/WJ8-D62G1AI/AAAAAAAAOtE/OVIowJRA5Rs/image_thumb%25255B1%25255D.png?imgmax=800" width="644" height="348"></a></p>
<p>I have dragged the behaviour on both of them. One will show my blog’s logo (that’s a landscape picture) and one comes from an Azure Blob container and shows portrait oriented picture of… well, see for yourself. If you deploy this app – or just hit the play button in Unity, the will initially show this:</p>
<p><a href="https://lh3.googleusercontent.com/-u7iSvyVM8O4/WJ8-EYLACjI/AAAAAAAAOtI/fi5h7Hg0K7M/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-FUJlviuio_U/WJ8-EnG8UsI/AAAAAAAAOtM/mB0EJMNkMoA/image_thumb%25255B5%25255D.png?imgmax=800" width="644" height="284"></a></p>
<p>If you air tap on one of the pictures you get this:</p>
<p><a href="https://lh3.googleusercontent.com/-AbnwWlc46Hc/WJ8-FYh6k4I/AAAAAAAAOtQ/m6lwaXcg36A/s1600-h/image%25255B10%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-f_VjsKcEiCg/WJ8-F1-rPkI/AAAAAAAAOtU/Q5DlsY32CII/image_thumb%25255B4%25255D.png?imgmax=800" width="644" height="325"></a></p>
<p>In the second picture, the pictures are a lot larger as they fit ‘better’ into the pane. If you click a picture again they will swap back. The only thing that actually changes it the value of ImageUrl. </p>
<p>Bonus brownie points and eternal fame, by the way, for the first one who correctly tells me who the person in the picture is, and at what occasion this picture was taken :D.</p>
<h2>Some concluding remarks</h2>
<p>If you value your sanity, don’t mess with the rotation of the Planes <em>themselves</em>. Just pack them into an empty game object and rotate <em>that</em>, so the local coordinate system is still ‘flat’, as far as the Planes are concerned. I have had all kinds of weird effects if you start messing with Plane orientation and location. Not sure why this is, probably things I don’t quite understand that but – you have been warned.</p>
<p>This is (or will be part) of something bigger, but I wanted to share the lessons learned separately, preventing them to get lost in some bigger picture. In the mean time, you can view the <a href="https://github.com/LocalJoost/UnityTextureDownload" target="_blank">demo project with source here</a>. </p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/Tqev5KWZCoI" height="1" width="1" alt=""/>2017-02-22T13:50:39.887+01:000http://dotnetbyexample.blogspot.com/2017/02/a-behaviour-for-dynamically-loading-and.htmlUsing a HoloLens scanned room inside your HoloLens apphttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/OrPjeJXY-ko/using-hololens-scanned-room-inside-your.htmlHoloLensUnity3Dnoreply@blogger.com (Joost van Schaik)Fri, 03 Feb 2017 13:04:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-2446186333508681868<p>HoloLens can interact with reality – that’s why it’s a mixed reality device, after all. The problem is, sometimes, the reality you need is not always available. For example, if the app needs to run in a room or facility at a (client) location you only have limited access to. And you have to locate stuff on places relative to places in the room. Now you can of course use the simulation in the device portal and capture the room. </p> <p><a href="https://lh3.googleusercontent.com/-iqmfhq4u_-g/WJTwK9JieFI/AAAAAAAAOpE/w5NHi9HfO9M/s1600-h/image%25255B4%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-HguwOE42N40/WJTwLkh0h9I/AAAAAAAAOpI/SbpNK-8exZ4/image_thumb%25255B2%25255D.png?imgmax=800" width="654" height="421"></a></p> <p>You can save the room into an XEF file and upload that to (another) HoloLens. That works fine runtime, but in Unity that doesn’t help you much with getting a feeling of the space, and moreover, it messes with your HoloLens’ spatial mapping. I don’t like to use it on a real live HoloLens.</p> <p>There is another option though, in the 3D view tab</p> <p><a href="https://lh3.googleusercontent.com/-lyPD6orWqRQ/WJTwMdAOs2I/AAAAAAAAOpM/_7iGAV0-Q24/s1600-h/image%25255B8%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-_ZeXY9N1Ch0/WJTwM4niPhI/AAAAAAAAOpQ/8Lvp08eLz8s/image_thumb%25255B4%25255D.png?imgmax=800" width="654" height="524"></a></p> <p>If you click “update”, you will see a rendering of the space the HoloLens has recognized, belonging to the ‘space’ it finds itself in. Basically, the whole mesh. In this case, my ground and first floor of my house (I never felt like taking the HoloLens to 2nd floor). If you click “Save” it will offer to save a SpatialMapping.obj. That simple WaveFront Object format. And this is something you actually <em>can use in Unity</em>.</p> <p>Only it looks rather crappy. Even if you know what you are looking at. This is the side of my house, with left bottom the living (the rectangular thing is the large cupboard), on top of that the master bedroom* with the slanted roof, and if you look carefully, you can see the stairs coming up from the hallway a little right of center, at the bottom of the house. </p> <p><a href="https://lh3.googleusercontent.com/-RMLkvb5RuY0/WJTwN_HLmhI/AAAAAAAAOpU/n2EFicBXYLw/s1600-h/image%25255B13%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-D0IESvRI8Is/WJTwOveTqlI/AAAAAAAAOpY/hhU9IDL2fYA/image_thumb%25255B7%25255D.png?imgmax=800" width="654" height="462"></a></p> <p>What is also pretty confusing it the fact meshes can have only one side. This has the peculiar effect that at a lot of places you can look into the house from outside, but not outside the house from within. Anyway. This mesh is way too complex (the file is over 40mb) and messy. </p> <p><a href="https://lh3.googleusercontent.com/-U_zfMW7iq7s/WJXiNZyuIiI/AAAAAAAAOsI/ZXNje-r0YL0/s1600-h/image3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-6bOiOhpLxbA/WJXiN7Z9PQI/AAAAAAAAOsM/wbwAQDmmoEQ/image_thumb.png?imgmax=800" width="53" align="right" height="58"></a>Fortunately – there’s <a href="http://www.meshlab.net/" target="_blank">Meshlab</a>. And it’s free too. Thank heavens, because after you have bought a HoloLens you are probably quite of money ; )</p> <p>Meshlab has quite some tools to make your Mesh a bit smoother. Usually, when you look at a piece of mesh, like for instance the master bedroom, it looks kinda spikey – see left. But after choosing Filter/Remeshing, Simplication and Reconstruction/Simplification: Quadratic Edge Collapse Decimation</p> <p><a href="https://lh3.googleusercontent.com/-zsEw2JNrkUQ/WJXiOiVG3AI/AAAAAAAAOsQ/t-Osptgkgh0/s1600-h/image172.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-BDFOXNRvaHk/WJXiPVzM0vI/AAAAAAAAOsU/9wpig_1YW60/image17_thumb1.png?imgmax=800" width="342" height="308"></a><a href="https://lh3.googleusercontent.com/-SW0DGaPgKI8/WJXiQF8B2nI/AAAAAAAAOsY/ICrrbK2wMCM/s1600-h/image201.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-56TgbAOTVDY/WJXiQ4tyBoI/AAAAAAAAOsc/HVor1IUehEk/image20_thumb.png?imgmax=800" width="304" height="308"></a></p> <p>My house starts to look at lot less like the lair of the <a href="http://hyperioncantos.wikia.com/wiki/Shrike" target="_blank">Shrike</a> – it’s more like an undiscovered Antonio Gaudi building now. Hunt down the material used (in the materials subfolder), set it to transparent and play with color and transparency. I thought this somewhat transparent brown worked pretty well. Although there’s definitely still a lot of ‘noise’, it now definitely looks like my house, good enough for me to know where things are – or ought to be. </p><iframe height="365" src="https://www.youtube.com/embed/YrVnPaQjvOo" frameborder="0" width="650" allowfullscreen></iframe> <p>Using this Hologram of a space you can position <a href="http://dotnetbyexample.blogspot.com/2017/01/scanning-physical-objects-with-xbox-one.html" target="_blank">Kinect-scanned objects</a> or 3d models relative to each other based upon their true relative positions without actually being in the room. Then, when you go back to the real room, all you have to to is to make sure the actual room coincides with the scanned room model – add world anchors to the models inside the room, and then get rid of the room Hologram. Thus, you can use this virtual room as a kind of ‘staging area’, which I successfully did for a client location to which physical access is very limited indeed.</p> <p>You might notice a few odd things – there are two holes in the floor in the living room – that is where the black leather couch and the black swivel chair are. As I’ve noticed before, black doesn’t work well with the HoloLens spatial mapping. Fascinating I find also the rectangular area that seems to float about a meter from left side of the house. That’s actually a large <em>mirror</em> that hangs on the bedroom wall, but the HoloLens spatial mapping apparently sees it as a reclined area. Very interesting. So not only this gives you a view of my house, but also a bit about HoloLens quirks. </p> <p>The project showed above, with both models (the full and the simplified one) in it, <a href="https://github.com/LocalJoost/SpatialObjDemo" target="_blank">can be found here.</a></p> <p><font size="1">* I love using the phrase “master bedroom” in relation to our house – as it conjures up images of a very large room like found in a typical USA suburban family house. I can assure you neither our house nor our bedroom do justice to that image. This is a Dutch house. But is <em>is</em> made out of concrete, unlike most houses in the USA, and will probably last way beyond my lifespan</font></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/OrPjeJXY-ko" height="1" width="1" alt=""/>2017-02-05T16:30:15.251+01:000http://dotnetbyexample.blogspot.com/2017/02/using-hololens-scanned-room-inside-your.htmlScanning physical objects with an Xbox One Kinect to use as Holograms in HoloLenshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/tWilmoTHIoE/scanning-physical-objects-with-xbox-one.htmlHoloLensKinectnoreply@blogger.com (Joost van Schaik)Tue, 31 Jan 2017 03:28:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-4537276196905090091<h2>Intro</h2> <p><a href="https://lh3.googleusercontent.com/-xm394bxhGno/WJB0t4tfOtI/AAAAAAAAOnQ/uZHKsxDglbc/s1600-h/image%25255B2%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-9kND7Q3Ay7o/WJB0uZAJDzI/AAAAAAAAOnU/eMrWOCaAf2U/image_thumb.png?imgmax=800" width="244" align="right" height="215"></a>There are several demos out there that show obviously <i>scanned</i> models of people or physical objects used in HoloLens applications. I think it was fellow MVP and Dresdener 3D genius <a href="https://twitter.com/rschu" target="_blank">René Schulte</a> who first <a href="https://twitter.com/rschu/status/737767579108974593" target="_blank">used his bust in a demo</a> or at least went public with it. Unfortunately I have not been able to find very much about the actual process used to do the scanning and get to this result, so I have been trying to kludge together a procedure to make a full color 3d scan of a physical object – myself – and show it in a HoloLens.</p> <h2>Shopping list</h2> <p>For this you will need the following hardware: <ul> <li>A PC with a USB3 port. A Surface 4 Pro i7 has been successfully used for this goal <li><a href="https://www.microsoftstore.com/store/msusa/en_US/pdp/Kinect-Sensor-for-Xbox-One/productID.2267482500" target="_blank">A Kinect sensor for Xbox one, aka Kinect 2</a> <li><a href="https://www.microsoftstore.com/store/msusa/en_US/pdp/Kinect-Adapter-for-Xbox-One-S-and-Windows-PC/productID.2233937600" target="_blank">A Kinect Adapter for Xbox One S and Windows PC</a> </li></ul> <p>And the following software: <ul> <li>The <a href="https://www.microsoft.com/en-us/store/p/3d-scan/9nblggh68pmc" target="_blank">3D Scan app</a> (see below) <li>&nbsp;<a href="http://cloudcompare.org/release" target="_blank">CloudCompare Stereo</a> <li>This special <a href="https://forum.unity3d.com/attachments/unityvc-zip.213471/" target="_blank">shader</a> that I nicked from <a href="https://forum.unity3d.com/threads/standard-shader-with-vertex-colors.316529" target="_blank">this thread</a> on the Unity forms</li></ul> <h2>Setting up the hardware</h2> <p>The Kinect adapter, when you remove it from the box, seems to be quite an intricate contraption of two boxes and three wires, one of them permanently connected to one of the boxes. That box – I’ll call it box #1 - is about the size of a package of cigarettes, and is the power supply. There is a wire with a mains plug that needs to be connected to this box. The other wire – that is permanently attached to box # 1– has a round plug on the other side.That plug goes into box #2 (the one without fixed wire). It’s about the size of an overly thick Mars bar. Now we have one wire unaccounted for – one with a weird squarish plug on one side, and a USB-3 plug on the other side. The squarish part <i>also</i> goes into box #2, next to the round plug coming from the power supply. Now there’s only one hole unaccounted for - on the other side box #1 is another hole – that’s where you need to connect the actual Kinect 2. <p>Plug in the mains connected to box #1, and connect the USB plug coming from box #2 to your computer’s USB3.0 port. This will start the installation of a couple of drivers, and I seem to recall it also automatically installed the 3D Scan app that you also can find in the store here. </p> <h2>Downloading additional software</h2> <p>If the 3D scan app did not install automatically, you will need to install it from the Windows Store. Also, download and install <a href="http://cloudcompare.org/release/CloudCompareStereo_v2.8_setup_x64.exe" target="_blank">CloudCompare</a>. Finally download the <a href="https://forum.unity3d.com/attachments/unityvc-zip.213471/" target="_blank">shader</a>. That’s a zip file – we will need that later in the process. So store that in your downloads folder and unblock it. </p> <h2>The actual scanning</h2> <p>The 3D Scanning app is actually rather straightforward. You can set a few options, and I noticed that you have to fiddle a lot with the settings to get the effect you want. A higher scan resolution give a lower depth, for instance. Also, Kinect sometimes just loses track of the object it tries to track. What you do have to make sure is that there is plenty of light and prevent shadows, because although the actual object <i>tracking</i> is done via Kinect’s magic sensors, the overlaying of <i>color</i> is using the <i>camera</i> and that just needs light. And move slowly. <em>Very</em> slowly.</p> <p>I scanned myself sitting on a rotating chair and a 120 seconds settings, rotating very very slowly. For this I used the “Kinect Sensor-stationary” setting. When it’s done, it will open the 3D builder app. That usually complains about invalid geometries that need to be fixed. Click the popup, prepare to wait for quite a while, and then finally the 3D object will be ready for the next step </p> <h2>Converting to a format usable for Unity</h2> <p>The annoying thing is that although the 3D builder can save the scanned object as a WaveFront Object (obj) file – that is readable by Unity -&nbsp; it will strip any color from it. So we will need to take an in-between step. To that effect, don’t save the file in OBJ but in PLY format. Then start CloudCompare, and open the PLY file in that you just created. Hit file/save as and save the file now as an FBX file. That gives you a number of options – I usually just use FBX binary. </p> <h2><a href="https://lh3.googleusercontent.com/-0L4RlvRG7Xo/WJB0u9sCvdI/AAAAAAAAOnY/Fm6FyxvCgKU/s1600-h/image%25255B31%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-mDxJpsQp62U/WJB0vUiUgPI/AAAAAAAAOnc/8ACNYNTYVkQ/image_thumb%25255B21%25255D.png?imgmax=800" width="244" align="right" height="188"></a>Importing in Unity <i>and showing the colors</i></h2> <p>When you import this into Unity and drag the scanned object on the canvas, you will soon notice a few things about me </p> <ul> <li>I’m rather large – like statue-of-Roman-emperor-with-overly-inflated-ego large <li>I seem to hang at a random place at a random angle in the sky, even if position and rotation are 0,0,0 <li>I am … rather pale.</li></ul> <p>Getting to let me look less like a roman statue uses requires the special shader. Using my standard folder structure, I added a folder “Shaders” and copied the contents of the UnityVC zip file into it. Net result: </p> <p><a href="https://lh3.googleusercontent.com/-ki0Jl1Z5lvM/WJB0v-EM6iI/AAAAAAAAOng/mDOx2hM-oJs/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-IHBjrgWOmiY/WJB0waW4SII/AAAAAAAAOnk/n-scE6cJzG0/image_thumb%25255B3%25255D.png?imgmax=800" width="304" height="184"></a>&nbsp;</p> <p>Then in your assets folder, go to the folder where you have imported the scanned model into. In my case that’s App/Models:</p> <p><a href="https://lh3.googleusercontent.com/-ZGiZVWbCQ3o/WJB0w-tbe9I/AAAAAAAAOno/_wdub_b4Kog/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-OO5RH0wU90o/WJB0xWE79II/AAAAAAAAOns/nMUFHt8ajhk/image_thumb%25255B5%25255D.png?imgmax=800" width="304" height="196"></a></p> <p>In that you will find a Material ColorMaterial</p> <p><a href="https://lh3.googleusercontent.com/-VBiSf1LbT6o/WJB0x1N2KYI/AAAAAAAAOnw/aXqHpUhwkTM/s1600-h/image%25255B35%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-PmvkukeEpgo/WJB0yf2QSmI/AAAAAAAAOn0/naes1ZHMwUA/image_thumb%25255B23%25255D.png?imgmax=800" width="354" height="199"></a></p> <p>Select that Material, go into it’s properties in the Inspector over on the right and select either “Standard (Vertex Color)” or “Standard Specular (Vertex Color)”</p> <p><a href="https://lh3.googleusercontent.com/-mqVLKf01TjY/WJB0yz_vYdI/AAAAAAAAOn4/LBZltwl__Lk/s1600-h/image%25255B39%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-pK5zBYk-Q9I/WJB0zfba9jI/AAAAAAAAOn8/kp-ThVOgaf8/image_thumb%25255B25%25255D.png?imgmax=800" width="404" height="174"></a></p> <p>And boom. There I am, in full color color <strike>and glory</strike>&nbsp;<strike>looking like a zombie</strike>.</p> <p><a href="https://lh3.googleusercontent.com/-l5zwL0OSqMQ/WJB0z3lLH4I/AAAAAAAAOoA/DkoItuhsBe0/s1600-h/image%25255B43%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-dOtcnByqWZo/WJB00RfBqhI/AAAAAAAAOoE/ACwpZC6697A/image_thumb%25255B27%25255D.png?imgmax=800" width="354" height="320"></a></p> <p>Now to make me appear in front of the HoloLens view and not seem like some giant ancient, balding and grey god of… whatever descending from Heaven, I used the following settings on the scanned Hologram:</p> <p><a href="https://lh3.googleusercontent.com/-JVxSIj6AFZY/WJB018qKQHI/AAAAAAAAOoI/P5-luO4lWRA/s1600-h/image%25255B51%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-bHulTi0mYkE/WJB02Qbd0sI/AAAAAAAAOoM/QmCiWGyVCOo/image_thumb%25255B31%25255D.png?imgmax=800" width="354" height="129"></a></p> <p>And then you get a more or less life-sized floating ghost/zombie/Borg me-like appearance</p> <p><a href="https://lh3.googleusercontent.com/-GM9up6ydS6s/WJB026PaXfI/AAAAAAAAOoQ/OOK47oIwzqE/s1600-h/image%25255B52%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-B83Yh1jr_P4/WJB03cyRGnI/AAAAAAAAOoU/02Mm_zGE3mE/image_thumb%25255B32%25255D.png?imgmax=800" width="304" height="337"></a></p> <p>I had the dubious pleasure of walking around myself, noticing the back part was missing. </p><iframe height="365" src="https://www.youtube.com/embed/CjIPBTlSc_o" frameborder="0" width="650" allowfullscreen=""></iframe> <h2>Lessons learned</h2> <ul> <li>First of all – already mentioned in passing, <em>make sure there is enough light, and prevent shadows</em>. This is harder than it sounds. You will notice scanned Holograms tend to look rather pale when created with insufficient light. <li>The handheld setting is way harder to use than the stationary setting. Consider placing objects on a rotary platform rather than moving around it with the Kinect <li>Move or rotate the object you want to scan <em>slowly</em> (or move the Kinect slowly) <li>You will have to fiddle a lot with settings before you get the result you want <li>Larger objects (like humans) are way easier to scan than small objects <li>Make anything you don’t want on the scan as black as and non-reflective as possible</li></ul> <h2>Concluding remarks</h2> <p>Unity will complain regularly about an error in the shader in the editor, but it still seems to work fine. Having no clue about shaders and how to write them yet, I tend to ignore it. The resulting project, although containing no code written by me at all, <a href="https://github.com/LocalJoost/KinectScanDemo" target="_blank">can be found here</a>.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/tWilmoTHIoE" height="1" width="1" alt=""/>2017-02-24T07:52:18.809+01:002http://dotnetbyexample.blogspot.com/2017/01/scanning-physical-objects-with-xbox-one.htmlManipulating Holograms (move, scale, rotate) by gestureshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/8Eyx4TSCKYc/manipulating-holograms-move-scale.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Wed, 25 Jan 2017 03:15:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-2876854214767719033<h2>
Intro</h2>
This tutorial will show you how to manipulate (move, rotate and scale) Holograms by using gestures. It uses speech commands to change between moving, rotating and scaling. It will build upon <a href="http://dotnetbyexample.blogspot.com/2017/01/dragging-holograms-with-gaze-and.html" target="_blank">my previous blog post</a>, re-using the collision detection. The app we are going to create will work like this (turn on sound to hear the voice commands):<br />
<iframe allowfullscreen="" frameborder="0" height="365" src="https://www.youtube.com/embed/XCHK1T3AHVs" width="650"></iframe> <br />
<h2>
Setting the stage</h2>
Once again, we start off with a rather standard setup… <br />
<a href="https://lh3.googleusercontent.com/-9qtGVqs1ezg/WH9OH5-ab-I/AAAAAAAAOlc/22nbd0NjGks/s1600-h/clip_image001%25255B4%25255D%25255B2%25255D.png"><img alt="clip_image001[4]" border="0" height="220" src="https://lh3.googleusercontent.com/-Dv1So8wsUwQ/WH9OIecOM9I/AAAAAAAAOlg/6VgMrLNM5JQ/clip_image001%25255B4%25255D_thumb.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: inline; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="clip_image001[4]" width="215" /></a> <br />
only now we have a cube and a sphere. Neither of their settings are particularly spectacular, and this will show the sphere initially a bit to the left, and a rectangular box a bit to the right <br />
<a href="https://lh3.googleusercontent.com/-WXo0ASYt2bg/WH9OIyvKHXI/AAAAAAAAOlk/f28FcLfDRwI/s1600-h/clip_image002%25255B9%25255D%25255B16%25255D.png"><img align="left" alt="clip_image002[9]" border="0" height="107" src="https://lh3.googleusercontent.com/-fb5xsd0l0-Q/WH9OJVhr6MI/AAAAAAAAOlo/MlUIRoZ27W0/clip_image002%25255B9%25255D_thumb%25255B14%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: inline; float: left; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="clip_image002[9]" width="329" /></a><a href="https://lh3.googleusercontent.com/-rvyP2Gztt34/WH9OJ92vidI/AAAAAAAAOls/H98csOnUzH4/s1600-h/clip_image003%25255B4%25255D%25255B10%25255D.png"><img align="right" alt="clip_image003[4]" border="0" height="107" src="https://lh3.googleusercontent.com/-KjtsbjBqZ2k/WH9OKnURlGI/AAAAAAAAOlw/8be0tGhJWgM/clip_image003%25255B4%25255D_thumb%25255B8%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: inline; float: right; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="clip_image003[4]" width="329" /></a> <br />
&nbsp; <br />
&nbsp; <br />
&nbsp; <br />
&nbsp; <br />
If we actually want this Holograms to do something we need to code some stuff: <br />
<ul>
<li>A behaviour to select and activate Holograms – TapToSelect </li>
<li>A behaviour that acts on speech commands, so the next behaviour knows what to do – this is SpeechCommandExecuter </li>
<li>A behaviour that responds to tap-hold-and-move gesture and does the actual moving, rotating and scaling – this is SpatialManipulator </li>
<li>Something to wire the whole thing together, a kind of app state manager – AppStateManager and its base class, very originally called BaseAppStateManager.</li>
</ul>
<h2>
Selecting a hologram</h2>
The fun thing with the new <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a> is that we don’t need do much anymore to listen or track hand movements and speech. It’s all done by the InputManager. We only need to implement the right interfaces to be called with whatever we want to know. So when we want an object to receive a tap, we only need to add a behaviour that implements IInputClickHandler:<br />
<pre style="font-size: 12px;">public class TapToSelect : MonoBehaviour, IInputClickHandler
{
public virtual void OnInputClicked(InputEventData eventData)
{
if (BaseAppStateManager.IsInitialized)
{
// If not already selected - select, otherwise, deselect
if (BaseAppStateManager.Instance.SelectedGameObject != gameObject)
{
BaseAppStateManager.Instance.SelectedGameObject = gameObject;
}
else
{
BaseAppStateManager.Instance.SelectedGameObject = null;
}
var audioSource = GetAudioSource(gameObject);
if (audioSource != null)
{
audioSource.Play();
}
}
else
{
Debug.Log("No BaseAppStateManager found or initialized");
}
}
}</pre>
This basically just gives the object to the App State Manager – or more it’s base class, later more about it, and optionally plays a sound – if the omitted GetAudioSource method can find and AudioSource in either the object or its parent. In the app it does, it plays a by now very recognizable ‘ping’ confirmation sound. <br />
<h2>
Speech command the new way – listen to me, just hear me out*</h2>
Using speech commands has very much changed with the new HoloToolkit. There are actually two ways to go about it:
<br />
<ul>
<li>Using a SpeechInputSource and implementing an ISpeechHandler ‘somewhere’. This is rather straightforward and is very much and analogy of how the IInputClickHandler works. Disadvantage is that you have to define your keywords twice – both in the SpeechInputSource and in the ISpeechHandler implementation
</li>
<li>Using a KeywordManager to define your keywords and map them to some object’s method(s).</li>
</ul>
This sample uses the latter method. It’s a bit of an odd workflow to get it working, but once that’s clear, it’s rather elegant. It’s also more testable, as the <em>interpretation</em> of keywords is separated from the <em>execution</em>. We are implementing that execution in the SpeechCommandExecuter. Its public methods are Move, Rotate, Scale, Done, Faster and Slower which pretty much maps to the available speech commands. And if you look in the code, you will see what it does internally is just call private methods, which in turn try to find the selected objects’ SpatialManipulator and call methods there.<br />
<pre style="font-size: 12px;">private void TryChangeMode(ManipulationMode mode)
{
var manipulator = GetSpatialManipulator();
if (manipulator == null)
{
return;
}
if (manipulator.Mode != mode)
{
manipulator.Mode = mode;
TryPlaySound();
}
}
private void TryChangeSpeed(bool faster)
{
var manipulator = GetSpatialManipulator();
if (manipulator == null)
{
return;
}
if (manipulator.Mode == ManipulationMode.None)
{
return;
}
if (faster)
{
manipulator.Faster();
}
else
{
manipulator.Slower();
}
TryPlaySound();
}
private SpatialManipulator GetSpatialManipulator()
{
var lastSelectedObject = AppStateManager.Instance.SelectedGameObject;
if (lastSelectedObject == null)
{
Debug.Log("No selected element found");
return null;
}
var manipulator = lastSelectedObject.GetComponent&lt;SpatialManipulator&gt;();
if (manipulator == null)
{
manipulator = lastSelectedObject.GetComponentInChildren&lt;SpatialManipulator&gt;();
}
if (manipulator == null)
{
Debug.Log("No manipulator component found");
}
return manipulator;
}</pre>
So why this odd arrangement? That’s because KeywordManager needs an object with parameterless methods to call on keyword recognition. So, we add this SpeechCommandExecuter and a KeywordManager (from the HoloToolkit) to the Managers object, and then we are going to make this work. The easiest way to get going is
<br />
<ul>
<li>Expand “Keywords and Responses”,
</li>
<li>Change “size” initially in one.
</li>
<li>Type “move object” into keyword,
</li>
<li>Click + under “Response”
</li>
<li>Drag the “Managers” object in the now visible “None” field. This is best explained by an image:</li>
</ul>
<a href="https://lh3.googleusercontent.com/-S073nKoJ27c/WH9OLBqEbNI/AAAAAAAAOl0/EnvPB_0kPiI/s1600-h/clip_image002%25255B11%25255D%25255B4%25255D.png"><img alt="clip_image002[11]" border="0" height="486" src="https://lh3.googleusercontent.com/-v0ZKEohEWJA/WH9OLhzVL1I/AAAAAAAAOl4/fKtDXEM4B_I/clip_image002%25255B11%25255D_thumb%25255B2%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: inline; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="clip_image002[11]" width="604" /></a><br />
And then you have to select to what method of which object you want to map this speech command. To do this, click the dropdown menu next to “Runtime Only”, that will initially say “no function”. From the drop down first select the object you want (SpeechCommandExecuter) and then the method you want (Move).
<br />
<a href="https://lh3.googleusercontent.com/-VeJgXVA1BRU/WH9OMoxYFsI/AAAAAAAAOl8/P8L4cFNEV4U/s1600-h/clip_image004%25255B6%25255D.png"><img alt="clip_image004" border="0" height="362" src="https://lh3.googleusercontent.com/-flOvPmsJDQU/WH9ONeOxBrI/AAAAAAAAOmA/3bQgc5xKusE/clip_image004_thumb%25255B3%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: inline; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="clip_image004" width="304" /></a>
<br />
Unfortunately, all objects in the game object are displayed, as are all public methods and properties from every object you select – plus those of its parent classes. You sometimes really have to hunt them down. It’s a bit confusing at first, but once you have done it a couple of time you will get the hang of it. It might feel as an odd way of programming if you are used to the formal declarative approach of things in XAML, but that’s the way it is.
<br />
Then change the size to 6 and add all the other keyword/method combinations. By using this method you only have to drag the Managers object once, as the Unity editor will copy all values of the first entry to the new ones.<br />
<h2>
Spatial Manipulation</h2>
This is the behaviour that does most of the work. And it’s surprisingly simple. The most important (new) part is like this: <br />
<pre style="font-size: 12px;">using UnityEngine;
using HoloToolkit.Unity.InputModule;
namespace LocalJoost.HoloToolkitExtensions
{
public class SpatialManipulator : MonoBehaviour
{
public float MoveSpeed = 0.1f;
public float RotateSpeed = 6f;
public float ScaleSpeed = 0.2f;
public ManipulationMode Mode { get; set; }
public void Manipulate(Vector3 manipulationData)
{
switch (Mode)
{
case ManipulationMode.Move:
Move(manipulationData);
break;
case ManipulationMode.Rotate:
Rotate(manipulationData);
break;
case ManipulationMode.Scale:
Scale(manipulationData);
break;
}
}
void Move(Vector3 manipulationData)
{
var delta = manipulationData * MoveSpeed;
if (CollisonDetector.CheckIfCanMoveBy(delta))
{
transform.localPosition += delta;
}
}
void Rotate(Vector3 manipulationData)
{
transform.RotateAround(transform.position, Camera.main.transform.up,
-manipulationData.x * RotateSpeed);
transform.RotateAround(transform.position, Camera.main.transform.forward,
manipulationData.y * RotateSpeed);
transform.RotateAround(transform.position, Camera.main.transform.right,
manipulationData.z * RotateSpeed);
}
void Scale(Vector3 manipulationData)
{
transform.localScale *= 1.0f - (manipulationData.z * ScaleSpeed);
}
}
}</pre>
The manipulation mode can be either Move, Rotate, Scale – or None, in which case this behaviour does nothing at all. So, when ‘something’ supplies a Vector3 to the Manipulate method, it will either move, rotate or scale the object.
<br />
<ul>
<li>In move mode, when you move your hand, the object will follow the direction. So, if you pull towards you, it will come toward you. Move up, it will move up. Elementary.
</li>
<li>Scale is even more simple. Pull toward you, the object will grow, push from you, it will shrink.
</li>
<li>Rotate is a bit tricky. Push from you, the object will rotate around the horizontal axis. That is, an axis running through your view from left to right. Effectively, the top of the object will be moving <i>from</i> you and the bottom <i>to </i>you. Move your hand from left to right, or right to left, and the object will rotate around and axis that is running from top to bottom of your view. Last and most tricky – and least intuitive: move your hand from top to bottom and the object will rotate clockwise over the z axis – that is, the axis ‘coming out of your eyes’ </li>
</ul>
There are two more methods – Faster and Slower, which are called via the SpeechManager as you have seen, and their function is not very spectacular: they either multiply the speed value of the currently active manipulation mode by two, or divide it by two. So by saying “go faster” you will make the actual speed at which your Hologram moves, rotates or scales go twice as fast, depending on what you are doing. “Go slower” does the exact opposite.<br />
<pre style="font-size: 12px;">using UnityEngine;
using HoloToolkit.Unity.InputModule;
namespace LocalJoost.HoloToolkitExtensions
{
public class SpatialManipulator : MonoBehaviour
{
public float MoveSpeed = 0.1f;
public float RotateSpeed = 6f;
public float ScaleSpeed = 0.2f;
public ManipulationMode Mode { get; set; }
public void Manipulate(Vector3 manipulationData)
{
switch (Mode)
{
case ManipulationMode.Move:
Move(manipulationData);
break;
case ManipulationMode.Rotate:
Rotate(manipulationData);
break;
case ManipulationMode.Scale:
Scale(manipulationData);
break;
}
}
void Move(Vector3 manipulationData)
{
var delta = manipulationData * MoveSpeed;
if (CollisonDetector.CheckIfCanMoveBy(delta))
{
transform.localPosition += delta;
}
}
void Rotate(Vector3 manipulationData)
{
transform.RotateAround(transform.position, Camera.main.transform.up,
-manipulationData.x * RotateSpeed);
transform.RotateAround(transform.position, Camera.main.transform.forward,
manipulationData.y * RotateSpeed);
transform.RotateAround(transform.position, Camera.main.transform.right,
manipulationData.z * RotateSpeed);
}
void Scale(Vector3 manipulationData)
{
transform.localScale *= 1.0f - (manipulationData.z * ScaleSpeed);
}
}
}</pre>
It’s not quite rocket science as you can see. Notice the re-use of the collision detector <a href="http://dotnetbyexample.blogspot.com/2017/01/dragging-holograms-with-gaze-and.html" target="_blank">I introduced in my previous blog post</a>. <br />
<h2>
App state – the missing piece</h2>
The only thing missing now is how it’s all stitched together. That’s actually done using two classes – a BaseStateManager and a descendant, AppStateManager
<br />
The BaseStateManager doesn’t do much special. Its main feats are having a property for a selected object and notifying the rest of the world of getting one. And that’s not even used in this sample app but I consider it useful for other purposes, so I left it in. It also calls a virtual method if the selected object is changed.<br />
<pre style="font-size: 12px;">using System;
using HoloToolkit.Unity;
using UnityEngine;
namespace LocalJoost.HoloToolkitExtensions
{
public class BaseAppStateManager : Singleton&lt;BaseAppStateManager&gt;
{
private GameObject _selectedGameObject;
public GameObject SelectedGameObject
{
get { return _selectedGameObject; }
set
{
if (_selectedGameObject != value)
{
ResetDeselectedObject(_selectedGameObject);
_selectedGameObject = value;
if (SelectedObjectChanged != null)
{
SelectedObjectChanged(this,
new GameObjectEventArgs(_selectedGameObject));
}
}
}
}
protected virtual void ResetDeselectedObject(GameObject oldGameObject)
{
}
public event EventHandler&lt;GameObjectEventArgs&gt; SelectedObjectChanged;
}
}</pre>
There is also a class GameObjectEventArgs but that’s too trivial to show here. Note, by the way, I stick to C# 4.0 concepts as this is what Unity currently is limited to.<br />
The actual AppStateManager glues the whole thing together:<br />
<pre style="font-size: 12px;">public class AppStateManager : BaseAppStateManager, IManipulationHandler
{
void Start()
{
InputManager.Instance.AddGlobalListener(gameObject);
}
public static new AppStateManager Instance
{
get { return (AppStateManager)BaseAppStateManager.Instance; }
}
protected override void ResetDeselectedObject(GameObject oldGameObject)
{
var manipulator = GetManipulator(oldGameObject);
if (manipulator != null)
{
manipulator.Mode = ManipulationMode.None;
}
}
public void OnManipulationUpdated(ManipulationEventData eventData)
{
if (SelectedGameObject != null)
{
var manipulator = GetManipulator(SelectedGameObject);
if (manipulator != null)
{
manipulator.Manipulate(eventData.CumulativeDelta);
}
}
}
protected SpatialManipulator GetManipulator(GameObject obj)
{
if (obj == null)
{
return null;
}
var manipulator = obj.GetComponent&lt;SpatialManipulator&gt;() ??
obj.GetComponentInChildren&lt;SpatialManipulator&gt;();
return manipulator;
}
}</pre>
It implements the IManipulationHandler, which means our almighty IManipulationHandler will call it’s OnManipulationUpdated whenever it detects hand with a tap-and-hold gesture (thumb and index finger pressed together) while moving. And it will give that data to the SpatialManipulator in the select object, that is – if the selected object <i>has</i> one. It also makes sure the currently active object gets deactivated once you select a new one. Note, IManipulationHandler requires you to implement three more methods, omitted here, as they are not used in this app.
<br />
There is an important line in the Start method, that will define this object as a <em>global input handler</em>. Its OnManipulationUpdated always gets called. Normally, this get only called when it is selected – that is, if your gaze strikes the object. That makes it very hard to move it, as your gaze most likely will most likely fall <em>off</em> the object as you move it. This approach has the advantage you can even manipulate objects even if you are not exactly looking at them.
<br />
<h2>
Wiring it all together in Unity </h2>
<a href="https://lh3.googleusercontent.com/-Bblu2wB25ew/WH9sRGGSknI/AAAAAAAAOmc/Wx4YIQZBYbA/s1600-h/image3%25255B1%25255D.png"><img align="right" alt="image" border="0" height="328" src="https://lh3.googleusercontent.com/-DewL4r04cPc/WH9sRifqnsI/AAAAAAAAOmg/k8C2uPEl6Nc/image3_thumb.png?imgmax=800" style="background-image: none; border-bottom: 0px; border-left: 0px; border-right: 0px; border-top: 0px; display: inline; float: right; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="image" width="349" /></a>This is actually really simple now we have all the components. Just go to the Cube in your hierarchy and add these three components:<br />
<br />
And don’t forget to drag the InputManager and the Cube itself on the Stabilizer and the Collision Detector fields as I explained in my previous blog post. Repeat for the Sphere object. Build your app, and you should get the result I show in the video.<br />
<h2>
Concluding remarks</h2>
It’s fairly easy to wire together something to move stuff around using gestures. There are a few limitations as to the intuitiveness of the rotation gesture, and you might also notice that while moving the object around uses collision detection, rotating and scaling do not. I leave those as ‘exercise to the reader’ ;). But I do hope this takes you forward as a HoloLens developer.<br />
Full code <a href="https://github.com/LocalJoost/HoloManipulateByGesture" target="_blank">can be found here</a><br />
<br />
<br />
<span style="font-size: xx-small;">*bonus points if you actually immediately recognized this phrase</span><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/8Eyx4TSCKYc" height="1" width="1" alt=""/>2017-02-22T13:51:22.634+01:009http://dotnetbyexample.blogspot.com/2017/01/manipulating-holograms-move-scale.htmlDragging holograms with gaze and tapping them in place on a surfacehttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/cKlfUYVfryM/dragging-holograms-with-gaze-and.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Tue, 17 Jan 2017 23:00:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-6136471972720356548<h2>Intro</h2> <p>Simply put – what I was trying to do create is the effect of what you get in the – in the meantime good old – Holograms app: when you pull a hologram out a menu, it ‘sticks to your gaze’ and follows it. You can air tap and then it stays hanging in the air where you left it, but you can also put it on a floor, on a table, or next to a wall. You can’t push it through a surface. That is, most of the time ;). So, like this: <p><iframe height="365" src="https://www.youtube.com/embed/en1DzTZXdU4" frameborder="0" width="650" allowfullscreen></iframe></p> <p>In the video, you can see it follows the gaze cursor floating through the air till it hits a wall to the left and then stops, then goes down till it hits the bed and then stops, then up again till I finally place it on the floor.</p> <h2>A new year, a new toolkit</h2> <p>As happens often in the bleeding end of technology, things tend to change pretty fast. This is also the case in HoloLens country. I have taken the plunge to Unity 5.5 and the new <a href="https://github.com/Microsoft/HoloToolkit-Unity">HoloToolkit</a> which has a few breaking changes. Things have gotten <i>way </i>simpler since the previous iteration. Also, I would like to point out that for this tutorial I am using <a href="https://unity3d.com/unity/qa/patch-releases">the latest patch release</a>, which at the time of this writing it 5.5.0p3, released December 22, 2016. </p> <h2>Setting up the initial project</h2> <p>This is best illustrated by a picture. If you have setup the project we basically only need this. Both Managers and HologramCollection are simply empty game objects meant to group stuff together, then don’t have any specific other function here. Drag and drop the four blue prefabs in the indicated places, then set some properties for the cube <p><a href="https://lh3.googleusercontent.com/-RgfMEgPGNz0/WH8dDRfnh0I/AAAAAAAAOlI/eL3c18z0e-Q/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-ylgbNWYC15U/WH8dD0Ws0DI/AAAAAAAAOlM/Tvi_YallNdU/image_thumb%25255B1%25255D.png?imgmax=800" width="161" height="210"></a><a href="https://lh3.googleusercontent.com/-yrUi4xeXyDo/WG6fkJSabKI/AAAAAAAAOkE/dIp4ZnvZbxk/s1600-h/image%25255B9%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 13px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-SD-ajNS86ZM/WG6fkvOOPOI/AAAAAAAAOkI/-sFp_871RNU/image_thumb%25255B5%25255D.png?imgmax=800" width="421" height="124"></a></p> <p>The Cube is the thing that will be moved. Now it’s time for ‘some’ code.</p> <h2>The main actors</h2> <p>There are two scripts that play the leading role, with a few supporting roles. </p> <ul> <li>MoveByGaze <li>IntialPlaceByTap</li></ul> <p>The first one makes an object move, the second one actually ends it. Apropos, the actual moving is done by our old friend <a href="http://itween.pixelplacement.com/gettingstarted.php">iTween</a>, whose usefulness and application was already described in <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-5-smooth.html">part 5 of the AMS HoloATC series</a>. So, you will need to include this in the project to prevent all kind of nasty errors. Anyway, let’s get to he star of the show, MoveByGaze. </p> <h2>Moving with gaze</h2> <p>It starts like this:</p><pre style="font-size: 12px">using UnityEngine;
using HoloToolkit.Unity.InputModule;
using HoloToolkit.Unity.SpatialMapping;
namespace LocalJoost.HoloToolkitExtensions
{
public class MoveByGaze : MonoBehaviour
{
public float MaxDistance = 2f;
public bool IsActive = true;
public float DistanceTrigger = 0.2f;
public BaseRayStabilizer Stabilizer = null;
public BaseSpatialMappingCollisionDetector CollisonDetector;
private float _startTime;
private float _delay = 0.5f;
private bool _isJustEnabled;
private Vector3 _lastMoveToLocation;
private bool _isBusy;
private SpatialMappingManager MappingManager
{
get { return SpatialMappingManager.Instance; }
}
void OnEnable()
{
_isJustEnabled = true;
}
void Start()
{
_startTime = Time.time + _delay;
_isJustEnabled = true;
if (CollisonDetector == null)
{
CollisonDetector =
gameObject.AddComponent&lt;DefaultMappingCollisionDetector&gt;();
}
}
}
}</pre>
<p>Up above are the settings:
<ul>
<li>MaxDistance is the maximum distance from your head the behaviour will try to place the object on a surface. Further than that, and it will just float in the air.
<li>IsActive determines whether the behaviour is active (duh)
<li>DistanceTrigger is the distance your gaze has to be from the object your are moving, before it actual starts to move. It kind of trails your gaze. This prevents the object from moving in a very nervous way.
<li>Stabilizer is the stabilizer made, used and maintained by the InputManager. You will have to drag the InputManager from your scene on this field to use the stabilizer. It’s not mandatory, but <i>highly </i>recommended
<li>CollisionDetector is a class we will see later – it basically makes sure the object that you are dragging is not pushed through any surfaces. You will need to add a collision detector to the game object that you are dragging along – or maybe a game object that is part of the game object that you are dragging. That collision detector needs then to be dragged on this field on the MoveByGaze This is <i>not</i> mandatory. If you don’t add one, the object you attach the MoveByGaze to will just simply follow your gaze, and move right through any object. That’s the work of the DefaultMappingCollisionDetector who is essentially a <a href="https://en.wikipedia.org/wiki/Null_Object_pattern">null pattern</a> implementation.&nbsp; </li></ul>
<p>Anyway, in the Update method all the work is done: <pre style="font-size: 12px">void Update()
{
if (!IsActive || _isBusy || _startTime &gt; Time.time)
return;
_isBusy = true;
var newPos = GetPostionInLookingDirection();
if ((newPos - _lastMoveToLocation).magnitude &gt; DistanceTrigger || _isJustEnabled)
{
_isJustEnabled = false;
var maxDelta = CollisonDetector.GetMaxDelta(newPos - transform.position);
if (maxDelta != Vector3.zero)
{
newPos = transform.position + maxDelta;
iTween.MoveTo(gameObject,
iTween.Hash("position", newPos, "time", 2.0f * maxDelta.magnitude,
"easetype", iTween.EaseType.easeInOutSine, "islocal", false,
"oncomplete", "MovingDone", "oncompletetarget", gameObject));
_lastMoveToLocation = newPos;
}
else
{
_isBusy = false;
}
}
else
{
_isBusy = false;
}
}
private void MovingDone()
{
_isBusy = false;
}</pre>
<p>Only if the behaviour is active, not busy, and the first half second is over we are doing anything at all. And the first thing is – telling the world we are busy indeed. Thid method, like all Updates, is called 60 times a second and we want to keep things a bit controlled here. Race conditions are annoying.
<p>Then we get a position in the direction the user is looking, and if that exceeds the distance trigger – or this is the first time we are getting here – we start off finding how far ahead along this gaze we can place the actual object by using CollisionDetector. If that’s is possible – that is, if the CollisionDetector does not find any obstacles, we can actually move the object using iTween. Important is to note that whenever the move is <i>not</i> possible, _isBusy immediately gets set to false. Also, note the fact that the smaller the distance, the faster the move. This is to make sure the final tweaks of setting the object in the right place don’t take a long time. Otherwise, _isBusy is only reset after a successful move.
<p>Then the final pieces of this behaviour: <pre style="font-size: 12px">private Vector3 GetPostionInLookingDirection()
{
RaycastHit hitInfo;
var headReady = Stabilizer != null
? Stabilizer.StableRay
: new Ray(Camera.main.transform.position, Camera.main.transform.forward);
if (MappingManager != null &amp;&amp;
Physics.Raycast(headReady, out hitInfo, MaxDistance, MappingManager.LayerMask))
{
return hitInfo.point;
}
return CalculatePositionDeadAhead(MaxDistance);
}
private Vector3 CalculatePositionDeadAhead(float distance)
{
return Stabilizer != null
? Stabilizer.StableRay.origin + <br> Stabilizer.StableRay.direction.normalized * distance
: Camera.main.transform.position +
Camera.main.transform.forward.normalized * distance;
}</pre>
<p>GetPostionInLookingDirection first tries to get the direction in which you are looking. It tries to use the Stabilizer’s StableRay for that. The Stabilizer is a component of the InputManager that stabilizes your view – and the cursor uses it as well. This prevents the cursor from wobbling too much when you don’t keep your head perfectly still (which most people don’t – this includes me). The stabilizer takes an average movement of 60 samples and that makes for a much less nervous-looking experience. If you don’t have a stabilizer defined, it just takes your actual looking direction – the camera’s position and your looking direction.
<p>Then it tries to see if the resulting ray hits a wall or a floor – but no further than MaxDistance away. If it sees a hit, it returns this point, if it does not, if gives a point in the air MaxDistance away along an invisible ray coming out of your eyes. That’s what CalculatePositionDeadAhead does – but also trying to use the Stabilizer first to find the direction.</p>
<h2>Detect collisions</h2>
<p>Okay, so what is this famous collision detector that prevents stuff from being pushed through walls and floors, using the spatial perception that makes the HoloLens such a unique device? It’s actually very simple, although it took me a while to actually <em>get</em> it this simple.</p><pre style="font-size: 12px">using UnityEngine;
namespace LocalJoost.HoloToolkitExtensions
{
public class SpatialMappingCollisionDetector : BaseSpatialMappingCollisionDetector
{
public float MinDistance = 0.0f;
private Rigidbody _rigidbody;
void Start()
{
_rigidbody = GetComponent&lt;Rigidbody&gt;() ?? gameObject.AddComponent&lt;Rigidbody&gt;();
_rigidbody.isKinematic = true;
_rigidbody.useGravity = false;
}
public override bool CheckIfCanMoveBy(Vector3 delta)
{
RaycastHit hitInfo;
// Sweeptest wisdom from
//http://answers.unity3d.com/questions/499013/cubecasting.html
return !_rigidbody.SweepTest(delta, out hitInfo, delta.magnitude);
}
public override Vector3 GetMaxDelta(Vector3 delta)
{
RaycastHit hitInfo;
if(!_rigidbody.SweepTest(delta, out hitInfo, delta.magnitude))
{
return KeepDistance(delta, hitInfo.point); ;
}
delta *= (hitInfo.distance / delta.magnitude);
for (var i = 0; i &lt;= 9; i += 3)
{
var dTest = delta / (i + 1);
if (!_rigidbody.SweepTest(dTest, out hitInfo, dTest.magnitude))
{
return KeepDistance(dTest, hitInfo.point);
}
}
return Vector3.zero;
}
private Vector3 KeepDistance(Vector3 delta, Vector3 hitPoint)
{
var distanceVector = hitPoint - transform.position;
return delta - (distanceVector.normalized * MinDistance);
}
}
}
</pre>
<p>This behaviour first tries to find a RigidBody, and failing that, adds it. We will need this to check the presence of anything ‘in the way’. But – this is important – we will set ‘isKinematic’ to true and ‘useGravity’ to false, or else or object will come under control of the Unity physics engine and drop on the floor. In this case, <i>we</i> want to control the movement of the object.
<p>So, this class has two public methods (it’s abstract base class demands that). One, CheckIfCanMoveBy (that we don’t use now), just says if you can move your object in the intended direction over the intended distance without hitting anything. The other essentially does the same, but if it finds something in the way, it also tries to find a distance over which you <i>can </i>move in the desired direction. For this, we use the SweepTest method of RigidBody. Essentially you give it a vector, a distance along that vector, and it has an out variable that gives you info about a hit, should any occur. If a hit <i>does</i> occur, it tries at again at 1/4th, 1/7th and 1/10th of that initially found distance. Failing everything, it returns a zero vector. By using this rough approach, and object moves quickly in a few steps till it can no more.
<p>And then it also moves the object back over a distance you can set from the editor. This keeps the object just a little above the floor or from the wall, show that be desired. That’s what KeepDistance is for.
<p>The whole point of having a base class BaseSpatialMappingCollisionDetector, by the way, is a) enabling null pattern implementation which as implemented by DefaultMappingCollisionDetector and b) make different collision detectors based upon different needs. A bit of architectural considerations within the sometimes-bewildering universe of Unity development.
<h2>Making it stop – InitialPlaceByTap</h2>
<p>Making the MoveByGaze stop is very simple – set the IsActive field to false. Now we only need something to actually make that happen. With the new HoloToolkit, this is actually very very simple:</p><pre style="font-size: 12px">using UnityEngine;
using HoloToolkit.Unity.InputModule;
namespace LocalJoost.HoloToolkitExtensions
{
public class InitialPlaceByTap : MonoBehaviour, IInputClickHandler
{
protected AudioSource Sound;
protected MoveByGaze GazeMover;
void Start()
{
Sound = GetComponent&lt;AudioSource&gt;();
GazeMover = GetComponent&lt;MoveByGaze&gt;();
InputManager.Instance.PushFallbackInputHandler(gameObject);
}
public void OnInputClicked(InputEventData eventData)
{
if (!GazeMover.IsActive)
{
return;
}
if (Sound != null)
{
Sound.Play();
}
GazeMover.IsActive = false;
}
}
}</pre>
<p>By implementing IInputClickHandler the InputManager will send an event to this object when you air tap <em>and </em>it is selected by gaze. But by pushing it as fallback handler it will get this event <em>also</em> when it’s not selected. The event processing is pretty simple – if the GazeMover in this object is active, it’s de-activated. Also, if there’s an AudioSource detected, it’s sound is played. I very much recommend this kind of audio feedback.</p>
<h2>Wiring it all together</h2>
<p>On your cube, you drag the MoveByGaze, SpatialMappingCollisionDetector, and InitialPlaceByTap scripts. Then you drag the cube itself again on the CollisionDetector field of MoveByGaze, and the InputManager on the Stabilizer field. Unity itself will select the right component.</p>
<p><a href="https://lh3.googleusercontent.com/-PjQZ_i9MgCc/WHJH-sVUhvI/AAAAAAAAOkk/nNHBLUHxksk/s1600-h/image%25255B4%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-MF5W9I776GI/WHJH_NapeLI/AAAAAAAAOko/z0ZCKst6TSM/image_thumb%25255B2%25255D.png?imgmax=800" width="654" height="630"></a></p>
<p>So, in this case I could also have used GetComponent&lt;SpatialMappingCollisionDetector&gt; in stead of a field where you need to drag something on. But this way is more flexible – in app I did not want to use the whole object’s collider, but only that of a child object. Note I have set the MinDistance for the SpatialMappingCollisionDetector for 1 cm – it will keep an extra centimeter distance from the wall or the floor.</p>
<h2>Concluding remarks</h2>
<p>So this is how you can more or less replicate part of the behavior of the Holograms App, by moving around objects with your gaze and placing them on surfaces using air tap. The unique capabilities of the HoloLens allow us to place objects next to or on top of physical objects, and the new HoloToolkit makes using those capabilities pretty easy.</p>
<p>Full code, as per my MVP ‘trademark’, <a href="https://github.com/LocalJoost/HoloDragByGaze" target="_blank">can be found here</a></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/cKlfUYVfryM" height="1" width="1" alt=""/>2017-03-01T11:18:13.994+01:001http://dotnetbyexample.blogspot.com/2017/01/dragging-holograms-with-gaze-and.htmlUnity 5.5/HoloLens: AudioPluginMsHRTF.dll has failed the AppContainerCheck checkhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/4AJ_E0uXDxI/unity-55hololens-audiopluginmshrtfdll.htmlHoloLensSpatial SoundUnity3DUWPnoreply@blogger.com (Joost van Schaik)Wed, 28 Dec 2016 06:20:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-5388813753192183057<p>All right, so I was finally past a couple of deadlines with paid HoloLens apps and was ready to take the plunge to update my app <a href="ms-windows-store://pdp/?ProductId=9NBLGGH52SZP" target="_blank">AMS HoloATC</a> to Unity 5.5 and the newest <a href="https://github.com/Microsoft/HoloToolkit-Unity" target="_blank">HoloToolkit</a>. After wrestling my way past some some interesting breaking changes (more about that later), I was finally at the point where everything worked again as it did before – and started adding new features. Today was the day I thought it was ready enough to justify a new submission to the store. I had tested everything thoroughly, so what could possibly go wrong.</p> <p><a href="https://lh3.googleusercontent.com/-74KF2g0Rc_Y/WGPKPdNBsWI/AAAAAAAAOi8/FIvRKDZjqJo/s1600-h/image%25255B12%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-kqkqUUcvc54/WGPKQHWw7OI/AAAAAAAAOjA/5TyBXLT5cOo/image_thumb%25255B6%25255D.png?imgmax=800" width="654" height="416"></a></p> <p><img title="View source image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="Image result for wtf" src="http://evolutionaryparenting.com/wp-content/uploads/2015/01/picard-wtf.jpg" width="182" align="right" height="143"></p> <p>Why, if it ain’t my dear old friend the WACK. The 'binary analyzer' has found a file <strong>AudioPluginMsHRTF.dll</strong> that does not pass the AppContainerCheck and the WACK gives me a whole bunch of suggestions that don't mean anything to me. Apparently I have to add a couple of options when I link the app, but I have not seen a linker since I wrote my last C program, which was – I think – before 2000. Now I know the WACK has some issues, so I tried submitting it anyway – but no dice. The Windows Store spitted it right back to me, with the same error. So now what?</p> <p>Fortunately I have had <a href="http://dotnetbyexample.blogspot.com/2016/11/a-hololens-airplane-tracker-7activating.html" target="_blank">dealt with Spatial Sound before</a> and recognized the “Ms HRTF Spatializer” as something I just selected in Unity<em>. </em>After numerous (time consuming) failures trying to juggle the WACK, I took a desperate measure. I went back to a machine that had a previous installation of Unity on it – 5.4 – and checked AudioPluginMsHRTF.dll there. I found it in <br>C:\Program Files\Unity HoloLens 5.4.0f3-HTP\Editor\Data\VR\Unity\WindowsStoreApps. It reports being its size as 2931200 bytes<br>For Unity 5.5, in <br>C:\Program Files\Unity550f3\Editor\Data\UnityExtensions\Unity\VR\WindowsStoreApps\uap_x86, there’s a file of 13312 bytes big. Only 13kb. Yet, when you deploy and app with this only 13kb dll to a HoloLens Spatial Sound works. You can clearly hear the difference – simple stereo versus spatial sound.</p> <p>So I went about and looked around where this dll is taken from when Visual Studio project is built, and found it comes from the plugins directory in the project directory of the generated solution. In my case that’s C:\Projects\AMS_HoloATC\AMS HoloATC\App\AMS HoloATC\Plugins\X86. I replaced the 13kb AudioPluginMsHRTF.dll by the one I had taken from the 5.4 installation. That fixed the WACK. Unfortunately, it also made the app crash whenever I used Spatial Sound. Nice try, no cigar. But it conformed my suspicions that the 5.5 version of this dll is indeed the thing that makes the WACK protest.</p> <p><a href="https://lh3.googleusercontent.com/-T6YS8Jhe10k/WGQLXhwKHLI/AAAAAAAAOjU/Jwg6SYL5Rfw/s1600-h/image%25255B5%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 7px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-3ZTkqjglduM/WGQLX8lupZI/AAAAAAAAOjY/x3vHCHFUV64/image_thumb%25255B3%25255D.png?imgmax=800" width="194" align="left" height="244"></a>So what I did, in the end, was just get rid of ‘real’ Spatial Sound. I went to Edit/Project settings in Unity and set “Spatializer Plugin” to “None”, disabling the Ms HRTF Spatializer. Re-created the Visual Studio solution from complete scratch, and the reference to AudioPluginMsHRTF.dll and AudioPluginMsHRTF.pdb were gone. And so was the WACK error.</p> <p>But of course, now my app has no real spatial sound anymore – it’s reduced to ‘normal’ Unity Stereo Panning. But a running app is better than one that does not. I have filed a bug at Unity with a repro and contacted some people within Microsoft. I will keep you posted about any progress. In the mean time, this should keep you going forward.</p> <p>For those who want to try out themselves, I have made this <a href="https://1drv.ms/u/s!AjesCJDYqwYk1NlSQpOzMEG6Qvxu-w" target="_blank">zipped up solution including all the files I built</a>, and you can easily check the resulting app does not pass the WACK. </p> <p><strong>Update 30-dec-2016 </strong>– the issue has been <a href="https://issuetracker.unity3d.com/product/unity/issues/guid/865507/" target="_blank">confirmed as a bug by Unity</a> and will be addressed.</p> <p><strong>Update 31-jan-2016 </strong>– the issue has been fixed – in Unity 5.5.1p2 or higher.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/4AJ_E0uXDxI" height="1" width="1" alt=""/>2017-02-17T17:55:43.813+01:006http://dotnetbyexample.blogspot.com/2016/12/unity-55hololens-audiopluginmshrtfdll.htmlA HoloLens airplane tracker 8–adding the church and the billboardhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/CtaFQguiGLo/a-hololens-airplane-tracker-8adding.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 19 Nov 2016 12:10:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-2046027210015051092<h2>Intro</h2> <p>All the code to track aircraft in a HoloLens is already yours. Now the only thing I have left to write is how the <a href="http://www.wortell.nl/" target="_blank">Wortell</a> Church and the billboard that appears on the map came to be. It’s not technically a part of the aircraft tracker per se, but a fun exercise in code that you might use if your company is near an airport too.</p> <h2>Required assets</h2> <p>We need two things: </p> <ul> <li>A Church <li>A Billboard (and a logo to put on it)</li></ul> <h2>Getting and importing the church</h2> <p>This was actually the hardest one. It turns on that there are lots of churches on <a href="https://www.cgtrader.com/">cgtrader</a>, but if you download for instance <a href="https://www.cgtrader.com/free-3d-models/architectural-exterior/historic/church-in-vasteras--2">this church</a> as FBX or OBJ and import it into Unity, you don’t get the promised church on the left– you get the ghost thing on the right! </p> <p><a href="https://lh3.googleusercontent.com/-eNiLtzANBNQ/WDCxijILlCI/AAAAAAAAOb8/UQjfEw0qSi8/s1600-h/image%25255B16%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-L8_Mjs4QdcU/WDCxi-Q4CsI/AAAAAAAAOcA/aCy9WJ1uC48/image_thumb%25255B9%25255D.png?imgmax=800" width="221" align="right" height="334"></a></p><a href="https://lh3.googleusercontent.com/-qL3VaZjFid8/WDCxjUI6VCI/AAAAAAAAOcE/-SHj00kOj14/s1600-h/image%25255B13%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/--oHSNhvj9tA/WDCxj3vkPZI/AAAAAAAAOcI/xwEdq9Z5Xh4/image_thumb%25255B6%25255D.png?imgmax=800" width="304" height="334"></a> <p>As one of my colleagues - who lived in the UK for a long time before joining us - would put it: “<a href="https://twitter.com/SoVeryBritish/status/517736334841352192">it’s not quite what I had in mind</a>”. And this one actually looks quite good – I have seen churches with not only textures but complete parts missing. Apparently not everyone checks the result of every format they convert to before uploading. Fortunately, the author writes he or she has created the model in Sketchup. Ain’t that nice, as you can <a href="http://www.sketchup.com/download">download a free trial version</a> of that program. So download that – and the original Sketchup model. Unpack the zip file, and rename the 3d-model.skp that’s in there to church.skp. Then double-click from the explorer and Sketchup will open, after possibly asking you about trial licensing first. I had to zoom out a little, but this is what I got</p> <p><a href="https://lh3.googleusercontent.com/-gypMyH3PKwk/WDCxkTm__0I/AAAAAAAAOcM/7Pz5XZylAUo/s1600-h/image%25255B21%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-ovm9wU7TFwo/WDCxkrxYf1I/AAAAAAAAOcQ/ld8WmnkmhjU/image_thumb%25255B12%25255D.png?imgmax=800" width="654" height="354"></a></p> <p><a href="https://lh3.googleusercontent.com/-yLBIa7kbxvw/WDCxlLifW-I/AAAAAAAAOcU/ef_Lv75VCMM/s1600-h/image%25255B32%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-N2sH_nwiZww/WDCxljYmXBI/AAAAAAAAOcY/KP0vY36TTHM/image_thumb%25255B16%25255D.png?imgmax=800" width="304" align="right" height="62"></a>And that was indeed what I had in mind. Now hit File/Export/3D model, select FBX file (*.fbx) for “Save as “type”, choose a name that suits you and hit save</p> <p>If you would import the resulting church.fbx into Unity you get this: </p> <p><a href="https://lh3.googleusercontent.com/-hwT7DQJ9mts/WDCxmP80k_I/AAAAAAAAOcc/AoqqonFQapo/s1600-h/image%25255B33%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 8px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-l2WJCs7Ncb0/WDCxmhTsHnI/AAAAAAAAOcg/uCIKEHu697I/image_thumb%25255B17%25255D.png?imgmax=800" width="189" align="left" height="304"></a></p> <p><a href="https://lh3.googleusercontent.com/-4oqmYDjoWbQ/WDCxnESrG4I/AAAAAAAAOck/FSXg_wK-kt4/s1600-h/image%25255B40%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 6px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-Uzj2r4aHC7Q/WDCxnhaFqjI/AAAAAAAAOco/TRaVkUxa8XI/image_thumb%25255B19%25255D.png?imgmax=800" width="244" align="left" height="169"></a>Although not nearly as pretty as in Sketchup, that’s more like it! To keep your project clean and make it easier to swap out this church for another one (or other building)later, don’t drag the FBX into the root of your assets, but first add a folder, <em>then</em> drag in the church, like displayed on the right.This way, the materials of all the imported objects don’t get mixed up. </p> <p>To keep the church and the (future) billboard together, I created an empty GameObject “Wortell” in HoloGramCollection. I dragged the church in there. You will find the church too big (like <em>way way </em>too big), so scale it to 0.025 in all directions.Then set for location X to –0.076 and Z to 0.2915. Finally, set Y rotation to 158.</p> <p>&nbsp;</p> <p><a href="https://lh3.googleusercontent.com/-o_oHCKk8lGc/WDCxoP_MK1I/AAAAAAAAOcs/k4TdyJtzyE4/s1600-h/image%25255B44%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-pMMMYh4K5R0/WDCxok8EnrI/AAAAAAAAOcw/a3pURoUYizs/image_thumb%25255B21%25255D.png?imgmax=800" width="304" align="left" height="138"></a></p> <p><a href="https://lh3.googleusercontent.com/-5cAMk9m6DIk/WDCxpNxFLbI/AAAAAAAAOc0/qB9pYJmzqXk/s1600-h/image%25255B55%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 12px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-W73WCVDmDpg/WDCxpYSQkqI/AAAAAAAAOc4/3kG--Bi4j3I/image_thumb%25255B24%25255D.png?imgmax=800" width="244" align="left" height="222"></a></p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>&nbsp;</p> <p>This will put the church nicely where it’s more or less supposed to be, although I think it’s about three times too big with respect to the map but otherwise it would be nearly invisible. For the record – this is not a model of <a href="https://www.google.nl/maps/@52.3466953,4.7509796,3a,75y,5.42h,112.06t/data=!3m6!1e1!3m4!1se3qdt22N-yI2cvSIPT1Sew!2e0!7i13312!8i6656?hl=nl" target="_blank">the actual church</a> that is converted to the Wortell offices, but it resembles it a little.</p> <h2><a href="https://lh3.googleusercontent.com/-7LwiVYtoIUg/WDGg4dGxcDI/AAAAAAAAOdM/QEzgHY61obs/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-ewymOIUKnUI/WDGg4xZyGCI/AAAAAAAAOdQ/ELBUHjlJ-UQ/image_thumb%25255B1%25255D.png?imgmax=800" width="244" align="right" height="143"></a>Getting a billboard and a logo</h2> <p>Getting this one is actually very easy. Go to the Unity Asset store – that is, click Window/Asset Store and type “Billboard” in filter. You will get an enormous amounts of hits but you just need the very first one, that I indicated in red. Click on it, then click “download” (or “import”, if you have downloaded it before) .</p> <p>When you actually import, make sure to deselect the sample scene, as we really don’t need it:</p> <p><a href="https://lh3.googleusercontent.com/-nDFgTTu0-Mo/WDGg5MoDx7I/AAAAAAAAOdU/irPooQeBpYM/s1600-h/image%25255B10%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-17kwK_z3nRg/WDGg5kuF-pI/AAAAAAAAOdY/OFk3TbNiU6s/image_thumb%25255B2%25255D.png?imgmax=800" width="223" height="244"></a></p> <p><a href="https://lh3.googleusercontent.com/-OfQcg5kxg4k/WDGg6LcypfI/AAAAAAAAOdc/KlHrSdFmNoY/s1600-h/image%25255B18%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-pB-ZpLdi8bo/WDGg6bWPVAI/AAAAAAAAOdg/e3S1rzF6DMI/image_thumb%25255B5%25255D.png?imgmax=800" width="304" align="right" height="108"></a>Drag the billboard prefab from Assets/Billboard inside the Wortell object on the hierachy. You will see that once again this is way too big. Scale it to 0.03 for x, y and z, give it the same position as the church and set rotation to 0 for x, 0 for y, and 90 for z. The billboard will now look like this:</p> <p><a href="https://lh3.googleusercontent.com/-QqeT91AFILA/WDGg7NNbIJI/AAAAAAAAOdk/GIwK8pDizOA/s1600-h/image%25255B21%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-cGzLFMQtG50/WDGg7YziWzI/AAAAAAAAOdo/-P9_VMNcNu8/image_thumb%25255B6%25255D.png?imgmax=800" width="199" height="244"></a></p> <p>So the only thing missing is the Wortell logo – or whatever you want to put on it. I am not sure if this is the right procedure, but I went with this:</p> <ul> <li><a href="https://lh3.googleusercontent.com/-gJhURcOR5EU/WDGyduH8YRI/AAAAAAAAOd4/8neRT0E2M9s/s1600-h/billboard%252520texture%25255B7%25255D.jpg"><img title="billboard texture" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="billboard texture" src="https://lh3.googleusercontent.com/-_Fq0e3JgmDo/WDGyeNJZukI/AAAAAAAAOd8/oVohvmlndBM/billboard%252520texture_thumb%25255B2%25255D.jpg?imgmax=800" width="54" align="right" height="54"></a>Add a 512x512 image with your logo to Assets/App/Textures. I took this one. Notice the Wortell text is just about halfway – this should put the text at the bottom of the billboard (as is customary with our logo) <li>Click on the Billboard in the hierarchy. <li>Expand the “board” material in the inspector <li>Drag your logo texture on the square that says “None (texture)” <li>Set X and Y tiling both to 2</li></ul> <p>Net result should be like left, and the billboard standing above the church should now look to the right:</p><a href="https://lh3.googleusercontent.com/-_7-aL_--ogA/WDGyelKdlhI/AAAAAAAAOeA/RDITaUxdqpQ/s1600-h/image%25255B40%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-9V1ov-13W1M/WDGyfIOE_hI/AAAAAAAAOeE/LikOP_UspPw/image_thumb%25255B13%25255D.png?imgmax=800" width="240" height="324"></a><a href="https://lh3.googleusercontent.com/-e1Hn90eaDL0/WDGyf3nvx9I/AAAAAAAAOeI/ncRXQ7uKZ7o/s1600-h/image%25255B41%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 37px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-62dN2zWoCBw/WDGygYG82ZI/AAAAAAAAOeM/_AHQaqIUoHU/image_thumb%25255B14%25255D.png?imgmax=800" width="284" height="324"></a> <p>Getting the logo the way you want it to be takes a little fiddling. I really can’t tell you why you need to set those tiling values – I just found out it worked this way. </p> <h2>Adding some life to it</h2> <p>It looks pretty nice, but that billboard has all the interactivity of a wooden shed – it’s sitting there doing nothing. We want it to do two things:</p> <ul> <li>When it’s showing, it needs to be rotated to face the user – so it’s always readable <li>Initially it won’t be visible at all, air tapping the church will make it visible (and invisible again, like a toggle)</li></ul> <p>After all, that’s what’s in the original video, and this very feature was even <a href="https://youtu.be/uSkv_XFTdc4?t=2589" target="_blank">demonstrated live on Windows Weekly</a> by none other than <a href="https://twitter.com/maryjofoley" target="_blank">Mary Jo Foley</a> herself&nbsp; ;) when she and <a href="https://twitter.com/thurrott" target="_blank">Paul Thurrott</a> visited the Wortell offices on <a href="https://youtu.be/uSkv_XFTdc4" target="_blank">November 16, 2016</a>. </p> <h2>Making the billboard face the user</h2> <p>This requires actually zero programming effort, as a billboard <em>behaviour</em> is already in the HoloToolkit. So basically all you have to do is:</p> <ul> <li>Got to Assets/HoloToolkit/Utilities/Scripts <li>Locate the Billboard.cs script <li>Drag it on top of your billboard game object in the hierarchy <li>Set the script’s pivot axis to X</li></ul> <p><a href="https://lh3.googleusercontent.com/-ga_P4GzVvVg/WDG3EEfLJ8I/AAAAAAAAOeg/pLFbrN_yveoUHU7XPy2xt1VjaL5NCldlACHM/s1600-h/image%255B46%255D"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-FfViM-63VnM/WDG3Eru2_SI/AAAAAAAAOek/RHTS7354Jfwu6tSPPduatVpM7wrsOIYpQCHM/image_thumb%255B17%255D?imgmax=800" width="404" height="57"></a></p> <p>And you are done. If you deploy the app now to a HoloLens or emulator you will see the billboard now follows the camera. Now this may seem odd, as the billboard should rotate around the axis that runs from to top bottom – in other words, Y. But since the billboard is default rotated 90⁰ around it’s <em>Z-axis</em> and rotates around it’s <em>own</em> axis rather than a world axis, this makes sense. Kind of ;)</p> <p>If you are smart, you are now are wondering why I did not use that for the <em>aircraft label</em>, as the billboard script also supports full free axis movement. That is simple – at the time I wrote the code for the label, I simply had not thought of looking into the HoloToolkit. So I wrote a custom script. And I kept it that way, as a kind of way to show you how I am still learning as well :)</p> <h2>Preparing the billboard folding in or out by air tapping</h2> <p>So what we want to achieve is to be able to air tap the <em>church</em>, but this should have an effect on the <em>billboard</em>. I have solved this as follows:</p> <ul> <li>Create a collider on the church <li>Make a behaviour attached to the billboard that handles the folding in/out of the billboard <li>Make a little behaviour attached to the church that receives the air tap and passes that to the behaviour mentioned in the previous bullet point.</li></ul> <p><a href="https://lh3.googleusercontent.com/-DsfFdRWivX4/WDHjHEysRrI/AAAAAAAAOe0/x1SMCY8y5TA/s1600-h/image%25255B59%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-Xqenuv2O-r8/WDHjHlXpQEI/AAAAAAAAOe4/lC3IB6jKClc/image_thumb%25255B21%25255D.png?imgmax=800" width="244" align="right" height="89"></a></p> <p>First the collider. As <a href="http://dotnetbyexample.blogspot.com/2016/11/a-hololens-airplane-tracker-7activating.html" target="_blank">explained in the previous episode</a> , to be able to show a cursor on an object or select it, it needs to have a collider. I once again took the box collider for easy selection.</p> <p>Only then we notice something peculiar when we look at the church from the top:</p> <p><a href="https://lh3.googleusercontent.com/-hZgPZOqRjMk/WDHjIInUG3I/AAAAAAAAOe8/swCWxOWCXks/s1600-h/image%25255B63%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-ZGTGzcJC1Mw/WDHjIsrKojI/AAAAAAAAOfA/HdshNmtkOD4/image_thumb%25255B23%25255D.png?imgmax=800" width="481" height="304"></a></p> <p>Meh. That’s annoying. Apparently the creator of this model has created it on an angle, or there’s something wrong with the conversion. Either way we will have to fix it. We do this by digging into the church object in the hierarchy and rotate the Group1 component 25⁰ over it’s Y-axis so it’s visually aligned with the collider:</p> <p><a href="https://lh3.googleusercontent.com/-WlzLCxDVODU/WDHjJHxgPzI/AAAAAAAAOfE/_xajqiUcmiY/s1600-h/image%25255B76%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-C9jFOgFWBKo/WDHjJuCA00I/AAAAAAAAOfI/zidiM5UdUXw/image_thumb%25255B28%25255D.png?imgmax=800" width="362" height="183"></a><a href="https://lh3.googleusercontent.com/-s0raFwzGmBo/WDHjKKWC_DI/AAAAAAAAOfM/MVwD9j_cFfs/s1600-h/image%25255B75%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 13px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-bS4uDqHzGTI/WDHjL9rC8bI/AAAAAAAAOfQ/vhEWCftZWTc/image_thumb%25255B27%25255D.png?imgmax=800" width="244" height="183"></a></p> <p>Now of course church itself is no longer aligned with the road. But if you subtract the 25⁰ from the 158⁰ the church object it is rotated:</p> <p><a href="https://lh3.googleusercontent.com/-taWSHk9GTsQ/WDHjMM_5XTI/AAAAAAAAOfU/82WjPu3m-cE/s1600-h/image%25255B83%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-MuwxilxH5vI/WDHjMgdZn-I/AAAAAAAAOfY/fB0bQE8yAZE/image_thumb%25255B31%25255D.png?imgmax=800" width="273" height="244"></a><a href="https://lh3.googleusercontent.com/-QNYHgHxprB0/WDHjNGE-dyI/AAAAAAAAOfc/9jmZayBApZ8/s1600-h/image%25255B82%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 13px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-Bevb-XLrcOE/WDHjNhODsiI/AAAAAAAAOfg/hqT0Eqb_OwE/image_thumb%25255B30%25255D.png?imgmax=800" width="235" height="244"></a>&nbsp;</p> <p>everything is now correctly rotated and aligned. And now for some action.</p> <h2>Adding the behaviours</h2> <p>We are almost there. First, we need the behavior that does the actual folding and unfolding of the billboard:</p><pre>using System;
using UnityEngine;
public class BillBoardToggler : MonoBehaviour
{
private float _startScale;
void Start()
{
_startScale = transform.localScale.x;
Hide();
}
public void Toggle()
{
if (Math.Abs(transform.localScale.x) &lt; 0.001f)
{
// Pop up
transform.localScale = new Vector3(0, _startScale, _startScale);
iTween.ScaleTo(gameObject,
new Vector3(_startScale, transform.localScale.y,
transform.localScale.z), 1f);
}
else
{
// Pop down
iTween.ScaleTo(gameObject, iTween.Hash("scale",
new Vector3(0, transform.localScale.y, transform.localScale.z),
"time", 1f, "oncomplete", "Hide"));
}
}
private void Hide()
{
transform.localScale = new Vector3(0, 0, 0);
}
}
</pre>
<p>So at start we save the x scale, and hide the complete billboard by scaling it to 0,0,0, effectively making it infinitely small – and thus invisible. Now if the Toggle method is called:</p>
<ul>
<li>If the scale bigger than 0.001, the billboard apparently was popped down. So we first set the Y and Z scale to their original value – and keep X to 0, then proceed to animate the X scale from 0 to its original value, using our old friend <a href="http://itween.pixelplacement.com/gettingstarted.php" target="_blank">iTween</a>, which we saw already in <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-5-smooth.html" target="_blank">the 5th episode</a> of this series. This lets the billboard rise out of the ground
<li>otherwise, the billboard was already popped up. So we animate the x scale to 0. If you look closely, you will see the billboard being reduced to its outline on the airport map. But just after that, Hide is called, letting the outline also pop out of existence.</li></ul>
<p>Drag this behaviour on top of the billboard. Only one thing left to do – make sure this thing gets called when the user air taps the <em>church</em>. For that, we only need this simple behaviour:</p><pre>using UnityEngine;
public class ChurchController : MonoBehaviour
{
private BillBoardToggler _toggler;
// Use this for initialization
void Start()
{
_toggler = transform.parent.GetComponentInChildren&lt;BillBoardToggler&gt;();
}
private void OnSelect()
{
if (_toggler != null)
{
_toggler.Toggle();
}
}
}</pre>
<p>Drag this on on top of the church. On start it will try to find a BillBoardToggler script on a sibling, and when the OnSelect method is called – a standard message sent by the HoloToolkit GestureManager – it will call the Toggle method on that, letting the billboard pop up (or make it go down again). So if you move your gaze cursor to the church and air tap on it:</p>
<p><a href="https://lh3.googleusercontent.com/-B5Eo51Ss61E/WDID8p6CYRI/AAAAAAAAOgE/eUKDUi7RAzA/s1600-h/image%25255B87%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-NxldE62VJaU/WDID9VdWBDI/AAAAAAAAOgI/pFzkHTmVreM/image_thumb%25255B33%25255D.png?imgmax=800" width="654" height="370"></a></p>
<h2>Conclusion</h2>
<p>So that’s it. No more beans to spill. This the the app and all of the app, except for the live data feed. In fact, it’s <em>better</em> then the app in the store, because backtracking the road I took making this app I skipped a few detours, false starts and messy things that have not turned up in this cleaned up tutorial code. That is why I can recommend everyone writing an extensive tutorial once in a while – because not only the students will learn, but the teacher as well. </p>
<p>The final and completed code of this app <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog8" target="_blank">can be found here</a>. I don’t rule out the possibility that I will write about extensions to this app at some later point, but this is what I wanted to share. I enjoyed immensely writing about it, and I hope you enjoyed reading. If you made it this far – kudos to you. I only ask you one thing – if this inspires you to write HoloLens apps too – consider writing a tutorial as well. Maye I will learn something from you as well ;)</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/CtaFQguiGLo" height="1" width="1" alt=""/>2016-11-20T21:19:59.402+01:000http://dotnetbyexample.blogspot.com/2016/11/a-hololens-airplane-tracker-8adding.htmlA HoloLens airplane tracker 7–activating an aircraft by air tappinghttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/brO83iMNGBs/a-hololens-airplane-tracker-7activating.htmlAzureHoloLensSpatial SoundUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 12 Nov 2016 22:03:00 PSTtag:blogger.com,1999:blog-5295746446529817470.post-5550570692610386573<h2>Intro</h2> <p>In this episode I am going to describe how the airplanes are selected in the sky by air tapping them. This requires some changes made to both the Azure Mobile App service and the app. The change to the app service is necessary as I want to share the selected airplane between all possible HoloLenses using the app. Why? Because ;). You actually can do this with the app in the store. This is why sometimes airplanes get selected even when you don’t select them – that is because someone else, somewhere in the world, selects an airplane.</p> <h2>Adapting the service</h2> <p>We need to be able to post to the service which aircraft is active, and in the data that is pulled from the server the active airplane needs to be indicated. Remember in the very <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-1.html">first episode</a> the “Flight” object had an unused “IsActive” field? It won’t be unused very much longer. First, you add a reference to System.Runtime.Caching </p> <p><a href="https://lh3.googleusercontent.com/-JgzKJpEKk3Q/WCgCFWxMaFI/AAAAAAAAOYY/8q2_8EbP1IE/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-QCL9lxS4dKE/WCgCFmDEflI/AAAAAAAAOYc/DHju8giURX0/image_thumb%25255B1%25255D.png?imgmax=800" width="448" height="204"></a></p> <p>Then, add using statements:</p><pre>using System.Runtime.Caching;
using System.Net.Http;
using System.Net;</pre>
<p>Proceed by adding the following two fields:</p><pre>private ObjectCache Cache = MemoryCache.Default;
private const string ActiveIdKey = "ActiveId";</pre>
<p>We will need to add a method to post the desired active plane’s Id to the server:</p><pre>[HttpPost]
public HttpResponseMessage Post(string activeId)
{
Cache[ActiveIdKey] = activeId;
return Request.CreateResponse(HttpStatusCode.OK);
}</pre>
<p>Then, in the Get method, insert right before the “return flights” statement the following code:</p><pre>var activeKey = Cache[ActiveIdKey] as string;
if (activeKey != null)
{
var activeFlight = flights.FirstOrDefault(p =&gt; p.Id == activeKey);
if (activeFlight != null)
{
activeFlight.IsActive = true;
}
}</pre>
<p>This is a rather naive way of ‘storing’ data on a server but I am just do darn lazy to set up a database if I only want to have one simple id kept active. So I am using the MemoryCache. If you post an id to the Post method, it will simply set the cache key – which will be copied into the data in the next data download, provided it’s actually an id used in the data. Publish your service, and let’s go the Unity project again</p>
<h2>Adding a gaze cursor – and messing a bit with it</h2>
<p>The HoloToolkit has a number of gaze cursors, but I am not quite happy with those, so I made one for myself – by adding a bit into an existing one. The procedure is as follows:</p>
<ul>
<li>From HoloToolkit/Input/scripts, drag Handsmanager on top of the HologramCollection
<li>From HoloToolkit/Input/scripts, drag Gazemanager on top of the HologramCollection
<li>From HoloToolkit/Input/Prefabs, drag Cursor on top of the HologramCollection</li></ul>
<p>Then we create a little extra behavior, called HandColorCursor </p><pre>using UnityEngine;
using HoloToolkit.Unity;
public class HandColorCursor : Singleton&lt;HandColorCursor&gt;
{
// Use this for initialization
private GameObject _cursorOnHolograms;
public Color HandDetectedColor;
private Color _originalColor = Color.green;
void Start()
{
_cursorOnHolograms = CursorManager.Instance.CursorOnHolograms;
_originalColor =
_cursorOnHolograms.GetComponent&lt;MeshRenderer&gt;().material.color;
}
// Update is called once per frame
void LateUpdate()
{
if (_cursorOnHolograms != null &amp;&amp; HandsManager.Instance != null)
{
_cursorOnHolograms.GetComponent&lt;MeshRenderer&gt;().material.color =
HandsManager.Instance.HandDetected ? Color.green : _originalColor;
}
}
}</pre>
<p>Drag this behavior on top of the Cursor prefab. Now if you build and run the app and point the gaze cursor to the airport it should become a kind of blue circle. If you move your hand into view, the cursor should appear and go green. </p>
<p><a href="https://lh3.googleusercontent.com/-T7eIrPSZyyU/WCiv4rAeK9I/AAAAAAAAOYw/385PqXqzqHc/s1600-h/image%25255B4%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-yVPEzx-SgzI/WCiv43EcRtI/AAAAAAAAOY0/lvwRXDsx6Z0/image_thumb%25255B1%25255D.png?imgmax=800" width="654" height="370"></a></p>
<p><a href="https://lh3.googleusercontent.com/-F44887GrGnU/WCjDxxsGsFI/AAAAAAAAOZE/UKUBcsC1Ohg/s1600-h/image%25255B30%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-nMCCucl8A5A/WCjDyVUOD1I/AAAAAAAAOZI/2x65NpMOQsM/image_thumb%25255B13%25255D.png?imgmax=800" width="54" align="right" height="55"></a>If you move it back, the cursor should again become blue. Unfortunately, if you point it at anything else, you will still see the fuzzy ‘CursorOffHologram’ image. That it’s because the airport is the only thing that has a collider. To make aircraft selectable, we will need to do some work at AircraftHolder.</p>
<h2>Updating the airplane</h2>
<p>To be selectable, it needs to have a collider. I have chosen a box collider that is pretty big with respect to the actual airplane model – to make it easier to actually select the airplane (as it can be quite some distance away, or at least seem to be) and a box makes it possible to actually see the cursor sitting pretty stable on the selection area as it hits the collider (as opposed to a mesh collider that will make it wobble around). Go to your AircraftHolder prefab in the App/Prefab folder, and add an 80x80x80 Box Collider </p>
<p><a href="https://lh3.googleusercontent.com/-dTb2bgqaces/WCjDzElU_lI/AAAAAAAAOZM/wl8vGaOiV0E/s1600-h/image%25255B15%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-gw0RRmzHliI/WCjDz5DLMZI/AAAAAAAAOZQ/kA0o7Q4gxUY/image_thumb%25255B6%25255D.png?imgmax=800" width="304" height="70"></a></p>
<p>And you will indeed see the cursor appear over the airplane</p><a href="https://lh3.googleusercontent.com/-gzHc9aCHh4k/WCjD0EUB72I/AAAAAAAAOZU/rriLQ22ASdQ/s1600-h/image%25255B19%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-T0BDz2RNOcw/WCjD0nTYzcI/AAAAAAAAOZY/w2BDRKUoNGQ/image_thumb%25255B8%25255D.png?imgmax=800" width="544" height="308"></a>
<h2>Getting some selecting done – code first</h2>
<p>From HoloToolkit/Input/scripts, drag GestureManage on top of the HologramCollection. GestureManager sends an OnSelect message to any GameObject that happens to be hit by a gaze (and where, incidentally, the cursor is hanging out out) using this code:</p><pre>private void OnTap()
{
if (FocusedObject != null)
{
FocusedObject.SendMessage("OnSelect");
}
}</pre>
<p>So there needs to be an OnSelect method in the AircraftController to handle this. First we will need to add the following fields to the AircraftController:</p><pre>private bool _isActive;
private AudioSource _sound;
private Dictionary&lt;MeshRenderer, Color&gt; _originalColors;</pre>
<p>To the Start method, we add the following code, just in front of the “_initComplete = true;” statement</p><pre>_sound = GetComponent&lt;AudioSource&gt;();
_originalColors = new Dictionary&lt;MeshRenderer, Color&gt;();
foreach (var component in GetComponentsInChildren&lt;MeshRenderer&gt;())
{
_originalColors.Add(component, component.material.color);
}</pre>
<p>So this actually initializes some things by getting the audio source and reads the color of all sub components – so we can revert it easily later. Then add this method:</p><pre>private void SetHighlight()
{
if (_isActive != _flightData.IsActive)
{
foreach (var component in GetComponentsInChildren&lt;MeshRenderer&gt;())
{
if (component.material.color == Color.white ||
component.material.color == Color.red)
{
component.material.color = _flightData.IsActive ?
Color.red : _originalColors[component];
}
}
if (_flightData.IsActive)
{
_sound.Play();
}
else
{
_sound.Stop();
}
_isActive = _flightData.IsActive;
}
}</pre>
<p>If the _isActive field of the current airplane differs from the new _flightData.IsActive, then flip white things to red – or red things back to white, depending on the actual value of _flightData.IsActive. And it turns on the sound – the annoying sonar ping you saw in the movie I made in the first episode. And fortunately it can also turn in it off ;)</p>
<p>Next, add a call to this new method at the end of the already existing “SetNewFlightData” method. Then, add this code:</p><pre>private void OnSelect()
{
SendMessageUpwards("AircraftSelected", _flightData.Id);
}</pre>
<p>This is like SendMessage, but then <em>upwards</em> through the object hierarchy, in stead of down to the children. So this will look for an AircraftSelected method in the parent. </p>
<p>Save the file. Now we go to the DataService, and add the following method. Make sure it is sitting between #if UNITY_UWP – #endif directives.</p><pre>#if UNITY_UWP
public async Task SelectFlight(string flightId)
{
var parms = new Dictionary&lt;string, string&gt;
{
{ "activeId", flightId }
};
await _client.InvokeApiAsync&lt;List&lt;Flight&gt;&gt;("FlightData",
HttpMethod.Post, parms);
}
#endif</pre>
<p>This does the actual calling of the App service. And then we need to go to AircraftLoader, which plays the select sound and passes the call trough to the DataService:</p><pre>public void AircraftSelected(object id)
{
_sound.Play();
#if UNITY_UWP
DataService.Instance.SelectFlight(id.ToString());
#endif
}</pre>
<p>Save everything, go back to Unity.</p>
<h2>Getting some selecting done – connecting the dots in Unity</h2>
<p>There are a few things we need to be doing in Unity, but one thing is very important. One I forgot in my first CubeBouncer demo app <em>and</em> in the app that’s in the store, simply because I was not aware of it. Fortunately someone who shall remain nameless here made me aware of it.</p>
<p>First, in the main Unity window, hit Edit/Project Settings/Audio. Then check “Ms HRTF Spatialization” for “Spatializer plugin”</p>
<p><a href="https://lh3.googleusercontent.com/-eDCGaUuMuBY/WCjpACkXgmI/AAAAAAAAOZs/mwakbAG-hX0/s1600-h/image%25255B34%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-3uLHZYNntBM/WCjpAS4qvdI/AAAAAAAAOZw/TBbXVnMk8WY/image_thumb%25255B15%25255D.png?imgmax=800" width="354" height="213"></a></p>
<p>Then open the AircraftHolder prefab again, and add an AudioSource. Important advice, also from the person mentioned above: hit the “Spatialize” checkbox. Then select the “Loop” checkbox and <em>uncheck</em> the “Play On Awake” checkbox. Finally, find a sound you would like to be played by a selected airplane. I took a sonar ping. I put it in the Assets/App/Audio folder, and dragged it on the Audio Clip property. I also moved the “Spatial Blend” slider all the way to the right but I don’t think that is necessary anymore now I use the Spatializer. Net result:</p>
<p><a href="https://lh3.googleusercontent.com/-_0-aHnJ-nKs/WCkxIkx2nyI/AAAAAAAAOaE/3he4VJFNwgo/s1600-h/image%25255B38%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-Vk8PqFPmH3g/WCkxI3OwU0I/AAAAAAAAOaI/faEYZDYipK4/image_thumb%25255B17%25255D.png?imgmax=800" width="354" height="315"></a></p>
<p>And sure enough, if you now air tap an airplane it shows like below, and it starts pinging:</p>
<p><a href="https://lh3.googleusercontent.com/-kNwtvuOlnrQ/WCkxJZTUqQI/AAAAAAAAOaM/8tLilF8Ck_M/s1600-h/image%25255B41%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="https://lh3.googleusercontent.com/-nckrmJ0rEec/WCkxJmFTc-I/AAAAAAAAOaQ/cFqen_v5haU/image_thumb%25255B18%25255D.png?imgmax=800" width="126" height="68"></a></p>
<h2>Adding confirmation sound to selection</h2>
<p>I think it’s good practice to add a confirmation sound of some sorts to any user input that is ‘understood’. This is especially necessary with speech commands, but I like to add it to air taps. Sometimes the effect of what you do is not immediately visible, and an audio confirmation the HoloLens has understood you and is working on making your command getting executed is very nice. I tend to add a kind of “pringgg” sound. I used the “Ready” file that is in Assets/App/Audio.</p>
<p>First, add an AudioSource to the HologramCollection and drag the “Ready” sound on top of the “AudioClip” field. Don’t forget to uncheck the “Play On Awake” checkbox.Then go to the AircraftLoader behaviour. Since this receives the message from every AircraftController and passes it through to the DataService, this is a nice central place to play the confirmation sound. So we add a new field:</p><pre>private AudioSource _sound;</pre>
<p>that needs to be initialized in the Start method:</p><pre>_sound = GetComponent&lt;AudioSource&gt;();
</pre>
<p>and finally in the AircraftSelected method, insert this line at the top of the method:</p><pre>_sound.Play();</pre>
<p>And we are done. Now if you airtap on the aircraft your HoloLens will sound a ‘pringgg’ sound. That is, if you tapped correctly.</p>
<h2>Conclusion</h2>
<p>And that’s that. Selecting an airplane by by air tapping it it, and posting an id to the Azure Mobile App service is pretty easy indeed. A lot of the interaction components are basically dragged in from the HoloToolkit, very little actual programming is required. This is the power of the HoloToolkit – I use as much from what is already there, and make as little as possible myself. As any good programmer I am a lazy programmer ;)</p>
<p>This blog post written far, far away from home, in Sammamish, WA, USA, shortly after the MVP summit that left me very inspired – again. After posting this I will soon retire – and go to SEA-TAC tomorrow for my flight back home in the Netherlands. </p>
<p>Code can be found, as always, <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog7">here on github</a>. </p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/brO83iMNGBs" height="1" width="1" alt=""/>2016-12-29T08:38:33.125+01:000http://dotnetbyexample.blogspot.com/2016/11/a-hololens-airplane-tracker-7activating.htmlRunning a Hololens app on a Raspberry PI2–and controlling it with a keyboardhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/Z_SYyQEm7jg/running-hololens-app-on-raspberry.htmlHoloLensIoTRaspberry PI2UWPnoreply@blogger.com (Joost van Schaik)Sat, 29 Oct 2016 13:30:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-5288502641489079305<h2>Intro</h2> <p>No, I am not drunk, nor did I smoke some peculiar produce our capital Amsterdam is famous for, and I did not bump my head a bit too hard on my car while getting out of it either. I was just curious how far the U of UWP would carry me. And it turns out, a lot further than I thought. It’s actually possible to run a HoloLens app on a Raspberry PI2. When I saw it happen, I actually had a bit trouble believing my own eyes, but it is possible indeed. With surprisingly little work. Although I added a little extra work to make it more fun.</p> <h2>Parts required &amp; base material</h2> <ul> <li>One Raspberry PI2 or 3 running the latest version of <a href="https://developer.microsoft.com/en-us/windows/iot" target="_blank">Windows 10 IoT Core</a>. <li>A keyboard that plays nice with the above <li>A screen to see the result on</li></ul> <p>I used a Raspberry PI2 running the latest Windows 10 IoT Core Insider’s build and a <a href="http://nexdock.com/" target="_blank">NexDock</a>. Mainly because I have one, it’s handy and it looks cool. Be aware that to connect the NexDock <em>keyboard</em> you will need a Bluetooth dongle for your RP2 – or you will need use a RP3 which has it on board.</p> <p>This post builds upon my (as of the time of this writing still incomplete) series about the <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-1.html" target="_blank">Hololens Aircraft tracker</a>, but that’s only because that’s a cool app to demo the idea on. It is no part of the series. I made a separate branch of the app at the end of the 6th episode. So this is like an interlude. </p> <h2>Some brute-force demolishing to get things to compile</h2> <p>Basically it’s as simple as setting the build target to ARM:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb.png" width="304" height="35"></a></p> <p>The only problem is, if you do that and then rebuild the project, it will complain about this</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb_3.png" width="604" height="96"></a></p> <p>For the very simple reason this dll, which is part of the HoloToolkit, is not available for ARM. Hell, HoloLens runs x86 only, so why <em>should</em> it be available. No-one ever anticipated using a <strike>dolt</strike> peculiar person like me trying to run it on a bloody Raspberry PI2. </p> <p>There is a rather crude way of fixing it. We are not using SpatialUnderstanding anyway, most certainly not on the Raspberry PI2. So I got rid of the plugin that Visual Studio complained about, by going to this folder in Unity:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb_4.png" width="304" height="371"></a></p> <p>And hitting delete. Rebuild project in Unity, try to compile it in Visual Studio. Alas. Still no success</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_5.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb_5.png" width="604" height="116"></a></p> <p>But this is a <em>different</em> plugin dll – PlaneFinding. Also something we don’t use. Now this is a bit confusing, as there are three folder containing a Planefinding.dll. Maybe that’s an error in the HoloToolkit.</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_6.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb_6.png" width="182" height="244"></a>&nbsp;</p> <p>Whatever. Let’s get rid of the whole plugins folder under SpatialMappping. Once again, rebuild in Unity, compile in Visual Studio. And wouldn’t you know it…</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_7.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_13865/image_thumb_7.png" width="604" height="249"></a></p> <p><img style="float: right; display: inline" src="https://mymorningmeditations.files.wordpress.com/2013/09/square-peg-round-hole-smash.jpg" width="100" align="right" height="113">The sweet smell of success. Now I want you to understand this is no way to go about to make a HoloLens project compatible with a Raspberry PI2. This is using a mallet to hit very hard on a square peg to make it go through a round hole. I have demolished two components you might want to use in the future version of your HoloLens app. But this is not serious development – this is hacking for fun’s sake to prove a point. That is why I have made a separate branch ;).</p> <h2>Controlling the app’s viewpoint</h2> <p>When you run the app on the HoloLens this is no issue at all. If you want to see the airport and it’s planes from closer up or from a different angle, you just move your head, walk to the object of your interest – or around it. If if runs on a screen, things are a bit different. So I created this little behaviour (with “ou” indeed, which suggests the good folks at Unity have been educated in The Queen’s English) that more or less replicates the key mappings of the HoloLens emulator:</p> <p>It’s crude, ugly, the result is a bit stuttering – but it does the job.</p><pre>using UnityEngine;
public class KeyboardCameraController : MonoBehaviour
{
public float Rotatespeed = 0.4f;
public float MoveSpeed = 0.02f;
public float FastSpeedAccleration = 7.5f;
private Quaternion _initialRotation;
private Vector3 _initialPosition;
void Start()
{
_initialRotation = Camera.main.transform.rotation;
_initialPosition = Camera.main.transform.position;
}
void Update()
{
var speed = 1.0f;
if (Input.GetKey(KeyCode.LeftShift) || Input.GetKey(KeyCode.RightShift))
{
speed = FastSpeedAccleration * speed;
}
if (Input.GetKey(KeyCode.LeftArrow))
{
Camera.main.transform.RotateAround(Camera.main.transform.position,
Camera.main.transform.up, -Rotatespeed * speed);
}
if (Input.GetKey(KeyCode.RightArrow))
Camera.main.transform.RotateAround(Camera.main.transform.position,
Camera.main.transform.up, Rotatespeed * speed);
if (Input.GetKey(KeyCode.UpArrow))
{
Camera.main.transform.RotateAround(Camera.main.transform.position,
Camera.main.transform.right, -Rotatespeed * speed);
}
if (Input.GetKey(KeyCode.DownArrow))
Camera.main.transform.RotateAround(Camera.main.transform.position,
Camera.main.transform.right, Rotatespeed * speed);
if (Input.GetKey(KeyCode.A))
Camera.main.transform.position +=
Camera.main.transform.right * -MoveSpeed * speed;
if (Input.GetKey(KeyCode.D))
Camera.main.transform.position +=
Camera.main.transform.right * MoveSpeed * speed;
if (Input.GetKey(KeyCode.W))
Camera.main.transform.position +=
Camera.main.transform.forward * MoveSpeed * speed;
if (Input.GetKey(KeyCode.S))
Camera.main.transform.position +=
Camera.main.transform.forward * -MoveSpeed * speed;
if (Input.GetKey(KeyCode.PageUp))
Camera.main.transform.position +=
Camera.main.transform.up * MoveSpeed * speed;
if (Input.GetKey(KeyCode.PageDown))
Camera.main.transform.position +=
Camera.main.transform.up * -MoveSpeed * speed;
if (Input.GetKey(KeyCode.Escape))
{
Camera.main.transform.position = _initialPosition;
Camera.main.transform.rotation = _initialRotation;
}
}
}</pre>
<p>Drag this behaviour on top of the camera (or for what matters, anything at all). You will be able to control the camera’s standpoint via what I have been told is the the standard PC gamer’s WASD and arrow key mappings. PageUp/PageDown will move the camera standpoint up and down, ESC will bring you back to the original viewpoint when the app started, and using SHIFT will make things go faster. </p>
<h2>Deploy and run</h2>
<p>Deploy the app to your Raspberry PI2 or 3 using Visual Studio – and select your PI as remote machine. Use either “Release” or “Master” build configuration. The latter should - in theory – go faster, but takes much longer (as in <em>very much</em> longer) to compile and deploy. Also, if you choose “Master”, the app does not always start on my PI, it’s sometimes only deployed – so you have to get it going via the device’s Device Portal. This may have something to do with me running an Insider’s build.</p>
<p>Either way – if you have WiFi either built-in or via dongle, that’s fine, but unless you particularly like waiting, connect your PI2/3 to a wired connection while deploying. I also observed issues when the app runs while only having WiFi connectivity – data comes in pretty slow, it can take minutes before the first airplanes appear, while on wired it takes like 30 seconds, max. Apparently my 2.4Ghz network (which the PIs use) is not that strong compared to the 5G all the other devices in the house employ.</p>
<p>And it works. Here are some pictures and a video of the app in action. The performance is not stellar (big surprise here, matching a $35 device against a $3000 device that comes straight out of Star Trek), but still – it works pretty reasonable. </p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_81D2/IMG_4786.jpg"><img title="IMG_4786" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="IMG_4786" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_81D2/IMG_4786_thumb.jpg" width="324" height="217"></a><a href="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_81D2/IMG_4792.jpg"><img title="IMG_4792" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="IMG_4792" src="http://www.schaikweb.net/dotnetbyexample/Running-a-Hololens-app-on-a-Raspberry-PI_81D2/IMG_4792_thumb.jpg" width="324" height="217"></a></p><iframe height="396" src="https://www.youtube.com/embed/1D7NxUwqrQg" frameborder="0" width="650" allowfullscreen></iframe>
<h2>Conclusion</h2>
<p>Looks like the U in Universal Windows Platform is pretty universal indeed. Microsoft weren’t talking <strike>BS</strike> rubbish about this promise. This app can also be deployed to PCs (I learned that by accidentally choosing “Local Machine” as a target) and I don’t doubt it will run on phones and even XBox One, although I think I would have to ponder a little about the way to control the viewpoint on those devices as they don’t have a keyboard. Yet, an impressive result. </p>
<p>Code <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog6a" target="_blank">can be found here</a>.</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/Z_SYyQEm7jg" height="1" width="1" alt=""/>2016-10-31T15:22:42.749+01:000http://dotnetbyexample.blogspot.com/2016/10/running-hololens-app-on-raspberry.htmlA HoloLens airplane tracker 6–adding an airport (and a tower)http://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/kmScGK-FH94/a-hololens-airplane-tracker-6adding.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Wed, 26 Oct 2016 04:32:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-891712076559964409<h2>Intro</h2> <p>This episode is nearly codeless, it’s all Unity and general geeky GIS stuff. We are going to cut out a piece of <a href="https://www.openstreetmap.org" target="_blank">OpenStreetMap</a> to depict the location of Schiphol. We are also putting a real 3D ATC tower on the map. If you look closely, it has only a very superficial liking to that of the actual Schiphol ATC tower, but those are just <em>minor</em> details ;) </p> <h2><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/schiphol3.png"><img title="schiphol[3]" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="schiphol[3]" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/schiphol3_thumb.png" width="228" align="right" height="304"></a>Getting and positioning the map</h2> <p>Getting it was easy enough. As I wrote, I went about to cut out a piece of rectangular OpenStreetMap that contains Schiphol airport and, incidentally and certainly not coincidentally, the <a href="https://www.google.nl/maps/@52.3467827,4.7507884,3a,75y,23.14h,90.64t/data=!3m6!1e1!3m4!1s4483lSDzlR1-QvRvlUB_Iw!2e0!7i13312!8i6656" target="_blank">Wortell offices</a>. We will deal with that in the final episode of the series.</p> <p>Now <em>positioning</em> that map is a little tricky, as I have set the center Unity’s coordinate system to the center of Schiphol according to Google Maps. But the center of Schiphol, according to <a href="http://www.latlong.net/" target="_blank">this nifty site</a> which also displays coordinates, is found at a location that is about 25% from the bottom of the image and a little to the left. </p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 6px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image_thumb.png" width="197" align="left" height="192"></a> <p>The way I solved this was with a little bit of knowledge of geo coordinates and an advanced technique I learned in my years in Geo Information Systems, which can be described as ‘fiddling until it fits’ ;)</p> <p>First order of business is to try and find out as closely as you can the lat/lon coordinates of the upper left and the lower right corner of the image. I also used <a href="http://www.latlong.net/" target="_blank">latlong.net</a> for that. As far as I could see, top/left is (52.368739, 4.706697) and bottom right (52.285193, 4.811196), while the center of Schiphol is (52.307687, 4.767424)</p> <p>Now we need to go back to <a href="http://dotnetbyexample.blogspot.com/2016/09/converting-latlon-coordinates-to-local.html" target="_blank">a precursor to this series</a> in which I explained how to convert lat/lon coordinates to Unity’s X/Y/Z coordinates using a tangential plane. We can convert these coordinates using this code</p><pre>double x, y, z;
GpsUtils.GeodeticToEnu(52.368739, 4.706697, 0, 52.307687, 4.767424, 0,
out x, out y, out z);
Debug.WriteLine($"{x / 15000},{y / 15000}");
GpsUtils.GeodeticToEnu(52.285193, 4.811196, 0, 52.307687, 4.767424, 0,
out x, out y, out z);
Debug.WriteLine($"{x / 15000},{y / 15000}");</pre>
<p>Note I ignore the Z coordinate, and I divide the result by 15000 as that is how I scaled the airplane data as well. The result is</p>
<p>-0.275750329947415,0.453014630195657<br>0.199135521004053,-0.166804762707821</p>
<p>So we now know the map in Unity is 0.474885850951468 (east-west) by 0.619819392903478 (north-south) meters. We will need this knowledge soon. But first, drag your map cut-out into your App/Textures folder in Unity:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb.png" width="244" height="144"></a></p>
<p>Then proceed to create a <em>material</em> based upon this texture. In the Materials folder, right-click, hit Create/Material and call the material “SchipholMaterial” (or whatever it is you fancy). Then drag your Schiphol texture on top of the “Albedo map” square</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_3.png" width="244" height="104"></a></p>
<p>Next, click the white rectangle to the right of the Albedo square. That pops up a color picker – set the color to B3B3B3FF to make the Schiphol map a bit less overly bright.</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image_thumb_3.png" width="154" height="292"></a></p>
<p>Then, in HologramCollection, create a 3D Plane object. Initially this will look like this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_4.png" width="654" height="348"></a></p>
<p>What is important to wrap your head around is that at scale 1:1, a plane’s size is 10x10 of these blocks which is 10x10 <em>meters</em> in HoloLens. We need to reduce that size so that it’s about 0.62 by 0.475 meters as that is 1:15000 scale. As 10m = scale 1, we need to scale the plane by 0.0475 for X, and 0.062 for Z (leave Y to 1). </p>
<p>Now drag your SchipholMaterial below the “Add Component” button on your Plane. Yes! Unity shows or Schiphol map, with the right dimensions, the right size…</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_5.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_5.png" width="504" height="260"></a></p>
<p>… upside down. Life – and Unity is - full of these little surprises. I have <em>no</em> idea why this is necessary, but if you go to the transform section of the object and hit “180” in the Rotation Y box, </p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_6.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_6.png" width="504" height="262"></a></p>
<p>… that will fix it. Unfortunately, we are still not done, and I will show you why. Add a Sphere to the HologramCollection, size it 0.015 in all directions, give it the Trail Line as material. You will notice a blue dot in the middle of the image if you look from above. That’s 0,0,0 according to Unity.</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_7.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_7.png" width="304" height="397"></a></p>
<p>(I toned down the light a little to make this better visible). Unfortunately, as I stated before, 0,0,0 should be on another place, quite a bit down and to the right, near the “Schiphol” label. If we continue to use this, airplanes taking off won’t be properly aligned with the runways :(. </p>
<p>Back to what we found earlier. The corners of the map should be </p>-0.275750329947415,0.453014630195657<br>0.199135521004053,-0.166804762707821
<p>on <a href="http://dotnetbyexample.blogspot.com/2016/09/converting-latlon-coordinates-to-local.html" target="_blank">the tangential plane we created and used the center of Schiphol</a> as found by Google Maps as 0,0 in the horizontal plane. Now what follows this might look odd, but there is logic to that.</p>
<ul>
<li>Summarize both X values, and divide the result by two. That is half of the <em>difference in distance to what should be the X-center</em>. This is about –0.038
<li>Summarize both Y values, and divide that result by two as well. That is half of the <em>difference in distance to what should be the Z-center </em>(yeah, gets me every time too – what you tend to think of as Y from your Math class, is Z in Unity). This is about 0.143
<li>Add those values to X and Z position in the Plane’s transform:</li></ul>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_8.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_8.png" width="304" height="79"></a></p>
<p>And boom. Compare that to the Google maps marker coordinate and that seems to fit <em>pretty </em>well. </p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_9.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_9.png" width="275" height="204"></a><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 6px 0px 20px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_E4C9/image_thumb.png" width="197" height="200"></a>
<p>Not an exact fit, but that’s because we manually determined the corner’s lat/lon positions. But we are not talking zoning permits here (lengthy legal battles have been fought over mere centimeters on those in my lovely country) so it will do for this app. </p>
<p>I left the sphere in the project as a reference but disabled it, as we don’t need it anymore but you can enable it again and see it indeed shows up at the center of the map.</p>
<p>So. Now we have a properly sized and positioned map. There is a tiny detail missing.</p>
<h2>Adding an ATC tower</h2>
<p>Once again we go to CGTrader and browse around for a tower. Unfortunately the <a href="https://www.bing.com/images/search?q=schiphol+atc+tower&amp;view=detailv2&amp;&amp;id=33D8C20BE99F79748625D06979EBFA8E44BEF69B&amp;selectedIndex=3&amp;ccid=V6m%2fdRb4&amp;simid=608023454551377581&amp;thid=OIP.M57a9bf7516f8b006141d4e0125d6fe61o0&amp;ajaxhist=0" target="_blank">iconic Schiphol ATC tower</a> is not available (paid or free) so I settled for the <a href="https://www.cgtrader.com/free-3d-models/architectural-exterior/industrial/hong-kong-airport-control-tower" target="_blank">Hong Kong ATC tower</a> that bears some superficial resemblance to it. Unfortunately there is some building in the attached to the actual tower in model - that I neither need or want. I followed this rather crude work flow to deal with that:</p>
<ul>
<li>I downloaded the model fbx model zip file
<li>Unpacked it, and renamed 3d-model.fbx to tower.fbx
<li>I created a folder “Tower” in my Assets folder in the Unity editor
<li>I dragged the tower.fbx into that folder
<li>And then I dragged the resulting tower prefab on the HologramCollection. It will be<em> very</em> large.
<li>Set its position to far far away, like 5000,5000,5000 to be out of the way of what we have already created.
<li>Double click on “Tower” to get in into view. I got about this:</li></ul>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_10.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_10.png" width="304" height="387"></a></p>
<p>Rotate around it till you get about this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_11.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_11.png" width="304" height="314"></a></p>
<p>Now what we want to get rid off, is the <em>building</em> next to the tower. I don’t know if I am legally entitled to <em>change</em> a model, so I limit myself to <em>hiding parts of it</em>. This works as follows.</p>
<ul>
<li>First, make sure this button is selected. </li></ul>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_12.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 6px 37px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_12.png" width="189" height="65"></a></p>
<ul>
<li>Select the model by clicking on the <em>building. </em>Not the tower. This will select the whole model nonetheless </li>
<li>Click the building again. This will only select the building
<li>Go to the inspector. Uncheck the checkbox all the way to the top left</li></ul>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_13.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_13.png" width="304" height="134"></a></p>
<p>That will get rid of most of the unwanted stuff:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_14.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_14.png" width="304" height="330"></a></p>
<p>See? Rinse and repeat until all the stuff you don’t want is invisible. Remember: click once selects the whole model, click twice to select only what you want. Hidden too much? CTRL-Z is your friend. It’s a bit of a hassle, especially the small floating bits just right of the tower, but in the end you should be left with just this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_15.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_15.png" width="304" height="289"></a></p>
<p>When you are done, scale the tower to 0.001,0.001,0.001 and put it’s location to 0,0,0. Select the Main Camera, then GameObject/Align View to Selected and you will see Schiphol again, with a tiny ATC tower standing on top of it. If you look at it almost dead from above, you will see that it’s a bit off to the right and the bottom, according to this aerial picture from Google Maps that ironically enough blocks the view of the actual tower itself with a label – so I marked it with a red circle</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_16.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_16.png" width="244" height="222"></a><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_17.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; margin: 0px 0px 0px 15px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_17.png" width="287" height="222"></a></p>
<p>As it is the wrong tower – and as it’s not to scale, I did not bother with calculating it exact position but dragged it around a bit until it looked like it was in the same spot, which was –0.0283 for X and 0.0316 for Z. Fiddling a bit around with the view spot so I could see the tower’s base hitting the ground made it look like this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_18.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-6Adding-an-a_8491/image_thumb_18.png" width="393" height="304"></a></p>
<p>And that’s fine with me. I’ll call it a day.</p>
<h2>Conclusion</h2>
<p>I have shown you how to orientate, scale and properly position (geographical) maps and objects on those maps in Unity, using some GIS knowledge, calculus, common sense and fiddling around. This makes your app look just that little extra pretty, plus it bases it in ‘real space’ which makes it extra cool.</p>
<p>Code – well, more like assets and config, <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog6" target="_blank">can be found here</a>. And I’d like to say hi to my fellow MVP <a href="https://twitter.com/tomverhoeff" target="_blank">Tom Verhoeff</a> and his trainee Ryan, who seem to like this tutorial so I hope I made both their days ;)</p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/kmScGK-FH94" height="1" width="1" alt=""/>2016-10-31T11:33:23.826+01:002http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-6adding.htmlA HoloLens airplane tracker 5-smooth movement with iTween and adding a trailhttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/31hxc0mH9_I/a-hololens-airplane-tracker-5-smooth.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Mon, 17 Oct 2016 11:54:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-6393857213162113572<h2>Intro</h2> <p>In the video <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-1.html" target="_blank">in the first post</a> of this series&nbsp; you can see the airplane models leaving a kind of trails in the air, while they move and rotate in a smooth fashion - in stead of merely hopping from place to place. The first is accomplished by adding a <a href="https://docs.unity3d.com/Manual/class-TrailRenderer.html" target="_blank">TrailRender</a> to the AircraftHolder prefab – the second by using an awesome Unity plugin called <a href="http://itween.pixelplacement.com/gettingstarted.php" target="_blank">iTween</a>. </p> <h2>Adding the trail</h2> <p><a href="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; margin: 0px 6px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image_thumb.png" width="304" align="right" height="333"></a>Before you can add the trail, it must have a <em>material</em> from which it’s made. This the color of the line. I will freely admit materials are still a bit vague to me, and after quite some fiddling I came to the following settings:</p> <ul> <li>Shader: HoloToolkit/Fast <li>Albedo: hex color 120067FF <li>Metallic: 1 <li>Emission: hex color 021F61</li></ul> <p>This leads to a kind of metallic light blue to dark blue trail line (depending on where the ‘directional light’) is shining, which is quite satisfactory. Feel free to mess around with the parameters. I did ;).</p> <p>Then go to App/Scripts and open the AircraftHolder prefab. Click on “Add Component”, then select “Trail Render”. Use the following workflow/ setting:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; margin: 0px 4px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image_thumb_3.png" width="304" align="right" height="219"></a></p> <ul> <li>Expand “Materials”. Drag the Trail Line Material that we just created on top of the “Element 0” field <li>Set start width to 0.003 <li>Set end width to 1e-05.</li></ul> <p>This means the trail will be visible for 120 seconds, so the tail of the trail line will be at the place where the airplane was two minutes ago. Older segments are automatically deleted. Also, near the airplane the trail will be 0.003 meter wide, petering out to 0.00001 meter (1e-05) at the very end. And sure enough:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/Smooth-movingrotating-with-iTween-and-ad_11D63/image_thumb_4.png" width="674" height="381"></a></p> <p>Unity takes care of the drawing, erasing, and making-smaller-toward-the-end part. The only thing we have to do is move the airplane. Which is what we were doing already anyway. It once again almost feels like cheating.</p> <h2><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image_thumb.png" width="244" align="right" height="242"></a>Introducing and obtaining iTween</h2> <p>In a <a href="http://dotnetbyexample.blogspot.com/2016/07/hololens-cubebouncer-application-part-4.html" target="_blank">blog post about an earlier project, the CubeBouncer</a>, I talked about using <a href="http://docs.unity3d.com/Manual/Coroutines.html" target="_blank">coroutines</a> to achieve smooth animations. In the mean time I have stumbled upon <a href="http://itween.pixelplacement.com/gettingstarted.php" target="_blank">iTween</a>, an awesome Unity plugin created by one <a href="http://www.itween.pixelplacement.com/credits.php" target="_blank">Bob Berkebile</a>, that makes creating animations of all kinds a <em>lot</em> easier. Although some concepts are a bit odd – particularly the the use of the Hash object to pass in an arbitrary number of parameters into methods – once you wrap your head around it, it’s very easy to use. Oh. And it’s free too.</p> <p>The easiest way to obtain it, is via the Unity Asset Store (Window/Asset Store or CTRL-9). Then enter iTween as search term. Click the iTween Logo, click import, and then the following dialog, as displayed to the right (below this paragraph), pops up. I usually deselect everything but the actual plugin, but if you would like to see samples and the Readme, feel free to leave everything checked.<a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image_thumb_3.png" width="245" align="right" height="274"></a></p> <p>When the process is finished, your project tab should contain the iTween plugin like below</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-5-smooth-mov_EA12/image_thumb_4.png" width="169" height="179"></a></p> <p>Once you have the iTween plugin installed, build the project, and move over to Visual Studio.</p> <h2>Using iTween for moving and rotating</h2> <p>So we head back to the AircraftController’s SetLocationOrientation method. We are going to change that a little:</p><pre>private void SetLocationOrientation()
{
SetNewFlightText();
if (!_firstMove)
{
transform.localPosition = GetFlightLocation();
if (_flightData.Heading != null)
{
transform.localEulerAngles = GetNewRotation();
}
transform.localScale = new Vector3(0.0015f, 0.0015f, 0.0015f);
_firstMove = true;
}
else
{
iTween.MoveTo(gameObject, iTween.Hash("position", GetFlightLocation(),
"time", 7f, "islocal", true));
if (_flightData.Heading != null)
{
iTween.RotateTo(gameObject, iTween.Hash("rotation", GetNewRotation(),
"time", 7f, "islocal", true));
}
}
}</pre>
<p>So if the first set of data (or the initial move to a location) as has not been applied yet – that is, this is a newly created airplane – just do the same as we did before – plonk the airplane at it’s location with the proper rotation and attitude, and give it a size that makes sense. But if this aircraft already exists – move it smoothly to it’s new location in 7 seconds, using local position. And rotate it too, if needed, in the same time. And that is <em>all</em> <em>you need for a smooth transition.</em> Here you see the Hash in action. iTween has a few overloads for both Move and Rotate where you don’t need it, but in effect, when you want to do something only a little advanced – like using local space – say hello to Mr. Hash. </p>
<p>You can of course rant against these ‘stringly typed’ variables and talk about type safety and stuff like that – but this is how it works, and it’s perfectly usable, albeit a bit odd for those who are fans of tight architecture and clean code. But then again, HoloLens is a pretty out-of-the-ordinary device anyway. When in Rome, act like Romans. ;)</p>
<h2>Making the text fade in and out</h2>
<p>Now the only thing we are missing is that nice effect where the text fades out just before the airplane moves – and appears again – with the newly updated flight data. This prevents the text from flashing and also gives a nice indication whether or not we are receiving updates for an aircraft (you may have noticed by now that aircraft that have landed tend to hang around for a while just above the airport before disappearing).</p>
<p>First, we need to add another field:</p><pre>private GameObject _label;</pre>
<p>And of course, initialize that in Start:</p><pre>void Start()
{
_text = transform.GetComponentInChildren&lt;TextMesh&gt;();
<strong>_label = transform.FindChild("Label").gameObject;
</strong> _initComplete = true;
}</pre>
<p>I already warned you Start would grow, didn’t I? ;) Then we add a new method:</p><pre>private void StartSetLocationOrientation()
{
if (_firstMove)
{
iTween.FadeTo(_label,
iTween.Hash("alpha", 0.0f, "time", 0.5f,
"oncomplete", "SetLocationOrientation",
"oncompletetarget", gameObject));
}
else
{
SetLocationOrientation();
}
}</pre>
<p>So when the first move is indeed done, it will fade the label’s alpha channel to 0 in half a second. When that is done, call the original SetLocationOrientation. So this is a <em>callback</em>. What is a <em>very</em> important iTween convention to understand – default the callback will be called <em>on the object you are operating on</em> – and that is the <em>label.</em> Not the airplane. Therefore I added an “oncompletetarget” entry with the current gameObject as value – so iTween knows it should call SetLocationOrientation and use my gameObject as a target, not the label. </p>
<p>Oh, in the else - if the airplane was just created and does not have an initial location - just move it to it’s initial location without using iTween.</p>
<p>Then we move to SetNewFlightData and change the call to SetLocationOrientation to <strong>Start</strong>SetLocationOrientation: </p><pre>public void SetNewFlightData(Flight newFlightData)
{
if (_initComplete)
{
var move = _flightData == null ||
!_flightData.Location.LocationEquals(newFlightData.Location);
_flightData = newFlightData;
ExtractSpeedAndHeading();
if (move)
{
<strong>Start</strong>SetLocationOrientation();
}
else
{
SetNewFlightText();
}
}
}</pre>
<p>And finally, to SetLocationOrientation we add all the way to the end this line:</p><pre>iTween.FadeTo(_label, iTween.Hash("alpha", 1f, "time", 0.5f, "delay", 0.2f));</pre>
<p>This fades the label’s alpha channel (back) to 1, in 0.5 second, after a delay of 0.2 seconds. I found these values after experimenting. Feel free to play with times and stuff.</p>
<h2>Conclusion</h2>
<p>I have shown you how to add a trail line to an airplane, and how to use iTween for smooth transition effects. Once the basics of your HoloLens app stand, it’s quite easy to make just a little more effort and get it smooth and cool. Using a powerful tool like iTween makes it almost child’s play IMHO, once you get your head around how to using it.</p>
<p>Code, as always, <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog5" target="_blank">can be found here</a>. </p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/31hxc0mH9_I" height="1" width="1" alt=""/>2016-10-20T08:01:27.100+02:002http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-5-smooth.htmlA HoloLens airplane tracker 4–Reading data and positioning airplaneshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/52UKk1l_VYE/a-hololens-airplane-tracker-4reading.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 15 Oct 2016 05:05:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-4004494303119930123<h2>Time to rock and roll</h2> <p>All right my friends, it’s time to show the heart of the matter – how to read data from an Azure Mobile App Services data service and make the airplanes appear in the right places. </p> <h2>Adding the JSON data object scripts</h2> <p>As the projects are partially regenerated, and have a total different .NET baseline, I have not found a good way to share classes as binaries between full .NET code and Unity without getting in all kinds of trouble, so I choose the easy way out: I took</p> <ul> <li>TrackType.cs <li>Coordinate.cs <li>Flight.cs</li></ul> <p>from the FlightDataService\DataObjects, moved them the App/Scripts/DataObjects folder in the Unity project. Then of course the service does not compile anymore, but I added the moved file <em>as link</em>. Then at least we have once source of truth as far as these classes are concerned.</p> <p>Then we need a simple class to hold a list of flights with a timestamp (add that to the DataObjects folder as well):</p><pre>using System;
using System.Collections.Generic;
namespace FlightDataService.DataObjects
{
public class FlightSet
{
public FlightSet()
{
}
public FlightSet(List&lt;Flight&gt; flights)
{
Flights = flights;
TimeStamp = DateTimeOffset.Now;
}
public DateTimeOffset TimeStamp { get; set; }
public List&lt;Flight&gt; Flights { get; set; }
}
}</pre>
<h2>Adding a DataService to read data</h2>
<p>I opted to add the DataService as a Singleton that can be accessed in the app. But we have a bit of an issue here -&nbsp; Unity runs on Mono, e.g. .NET 3.something, and does not have things like packages for reading App services, or support async and Tasks and stuff. But remember – a HoloLens app is a two stage rocket. Unity does not generate an app for the HoloLens, but <em>an UWP project</em>, and that UWP project <em>can</em> use all the goodness from all the latest stuff. Meet the simple way to close the gap - my new friend UNITY_UWP.</p><pre>using System;
using System.Collections.Generic;
using FlightDataService.DataObjects;
using HoloToolkit.Unity;
#if UNITY_UWP
using Microsoft.WindowsAzure.MobileServices;
using System.Net.Http;
using System.Threading.Tasks;
#endif
public class DataService : Singleton&lt;DataService&gt;
{
public string DataUrl = "http://yourflightdataservice.azurewebsites.net/";
#if UNITY_UWP
private MobileServiceClient _client;
#endif
public DataService()
{
#if UNITY_UWP
_client = new MobileServiceClient(new Uri(DataUrl));
#endif
}
#if UNITY_UWP
public async Task&lt;List&lt;Flight&gt;&gt; GetFlights()
{
var result = await _client.InvokeApiAsync&lt;List&lt;Flight&gt;&gt;(
"FlightData", HttpMethod.Get, null);
return result;
}
#endif
}
</pre>
<p>Code between #if UNITY_UWP – #endif blocks is invisible to Unity – <em>but will be accessible in the UWP HoloLens app that is generated by Unity</em>. You will see Unity swallows it – no problems at all. There is only this tiny thing – you cannot add NuGet packages to Unity. So we have to do that <em>in the UWP app</em>, in Visual Studio. But… the UWP app is generated from Unity by File/Build and overwritten. The trick is to know not the whole app is overwritten, but only part of it. What I did was the following:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_12138/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_12138/image_thumb.png" width="244" align="right" height="174"></a></p>
<ul>
<li>Generate the app in a subfolder “App” of the Unity project. This should be &lt;your_root&gt;\HoloATC_Demo\AMS HoloATC Demo\App
<li>Open the solution in HoloATC_Demo\AMS HoloATC Demo\App (<em><strong><u>not </u></strong>the one in HoloATC_Demo\AMS HoloATC Demo</em>)
<li>Go to the project Assembly-Csharp
<li>Add the NuGet Package Microsoft.Azure.Mobile.Client
<li>Find the project.json file that belong to this project. You will find it’s not under AMS HoloATC Demo\App at all, but in AMS HoloATC Demo\UWP\Assembly-CSharp\project.json
<li>This is the only file that has been changed with respect to the generated code. <em>You will have to add this to source control</em>. Even if Git claims it’s an ignored file.
<li>After that – if you checkout the source in a different location or on a different machine, you will only have to regenerate the solution in the right place (the App folder), and revert this specific file. You will need to do this, or else the UWP app won’t compile.</li></ul>
<h2>Adding the AircraftLoader to create and update aircraft</h2>
<p>I have created one class that is actually responsible for actually getting the aircraft data from the DataService – and one that is responsible for creating, updating and deleting aircraft. The updating – moving to a new position – is handled by the Aircraft<em>Controller</em> – basically all the AircraftLoader says to the airplane is ‘here is new data for you – handle it’ </p>
<p>Remember – it’s always best to start in Unity. So create an AircraftLoader and an AircraftController script in Assets/App/Scripts. Then proceed to double-click on AircraftLoader. This will open a Visual Studio instance. Be aware – this is not the project your will be running. This is the other solution, the one that is in the project root – I call this the ‘Edit’ solution. First, we are going to make sure the “usins” of the class are properly organized – that is, in a way that doesn’t make Unity go belly-up:</p><pre>using System;
using UnityEngine;
using System.Collections.Generic;
using System.Linq;
using FlightServices.Data;
#if UNITY_UWP
using System.Threading.Tasks;
using Windows.Web.Http;
using Newtonsoft.Json;
#endif</pre>
<p>Then we are going to add a whole lot of fields. And yes, some are public. These are things that can be set from the Unity editor. I have talked about that before.</p><pre>public GameObject Aircraft;
public string TopLevelName = "HologramCollection";
private Dictionary&lt;string, GameObject&gt; _aircrafts;
private readonly TimeSpan _waitTime = TimeSpan.FromSeconds(5);
private DateTimeOffset _lastUpdate = DateTimeOffset.MinValue;
private Queue&lt;FlightSet&gt; _receivedData;
private GameObject _topLevelObject;</pre>
<ul>
<li>The Aircraft GameObject field will be used to drag our AircraftHolder onto – so this behaviour knows which game object to create
<li>TopLevelName is the name of the parent object to which all the aircraft are added to. This&nbsp; “HologramCollection” by default
<li>_aircrafts is a dictionary of aircraft game objects with their id as key, so we can update/delete existing aircraft game objects based upon the data coming in
<li>waitTime – minimal time to load new data
<li>_lastUpdate – the time the last update of aircraft was completed
<li>_receivedData – a queue with data coming from the service. I use this pattern all the time. I queue up data from the service, then read it in the Update method that Unity calls automatically 60 times a second. Don’t ever try to update game objects from .NET callback methods or events – you might regret it, or just get plain crashes – kind of like happens in UWP XAML apps, where you need the Dispatcher to take care of that
<li>_topLevelObject – to store the game object with the TopLevelName in it. I have been told the method I use to find an object by name is quite heavy on performance – so better do it once and retain the result, right.</li></ul>
<p>The Start method simply initializes some stuff:</p><pre>void Start()
{
_aircrafts = new Dictionary&lt;string, GameObject&gt;();
_receivedData = new Queue&lt;FlightSet&gt;();
_topLevelObject = GameObject.Find(TopLevelName);
}</pre>
<p>And then there’s this little method – that is used as callback for the DataService’s GetFlight method:</p><pre>#if UNITY_UWP
private void ProcessData(Task&lt;List&lt;Flight&gt;&gt; flightData)
{
if (flightData.IsCompleted &amp;&amp; !flightData.IsFaulted)
{
var set = new FlightSet(flightData.Result);
_receivedData.Enqueue(set);
}
}
#endif</pre>
<p>In short - when the data received and it is ok, just add it to the queue of received data. And then let Update handle it. As a matter of fact – like this:</p><pre>private bool _isUpdating;
void Update()
{
if ((_lastUpdate - DateTimeOffset.Now).Duration() &gt; _waitTime)
{
_lastUpdate = DateTimeOffset.Now;
#if UNITY_UWP
DataService.Instance.GetFlights().ContinueWith(ProcessData);
#endif
}
if (!_isUpdating)
{
_isUpdating = true;
if (_receivedData.Any())
{
var set = _receivedData.Dequeue();
var flightIds = set.Flights.Select(p =&gt; p.Id).ToList();
var aircraftToDelete =
_aircrafts.Keys.Where(p =&gt; !flightIds.Contains(p)).ToList();
DeleteAircraft(aircraftToDelete);
var keysToUpdate = _aircrafts.Keys.Where(p =&gt; flightIds.Contains(p));
var aircraftToUpdate =
set.Flights.Where(p =&gt; keysToUpdate.Contains(p.Id)).ToList();
UpdateAircraft(aircraftToUpdate);
var aircraftToAdd =
set.Flights.Where(p =&gt; !_aircrafts.Keys.Contains(p.Id)).ToList();
CreateAircraft(aircraftToAdd);
}
_isUpdating = false;
}
}</pre>
<p>First it finds out if it’s necessary to download new data, but downloading and adding happens asynchronously. Then, if there’s any data in the queue, it first makes a list of all the id’s in the newly received flights.</p>
<ul>
<li>Aircrafts that are in the aircraft game object dictionary (with the flight id as key) but are no longer in the list of flights, can be deleted (they have landed or moved out of Dutch airspace)
<li>Aircraft that appear in that dictionary <em>and</em> in the list of flights need to be updated – possibly moved to a new position
<li>Aircraft that do <em>not</em> have an entry in the game object dictionary are new, and need to be created. </li></ul>
<p>It’s not that hard, see ;). The methods for creating, updating and deleting aircrafts are not that hard either.</p><pre>private void CreateAircraft(IEnumerable&lt;Flight&gt; flights)
{
foreach (var flight in flights)
{
var aircraft = Instantiate(Aircraft);
aircraft.transform.parent = _topLevelObject.transform;
aircraft.transform.localScale = new Vector3(0f, 0f, 0f);
SetNewFlightData(aircraft, flight);
_aircrafts.Add(flight.Id, aircraft);
}
}
private void UpdateAircraft(IEnumerable<flight> flights)
{
foreach (var flight in flights)
{
if (_aircrafts.ContainsKey(flight.Id))
{
var aircraft = _aircrafts[flight.Id];
SetNewFlightData(aircraft, flight);
}
}
}
private void DeleteAircraft(IEnumerable&lt;string&gt; keys)
{
foreach (var key in keys)
{
var aircraft = _aircrafts[key];
Destroy(aircraft);
_aircrafts.Remove(key);
}
}</pre>
<p>The CreateAircraft is the most interesting – it instantiates the aircraft game object, makes the HoloGramCollection it’s parent, passes the actual flight data to the object, and adds it to the dictionary of aircraft game objects using the flight id as the key. Oh, and it also sets the scale to 0, making the plane effectively invisible. This is because, without setting a location on instantiation, the aircraft will appear on 0,0,0 before – where in future episodes we will place the center of Schiphol, very near the tower, and that’s not a place where aircraft belong. </p>
<p>UpdateAircraft just sends flight data to the game object, and DeleteAircraft – well, deletes it. There is only this matter of SetNewFlightData: </p><pre>private void SetNewFlightData(GameObject aircraft, Flight flight)
{
var controller = aircraft.GetComponent&lt;AircraftController&gt;();
if(controller != null)
{
//controller.SetNewFlightData(flight);
}
}</pre>
<p>But that has the actual method that moves the flight data to the AircraftController commented out, because that does not even <em>exist</em>. What is worse, the Aircraft does not even have to component. So let’s head over back to the Unity Editor.</p>
<h2>Wiring up some stuff in Unity </h2>
<p>First of all, we drag the DataService and AircraftLoader Script from the App/Scripts folder on top of the HologramCollection’s inspector page. Then we drag AircraftHolder prefab from App/Prefabs on top of the AircraftLoader’s “Aircraft” property. Net result should be this:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_A318/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_A318/image_thumb.png" width="354" height="240"></a></p>
<p><em>Please make sure the Data Url property of the Data Service indeed points to the place where your data service <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-1.html" target="_blank">as created in the first post</a> is published.</em></p>
<p>Then go to the App/Prefabs folder, open the AircraftHolder prefab itself, and drag the (still default) AircraftController on top of the AircraftHolder’s inspector page.</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_A318/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_A318/image_thumb_3.png" width="354" height="178"></a></p>
<p>Save the scene, then rebuild the app with File/Build settings etc. Go back to Visual Studio once the building is finished.</p>
<h2>Creating the AircraftController</h2>
<p>Having programmed OO since the start of this century, I tend to put control where I logically think it belongs, so rather than programming the ‘flight logic’ into a class that loads and translates data too (the AircraftLoader) I opted for putting it into a separate class that would be part of the game object. I tend to conceptualize these combined game objects and components as OO objects - although that is not completely right, it helps me think about it. The start is simple enough</p><pre>public class AircraftController : MonoBehaviour
{
private Flight _flightData;
private float? _speed;
private float? _heading;
private TextMesh _text;
private bool _initComplete;
private bool _firstMove;
void Start()
{
_text = transform.GetComponentInChildren&lt;TextMesh&gt;();
_initComplete = true;
}
}</pre>
<p>A few private fields to retain some data. </p>
<ul>
<li>_flightData just keeps the last provide flight data available
<li>_speed and _heading keep the last speed and heading. The stream of data sometimes misses a beat and provides no speed and/or heading – I prefer then to display the latest data, in stead of heaving the aircraft suddenly rotate to the North (heading 0) and back again when the next set of data is correct again
<li>_text keeps a reference to the text mesh so I don’t have to look it up every update - potentially 60 times a second
<li>the _initComplete boolean is a trick I use regularly to prevent other routines using variables like _text before the are initialized. Remember, the Update loop is called independently of what you do. It may look a bit overdone now with only one initialization statement, but believe me – there will be more.</li></ul>
<p>Well then, finally the infamous SetNewFlightData method</p><pre>public void SetNewFlightData(Flight newFlightData)
{
if(_initComplete)
{
var move = _flightData == null ||
!_flightData.Location.LocationEquals(
newFlightData.Location);
_flightData = newFlightData;
ExtractSpeedAndHeading();
if (move)
{
SetLocationOrientation();
}
else
{
SetNewFlightText();
}
}
}</pre>
<p>I like to write the code at high level as almost self-explanatory. First we determine if the aircraft needs to be moved, then we ingest the data, and extract speed and heading. Then, when the aircraft needs to be moved, change it’s location, if not, just update the label. The ExtractSpeedAndHeading is pretty straightforward. There is only the thing with heading - that sometimes comes in a negative value, and although Unity3D has no problem with that in positioning the aircraft, I think it looks ugly in the label. So I make sure it's always positive.</p><pre>private void ExtractSpeedAndHeading()
{
if (_flightData.Heading != null)
{
_heading = (float)_flightData.Heading;
}
if (_heading &lt; 0)
{
_heading += 360;
}
if (_flightData.Speed != null)
{
_speed = (float)_flightData.Speed;
}
}</pre>
<p>SetFlightText is also rather trivial </p><pre>private void SetNewFlightText()
{
var speedText =
_speed != null ? string.Format("{0}km/h", _speed) : string.Empty;
var headingText =
_heading != null ? string.Format("{0}⁰", _heading) : string.Empty;
var text = string.Format("{0} {1} {2}m {3} {4}", _flightData.FlightNr,
_flightData.Aircraft, _flightData.Location.Alt, speedText,
headingText).Trim();
_text.text = text;
}</pre>
<p>Just some clever formatting to prevent empty postfixes like km/h and degrees in the label. By the way – stick to ye olde string.Format and don’t be tempted to use C# 6 string interpolation <a href="http://dotnetbyexample.blogspot.com/2016/07/gotchathe-associated-script-could-not.html" target="_blank">or you will be sorry</a> (unless you put it between “#if UNITY_UWP – #endif). Anyway, on to the next routine, that actually does all the aircraft manipulation:</p><pre>private void SetLocationOrientation()
{
SetNewFlightText();
transform.localPosition = GetFlightLocation();
if (_flightData.Heading != null)
{
transform.localEulerAngles = GetNewRotation();
}
if (!_firstMove)
{
transform.localScale = new Vector3(0.0015f, 0.0015f, 0.0015f);
_firstMove = true;
}
}</pre>
<p>It sets the text too, then set’s the location based on the flight’s location, and set rotation based upon the heading of the aircraft <em>and</em> whether it’s going up or down. Notice I use <em>local</em> position, heading, and scale. This means all those things are <em>relative</em> to the position, rotation and scale <em>of the containing GameObject</em> – HologramCollection. This has the advantage that I can move, rotate and scale the containing object and <em>in one go everything that is in it follows suit</em>. So you don’t have to do all calculations for that – Unity takes care of that form me. I don’t use it in this app just yet, but the actual app already has some experimental code to do that (although that code is not yet in the store version).</p>
<p>Also, note the fact the airplane gets it’s size here (when it’s created it’s 0,0,0). I have the feeling the model is actually a 1:1 scale representation of the actual aircraft – the first time I saw it with the HoloLens I could not find it at first, then turned around and had a “<a href="https://en.wikipedia.org/wiki/Falling_from_the_Sky:_Flight_174" target="_blank">Glimly Glider</a> experience” - a giant aircraft silently swooping down on me from what looked only 10-15 meters. I decide to scale it down to 0.0015 of it’s original size so it appears to be about 10-15cm in a HoloLens – a size more beneficial for getting an good overview, not to mention the blood pressure and heart rate of the average user ;).</p>
<p>Next up are these two routines:</p><pre>private Vector3 GetFlightLocation()
{
return GetLocalCoordinates(_flightData.Location);
}
private Vector3 GetLocalCoordinates(Coordinate c)
{
return new Vector3((float)c.X / 15000,
c.Alt != null ? (float)c.Alt / 2000.0f : 0f,
(float)c.Y / 15000);
}</pre>
<p>I already discussed the how and why of scaling the down coordinates 15000 times in horizontal direction and 2000 times in vertical direction in a <a href="http://dotnetbyexample.blogspot.com/2016/09/converting-latlon-coordinates-to-local.html" target="_blank">recent blog post about converting lat/lon/alt coordinates into the Unity3D X/Y/Z system</a>, so I am not going through that again, because the next part is a lot more interesting. First I will show the last method, GetVerticalAngle</p><pre>private float GetVerticalAngle()
{
var tracksize = _flightData.Track.Count;
if (tracksize &gt; 2)
{
var pLast = _flightData.Track[tracksize - 1];
var pSecondLast = _flightData.Track[tracksize - 3];
var delta = pLast.Alt - pSecondLast.Alt;
if (Math.Abs(delta.Value) &gt; 2.5f)
{
return delta &lt; 0 ? 10 : -20;
}
}
return 0;
}</pre>
<p>I found that the data, although it provides information about whether the aircraft is actually ascending or descending, that data is not always correct. So I decided to calculate that myself, based upon the difference between the current location and an older location. Now since an aircraft usually descends a lot slower than it takes off (which is very fortunate for the passenger’s – or at least my – peace of mind) this method basically returns –20 when the aircraft is going up, and 10 when it’s going down. And then we get some beautiful Unity3D math again – or more accurately, methods that <em>prevent</em> you from having to use all kinds of advanced 3D math: </p><pre>private Vector3 GetNewRotation()
{
var heading = _heading ?? 0;
var rotation = Quaternion.AngleAxis(heading, Vector3.up).eulerAngles +
Quaternion.AngleAxis(GetVerticalAngle(), Vector3.right).eulerAngles;
return rotation;
}</pre>
<p><img style="float: right; display: inline" src="http://2.bp.blogspot.com/-18VdaBXv2sM/T9GzlEMuu3I/AAAAAAAAAEE/SCtfkqZlEYs/s1600/roll+pitch+yaw.png" width="227" align="right" height="179">Try to picture in your mind how this works:</p>
<ul>
<li>For the <em>heading</em> we have to rotate around the axis that is going <em>up</em> (and down, too) from the center of the aircraft – this what they call <em>yaw</em> in aviation
<li>For the vertical angle – that makes it look whether the aircraft is going up or down – we have to rotate around the axis that goes to the <em>right </em>(and left – so basically over the wings) from the center of the aircraft. In aviation, this is called <em>pitch</em>&nbsp;</li></ul>
<p>You use summarize Quaternion.AngleAxis(angle, Vector3.&lt;the axis you desire&gt;).eulerAngles over multiple axes to get the combined rotation of an object and assign that in one go. Hence you see in SetLocationOrientation() the statement "transform.localEulerAngles = GetNewRotation();"</p>
<h2>The final things</h2>
<p>Go to <em><strong>AircraftLoader</strong></em> 's SetNewFlightData and uncomment the line <pre> //controller.SetNewFlightData(flight); </pre>
<p>Because we have implemented that in the previous section. Then there’s is the method LocationEquals in Coordinate, that we used in AircraftController but that was not implemented yet ;)</p><pre>public bool LocationEquals(Coordinate other)
{
if (other == null) return false;
return (X == other.X &amp;&amp; Y == other.Y &amp;&amp; Alt != null &amp;&amp;
other.Alt != null &amp;&amp;
Alt.Value == other.Alt.Value);
}</pre>
<p>And if you run this in a HoloLens or the Emulator, you will see aircraft!</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_12138/image_3.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-4Reading-dat_12138/image_thumb_3.png" width="634" height="358"></a></p>
<p>You will notice they won't move through the air but jump from position to position. They also don't show their track, there is no Schiphol Airport map, no ATC tower, no church, no gaze cursor - and you cannot select anything yet - but that is because this post is long enough as it is. The base is here. 3D visualization of a JSON stream. What is left, is basically making things more slick :) </p>
<h2>Conclusion</h2>
<p>In hindsight I might better have splitted this episode in two blog post still, but I hope you have made it to the end. I feel this blog post is the heart of the matter – reading a data stream and turning it into a 3D model – making ‘dry records’ come to life, almost literally. I also showed you some key concepts about positioning and rotating stuff in 3D. </p>
<p>As usual, the code is <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog4" target="_blank">here on GitHub.</a></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/52UKk1l_VYE" height="1" width="1" alt=""/>2016-10-15T14:12:55.631+02:004http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-4reading.htmlA HoloLens airplane tracker 3–Creating an annotated airplanehttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/EN7wzqwN1ZY/a-hololens-airplane-tracker-3creating.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Sat, 08 Oct 2016 01:51:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-5858039139626991673<h2>Intro</h2> <p>In this post we are going to create a game object holding the aircraft and a label showing aircraft data that always looks at the user. We are not going to worry about scaling yet – we will handle that from code, in the next blog post. </p> <h2>First some housekeeping</h2> <p>Just as in the CubeBouncer project, we are going to create some folders in the Assets first to keep our custom assets nicely separated. Create a folder App, and in this folder the following subfolders:</p> <ul> <li>Audio <li>Materials <li>Prefabs <li>Scripts <li>Textures <li>UWPAssets</li></ul> <h2>Creating the Aircraft Holder and including the aircraft</h2> <p>Inside the HologramCollection, create an empty GameObject called AircraftHolder. Then drag into that object the A320 object form the A320 folder. A rather large A320 appears in the Scene, as indicated in the <a href="http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-2setting-up.html" target="_blank">previous blog post</a>. Just ignore this for now.Open the A320 object (in the hierarchy, not the object in the A320 folder), disable the animator, and make sure X,Y,Z position and rotation are all 0,0,0 and scale 1,1,1: </p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb.png" width="354" height="258"></a></p> <p>At this point I have no idea what the Animator is supposed to do, but we don’t need it – I think. </p> <h2>Create a label</h2> <p>In this section we create the label above the airplane that displays flight information, like flight number, aircraft type, speed, altitude, etc. Right-click the AicraftHolder and create a “3D Text” Object. Call this “Label”. That label now sits at the bottom of the airplane, where apparently it’s origin is.</p> <p>Open the label in the inspector, set the Y position to 15 and the Y rotation to 270. That places the text 15 meters above the airplane, placing it parallel above the fuselage, making the default text “Hello World” readable from the right side of the airplane</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_3.png" width="354" height="106"></a></p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_4.png" width="354" height="141"></a></p> <p>The go down to the “Text Mesh”. Change the following things:</p> <ul> <li>Anchor to lower center <li>Change font size to 80 <li>Change the color to red</li></ul> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_5.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_5.png" width="354" height="201"></a></p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_6.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_6.png" width="354" height="170"></a></p> <p>Now hit “Add component”, and add a Mesh Collider. Select the “Convex” checkbox:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_7.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_7.png" width="354" height="75"></a></p> <p>Of course, it’s not very practical if the airplane’s data is only readable from the left side. So to close off this article, we actually going to write code – by adding a simple script that keeps the text readable from every angle. Go to the Assets/App/Scripts folder, right-click it and hit “Create Script”. Call the script “LookAtCamera”. Double-click it – that will open Visual Studio – and change the update method to this:</p><pre>void Update()
{
gameObject.transform.LookAt(Camera.main.transform);
gameObject.transform.Rotate(Vector3.up, 180f);
}</pre>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_8.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_8.png" width="244" align="right" height="76"></a>Save the file, go back at Unity. Drag the new script on top of the Label inside the AircraftHolder game obejct To see this actually works: hit the Unity “Play” button</p>
<p>Now the fun thing is – as long as you are in play mode, changes in settings are not stored so this is great for fooling around. Initially your game screen looks like this – you are looking from under the plane right up to the front landing gear.</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_9.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_9.png" width="354" height="300"></a></p>
<p>Select the AircraftHolder in the hierarchy, then change Z to 500. </p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_10.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_10.png" width="354" height="261"></a></p>
<p>The plane flicks forward… and the text is still readable although we are not looking form the left, but the back. Change rotation X to –20 and rotatation Y to –30:</p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_11.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_11.png" width="354" height="285"></a></p>
<p>The plane is rotated, we are now looking more or less form the right, but the text <em>is still pointing straight to you</em>, and is readable. Mission accomplished. Not bad for two lines of code!</p>
<p>Exit Play mode. Go to the label once more, and remove the text “Hello World”. The label seems to disappear, but it’s still there, it’s now just empty. It will be filled by code.</p>
<h2>And finally…</h2>
<p>Select the Prefab folder in Assets/App, and drag the AircraftHolder into it. After you have done it, remove it from the HologramCollection. And save the Scene.</p>
<p>Net result: </p>
<p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_12.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-3_C931/image_thumb_12.png" width="354" height="539"></a></p>
<p>The reason why I am making the AircraftHolder in stead of adding components directly to the A320 model – which is perfectly possible – is that by using this way I can easily switch out the actual airplane model, while all logic and additional components are preserved. If you find a different (more pretty) aircraft 3D model this way saves you work.</p>
<p>Code so far can be <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog3" target="_blank">downloaded from Github here.</a></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/EN7wzqwN1ZY" height="1" width="1" alt=""/>2016-11-22T10:18:03.874+01:004http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-3creating.htmlA HoloLens airplane tracker 2–Setting up the project and getting airplane assetshttp://feedproxy.google.com/~r/blogspot/dotnetbyexample/~3/x004v0UaGqY/a-hololens-airplane-tracker-2setting-up.htmlHoloLensUnity3DUWPnoreply@blogger.com (Joost van Schaik)Thu, 06 Oct 2016 12:38:00 PDTtag:blogger.com,1999:blog-5295746446529817470.post-6699867580980052515<h2>Intro</h2> <p>All right my friends, it’s time to hit Unity again. In a previous <a href="http://dotnetbyexample.blogspot.com/2016/07/hololens-cubebouncer-application-part-1.html" target="_blank">blog post about the Cube Bouncer</a> I have told you how to set up a project. That was quite a convoluted procedure. Luckily, things have improved considerably since, especially where it comes to the HoloToolkit. So this ‘setting up’ post will be a lot shorter and simpler.</p> <h2>Creating the project</h2> <p>Don’t start with creating a project – first <a href="https://github.com/Microsoft/HoloToolkit-Unity">find the HoloToolkit here on GitHub</a>, and clone it at some place on your disk. Then follow carefully the first two steps of <a href="https://github.com/Microsoft/HoloToolkit-Unity/blob/master/GettingStarted.md" target="_blank">procedure as described here</a>. </p> <p><em>Then</em> you create the Unity project. </p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb.png" width="644" height="372"></a></p> <h2>Initializing the project</h2> <p>Proceed with following the third step of the <a href="https://github.com/Microsoft/HoloToolkit-Unity/blob/master/GettingStarted.md" target="_blank">getting started document</a>: import the package. This will give you two folders in your Assets folder plus some stuff in the root – and an extra menu-item, called “HoloToolkit” </p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_3.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_3.png" width="304" align="left" height="172"></a></p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_4.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: right; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_4.png" width="304" align="right" height="108"></a></p> <p style="clear: both">I usually delete everything under HoloToolkit-Examples. And then I hit File/Save Scene to save the current scene as “Main”. Now do the fourth step of the <a href="https://github.com/Microsoft/HoloToolkit-Unity/blob/master/GettingStarted.md" target="_blank">getting started document</a>. I usually skip the step of adding ManualCameraControl.cs to the camera. Allow Unity to reload the project when it has to, and accept it’s offer to save the Main scene when it asks to. Now your project has been set up the way the HoloToolkit people think it’s a good default. Only I don’t quite agree with all settings, so we are going to change a few ;)</p> <h2>Tweaking the camera</h2> <p>There are two major things I want to do to the camera:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_5.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; float: left; padding-top: 0px; padding-left: 0px; margin: 0px 10px 0px 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_5.png" width="354" align="left" height="382"></a></p> <p>Every manual about the HoloLens I saw so far say you should put the camera at 0,0,0 and the near clipping plane at 0.85m. To make things clear – the clipping plane is the minimum virtual distance you can get to a Hologram before it winks out. I found it to be very cool to be able to get up close to the airplanes and the airport and see the amazing level of detail HoloLens is capable of. As to the camera location – this is essentially a 3D map, I am a GIS person – and a stubborn b*st*rd as well. So I put 0,0,0 at the center of my <em>display data, </em>a logical origin – the center of Schiphol airport according to Google Maps. The camera then is stationed a little South and above it. So when the app starts, you will see Schiphol float about 2 meters before you and 25 cm down from your horizontal line of sight. You will be looking due North - which gives you a nice and compelling overview of airplanes going out over the ocean and around Amsterdam, or coming into it. </p> <p>After you have made the settings, double-click the camera in the hierarchy.</p> <h2>Directional light</h2> <p>Basically this can be whatever you like, but I opted to put it right over the airport. </p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_7.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_7.png" width="354" height="138"></a></p> <h2>The hat stand</h2> <p>Finally, add an empty game object “HologramCollection” to the main scene by right-clicking in the hierarchy, and select “Create Empty”. Make sure both position and rotation X, Y and Z are all 0 (zero) and Scale all 1:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_8.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_8.png" width="354" height="137"></a></p> <p>This will be the hat stand – the place where all other Holograms will be placed in or moved within.</p> <h2>Getting the airplanes assets</h2> <p>There is of course the awesome Unity Asset store, but there is also a great site where you can find a lot of free (and paid) 3D models. It’s called an awesome site called <a href="https://www.cgtrader.com/" target="_blank">CGTrader</a> and claims to have 520000 3D models to choose from. </p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_9.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_9.png" width="644" height="469"></a></p> <p>Models are available in a range of formats – we will need to have an airplane that is provided in OBJ or FBX format, because that is what Unity likes to have. You can just look for “aircraft free” and find a lot of airplanes Lo and behold, <a href="https://www.cgtrader.com/free-3d-models/aircraft/commercial/template-airbus-a330-200--2" target="_blank">here’s a beautiful A320</a> and it’s free, too:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_10.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_10.png" width="644" height="385"></a></p> <p>Before you use a model, make sure the license allows you to do so. When in doubt, contact the person who uploaded it – the site offers that capability. In my case, I contacted a person called <a href="https://www.cgtrader.com/2001kraft" target="_blank">Maxim Kraft</a> and he (I assume he) graciously gave me permission to use it. He has a whole range of excellent models – check him out!</p> <p>Anyway - hit “Free download” and select “3d-model.fbx.zip”. Unblock and unzip the file. Rename the file that comes out of it 3d-model.fbx – to A320.fbx. Then go to Unity, make a folder “A320” in the root and drag the A320.fbx from your explorer on top of it. Net result should be this:</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_11.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_11.png" width="354" height="148"></a></p> <p>And if you drag the A320 on top of your scene (don’t!) you will see this ginormous airplane (and this is zoomed out considerably!)</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_12.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_12.png" width="644" height="283"></a></p> <p>That clearly needs to be scaled down a little first. We will do that in the 4th blog post of this series.</p> <h2>A word about source control settings</h2> <p>I had quite some ‘fun’ getting this solution properly in source control. I am not absolutely sure this is necessary, but I think – at least for Git – it is. Hit Edit/Project Settings/Editor and select “Visible Meta Files”</p> <p><a href="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_6.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://www.schaikweb.net/dotnetbyexample/A-HoloLens-airplane-tracker-1an-Azure-da_12477/image_thumb_6.png" width="354" height="183"></a></p> <p>As to the rest – I am not a Git guru. I have included two .gitignore files, one for the FlightDataService in my previous post, and one for the Unity project. I have learned the hard way that for checking in Unity projects in Git, you make sure that everything in Assets and ProjectSettings gets checked in. As far as these directories are concerned, there is <em>no such thing as an ignored file. </em>So make sure everything under that is added to your repo. This includes dlls.</p> <p>Code so far can be found <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog2" target="_blank">here</a>. That’s a short post, eh? I did say the procedure for setting up a project was dramatically simpler now, and it is indeed!</p> <p>The code, as always, can be found online – <a href="https://github.com/LocalJoost/HoloATC_Demo/tree/Blog2" target="_blank">here on github</a></p><img src="http://feeds.feedburner.com/~r/blogspot/dotnetbyexample/~4/x004v0UaGqY" height="1" width="1" alt=""/>2016-10-15T09:57:41.323+02:002http://dotnetbyexample.blogspot.com/2016/10/a-hololens-airplane-tracker-2setting-up.html