FlightGear WorldTerrain SkyBox Server

This article describes content/features that may not yet be available in the latest stable version of FlightGear (Release 2016.3).You may need to install some extra components, use the latest development (Git) version or even rebuild FlightGear from source, possibly from a custom topic branch using special build settings: .This feature is scheduled for FlightGear (unknown).

Objective

The purpose of this project is to make FlightGear's extensive library of world data, weather & environment logic, and map editing tools, available for the realtime use of other applications over a socket connection, either locally or over a network.

The initial driving application for this ability is a first person game engine. When this project was initiated in 2013, the Unity engine was chosen for the flagship demo, but we have since switched back to T3D in order to maintain complete source access and a totally open license (and in recognition of the impressive work being done by the Torque3D steering committee and community).

At present, only two types of data are transferred from FlightGear to the client game engine: skybox images and terrain data. A custom camera group has been created (skybox-cameras.xml) which creates a set of five images capable of being mapped to a skybox in the client (there is no bottom image at this time). Meanwhile, as the player moves around the world, the terrainPager class in T3D sends requests for terrain data, which are answered in FlightGear by the creation of binary float array files for heights and integer array files for texture reference. These are named with the protocol "hght.(coordinates).bin" and "text.(coordinates).bin" respectively.

The application is still under heavy development, but the current version uses a custom networking class called "dataSource", which is responsible for opening and maintaining the socket connection, and sending a basic packet of information, and "worldDataSource" which inherits dataSource and handles all the real work.

This structure is up for debate and it is very likely that it will be discarded in the future in favor of a more general networking base class, probably from SimGear.

(Apologies, but the rest of this page is a group of somewhat random quotes on this subject, mostly from myself, helpfully assembled by Hooray who also created this page. I will attempt to make this a more organized presentation of the project at some point in the future. In the meantime, check out the blog at [1].)

My ultimate goal here is a real-world-terrain-based MMO game using Unity for the FPS engine and FlightGear as a backend Skybox (video games) server.

I would also like to make this skybox/terrain mechanism available to others, however, for many different types of projects.

Part of my plan here is to go off on a fork of terragear et al, at whatever level is necessary to change the interpretation of the GIS data, eg to substitute something more blasted and smoking where the urban areas are in the current flightgear world. Once I get that handled I plan to use GIS shapefiles as a format for creative game-related mapping work, so we're not limited to just the existing real world data, but can paint towns and roads wherever we want.

Long story short, it's useful because I want to create a first person MMO game based in real life terrain, and I want to leverage flightgear's very long view distances, awesome weather and day/night simulation, GIS-based terrain texturing, tree shaders, etc. to give the player the sense of truly being immersed in a planetary scale game environment. And, ultimately, to let them play the game right in their own home town, when I get all the parts put together.

I'm really aiming larger than just making a single game here, though. Ultimately I would like to make this into an open source simulation tool / online metaverse explorer that could be used for many different purposes.

It's also in no way locked to Unity, that's just the environment I'm focusing on right now, due to its ease of cross platform export, among other things. I have already written a terrain pager in Torque, and could just as easily pipe these skybox images there, which would result in a 100% open source solution... but only for windows or possibly Mac at the moment, no linux.

Status

my skybox is working pretty well now, and I have the FG textures also imported into my Unity terrain, so if I can find a quick way to determine which texture goes where in flightgear when I load a new terrain, I will be a big step forward in the process of replicating my FG experience on the Unity terrain.

I'm in the process of retrofitting the above described prototype so as to allow it to save out an array of finished Unity terrains and skyboxes, using the data provided once by flightgear during development, and then page through this precompiled data at runtime. This way the end user never has to have flightgear or terragear installed, they just run the Unity game, and only the developer has to deal with the flightgear side of things.

I'm still working on my "FlightGear World Server", and the current mission is to replace my (achingly slow) skybox generator. My previous method was to rotate the camera via PropertyTree changes, and save screenshots at each cardinal direction. I finally got through the literature on CameraGroups and Cameras, however, and managed to hook up a set of five cameras which simultaneously record all five desired directions, with the correct field of view.

Gallery

Approach

Screen shot showing the camera group setup used for creating a single view that can be processed as a screen shot and is turned into textures for all skybox faces.

The skyboxes are currently handled on the FlightGear side by the use of a custom camera group, located in a file called skybox-cameras.xml. I haven't worked out the rest of my data pruning issues and hence have not uploaded anything to the fgdata side of my FlightGear repository for this project, but the skybox file is necessary to run the project. Here is the skybox-cameras.xml file:

<!--************************************************************************Definitions for four cameras, tracking three cardinal directions ESW (main camera handles North) and then one for straight up.************************************************************************--><PropertyList><camera-group><window><name>FlightGear World Server</name><host-nametype="string"/><display>0</display><screen>0</screen><width>4000</width><height>800</height><decorationtype="bool">false</decoration></window><camera><window><name>FlightGear World Server</name></window><viewport><x>0</x><y>0</y><width>800</width><height>800</height></viewport><view><heading-degtype="double">90.0</heading-deg></view><frustum><top>480</top><bottom>-480</bottom><left>-480</left><right>480</right><near>480</near><far>40000</far><fixed-near-far>true</fixed-near-far></frustum></camera><camera><window><nametype="string">FlightGear World Server</name></window><viewport><x>800</x><y>0</y><width>800</width><height>800</height></viewport><view><heading-degtype="double">0</heading-deg></view><frustum><top>480</top><bottom>-480</bottom><left>-480</left><right>480</right><near>480</near><far>40000</far><fixed-near-far>true</fixed-near-far></frustum></camera><camera><window><name>FlightGear World Server</name></window><viewport><x>1600</x><y>0</y><width>800</width><height>800</height></viewport><view><heading-degtype="double">-90.0</heading-deg></view><frustum><top>480</top><bottom>-480</bottom><left>-480</left><right>480</right><near>480</near><far>40000</far><fixed-near-far>true</fixed-near-far></frustum></camera><camera><window><name>FlightGear World Server</name></window><viewport><x>2400</x><y>0</y><width>800</width><height>800</height></viewport><view><heading-degtype="double">-180.0</heading-deg></view><frustum><top>480</top><bottom>-480</bottom><left>-480</left><right>480</right><near>480</near><far>40000</far><fixed-near-far>true</fixed-near-far></frustum></camera><camera><window><name>FlightGear World Server</name></window><viewport><x>3200</x><y>0</y><width>800</width><height>800</height></viewport><view><heading-degtype="double">0.0</heading-deg><pitch-degtype="double">90.0</pitch-deg></view><frustum><top>480</top><bottom>-480</bottom><left>-480</left><right>480</right><near>480</near><far>40000</far><fixed-near-far>true</fixed-near-far></frustum></camera><gui><window><nametype="string">FlightGear World Server</name></window></gui></camera-group></PropertyList>

Also, here is my preferences file, where the skybox-cameras file is included:

what I'm primarily doing is using flightgear to draw skyboxes for the Unity (or other FPS style) engine.

All I get out of flightgear is five textures, which draw to the front, back, sides and top of the skybox. The local terrain in Unity I'm actually handling on my own, at much higher resolution than flightgear's terrain (10m instead of 90m(?) for flightgear), although I would like to figure out how to also export the terrain height data from flightgear for the rest of the world beyond my current 50km area of high res data.

I would also like to be able to lift the terrain texture for a given area out of flightgear and inform Unity of it, but I'm taking it one step at a time...

the way I have it designed at the moment is all over a TCP/IP socket, so we should be pretty much free and clear - the flightgear end stays GPL, and the other end doesn't have to really care, it just sends a world position and gets textures back through the socket.

There may be some overlap when it comes to objects - I may end up converting some of the .ac files in flightgear into FBX or collada, so I can load them into Unity, but more often I think I'll be going the other way and adding my FBX game models into flightgear, ala this thread.

As it is now, though, I can run and run across 2500 square km of area, and wherever I go I can refresh the skybox to see flightgear's view of my position. Only a very limited region of terrain has to be actually instantiated in Unity at any given time.

This is all done using Unity C# scripts, opening up sockets to communicate with A) a tiny little terrain server I wrote in C++, which accepts a starting latitude/longitude and a width/height of the area, and returns height data from a giant ten-meter-resolution height data file I made from DEM maps. In Unity I have a 3x3 set of tiny terrains (640m each) which page across the landscape, loading new data and moving as necessary to stay in front of the player. And then, B) a modified flightgear instance, which takes a socket connection from which it reads latitude, longitude, and elevation, and returns five screenshots taken at 90 degree angles to each other, with a 90 degree field of view.

The result was actually better than I expected, although the lighting comes out pretty sharply different between the different camera angles. There is no hiding the fact that I'm painting to a skybox - but for a first pass, it's actually pretty entertaining anyway.

Implementation

Development

My suggestion would be to introduce 2-3 well-defined layers/components here:

the screen shot/image handling stuff

the IPC/RPC (networking) layer

the skybox retrieval method (which could also be IPC based at some point).

Networking

Your current implementation looks like a telnet-like approach to me - so, you could just as well modify the existing built-in telnet protocol/daemon to add support for new "skybox" related commands there - I've added corresponding pointers to the article on doing just that. I did something pretty much like that a few years ago, and it was really dead-simple.

I am not saying that this is necessarily the way to keep this working - but for the sake of prototyping, I would definitely focus on extending the built-in telnet daemon/server first.

We already have an RPC mechanism built-on top of so called "fgcommands" - those can also be easily invoked via telnet.
Which basically means that telnet can be used for arbitrary IPC, and fgcommands can be used for creating/extending functionality.
Equally, we already have a httpd-based jpeg-server that serves screen shots using native OSG machinery.

In other words, your current approach could be easily simplified and generalized, so that it would also become more efficient/faster.

I would probably continue prototyping, but on top of the built-in telnet protocol.
That will greatly simplify your code - once you need something more powerful, e.g. for also serving binary data (think image blobs), I would consider adding a custom I/O protocol. I can tell you exactly how to do that and also provide stubs to get you going quickly, including all the cmake magic to integrate things properly. But I would prefer using telnet for starters, to move away from your current hard-coded I/O approach.

There's cross-platform/multi-platform socket handling code in simgear for all the socket stuff you are doing there e.g. see SGSocket Internally, I/O protocols and network stuff usually live in $FG_SRC/Network
Which is also where you can see, that there's the concept of an "FGIOChannel" for encapsulating all the logic you added to fg_init.cxx
fg_init.cxx will only set up the IO channel, which just inherits from the corresponding interface class: http://api-docs.freeflightsim.org/simgear/classSGIOChannel.html

For examples, look at the httpd or telnet/props stuff

You will find that using these SimGear 2-3 APIs will greatly simplify and reduce your code, while making it much more straightforward to move it into a separate module, that also compiles across all platforms.

Equally, there's a ton of SGGeo* helpers available for doing all lat/lon computations that can dream of

Demo using Python

Using the Python code available at [2], the following snippet of code can be used to display a FlightGear-based SkyBox using Python:

Issues

it's still using the chunky 90m terrain, and my height sampling gets glitchy here and there (hence the needle problem down by the corner of the skybox). It's also slower than cold molasses when it comes to making the skyboxes and doing the terrain sampling, but I have a long list of planned optimizations and improvements going forward from here. Even as is, however, it should still be adequate for a player moving at normal human speed.

Roadmap

There are several flightgear related tasks remaining to figure out. (Warning, gross understatement.) It would work much better if I could somehow turn off rendering for everything within a certain radius of the camera, so I would not accidentally fill half my sky with one tree. :-P The best solution would be to somehow cut out the exact square that I'm rendering locally in Unity, but I'm quite willing to accept cheap hacks and half measures at this point.

I also need to figure out a quick & easy way to raycast for a terrain height at any given place in the flightgear world. And figure out enough of terragear to know how the terrain textures are placed. And on and on and on... but I thought some folks here might get a laugh out of this little project, at least.

Rendering is done for a near camera and a far camera anyway, so I think all you need to do is not to render the near camera, and that's it... (I think this was specified somewhere in camera_group.cxx ).