LightSwitch for Games Part 4: OData Access from C++ Client Code with the C++ REST SDK

NOTE: Although this series is aimed at small game developers, it is equally applicable to anyone wishing to learn how to use LightSwitch.

In Part 2 of this series we built a user account and profile database on our LightSwitch server, and in part 3 we showed how to make a web interface to allow users to edit their account details. In this part, we will look at how to enable new users to register and existing users to log in direct from your C++ game (or application) code. If you’ve ever played a console game which requires log in to EA’s Origin servers or something similar, you will be familiar with this workflow and why it is useful to have in your game; that is, it saves users from having to go to a web site to make an account before they can play.

You don’t need to have completed part 3 in order to follow along with the tutorial below, but your LightSwitch project and database need to be in a state that matches at least the end of part 2. The web interface from part 3 is a fully distinct code path from what we will do below so it is not required for this code to work.

This article assumes some familiarity with:

HTTP requests and responses, methods and headers

JSON

OData transactions (covered in part 2)

a moderate understanding of C++ (including C++11 lambda functions)

a basic understanding of threading

setting up include and library directories for a project in Visual Studio

You will learn:

How to interact with LightSwitch OData endpoints using the C++ REST SDK (codename ‘Casablanca’)

How to use PPL(X) tasks and continuations

How to use OData to create new users (write table rows) and fetch user profiles (read table rows) from our server’s database programmatically in C++

How to update and delete rows with HTTP/OData directly or from within C++ programmatically

How to make the code error-resistant (for example if the user is disconnected from the internet)

How to separate the client-side logic (interacting with the server) from the user interface

How to access the server asynchronously (that is, using multiple threads so that the rest of your game or application does not stall or block while waiting for the server to respond)

How to create a basic framework of C++ classes to make your code easily re-usable and extensible

Project Goals

We need a client-side framework for communicating with the LightSwitch server before we start adding game-specific features and plugging the code into an actual game, so for this part our goal will be to create a simple console test application which allows us to create new users and fetch their profiles.

NOTE: The code presented below makes heavy use of C++11 features. You need Visual Studio 2013 Preview or later to complete the tutorials in this article. You can re-write portions of the code without these features if you need to compile it with Visual Studio 2012.

C++ REST SDK

There are lots of ways to process HTTP requests and responses in C++, right from barebones WinInet calls (not recommended!) to Boost or POCO. In this article, we shall use Microsoft’s C++ REST SDK. This was available as a beta with the name Casablanca (version 0.6) during Visual Studio 2012, and also included the Windows Azure SDK for a while which has now been split off into a separate SDK. Visual Studio 2013 ships with version 1.0 of the C++ REST SDK and newer versions (2.0 at the time of writing) are available on CodePlex.

REST stands for Representational State Transfer and although it has many implementations, for our purposes it is essentially a simple way of issuing requests and commands to a remote server via HTTP calls. You can use the GET method to query tables, POST to insert rows or call remote functions, PUT to update rows and DELETE to delete rows. In other words, it is conveniently exactly like the interface supplied by the OData endpoints on our LightSwitch project (and in the case of remote functions, calling any WCF RIA Services we make – see parts 2 and 3 for details and examples).

While you can download and configure the SDK in your projects manually, the easiest way is to use the NuGet package manager and add a reference to the C++ REST SDK (search for ‘Casablanca’ in the Manage NuGet Packages search box to find it) in your project. This will download and install the SDK and add all the appropriate include and library paths so you can get going quickly.

NOTE: The C++ REST SDK only supports dynamic linking (you must compile with the /MDd or /MD flags – this is the default). Since you may be integrating the code with game engines or other libraries that only support static linking, I have produced an article explaining how to re-compile the C++ REST SDK to support static linking. It is not necessary for this tutorial, but note if you try to statically link the code below without re-compiling the SDK, it will crash with debug assertion errors.

Make a new, empty C++ project in Visual Studio and reference the SDK in whichever way you prefer, then we can get cracking!

PPLX Primer

PPLX is a Linux-compatible version of PPL included with the C++ REST SDK. PPL stands for Parallel Patterns Library and is an SDK which allows for some neat syntactical sugar to create multi-threaded applications. Everything in the C++ REST SDK is based on PPL tasks and asynchronous operation, and as such there is a bit of a steep learning curve for those not used to this kind of programming. You don’t need to know everything to use the SDK, but to help things along, I’ll give a brief introduction into the basic techniques required.

As it happens, a network SDK based on PPL is very useful for our purposes because we really don’t want our game to stall while it is waiting for the server to respond. Usually we take care of this by making network communication run in the background by starting additional threads so that the game’s main thread can continue uninterrupted. With PPL, the thread management is taken care of for us automatically, making things much easier.

PPL Tasks

Instead of performing a computation directly on our main thread, we can wrap it in a pplx::task<T> object (where T is the return type of the function encapsulating the task, ie. the variable type of the task’s output). The task will run automatically in another thread.

The C++ REST SDK has many functions which return tasks instead of the result directly.

For example the http_client object’s request() method returns a pplx::task<http_response> object, rather than an http_response directly. This means that when you call request() to execute an HTTP request, it returns immediately (doesn’t block) and the task automatically starts to run in another thread.

Notice that we have not actually made mention of pplx::task<http_response> in the code itself. We don’t actually generally need to deal with tasks directly unless we’re doing something special. It is assumed that the thread which receives the actual HTTP response will signal the application that the request has completed and provide the result (we’ll see how to do this below).

Task waits and results

If you want to wait for a task to finish – blocking the current thread – use wait() on a task:

Continuations

A continuation is a construct which indicates what should happen when a task is completed. You can add continuations to tasks by using the then() method to generate a new task which includes the continuation. You can chain as many then() calls as you want together.

then() takes a single argument which is the function to call when the task completes. In the above value-based continuation, the target function receives a single argument which is the result of the task. By using C++11 lambda functions as above, and chaining together continuations with then(), we can essentially write the code serially in layout even though it really executes in parallel.

NOTE FOR EXPERTS: The target function executes in the same thread as the task by default, and this can sometimes be a problem. The SDK provides a way for you to indicate that the continuation should be run on the original thread from where the task was created using concurrency::task_continuation_context::use_current(), but since this is only supported in Windows 8, we show another way to deal with this problem below.

You can also create a task-based continuation, where the target function receives a new task which wraps the result of the previous task, instead of receiving the result of the previous task directly:

The main reason to use this for us is exception handling. If the server is down or the user isn’t connected to the internet, attempting to generate an http_response will cause an http_exception exception to be thrown. In a value-based continuation there is no way to handle this, so we have to wrap all our network task generation calls from the main thread in try/catch blocks), but in a task-based continuation we can just put the try/catch block inside the continuation and keep things tidy. More on this below.

A note on strings

The C++ REST SDK uses the platform string string format for virtually everything involving strings. This basically means the default string format on your PC (ANSI, UTF-8, Unicode etc.). This can make things a bit tricky if you’re used to just using std::string, std::cout and so forth, because most machines default to a wide string format (as opposed to a ‘narrow’ 8-bit format) nowadays. Most of the C++ library string manipulation functions have wide versions with the same names as the narrow ones but with the letter ‘w’ in front, eg. std::wstring, std::wcout and so on. You can hard-code for this if you want (in which case remember to pre-pend all string literals with L to make them long/wide), or you can use some syntactical sugar:

The C++ REST SDK provides utility::string_t (and utility::stringstream_t etc.) which maps to either std::string or std::wstringdepending on your environment. You can use the _XPLATSTR() macro (or just U() as a shortcut) to convert any string literal into the platform default.

In the code below, I have just hard-coded everything to use wide strings.

Walkthrough example: Create a new user

Let’s make a bare-bones console application which creates a new user. First, the boilerplate code:

If you installed the C++ REST SDK using NuGet, the include path for the SDK’s header files will be cpprest/* as shown above, otherwise you may need to change this. The namespaces are all defined by the SDK. Replace the account details in main() with pleasing defaults! Note that CreateUser() will return a PPL task, so we call wait() on it to make sure the application doesn’t exit before the server has responded to the request.

NOTE: I have used C++11 raw string literals above. Earlier versions of Visual Studio do not support this, so you must replace them with normal string literals.

Recall that we made an account __userRegistrant with special privileges in part 2 to allow the anonymous creation of new player accounts. To log in an actual user later on, simply replace the arguments to credentials‘ constructor with the user’s username and password.

Remember that HTTP is a stateless protocol, so the correct username and password must be supplied with every request. There are no session keys. Since HTTP Basic auth involves the transmission of the password in plaintext (unencrypted), it is critical that you use an SSL-encrypted connection to the server for authenticated requests; be sure to use https:// rather than http:// in your URL paths to make sure SSL is turned on. If you are using Windows Azure to host your LightSwitch server, SSL is configured and enabled for you when you provision a new web site, otherwise you will need to do the server-side configuration yourself.

We construct the JSON request manually in requestBody (this isn’t good practice and later we’ll see how to do this properly; for one thing, if one of the user-supplied fields contains a backslash, the above code will fail to encode it properly), set the HTTP method to POST, set the endpoint to the UserProfiles table, specify that the request is in JSON (rather than XML) and that we also want the response in JSON too.

Processing the response

We now create a task for the request with a continuation to deal with the response. We start by getting the HTTP response status code and body:

If not, we try to find out why. In the case of status code 500, the LightSwitch server returns some XML with error codes and descriptive error text. The <Id> tag contains the error code enum value. This doesn’t change regardless of the server’s LightSwitch version or locale so you should prefer to inspect this tag when deducing which error has occurred. You can use an XML parser if you want, but it’s much simpler to just do a brute-force string search:

And that’s it. If you now run the example numerous times with various valid and invalid username/password combinations, try changing the request URI to one that doesn’t exist on the server and so forth, you should find that it all behaves exactly as you would expect – as long as the server is up and you’re connected to the internet.

Walkthrough example: Fetch a user’s profile

Let us now write code to fetch a user’s profile. First add the following code to the previous example:

Notice that the constructor for credentials only allows platform strings so we had to use wstring for the argument types here.

Querying a database table is much simpler than inserting one because we don’t need to set up a POST request body or supply additional HTTP headers, so we can use a simple overload of request() as follows:

We check for the condition that the authentication failed (error code 401 – Unauthorized) and for any other unexpected HTTP status code in the response. If everything is ok, we proceed to extract the relevant data from the JSON response:

Whereas before we used extract_string() to get the text of the HTTP response body, here we use extract_json() instead (returns pplx::task<json::value>), which converts the response text into a json::value object.

When you query a LightSwitch table in JSON format, what you get back is a single object containing two items: odata.metadata which you can safely ignore, and value which contains the query result. Specifically, value is an array with one element per retrieved row, and each element is an object which has one property for each field in the retrieved row. json::value has an overloaded [] operator which lets us retrieve items using standard C++ syntax, so the code responseJ[L”value”][0] returns a json::value representing the first (and in this case, only) retrieved row.

When you pull a value out of a json::value via an indexer as above, what you get is another json::value (think of it as tree traversal). To convert the leaf textual values to actual C++ strings, use as_string() as shown above. There are various as_*() functions for the different types you might want to convert to.

The final code retrieves the specified user’s profile and prints their full name and email address to the console.

Dealing with no internet connection

If there is no internet connection, the task which generates an http_response will throw an http_exception. The simplest way to deal with this is to wrap all of the relevant code (not just the task-generating code; that on its own won’t raise an exception) in a try/catch block as follows:

Error code 12007 is defined somewhere in the Windows API as ERROR_INTERNET_NAME_NOT_RESOLVED – in other words a DNS failure, which is what is likely to happen if the user’s internet connection is off or has failed. We simply check for this error code so we can print a meaningful error message, or print the error message supplied with the exception if something else went wrong.

Obviously, wrapping everything in error-handling code like this creates a lot of repetition and isn’t very readable or maintainable. A better way is to use a task-based continuation by changing code like this:

Maintenance and extensibility

What we’ve done so far works but it is far from optimal from a development point of view. Here are some of the problems:

username and password must be supplied with every request

due to the stateless nature of the protocol, there is no way to know if the user is metaphorically “logged in” or not

the interface (output or other application-specific processing of results) is mixed up with the request/response logic. We would like to separate these so we can re-use our network code in multiple games/apps.

the universal LightSwitch/OData handling code is mixed up with the code specific to the requests/functions/tables available in our game network. We would like to separate these so the LightSwitch client code can be re-used in other applications that aren’t related to our game network project.

adding new request functions means we’ll have to add new response/error checking/validation code that will be similar for many requests

we are not constructing JSON requests in a safe way (recall that in the CreateUser example we made the request by joining strings together)

iterating through many JSON objects is syntactically messy. We would like to convert returned rows to C++ structs with a property for each field.

there is no way for the main thread to know if the request was successful or an error occurred

there is no way for the main thread to know if the network code is still busy processing the request without also blocking it (using pplx::task::wait())

tracks whether the current login credentials were valid last time they were used (indicating ‘successful logon’)

maintains state in a thread-safe manner about whether the network code is busy processing a request, which can be polled by the main thread

has an inheritance hierarchy that separates LightSwitch/OData logic, game network logic and logic for our specific game

is used in each game by a separate application-specific class containing the game’s interface which will be linked to the network code via task continuations

The full source code for just such a framework can be found at the bottom of the article. I’m not going to go over it line by line but I will highlight a few features of the code we haven’t looked over yet.

GameNetwork – derives from LightSwitchClient and includes the functions/tables supported by our GameNetwork LightSwitch project

GameClient – the game’s interface and has GameNetwork as a member through which the LightSwitch server is accessed

If your game network will have various functions that aren’t game-specific as well as some that are – and this is probably going to be the case – you may wish to further derive from GameNetwork so that this class does not have to be modified with game-specific code.

Usage:

Error handling

Instead of outputting error messages directly, we store them for later retrieval and the client code can call LightSwitchClient::GetError() to check if an error occurred. All error types – no internet connection, HTTP error status codes and LightSwitch errors are funneled through this mechanism so that error-checking by the client can be done in a simple unified way.

Credentials

The desired user’s login and password can be set via LightSwitchClient::SetUser(). This is initially assumed to be a valid user and this assumption changes if a request returns a 401 Unauthorized error, or if LightSwitchClient::LogOut() is called, clearing the stored credentials. The login state can be checked via bool LightSwitchClient::LoggedIn().

Busy state

We create a type ThreadSafeBool which can be converted to the standard bool type and back via overloaded operators. The class essentially wraps a single bool in a Windows CRITICAL_SECTION such that it can be read and written by multiple threads without corruption. We then store an instance of this object in our class framework which is set to true at the start of any request and false when the request completes (with or without errors). Call LightSwitchClient::Busy() to get the busy state.

Techniques (the following code is all included in the framework; it is provided here for educational purposes if you want to roll your own):

LightSwitch error handling

You can extract the error code from a failed LightSwitch request as follows:

Some responses (mainly those as a result of a 404 Not Found error) don’t have <Id> tags with an error code, so in those cases we try to extract the error text from the <Message> tag instead. If neither are found, a default error message is returned.

Extracting JSON data

Unlike querying a row which was described earlier, inserting a row in the database will return a JSON object which contains the inserted fields, without the extra object/array wrapping. You can bypass all of this and ensure you get the data you want regardless of request type as follows:

In a nutshell, if the response JSON data is a 2-element object where one of the properties is called value and is itself an array, then it’s most likely we have just received the query results of one or more rows so we return the array directly; in all other cases we return the original response. If the JSON data is anything besides an object, it is probably corrupt data.

Encapsulating JSON data

We define a type JsonFields which is a simple mapping of keys to values using std::map as follows:

typedef std::map<wstring, wstring> JsonFields;

Unlike json::value, we can use a C++11 initializer list to populate this very easily; for example, to create a user profile JSON object we could write something like:

Note that we have to return a pplx::task, but when an error occurs there is no task to perform. Luckily we can use pplx::task_from_result(T value) to generate a task that simply returns the supplied value immediately.

Retrieve only the first row matching a query

You can add the OData directive $top=1 to a URL’s query string to fetch only the first matching row of a query, and then look at the first element of the array returned by SanitizeJSON above (the framework includes a function LightSwitchClient::QueryFirstRow() to do this for you).

Updating table rows

Although our registration and login example doesn’t require it, the framework also allows you to update rows with one or more changed fields. OData uses the HTTP PATCH method to do this. The HTTP request should be formed in the same way as for inserting rows but with one additional header:

If-Match: *

This is a requirement in LightSwitch and simply means that any matching entity (row) can be updated.

The URL should point to the row or rows to be updated. To point to a single row in a LightSwitch application, the auto-generated Id field for each table is used as the primary key. Brackets are added to the table name to select a row by its primary key:

The LightSwitchClient::Update(wstring table, int key, JsonFields fields) function in the framework will handle table row updates for you automatically.

Note that on a successful update, the server will return 204 No Content with an empty response body.

NOTE: HTTP PUT can also be used but this updates all the fields in a matching row, even if you don’t specify them in the request (in that case, they will be blanked).

Deleting table rows

Once again not called for in our example code but available in the framework, deleting rows uses the HTTP DELETE method and has the same URL and HTTP header requirements as for updating rows, but no request body needs to be specified as there is nothing to update. Deleting rows also returns 204 No Content with an empty response body from the server on success.

Warning about row update/delete security

Ensure that users can only modify table rows that they should be modifying!

While in this case, users are restricted to viewing their own profile row and will encounter a 404 Not Found error if they try to access someone else’s, there is no harm in being paranoid! In the server code following on from part 3, I added the following business logic to the UserProfiles table (C#):

Be careful though. In part 2 we allowed __userRegistrant to add users by performing a temporary privilege elevation. However, we implemented this in SaveChanges_Executing which actually runs beforeUserProfiles_CanDelete in the save pipeline, so as things stand now the delete will always be allowed. To fix this, move this line:

out of SaveChanges_Executing() and insert it at the beginning of UserProfiles_Inserting() instead.

Game Network implementation

We will now layer functions specific to our GameNetwork LightSwitch project from the rest of the series over the LightSwitchClient class.

Creating a C++ struct to represent a JSON object

Here is an example of how to create a struct that is easily convertible to and from a JSON object. The more adventurous among you may want to use type reflection to avoid having to write the ToJSON() and FromJSON() methods for every new type.

The code should be fairly self-explanatory, but note that – crucially – Id is defined last so that you can use an initializer list to create a new UserProfile without specifying an ID, since that will be automatically assigned by the LightSwitch server.

WARNING: For reasons known only to Microsoft, trying to return a UserProfile created with an initializer list directly in FromJSON() crashes the Visual Studio 2013 C++ compiler and returns an empty struct with the November 2013 CTP compiler. This is why I create it in “p” first. If you declare Id as the first item in the struct, returning directly with an initializer list works as expected on both compilers.

Game Network client implementation

We define one method in GameNetwork for each possible action we want to perform on the server. In our example, we are creating a user and fetching a user’s profile so we need two methods. We also define callbacks that will trigger when a request completes, such that the main application knows a response has been received – this solves the signalling problem described earlier.

Fetch user profile

Line 1 of the return statement fetches the first row from UserProfiles whose UserName field matches the name of the currently logged in user (it has to even without any query parameters, because in Part 2 we configured the server so that it would only return the current user’s profile row for security reasons), and fetches the row as a json::value.

Line 2 converts the json::value into a UserProfile object.

Line 3 calls the application-defined callback if one has been set.

Line 4 returns the UserProfile object to the thread which created the task.

Create new user

Line 1 sets the current user to the special user registration account __userRegistrant which we defined in part 2.

Line 2 converts the supplied new UserProfile object to a JsonFields object, inserts it into the database (which calls the UserProfiles table business logic we defined on the server to validate all the fields and update the ASP.NET Membership database at the same time, as well as assigning the new user to the Player role), and fetches the sever’s version of the new profile as a json::value.

Line 3 converts the json::value into a UserProfile object.

Line 4 sets the UserName field. This is important, because the application-defined callback may need it, but if an error occurred, the server will not return a new JSON profile object, so when the conversion takes place in line 3, the resulting UserProfile object will not have any of its fields populated. When a new user is created successfully, this line of code has no effect.

Line 5 calls the application-defined callback if one has been set.

Line 6 returns the UserProfile object to the thread which created the task.

As you can see, adding new functions to the GameNetwork implementation will be trivially easy in most cases thanks to the dirty work being done in LightSwitchClient for us.

Game interface implementation

Now we turn to the final piece of puzzle: the game, which actually calls these functions in GameNetwork and does something with the results. Because all of the client-server logic is now abstracted away, we can now plug in whatever behaviours we want and re-use all of the previous code in any game or application. So let us now re-write the previous examples to use this new framework.

In this simple example, we define one method Run() which will be the actual main application code, and two callbacks which are called when a new user is created or a profile is fetched (or an error occurs trying to do either of these things).

The full source code is available at the end of the article, but the relevant part of the Run() implementation is:

As you can see, we merely call GameNetwork::CreateUser() and GameNetwork::GetProfile() with appropriate arguments and sit back and wait until the work is done. Instead of blocking the thread with pplx::task::wait() as we did in the original examples, we now poll the GameNetwork object’s Busy() function repeatedly until it becomes false. For the sake of proving that the network code does in fact run in another thread, we print dots every 10ms until each request completes (note: you may notice when running this code that the order of output of text and dots on the console is not correct; this is because console writes are not atomic operations and therefore, not thread-safe and may be executed out of order. In a DirectX/OpenGL or Windows GUI application this will not be an issue).

std::bind is used to set the callback to a method of an object instance. The syntax:

std::bind(&MyClass::MyMethod, this, _1)

can be used anywhere in C++ where you might need a function pointer that is a pointer to a member function of the calling object.

The actual callback functions merely print out a friendly error message where possible if an error occurred, or the actual result of the server request if it completed successfully:

Note the method of checking for errors: when creating a user, UserProfile::Id will be zero if creation failed; when fetching a user profile, UserProfile::UserName will be blank if the fetch failed. GameNetwork::GetError() (which inherits from LightSwitchClient::GetError()) is used to find the relevant error code or error message. In the case of LightSwitch error codes, the callback converts them into human-readable error messages.

Wrapping up

Now the low-level stuff is out of the way, we are ready to move on to integrating the client code with a graphical interface, which will be the subject of part 5. I’m still sick so please donate to my final wishes crowdfund if you found this article useful!