What the hell...

Today first person shooter games like Counter Strike, Quake and Unreal Tournament are played online mostly. Each game ships with a built-in server-browser to show active servers on the net. A lot of people would like to have such a server-browser or similar tools as standalone applications. Maybe they want to look for more than one game at the same time, maybe standalone tools have additional features (like enhanced filtering) or are lightweight applications that do their specific job much faster than in the in-game-browsers, whatever. With Unreal Tournament 2003 being available soon, I thought it would be a good idea to share my code. Provided with this backend-class, you are able to develop your own server-browser, server-watching-tool or whatever you like.

Introduction

There are several sources of information on how to get the latest status of an Unreal Tournament server (see the Links-section below). (I personally play none of the other games, so I don't know about their query-protocols, but i believe, they are similar to the Unreal-ones presented here.) When speaking about Unreal Tournament 2003, there are two protocols available. The GameSpy-protocol that already worked with Unreal Tournament (the old version of the game) and a brand-new protocol, that was first seen with the Unreal Tournament 2003 Demo available in mid-September 2002 (when this article was written).

The 'old' GameSpy-protocol is well documented by Epic (the company that does all the Unreal-series games). On Epic's Unreal Technology Site there is a page on server-communication that gives us a lot of information. In short: send '\status\' via UDP to the server, but add 1 to the official gameport (e.g: server-address = 62.93.201.43:8980 - send query to 62.93.201.43:8981). What you get back is a bunch of packets, each in the following format: "\property\value\property\value\...\queryid\X.Y".

Each query is assigned a unique ID to - all the packets you receive from one query should have the same X-value. The second number Y is the packet-number (1-indexed). This is essential, since UDP is a connectionless protocol, thus packages may take different routes through the internet and may arrive out of order. Additionally from time to time packets get lost. Checking the packet-numbers you can asure, that you received all packets in the correct ordering and that none is missing. The last package ends with the string '\final\'.

To give you an idea, of how a typical answer looks like, here is a very simple one:

Currently there's no player active on the server and we received all the information in one single packet.

Querying an 'old' UT-server, you would use Ip:gameport+1 with that GameSpy-protocol (as written in that official document I mentioned above). Now in times of UT2003, that protocol was moved over to gameport+10. On gameport+1 there's this new, let's call it Unreal-protocol, which is byte-oriented.

There's no official documentation on this protocol, but since it is far more robust (no backslash-bug, see below) and faster to parse, it will be used a lot in the future. See the Links-section for pages of people that looked at the packets (sent by the original built-in UT2003 server-browser) and have written down, what they found out.

Update: After observing the two protocols for a few weeks now, there are certain important things to mention. See the Protocol Discussion Section below.

You guess what the two 'in-depth' parts of this article (and the main problems writing the code for our purpose) are: connect to the server and parse the answer. But before getting into detail, let's have a look at how to use this class:

Usage

First thing to do is to add the provided C#-file UT2003Server.cs into your project.

Next you construct your server-object and call at least once the Refresh()method. This method connects to the server, parses the answer and assigns the current values to all the server-objects properties. Well, and then you just use them for your displaying pleasure.

Pretty simple. I would strongly recommend to put the Refresh() into a try-block and check for GameserverQueryExceptions and handle these cases accordingly in your application. UDP really isn't that reliable and from time to time the server doesn't respond or responds too slow or the user just isn't online. These exceptions do really occur.

Refinement I: Using server.Timeout you can adjust the maximum number of milliseconds, the query-class waits for a response before raising a ServerNotReachableException (which is aGameserverQueryException) when executing a refresh.

Refinement II: Using server.Protocol you can decide what query-protocol is used, the next time you call Refresh(). What protocol you use is up to your needs. The GameSpy-protocol delivers less information on the game settings but current player-statistics. The Unreal-protocol gives outdated player-information but more general settings. I've introduced a third protocol, that is a mixture of the two. First it uses the GameSpy-protocol to get the current player-statistics and after that a part of the Unreal-protocol enlarges the information on the game-settings.

Attention: Be aware that some properties of the server-object may equal to null after refreshing. Some servers just don't deliver all possible information. Additionally future patches of the game may change or add certain properties. Hopefully I have the time to update this article each time something changes dramatically.

Update: In version 2.1 there was a new property introduced to the class, named ServerVars. This is a collection of all the server variables available, making the usage of the class far nicer and easier. The 108 lines of code for displaying all these values in the example-application dropped down to 7 lines.

The new static function Prop2En translates a property-name given by the server into a human readable English phrase (e.g. "gamestats"→"Stats logging" or "maptitle"→"Map Title").

There are a few things left to mention about ServerVars. As, for ease of use (i.e. without casting), all the items in the collection need to be of the same type (which is string), the properties, that are not strings by nature are converted internally. So you don't get Port, NumPlayers, MaxPlayers etc back as ints, but as strings, when using server.ServerVars["numplayers"] instead of server.NumPlayers. But as both are still available, you can choose the way you like it. The same with mutators: server.Mutators returns a string array containing all mutators, server.ServerVars["mutator"] one string containing all mutators as a comma-separated list. Another advantage is the fact, that the ServerVars-collection is cleared, when you assign a new IP to the server-class.

The rule is: When you want to display all server-properties in a list or similar, loop through ServerVars. When you need access to a certain server-variable use the corresponding property.

Querying the Masterserver

There is no real querying of the masterserver, at least if you don't know the authentication-algorithm, that UT2003 uses to connect to the masterserver. EPIC doesn't want everyone to query the server via TCP (as in good old UT times), but rather updates a textfile every 20 seconds (or something in that dimension) which is free for download. There's one file for the UT2003 demo-version and another for the retail:

Protocol Discussion

How to ask?

The UT2003Server.QueryProtocol enumeration contains three entries. And it's up to you to choose, which protocol is used when querying a server. Some thoughts may help you to decide, what fits your needs best. But there's nothing, you could damage, so just try, if you're unsure.

None of the two protocols is better in each and every case. It seems, that the output of the new Unreal-protocol is cached on the server and far from being up-to-date. If you query a server every 2 seconds, you see that the output of the GameSpy-protocol changes everytime something happened (especially the ping and frags of the players) while the Unreal-protocol still delivers things from the past. On the other hand, the GameSpy-protocol gives you less information on the game and the server in general. Thus getting information about mutators, friendly fire, gamespeed, etc., you have to use the new Unreal-protocol. Summary: only using both protocols gives you complete and up-to-date information. That's why I introduced the Mixed-Protocol, that does the job for you.

The following table outlines the properties, the protocols deliver exclusively (i.e. the other one doesn't):

GameSpy

Unreal

current player statistics

current number of players

name of the level (not the file name), i.e.: Temple of Anubis instead of BR-Anubis

game name

game version

min client version

server mode

game speed

game stats

mutators

goal score

time limit

translocator

friendly fire

min players

Where to ask?

Some Servers don't have their GameSpy-Queryport on Gameport+10, since this is freely configurable by the server-admin. When I wrote this, there were 1196 servers online, 951 (79.5%) of them using Gameport+10 (default), 118 (9.8%) using Gameport+12, 67 (5.6%) using Gameport+1 and 60 (5.0%) using other Offsets (even negative ones, i.e. Gameport > Queryport). That means: there's no 100% reliable way to get the Queryport out of the Gameport, but with +10 or +12 you are right with a probability of 90%. Using Gameport+1 will work with half of the remaining 10%, but I have no idea, where these servers put the Unreal-protocol Queryport (since Gameport+1 is the default value here). But it turns out, that the entire story isn't that frustrating, because EPICs masterserver just tells you the GameSpy-Queryport - so if you get your servers from the masterserver, everything is alright. If you want to observe a certain server given by IP and Gameport, you have to ask the administrator for the GameSpy-Queryport or try yourself. The message is: it is not always Gameport+10!

If you just want to use the class and don't care much about details and the problems I had to face, you can stop reading here and do a little C# instead.

Connect to the Server

Well, now lets dive into the Details. First some preparatory work: The UdpClient-class in the System.Net.Sockets-namespace offers a blockingReceive() method, that doesn't return until a UDP-packet arrives at your computer. What I wanted is something that does return null or raises an exception, if a timer expires, such that you don't have to wait infinitely if something goes wrong at the network communication level. Daniel Turini sent me with this post here on the codeproject forums in the direction to try

but that just didn't work - would have been too nice. Notice the Client-property is protected, you have to build your own child-class to be able to access it. And that's what I did in the end to solve the problem: I created a new TimedUdpClient class which I present shortly here:

It makes a new Receive()-function available, that starts a "watchdog"-thread before calling the original blocking Receive()-function from its base class UdpClient. The watchdog just waits the specified amount of time and then just sends a single byte on the same socket the client is listening to. Unfortunately this byte is not received normally (so that one could distinguish between packets sent from outside and packets sent by myself), instead an exception is raised somewhere inside UdpClient::Receive() and the function returns. Hey, actually that's the only thing we wanted. Maybe there are nicer ways, maybe not. But it works.

Having that and the base functionality of UdpClient due to inheritance, we can start querying the server.

The first version of this article had a lot of code here, which was kind of straightforward and only lengthened the page needlessly. I will give you only the core parts now, if you're interested you will look at the code anyway

Sending the query via UDP needs the query itself to be in a byte-array, which is easily constructed with the help of the Encoding-namespace. As you know from the previous discussions gameSpyQueryOffset will be equal to 10 in most cases. The AnalyseGameSpyAnswer-function is described later on in the parsing-part of this article.

Update: I found one server on the masterlist, that returned nonsense, when asked with '\status\' on the correct port. Since I want that class to work with every server on the masterlist, I updated the piece of code above with an additional timeout-condition, when there is something received, but no '\final\' could be found within the given timeout. The ominous server sent an increasing stream of zero-bytes...

The Unreal-Query is devided into three parts (see the documentation for details), where all three queries are sent and evaluated, when the Unreal-Protocol is used, and only the second query is used, when working with the mixed protocol (since basic and player information is already there from the GameSpy-Answer in that case).

This is the code for the second query, which receives the server properties packet. The two other ones are analog. Afterwards the AnalyseUnrealAnswer-function is called, which extracts the correct information from the byte-arrays.

This answer comes in two packets (29.1 and 29.2) in correct ordering and 29.2 is definitely the last packet, because it ends with '\final\'. There are 11 players currently on the server - the demo-version had a bug, that caused the server to send a weird team-value. The class takes care of that and simply sets the team-value to an empty string, when querying a UT2003-Demo server.

There's another problem in that answer, that is well known to GameSpy-Style-Query-Programmers. The 'very cool' Player named 'Kee\/n' confuses our nice little answer.Split(newchar[] {'\\'});-parser, because he uses the splitter-character (the backslash) in his name. If you don't take care of that, you will get his name has 'Kee' and after that a property named '/n' with the value 'frags_4' - which is nonsense. *grmpf*

But hey, you're a programmer and programmers are there to solve problems. Well, at least I solved the problem with the following hack: if the next property immediately after 'player_x' is not 'frags_x' or 'queryid' a playername with a backslash is detected and handled appropriately. In fact a backslash ('\') and the nonsense-property-name ('/n') (which is the second part of the players name) are appended to the name and the nonsense-value ('frags_4') becomes the new property-name. Put that into a while loop (to handle even cooler players using more than one backslash in their name) and everything goes fine.

Since there's nothing technically new and it should be an easy read, just go ahead and look at the code, if you're interested in the details.

UT2003-protocol

When you stick to the specification, parsing the byte-arrays is pretty straightforward. I needed a function that extracts a string from the byte-array, another reading an int and the rest is ++pos.

Notice, that this protocol may change with each patch version released for UT2003. Currently it is working with the UT2003 Demo and Retail version! Maybe some patched servers deliver additional information, maybe something is altered - please contact me, if you think you've found a server, that sends things that are not taken care of by my class.

Other interesting Problems

What about the player-properties? I included a very basic Player-class and wanted the user from outside to use a simple Player-array. The problem with arrays is, that you have to know how big they are, when you create them. The other possibility, an ArrayList, is more dynamic by nature, but it only handles objects. So I made a tradeoff and went the middle way: When parsing the answer, the players are put together in a growing ArrayList and at the very end it is kind of converted into an array, which is fast accessible and doesn't require the user to cast anything. The same principle is used for the returned string-array containing the mutators and the list of servers returned by the masterserver query function.

Btw: I won't rely on the 'numplayers'-property to be there all the time. Using that (and assuming it always is delivered before the first player-property) might be a solution to know the size of the player-array before the first player occurs.

A little request: if you're using that class when debugging your application or for private use, I would be glad if you set RaiseExceptionOnUnhandledProperty to true. This raises an exception, when the class receives a property that is not known so far. When this member is false (default) unknown properties are just ignored. I've tested the class a lot, but you never know. If you've found an exotic server delivering unknown properties, please contact me and I will extend the class with the new property.

Version 2.0 - 10/08/2002

new properties handled by the Unreal-Protocol: gamestats, translocator, password, gamespeed

mutators are returned as a string-array

new properties handled by the GameSpy-Protocol: password, teams

query-port offsets configurable

mixed protocol query

masterserver-query

Version 1.1 - 09/25/2002

fixed a little bug in the demo-application, nothing about the query-class

Version 1.0 - 09/25/2002

things seem to work

Unreal-protocol is in but tested too little

Aftermath

Sorry for my bad English. I'm trying to improve it, but here and there, you see I'm natively German-speaking.

Prospective: When future patches are released, I will make sure, the class runs with all versions of UT2003. This is my first article. Feel free to make suggestions, both on the code and on the article itself. Email: ruepel@gmx.li

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Because I was not able to find out how to access the SetSocketOption method of the Socket class for an object of the UdpClient class.

Also the nice idea of sending an empty packet to interrupt the blocking UdpClient receive method didn't work for me because my application needs to listen to a certain Endpoint, and therefore a datagram send by myself won't be noticed by the receive method.

In the end I am using the base Socket class now which makes things a bit less easy but of course everything offered by the UdpClient class is also included in the Socket class somehow.

//Uses the protected Active property belonging to the UdpClient base class to determine if a connection is established.
if (this.Active){
//Calls the protected Client property belonging to the UdpClient base class.
Socket s = this.Client;
//Uses the Socket returned by Client to set an option that is not available using UdpClient.
s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Broadcast, 1);
}

Hi, I ported your code to work with a fav game and it works lol. However i noticed it only does one server at a time...could you possible do something with udp sockets so multiple connections can be made?

I'm gonna attempt to port how this source does it http://www.csharphelp.com/archives/files/archive130/sampleTcpUdpClient1.cs

O yea when i tried to do multiple servers, my dialup was showing errors...albet cause i was using the same port for all requests,..gonna experiment if a differant port is used if it will work that way.

here's my prograss so far, i redid your code as a socket and seems alot shorter...(left out other protocol stuff cause only intended to work for one game...also this game responds if you send it a "\"...no need for a @"\status\"...so whatever lol. This is still not asynch...still tryin to figure that out.

using System;
using System.Net;
using System.Windows.Forms;
using System.Net.Sockets;
using System.Text;

Great Work, really !!
I used this Class on an ASP.Net-Page, and it's awsome good !
Now, how much work would it be to use this class with other Games ?
Wouldn't it just be an extension of interpreting the other Game-Protocols, as i knw it from PHP-Solutions ?
Does anyone know where to find the Protocols for other Games, ex. Battlefield1942 ?

In that case, i would love to extend this class with other games and put it on this page too ...

more protocols:
i don't know any page, where protocols of different games are explained, but you can head over to qstat.org get the source code and figure it out yourself. but there are also some quake-protocol links on the protocols-page.

Rodrigo Reyes took the sources presented here, modified them and wrote a really nice server browser application. the tool is meant to be extensible to query other games. (kind of a plugin structure, ut2003 is just the first plugin)

Well, I download the serverlist from the UT2003-Master. Now I randomly pick a server, e.g.

208.224.174.9677777787

This is one random server from the list.

7777 is the gameport, and 7787 is the queryport. So I send the packet to port 7787, but I also tried 7778, since you mentioned it must be gameport+1. But in both cases I only get a timeout, whereas this method works if I send "\status\".

But I have one more question:
in the debugger my packet, e.g. ut2003_ruleinfo[], looks like this:

read the article! i've explained the ports there. the one you get from the master is the gamespy-port. so there and only there '\status\' works. to use the other queries you need to query at gameport+1. and only there.

the debug-output looks strange. ut2003_ruleinfo[3] should be zero and ut2003_ruleinfo[4] what you've specified. looks that your memory got screwed up somehow...

I'm trying to modify this slightly to search through the servers for specific player names - and at the moment I am going through each of the servers found and grabbing it's information using the Refresh() function.

Alas, when there is nearly 2000 servers to go through, you can quite imagine how slow it currently is.

Anyway, within UT2003 itself, you can list buddies quite quickly so I'm sure it can be done.

the buddy-list is completely done directly on the masterserver who automatically gets all that data from all gameservers. you can only query this list from the masterserver if you manage to send the right authentication-string.

i have re-engineered the rest of the protocol (which wasn't too hard with ethereal). but that's all useless without the authentication-algorithm.

the problem is: there's no unauthorized tcp-communication with the masterserver. that we are able to receive a list of the current gameservers is only possible, because epic generously provides us with a tiny little text-file containing all ips and ports. that's not the way your ut2003 gets the list. it does authenticated direct tcp and special ut2003-protocol things. and that's why it is so fast.

Well. I'm all over and around the world. No, been reading here for quite a few years now. But this article I found on your HP checking which sites to keep hosted. Never been doing anything on C since C# was out.

Much better. This version is running much cleaner on my Notebook here at work. The previous version would crash if I tried to look at other servers. This one runs great! Now, give us a listing of servers, sortable, and move the picture to the side rather than the bottom.