Blog Entries

This entry covers the historical context of Space War!, and instructions for working with our in-browser emulator. The system doesn’t require installed plugins (although a more powerful machine
and recent browser version is suggested).

The JSMESS emulator (a conversion of the larger MESS project) also contains a real-time portrayal of the lights and switches of a Digital PDP-1, as well as
links to documentation and manuals for this $800,000 (2014 dollars) minicomputer.

Summary

The Modern SDK contains some URI related functionality as do libraries available in particular projection languages. Unfortunately, collectively these APIs do not cover all scenarios in all
languages. Specifically, JavaScript and C++ have no URI building APIs, and C++ additionally has no percent-encoding/decoding APIs.

The Windows.Foudnation.Uri type is not projected into .NET modern applications. Instead those applications use System.Uri and the
platform ensures that it is correctly converted back and forth between Windows.Foundation.Uri as appropriate. Accordingly the column marked WinRT above is applicable
to JS and C++ modern applications but not .NET modern applications. The only entries above applicable to .NET are the .NET Only column and the WwwFormUrlDecoder in the bottom left which is available to .NET.

Scenarios

Parse

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS, and by System.Uri in .NET.

Parsing a URI pulls it apart into its basic components without decoding or otherwise modifying the contents.

WsDecodeUrl (C++)

WsDecodeUrl is not suitable for general purpose URI parsing. Use Windows.Foundation.Uri
instead.

Build (C#)

URI building is only available in C# via System.UriBuilder.

URI building is the inverse of URI parsing: URI building allows the developer to specify the value of basic components of a URI and the API assembles them into a URI.

To work around the lack of a URI building API developers will likely concatenate strings to form their URIs. This can lead to injection bugs if
they don’t validate or encode their input properly, but if based on trusted or known input is unlikely to have issues.

WsEncodeUrl(C++)

WsEncodeUrl, in addition to building a URI from components also does some encoding. It encodes non-US-ASCII characters
as UTF8, the percent, and a subset of gen-delims based on the URI property: all :/?#[]@ are percent-encoded except :/@ in the path and
:/?@ in query and fragment.

Accordingly, WsEncodeUrl is not suitable for general purpose URI building. It is acceptable to use in the following
cases:

- You’re building a URI out of non-encoded URI properties and don’t care about the difference between encoded and decoded characters. For instance
you’re the only one consuming the URI and you uniformly decode URI properties when consuming – for instance using WsDecodeUrl to consume the URI.

- You’re building a URI with URI properties that don’t contain any of the characters that WsEncodeUrl encodes.

Normalize

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET. Normalization is applied during construction of the Uri object.

This is modulo Win8 812823 in which the Windows.Foundation.Uri.AbsoluteUri property returns a normalized IRI not a normalized URI. This bug does not affect System.Uri.AbsoluteUri which returns a normalized URI.

Equality

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET.

URI equality determines if two URIs are equal or not necessarily equal.

varuri1 =newWindows.Foundation.Uri("HTTP://EXAMPLE.COM/p%61th foo/"),

uri2 =newWindows.Foundation.Uri("http://example.com/path%20foo/");

console.log(uri1.equals(uri2));// true

Relative resolution

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET

Relative resolution is a function that given an absolute URI A and a relative URI B, produces a new absolute URI C. C is the combination of A and B
in which the basic components specified in B override or combine with those in A under rules specified in RFC 3986.

Encode data for including in URI property

This functionality is available in JavaScript via encodeURIComponent and in C# via System.Uri.EscapeDataString. Although the two methods
mentioned above will suffice for this purpose, they do not perform exactly the same operation.

Additionally we now have Windows.Foundation.Uri.EscapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to
Windows.Foundation.Uri). This is also slightly different from the previously mentioned mechanisms but works best for
this purpose.

Encoding data for inclusion in a URI property is necessary when constructing a URI from data. In all the above cases the developer is dealing with
a URI or substrings of a URI and so the strings are all encoded as appropriate. For instance, in the parsing example the path contains “path%20segment1” and not “path segment1”. To construct a URI one must first construct the basic components of the URI which involves encoding the data.
For example, if one wanted to include “path segment / example” in the path of a URI, one must percent-encode the ‘ ‘ since it is not allowed in a URI, as well as the
‘/’ since although it is allowed, it is a delimiter and won’t be interpreted as data unless encoded.

If a developer does not have this API provided they can write it themselves. Percent-encoding methods appear simple to write, but the difficult
part is getting the set of characters to encode correct, as well as handling non-US-ASCII characters.

WsEncodeUrl(C++)

In addition to building a URI from components, WsEncodeUrl also percent-encodes some characters. However the API is not recommend for this scenario given the particular set of characters that are encoded and the convoluted nature in which a developer
would have to use this API in order to use it for this purpose.

There are no general purpose scenarios for which the characters WsEncodeUrl encodes make sense: encode the %, encode a subset of gen-delims but not also encode the sub-delims. For instance this could not replace encodeURIComponent in a C++ version of the following code snippet since if ‘value’ contained ‘&’ or ‘=’ (both sub-delims) they wouldn’t be
encoded and would be confused for delimiters in the name value pairs in the query:

Since WsEncodeUrl produces a string URI, to obtain the property they want to encode they’d need to parse the resulting URI.WsDecodeUrl won’t work because it decodes the property but Windows.Foundation.Uri doesn’t
decode. Accordingly the developer could run their string through WsEncodeUrl then Windows.Foundation.Uri to extract the property.

Decode data extracted from URI property

This functionality is available in JavaScript via decodeURIComponent and in C# via System.Uri.UnescapeDataString. Although the two
methods mentioned above will suffice for this purpose, they do not perform exactly the same operation.

Additionally we now also have Windows.Foundation.Uri.UnescapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to
Windows.Foundation.Uri). This is also slightly different from the previously mentioned mechanisms but works best for
this purpose.

Decoding is necessary when extracting data from a parsed URI property. For example, if a URI query contains a series of name and value pairs
delimited by ‘=’ between names and values, and by ‘&’ between pairs, one must first parse the query into name and value entries and then decode the values. It is necessary to make this an extra step separate from parsing the URI property so that sub-delimiters (in this case ‘&’ and ‘=’) that are encoded will
be interpreted as data, and those that are decoded will be interpreted as delimiters.

If a developer does not have this API provided they can write it themselves. Percent-decoding methods appear simple to write, but have some tricky
parts including correctly handling non-US-ASCII, and remembering not to decode .

In the following example, note that if unescapeComponent were called first, the encoded ‘&’ and ‘=’ would be decoded and interfere with the parsing of the name
value pairs in the query.

WsDecodeUrl (C++)

Since WsDecodeUrl decodes all percent-encoded octets it could be used for general purpose percent-decoding but it takes a URI so would require the dev to construct a stub URI around the string they want to decode. For example they could prefix “http:///#” to their string, run
it through WsDecodeUrl and then extract the fragment property. It is convoluted but will work correctly.

Parse Query

The query of a URI is often encoded as application/x-www-form-urlencoded which is percent-encoded name value pairs delimited by ‘&’ between pairs and ‘=’ between corresponding names and values.

In WinRT we have a class to parse this form of encoding using Windows.Foundation.WwwFormUrlDecoder. The queryParsed property on
the Windows.Foundation.Uri class is of this type and created with the query of its Uri:

The QueryParsed property is only on Windows.Foundation.Uri and not System.Uri and accordingly is not
available in .NET. However the Windows.Foundation.WwwFormUrlDecoder class is available in C# and can be used manually:

Build Query

To build a query of name value pairs encoded as application/x-www-form-urlencoded there is no WinRT API to do this directly. Instead a developer must do this manually making use of the code
described in “Encode data for including in URI property”.

In terms of public releases, this property is only in the RC and later builds.

“A Slower Speed of Light is a first-person game in which players navigate a 3D space while picking up orbs that reduce the speed of light in increments. A custom-built, open-source relativistic
graphics engine allows the speed of light in the game to approach the player’s own maximum walking speed. Visual effects of special relativity gradually become apparent to the player, increasing
the challenge of gameplay. These effects, rendered in realtime to vertex accuracy, include the Doppler effect; the searchlight effect; time dilation; Lorentz transformation; and the runtime
effect.

Why do we use a paper size that is so unfriendly for the basic task of reading? According toa very interesting post by Paul Stanley, the rough dimensions of office paper evolved to
accommodate handwriting and typewriters with monospaced fonts, both of which rendered many fewer characters per line. “Typewriters,” he explains, “produced 10 or 12 characters per inch: so on
(say) 8.5 inch wide paper, with 1 inch margins, you had 6.5 inches of type, giving … around 65 to 78 characters.” This, he says, is “pretty close to ideal.”

Issue

There's no input validation on the namespace parameter and it is injected into the SQL query with no encoding applied. This means you can use the '%' character as the namespace which is the
wildcard character matching all secrets.

Notes

Code review red flag was using strings to query the database. Additional levels made this harder to exploit by using an API with objects to construct a query rather than strings and by running a
query that only returned a single row, only ran a single command, and didn't just dump out the results of the query to the caller.

Issue

There's little input validation on username before it is used to constrcut a SQL query. There's no encoding applied when constructing the SQL query string which is used to, given a username,
produce the hashed password and the associated salt. Accordingly one can make username a part of a SQL query command which ensures the original select returns nothing and provide a new SELECT via
a UNION that returns some literal values for the hash and salt. For instance the following in blue is the query template and the red is the username injected SQL code:

In the above I've supplied my own salt and hash such that my salt (word) plus my password (pass) hashed produce the hash I provided above. Accordingly, by providing the above long and
interesting looking username and password as 'pass' I can login as any user.

Notes

Code review red flag is again using strings to query the database. Although this level was made more difficult by using an API that returns only a single row and by using the execute method which
only runs one command. I was forced to (as a SQL noob) learn the syntax of SELECT in order to figure out UNION and how to return my own literal
values.

When they went to the Moon, they received the same per diem compensation as they would have for being away from base in Bakersfield: eight dollars a day, before various deductions (like for
accommodation, because the government was providing the bed in the spaceship).

Fixed in Windows 8 is intra-line tab completion - you can try it out on the Windows 8 Consumer Preview now. If you open
a command prompt, type a command, then move your cursor back into a token in the middle of the command and tab complete, the tab completion works on that whitespace delimited token and doesn't
erase all text following the cursor. Like it does in pre Windows 8. And annoys the hell out of me. Yay!

“On The Verge is ready for a lot of things, but we clearly weren’t ready for renowned astrophysicist Dr. Neil deGrasse Tyson, who stopped by to talk space exploration, life as a meme, and
why he carries a slightly-illegal laser with him at all times.”

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Worse than the lame blog comments hating on percent-encoding is the shipping code which can do actual damage. In one very large project I won't name, I've fixed code that decodes all
percent-encoded octets in a URI in order to get rid of pesky percents before calling ShellExecute. An unnamed developer with similar intent but clearly much craftier did the same thing in a loop
until the string's length stopped changing. As it turns out percent-encoding serves a purpose and can't just be removed arbitrarily.

Percent-encoding exists so that one can represent data in a URI that would otherwise not be allowed or would be interpretted as a delimiter instead of data. For example, the space character
(U+0020) is not allowed in a URI and so must be percent-encoded in order to appear in a URI:

http://example.com/the%20path/

http://example.com/the path/

In the above the first is a valid URI while the second is not valid since a space appears directly in the URI. Depending on the context and the code through which the wannabe URI is run one
may get unexpected failure.

For an additional example, the question mark delimits the path from the query. If one wanted the question mark to appear as part of the path rather than delimit the path from the query, it must
be percent-encoded:

http://example.com/foo%3Fbar

http://example.com/foo?bar

In the second, the question mark appears plainly and so delimits the path "/foo" from the query "bar". And in the first, the querstion mark is percent-encoded and so
the path is "/foo%3Fbar".