Archive

Yesterday I did something that I didn’t think I was capable of. It involved quaternions, matrices and a decent amount of math. Let’s start with the problem definition:

We have a system that tracks rotations using quaternions

We would like people to be able to use Euler coordinates (Tait-Bryan angles if you’re picky)

Our world is Y-UP

Doesn’t sound too hard. In fact, half of it was fairly easy. Turning a set of Euler coordinates into a quaternion is as simple as creating three quaternions using angle axis pairs derived from the incoming Euler rotation. For us, the three Quaternions were as follows:

[Angle: Yaw, Axis: [0, 1, 0]]

[Angle: Pitch, Axis: [0, 0, 1]]

[Angle: Roll, Axis: [1, 0, 0]]

We create a combined rotation quaternion by conjugating the three quaternions together in the following order:

Roll * (Pitch * (Yaw) * Pitch-1) * Roll-1

This is all that is required to produce a quaternion from a set of Euler coordinates. Now, we also had to retrieve Euler coordinates from a given quaternion. This is more difficult than taking Euler coordinates and converting them into a quaternion as the task requires a fairly solid understanding of the math. This too would be simple, if it weren’t for the fact that the entire Internet agrees that Z-Up is the way to go for Euler/Quaternion conversions. From what I could tell, the equations for converting a rotation quaternion into a Y-Up Euler coordinate do not exist! (On the internet).

To my shock, horror and dismay, this meant that I had to derive the equations myself. Needless to say, this was a daunting task. But, after an arm and a leg… and the other arm… and the other leg… and parts of my face/body and an assortment of pieces from whichever organs remain… I was victorious. My glorious victory over the maths was quite euphoric.

First, let’s talk about why Y-Up Euler coordinates are so different from Z-Up Euler coordinates. The idea behind Euler coordinates is that given three angular values, you apply three rotational transformations in a predetermined order around three dynamic or constant axes (depending on convention) and arrive at a destination rotation.

Rotational transformations are not commutative, that means that applying them in a different order will produce a different rotation all-together. I won’t get into why that is but it has to do with the fact that the space of all 3D rotations is something akin to the surface of a 4 dimensional hypersphere and applying a rotation is like traveling from one point on the surface of that sphere to another surface point while always remaining on the surface. (Feel like going for a dip in some advanced math? Read here).

The effect is that a rotation quaternion composed from rotations around axes in the order ZYX is fundamentally different from one composed from rotations in the order YZX, which is what we want. So, the equations must be derived. I won’t delve very much into why the following works as that could take several blogs. First, we must build a rotation matrix that will compose our Euler coordinates in the order that we want. This is as simple as getting the angle-axis matrix formula and plugging some numbers into it. That formula is the following:

Where c is cos(θ), s is sin(θ) and C is 1-c, θ is the angle of rotation and (x,y,z) are the vector components of the axis. This is taken from here . By plugging in axis and angle, we construct three rotation matrices:

Yaw, about [0, 1, 0], angle sign Ψ (Psi):

Pitch, about [0, 0, 1], angle sign Θ (Theta):

Roll, about [1,0,0], angle sign Φ (Phi):

Now, we multiply the matrices together in the order (Roll)(Pitch)(Yaw), producing the following:

Next step is to refer to the Quaternion -> Rotation matrix formula:

This formula was taken from here. The quaternion components are [w, x, y, z] -> [0, 1, 2, 3]

Finally, to retrieve the Euler coordinates we must match the matrix form of the rotation quaternion to our RPY rotation matrix exploiting cells where the values are easy to retrieve. In this case, the values are as follows (Notation, Matrix Row Column where row and column are in the range [1,3]):

This is almost enough. What is left is to test for pitch singularities (+90, -90) and treat them specially. Since this blog has gone on for far too long, I will stop now as what has been given is certainly enough to get a similar system working. In my next blog, I will detail how we could cheaply test for the singularities and what we could do in that case. You can do some advance reading on that here.

Here we are at the bottom end of OSD600. This course has been fun and has taught me a lot. This is why it is all the more unfortunate that my 0.4 release is not nearly as atomic as my 0.3. For my 0.4, I had 2 projects to work on alongside another course. One project was migrating XSS Me from the current tab-based threadpool implementation to Web Workers. This didn’t go so well. The conclusion of my research into Web Workers deemed them unusable for XSS Me’s requirements. So instead, a very minor fixup was in order.

As for Paladin, as usual, quite a lot of work went into the configurator. The configurator is now closer than ever to landing but still has not landed. This iteration, quite a number of changes went into the configurator. I’ve itemized the changes below:

Store/load now accept named parameters

Configurator objects were receiving undefined guids, now test suite fails if that happens

Whew! It’s been a while but my 0.3 has now been released! In this release, I’ve been working on Paladin/configurator subsystem. By now, it has become quite mature and achieves all required functionality. Also being used is IndexedDB as the load/store backend. Also important is that the codebase has now been brought up-to-date with the rest of the engine.

Changelog:

Added comments to all of the Configurator tests that didn’t have any. Improved some of the comments that were there before as well as improving some test structures slightly.

Removed Cookie.js from the engine. Cookies are no longer used as load/store backend, now replaced with IndexedDB.

Many functions had vague local variable names; their names have been improved.

load/store are now both asynchronous.

Added config/default.js for local configurations that need to be baked into the engine.

As you can see, I forced the reported values for the cursor’s position to be 17000/18000 for the screenX/screenY members and 7000/8000 for the clientX/clientY members. This is a pretty naive way to do it but it proved to produce results. Check it out!

This is what happens now when a user attempts to right click anywhere in the page. As you can see, the context menu appears far off into the right bottom corner ( since the coordinates go off screen ).

Interestingly enough, the changes to the event class do not affect the native windows right click context menu ( can be triggered by right clicking in the same vertical space as the minimize/maximize/close buttons ), as can be seen here. Also not affected is text selection and clicking buttons/links and forms.

Also interesting is that no assertion errors are usually thrown. I have managed to provoke an assertion error however simply by middle clicking ( click the wheel of your mouse if you have a mouse wheel ) somewhere inside the page; the page being used should have sufficient text for the quick drag middle click popup to appear, for an example this page should work. Scroll around a bit if you don’t first succeed.

In conclusion, it seems that it is not immediately obvious how MouseEvent is used internally. On the one hand, the values reported through the event to the js engine are the only way for the js to figure out where the cursor is; so as far as the js is concerned, whatever we return here will be where the js believes the cursor to be. On the other hand, some internals seem to use the values returned from the getter functions of the mouse event object as well; this may make updating the values to our whims possibly difficult due to the sheer size of firefox’s internals and the fact that we don’t *exactly* know what will be affected by our meddling.

P.S. The MouseEvent class also defines some longs (screenX, screenY, clientX, clientY); from what I could tell, these are only updated during initialization:

I am now calling the 0.2 version of the configuration registry released! The configuration registry is now almost completely useable. In this revision, I’ve implemented loading/storing to local browser storage ( ie a cookie ); as well, clearing and JSON serialization were implemented in this revision. The new features are backed up by rigorous ( more or less 🙂 ) unit tests.

The only thing left to implement is loading using xhr. Due to the lack of a good way to unit test xhr, I’ve been hesitant to start writing this feature in. Version 0.3 of this bug will include xhr loading.

Since only xhr loading is left, I plan to start working on other bugs/other projects for the 0.3 release.

I look forward to my code being reviewed. If you have the time, come over to the pull request thread and drop a word or two 🙂

I just finished tearing apart my git repo and putting it back together. The reason for this undertaking was that a number of lf/crlf EOL errors had crept into my commits somehow. The culprit, as I found out yesterday, was git config’s core.autocrlf configuration option. Initially, it was set to true, meaning that files checked out from a repo are translated from lf to crlf and files committed back in are translated from crlf back to lf. Problem is, I had some files that needed to be checked out as lf in another repository. Foolishly, I set core.autocrlf to false globally. This set in motion a set of events that led to some of my files in the paladin repo being checked in with all the line endings set to crlf. As you may guess, this absolutely destroys any chance of a meaningful review diff being produced. Clearly, this was my fault and I had to own upto it.

This is when I set out to export my commits to patches, clean up the line endings in the patches, remove any nit-picky changes ( ie a change hunk where only trailing whitespace was removed ), shove the changes back into my repo, updated the old branch pointer to look at my shiny new commits, forced github to forget my old history and finally my repo was sanitized.

What I’ve learned from this is to not let anyone or any thing handle things like line endings for me in the future as I may blow it up inadvertently while trying to do something unrelated. core.autocrlf will stay permanently set to false. The solution? My favourite text editor is now set to produce lf line endings by default; let the chips fall as they may.

It’s Wednesday, October 26, 2011. I’ve been on something of a hiatus for a week or two as I’ve been mostly preoccupied with EJB605. It’s time to get back to work on OSD and not just because 0.2 is due tomorrow. Luckily, my particular project involves a steady linear progression towards completion at some point. I imagine that 0.2 will be the last release that I do for the Configurator Registry.

With that in mind, let me take a minute to point out how awesome it has been to work on Paladin. Coming from a background with a lot of interest in game development as well as a sizeable amount of work invested in other game engines over the past 2 or 3 semesters, I found Paladin to be very nicely written. The organization and enthusiasm of its volunteers and project leads is quite refreshing. This is why I will be going back to Paladin to help them out whenever I can in the near future. However, for my 0.3 release, I thought that I would like to change course a little bit.

XSS Me is an explicitly open-source FF add-on that tests sites for XSS vulnerabilities. It is a tool used and loved by many web devs around the world to test their sites for XSS flaws. For the uninitiated, an XSS flaw ( Cross-Site Scripting, shortened to XSS ) is a type of security flaw that allows an attacker to execute arbitrary Javascript on a website by submitting specially crafted user-data to that site. This can be something like a Facebook post or a user blog on a blogging site. This is very dangerous on websites like Facebook that thrive on user-submitted data. The results of a successful XSS attack can be devastating considering that the browser of each user viewing the attacker-submitted data will be executing whatever Javascript the attacker desires.

XSS Me partially mitigates this threat by doing the attacker’s work for you, thereby giving a webdev some indication as to whether their website is vulnerable to XSS attacks. I had the chance to work on XSS Me when I worked for Security Compass ( also see SecCom Labs ) as a part of Seneca’s coop program. What makes XSS Me interesting is that it does its work fast, really fast. XSS Me will send many requests to a target website, encoding an XSS attack vector into each request; it will then analyze each response, scanning for evidence that the attack vector had caused the site to execute specific Javascript encoded in each attack vector. Since Javascript is single threaded, one would think that this process is quite slow once you factor in the hundreds of attack vectors that have to be sent to the site.

To achieve its speed however, XSS Me gets clever. Since XSS Me is an FF Add-on and not a web-page Javascript, it gets more privileges as it executes in the browser’s context. XSS Me leverages this power by opening up multiple tabs and executing Javascript code in each one. In this way, XSS Me fakes real threading into Javascript.

This mostly works. However, some nasty bugs have emerged from this construct that are difficult or impossible to diagnose; one heisen-bug in particular caused me much grief as I tried to squash it, failing time and time again despite numerous attempts. With the advent of HTML5 and web workers, I saw a chance to update the core threading component of XSS Me to utilize web workers. I find this to be an exciting opportunity because I like concurrency J

None of this is for certain as I still have to discuss this plan with Humph; should it pan out however, I think that it would be an exciting way to enter the world of HTML5 threading. As well, if anyone else is interested in working on XSS Me, there’s plenty of work to go around.