On php 5.2.2 with a copy of a live site (with many other modules), for example, I see a speed improvement from 684ms to 663ms per page load of the user login form 'user', and that's with all of the files totally loaded into the Linux file buffers. Your results will likely differ.

The module is distributed as a patch against pathauto 5.x-1.2. You can download it below.