Saturday, May 14, 2011

py360 - A speed increase (mostly)

I've updated py360 (from the patch note):

Changed the Partition class from preprocessing the entire partition during its constructor, instead it now will resolve files and directories on demand and store the results for later. Basically trading precomputation for memoization. gamertags.py runs about 90x faster, report360.py runs about the same (since it touches every file) and mounting is about 100x faster. These improvements are at the cost of all first time reads being slightly slower but no wasted preprocessing is done.

What isn't mentioned is that this will also make it much more likely that corrupted partitions will mount (which was why I started looking at this change at all). Also, gamertags.py and report360.py have been changed to be compatible with the new changes. The main difference is that using partition.allfiles does not necessarily return all files but rather all the processed files, use partition.walk() to get all files. The old behaviour is still available by passing in precache=True to the Partition constructor.

This is a fairly experimental patch so let me know if it doesn't work for you. Next on the dev list remains better output from report360.py (and STFS / XDBF).