It occurred to me last night that a big list of passwords could be abstracted out into their equivalent masks, and then a frequency count of those masks could be generated, which could then be exhausted in frequency order.

First, I extracted a frequency count of character set combinations (masks) from all eight-characters-longthe RockYou breach's password list, yielding a list of the form:

Then it occurred to me that if someone else had published this info, and had used real corpora of passwords as the input, then our frequency lists would probably look similar. So I did the following Google search:

Dangit! :-) But at least I'm catching up to the state of the art; the KoreLogic article was published in April 2014. :-)

I got the idea from work I had done on some license-plate-collecting stuff I do on the side. I thought of it for capturing high-level patterns in serials, so that people can search for a plate based on the serial. A plate with "BDT 606" on it would match any plate whose serial "mask" is "AAA 999" using my notation. (I then match more closely, but it's used for a high-level search first).

I haven't watched the KoreLogic presentation yet, but I can definitely improve upon my own approach, because I'm being overly aggressive in turning then entire set of non-alphanumeric-but-printable characters into 's':

| tr "[\ !\"#$%&\'()*+,-./:;<=>?@\[\\\]^_\`{|}~]" 's' \

... when most folks use the simple ones (#$%@, etc.) I could create a custom charset for this using the notation as noted here ... and then turn the remaining characters into another custom charset that is the remaining characters.

I then found PACK - the Password Analysis and Cracking Kit, which is is a set of Python scripts to manage masks, including optimizing a set of masks based on a given timeframe (or, "I have 24 hours. Which masks should I use to maximize how many passwords I can crack?")

I'll be using this post to store information about LSI HBA firmware, with a focus on FreeBSD (but also drawing upon Linux information). It may also be useful for users of FreeNAS, PC-BSD, unRAID, Nexenta, or ZFSguru.

Why - SATA port density on a budget

If you are using ZFS, you do not need RAID -- you just need lots of fast SATA ports. To maximize the features of ZFS, it needs to directly access attached drives in JBOD mode rather than as RAID. If you can afford them, you can buy the LSI 9211-8i HBA card. Alternatively, you can also buy a less expensive card, and then replace its stock "IR" (Initiator-RAID) firmware by "crossflashing" to an "IT" (Initiator-Target) version of LSI's general firmware for 9211-8i hardware. This option is useful for people building home NAS systems on a budget. Popular cards include the Dell PERC H200 and the IBM ServeRAID M1015. This ServeTheHome post introduces the topic well.

Here is the relevant dmesg for a Dell PERC H200 Internal (H200I) under FreeBSD 8.4-RELEASE. (Note that this particular card's LSI firmware (Phase 9) is out of sync with the FreeBSD driver (Phase 14), which may have unexpected side effects. The system was initially built as a FreeBSD 8.1-RELEASE system in 2010.)

General flashing tips

Before flashing, and especially before erasing any flash, use the sas2flsh.exe -listall option to note the SAS ID of your device (usually beginning with "0x590"). If you accidentally erase the entire flash (sas2flsh.exe -o -e 6 will retain your SAS ID, but sas2flsh.exe -o -e 7 will wipe it), you will not be able to re-flash the device unless you have this ID. Write it down.

Some earlier versions of sas2flsh.exe allow cards to be flashed from IR firmware to IT firmware; others do not. I and others have had luck with the one that comes with LSI's Phase 7 (AKA P7 or P07) firmware. (Try this link, or search LSI.com for "9211_8i_Package_For_P7_Firmware_BIOS_Upgrade_on_MSDOS_and_Windows" to download the package that contains this version of sas2flsh.exe.

To flash the firmware on cards installed in non-UEFI motherboards, you can create a DOS-bootable USB key using a tool like Rufus. Rufus will make the device bootable with FreeDOS or MS-DOS (well, actually, Windows ME!). I and others have had better luck using the MS-DOS option. (According to that thread, LSI themselves recommend MS-DOS rather than FreeDOS).

Also note that when flashing using sas2flsh.exe there are two different components to be flashed: the firmware (contained in a filename sometimes ending with .fw, and usually named after the device in some way) and the BIOS (usually named something like MPTSAS2.ROM). The firmware component is what your OS driver communicates with. The BIOS component allows you to configure the firmware at boot time, and can enumerate the list of attached hard drives. For ZFS and JBOD purposes, the BIOS is not strictly necessary, and has even been reported to cause problems when present. Erasing the firmware areas sas2flsh.exe -o -e 6 and then just applying the firmware without the BIOS will also result in faster boot times.

FreeBSD flashing considerations

At this writing (2015-01), there have been reports of Phase 20 not playing well with FreeNAS and FreeBSD. Downgrading to Phase 16 (FreeBSD 9.3 and 10.0) or Phase 19 (FreeBSD 10.1) is reported to be more stable.

Under FreeBSD, PC-BSD, and FreeNAS, the desired end state is for the "Firmware" and "Driver" ports of the dmesg line to use identical firmware versions. For FreeBSD 10.1-RELEASE, this is the Phase 19 version. In the dmesg output, the Firmware item is what's on the card, and the Driver item is what the OS supplies.

(I also list all of the firmware/OS pairings I know of towards the end of this post.)

Beware when upgrading a FreeBSD-based OS. Depending on the combination of firmware and driver, your drives may disappear from the OS' view until you reflash. This can be especially troublesome if your root filesystem is ZFS.

How to reflash the Dell Internal Tape Adapter 15MCV card as a 9211-8i

There is a card from Dell that looks almost identical to the H200I card, but is actually a Dell Internal Tape Adapter board (Dell part number 15MCV). This is identified in various levels of firmware and utilities as "Int Tape Adapter" or "IntTapeAdptr", and identified under Linux as: