SANS Digital Forensics and Incident Response Blog

As some of you may remember, I've previously written about a technique for mounting EXT3 file system images with the read-only option, even when power was abruptly removed from the system- as is typical during forensic seizure- and the file system is still "dirty". In these cases, my technique involves using an alternate superblock, which will not have the "needs journal recovery" flag set, and using the "-t ext2" option to ignore any entries in the EXT3 file system's journal.

In the last year, however, I've been starting to see cases where I've had to analyze the newer EXT4 file system (hence my recent series of articles on EXT4 internals). It turns out that the technique I developed for mounting EXT3 file systems does not work with EXT4:

So, unfortunately, our trick of using "-t ext2" to get the file system drivers to ignore the journal is not going to work for us here, because the EXT2 drivers don't recognize many of the new file system options in EXT4.

So what can we do to mount our EXT4 file systems? When in doubt, refer to the man page:

Note that, depending on the filesystem type, state and kernel behavior, the system may still write to the device. For example, Ext3 or ext4 will replay its journal if the filesystem is dirty. To prevent this kind of write access, you may want to mount ext3 or ext4 filesystem with "ro,noload" mount options or set the block device to read-only mode, see command blockdev(8).[...]noload Do not load the ext3 filesystem's journal on mounting.[...]

You can see the "needs journal recovery" flag in the output of the file command, so our file system is definitely dirty. But happily the "noload" option does indeed allow us to mount the EXT4 file system.

But what about EXT3? The manual page suggests that "noload" will work there as well. Unfortunately, this doesn't appear to be correct:

It appears that the "noload" option does in fact cause the journal not to be loaded. But the EXT3 drivers apparently regard this as an error condition and refuse to do the mount. And before you ask, adding "-t ext2" to the command line above doesn't work either.

So at this point I was stuck with having one method for mounting EXT4 and a different, painful method for mounting EXT3. But then I got an email from Gebhard Zocher, which pointed out a clever solution:

Since the EXT4 drivers are backwards compatible with EXT3 file systems, you can just specify "-t ext4" and then use "noload" to mount your EXT3 file systems without mucking around with alternate superblocks. And that means we now have a consistent solution for mounting both EXT3 and EXT4 file systems. Thanks Gebhard!

Abhishek Datta

Hal Pomeranz

Abhishek, that's an odd error message. It makes it sound like your Ubuntu system doesn't have support for EXT4. But I know Ubuntu 10.04 supports EXT4 because I'm typing this response from my Ubuntu 10.04 system, booted off of an EXT4 file system. Have you recompiled your kernel or are you running some sort of special distro?

Anonymous Coward

I just realized that the root user does not always respect file permissions, so even if a image is set to be read-only, you can still destroy it by running "echo > image.dd".Are you confident that mounting an image with the -o=ro option will not alter the image file in any way?Using "losetup -r /dev/loop0 image.dd" provides an extra layer of write protection. I haven't yet checked how this may impact performance.Speaking of loopdevices, they are usefull to get access to a disk image through VirtualBox: Set up the loop device like described above, and see http://www.virtualbox.org/manual/ch09.html#rawdisk. Again, I don't know how this may impact performance, but I suspect it will be better than by going through shared folders.

Hal Pomeranz

If you're really concerned about accidental damage to your image, consider "chattr +i image.dd" to set the immutable flag on the file (can be removed later with "chattr -i image.dd"). You can also "set -o noclobber" in your .bashrc, which would also prevent the sort of output redirection error that you mention.But in general, yes, I am confident that "-o ro" works as advertised. Do you have evidence to the contrary?

Anonymous Coward

Thanks for taking the time to anwser.The only evidence I have to the contrary is something I read a while back, but it seems to no longer be an issue (http://www.forensicswiki.org/wiki/Forensic_Linux_Live_CD_issues#Example). Using the flags you recommended in this article with the SIFT Workstation 2.12 works great, and the checksums are the same before and after mounting.Thanks for the tip on the immutable flag. It seems like a pain to work with, but then I guess that's kind of the point.It would be interesting to see a discussion on whethe using mount -o ro, losetup -r or chattr i is a viable alternative to using a hardware write blocker. Hardware write blockers can theoretically fail (either pysically or because of user error), they are expensive, and they are impractical when working with e.g. laptops where the disk is not always easily accessible. The problems I see with software blocking methods in linux are user error (like getting the flags wrong) or implementation errors like those described in the forensicswiki article I referred to.

Ikiris

Thanks for posting this! I have been trying to setup a process for recovering VMDKs from a linux snapshot and finally got down to the filesystem recovery which used your simple tricks to get past the journaling issue.

Frank Thynne

I needed to repair a system installed as alternative boot to Windows. The system refused to boot into Ubuntu. This blog was very helpful, but I could not see how to repair the filesystem. These steps worked for me:# Recovering an ext4 filesystem requiring journal recovery.# Ubuntu was hosted in Windows and grub loader could not load the root file system# Repair steps# Boot from an XUbuntu CD# Start terminal and become root# Create mount points for host system and broken root filesystem3 cd /4 mkdir /root.disk5 mkdir /windows# Mount host filesystem6 mount /dev/sda3 /windows# Check presence of hosted filesystems8 cd /windows/ubuntu9 cd disks# Many aborted steps removed# Mount damaged filesystem read-only and ignore journal.34 mount -o loop,ro,noexec,noload /windows/ubuntu/disks/root.disk /root.disk# Backup everything to host as found39 cd /root.disk40 tar -cvzf /windows/root.disk.tgz .41 tar -tvzf /windows/root.disk.tgz |less# Repair filesystem. I think this uses what it can of the journal.62 fsck.ext4 /windows/ubuntu/disks/root.disk# Two errors were found and fixed.# Success! Could now restart normally!

Hal Pomeranz

Just a quick follow-up after talking with Joey. Turns out he was using an older Linux distro with an early version of the EXT4 drivers that didn't properly support the "noload" option. Works fine on recent releases.