Friday, September 15, 2017

Best Practices Of Running File System Check On Ext4 Or Xfs

Generally, running a file system check/repair command in Linux is expected to automatically repair at least some of the inconsistencies it finds. In some cases, severely damaged i-nodes or directories may be discarded if they cannot be repaired, hence, significant changes to the file system may occur which may result in data loss.

To ensure that unexpected or undesirable changes are not permanently made, perform the following precautionary steps:

Dry run (read-only mode)

-----------------------------------------Most file system checkers have a mode of operation which checks but does not repair the file system. In this mode, the checker will print any errors that it finds and actions that it would have taken, without actually modifying the file system. Using "e2fsck" on EXT4 (ext family file systems) and "xfs_repair" for XFS.

--> From man page of "e2fsck" : e2fsck -n <BlockDevice>-n Open the filesystem read-only, and assume an answer of ‘no’ to all questions. Allows e2fsck to be used non-interactively. This option may not be specified at the same time as the -p or -y options.

--> Snap shows how to run file system check in dry-run mode using "e2fsck" command:[root@localhost ~]# e2fsck -n /dev/sdc1

Operate first on a file system image, if available--------------------------------------------------------------------

Most file systems (EXT2, EXT3, EXT4, XFS etc.,) supports the creation of a metadata image (using "e2image" command or using "xfs_metadump" in case of XFS file system), a sparse copy of the file system which contains only metadata. Because file system checkers operate only on metadata, such an image can be used to perform a dry run (if available) of an actual file system repair, to evaluate what changes would actually be made. If the changes are acceptable, the repair can then be performed on the file system itself.

--> Using 'e2image' command (for ext file systems):

So, if there is a metadata dump stored which is in good condition could be used to repair corrupted file system structure as demonstrated below. Here, I'd take a backup file system metadata using "e2image" and then run file system check on the metadata dump and see how it works:

--> Taking image backup using "e2image" (need to use "r" option to take raw image backup of metadata): Need to un-mount file system or mount read-only before proceeding....

The -I option will cause e2image to install the metadata stored in the image file back to the device. It can be used to restore the filesystem metadata back to the device in emergency situations.

WARNING!!!! The -I option should only be used as a desperation measure when other alternatives have failed. If the filesystem has changed since the image file was created, data will be lost. In general, you should make a full image backup of the file system first, in case you wish to try other recovery strategies afterwards.

[root@managed1 ~]# e2image -I /dev/sdb1 /tmp/sdb1.image.out

e2image 1.42.9 (28-Dec-2013)

[root@managed1 ~]# echo $?

0

[root@managed1 ~]# e2image -I /dev/sdb1 /tmp/sdb1.image.out

e2image 1.42.9 (28-Dec-2013)

[root@managed1 ~]# mount /dev/sdb1 /data

[root@managed1 ~]# df -PTh /data

Filesystem Type Size Used Avail Use% Mounted on

/dev/sdb1 ext4 190M 41M 136M 24% /data

<<< metadata file restored and device mounted, accessed successfully. This is only applicable if block device superblock is corrupted. There is no way to recover if disk is physically damaged or layout is damaged. >>>

--> Destroy file system metadata and recover it later using image file:

[root@managed1 ~]# dd if=/dev/zero of=/dev/sdb2 bs=1024 count=2

2+0 records in

2+0 records out

2048 bytes (2.0 kB) copied, 0.0203458 s, 101 kB/s

[root@managed1 ~]# mount /dev/sdb2 /data2

mount: /dev/sdb2 is write-protected, mounting read-only

mount: unknown filesystem type '(null)'

[root@managed1 ~]# mount /dev/sdb2 /data2 -t xfs

mount: wrong fs type, bad option, bad superblock on /dev/sdb2,

missing codepage or helper program, or other error

In some cases useful info is found in syslog - try

dmesg | tail or so.

--> Now, try to restore image data using "xfs_mdrestore" from the file "/tmp/sdb2.image.out":

[root@managed1 ~]# xfs_mdrestore /tmp/sdb2.image.out /dev/sdb2

[root@managed1 ~]# mount /dev/sdb2 /data2

[root@managed1 ~]# df -PTh /data2

Filesystem Type Size Used Avail Use% Mounted on

/dev/sdb2 xfs 197M 11M 187M 6% /data2

Save a files system image for support investigations----------------------------------------------------------------------------

A pre-repair file system metadata image can often be useful for support investigations if there is a possibility that the corruption was due to a software bug. Patterns of corruption present in the pre-repair image may aid in root-cause analysis. If possible, a regular practice of taking image backup or at least metadata backup would be a lifesaver.

Operate only on unmounted file systems---------------------------------------------------------

Disk errors-----------------

File system check tools cannot repair hardware problems. A file system must be fully readable and writable if repair is to operate successfully. If a file system was corrupted due to a hardware error, the file system must first be moved to a good disk, for example with the dd utility or in case of xfs the command "xfs_copy" could be used.

PageViews

About Me.

This is just about me, my experience, my interest, my passion.... I am from Mysuru which is so popularly known as Cultural Capital of Karnataka(India). I started my carrier as a technical support person, my interest grew on Linux, started working on Linux operating system, currently working as Infrastructure Lead in a private company. Although I got enough experience in this Linux world I still feel there is some empty space which needs to be filled up every day, so, every day there is a new learning, that is what we say, learning is a never ending path. We are all here to share/learn/spread the knowledge. There is much more to learn/gain in this open-source world. Happy Learning.....