Tuesday, May 06, 2008

About resizing and recovering filesystems

A colleague of mine did something horrible last night. He resized an LVM volume along with the file system on it without making a backup first. Murphy's law kicked in and we ended up with a file system that was 99% full, but with no files on it.

This morning, after enjoying the luxury of some five hours sleep, it dawned on me that the problem might well be the journal of said ext3 filesystem. Knowing that an ext3 filesystem is an ext2 file system with extra goodies, I tried mounting it as ext2:


# mount -t ext2 -o ro /dev/vg/volume /mnt/recover
mount: /dev/vg/volume already mounted or /mnt/recover busy


First line of defense: turn that error into a google search. This eventually tells me that the problem must lie with device mapper holding on to the device. After a bit of deliberation, I remove the mapped device:


dmsetup remove /dev/mapper/vg-volume


Then I do an lvscan to recreate it:


# lvscan
... snip ...
inactive '/dev/vg/volume' [70.00 GB] inherit
... snip ...


And activate it:


# lvchange -a y /dev/vg/volume


And just like that, I could mount it as an ext2 filesystem and get my files back.

The moral of this story, I think, is to remove the ext3 journal before you resize things, and add it back afterwards.

0 Comments:

Post a Comment

<< Home