How to Recover Partitions from LVM Volume

Yesterday I had a problem with a disk… while trying to increase the size of a LVM volume, I lost the disk. What I did was: add the new disk to the LVM volume, mirror the volume and removing the original disk. But the problem is that I added the disk back again and the things started to go wrong. The volume did not boot, etc.

The original system was a Scientific Linux 6.7 and it had different partitions: one /boot ext-4 partition and a LVM volume in which we had 2 partitions: lv-root and lv-swap.

At the end of the LVM problem I had the original volume with LVM bad metadata that did not allow me to use the original information. Luckily I had not written any other information in the disks… so the information had to be there.

Once the disaster was there…

I learned how to recover the partitions from a LVM volume.

I had to recover the partitions, to create a new disk with the partitions.

Recovering partitions

After some failed tests, I got to the situation in which I had /dev/sda with a single GPT partition in there. I remembered about TestDisk, which is a tool that helps in forensics, and I started to play with it.

1

The first that I did was to try to figure out what could I do with my partition. So I started my system with a ubuntu 14.04 desktop LiveCD and /dev/sda, downloaded TestDisk and tried

$ ./testdisk_static /dev/sda

Then I used the GPT option and analysed the disk. In the disk I found two partitions: my boot partition and a LVM partition. I wrote the partition table and got to the initial page where I entered in the advanced options to dump the partition (Image Creation).

2

Then I had the boot partition dumped and it could be used as a raw partition dumped with dd.

Then I exited from the app and started it back again, because the LVM partition was now accesible in /dev/sda2. Now I tried

$ ./testdisk_static /dev/sda2

Now I selected the disk and selected the Intel partition option. TestDisk found the two partitions: the Linux and the Linux Swap.

3.png

And now I dumped the Linux partition.

Disclaimer

This is my specific organization for the disk, but the key is that TestDisk helped me to figure out where the partitions were and to get their raw images.

Creating the new disk

Now that I have the partition images: image.dd (for the boot partition) and sda1.dd, and now I have to create a new disk. So I booted the ubuntu desktop again, with a new disk (/dev/sdb).

The first thing is to get the size of the partitions and we will use the fdisk utility on the dumped files:

$ fdisk -l image.dd
$ fdisk -l sda1.dd

Using these commands I will get the size in sectors of the disk. And using that size I can make the partitions in /dev/sdb. According to my case, I will create a partition for the boot and other for the filesystem. My options were the next (please pay attention to the images to check that the values of the sectors match between the size of the partitions and the size of the NEW partitions and so on).

$ fdisk /dev/sdb
n
<enter>
<enter>
<enter>
+1024000
n
<enter>
<enter>
<enter>
+15810560
w

3.png

The key is to pay attention to the size (in sectors) of the partitions obtained with the fdisk -l command issued before. If you have any doubt, please check the images.

And now you are ready to dump the images in the disk:

$ dd if=image.dd of=/dev/sdb1
...
$ dd if=sda1.dd of=/dev/sdb2
...

Check this cool tip!

The process of dd costs a lot. If you want to see the progress, you can open other commandline and issue the command

$ killall -USR1 dd

The dd command will output the size that it has dumped in its console. If you want to see the progress, you can issue a command like this one:

$ watch -n 10 killall -USR1 dd

This will make that dd outputs the size dumped every 10 seconds.

More on this

Once I had the partition dumped, I used gparted to resize the second partition (as it was almost full). My disk was far bigger than the original, but if you only want to get the data or you have free space, this won’t probably be useful for you (so I am skipping it).

 

Advertisements

How to compact a QCOW2 or a VMDK file

When you create a Virtual Machine (VM), you usually have the option of use a format that reserves the whole size of the disk (e.g. RAW), or to use a format that grows according to the used space in the disk (e.g. QCOW or VMDK).

The problem is that the space actually used in the disk grows as the disk files are written, but it is not decreased as they are deleted. But if you writed a lot of files and you deleted after they were needed, you’d probably have a lot of space reserved for the VMDK file, while that space is not actually used. I wanted to reclaim that space, to move the VMs using less space, and so this time…

I learned how to compact a VMDK file (the same method applies to QCOW2)

The method is, in fact, very easy… you simply have to re-encode the file using the same output format. If you have your original-disk.vmdk file, you simply have to issue a command like this one:

$ qemu-img convert -O vmdk original-disk.vmdk compressed-disk.vmdk

And that will make the magic (easy, isn’t it?).

But if you want to compact it more, you can claim more space from the disk before re-enconding the disk. First, I’d go to the solution and then I’ll explain it:

If the VM is a linux-based, you can boot it and create a zero-file, and once the file has exhausted the disk, delete it:

$ dd if=/dev/zero of=/tmp/zerofile.raw...$ rm /tmp/zerofile.raw

If the VM is a Windows-based, you can get the command sdelete from Microsoft website decompress it and execute the following commandline:

c:\> sdelete -z c:

Now you can power off the VM and issue the qemu-img command. You’ll get a file that correspond to only the used space in the disk:

$ qemu-img convert -O vmdk original-disk.vmdk compressed-disk.vmdk

Explanation

(Disclaimer: Please take into account that this is a simple and conceptual explanation)

If you knew about how the disks are managed, you’d probably know that when a file is deleted, it is not actually deleted from the disk. Instead, the space that it was using is marked as “ready to be used in case that it is needed”. So if a new file is created in the disk, it is possible that it uses that physical space (or not).

That is the trick from which the file recovery applications work: trying to find those “ready to be used” sectors. And that is why the “low-level format” exists: in order to “zero” the disk and to avoid that files are recovered.

When you created the /tmp/zerofile.raw file, you started to write zeros in the disk. When the physical empty space was exhausted, the disk controller started to use the “ready to be used” sectors, and the zerofile wrote zeros on them, and the zeros were written in the VMDK file.

The good thing here is that when a VMDK file is created (from any format… in our case, it is VMDK), the qemu-img application does not write those zeros in the file that contains the disk, and that is how the storage space is reclaimed.