complete fs corruption after fsck
 

Hi,

we are currently facing the worst of all possible scenarios on one of
our virtualized database servers (kvm, with a qcow2 image).

After trying to repair a broken database, the kernel reported an ext3
error, put the filesystem in r/o mode and suggested an fsck.

The story ends with a completely destroyed filesystem, the only folder
existing is lost+found, containing huge amounts of #12345 style files
and folders.

Now my problem is that unfortunately most of the stuff that went into
lost+found is just unuseable garbage, there is hardly any "regular" file
in lost+found and its subdirectories.

What I desperatly need is at least one of the many +400M backup files
located on the server. They happen to be so called postsgresql "custom"
dump files, identifyable by their "PGDMP" file header.

In order to find out wether such files or fragments exist in lost+found
I fgrep'ed the entire folder, w/o success.

On the other hand, if I fgrep the qcow2 image file for the string, I get
results soon.

So, are there any options left to get more data back? I've played with
lde [1], but I must admit that the magic required to understand lde is
beyond my level of magic ...

If there is a chance, we would also be willing to pay for the efforts
required.

Thanks in advance.

Udo Rader

[1] http://lde.sourceforge.net/

--
Udo Rader, CTO
http://www.bestsolution.at
http://riaschissl.blogspot.com

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

complete fs corruption after fsck
 

On Fri, Aug 06, 2010 at 11:21:47AM +0200, Udo Rader wrote:
> What I desperatly need is at least one of the many +400M backup files
> located on the server. They happen to be so called postsgresql "custom"
> dump files, identifyable by their "PGDMP" file header.

They are probably too big to be in one piece on the disk, and since the
pgdump custom format does not sound like something easily detectable as
correct (it looks like it's usually gzip compressed, which means it would
look like random data), they probably won't recover correctly.

> On the other hand, if I fgrep the qcow2 image file for the string, I get
> results soon.
>
> So, are there any options left to get more data back? I've played with
> lde [1], but I must admit that the magic required to understand lde is
> beyond my level of magic ...

You can try one of the following:

http://www.cgsecurity.org/wiki/TestDisk
http://www.itu.dk/people/jobr/magicrescue/
http://www.sleuthkit.org/
http://www.keldix.com/keld/readme-salvage.html

or some of the tools from:
https://ext4.wiki.kernel.org/index.php/Undeletion

you would want to make a copy of qcow2 image first, as some of the tools
will damage it even more than it was in the beginning, so you want to start
with fresh copy on each try.

Good luck...

--
Opinions above are GNU-copylefted.

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

complete fs corruption after fsck
 

On 08/07/2010 11:02 PM, Matija Nalis wrote:
> On Fri, Aug 06, 2010 at 11:21:47AM +0200, Udo Rader wrote:
>> What I desperatly need is at least one of the many +400M backup files
>> located on the server. They happen to be so called postsgresql "custom"
>> dump files, identifyable by their "PGDMP" file header.
>
> They are probably too big to be in one piece on the disk, and since the
> pgdump custom format does not sound like something easily detectable as
> correct (it looks like it's usually gzip compressed, which means it would
> look like random data), they probably won't recover correctly.
>
>> On the other hand, if I fgrep the qcow2 image file for the string, I get
>> results soon.
>>
>> So, are there any options left to get more data back? I've played with
>> lde [1], but I must admit that the magic required to understand lde is
>> beyond my level of magic ...
>
> You can try one of the following:
>
> http://www.cgsecurity.org/wiki/TestDisk
> http://www.itu.dk/people/jobr/magicrescue/
> http://www.sleuthkit.org/
> http://www.keldix.com/keld/readme-salvage.html
>
> or some of the tools from:
> https://ext4.wiki.kernel.org/index.php/Undeletion
>
> you would want to make a copy of qcow2 image first, as some of the tools
> will damage it even more than it was in the beginning, so you want to start
> with fresh copy on each try.

Thanks a lot for those links - as it seems some kind of real survivial
kits for situations like mine.

This is at least something more to try, although, as you write, my hope
is limited due to the internal PGDMP format (that indeed gzip's data
inside).

Thanks again!

Udo Rader

--
Udo Rader, CTO
http://www.bestsolution.at
http://riaschissl.blogspot.com

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users