recovery recommendations
 

Recently a 640GB external enclosure was PEBKAC'd with the following
command:

dd if=/some-185mb-linux-install.iso of=/dev/sdx

[not of=/dev/sdx1]

Since the PEBKAC, the drive has not been written to beyond being
unplugged. I have made an image of the drive and have attempted to run
against it: testdisk, fsck.ext3 with alternate superblocks, and even
"fsck.ext3 -Sy", to no avail.

So. I am *convinced* that my 550gb of data is recoverable. It seems that
[obviously] the first 185mb is gone - whatever files those were.

I have googled, but to be honest, I am not entirely sure I'm searching
the right strings. Most software seems geared towards partition recovery
where a partition has become damaged [which is technically what's
happened], but not exactly; or partition undeleting [which again isn't
technically what happened].

Please, any recommendations that don't involve a time machine are much
appreciated.

If this isn't the place to be asking, please point me in the right
direction.

Thank you.

-mp- - kicking self -


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

On 1/21/11 12:36 PM, m.p. wrote:
> Recently a 640GB external enclosure was PEBKAC'd with the following
> command:
>
> dd if=/some-185mb-linux-install.iso of=/dev/sdx
>
> [not of=/dev/sdx1]
>
> Since the PEBKAC, the drive has not been written to beyond being
> unplugged. I have made an image of the drive and have attempted to run
> against it: testdisk, fsck.ext3 with alternate superblocks, and even
> "fsck.ext3 -Sy", to no avail.
>
> So. I am *convinced* that my 550gb of data is recoverable. It seems that
> [obviously] the first 185mb is gone - whatever files those were.

You really need to restore from backups. You just overwrote 30% of
your filesystem with something else... you've lost partition data,
metadata, directory data, file data ... whatever was in that first
185M.

I think the best you can do is low-level recovery at this point,
groveling around for file fragments with something like the photo recovery
tools.

> I have googled, but to be honest, I am not entirely sure I'm searching
> the right strings. Most software seems geared towards partition recovery
> where a partition has become damaged [which is technically what's
> happened],

You've done much more than that...

> but not exactly; or partition undeleting [which again isn't
> technically what happened].
>
> Please, any recommendations that don't involve a time machine are much
> appreciated.

Maybe, just -maybe- if you can find a backup superblock, fsck might try
to piece a few things together.

But you can't generally overwrite 1/3 of a filesystem and expect normal
tools to recover for you, I'm afraid.

-Eric

> If this isn't the place to be asking, please point me in the right
> direction.
>
> Thank you.
>
> -mp- - kicking self -
>
>
> _______________________________________________
> Ext3-users mailing list
> Ext3-users@redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

--On 21 January 2011 10:36:41 -0800 "m.p." <martin@ebcl.lib.id.us> wrote:


dd if=/some-185mb-linux-install.iso of=/dev/sdx


You wrote to /dev/sdx not /dev/sdxN (where N is a number)?

You will have zapped your partition table. If you only had one partition
on there, you can recreate this by running fdisk on the image first.

Note ext3 recovery tools want a partition image, not a disk image.
You can use dd with the appropriate skip (or use sparsecopy
from
http://blog.alex.org.uk/2010/12/02/copying-sparse-files/
) to get the partition out from the disk image.

--
Alex Bligh

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

On 2011-01-21, at 12:06, Eric Sandeen wrote:
> On 1/21/11 12:36 PM, m.p. wrote:
>> Recently a 640GB external enclosure was PEBKAC'd with the following
>> command:
>>
>> dd if=/some-185mb-linux-install.iso of=/dev/sdx
>>
>> [not of=/dev/sdx1]
>>
>> Since the PEBKAC, the drive has not been written to beyond being
>> unplugged. I have made an image of the drive and have attempted to run
>> against it: testdisk, fsck.ext3 with alternate superblocks, and even
>> "fsck.ext3 -Sy", to no avail.
>>
>> So. I am *convinced* that my 550gb of data is recoverable. It seems that
>> [obviously] the first 185mb is gone - whatever files those were.
>
> You really need to restore from backups. You just overwrote 30% of
> your filesystem with something else... you've lost partition data,
> metadata, directory data, file data ... whatever was in that first
> 185M.

Eric, note 185 Megabytes, vs 550 Gigabytes, so only the first 1.5 groups were clobbered (which may have been largely filled by the journal, depending on when the filesystem was formatted).

> I think the best you can do is low-level recovery at this point,
> groveling around for file fragments with something like the photo recovery
> tools.

Actually, I think the chance for significant data recovery is pretty good.

> Maybe, just -maybe- if you can find a backup superblock, fsck might try
> to piece a few things together.

I think the first thing to do is to recover the partition table EXACTLY the way it was before. There was a tool for this, "gpart" or something similar, that could recover partition tables.

Alternately, if you build and run the "findsuper" tool from e2fsprogs sources (I've attached an x86_64 binary here, maybe it will work for you), it will tell you the starting and ending offset of each ext* filesystem, based on superblocks that it finds on the disk. You need to take the byte offsets and convert them into kB or sector offsets for use with fdisk.

Once you get the partition table correct, e2fsck should have no problem to recover the filesystem, though it will be missing the root directory and possibly a bunch of other files that were stored in the first 185MB of disk. The possibly good news is that the journal _used_ to be stored at the start of the filesystem, and would fill 32MB or 128MB of the start of the disk, and in turn that dissuaded the allocator from putting files into that group. Sadly (maybe), the journal is now allocated in the middle of the filesystem for performance reasons and that coincidental "data protection" is no longer with us.

> But you can't generally overwrite 1/3 of a filesystem and expect normal
> tools to recover for you, I'm afraid.

Surprisingly, extN is very robust in this regard. I've been able (required) to recover data from similarly corrupted filesystems, and a lot can be done. With changes like flex_bg and UNMAP it will get a lot harder, however.


Cheers, Andreas






_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

On 1/21/11 2:05 PM, Andreas Dilger wrote:
> On 2011-01-21, at 12:06, Eric Sandeen wrote:
>> On 1/21/11 12:36 PM, m.p. wrote:
>>> Recently a 640GB external enclosure was PEBKAC'd with the
>>> following command:
>>>
>>> dd if=/some-185mb-linux-install.iso of=/dev/sdx
>>>
>>> [not of=/dev/sdx1]
>>>
>>> Since the PEBKAC, the drive has not been written to beyond being
>>> unplugged. I have made an image of the drive and have attempted
>>> to run against it: testdisk, fsck.ext3 with alternate
>>> superblocks, and even "fsck.ext3 -Sy", to no avail.
>>>
>>> So. I am *convinced* that my 550gb of data is recoverable. It
>>> seems that [obviously] the first 185mb is gone - whatever files
>>> those were.
>>
>> You really need to restore from backups. You just overwrote 30%
>> of your filesystem with something else... you've lost partition
>> data, metadata, directory data, file data ... whatever was in that
>> first 185M.
>
> Eric, note 185 Megabytes, vs 550 Gigabytes, so only the first 1.5
> groups were clobbered (which may have been largely filled by the
> journal, depending on when the filesystem was formatted).

Oops sorry, my old physics teacher would not be proud:
"think units before you think numbers" he said...

So ignore what I said, and listen to Andreas. Sorry about that!

-Eric

>> I think the best you can do is low-level recovery at this point,
>> groveling around for file fragments with something like the photo
>> recovery tools.
>
> Actually, I think the chance for significant data recovery is pretty
> good.
>
>> Maybe, just -maybe- if you can find a backup superblock, fsck might
>> try to piece a few things together.
>
> I think the first thing to do is to recover the partition table
> EXACTLY the way it was before. There was a tool for this, "gpart" or
> something similar, that could recover partition tables.
>
> Alternately, if you build and run the "findsuper" tool from e2fsprogs
> sources (I've attached an x86_64 binary here, maybe it will work for
> you), it will tell you the starting and ending offset of each ext*
> filesystem, based on superblocks that it finds on the disk. You need
> to take the byte offsets and convert them into kB or sector offsets
> for use with fdisk.
>
> Once you get the partition table correct, e2fsck should have no
> problem to recover the filesystem, though it will be missing the root
> directory and possibly a bunch of other files that were stored in the
> first 185MB of disk. The possibly good news is that the journal
> _used_ to be stored at the start of the filesystem, and would fill
> 32MB or 128MB of the start of the disk, and in turn that dissuaded
> the allocator from putting files into that group. Sadly (maybe), the
> journal is now allocated in the middle of the filesystem for
> performance reasons and that coincidental "data protection" is no
> longer with us.
>
>> But you can't generally overwrite 1/3 of a filesystem and expect
>> normal tools to recover for you, I'm afraid.
>
> Surprisingly, extN is very robust in this regard. I've been able
> (required) to recover data from similarly corrupted filesystems, and
> a lot can be done. With changes like flex_bg and UNMAP it will get a
> lot harder, however.
>
>
> Cheers, Andreas
>
>
>
>
>

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

Hello m.p.

On Friday 21 January 2011 19:36:41 m.p. wrote:
> Recently a 640GB external enclosure was PEBKAC'd with the following
> command:

A short time ago I overwrote the first 12GB of my single-disk 1TB system. Thise
12GB luckily contained only OS files I was about to replace anyway (still gotta
write that up as solved; see the archives for "Overwritten beginning of ext3
filesystem. Recovery?" on 27dec2010). I got all my important user data back in
one piece.

Depending on usage patterns and the kind of data you had on that disk, I think
your chances are pretty good to get most of it back.

> So. I am *convinced* that my 550gb of data is recoverable. It seems that
> [obviously] the first 185mb is gone - whatever files those were.

In my case, I overwrote an ext3 partition's first few GB with a valid ext3 FS
of identical size but different content. fsck then found lots of errors (in the
"overlaying" FS) due to it being incomplete. The problems were so bad that it
never actually got to recovering the good ("underlaying") data.

What did the trick for me was to zero out the offending stuff. In your case that
would be sth. like:

* Keep a copy of your "unrecovered" disk image! You don't want to touch the
original disk at all, so be prepared for making mistakes
* dd if=/dev/zero of=PARTIMAGE count=ISOSIZE
where ISOSIZE is the ISO image's size in bytes
* Extract the partition image like Alex Bligh wrote and work with that
* e2fsck -f -y /dev/sdx1 (perhaps -b and -C0 as well)
e2fsck will probably ask you to be run again (might want do that anyway to
make sure)

You'll then find the remainder of your data in lost+found/, even most directory
hierarchies will be intact. Some pretty much random(!) data blocks from
seemingly anywhere(!) on disk will be zeroed out, but at least those will be
easily recognizable. The exact outcome depends on what data you had on there,
as well as usage scenario and history.

It's imperative that you check every file of the results - any data block might
be missing, any subdirectory might be missing. If you lost a directory entry,
chances are high that it's contents will show up directly (and namelessly) in
lost+found/.

Hope that helps. If you find the time, please keep me posted about your
results.

--
With best regards,
Florian Weber

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

All,

Thank you *very* much for the salient & encouraging suggestions.

The usage of the drive is/was purely storage - stuffed inside an
external enclosure.

I will keep those interested parties apprised of any developments - it
will likely be several days - it took over 40 hours just to image the
murder-victim > rescue-drive.

And later on, I will submit to all the "where was your backup?" you care
to inflict. :) For now, it's too much agony. Thirty years worth of
scanned family fotos [among other irreplaceables] hang in the balance -
you know the drill. It's *never* the drive full of Robot Chicken
episodes that gets nuked ....

-m-

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

--On 21 January 2011 16:12:46 -0800 "m.p." <martin@ebcl.lib.id.us> wrote:


Thirty years worth of
scanned family fotos [among other irreplaceables] hang in the balance -
you know the drill.


I've had 100% success getting photos back with PhotoRec which doesn't
rely on metadata.

--
Alex Bligh

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

On Fri, Jan 21, 2011 at 10:51:20PM +0100, Florian Weber wrote:
> * Keep a copy of your "unrecovered" disk image! You don't want to touch the
> original disk at all, so be prepared for making mistakes

yes, that is always the most important step. And be extra extra carefull you
don't nuke it by accident... readonly jumper on the drive (if it has one)
helps.

> * dd if=/dev/zero of=PARTIMAGE count=ISOSIZE
> where ISOSIZE is the ISO image's size in bytes

actually, it is ISO image size in *blocks* [1], so this procedure would
actually destroy 512 times more data than original error... :-|

Anyway, let us know how it worked out...

And if this procedure fails you, I'd also second the great experiences with
http://www.cgsecurity.org/wiki/PhotoRec - works even if filesystem and
partition structures are completely foobared... If you use it and it works
for you, don't forget to donate to the guy, there's link on the pages.

[1] by default block size is 512 bytes, can be changed with bs= parametar to
dd(1) -- note that while you can use 1 byte as an unit, it will be terribly
slow; so it is probably best to use let's say bs=4K, divide ISOSIZE in bytes
by 4096, round it down to nearest integer, and use that as count= parameter.

--
Opinions above are GNU-copylefted.

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

recovery recommendations
 

A simple one would be:
dd if=/dev/zero of=PARTIMAGE bs=1M count=184
(presuming that your original ISO was 185MB.* -- this leaves up to 1MB of random crap, but (in my mind), that's better than tashing the good bits of data.


You can get more accurate results by varying the block size and the count so that they multiply out properly.
modern versions of DD recognize multipliers of K -> kilobytes, M->Megabytes, G->gigabytes
* .. and those are the CS units (now called kibibytes, mebibytes, gibibytes), not the marketing units.



On Sat, Jan 22, 2011 at 5:57 AM, Matija Nalis <mnalis-ml@voyager.hr> wrote:

On Fri, Jan 21, 2011 at 10:51:20PM +0100, Florian Weber wrote:

> * Keep a copy of your "unrecovered" disk image! You don't want to touch the

> * *original disk at all, so be prepared for making mistakes



yes, that is always the most important step. And be extra extra carefull you

don't nuke it by accident... readonly jumper on the drive (if it has one)

helps.



> * dd if=/dev/zero of=PARTIMAGE count=ISOSIZE

> * *where ISOSIZE is the ISO image's size in bytes



actually, it is ISO image size in *blocks* [1], so this procedure would

actually destroy 512 times more data than original error... :-|



Anyway, let us know how it worked out...



And if this procedure fails you, I'd also second the great experiences with

http://www.cgsecurity.org/wiki/PhotoRec - works even if filesystem and

partition structures are completely foobared... *If you use it and it works

for you, don't forget to donate to the guy, there's link on the pages.



[1] by default block size is 512 bytes, can be changed with bs= parametar to

* *dd(1) -- note that while you can use 1 byte as an unit, it will be terribly

* *slow; so it is probably best to use let's say bs=4K, divide ISOSIZE in bytes

* *by 4096, round it down to nearest integer, and use that as count= parameter.



--

Opinions above are GNU-copylefted.



_______________________________________________

Ext3-users mailing list

Ext3-users@redhat.com

https://www.redhat.com/mailman/listinfo/ext3-users



--
Stephen Samuel http://www.bcgreen.com* Software, like love,
778-861-7641* * * * * * * * * * * * * * * grows when you give it away


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users