FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 08-08-2012, 12:31 AM
Mark Knecht
 
Default Want to seriously test a NEW hard drive

On Tue, Aug 7, 2012 at 4:05 PM, Dale <rdalek1967@gmail.com> wrote:
> In
> the meantime, it is what it is.

I completely understand Dale, and I'm not picking on you. Just get
that second drive as soon as you can.

Cheers,
Mark
 
Old 08-08-2012, 12:32 AM
Mark Knecht
 
Default Want to seriously test a NEW hard drive

On Tue, Aug 7, 2012 at 4:06 PM, Peter Humphrey <peter@humphrey.ukfsn.org> wrote:
> On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:
>
>> Dead out of the box is dead. However a drive failing in a couple of
>> months _might_ have showed up in the smartctl output ...
>
> I wonder. Does anyone here know what most often causes HD failure after
> a couple of months? I imagine it's some component whose material is
> substandard, in which case it might show up with smartclt or it might
> not. I dunno.
>
> --
> Rgds
> Peter
>

This has come up before on this list I think. This is the overarching
sort of thinking about failure over lifetime...

http://en.wikipedia.org/wiki/Bathtub_curve
 
Old 08-08-2012, 02:19 AM
Dale
 
Default Want to seriously test a NEW hard drive

Paul Hartman wrote:
> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
>> I didn't know you could do low level formats anymore. Really? What
>> package provides that? Hmmm, I'm thinking about those HOURS spent
>> formatting a 100Mb drive and then thinking about how long it will take
>> to do a 3Tb drive. O_O I mean really O_O. LOL
> hdparm provides it. Do a search for "ATA secure erase" or "enhanced
> secure erase". It is as close as there is to a low-level format in
> modern drives. It is basically a erase/format within the drive's
> firmware, that resets it all back to factory, including bad sectors,
> with the same pattern of 1's and 0's and everything. You can do it
> with hdparm but it's tricky and contains many warnings about killing
> your drive. It is considered the only "true" way to properly erase a
> hard drive as anything else is just overwriting and does not
> necessarily touch all the areas that the firmware can touch. I think
> actual implementation of what the secure erases do varies from drive
> to drive, but they'll all format the whole disk for sure.
>
> The parted magic live CD contains a GUI tool to automate it and it is
> extremely simple to use. Choose your drive and go. On a 2tb drive I
> think it took 4 or 5 hours when I ran it. There is absolutely no
> feedback while it is running, so you're just waiting with no progress
> indicator or anything. You can also do SMART tests from within the
> parted magic live CD environment. And of course partitioning.
>
> That all being said, when performing this kind of operation I usually
> like to use a live CD and unplug ALL OTHER HARD DRIVES except for the
> one I'm going to destroy. I don't want to accidentally erase the wrong
> drive. (In fact I have an old Pentium 4 computer with no HDDs that I
> use solely for the purpose of testing live CDs, testing and formatting
> drives, partitioning new drives before i put them into a production
> machine)
>
>


I have seen where people use dd to do this sort of thing to. I read
somewhere that if you do a dd and put in all 1's, then all 0's then back
again that it is very hard to get any data back off the drive. I think
if you do it like over a dozen times, it is deemed impossible to get
anything back. I think that is the Government standard of it's gone.

4 or 5 hours huh. I guess drives are a lot faster now. Back in the
late 80's or early 90's, it took that long for those whimpy little 100Mb
drives. Ooops, my ages is showing again. lol

I got to go read up on hdparm. I already have it installed here. I'm
not planning to use this part but do want to read up on this.

Thanks.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 
Old 08-08-2012, 03:55 AM
William Kenworthy
 
Default Want to seriously test a NEW hard drive

On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
> Paul Hartman wrote:
> > On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
> >> I didn't know you could do low level formats anymore. Really? What
> >> package provides that? Hmmm, I'm thinking about those HOURS spent
> >> formatting a 100Mb drive and then thinking about how long it will take
> >> to do a 3Tb drive. O_O I mean really O_O. LOL
> > hdparm provides it. Do a search for "ATA secure erase" or "enhanced
> > secure erase". It is as close as there is to a low-level format in
...
>
> I have seen where people use dd to do this sort of thing to. I read
> somewhere that if you do a dd and put in all 1's, then all 0's then back
> again that it is very hard to get any data back off the drive. I think
> if you do it like over a dozen times, it is deemed impossible to get
> anything back. I think that is the Government standard of it's gone.
>
> 4 or 5 hours huh. I guess drives are a lot faster now. Back in the
> late 80's or early 90's, it took that long for those whimpy little 100Mb
> drives. Ooops, my ages is showing again. lol
>
> I got to go read up on hdparm. I already have it installed here. I'm
> not planning to use this part but do want to read up on this.
>
> Thanks.
>
> Dale
>
> :-) :-)
>


Goggle have a well known document
(http://research.google.com/archive/disk_failures.pdf) where they
analysed hard drive failures for a very large number of drives ... the
basic upshot is that a very large portion of failures happen with no
pre-warning, so testing a drive like you are proposing not going to
prove a thing.

They also found that smart (is quite dumb) and its tests were of little
use.

And high temperatures and work loads were also not a reliable guide to
trends in failure rates, both of which which surprised me.

Some of those bathtub curves that I was trained on when setting
maintenance schedules dont hold water here!

This anaysis of the paper looks quite good if you want the lite view:
http://storagemojo.com/2007/02/19/googles-disk-failure-experience/

BillK



BillK
 
Old 08-08-2012, 04:02 AM
Adam Carter
 
Default Want to seriously test a NEW hard drive

> I have seen where people use dd to do this sort of thing to. I read
> somewhere that if you do a dd and put in all 1's, then all 0's then back
> again that it is very hard to get any data back off the drive. I think
> if you do it like over a dozen times, it is deemed impossible to get
> anything back. I think that is the Government standard of it's gone.

I've heard the old attacks to recover data from a zerod drive are no
longer viable for disks of greater capacity than about 10G. I haven't
seen the information myself, however.

A single pass using dd would probably a good way of detecting any
existing bad blocks, so a smartctl then dd then smartctl again and a
diff of the results may be interesting.

I just use a 1TB software mirror for my backups.
 
Old 08-08-2012, 04:16 AM
William Kenworthy
 
Default Want to seriously test a NEW hard drive

On Wed, 2012-08-08 at 14:02 +1000, Adam Carter wrote:
> > I have seen where people use dd to do this sort of thing to. I read
> > somewhere that if you do a dd and put in all 1's, then all 0's then back
> > again that it is very hard to get any data back off the drive. I think
> > if you do it like over a dozen times, it is deemed impossible to get
> > anything back. I think that is the Government standard of it's gone.
>
> I've heard the old attacks to recover data from a zerod drive are no
> longer viable for disks of greater capacity than about 10G. I haven't
> seen the information myself, however.
>
> A single pass using dd would probably a good way of detecting any
> existing bad blocks, so a smartctl then dd then smartctl again and a
> diff of the results may be interesting.
>
> I just use a 1TB software mirror for my backups.
>

To wipe a drive use dban. - live CD which uses (US) gov approved
standards of wipe methods/patterns.

dd is only going to show sectors on a failed drive - too late!

To explain, modern drives have a store of locations they can use to
transparently replace any failed locations (apparently similar to the
way SSD's do it) - the internal drive electronics handle this and its
not visible externally though smart data seems to show it, but as google
says, smart is a bit suspect. The problem of a bad sector will only
show once all the reserved locations are used up, by which time the
drive is usually in rampant failure.

I do suspect this is one reason for googles results - actual failures of
the media (as against the motors/electronics are much as they always
have been, but the drives are not reporting them until its too late.

BillK
 
Old 08-08-2012, 04:18 AM
Dale
 
Default Want to seriously test a NEW hard drive

William Kenworthy wrote:
> On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
>> Paul Hartman wrote:
>>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
>>>> I didn't know you could do low level formats anymore. Really? What
>>>> package provides that? Hmmm, I'm thinking about those HOURS spent
>>>> formatting a 100Mb drive and then thinking about how long it will take
>>>> to do a 3Tb drive. O_O I mean really O_O. LOL
>>> hdparm provides it. Do a search for "ATA secure erase" or "enhanced
>>> secure erase". It is as close as there is to a low-level format in
> ...
>> I have seen where people use dd to do this sort of thing to. I read
>> somewhere that if you do a dd and put in all 1's, then all 0's then back
>> again that it is very hard to get any data back off the drive. I think
>> if you do it like over a dozen times, it is deemed impossible to get
>> anything back. I think that is the Government standard of it's gone.
>>
>> 4 or 5 hours huh. I guess drives are a lot faster now. Back in the
>> late 80's or early 90's, it took that long for those whimpy little 100Mb
>> drives. Ooops, my ages is showing again. lol
>>
>> I got to go read up on hdparm. I already have it installed here. I'm
>> not planning to use this part but do want to read up on this.
>>
>> Thanks.
>>
>> Dale
>>
>> :-) :-)
>>
>
> Goggle have a well known document
> (http://research.google.com/archive/disk_failures.pdf) where they
> analysed hard drive failures for a very large number of drives ... the
> basic upshot is that a very large portion of failures happen with no
> pre-warning, so testing a drive like you are proposing not going to
> prove a thing.
>
> They also found that smart (is quite dumb) and its tests were of little
> use.
>
> And high temperatures and work loads were also not a reliable guide to
> trends in failure rates, both of which which surprised me.
>
> Some of those bathtub curves that I was trained on when setting
> maintenance schedules dont hold water here!
>
> This anaysis of the paper looks quite good if you want the lite view:
> http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
>
> BillK
>
>
>
> BillK
>
>

Well, I am going by actual real experiences from other users of this
model of drive. I don't know what google was testing but I would bet it
is not the drive model I just bought. The users who bought this exact
model drive report that most failures are either out of the box or
within a few weeks to a month. I'm just going to try to increase my
odds even if it is just a little bit.

Smart may not always predict a failure but it is better than nothing at
all. Would you rather have a tool that may predict a failure or no tool
at all? Me, I'd rather have something that at least tries too. The one
drive I had to go bad, Smart predicted it very well. It said I had
about 24 hrs to get my stuff off. Sure enough, the next day, it
wouldn't do anything but spin. Without Smart and its prediction, I'd
have lost the data on the drive with no warning at all.

A couple questions. What if while I am testing this drive, it dies?
Does that prove that my testing benefited me then?

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 
Old 08-08-2012, 04:21 AM
Adam Carter
 
Default Want to seriously test a NEW hard drive

> To wipe a drive use dban. - live CD which uses (US) gov approved
> standards of wipe methods/patterns.

Or shred, which comes with coreutils.

> dd is only going to show sectors on a failed drive - too late!
>
> To explain, modern drives have a store of locations they can use to
> transparently replace any failed locations (apparently similar to the
> way SSD's do it) - the internal drive electronics handle this and its
> not visible externally though smart data seems to show it, but as google
> says, smart is a bit suspect. The problem of a bad sector will only
> show once all the reserved locations are used up, by which time the
> drive is usually in rampant failure.
>
> I do suspect this is one reason for googles results - actual failures of
> the media (as against the motors/electronics are much as they always
> have been, but the drives are not reporting them until its too late.

Ahh - go to know. My reasoning assumed that smart reports all remaps.
 
Old 08-08-2012, 05:42 AM
William Kenworthy
 
Default Want to seriously test a NEW hard drive

On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
> William Kenworthy wrote:
> > On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
> >> Paul Hartman wrote:
> >>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
...
> > Goggle have a well known document
> > (http://research.google.com/archive/disk_failures.pdf) where they
> > analysed hard drive failures for a very large number of drives ... the
> > basic upshot is that a very large portion of failures happen with no
> > pre-warning, so testing a drive like you are proposing not going to
> > prove a thing.
> >
> > They also found that smart (is quite dumb) and its tests were of little
> > use.
> >
> > And high temperatures and work loads were also not a reliable guide to
> > trends in failure rates, both of which which surprised me.
> >
> > Some of those bathtub curves that I was trained on when setting
> > maintenance schedules dont hold water here!
> >
> > This anaysis of the paper looks quite good if you want the lite view:
> > http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
> >
> > BillK
> >
> >
>
> Well, I am going by actual real experiences from other users of this
> model of drive. I don't know what google was testing but I would bet it
> is not the drive model I just bought. The users who bought this exact
> model drive report that most failures are either out of the box or
> within a few weeks to a month. I'm just going to try to increase my
> odds even if it is just a little bit.
>
> Smart may not always predict a failure but it is better than nothing at
> all. Would you rather have a tool that may predict a failure or no tool
> at all? Me, I'd rather have something that at least tries too. The one
> drive I had to go bad, Smart predicted it very well. It said I had
> about 24 hrs to get my stuff off. Sure enough, the next day, it
> wouldn't do anything but spin. Without Smart and its prediction, I'd
> have lost the data on the drive with no warning at all.
>
> A couple questions. What if while I am testing this drive, it dies?
> Does that prove that my testing benefited me then?
>
> Dale
>
> :-) :-)
>

Read the paper - its written by someone who buys drives in batches of
100's+, not by a few guys posting on a forum somewhere who bought one
random drive, who probably didn't use anti-static techniques handling
the drive, and thumped it around in the boot of the car or got it via
the courier who was famous for delivering TV's by throwing them over the
fence. It is a bit of an eye opener - read it.

My impression of models is that it is not really the model that has a
run of failures, but the batch so a different run of the same model will
have a different failure pattern. There are exceptions such as those
IBM 60G deathstar drives, but then again they fixed it and following
drives of the same model were fine.

My own experience of smart is it tells you something, but what it seems
to say is not right (notice I am not saying it tells lies, but that the
data and interpretation don't make sense on the drives Ive had)

Drive failure does seem to be a semi-random lottery, but I am seriously
doubtful that testing will do anything ... it has as much chance of
precipitating failure that wouldn't occur otherwise because you are
seriously hammering it, or weakening the drive so it will fail at some
random time, but perhaps weeks away rather than the years it otherwise
would, or nothing will happen except for wasted electrons.

Then again, I am of the view that modern electronics is
designed/programmed to fail a few seconds past warranty expiry (why else
do most devices have timekeeping built in

BillK
 
Old 08-08-2012, 06:24 AM
Dale
 
Default Want to seriously test a NEW hard drive

William Kenworthy wrote:
> On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
>> William Kenworthy wrote:
>>> On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
>>>> Paul Hartman wrote:
>>>>> On Tue, Aug 7, 2012 at 5:26 PM, Dale <rdalek1967@gmail.com> wrote:
> ...
>>> Goggle have a well known document
>>> (http://research.google.com/archive/disk_failures.pdf) where they
>>> analysed hard drive failures for a very large number of drives ... the
>>> basic upshot is that a very large portion of failures happen with no
>>> pre-warning, so testing a drive like you are proposing not going to
>>> prove a thing.
>>>
>>> They also found that smart (is quite dumb) and its tests were of little
>>> use.
>>>
>>> And high temperatures and work loads were also not a reliable guide to
>>> trends in failure rates, both of which which surprised me.
>>>
>>> Some of those bathtub curves that I was trained on when setting
>>> maintenance schedules dont hold water here!
>>>
>>> This anaysis of the paper looks quite good if you want the lite view:
>>> http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
>>>
>>> BillK
>>>
>>>
>> Well, I am going by actual real experiences from other users of this
>> model of drive. I don't know what google was testing but I would bet it
>> is not the drive model I just bought. The users who bought this exact
>> model drive report that most failures are either out of the box or
>> within a few weeks to a month. I'm just going to try to increase my
>> odds even if it is just a little bit.
>>
>> Smart may not always predict a failure but it is better than nothing at
>> all. Would you rather have a tool that may predict a failure or no tool
>> at all? Me, I'd rather have something that at least tries too. The one
>> drive I had to go bad, Smart predicted it very well. It said I had
>> about 24 hrs to get my stuff off. Sure enough, the next day, it
>> wouldn't do anything but spin. Without Smart and its prediction, I'd
>> have lost the data on the drive with no warning at all.
>>
>> A couple questions. What if while I am testing this drive, it dies?
>> Does that prove that my testing benefited me then?
>>
>> Dale
>>
>> :-) :-)
>>
> Read the paper - its written by someone who buys drives in batches of
> 100's+, not by a few guys posting on a forum somewhere who bought one
> random drive, who probably didn't use anti-static techniques handling
> the drive, and thumped it around in the boot of the car or got it via
> the courier who was famous for delivering TV's by throwing them over the
> fence. It is a bit of an eye opener - read it.
>
> My impression of models is that it is not really the model that has a
> run of failures, but the batch so a different run of the same model will
> have a different failure pattern. There are exceptions such as those
> IBM 60G deathstar drives, but then again they fixed it and following
> drives of the same model were fine.
>
> My own experience of smart is it tells you something, but what it seems
> to say is not right (notice I am not saying it tells lies, but that the
> data and interpretation don't make sense on the drives Ive had)
>
> Drive failure does seem to be a semi-random lottery, but I am seriously
> doubtful that testing will do anything ... it has as much chance of
> precipitating failure that wouldn't occur otherwise because you are
> seriously hammering it, or weakening the drive so it will fail at some
> random time, but perhaps weeks away rather than the years it otherwise
> would, or nothing will happen except for wasted electrons.
>
> Then again, I am of the view that modern electronics is
> designed/programmed to fail a few seconds past warranty expiry (why else
> do most devices have timekeeping built in
>
> BillK
>

Actually, I read the paper a long time ago. May give it another look
but I'm still going by what people have posted about this specific
model. If they make them all the same, then testing to at least see if
it is going to get past the initial stages is a good idea. I do think
some failures were because of the BIOS and I stated that in my original
post. Getting a DOA drive can happen but when there are mobos around
that can't see large drives, then one has to consider it. The ones I
worry about are the ones that worked for a few weeks or a month then
died. They obviously don't have BIOS issues but some other problem.

Still, all things considered, I'm going to test the drive. If it can
pass the test then I will feel better about putting my data on it. As
for the paper:

root@fireball / # ls -al /home/dale/Desktop/disk_failures.pdf
-rw-r--r-- 1 dale users 247492 May 21 17:58
/home/dale/Desktop/disk_failures.pdf
root@fireball / #


I read it back in May. It's still sitting on my desktop. I might also
add, it is about 5 years old. Drives have changed since then. For one,
they have gotten larger. We don't know what else may have changed either.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 

Thread Tools




All times are GMT. The time now is 05:00 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org