FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 09-29-2012, 08:26 PM
Frank Bauer
 
Default Debian not suitable for SSD due to apt/dpkg?

Hi,

I am considering migrating my Debian testing system to a SSD to speed things up.
Since SSD lifetime is severely limited (about 5000 overwrites for
consumer grade MLC), I wanted to know beforehand, how much writes does
my system generate.

iostat (part of sysstat) revealed, that simple apt-get update command
generated about 250MB of writes!
We are talking about current debian testing on ext4 filesystem with
noatime option enabled.
Daily apt-get update && apt-get upgrade consumes about 1.2GB of writes
(downloaded size is usually <100MB)

Do you have similar experience? Isn't that a bit excessive? Can it be avoided?

Best Regards,
Frank


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CAPds5_+8TC_x3mTSmOGSdo=r-BQcaiyDYYo1howfWom9MUuKkQ@mail.gmail.com
 
Old 09-29-2012, 08:46 PM
David Paleino
 
Default Debian not suitable for SSD due to apt/dpkg?

On Sat, 29 Sep 2012 22:26:57 +0200, Frank Bauer wrote:

> Hi,

Hello,

> I am considering migrating my Debian testing system to a SSD to speed things
> up. Since SSD lifetime is severely limited (about 5000 overwrites for
> consumer grade MLC), I wanted to know beforehand, how much writes does
> my system generate.
> [..]

Have a look at http://wiki.debian.org/SSDoptimization .

David

--
. '`. Debian developer | http://wiki.debian.org/DavidPaleino
: :' : Linuxer #334216 --|-- http://www.hanskalabs.net/
`. `'` GPG: 1392B174 ----|---- http://deb.li/dapal
`- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174
 
Old 09-29-2012, 09:15 PM
Bastian Blank
 
Default Debian not suitable for SSD due to apt/dpkg?

On Sat, Sep 29, 2012 at 10:26:57PM +0200, Frank Bauer wrote:
> I am considering migrating my Debian testing system to a SSD to speed things up.
> Since SSD lifetime is severely limited (about 5000 overwrites for
> consumer grade MLC), I wanted to know beforehand, how much writes does
> my system generate.

If you have 5000 erase cycles, it will run for 13 years if you overwrite
it once per day. Do you really expect this device to work until this?

> iostat (part of sysstat) revealed, that simple apt-get update command
> generated about 250MB of writes!

How does it reveal this?

> We are talking about current debian testing on ext4 filesystem with
> noatime option enabled.

You also need discard.

> Daily apt-get update && apt-get upgrade consumes about 1.2GB of writes
> (downloaded size is usually <100MB)

Disable pdiffs, mount /var/cache/apt as tmpfs, so at least the packages
are not written again.

> Do you have similar experience? Isn't that a bit excessive? Can it be avoided?

No idea, it is irrelevant. No. Partially.

Bastian

--
Ahead warp factor one, Mr. Sulu.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120929211544.GB17303@wavehammer.waldi.eu.org">ht tp://lists.debian.org/20120929211544.GB17303@wavehammer.waldi.eu.org
 
Old 09-30-2012, 10:49 AM
Frank Bauer
 
Default Debian not suitable for SSD due to apt/dpkg?

On Sat, Sep 29, 2012 at 11:15:45PM +0200, Bastian Blank wrote:
>
> If you have 5000 erase cycles, it will run for 13 years if you overwrite
> it once per day. Do you really expect this device to work until this?

Why not, my computer upgrade cycles are about 6-8 years and the
computer won't be idling all the time - especially considering modern
desktop environments running whole database engines to store
config/meta data.

Is writing of 160GB/day realistic? Hopefuly not, but see my apt
measurements below.

There is also something called SSD write amplification - the erase
blocks on the device are often larger than your normal filesystem
blocks, which might lead to up to 10x data actually writen to SSD,
i.e. down to 1.3years of overwrites in the extreme case.

> > iostat (part of sysstat) revealed, that simple apt-get update command
> > generated about 250MB of writes!
>
> How does it reveal this?

Running iostat -dm -p sda before and after apt-get, looking at the
MB_wrtn column.

Immediately after boot to xmonad:

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 81,24 1,96 0,12 175 10
sda1 3,38 0,01 0,00 1 0
sda2 64,64 1,89 0,12 170 10
sda3 1,78 0,01 0,00 0 0
sda4 0,02 0,00 0,00 0 0
sda5 1,82 0,01 0,00 0 0
sda6 3,65 0,02 0,00 1 0
sda7 5,56 0,02 0,00 1 0

After running apt-get update after about a week (relevant device only,
focus on the last column):

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda2 42,66 0,75 4,32 246 1413

so just updating the package info wrote 1.4GB of data (disabling pdiff
might help as you suggested)

After running apt-get upgrade (expected APT DL size: 87.2MB):

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda2 56,43 0,36 2,69 317 2376

so installing 90 MB of packages produced about 900 MB of writes

Running apt-get update again (there is nothing to update right now),
about 40 MB of data are written in each subsequent run.

> Disable pdiffs, mount /var/cache/apt as tmpfs, so at least the packages
> are not written again.

Good call, thanks

Frank


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CAPds5_KFZ+oHC5Z5iEErCPycP-yOEMq17FGuFQ=tMytwxr3xXQ@mail.gmail.com
 
Old 09-30-2012, 01:04 PM
Игорь Пашев
 
Default Debian not suitable for SSD due to apt/dpkg?

Relax :-)

It is a hardware problem.

Just keep doing your regular job till SDD become more robust.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CALL-Q8xFUS22ff1by=hXuQNd5giFMu5vdF5fh-TqbLriWb3iaQ@mail.gmail.com
 
Old 10-01-2012, 02:23 AM
Chow Loong Jin
 
Default Debian not suitable for SSD due to apt/dpkg?

On 30/09/2012 18:49, Frank Bauer wrote:
> Why not, my computer upgrade cycles are about 6-8 years and the
> computer won't be idling all the time - especially considering modern
> desktop environments running whole database engines to store
> config/meta data.
>
> Is writing of 160GB/day realistic? Hopefuly not, but see my apt
> measurements below.
>
> There is also something called SSD write amplification - the erase
> blocks on the device are often larger than your normal filesystem
> blocks, which might lead to up to 10x data actually writen to SSD,
> i.e. down to 1.3years of overwrites in the extreme case.

Have you done any actual calculation on this? A quick Google search on SSD write
cycles shows more articles debunking this theory than supporting it.

--
Kind regards,
Loong Jin
 
Old 10-01-2012, 11:32 AM
Frank Bauer
 
Default Debian not suitable for SSD due to apt/dpkg?

On Mon, Oct 01, 2012 at 10:23:32AM +0800, Chow Loong Jin wrote:
>
> Have you done any actual calculation on this? A quick Google search on SSD write
> cycles shows more articles debunking this theory than supporting it.

Reading specifications of intel's SSD 320 line at the following link:

http://www.intel.com/content/www/us/en/solid-state-drives/ssd-320-enterprise-server-storage-application-specification-addendum.html

states in section 2.3 Reliability that for 160GB drive the write
endurance is 15TB, which gives about 94 full overwrite cycles. Not
that much in my eyes.

The point, however, is not whether the SSD in question will last 94 or
100 overwrites, but whether Debian's packaging system is not causing
too much unneccessary write overhead.

Frank


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CAPds5_KNssZ_6oU+VVgtg2_jqvLp1yGGsvtAbz=isqTqj0fWr Q@mail.gmail.com
 
Old 10-01-2012, 01:12 PM
Michael Hanke
 
Default Debian not suitable for SSD due to apt/dpkg?

On Mon, Oct 01, 2012 at 10:23:32AM +0800, Chow Loong Jin wrote:
> On 30/09/2012 18:49, Frank Bauer wrote:
> > Why not, my computer upgrade cycles are about 6-8 years and the
> > computer won't be idling all the time - especially considering modern
> > desktop environments running whole database engines to store
> > config/meta data.
> >
> > Is writing of 160GB/day realistic? Hopefuly not, but see my apt
> > measurements below.
> >
> > There is also something called SSD write amplification - the erase
> > blocks on the device are often larger than your normal filesystem
> > blocks, which might lead to up to 10x data actually writen to SSD,
> > i.e. down to 1.3years of overwrites in the extreme case.
>
> Have you done any actual calculation on this? A quick Google search on SSD write
> cycles shows more articles debunking this theory than supporting it.

Just a data point:

I'm on an Intel SSD (120GB) since Aug 2009 -- running Debian testing all
the time. I do not upgrade daily, but often. I have _not_ done any of
the optimizations mentioned on the wiki. I have on average approx 15GB
free on the drive. Obviously, I ran a kernel <3.2 for most of the time.
I do lots of compiling on this SSD.

So far, I have nothing to complain about and consider this drive as
fairly reliable.

Michael





--
Michael Hanke
http://mih.voxindeserto.de


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20121001131246.GD11280@meiner
 
Old 10-01-2012, 01:17 PM
Andreas Beckmann
 
Default Debian not suitable for SSD due to apt/dpkg?

On 2012-10-01 13:32, Frank Bauer wrote:
> On Mon, Oct 01, 2012 at 10:23:32AM +0800, Chow Loong Jin wrote:
>>
>> Have you done any actual calculation on this? A quick Google search on SSD write
>> cycles shows more articles debunking this theory than supporting it.
>
> Reading specifications of intel's SSD 320 line at the following link:
>
> http://www.intel.com/content/www/us/en/solid-state-drives/ssd-320-enterprise-server-storage-application-specification-addendum.html
>
> states in section 2.3 Reliability that for 160GB drive the write
> endurance is 15TB, which gives about 94 full overwrite cycles. Not
> that much in my eyes.

And it will take about 70 days to do these 15 TB of 4KB writes
distributed over the full device (600 IOPS according to Table 1)
(15*10^12/4096/600/3600/24)
while 5000 sequential overwrites could be finished within 53 days
(5000*160*10^9/165/2^20/3600/24) (165 MB/s, Table 2)
Of course you may not interrupt your write workload by reading, which
would delay the wearout.

Do you have any information about the overprovisioning and erase block
size of that device?

> The point, however, is not whether the SSD in question will last 94 or
> 100 overwrites, but whether Debian's packaging system is not causing
> too much unneccessary write overhead.

Neither of the above extremes will probably match *your* workload.
I would assume apt-get update will mostly do sequential writes (on
several files) and this will cause some file system meta-data updates
which may result in "more random" I/Os (but restricted to a small
portion of the disk) and they go to the filesystem log, too.

That specification also gives some nice instructions how to use SMART
attributes to analyze your work load ... did you look into this?


Andreas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 506997EC.3070102@abeckmann.de">http://lists.debian.org/506997EC.3070102@abeckmann.de
 
Old 10-01-2012, 01:37 PM
Holger Levsen
 
Default Debian not suitable for SSD due to apt/dpkg?

Hi,

On Montag, 1. Oktober 2012, Michael Hanke wrote:
> Just a data point:

interesting, thanks.

what are the main (ssd related) advantages of running a 3.2 kernel instead of
the 2.6.32 from squeeze? (I don't want to run 3.2 due to wlan/intel gfx
problems, though last time I tried was three months ago, might been fixed by
now.)


cheers,
Holger


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201210011537.20403.holger@layer-acht.org">http://lists.debian.org/201210011537.20403.holger@layer-acht.org
 

Thread Tools




All times are GMT. The time now is 08:54 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org