FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 05-09-2008, 03:36 AM
NN_il_Confusionario
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

> * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
>Does anyone know how much memory fsck needs to check a large filesystem?

some (rather old) information is contained in

Linkname: E2fsprogs Release Notes
URL: http://e2fsprogs.sourceforge.net/e2fsprogs-release.html

for example, it says

E2fsprogs 1.07 (March 14, 1997)

E2fsck is now uses much less memory when checking really large
filesystems (or rather, filesystems with a large number of inodes).
Previously a filesystem with 1 million inodes required 4 megabytes of
memory to store inode count statistics; that storage requirement
has now been reduced to roughly half a megabyte.

E2fsprogs 1.15 (July 18, 1999)

E2fsck now uses mallinfo if it exists to get accurate statistics about
its memory usage.

Moreover:

Linkname: Filesystems (ext3, reiser, xfs, jfs) comparison on Debian Etch
URL: http://www.debian-administration.org/articles/388

Times of over two months to 'fsck' a filesystem have been reported for
'ext3' and XFS sometimes requires more than 4GB of memory to run 'fsck'
(it is possible to create and use an XFS filesystem on a system with
a 32 bit that can only be 'fsck'ed on a 64 bit CPU, and at least one
case has actually happened). The basic problem is that while very
large filesystems using JFS or XFS (or very recent 'ext3') perform
well on RAID storage, because they take advantage of the parallel nature
of the underlying storage system, 'fsck' is single threaded in every
Linux file system design that I have seen. Bad news. More details here:
http://www.sabi.co.uk/Notes/anno05-4th.html#051012
http://www.sabi.co.uk/Notes/anno05-4th.html#051009


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 05:24 AM
NN_il_Confusionario
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

> * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
>Does anyone know how much memory fsck needs to check a large filesystem?

Linkname: Considerations when creating ext3 filesystems - Ext4
URL:
http://ext4.wiki.kernel.org/index.php/Considerations_when_creating_ext3_filesystems#Memo ry_needed_to_fsck_the_filesystem

Unfortunately, even that answer does not contain a formula to calculate the
needed memory (or time).


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 10:58 PM
Mitchell Laks
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

On 07:24 Fri 09 May , NN_il_Confusionario wrote:
> > * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
> >Does anyone know how much memory fsck needs to check a large filesystem?
>
> Linkname: Considerations when creating ext3 filesystems - Ext4
> URL:
> http://ext4.wiki.kernel.org/index.php/Considerations_when_creating_ext3_filesystems#Memo ry_needed_to_fsck_the_filesystem
>
> Unfortunately, even that answer does not contain a formula to calculate the
> needed memory (or time).
>

In my case, the fsck took about 24 hours - amd64 machine with 1 terabyte ram
running kernel 2.4.18.

thanks,

Mitchell Laks



>
> --
> To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-10-2008, 02:30 AM
Ron Johnson
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/09/08 17:58, Mitchell Laks wrote:
> On 07:24 Fri 09 May , NN_il_Confusionario wrote:
>>> * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
>>> Does anyone know how much memory fsck needs to check a large filesystem?
>> Linkname: Considerations when creating ext3 filesystems - Ext4
>> URL:
>> http://ext4.wiki.kernel.org/index.php/Considerations_when_creating_ext3_filesystems#Memo ry_needed_to_fsck_the_filesystem
>>
>> Unfortunately, even that answer does not contain a formula to calculate the
>> needed memory (or time).
>>
>
> In my case, the fsck took about 24 hours - amd64 machine with 1 terabyte ram
> running kernel 2.4.18.

A mobo with 256 DIMM slots must be *huge*. Send pictures!!!

- --
Ron Johnson, Jr.
Jefferson LA USA

We want... a Shrubbery!!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIJQisS9HxQb37XmcRAuJuAJ92ag5KYBOa151rkgQqAu 8bjIPBeQCeItkL
mQ7SAJ71KELRZpxy4eRrUPQ=
=BxPx
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-10-2008, 11:19 PM
"Douglas A. Tutty"
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

On Fri, May 09, 2008 at 06:58:02PM -0400, Mitchell Laks wrote:
> On 07:24 Fri 09 May , NN_il_Confusionario wrote:
> > > * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
> > >Does anyone know how much memory fsck needs to check a large filesystem?

> In my case, the fsck took about 24 hours - amd64 machine with 1 terabyte ram
> running kernel 2.4.18.
>

I'm assuming that you mean 1 TB disk.

I don't suppose you know how much memory it used? If your / fs isn't 1
TB, and you have 24 hrs to spend, you could check just that 1 TB
filesysem with top (or something) running and see.



Doug.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-11-2008, 02:37 AM
Mitchell Laks
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

On 19:19 Sat 10 May , Douglas A. Tutty wrote:
> On Fri, May 09, 2008 at 06:58:02PM -0400, Mitchell Laks wrote:
> > On 07:24 Fri 09 May , NN_il_Confusionario wrote:
> > > > * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
> > > >Does anyone know how much memory fsck needs to check a large filesystem?
>
> > In my case, the fsck took about 24 hours - amd64 machine with 1 terabyte ram
> > running kernel 2.4.18.
> >
>
> I'm assuming that you mean 1 TB disk.

LOL!

I had only 1G ram, but the raid was a 1T hardware raid.

>
> I don't suppose you know how much memory it used? If your / fs isn't 1
> TB, and you have 24 hrs to spend, you could check just that 1 TB
> filesysem with top (or something) running and see.

I did peak once via ssh while I was running fsck.ext3 (at that
time which was close to hour 20 ). As I recall I thought I would see
large numbers in the %cpu and %mem columns. I dont remember
exactly what I saw, but it was something like 10% cpu and 10% mem
but I can't swear. It certainly was not hogging more. I perhaps should
have done something with nice to make it more agressive...

Mitchell



>
>
>
> Doug.
>
>
> --
> To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-11-2008, 01:54 PM
"Douglas A. Tutty"
 
Default fsck.ext3 -yv /dev/sda1 on 1 terabyte partition

On Sat, May 10, 2008 at 10:37:37PM -0400, Mitchell Laks wrote:
> On 19:19 Sat 10 May , Douglas A. Tutty wrote:
> > On Fri, May 09, 2008 at 06:58:02PM -0400, Mitchell Laks wrote:
> > > On 07:24 Fri 09 May , NN_il_Confusionario wrote:
> > > > > * From: "Douglas A. Tutty" <dtutty@porchlight.ca>
> > > > >Does anyone know how much memory fsck needs to check a large filesystem?
>
> I had only 1G ram, but the raid was a 1T hardware raid.
>
> >
> > I don't suppose you know how much memory it used? If your / fs isn't 1
> > TB, and you have 24 hrs to spend, you could check just that 1 TB
> > filesysem with top (or something) running and see.
>
> I did peak once via ssh while I was running fsck.ext3 (at that
> time which was close to hour 20 ). As I recall I thought I would see
> large numbers in the %cpu and %mem columns. I dont remember
> exactly what I saw, but it was something like 10% cpu and 10% mem
> but I can't swear. It certainly was not hogging more. I perhaps should
> have done something with nice to make it more agressive...

Nice wouldn't have helped; it was probably waiting on IO.

Doug.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 06:26 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org