FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > EXT3 Users

 
 
LinkBack Thread Tools
 
Old 02-25-2009, 04:42 PM
Ralf Hildebrandt
 
Default Questions regarding journal replay

* Ralf Hildebrandt <Ralf.Hildebrandt@charite.de>:

> After my initial report I removed the journal and created an external
> journal on a 32GB partition. Hoping it would be faster, since
> accoriding to the docs. the journal size is limited to 128MB.

That should read:

Hoping it would be faster, since -- according to the docs -- the journal
size is limited to 128MB.

--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Geschäftsbereich IT | Abt. Netzwerk Fax. +49 (0)30-450 570-962
Hindenburgdamm 30 | 12200 Berlin

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 04:44 PM
Eric Sandeen
 
Default Questions regarding journal replay

Ralf Hildebrandt wrote:
> * Theodore Tso <tytso@mit.edu>:
>> On Wed, Feb 25, 2009 at 10:31:42AM -0600, Eric Sandeen wrote:
>>> It'd be better to get to the bottom of the problem ... maybe iostat
>>> while it's happening to see if IO is actually happening; run blktrace to
>>> see where IO is going, do a few sysrq-t's to see where threads are at, etc.
>>>
>>> Can you find a way to reproduce this at will?
>>>
>>> Journal replay should *never* take this long, AFAIK.
>> Indeed. The journal is 128 megs, as I recall. So even if the journal
>> was completely full, if it's taking 800 seconds, that's a write rate
>> of 0.16 Mb/S (164 kb/second). That is indeed way too slow.
>
> The problem seems to be with the external journal which I recently
> changed to. It's a 32GB partition. My timings seem to indicate that
> ALL OF IT was being replayed
>

But you also saw this with an internal journal?

Perhaps you have uncovered 2 bugs ...

TBH external journals probably aren't tested that much (though they
certainly should work)

I'll give it a quick sanity test on ext4.

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 05:11 PM
Ralf Hildebrandt
 
Default Questions regarding journal replay

* Eric Sandeen <sandeen@redhat.com>:

> > The problem seems to be with the external journal which I recently
> > changed to. It's a 32GB partition. My timings seem to indicate that
> > ALL OF IT was being replayed
> >
>
> But you also saw this with an internal journal?

Yes.

> Perhaps you have uncovered 2 bugs ...
>
> TBH external journals probably aren't tested that much (though they
> certainly should work)
>
> I'll give it a quick sanity test on ext4.

They DO work, but apparently the docs are wrong! I mean, no sane
person needs 32GB of journal

--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Geschäftsbereich IT | Abt. Netzwerk Fax. +49 (0)30-450 570-962
Hindenburgdamm 30 | 12200 Berlin

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 05:44 PM
Theodore Tso
 
Default Questions regarding journal replay

On Wed, Feb 25, 2009 at 06:42:14PM +0100, Ralf Hildebrandt wrote:
> * Ralf Hildebrandt <Ralf.Hildebrandt@charite.de>:
>
> > After my initial report I removed the journal and created an external
> > journal on a 32GB partition. Hoping it would be faster, since
> > accoriding to the docs. the journal size is limited to 128MB.
>
> That should read:
>
> Hoping it would be faster, since -- according to the docs -- the journal
> size is limited to 128MB.
>

Increasing the journal size may speed up certain filesystem workloads
which are causing the journal to wrap very frequently. However,
increasing the journal *will* increase the time to replay the journal....

How long did the journal replay take when you were using the 128MB
internal inode? Was the 800 seconds to replay for the the case when
you were using the internal journal or the external journal?

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 05:46 PM
Theodore Tso
 
Default Questions regarding journal replay

On Wed, Feb 25, 2009 at 06:39:07PM +0100, Ralf Hildebrandt wrote:
>
> The RAID is made up from one logical volume, consisting of two drives
> sda and sdb, each containing 6 disks in a hardware RAID5 setup.

Do you know what the chunk size or strip size is for your hardware
RAID5?

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 05:52 PM
Theodore Tso
 
Default Questions regarding journal replay

On Wed, Feb 25, 2009 at 07:11:08PM +0100, Ralf Hildebrandt wrote:
> > TBH external journals probably aren't tested that much (though they
> > certainly should work)
> >
> > I'll give it a quick sanity test on ext4.
>
> They DO work, but apparently the docs are wrong! I mean, no sane
> person needs 32GB of journal

The docs don't warn against needing that large of a journal, yes. One
of the things which never got finished (although it was in the
original design of the jbd layer) was the ability to share the journal
across multiple filesystems. This would mean that it might more sense
to have a single large journal. Probably not 32GB in size, though.

Did you find some documentation that actually recommend that large of
an external journal?

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 06:01 PM
Ralf Hildebrandt
 
Default Questions regarding journal replay

* Theodore Tso <tytso@mit.edu>:

> Increasing the journal size may speed up certain filesystem workloads
> which are causing the journal to wrap very frequently. However,
> increasing the journal *will* increase the time to replay the journal....

Indeed. This is a Maildir-style mailbox server. Many small writes,
reads and deletes.

> How long did the journal replay take when you were using the 128MB
> internal inode?

800s

--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Geschäftsbereich IT | Abt. Netzwerk Fax. +49 (0)30-450 570-962
Hindenburgdamm 30 | 12200 Berlin

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 06:02 PM
Ralf Hildebrandt
 
Default Questions regarding journal replay

* Theodore Tso <tytso@mit.edu>:

> The docs don't warn against needing that large of a journal, yes. One
> of the things which never got finished (although it was in the
> original design of the jbd layer) was the ability to share the journal
> across multiple filesystems. This would mean that it might more sense
> to have a single large journal. Probably not 32GB in size, though.
>
> Did you find some documentation that actually recommend that large of
> an external journal?

It says the "journal has a maximum size of 128M"

--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Geschäftsbereich IT | Abt. Netzwerk Fax. +49 (0)30-450 570-962
Hindenburgdamm 30 | 12200 Berlin

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 06:28 PM
Eric Sandeen
 
Default Questions regarding journal replay

Ralf Hildebrandt wrote:
> * Eric Sandeen <sandeen@redhat.com>:
>
>>>> It'd be better to get to the bottom of the problem ... maybe iostat
>>>> while it's happening to see if IO is actually happening; run blktrace to
>>>> see where IO is going, do a few sysrq-t's to see where threads are at, etc.
>>> We had 24GB of reading from the journal device (or 12GB if it's
>>> 512byte blocks). I wonder why?
>> 24GB of reading from the journal device (during that 800s of replay
>> during mount?), and your journal is 128M ... well that's odd.
>
> After my initial report I removed the journal and created an external
> journal on a 32GB partition. Hoping it would be faster, since
> accoriding to the docs. the journal size is limited to 128MB.
>
>> You say journal device; is this an external journal? I didn't think so
>> from your first email, but is it?
>
> It is now.

...

> Journal block size: 4096
> Journal length: 8488436
> Journal first block: 2
> Journal sequence: 0x0027c611
> Journal start: 2
> Journal number of users: 1
> Journal users: 032613d3-6035-4872-bc0a-11db92feec5e

Ok we might be getting a little off-track here. Your journal is indeed
32G in size. But you also saw this with an internal journal, which
should be limited to 128M, and yet you still saw a very long replay, right?

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 02-25-2009, 06:31 PM
Ralf Hildebrandt
 
Default Questions regarding journal replay

* Eric Sandeen <sandeen@redhat.com>:

> > Journal block size: 4096
> > Journal length: 8488436
> > Journal first block: 2
> > Journal sequence: 0x0027c611
> > Journal start: 2
> > Journal number of users: 1
> > Journal users: 032613d3-6035-4872-bc0a-11db92feec5e
>
> Ok we might be getting a little off-track here. Your journal is indeed
> 32G in size. But you also saw this with an internal journal, which
> should be limited to 128M, and yet you still saw a very long replay, right?

800s for 128M, yes
--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Geschäftsbereich IT | Abt. Netzwerk Fax. +49 (0)30-450 570-962
Hindenburgdamm 30 | 12200 Berlin

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 

Thread Tools




All times are GMT. The time now is 11:36 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org