Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Crash Utility (http://www.linux-archive.org/crash-utility/)
-   -   LZO Compression Support (http://www.linux-archive.org/crash-utility/636472-lzo-compression-support.html)

HATAYAMA Daisuke 01-01-1970 01:00 AM

LZO Compression Support
 
Hello Kumagai-san,

From: Atsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>
Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
Date: Fri, 23 Mar 2012 16:33:03 +0900

> On Thu, 23 Feb 2012 10:33:54 +0900
> HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:
>
>> The following series implements LZO compression support to
>> makedumpfile. LZO is as good as in size but by far better in speed
>> than ZLIB, readucing down time during generation of crash dump and
>> refiltering.
>
> Sorry for late reply.
>
> [PATCH v2 4/4] allows to use -l option even without USELZO but
> lzo compression doesn't work without USELZO.
> I think that warning messages should be displayed when -l option is
> used on makedumpfile built without USELZO.
>
> So I made v3 patches attached to this mail.
> ([PATCH v2 1/4] and [PATCH v2 2/4] look good and I didn't change them.)
>
> v2 => v3:
>
> - Add warning messages for invalid -l option.
> - Show LZO support status with help messages.
> - Add description of USELZO in README and man page.
> - (and I will do some cleanups.)
>
> What do you think, Hatayama-san ?

Thanks. I think the warning message very helpful for users.

BTW, I have a question about future build option of LZO library. You
said previously you're going to introduce autotools. Then, do you
consider default enable if LZO library is present on the environment?

Thanks.
HATAYAMA, Daisuke

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

HATAYAMA Daisuke 01-01-1970 01:00 AM

LZO Compression Support
 
Hello Kumagai-san,

From: Atsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>
Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
Date: Wed, 28 Mar 2012 14:18:09 +0900

> Hello Hatayama-san,
>
> On Fri, 23 Mar 2012 17:26:08 +0900 ( )
> HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:
>
>> BTW, I have a question about future build option of LZO library. You
>> said previously you're going to introduce autotools. Then, do you
>> consider default enable if LZO library is present on the environment?
>
> I'm afraid I decide not to introduce autotools for the following reasons.
>

It's OK if you don't have plan to use autotools.

> First, because autotools cannot know whether LZO library is prepared in
> 2nd kernel environment, it cannot decide whether it should link LZO library
> dynamically even if LINKTYPE=dynamic is specified.
>

Yes, I of course agree with the fact that makedumpfile for kdump 2nd
kernel should always be built in statically linked way.

But I think two problems are different, that is, the problem making
users easily able to choose build option by autotools, and the problem
guranteeing that the makedumpfile for kdump 2nd kernel be built always
statically.

> Second, if autotools behaves differently depending on LINKTYPE, it is difficult
> for users to understand.

If autotools were introduced, then I think we would no longer use
LINKTYPE and we would choose building option through autotools
features. For example, through new options in configure script such as
--enable-lzo-{static,dynamic} for example?

>
> So, I think that current method is simpler and better than autotools.
>

I agree. I think simpler one is better if it's enough for the purpose.

Thanks.
HATAYAMA, Daisuke

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

HATAYAMA Daisuke 01-01-1970 01:00 AM

LZO Compression Support
 
Hello,

From: Bouchard Louis <louis.bouchard@canonical.com>
Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
Date: Wed, 28 Mar 2012 09:59:04 +0200

> Hello,
>
> Le 28/03/2012 09:50, HATAYAMA Daisuke a écrit :
>>
>> Yes, I of course agree with the fact that makedumpfile for kdump 2nd
>> kernel should always be built in statically linked way.
>>
>> But I think two problems are different, that is, the problem making
>> users easily able to choose build option by autotools, and the problem
>> guranteeing that the makedumpfile for kdump 2nd kernel be built always
>> statically.
>>
>
> I don't see why it should be statically linked. Debian & Ubuntu both use
> a dynamically linked makedumpfile for the 2nd kdump kernel.
>
> Debian executes makedumpfile on the 'real' root FS while Ubuntu runs it
> in the initramfs. It is simply a chroot on the root FS being mounted on
> /root.
>

Thanks. I've noticed the fact on Debian for the first time.

Root filesystem can be broken due to filesystem related bugs or disk
crash. So, on fedora/RHEL environments, root filesystem is not mounted
in kdump initrd/initramfs. Necessary resources in 2nd kernel,
including makedumpfile, are all packed in kdump initrd/initramfs.

Thanks.
HATAYAMA, Daisuke


--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Atsushi Kumagai 02-23-2012 07:16 AM

LZO Compression Support
 
Hello Hatayama-san,

On Thu, 23 Feb 2012 10:33:54 +0900
HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:

> The following series implements LZO compression support to
> makedumpfile. LZO is as good as in size but by far better in speed
> than ZLIB, readucing down time during generation of crash dump and
> refiltering.
>
> The RFC discussion was made here:
>
> http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
> http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
>
> How to build:
>
> 1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of
> the following:
>
> 1) Original author's website:
> http://www.oberhumer.com/opensource/lzo/
>
> 2) yum framework on fedora. Older releases don't have the packages.
>
> 2. Apply the patch set to makedumpfile v1.4.2.
>
> 3. Do make as follows:
>
> $ make USELZO=on
>
> Note: In default, no LZO compression support is included.
>
> How to use:
>
> Introduce new -l option. If a user specify this, makedumpfile
> generates dumpfile compressed by pages with lzo compression.
>
> Example)
> $ makedumpfile -l vmcore dumpfile
>
> Performance evaluation:
>
> - Kumagai-san's evaluation simulating working servers:
> http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
>
> - My evaluation focusing on the worst cases:
> http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
>
> LZO Support for crash:
>
> I'll post LZO support patch for crash after makedumpfile merges
> these patches.
>
> Changelog:
>
> v1 => v2:
>
> - Add build condition for LZO support. Enable LZO support if
> specifying USELZO=on to make command.
>
> - Avoid LONG_MAX/ULONG_MAX redefinitions.

Thank you for your work.
I will review your patches for v1.4.4.

Thanks
Atsushi Kumagai

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Atsushi Kumagai 03-23-2012 06:33 AM

LZO Compression Support
 
Hello Hatayama-san,

On Thu, 23 Feb 2012 10:33:54 +0900
HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:

> The following series implements LZO compression support to
> makedumpfile. LZO is as good as in size but by far better in speed
> than ZLIB, readucing down time during generation of crash dump and
> refiltering.

Sorry for late reply.

[PATCH v2 4/4] allows to use -l option even without USELZO but
lzo compression doesn't work without USELZO.
I think that warning messages should be displayed when -l option is
used on makedumpfile built without USELZO.

So I made v3 patches attached to this mail.
([PATCH v2 1/4] and [PATCH v2 2/4] look good and I didn't change them.)

v2 => v3:

- Add warning messages for invalid -l option.
- Show LZO support status with help messages.
- Add description of USELZO in README and man page.
- (and I will do some cleanups.)

What do you think, Hatayama-san ?


Thanks
Atsushi Kumagai


> The RFC discussion was made here:
>
> http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
> http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
>
> How to build:
>
> 1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of
> the following:
>
> 1) Original author's website:
> http://www.oberhumer.com/opensource/lzo/
>
> 2) yum framework on fedora. Older releases don't have the packages.
>
> 2. Apply the patch set to makedumpfile v1.4.2.
>
> 3. Do make as follows:
>
> $ make USELZO=on
>
> Note: In default, no LZO compression support is included.
>
> How to use:
>
> Introduce new -l option. If a user specify this, makedumpfile
> generates dumpfile compressed by pages with lzo compression.
>
> Example)
> $ makedumpfile -l vmcore dumpfile
>
> Performance evaluation:
>
> - Kumagai-san's evaluation simulating working servers:
> http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
>
> - My evaluation focusing on the worst cases:
> http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
>
> LZO Support for crash:
>
> I'll post LZO support patch for crash after makedumpfile merges
> these patches.
>
> Changelog:
>
> v1 => v2:
>
> - Add build condition for LZO support. Enable LZO support if
> specifying USELZO=on to make command.
>
> - Avoid LONG_MAX/ULONG_MAX redefinitions.
>
> ---
>
> HATAYAMA Daisuke (4):
> Add build condition for LZO support
> Add help and manual messages about LZO compression support
> Avoid LONG_MAX/ULONG_MAX redefinitions
> Add LZO Support
>
>
> Makefile | 5 ++++
> common.h | 4 +++
> diskdump_mod.h | 3 ++-
> makedumpfile.8 | 6 +++--
> makedumpfile.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++------
> makedumpfile.h | 4 +++
> print_info.c | 16 +++++++------
> 7 files changed, 86 insertions(+), 19 deletions(-)
>
> --
> HATAYAMA Daisuke
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Atsushi Kumagai 03-28-2012 05:18 AM

LZO Compression Support
 
Hello Hatayama-san,

On Fri, 23 Mar 2012 17:26:08 +0900 ( )
HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:

> BTW, I have a question about future build option of LZO library. You
> said previously you're going to introduce autotools. Then, do you
> consider default enable if LZO library is present on the environment?

I'm afraid I decide not to introduce autotools for the following reasons.

First, because autotools cannot know whether LZO library is prepared in
2nd kernel environment, it cannot decide whether it should link LZO library
dynamically even if LINKTYPE=dynamic is specified.

Second, if autotools behaves differently depending on LINKTYPE, it is difficult
for users to understand.

So, I think that current method is simpler and better than autotools.


Thanks
Atsushi Kumagai

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Bouchard Louis 03-28-2012 07:59 AM

LZO Compression Support
 
Hello,

Le 28/03/2012 09:50, HATAYAMA Daisuke a écrit :
>
> Yes, I of course agree with the fact that makedumpfile for kdump 2nd
> kernel should always be built in statically linked way.
>
> But I think two problems are different, that is, the problem making
> users easily able to choose build option by autotools, and the problem
> guranteeing that the makedumpfile for kdump 2nd kernel be built always
> statically.
>

I don't see why it should be statically linked. Debian & Ubuntu both use
a dynamically linked makedumpfile for the 2nd kdump kernel.

Debian executes makedumpfile on the 'real' root FS while Ubuntu runs it
in the initramfs. It is simply a chroot on the root FS being mounted on
/root.

HTH,

...Louis
--
Louis Bouchard
Backline Support Analyst
Canonical Ltd
Ubuntu support: http://landscape.canonical.com

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

HATAYAMA Daisuke 05-10-2012 12:38 AM

LZO Compression Support
 
Helllo Kumagai-san,

From: Atsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>
Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
Date: Thu, 23 Feb 2012 17:16:46 +0900

> Thank you for your work.
> I will review your patches for v1.4.4.

Could you tell me when you plan to release version 1.4.4? Not
necessarily exact date, rough schedule is enough. I need to post the
corresponding patch to the crash utility's side after LZO feature is
merged into makedumpfile.

Thanks.
HATAYAMA, Daisuke

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Atsushi Kumagai 05-10-2012 03:40 AM

LZO Compression Support
 
Hello HATAYAMA-san,

On Thu, 10 May 2012 09:38:29 +0900 (JST)
HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> wrote:

> Helllo Kumagai-san,
>
> From: Atsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>
> Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
> Date: Thu, 23 Feb 2012 17:16:46 +0900
>
> > Thank you for your work.
> > I will review your patches for v1.4.4.
>
> Could you tell me when you plan to release version 1.4.4? Not
> necessarily exact date, rough schedule is enough. I need to post the
> corresponding patch to the crash utility's side after LZO feature is
> merged into makedumpfile.

At first, I planned to merge features below into v1.4.4.

- vmalloc translation support for PPC32 (by Suzuki K. Poulose)
- LZO Compression Support (by HATAYAMA Daisuke)
- sadump: check if given cpu is online in per-cpu related helper functions
(by HATAYAMA Daisuke)

However, the first one is being discussed and the schedule is not clear.
So, I would like to postpone the feature for v1.4.5 and start testing
the other two for v1.4.4.

If I can start testing now, I will release v1.4.4 in this month.


Thanks
Atsushi Kumagai

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

HATAYAMA Daisuke 05-10-2012 04:40 AM

LZO Compression Support
 
From: Atsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>
Subject: Re: [PATCH makedumpfile v2 0/4] LZO Compression Support
Date: Thu, 10 May 2012 12:40:27 +0900

>
> At first, I planned to merge features below into v1.4.4.
>
> - vmalloc translation support for PPC32 (by Suzuki K. Poulose)
> - LZO Compression Support (by HATAYAMA Daisuke)
> - sadump: check if given cpu is online in per-cpu related helper functions
> (by HATAYAMA Daisuke)
>
> However, the first one is being discussed and the schedule is not clear.
> So, I would like to postpone the feature for v1.4.5 and start testing
> the other two for v1.4.4.
>
> If I can start testing now, I will release v1.4.4 in this month.
>

Thanks. So the release would be before the release of crash utility's
next version. I'll post the patch for crash utility soon.

Thanks.
HATAYAMA, Daisuke

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility


All times are GMT. The time now is 04:55 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.