FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 07-13-2012, 08:37 PM
Jonathan Nieder
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

Martin-Éric Racine wrote:

> I have already stated that the .config is copied as-is from the stock
> Debian 3.2 kernel's config.

Oh, interesting.

May we have a debdiff of the binary packages, too?



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120713203758.GA31432@burratino
 
Old 07-13-2012, 08:46 PM
Ben Hutchings
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

On Fri, Jul 13, 2012 at 11:02:57PM +0300, Martin-Éric Racine wrote:
> (putting back the CC to the bug, which will probably need to be
> reassigned to 'linux')
> 2012/7/13 Jonathan Nieder <jrnieder@gmail.com>:
> > (dropping dpkg maintainers from cc)
> > Martin-Éric Racine wrote:
> >> 2012/7/13 Jonathan Nieder <jrnieder@gmail.com>:
> >>> Martin-Éric Racine wrote:
> >
> >>>> the resulting DEB
> >>>> is a whopping 488MB in size, compared to 22MB for the stock Debian
> >>>> linux-image-3.2.0-3-686-pae built using the exact same .config file.
> >>>
> >>> That's because the Debian packaging strips debugging symbols into a
> >>> separate -dbg package.
> >>
> >> I'm already aware of the effects of stripping binaries. However, you
> >> gotta admit that 488MB compared to 22MB is just ridiculous; something
> >> is definitely broken in those 3.2 build scripts.
> >
> > Are the build scripts responsible for the size of debugging symbols?
> > I don't follow.
> >
> > Or do you mean that the size of the kernel -dbg packages is
> > ridiculous?
>
> I mean that the size of the kernel package produced by 'make deb-pkg'
> from the 3.2 vanilla tree, even after disabling debug symbols, is
> highly suspicious, compared to the stock Debian kernel with the same
> source.

Did you actually rebuild with CONFIG_DEBUG_INFO off, or did you
only re-run 'make deb-pkg'?

Ben.

--
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
- Albert Camus



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120713204622.GQ1894@decadent.org.uk">http://lists.debian.org/20120713204622.GQ1894@decadent.org.uk
 
Old 07-14-2012, 04:51 AM
Martin-Éric Racine
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

2012/7/13 Ben Hutchings <ben@decadent.org.uk>:
> On Fri, Jul 13, 2012 at 11:02:57PM +0300, Martin-Éric Racine wrote:
>> (putting back the CC to the bug, which will probably need to be
>> reassigned to 'linux')
>> 2012/7/13 Jonathan Nieder <jrnieder@gmail.com>:
>> > (dropping dpkg maintainers from cc)
>> > Martin-Éric Racine wrote:
>> >> 2012/7/13 Jonathan Nieder <jrnieder@gmail.com>:
>> >>> Martin-Éric Racine wrote:
>> >
>> >>>> the resulting DEB
>> >>>> is a whopping 488MB in size, compared to 22MB for the stock Debian
>> >>>> linux-image-3.2.0-3-686-pae built using the exact same .config file.
>> >>>
>> >>> That's because the Debian packaging strips debugging symbols into a
>> >>> separate -dbg package.
>> >>
>> >> I'm already aware of the effects of stripping binaries. However, you
>> >> gotta admit that 488MB compared to 22MB is just ridiculous; something
>> >> is definitely broken in those 3.2 build scripts.
>> >
>> > Are the build scripts responsible for the size of debugging symbols?
>> > I don't follow.
>> >
>> > Or do you mean that the size of the kernel -dbg packages is
>> > ridiculous?
>>
>> I mean that the size of the kernel package produced by 'make deb-pkg'
>> from the 3.2 vanilla tree, even after disabling debug symbols, is
>> highly suspicious, compared to the stock Debian kernel with the same
>> source.
>
> Did you actually rebuild with CONFIG_DEBUG_INFO off, or did you
> only re-run 'make deb-pkg'?

The plot thickens:

If I disable CONFIG_DEBUG_INFO again just before building, the kernel
indeed is MUCH smaller (I had probably forgotten to disable it before
attempting a new build; sorry for the confusion) and it easily builds
within the space allocated for /tmp using sysfs via initscripts.
However, it still comes out at about 8MB larger than the equivalent
stock Debian kernel:

-rw-r--r-- 1 perkelix perkelix 22M Jun 28 15:03
linux-image-3.2.0-3-686-pae_3.2.21-3_i386.deb
-rw-r--r-- 1 perkelix perkelix 30M Jul 14 01:31
linux-image-3.2.21-vanilla-686-pae_3.2.21-vanilla-686-pae-1_i386.deb

This is of course a minor issue at this point and something that can
be discussed separately from this bug.

As for bug #617299, the actual cause is three-folds:

1) Starting with Wheezy, /tmp is mounted as a tmpfs whose size is
determined by initscripts to 20% of available memory. On my laptop
with a modest 1GB of RAM, this meant a /tmp sized at 200MB.

2) As it happens, 'dpkg-deb -b' uses /tmp (unless somewhere else is
specified using TMPDIR) as temporary space to compress packages while
building them.

3) Whenever building a kernel using 'make deb-pkg' without first
having disabled CONFIG_DEBUG_INFO, the resulting *.deb can easily near
500MB in size using the stock .config on x86. This obviously exceeds
the available /tmp space allocated as tmpfs via initscripts, which is
how gzip ends up outputing the above error message about "No space
left on device" or "File not found" despite plenty of hard-disk space
being available.

The solution to bug #617299 is therefore to either define TMPDIR to
some actual hard-disk with sufficient storage space via a commandline
environment e.g. 'TMPDIR=/var/tmp make deb-pkg' whenever using
commands that require an unusually large amount of /tmp space, or to
completely disable the usage of a tmpfs for /tmp via initscripts
defaults in /etc/defaults/rcS and /etc/defaults/tmpfs, as appropriate.

Martin-Éric



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CAPZXPQeEwHVNjSRJc_zO7i+EMh5LWZ7a=CW3HALs6PAHM98JG w@mail.gmail.com
 
Old 07-14-2012, 04:58 AM
Ben Hutchings
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

On Sat, 2012-07-14 at 07:51 +0300, Martin-Éric Racine wrote:
> 2012/7/13 Ben Hutchings <ben@decadent.org.uk>:
[...]
> If I disable CONFIG_DEBUG_INFO again just before building, the kernel
> indeed is MUCH smaller (I had probably forgotten to disable it before
> attempting a new build; sorry for the confusion) and it easily builds
> within the space allocated for /tmp using sysfs via initscripts.
> However, it still comes out at about 8MB larger than the equivalent
> stock Debian kernel:
>
> -rw-r--r-- 1 perkelix perkelix 22M Jun 28 15:03
> linux-image-3.2.0-3-686-pae_3.2.21-3_i386.deb
> -rw-r--r-- 1 perkelix perkelix 30M Jul 14 01:31
> linux-image-3.2.21-vanilla-686-pae_3.2.21-vanilla-686-pae-1_i386.deb

Official packages are now compressed with xz whereas deb-pkg uses gzip.

> This is of course a minor issue at this point and something that can
> be discussed separately from this bug.
>
> As for bug #617299, the actual cause is three-folds:
>
> 1) Starting with Wheezy, /tmp is mounted as a tmpfs whose size is
> determined by initscripts to 20% of available memory. On my laptop
> with a modest 1GB of RAM, this meant a /tmp sized at 200MB.
[...]

This was reverted in version of 2.88dsf-26 of initscripts, due to issues
like this. I'm not sure all systems will get automatically changed back
though.

Ben.

--
Ben Hutchings
The generation of random numbers is too important to be left to chance.
- Robert Coveyou
 
Old 07-14-2012, 05:09 AM
Jonathan Nieder
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

clone 617299 -1
retitle -1 kernel-handbook: encourage disabling DEBUG_INFO and setting TMPDIR
submitter -1 !
reassign -1 debian-kernel-handbook 1.0.13
quit

Martin-Éric Racine wrote:

> The solution to bug #617299 is therefore to either define TMPDIR to
> some actual hard-disk with sufficient storage space via a commandline
> environment e.g. 'TMPDIR=/var/tmp make deb-pkg' whenever using
> commands that require an unusually large amount of /tmp space, or to
> completely disable the usage of a tmpfs for /tmp via initscripts
> defaults in /etc/defaults/rcS and /etc/defaults/tmpfs, as appropriate.

Sort of. It's still a dpkg (wishlist?) bug. I would even think it
might be possible for dpkg-deb to write its output directly to the
expected target path or another file in the same directory.

Cloning as a reminder to recommend setting TMPDIR in the kernel
handbook.

Thanks,
Jonathan



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120714050923.GB3693@burratino
 
Old 07-14-2012, 05:24 AM
Martin-Éric Racine
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

(dropping debian-kernel from CC as we're back to discussing the dpkg
issue itself)
2012/7/14 Jonathan Nieder <jrnieder@gmail.com>:
> clone 617299 -1
> retitle -1 kernel-handbook: encourage disabling DEBUG_INFO and setting TMPDIR
> submitter -1 !
> reassign -1 debian-kernel-handbook 1.0.13
> quit
>
> Martin-Éric Racine wrote:
>
>> The solution to bug #617299 is therefore to either define TMPDIR to
>> some actual hard-disk with sufficient storage space via a commandline
>> environment e.g. 'TMPDIR=/var/tmp make deb-pkg' whenever using
>> commands that require an unusually large amount of /tmp space, or to
>> completely disable the usage of a tmpfs for /tmp via initscripts
>> defaults in /etc/defaults/rcS and /etc/defaults/tmpfs, as appropriate.
>
> Sort of. It's still a dpkg (wishlist?) bug. I would even think it
> might be possible for dpkg-deb to write its output directly to the
> expected target path or another file in the same directory.

Writing to the target path would indeed make sense. Just a wild guess,
but dpkg probably uses /tmp because, by definition, it's guaranteed to
be temporary and deleted upon reboot, whereas writing to target path
might leave an increasing amount of cruft whenever a build repeatedly
fails.

Still, as the original reporter complained, blindly trying to figure
out why a package repeatedly fails to build with "No space left" -
even though it compiled succesfully - on a hard-disk with several GB
of space, when it turns out that it's simply because dpkg uses as a
scratchpad a /tmp that happens to be mounted as a small tmpfs, is a
genuine problem. I am curious to hear what dpkg developers have to say
about solving this particular one.

Martin-Éric



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/CAPZXPQeh-3opLMM6oLTT.yLbsNWktwXAMSDkLd-aH2-o9jrg@mail.gmail.com
 
Old 07-14-2012, 05:26 AM
Jonathan Nieder
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

Martin-Éric Racine wrote:

> Still, as the original reporter complained, blindly trying to figure
> out why a package repeatedly fails to build with "No space left" -
> even though it compiled succesfully - on a hard-disk with several GB
> of space, when it turns out that it's simply because dpkg uses as a
> scratchpad a /tmp that happens to be mounted as a small tmpfs, is a
> genuine problem. I am curious to hear what dpkg developers have to say
> about solving this particular one.

I'm guessing they say "patches welcome"?


Jonathan

I also suggested earlier that output from dpkg-deb --debug would be
helpful in solving this, but no one took up the offer.



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120714052614.GC3693@burratino
 
Old 07-14-2012, 05:29 AM
Guillem Jover
 
Default Bug#617299: dpkg-deb: should give a hint when it fails due to filling /tmp

On Sat, 2012-07-14 at 00:09:23 -0500, Jonathan Nieder wrote:
> Martin-Éric Racine wrote:
> > The solution to bug #617299 is therefore to either define TMPDIR to
> > some actual hard-disk with sufficient storage space via a commandline
> > environment e.g. 'TMPDIR=/var/tmp make deb-pkg' whenever using
> > commands that require an unusually large amount of /tmp space, or to
> > completely disable the usage of a tmpfs for /tmp via initscripts
> > defaults in /etc/defaults/rcS and /etc/defaults/tmpfs, as appropriate.
>
> Sort of. It's still a dpkg (wishlist?) bug. I would even think it
> might be possible for dpkg-deb to write its output directly to the
> expected target path or another file in the same directory.

Yes, it's possible, and it's on my TODO list pending merging some ar
code refactoring. The code needs to rewind the file descriptor to write
out the ar header for the final tar member size. WHich will imply way
faster .deb generation as the output will be only written once instead
of copyig data over multiple times.

thanks,
guillem



--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120714052934.GA8289@gaara.hadrons.org">http://lists.debian.org/20120714052934.GA8289@gaara.hadrons.org
 

Thread Tools




All times are GMT. The time now is 08:03 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org