FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 03-02-2011, 03:02 PM
Marius Vollmer
 
Default Speeding up dpkg, a proposal

Hi,

I have recently been looking into where dpkg spends most of its time
when installing very many small packages, and came up with the following
idea to speed it up.

- Most of the time is spent writing files very carefully, a lot of them
in /var/lib/dpkg/updates.

- We can avoid this by writing the files less carefully (without fsync)
and even skipping the journal entries in /var/lib/dpkg/updates
completely.

- Instead, we move all packages that are to be unpacked into
half-installed / reinstreq before touching the first one, and put a
big sync() right before carefully writing /var/lib/dpkg/status.

[ There are more details to this than this, please check the code before
trying to find the holes in this short version of the idea.
]

This should be just as safe as writing very many small journal entries,
but if dpkg does get interrupted harshly, it leaves its database behind
in a correct but quite outdated and not so friendly state. Many
packages that have not been touched will have to be reinstalled because
dpkg can't be sure that they have in fact not been touched.

This should only happen when the system goes down abruptly without any
chance for dpkg to write a checkpoint and without unmounting the
filesystem cleanly. In any other case, such as a maintainer script
failing or the user interrupting dpkg with C-c, dpkg will write a
accurate checkpoint as the last thing it does.


I have experimental code for this here, based on dpkg 1.15.8.8:

http://meego.gitorious.org/~mvo/meego-platform-security/reckless-dpkg/commits/mvo/reckless

It shows a speed up between factor six and two in our environment (ext4
on a slowish flash drive) . I am not sure whether messing with the
fundamentals of dpkg is worth a factor of two in performance, but I
still think the idea is sound and worth sharing here, if only to be shot
down.

So, opinions?


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 8762s1a4u4.fsf@big.research.nokia.com">http://lists.debian.org/8762s1a4u4.fsf@big.research.nokia.com
 

Thread Tools




All times are GMT. The time now is 06:36 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org