FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Development

 
 
LinkBack Thread Tools
 
Old 01-14-2009, 01:53 PM
Steven Harms
 
Default Deb binary diffs?

I noticed that with Fedora 11, they want to enable Presto / Yum by
default so that only the deltas of binary files are downloaded by
default. I believe zypper also downloads just the delta rpms for
their updates also. Is there anything planned for Jaunty that will
implement a similar feature?

I found https://wiki.ubuntu.com/APTPackageDeltas but I didn't see if
this has been implemented or will be.

Thanks,

Steven Harms

--
GPG Key ID: C92EF367 / 1428 FE8E 1E07 DDA8 EFD7 195F DCCD F5B3 C92E F367

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 01-16-2009, 10:00 PM
Justin Dugger
 
Default Deb binary diffs?

As noted in the discussion, Debian packages tend to have terrible
deltas. One change can propagate a long ways in gzip. The Rsync
author proposed a fix in like 2000 to reset the window regularly, to
limit the scope of changes at a price of a few percent compression.
Ultimately, rsync is a CPU intensive process on what I'm told is an
already overloaded server. xdelta without gzip hacks would effectively
double the size of the archive needed to support this proposal.

An interesting solution is zsync, which uses standard HTTP and does
all processing locally. But even here, we need packages to be just a
bit different than it is, or you wind up paying the rsync CPU overhead
just to download the whole archive. Zsync is smart enough to look
inside compressed archives, but doesn't know about the .deb format.
Using the command option --rsyncable should be a temporary workaround,
and the consequences are outlined in the gzip manpage.

Some initial work and analysis is already in the wiki.
https://wiki.ubuntu.com/apt-sync . I too would be interested to hear
anything new on the subject.

Justin Dugger

On Wed, Jan 14, 2009 at 8:53 AM, Steven Harms <thisdyingdream@gmail.com> wrote:
> I noticed that with Fedora 11, they want to enable Presto / Yum by
> default so that only the deltas of binary files are downloaded by
> default. I believe zypper also downloads just the delta rpms for
> their updates also. Is there anything planned for Jaunty that will
> implement a similar feature?
>
> I found https://wiki.ubuntu.com/APTPackageDeltas but I didn't see if
> this has been implemented or will be.
>
> Thanks,
>
> Steven Harms
>
> --
> GPG Key ID: C92EF367 / 1428 FE8E 1E07 DDA8 EFD7 195F DCCD F5B3 C92E F367
>
> --
> ubuntu-devel mailing list
> ubuntu-devel@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 01-19-2009, 04:22 PM
Matt Zimmerman
 
Default Deb binary diffs?

On Wed, Jan 14, 2009 at 09:53:06AM -0500, Steven Harms wrote:
> I noticed that with Fedora 11, they want to enable Presto / Yum by
> default so that only the deltas of binary files are downloaded by
> default. I believe zypper also downloads just the delta rpms for
> their updates also. Is there anything planned for Jaunty that will
> implement a similar feature?
>
> I found https://wiki.ubuntu.com/APTPackageDeltas but I didn't see if
> this has been implemented or will be.

It's something which has been talked about and even largely specified, but I
don't think there's much momentum behind it at the moment, and it's quite
late to start on such a change to critical infrastructure for Jaunty. If
someone wants to pick it up and run with it (including working with the
Launchpad developers on their end of it), I think it could be done for
Jaunty+1.

--
- mdz

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 01-24-2009, 02:11 PM
Paul Sladen
 
Default Deb binary diffs?

On Fri, 16 Jan 2009, Justin Dugger wrote:
> https://wiki.ubuntu.com/apt-sync .

IIRC, it's actually futher along from than that; the code exists, but is
hackish; alot of the hackish is involved in the expectation that we repack
the .deb (when dpkg will then immediately unpack it again)

Some further research/optimisation needed doing on a machine with a localish
full mirror (but people. was/is Canonical only(tm)[1]). mvo had a test
setup in his home folder, but I'm not/was not in a position to tinker.

To make things easier for an apt-sync, here's what needs doing:

1. dpkg to allow 'data.tar'[0]
2. hash of non-packed .deb in 'Packages.
(Or rather, sign the data, not the result of the encoding)

Then it needs integrating with the build process (checksum building), for
Debian we have source; for Ubuntu that would have to be a Canonical employee.

Once done, for further optimisation (over download size):
1. Debian/Ubuntu spec to allow any deflate packer (not just'gzip -9')
2. Guide compressor using previous-version deflate tables/decisions
3. Guide linker using previous-version linking decisions
4. Guide Compiler using previous-version compiler decisions

Being on a boat and stuck behind GPRS, I'm actually a person that would gain
from this massively at the moment.

There's people pulling in different directions:

(a) CD-size (see Jani's message this week)
(b) Mirror footprint
(c) Download size
(d) Upgrade size
(e) Buildd load
(g) Mirror CPU
(f) Local load

Pick one, maybe two. Canvas people. You can't have (b) or (g).

-Paul

[0] dpkg already allows non-packed 'control.tar', but not 'data.tar'
[1] I did ask at the time.
--
Why do one side of a triangle when you can do all three. Somewhere, GB.


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 

Thread Tools




All times are GMT. The time now is 04:13 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org