Distfile Patching Support for Portage
 

Hi,

I'm sending attached a patch that does the basic implementation of
Distfile Patching Support for Portage. This is part of my GSoC project
[1].

It relies on the tools provided by app-portage/distpatch, commited to
the tree today, and is able to get an old tarball, plus a XZ
compressed binary delta and reconstruct the needed tarball, if a delta
is available.

You may find a few sample deltas and a delta.db in my public page in
soc.dev.gentoo.org [2]. More deltas will be available soon. These ones
are just for tests.

Basic steps to test: (be careful with the commands! :)

# echo 'DELTAS_ROOT_URL="http://soc.dev.gentoo.org/~rafaelmartins/"'
>> /etc/make.conf
# echo 'FEATURES="${FEATURES} distpatch"' >> /etc/make.conf
# wget -O $(portageq distdir)/delta.db
http://soc.dev.gentoo.org/~rafaelmartins/delta.db
# emerge --fetchonly =grep-2.8
# rm $(portageq distdir)/grep-2.9.tar.xz
# emerge -av1 =grep-2.9

You'll see that the fetch size reported is pretty smaller than the
full tarball (80kB X >1MB). If you hit return, a delta will be fetched
and saved to $DISTDIR/deltas, the tarball will be reconstructed, the
checksums will be verified and the package will be installed. The
reconstructed tarball will be saved to $DISTDIR, if the checksums
match with the original tarball, ofrto $DISTDIR/delta-reconstructed,
if the checksums from compressed files unmatched but the checksums for
the uncompressed files matched. This is handled in a secure way by the
distpatch tools and the delta.db file.

Currently the delta.db file is fetched manually and placed in $DISTDIR
because we don't decided yet about how it will be shipped to users.
Probably through rsync, but we can't say this before have a "real"
deployment of the delta generators. This will require a little patch
soon.

This patch isn't obtrusive, and shouldn't affect any users with
FEATURES="-distpatch".

All the variable names can be changed, if wanted. I'm not very good at
choose names :)

That's it.

[1] - http://www.gentoo.org/proj/en/infrastructure/distpatch/
[2] - http://soc.dev.gentoo.org/~rafaelmartins/

Regards,

--
Rafael Goncalves Martins
Gentoo Linux developer
http://rafaelmartins.eng.br/

Distfile Patching Support for Portage
 

2011-08-11 10:12:04 Rafael Goncalves Martins napisał(a):
> + except subprocess.CalledProcessError, e:

Please test your patch with all Python versions supported by Portage (at least using runtests.sh).

--
Arfrever Frehtes Taifersar Arahesis

Distfile Patching Support for Portage
 

On Thu, Aug 11, 2011 at 12:29 PM, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta@gmail.com> wrote:
> 2011-08-11 10:12:04 Rafael Goncalves Martins napisał(a):
>> + Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* except subprocess.CalledProcessError, e:
>
> Please test your patch with all Python versions supported by Portage (at least using runtests.sh).
>

Fixed. Tests ok.

New patch attached.

Thanks.

--
Rafael Goncalves Martins
Gentoo Linux developer
http://rafaelmartins.eng.br/

Distfile Patching Support for Portage
 

Patch reworked, after Zac's feedback on #-portage.

Attached.

--
Rafael Goncalves Martins
Gentoo Linux developer
http://rafaelmartins.eng.br/

Distfile Patching Support for Portage
 

On Sun, Aug 21, 2011 at 3:42 PM, Rafael Goncalves Martins
<rafaelmartins@gentoo.org> wrote:
> Patch reworked, after Zac's feedback on #-portage.
>
> Attached.
>

Re-sending. I forgot a print statement in the code. :(

Sorry.

--
Rafael Goncalves Martins
Gentoo Linux developer
http://rafaelmartins.eng.br/