Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian Development (http://www.linux-archive.org/debian-development/)
-   -   Enabling uupdate to simply remove files from upstream source (http://www.linux-archive.org/debian-development/695132-enabling-uupdate-simply-remove-files-upstream-source.html)

Tollef Fog Heen 08-17-2012 07:41 PM

Enabling uupdate to simply remove files from upstream source
 
]] Pau Garcia i Quiles

> It's not a matter of using a better or worse script, having dpkg, or
> uupdate, or some other tool do that for us. It's a matter of
> repackaging. It's a matter of altering upstream's package for (IMHO)
> no good reason. It's a matter of breaking user confidence: users are
> no longer able to compare Debian's .orig.tar.gz with upstream's
> tarball, or with Gentoo's traball, or with Fedora's tarball, etc just
> because we are shipping a different source tarball.

It's customary to add +dfsg or similar when repackaging, so it's pretty
clearly marked that it's not the exact same tarball as upstream's
released and I believe user confusion is small.

Also, I question whether people just check fingerprints instead of
actually diffing the contents of the tarballs themselves.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87ipch71cw.fsf@xoog.err.no">http://lists.debian.org/87ipch71cw.fsf@xoog.err.no

gregor herrmann 08-17-2012 11:01 PM

Enabling uupdate to simply remove files from upstream source
 
On Fri, 17 Aug 2012 14:12:49 +0200, Andreas Tille wrote:

> > > So we finally have three independently developed solutions (we also have
> > > several instances of a debian/get-orig-source script in Debian Med
> > > team) and
> > pkg-perl variant:
> > http://anonscm.debian.org/gitweb/?p=pkg-perl/scripts.git;a=tree
> > repack.sh and repack.stub there
> Solution 4. ;-)

:)

BTW, there's even documentation on how to use it (and what else is
needed):
http://pkg-perl.alioth.debian.org/howto/repacking.html

> Right that's correct (and explains Daniel's hint there is no need for
> uupdate).
>
> 1. I propose a file debian/uscan.remove which contains regular expressions.
> The deletion process will loop over every line and does
>
> rm -rf ${MAIN_SOURCE_DIR}/<expression>

I also like Jonas' idea of using d/copyright.

> 2. uscan just knows the option --repack. Do we want to reuse this and start
> deleting files in case the file debian/uscan.remove exists or do we
> want to use an explicite option --repack-and-remove? I have some
> preference for just using --repack.

Hm, not sure if changing the semantics of an existing option is a
good idea.

What I like about our current way is that I -- once d/watch is
adjusted -- I don't have to do anything besides calling uscan.

> 4. In case something was removed the version string will be appended by
> '~dfsg' to express the fact that the content of the original source
> was changed.

+dfsg

What I also like about our current scripts is
- that the suffix is configurable if needed
- that it manipulates also MANIFEST (ok, that's perl-specific)


Cheers,
gregor

--
.'`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
: :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
`. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
`- NP: The Dubliners: Champion at keeping them rolling

gregor herrmann 08-18-2012 09:41 PM

Enabling uupdate to simply remove files from upstream source
 
On Sat, 18 Aug 2012 22:19:22 +0200, Andreas Tille wrote:

> 2. There was one vote from Gregor Hermann to use the --repack option
> of uscan. I personally admit that I do not fully agree with
> Gregor that this means changing the semantics of an existing option.
> We are just repackaging and without the additional information in
> debian/copyright the functionality remains exactly the same.

Sorry if I was unclear here: actually I don't really like to re-use
--repack because the "new" --repack would do something else (i.e:
more) than the "old" repack.

> Gregor made also the remark that he likes the pkg-perl teams
> method of doing things because once d/watch is adjusted you don't
> have to do anything besides calling uscan. I consider the method
> I like to suggest here as serving the very same purpose

Sure, I guess I will get used to adding "--$option" instead of just
using plain uscan, if needed :)


Cheers,
gregor

--
.'`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
: :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
`. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
`- NP: David Bowie: Starman

Tollef Fog Heen 08-19-2012 08:19 AM

Enabling uupdate to simply remove files from upstream source
 
]] Andreas Tille

> Hi,
>
> trying to summarise suggested changes for the proposal:
>
> 1. The new field Files-Excluded in debian/copyright contains
> a space separated list of regular expressions.

Why regexes rather than globs, which is used elsewhere in the file?

Also, your example uses globs, AFAICS.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87boi75m67.fsf@xoog.err.no">http://lists.debian.org/87boi75m67.fsf@xoog.err.no

Ansgar Burchardt 08-19-2012 09:55 AM

Enabling uupdate to simply remove files from upstream source
 
gregor herrmann <gregoa@debian.org> writes:
> On Fri, 17 Aug 2012 13:08:39 +0200, Andreas Tille wrote:
>> my suggestion was just to settle with a common and simple
>> solution. This should be pretty simple to implement (I'd volunteer to
>> do this but wanted to seek for comments before filing a bug report +
>> patch).
>
> Yup, that would be nice.
> But I don't think it belongs into uupdate but in uscan.

Please keep it usable for packages that do not use uscan to get the
upstream tarball. I have some packages where I generate the upstream
tarball from a VCS repository, but have to exclude some files. It would
be nice to use the tool for such cases as well.

Ansgar


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87mx1rfbom.fsf@marvin.43-1.org">http://lists.debian.org/87mx1rfbom.fsf@marvin.43-1.org

Andreas Tille 08-19-2012 10:09 AM

Enabling uupdate to simply remove files from upstream source
 
On Sat, Aug 18, 2012 at 11:41:24PM +0200, gregor herrmann wrote:
> > 2. There was one vote from Gregor Hermann to use the --repack option
> > of uscan. I personally admit that I do not fully agree with
> > Gregor that this means changing the semantics of an existing option.
> > We are just repackaging and without the additional information in
> > debian/copyright the functionality remains exactly the same.
>
> Sorry if I was unclear here: actually I don't really like to re-use
> --repack because the "new" --repack would do something else (i.e:
> more) than the "old" repack.
>
> > Gregor made also the remark that he likes the pkg-perl teams
> > method of doing things because once d/watch is adjusted you don't
> > have to do anything besides calling uscan. I consider the method
> > I like to suggest here as serving the very same purpose
>
> Sure, I guess I will get used to adding "--$option" instead of just
> using plain uscan, if needed :)

When reading this I wonder whether we actually will need any command
line option at all? Shouldn't it rather be the other way around that we
want to *prevent* repackaging in some cases if Files-Excluded is set and
otherwise just to the repackaging as a default. In other words: If it
is documented in debian/copyright that some files will be excluded the
proper way to act for uscan would be actually to exclude the files. We
rather would need an option

uscan --no-exclusion (or --fetch-original ???)

to prevent uscan from doing so. BTW, we also would need an according
USCAN_NO_EXCLUSION (or whatever name we decide) for /etc/devscripts.conf
and ~/.devscripts.

Kind regards

Andreas.


--
http://fam-tille.de


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120819100959.GB12722@an3as.eu">http://lists.debian.org/20120819100959.GB12722@an3as.eu

Andreas Tille 08-19-2012 10:25 AM

Enabling uupdate to simply remove files from upstream source
 
Hi,

On Sun, Aug 19, 2012 at 11:55:37AM +0200, Ansgar Burchardt wrote:
> > Yup, that would be nice.
> > But I don't think it belongs into uupdate but in uscan.
>
> Please keep it usable for packages that do not use uscan to get the
> upstream tarball. I have some packages where I generate the upstream
> tarball from a VCS repository, but have to exclude some files. It would
> be nice to use the tool for such cases as well.

Wouldn't it make way more sense to teach uscan to read a VCS repository?
I admit I would not volunteer for this coding because it seems more
complex to me, but IMHO this would be the proper way to handle this.

For the time beeing to help you with this we should probably come back
to the grep-dctrl suggestion from Jonas and write a simple shell script
that takes a directory as input and removes files by using find as
discussed above. Uscan could call this script and you could do so as
well.

What do you think?

Kind regards

Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120819102535.GD12722@an3as.eu">http://lists.debian.org/20120819102535.GD12722@an3as.eu

Russ Allbery 08-19-2012 05:35 PM

Enabling uupdate to simply remove files from upstream source
 
Ansgar Burchardt <ansgar@debian.org> writes:

> Please keep it usable for packages that do not use uscan to get the
> upstream tarball. I have some packages where I generate the upstream
> tarball from a VCS repository, but have to exclude some files. It would
> be nice to use the tool for such cases as well.

Teaching uscan how to do things like git archive might be very useful.
One of my packages currently has the following, which I bet could be
replaced by a sufficiently Git-aware uscan and a bit of glue for finding
the upstream Git repository and understanding the upstream tag format
(perhaps in debian/watch?).

# These variables are used by get-orig-source, to construct dkms.conf, and
# to set the build version. You will need to change TAG to package stable
# releases instead of experimental releases.
DEBIAN := $(shell dpkg-parsechangelog | grep ^Version: | cut -d' ' -f2)
DEBVERS := $(shell echo '$(DEBIAN)' | cut -d- -f1)
VERSION := $(shell echo '$(DEBVERS)' | sed -e 's/[+-].*//' -e 's/~//g')
TAG := $(shell echo 'openafs-stable-$(VERSION)' | sed 's/./_/g')
REPO := git://git.openafs.org/openafs.git

# Upstream does tarball releases for major releases, but not for point
# relesaes, and the tarball releases are split into src and doc and contain
# the WINNT directory. Dropping WINNT, which is not used on Debian, saves a
# substantial amount of space in the source package, and there's no reason
# to include the files generated by regen.sh when we're going to run it
# again ourselves anyway.
#
# This rule therefore generates an upstream tarball from the upstream Git
# tag, rather than the tarball release, without the generated files that are
# not in Git and without the WINNT directory. It assumes that git-core is
# installed and there's network connectivity to the upstream repository.
get-orig-source:
git archive --remote='$(REPO)' --prefix='openafs_$(DEBVERS).orig/'
--format=tar '$(TAG)' | tar xf -
rm -r openafs_$(DEBVERS).orig/src/WINNT
tar cf openafs_$(DEBVERS).orig.tar openafs_$(DEBVERS).orig
rm -r openafs_$(DEBVERS).orig
gzip -9 openafs_$(DEBVERS).orig.tar

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87ipceeqei.fsf@windlord.stanford.edu">http://lists.debian.org/87ipceeqei.fsf@windlord.stanford.edu

gregor herrmann 08-19-2012 10:10 PM

Enabling uupdate to simply remove files from upstream source
 
On Sun, 19 Aug 2012 12:19:57 +0200, Andreas Tille wrote:

> > Suffix should be configurable.

Ack.

> > I use ~dfsg by default, ~dfsg1 and bumping numbers for multiple
> > repackagings, and only +dfsg when the repackaging happens after a
> > non-repackaged version was released into Debian.
> >
> > Reason for this is that there is a slight chance upstream may re-release
> > same upstream version repackaged to fix a purely tarball-related issuem
> > and I would then have room for using that proper version instead of
> > using epoch or add a bogus .0 to the version.
>
> This was also my initial idea when firt proposing ~dfsg. On the other
> hand: I would *really* want to have upstream adding a new version number
> to the cleaned up release. It is just (uhmm, find your own word here)
> if people release the same named file with different content. So I do
> not see great harm if we would settle with +dfsg. Gregor, could you give
> better reasons than Jonas for +dfsg?

Well, I see Jonas' point but I haven't encountered it yet in my
experience; and often repackaging happens after detecticting that
it's needed, in which case +dfsg seems more logical.

> > > use Debian::Copyright;
> > That initial test by Gregor makes me worry if Debian::Copyright parser
> > might be too strict: Writing should be strict but parsing relaxed -
> > Copyright file format with undefined fields added should *not* be
> > treated as broken. Perhaps there are other surprises waiting to happen
> > :-/

Yup, I was just the first that came to my mind.

> Could anybody say something about this?

Next guess:

Dpkg::Control::Hash - parse and manipulate a block of RFC822-like fields
(libdpkg-perl)

Let's try:

in d/copyright:

Files-Excluded: doc/a src/b
bin/c

test script:

#v+
#!/usr/bin/perl

use strict;
use warnings;

use Dpkg::Control::Hash;

my $c = Dpkg::Control::Hash->new();
$c->load('debian/copyright');

my @excluded_files = split /s/, $c->{'Files-Excluded'};
print "@excluded_files
";
#v-

Output:

doc/a src/b bin/c


Cheers,
gregor

--
.'`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
: :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
`. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
`- NP: Vic Chesnutt: Thailand

gregor herrmann 08-19-2012 10:11 PM

Enabling uupdate to simply remove files from upstream source
 
On Sun, 19 Aug 2012 12:09:59 +0200, Andreas Tille wrote:

> > Sure, I guess I will get used to adding "--$option" instead of just
> > using plain uscan, if needed :)
> When reading this I wonder whether we actually will need any command
> line option at all? Shouldn't it rather be the other way around that we
> want to *prevent* repackaging in some cases if Files-Excluded is set and
> otherwise just to the repackaging as a default. In other words: If it
> is documented in debian/copyright that some files will be excluded the
> proper way to act for uscan would be actually to exclude the files.

Yup, that would at least be more convenient.


Cheers,
gregor

--
.'`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
: :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
`. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
`- NP: Tom Waits: Lullaby


All times are GMT. The time now is 01:02 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.