FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 03-07-2009, 06:28 PM
Steve Langasek
 
Default Bug#466550: Pristine source from upstream VCS repository

On Fri, Mar 06, 2009 at 11:03:57AM +1100, Ben Finney wrote:

> === modified file 'policy.sgml'
> --- policy.sgml 2009-03-05 08:44:48 +0000
> +++ policy.sgml 2009-03-05 23:59:38 +0000
> @@ -1907,12 +1907,21 @@
> <tag><tt>get-orig-source</tt> (optional)</tag>
> <item>
> <p>
> - This target fetches the most recent version of the
> - original source package from a canonical archive site
> - (via FTP or WWW, for example), does any necessary
> + This target generates the original source archive for
> + the package, such that its contents exactly match the
> + original source archive used to generate the package
> + for Debian. See the “Original source archive”
> + section, below, for policy details of this file.
> + </p>

Surely, given that this is sgml, you should be referencing "Original source
archive" by id instead of "see below"?

> +
> + <p>
> + The actions for this target fetch the original source
> + package, corresponding to the Debian package version,

This reads wrong. What are "actions for this target"? Should be "This
target fetches [...]" as in the original, or "The target fetches [...]" if
you're trying to avoid repetition.

> <p>
> @@ -1922,8 +1931,14 @@
> </p>
>
> <p>
> - This target is optional, but providing it if
> - possible is a good idea.
> + This target is <em>optional</em>. A common reason to
> + forego this target is that the upstream developers
> + make canonical original source archive files for
> + specific versions available for direct public
> + download; in these cases, the package only needs an
> + appropriate <file>debian/watch</file> configuration
> + for <prgn>uscan</prgn> to fetch the original source
> + archive.
> </p>
> </item>

Why is 'optional' emphasized?

I don't like the speculation about developers' reasons for forgoing (not
'foregoing', btw) the target. I think this is better:

This target is optional. In many cases it is not needed because the
upstream developers make canonical original source archive files for each
version available for direct public download; in this case, creating a
<file>debian/watch</file> configuration that tells <prgn>uscan</prgn> how
to download the original source archive is sufficient.

I also wonder if this ought to be a footnote instead. I don't think uscan
should be considered normative in policy.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-07-2009, 09:25 PM
Ben Finney
 
Default Bug#466550: Pristine source from upstream VCS repository

On 07-Mar-2009, Steve Langasek wrote:
> On Fri, Mar 06, 2009 at 11:03:57AM +1100, Ben Finney wrote:
>
> > === modified file 'policy.sgml'
[…]
> > + for Debian. See the “Original source archive”
> > + section, below, for policy details of this file.
> > + </p>
>
> Surely, given that this is sgml, you should be referencing "Original
> source archive" by id instead of "see below"?

My SGML-fu is very weak. The existing text also doesn't give a
reference, instead saying “the original source tar file format
described below”.

I've added a reference now.

> > +
> > + <p>
> > + The actions for this target fetch the original source
> > + package, corresponding to the Debian package version,
>
> This reads wrong. What are "actions for this target"? Should be
> "This target fetches [...]" as in the original, or "The target
> fetches [...]" if you're trying to avoid repetition.

I'm trying to avoid repetition but also being more specific by
following the Make documentation in distinguishing the target (which
is just a label) from the commands run to achieve that target. But I
used the wrong term; changed to “commands for this target”.

> > + This target is <em>optional</em>. A common reason to
[…]
>
> Why is 'optional' emphasized?

I misunderstood the “words … used to distinguish the significance of
the various guidelines in this policy document” paragraphs in the
“Scope” section, and thought they needed to be emphasised in order to
have the described effect. That's not true, though, as I see from the
rest of the document. Fixed.

> I don't like the speculation about developers' reasons for forgoing (not
> 'foregoing', btw) the target.
[…]
> I also wonder if this ought to be a footnote instead. I don't think uscan
> should be considered normative in policy.

Fixed.


Updated patch:


=== modified file 'policy.sgml'
--- policy.sgml 2009-03-05 08:44:48 +0000
+++ policy.sgml 2009-03-07 22:22:45 +0000
@@ -1907,12 +1907,21 @@
<tag><tt>get-orig-source</tt> (optional)</tag>
<item>
<p>
- This target fetches the most recent version of the
- original source package from a canonical archive site
- (via FTP or WWW, for example), does any necessary
+ This target generates the original source archive for
+ the package, such that its contents exactly match the
+ original source archive used to generate the package
+ for Debian.
+ </p>
+
+ <p>
+ The commands for this target fetch the original source
+ package, corresponding to the Debian package version,
+ from a canonical archive site (for example, via FTP,
+ WWW, or a public VCS repository), do any necessary
rearrangement to turn it into the original source
- tar file format described below, and leaves it in the
- current directory.
+ archive file format, and leave it in the current
+ directory. See <ref id="pkg-sourcearchives"> for
+ policy details of the original source archive.
</p>

<p>
@@ -1922,8 +1931,14 @@
</p>

<p>
- This target is optional, but providing it if
- possible is a good idea.
+ This target is optional, and in many cases is
+ unnecessary.<footnote>A common reason to omit this
+ target is that the upstream developers make available,
+ for direct download, canonical original source archive
+ files for each specific version. In these cases, it is
+ sufficient to create a <file>debian/watch</file>
+ configuration for <prgn>uscan</prgn> to fetch the
+ original source archive.</footnote>
</p>
</item>



--
“If society were bound to invent technologies which could only |
` be used entirely within the law, then we would still be sitting |
_o__) in caves sucking our feet.” —Gene Kan, creator of Gnutella |
Ben Finney <ben@benfinney.id.au>
 
Old 03-11-2009, 10:33 AM
Ben Finney
 
Default Bug#466550: Pristine source from upstream VCS repository

On 11-Mar-2009, Goswin von Brederlow wrote:
> Manoj Srivastava <srivasta@debian.org> writes:
>
> > I am wondering which is of more use to the end users as
> > well: I can always get the sources of the package I have
> > already on my disk from Debian, but getting the latest
> > munged source seems more useful to me.
>
> Full ACK. The way to get the current upstream source for a debian
> package is "apt-get source" or equivalent and then using the
> orig.tar.gz. Duplicating this in debian/rules seems wastefull.

That's not quite the same thing though. That will get the original
source archive *as in the Debian repository*; while the existing and
proposed ‘get-orig-source’ target gets the original source archive
*from the canonical upstream location itself*.

It's worth asking, then, what is the original purpose for which the
‘get-orig-source’ target specification was inserted into the policy?

* To get some copy of the original source archive. If so, that is
entirely redundant with making it available in the Debian
repository. Does anyone think this is what was intended by the
drafters of that policy clause?

* To get the *latest* version of the package as an original source
archive, regardless of the Debian version of the package. This is
largely duplicated by ‘uscan(1)’, but not for all cases.

* To get the original source archive corresponding to the package
directly from the canonical upstream location. That is the purpose
of the patch I've submitted to this bug report.

* To do something else. I haven't seen any other options not covered
here, but that doesn't mean the truth might not be different.

What is our best resource for discovering which of these options is
the actual intent of the ‘get-orig-source’ target when it was inserted
into policy?

--
“I took a course in speed waiting. Now I can wait an hour in |
` only ten minutes.” —Steven Wright |
_o__) |
Ben Finney <ben@benfinney.id.au>
 
Old 03-11-2009, 02:13 PM
Manoj Srivastava
 
Default Bug#466550: Pristine source from upstream VCS repository

On Wed, Mar 11 2009, Ben Finney wrote:

> On 11-Mar-2009, Goswin von Brederlow wrote:
>> Manoj Srivastava <srivasta@debian.org> writes:
>>
>> > I am wondering which is of more use to the end users as
>> > well: I can always get the sources of the package I have
>> > already on my disk from Debian, but getting the latest
>> > munged source seems more useful to me.
>>
>> Full ACK. The way to get the current upstream source for a debian
>> package is "apt-get source" or equivalent and then using the
>> orig.tar.gz. Duplicating this in debian/rules seems wastefull.
>
> That's not quite the same thing though. That will get the original
> source archive *as in the Debian repository*; while the existing and
> proposed ‘get-orig-source’ target gets the original source archive
> *from the canonical upstream location itself*.


In most cases, the sources are identical; and even otherwise, it
is just a matter of looking into debian/copyright and doing a
wget. This is not a compelling enough case to create a whole new target
in debian/rules and policy.

Indeed, the whole rationale for the target, and why it got into
policy, was for the secific cases where extended munging of upstream
had to be done, usually, but not restricted to, the cases where
upstream had DFSG violations and the upstream source could not be
allowed in the main section of the archive.

This is what diferentiates is from uscan; indeed, I use uscan in
the cases where I provide the target, The target unpacks the
raw upstream source, munges it (by, say, removing a subdir which has
non-dfsg stuff, or removes the debian dir, applies patches, or whatever
other processing is required.

There is no need to do this for the current version; the mungeds
sources already are an apt-get source away.


This is not a trivial replacement for uscan watch file with a
version number in the regexp, which is the trivial way of implementing
the task of getting the current version from upstream). This target is
for the developers of munged sources to provide a script to munge the
latest sources.

Keep in mind that sometimes an unmunged source will not build
with the debian/rules file in the archive; so it is not just a
matter of dropping in ./debian into the latest upstream tarball.

That facility is being dropped silently, and I object to there
being no transition plan, and nothing to replace the functionality
currently provided by the target.

If we move over to the new docbook format (and I have one that
seems to kinda work, but I got demotivated before I published it), the
rationalew would have spelled out _why_ we needed such a target, and
what it was originally designed to be used for.

> It's worth asking, then, what is the original purpose for which the
> ‘get-orig-source’ target specification was inserted into the policy?
>
> * To get some copy of the original source archive. If so, that is
> entirely redundant with making it available in the Debian
> repository. Does anyone think this is what was intended by the
> drafters of that policy clause?

No.

> * To get the *latest* version of the package as an original source
> archive, regardless of the Debian version of the package. This is
> largely duplicated by ‘uscan(1)’, but not for all cases.

As long as no munging is t be done.

> * To get the original source archive corresponding to the package
> directly from the canonical upstream location. That is the purpose
> of the patch I've submitted to this bug report.

I think this is not useful enough to take over the original
purpose of the target. I suggest if you think that looking into
debian/copyright + wget is too hard, then create a new policy target.

> * To do something else. I haven't seen any other options not covered
> here, but that doesn't mean the truth might not be different.

Indeed. You missed the whole "munge source to make it
acceptable for debian and allow it to be built with ./debian files"

> What is our best resource for discovering which of these options is
> the actual intent of the ‘get-orig-source’ target when it was inserted
> into policy?

Ask me. There is institutional knowledge in the post of leading
the policy development for 10 years. And my memory is not that bad --
yet, though doubtless senility encroaches.

manoj
--
Under deadline pressure for the next week. If you want something, it
can wait. Unless it's blind screaming paroxysmally hedonistic...
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 02:21 PM
Manoj Srivastava
 
Default Bug#466550: Pristine source from upstream VCS repository

Hi,

The best way to get the exact sources for the current version
probably should be a new watch file (watch-current) which has a static
version number in the regexp, but can use all the other facilities f
uscan -- wild carded directory, looking thoiugh an index.html page for
a matching href, and so on.

This allows us not to reinvent the wheel in policy for uscan,
does not require every one to do their own uscan, often replicating
uscan poorly, and allows for simple scripting to grab the watch-current
file from, say, the pts or packages.d.o and not havce to unpack the old
source to run make on debian/rules.

It is simpler for lintian to check, avoids the NIH syndrome,
does not silently change the semantics of a policy rule, and continues
to allow the current target to be used for the original purpose of
munging raw upstream sources. It also means we will not be doing design
of the dowloader in policy.

There was a reason the original policy dictum was vague: we did
not want to limit the things people do to munge upstream sources to
make it palatable to Debian.

Perhaps it is time for me to play a more active role in policy
again, if Russ is willing to let me back in.


manoj
--
You mean you didn't *know* she was off making lots of little phone
companies?
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 05:09 PM
Russ Allbery
 
Default Bug#466550: Pristine source from upstream VCS repository

Manoj Srivastava <srivasta@debian.org> writes:

> Perhaps it is time for me to play a more active role in policy
> again, if Russ is willing to let me back in.

Good heavens, yes. I've always found your Policy work to be extremely
valuable, and whatever time you're willing to spend on the work is greatly
appreciated.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 10:44 PM
Ben Finney
 
Default Bug#466550: Pristine source from upstream VCS repository

On 11-Mar-2009, Manoj Srivastava wrote:
> On Wed, Mar 11 2009, Ben Finney wrote:
> > It's worth asking, then, what is the original purpose for which the
> > ‘get-orig-source’ target specification was inserted into the policy?
>
> Indeed, the whole rationale for the target, and why it got into
> policy, was for the secific cases where extended munging of upstream
> had to be done, usually, but not restricted to, the cases where
> upstream had DFSG violations and the upstream source could not be
> allowed in the main section of the archive.
[…]
>
> This is not a trivial replacement for uscan watch file with a
> version number in the regexp, which is the trivial way of implementing
> the task of getting the current version from upstream). This target is
> for the developers of munged sources to provide a script to munge the
> latest sources.
>
> Keep in mind that sometimes an unmunged source will not build
> with the debian/rules file in the archive; so it is not just a
> matter of dropping in ./debian into the latest upstream tarball.
>
> That facility is being dropped silently, and I object to there
> being no transition plan, and nothing to replace the functionality
> currently provided by the target.

This purpose isn't clear to me at all in the existing policy document;
hence my (and presumably others) misunderstanding of its purpose and
implementing a ‘get-orig-source’ target with a different purpose.

> Ask me. There is institutional knowledge in the post of leading
> the policy development for 10 years. And my memory is not that bad --
> yet, though doubtless senility encroaches.

I would very much like to see a patch from you, submitted to this bug
report, that re-works this policy section to more explicitly state the
intent and purpose of the ‘get-orig-source’ target.

--
“To label any subject unsuitable for comedy is to admit |
` defeat.” —Peter Sellers |
_o__) |
Ben Finney <ben@benfinney.id.au>
 
Old 03-12-2009, 08:05 AM
Steve Langasek
 
Default Bug#466550: Pristine source from upstream VCS repository

On Wed, Mar 11, 2009 at 10:13:51AM -0500, Manoj Srivastava wrote:
> This is what diferentiates is from uscan; indeed, I use uscan in
> the cases where I provide the target, The target unpacks the
> raw upstream source, munges it (by, say, removing a subdir which has
> non-dfsg stuff, or removes the debian dir, applies patches, or whatever
> other processing is required.

> There is no need to do this for the current version; the mungeds
> sources already are an apt-get source away.

For several packages I (co)maintain where I have to munge the upstream
tarball, the standard procedure (inherited from past maintainers) is:

- increment the version number in the debian packaging
- call the get-orig-source target

I think it's perfectly reasonable to want the get-orig-source target to give
you a *specified* version of an upstream tarball, rather than the *newest*
version of an upstream tarball. Packaging a new upstream version doesn't
necessarily mean packaging the latest that uscan can find.

It's also useful for third parties to be able to easily examine the
provenance of specific Debian tarballs. A get-orig-source target provides a
much more concise description of the Debian changes than examining the diff
between the two tarballs.

So I certainly agree that uscan doesn't obsolete the get-orig-source target,
but I disagree that it's not useful to have such a target generate a tarball
for the 'current' upstream version.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-12-2009, 01:12 PM
Gunnar Wolf
 
Default Bug#466550: Pristine source from upstream VCS repository

Steve Langasek dijo [Thu, Mar 12, 2009 at 02:05:42AM -0700]:
> I think it's perfectly reasonable to want the get-orig-source target to give
> you a *specified* version of an upstream tarball, rather than the *newest*
> version of an upstream tarball. Packaging a new upstream version doesn't
> necessarily mean packaging the latest that uscan can find.
>
> It's also useful for third parties to be able to easily examine the
> provenance of specific Debian tarballs. A get-orig-source target provides a
> much more concise description of the Debian changes than examining the diff
> between the two tarballs.
>
> So I certainly agree that uscan doesn't obsolete the get-orig-source target,
> but I disagree that it's not useful to have such a target generate a tarball
> for the 'current' upstream version.

Good point you have here - But (and I know it is not being discussed
yet, maybe you want to teleport this thread a couple of years into the
future) I feel this should clearly be an optional target, and the
canonical location for orig.tar.gz files should still be our archive -
Down the other road lies Gentoo's BSD ports' madness, where an
upstream site restructure means packages become unreachable and
insta-FTBFS.

--
Gunnar Wolf - gwolf@gwolf.org - (+52-55)5623-0154 / 1451-2244
PGP key 1024D/8BB527AF 2001-10-23
Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-12-2009, 02:31 PM
Manoj Srivastava
 
Default Bug#466550: Pristine source from upstream VCS repository

On Thu, Mar 12 2009, Steve Langasek wrote:

> On Wed, Mar 11, 2009 at 10:13:51AM -0500, Manoj Srivastava wrote:
>> This is what diferentiates is from uscan; indeed, I use uscan in
>> the cases where I provide the target, The target unpacks the
>> raw upstream source, munges it (by, say, removing a subdir which has
>> non-dfsg stuff, or removes the debian dir, applies patches, or whatever
>> other processing is required.
>
>> There is no need to do this for the current version; the mungeds
>> sources already are an apt-get source away.
>
> For several packages I (co)maintain where I have to munge the upstream
> tarball, the standard procedure (inherited from past maintainers) is:
>
> - increment the version number in the debian packaging
> - call the get-orig-source target
>
> I think it's perfectly reasonable to want the get-orig-source target
> to give you a *specified* version of an upstream tarball, rather than
> the *newest* version of an upstream tarball. Packaging a new upstream
> version doesn't necessarily mean packaging the latest that uscan can
> find.

Fair enough. But that is not the semantics of the target
currently: get-orig-source is defined right now to get the /latest/
source, and while it is reasonable to have both behaviours, it is not
necessary to expect both from the same target.

To recap:
1) apt-get source is enough to get the latest Debian source from the
archive (and whet for older sources)
2) In the absence of munging, uscan, with a watch and watch-current
files, is adequate to get either the latest or a specific version
from upstream
3) It is reasonable to get the latest, or a specific version, from
upstream, and munge it.

So, for case 3: get-orig-source has been defined to get the
latest sources (with munging, if needed). If we want to get a specific
version, we can:
a. over-load get-orig-source to take a version number, some how,
through an env variable, perhaps
b. create a brand new target, which looks at the env variable, and
falls back to the version in the changelog.

I think case a is harder from a policy creation perspective,
since it should not outlaw currently conforming implementations. The
new target method can be deployed, tested in the wild, and then made
into policy when the kinks have been ironed out.

manoj
--
Writing is turning one's worst moments into money. J.P. Donleavy
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 09:34 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org