FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 02-11-2008, 04:26 PM
Russ Allbery
 
Default How to cope with patches sanely

Manoj Srivastava <srivasta@debian.org> writes:
> On Thu, 7 Feb 2008 11:48:47 +0000, Matthew Johnson <mjj29@debian.org> said:
>> On Thu Feb 07 22:42, Ben Finney wrote:

>>> In the scenario Manoj presents above, the modifications applied to
>>> upstream are easily available all in one place: the foo.diff.gz.

>> But all as one patch, not as your nice separated list of commits
>> and/or branches.

> True. We now have to evaluate the benefits of providing sources
> *that the binary packages are built from with no fuss (dpkg -x); which
> can then be inspected and patched.

Well, this whole thread was really about how to have our cake and eat it
too, so I don't think we should assume that we can't provide the sources
with no muss and not *also* use a patch system. That's what many of us
were trying to work towards with the discussion of how quilt would fit
into wig&pen.

> What is the use case this effort is designed to address? I
> have not actually heard NMU/porters express a need for converting
> monolithic patches to patch series. Have I missed the need statements?

It's to let those of us who think of a patch as a useful unit of
development instead of a branch continue to develop Debian packages using
the methodology that we feel makes us the most productive without causing
extra problems for the security team and NMUs.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
 
Old 02-16-2008, 08:24 AM
martin f krafft
 
Default How to cope with patches sanely

Sorry for the late reply. I'll send only a single reply this time
since my flood reply seemed to annoy some people. I still think
short single replies are better than long, unified ones...

also sprach Pierre Habouzit <madcoder@debian.org> [2008.02.03.2038 +1100]:
> > > I'm less and less sure that a git-based format is a brilliant
> > > idea. I like git more than a lot, but it's a poor idea to base
> > > source packages on them.
> >
> > Why?
>
> Because it's a way to high level tool for the task. Git (and
> I suppose that the following is true for other DVCS, and if not,
> they _REALLY_ suck, and I mean it, really) has been designed so
> that you can only exchange patches.

Many people actually tend to think of Git as a filesystem...

> > > IOW, all that you should need to grok what is in a source package
> > > should basically be tar/ar/cpio/... and vi.
> >
> > Are you sure that this has to be the case in 10 years?
>
> Are you sure git will be there and backward compatible in 10 years ?
> Okay, it's likely given the current upstream, that focuses a _lot_ on
> backward compatibility. But still. Some repository formats have been
> deprecated (the object store is of version3 and IIRC git doesn't grok
> version1 anymore, but I may be mistaken). That's the kind of gamble I
> wouldn't take.
>
> Whereas I guess tar will always grok tarballs it generates today in 10
> years, and gzip/bzip2/lzma/$compressor won't change either, and text is
> text for enough time to assume it'll remain as editeable in 10 years.

Fair enough, but why would e.g. version3 ever be removed? It's not
like versioned storage formats need a whole lot more support than
the infrastructure to identify them, which is already in place.

> > Nice, I didn't see this before. This *is* in fact nice and puts
> > the quilt *format* very high on my list.
>
> Yes, quilt preserves *everything* you put in the header, so if you use
> git (or $scm) to generate the patches with authoring information, commit
> message and so on, it wont be lost.

Excellent.

> What I ask you is just to be consistent. Either we _will_ modify
> source packages, either we won't. If we will, adding features to it is a
> good idea, if we won't, let's just focus on how to let it be expressive
> enough to encode in it all what we use as new features upstream from it.
> And as a matter of a fact, quilt is enough for that.

Good arguments. Thanks!



also sprach David Nusinow <dnusinow@speakeasy.net> [2008.02.03.1154 +1100]:
> > - patch/feature-branch-specific history. Say feature branch 'foo'
> > has a bug, so you check it out and work on it again... now
> > you're suddenly in the context of all the work you've previously
> > done and you can trivially browse the history of changes
> > applicable only to what you're currently working on.
>
> How do you decide the order to apply your feature branches? Do you encode
> it in the branch name? Congratulations, you've just reinvented dbs. Do you
> have a separate file in the debian directory that tells you the merge
> order? Congratulations, you've just reimplemented quilt. Do you write a
> script to update your branches automatically, only failing when you have to
> manually update yourself? Congratulations, you've just reinvented every
> other patch system in existence.

Good points, thanks David.



also sprach Manoj Srivastava <srivasta@debian.org> [2008.02.05.1751 +1100]:
> On Sat, 26 Jan 2008 11:34:27 -0500, David Nusinow
> <dnusinow@speakeasy.net> said:
> [...]
> > An alternate idea I keep seeing is feature branches. I have absolutely
> > no idea why these are considered preferable to most people. In the
> > case of the X server we regularly carry around 30 patches on our
> > stack. If we were to put each of these on its own unique feature
> > branch, merging them becomes an exercise in pain. Sure, we could
> > implement our own custom scripts and have very strict naming policies,
> > but then you've re-invented the wheel and congratulations, you have
> > yet another custom piece of crap software that you have to maintain on
> > top of the already complicated one you actually care
> > about. Additionally, you have a lot more branches getting in the way
> > when you're trying to figure out which one you're looking for.
>
> In my experience, once I have created the integration branch,
> most upstream versions require little to none re-merging; I just apply
> the delta to each of the feature branches, _and_ the integration
> branch. Very rarely do I have to fix up a feature branch, and then
> apply a second delta with that fix up to the integration branch; but it
> has not been, for me, painful at all, nor do I have to jump through
> hoops every single time to accommodate new upstream.

But sometimes you will have to touch an integration branch and then
things get messy, especially if there are dependencies between
feature branches. I think David is making a very strong point
here...



also sprach Ben Finney <bignose+hates-spam@benfinney.id.au> [2008.02.07.2242 +1100]:
> > On Thu, Feb 07, 2008 at 05:12:00AM +0000, Manoj Srivastava
> > wrote: That's not really about "bring your feature branches"
> > into a patch system, but rather export them into something that
> > looks like one so that the security team or the NMUer can have
> > a good outlook of the modifications you apply to upstream.
>
> In the scenario Manoj presents above, the modifications applied to
> upstream are easily available all in one place: the foo.diff.gz.

But I may want to see the differences from upstream nicely
separated into patches, rather than whole chunks? This is the same
argument why 32k diffs provided on patches.ubuntu.com are useless to
Debian maintainers.

> Whereas with a patch bundle system, the security team needs to
> know *every* patch bundle system in use and how to use it, just to
> have a chance of operating on arbitrary Debian source packages.

It looks like they'll just have to know quilt, or any other
compatible system.




Thanks to everyone for a good discussion. I am sure this isn't the
last we've heard on this front.


With Extremadura coming up, who would be interested in joining
a "workgroup" to work on this stuff? I would volunteer to chair such
a session...

--
.'`. martin f. krafft <madduck@debian.org>
: :' : proud Debian developer, author, administrator, and user
`. `'` http://people.debian.org/~madduck - http://debiansystem.info
`- Debian - when you have better things to do than fixing systems

"if you can dream it, you can do it"
-- walt disney
 
Old 02-16-2008, 05:16 PM
Pierre Habouzit
 
Default How to cope with patches sanely

On Sat, Feb 16, 2008 at 09:24:12AM +0000, martin f krafft wrote:
> > Because it's a way to high level tool for the task. Git (and
> > I suppose that the following is true for other DVCS, and if not,
> > they _REALLY_ suck, and I mean it, really) has been designed so
> > that you can only exchange patches.
>
> Many people actually tend to think of Git as a filesystem...

I do too, it's just that if you know where you come from, all you need
to rebuild the same state are the patches. Most of the kernel and git
developments are done this way. For the kernel not even every major
contributor uses git (akpm didn't last time I checked) and their
exchange format is patch series, completely bijective with quilt format.

And given their current tremendous patch integration rate, I _really_
think this format has proven its robustness, while being good enough for
git.


> > Are you sure git will be there and backward compatible in 10 years ?
> > Okay, it's likely given the current upstream, that focuses a _lot_ on
> > backward compatibility. But still. Some repository formats have been
> > deprecated (the object store is of version3 and IIRC git doesn't grok
> > version1 anymore, but I may be mistaken). That's the kind of gamble I
> > wouldn't take.
> >
> > Whereas I guess tar will always grok tarballs it generates today in 10
> > years, and gzip/bzip2/lzma/$compressor won't change either, and text is
> > text for enough time to assume it'll remain as editeable in 10 years.
>
> Fair enough, but why would e.g. version3 ever be removed? It's not
> like versioned storage formats need a whole lot more support than
> the infrastructure to identify them, which is already in place.

Well, you still don't know if upstream will become mad in the next 10
years, can you ? And git code isn't trivial, so I don't think you'll
be able to hack version3 back into it if it's stripped from upstream at
some point.


> > > Nice, I didn't see this before. This *is* in fact nice and puts
> > > the quilt *format* very high on my list.
> >
> > Yes, quilt preserves *everything* you put in the header, so if you use
> > git (or $scm) to generate the patches with authoring information, commit
> > message and so on, it wont be lost.
>
> Excellent.

In fact afaict, git-quiltimport is able to get authoring informations
and so on, and if it's not able to use embedded blob sha's yet, it's
probably not hard to build from there. As a matter of a fact, I'm
experimenting right now with the llvm2 packaging a way to share our
"patch branch" (with a co-maint) through debian/patches, and so far, it
doesn't work so badly. IOW, what the xorg people do for months actually,
except that I use git all over the place.

[ and to be fair we export patches through git-format-patch so there
isn't a series file, but the series file would basically built this
way: (ls -1 * 2>/dev/null || > series ]

> > What I ask you is just to be consistent. Either we _will_ modify
> > source packages, either we won't. If we will, adding features to it is a
> > good idea, if we won't, let's just focus on how to let it be expressive
> > enough to encode in it all what we use as new features upstream from it.
> > And as a matter of a fact, quilt is enough for that.
>
> Good arguments. Thanks!




> With Extremadura coming up, who would be interested in joining
> a "workgroup" to work on this stuff? I would volunteer to chair such
> a session...

I would be very much interested, but I'm not really sure that I'd have
the time to join a session. Note that my talk at FOSDEM could be also a
place to discuss some points too on the subject (If I get the time to
write the damn slides *cough*)

--
O Pierre Habouzit
O madcoder@debian.org
OOO http://www.madism.org
 
Old 02-16-2008, 10:29 PM
Manoj Srivastava
 
Default How to cope with patches sanely

On Sat, 16 Feb 2008 20:24:12 +1100, martin f krafft <madduck@debian.org> said:

> also sprach Manoj Srivastava <srivasta@debian.org> [2008.02.05.1751
> +1100]:
>> On Sat, 26 Jan 2008 11:34:27 -0500, David Nusinow
>> <dnusinow@speakeasy.net> said: [...]
>> > An alternate idea I keep seeing is feature branches. I have
>> > absolutely no idea why these are considered preferable to most
>> > people. In the case of the X server we regularly carry around 30
>> > patches on our stack. If we were to put each of these on its own
>> > unique feature branch, merging them becomes an exercise in
>> > pain. Sure, we could implement our own custom scripts and have very
>> > strict naming policies, but then you've re-invented the wheel and
>> > congratulations, you have yet another custom piece of crap software
>> > that you have to maintain on top of the already complicated one you
>> > actually care about. Additionally, you have a lot more branches
>> > getting in the way when you're trying to figure out which one
>> > you're looking for.
>>
>> In my experience, once I have created the integration branch, most
>> upstream versions require little to none re-merging; I just apply the
>> delta to each of the feature branches, _and_ the integration
>> branch. Very rarely do I have to fix up a feature branch, and then
>> apply a second delta with that fix up to the integration branch; but
>> it has not been, for me, painful at all, nor do I have to jump
>> through hoops every single time to accommodate new upstream.

> But sometimes you will have to touch an integration branch and then
> things get messy, especially if there are dependencies between feature
> branches. I think David is making a very strong point here...

No. You never, ever touch an integration branch. If you do,
you are correct in assuming the mouth of hell shall open and
spew forth the demons. The integration branch is only for pulling in
changes from the feature branches; this is why we have a "debian"
feature branch.

You hack in the sloppy branch. You merge change sets from the
sloppy branch into the feature branches. You merge the delta from the
feature branch into the integration branch. In my experience, there is
usually no new mess -- since you are only merging the delta in the
integration branch; the step you took earlier to deal with the mess
usually still apply.

If you are speculating there will be unforeseen and
unmentionable problems, all I can say is that I have yet to experience
them, and not all my packages are trivial packaging of upstream
versions.

> also sprach Ben Finney <bignose+hates-spam@benfinney.id.au>
> [2008.02.07.2242 +1100]:
>> > On Thu, Feb 07, 2008 at 05:12:00AM +0000, Manoj Srivastava wrote:
>> > That's not really about "bring your feature branches" into a patch
>> > system, but rather export them into something that looks like one
>> > so that the security team or the NMUer can have a good outlook of
>> > the modifications you apply to upstream.
>>
>> In the scenario Manoj presents above, the modifications applied to
>> upstream are easily available all in one place: the foo.diff.gz.

> But I may want to see the differences from upstream nicely separated
> into patches, rather than whole chunks?

You mean you do not want to see the differences between a
feature branch and upstream as one chunk, but nicely segregated in
smaller pieces?

> This is the same argument why 32k diffs provided on patches.ubuntu.com
> are useless to Debian maintainers.

Not really, unless your feature branches are poorly chosen.
Each feature branch, in my packages, represent onlogical feature that
belongs in the same patch (series).

manoj

--
Mountain Dew and doughnuts... because breakfast is the most important
meal of the day.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 05:04 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org