FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 07-28-2011, 01:36 PM
Lucas Nussbaum
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Hi,

It seems that many different teams have recently switched to Git, or are
considering switching. But each of them seems to re-implement the wheel
by designing their own Git workflow.

I think that it would be fantastic if we could collaborate on defining
a common Git workflow.
Pros:
- build on the experience of other teams
- easier for maintainers who are active in several teams
- share the documentation work, get better documentation
Cons:
- might require a migration when switching to that 'standard' workflow

I'm mostly interested in teams that maintain a large number of easy
packages with at most a handful of patches, not in teams that maintain a
few, complicated packages. I'm not sure if we can find a workflow that
suits both (but if we can, that's great, and I don't want to exclude
those teams from the discussion).

People from the Ruby, Perl and Java teams already expressed interest.

If you are interested in participating (which doesn't mean that you
commit on behalf of your team to switch to that workflow, just that you
want to participate in the discussions), add yourself (and your team) to
http://wiki.debian.org/GitPackagingWorkflow. If more than 3 teams are
interested in a BOF at DC11, I'll try to organize one on saturday (all
the video-broadcasted slots are taken, but we will make sure to take
notes). If the BOF doesn't happen, I'll contact interested people to see
how to continue.

I realize that it's unlikely that we will find a unique workflow that
fits everybody. But we can probably converge on many things, and then
document the few widely-used alternatives for the rest.

- Lucas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110728133607.GA9225@xanadu.blop.info">http://lists.debian.org/20110728133607.GA9225@xanadu.blop.info
 
Old 07-28-2011, 01:36 PM
Lucas Nussbaum
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Hi,

It seems that many different teams have recently switched to Git, or are
considering switching. But each of them seems to re-implement the wheel
by designing their own Git workflow.

I think that it would be fantastic if we could collaborate on defining
a common Git workflow.
Pros:
- build on the experience of other teams
- easier for maintainers who are active in several teams
- share the documentation work, get better documentation
Cons:
- might require a migration when switching to that 'standard' workflow

I'm mostly interested in teams that maintain a large number of easy
packages with at most a handful of patches, not in teams that maintain a
few, complicated packages. I'm not sure if we can find a workflow that
suits both (but if we can, that's great, and I don't want to exclude
those teams from the discussion).

People from the Ruby, Perl and Java teams already expressed interest.

If you are interested in participating (which doesn't mean that you
commit on behalf of your team to switch to that workflow, just that you
want to participate in the discussions), add yourself (and your team) to
http://wiki.debian.org/GitPackagingWorkflow. If more than 3 teams are
interested in a BOF at DC11, I'll try to organize one on saturday (all
the video-broadcasted slots are taken, but we will make sure to take
notes). If the BOF doesn't happen, I'll contact interested people to see
how to continue.

I realize that it's unlikely that we will find a unique workflow that
fits everybody. But we can probably converge on many things, and then
document the few widely-used alternatives for the rest.

- Lucas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110728133607.GA9225@xanadu.blop.info">http://lists.debian.org/20110728133607.GA9225@xanadu.blop.info
 
Old 07-29-2011, 10:26 AM
Iain Lane
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Hi there,

On Thu, Jul 28, 2011 at 03:36:07PM +0200, Lucas Nussbaum wrote:
> Hi,
>
> It seems that many different teams have recently switched to Git, or are
> considering switching. But each of them seems to re-implement the wheel
> by designing their own Git workflow.

None of us is at Debconf unfortunately, but the Mono/CLI teams are
interested in this. For all of our packages we use git-buildpackage and
the majority just use plain quilt (or maybe if we're feeling exotic with
gbp-pq) for patches.

I think patches are the most interesting area to focus on, as they are
where we can get a lot of the benefit of git. What you want:

1. Ability to cherry-pick from upstream
2. Proper history of patches
3. git-blame to work
4. Ability to drop patches merged upstream, etc
5. Use just the commands provided by git as far as possible (scripts
OK as long as they just automate git commands)

2 & 4 seem to me to basically mean that you want the patches /applied/
in your git tree. But it's also pretty nasty to version .pc (c.f.
#572204), so you probably want to use no patchsys or single-debian-patch
to avoid the pain at the expense of having a slightly less useful source
package.

Mirco 'meebey' Bauer came up with a pure git workflow for managing
patches for the mono source package, in which we cherry-pick a lot of
patches from upstream. Basically each patch is a branch of upstream,
which is itself a gbp branch. These are then merged in turn into master,
resolving all conflicts at this time. We can handle merging with new
upstream versions and dropping applied patches. Here's some command
sequences that might give you an idea of how this works

http://wiki.debian.org/Teams/DebianMonoGroup/Git

and a script also written by Mirco to test whether branches are already
merged in a new upstream version or if they need updating

http://anonscm.debian.org/gitweb/?p=pkg-mono/packages/mono.git;a=blob;f=debian/git-test-debian-patches;h=76f31265d026eb6c97448749c864aba39630f11e ;hb=refs/heads/master-experimental

It's clearly not perfect, but I think something like this is a good
start.

HTH,

--
Iain Lane [ iain@orangesquash.org.uk ]
Debian Developer [ laney@debian.org ]
Ubuntu Developer [ laney@ubuntu.com ]
PhD student [ ial@cs.nott.ac.uk ]
 
Old 07-29-2011, 02:13 PM
Thomas Koch
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Hi,

debconf is fantastic in that it gave me the possibility to discuss with many
people and rework my ideas about patch management. The ideas now feel right.

I've attached a draft of a description of the workflow.

Before continuing with the description I'll now try to find my way around the
git-buildpackage source code as long as Guido is still around. I'd love to
hear feedback whether the description is understandable and the ideas are
sane.

Best regards,

Thomas Koch, http://www.koch.ro
The proposed patch management workflow in this document has many similarities with git-pq. It differs in these points:

* Patch branches all originate in upstream or in the branches they depend on. git-pq puts all patches on one line.
* The commit objects of patch branches are conserved in a normal packaging workflow even though users only share the usual upstream, master, build branches and release tags.
* This workflow allows to make use of GIT's merge capabilities.

For the purpose of this document, the master branch is the branch where /debian/ lives. Others like to call this branch e.g. "debian".

==Most basic workflow, without patches, sane upstream==

upstream
|
| master
| |
| /* merge, new debian version (n commits)
|/ |
* | new upstream version
| |
| * debianize, debian release (n commits)
| /
|/
* import upstream

==Upstream needs clean up to be dfsg compliant==

upstream
|
| dfsg_clean
| |
| | master
| | |
| | * new debian version
| | /|
| |/ |
| * | merge, clean up (n commits)
| /| |
|/ | |
* | | import new unmodified upstream
| | |
| | * debianize, debian release
| | /
| |/
| * merge, clean up (n commits)
| /
|/
* import unmodified upstream

==Patches, dfsg free upstream==

===principles, requirements, concepts===

* Patches should not be merged to the master branch, because it's hard to "unmerge".
* We want to use GIT's merge capabilities and other features, therefor we'd like to work on patches in seperate branches per patch.
* It should be much easier to use then topgit
* We don't want to be forced to pull/push douzens of patch branches.
* We don't want to (permanently) pollute the branch namespace

===Design===

* When a new release has been merged to master, a new build branch is forked from master: build-$UPSTREAM_VERSION
* Patch branches are only merged in such build branches, never in the master branch
* Later we can delete the build branches but we still keep the commits since we tag each Debian release

* The primary storage of patches are patch files in quilt format in debian/patches in the master branch
* We provide tooling to convert patch branches into annotated quilt patches and recreate *identical* patch branches from quilt files.
* We add extra headers in [http://dep.debian.net/deps/dep3/ dep-3] format to the quilt files in debian/patches:
** git-commit: SHA-1 value of the HEAD of the patch branch
** git-base-commit: SHA-1 value of the base commit of the patch branch (only necessary if git-base-name is not the UPSTREAM_BRANCH)
** git-dependencies: list of branch names of dependencies of this patch branch

===Workflow===

Creating patches:
# Checkout patch branch from upstream (or from the branch it depends on).
# Hack, commit
# Create an annotated quilt patch from the patch branch
# repeat the above for other patch branches
# merge the patch branches into the build branch
# Once the patches are merged in the build branch, the patch branches can be deleted.

Editing patches:
# Recreate the patch branches with informations from the quilt annotations
# Hack, commit
# update quilt files
# merge updated patch branches into build branch

Update upstream:
# Import upstream, merge into master
# recreate the patch branches
# either merge upstream changes into each patch branch or rebase patch branches
# hack on patch branches
# update quilt files
# checkout a new build branch from master
# merge updated patch branches into build branch

===Tooling===

The outlined workflow could be done manually without new tools. It's however only practical, if additional tools are provided.
This section describes the necessary tools to be implemented.

environment variables:
* UPSTREAM_BRANCH - the name of the upstream branch, default "upstream", may need to be changed to dfsg_clean

====create quilt file====

options:
* dependency branches [optionally, defaults to empty=UPSTREAM_BRANCH]
* base [optionally, defaults to SHA-1 of UPSTREAM_BRANCH]
* head [optionally, defaults to SHA-1 of current HEAD]
* name [optionally, defaults to current HEAD name]
* target [optionally, defaults to debian/patches/$BRANCH_NAME]

Creates a quilt patch file from a patch branch or updates an existing one, inheriting options from an exisiting quilt file.

====checkout branch from quilt file====

options:
* branch name (translates to the quilt file's name)
* checkout [boolean, defaults to true]

Creates a branch with the given name pointing to the commit specified in the git-commit line of the quilt file. Checks out the branch if checkout option is true.

====merge patch branch(es)====

options:
* target branch
* name [optionally, n-times, names of branches to merge]

Merges all patch branches into the target branches. Creates the target branch if it doesn't exist yet.

====status====

options:
* build branch [optionally]

Gives the following informations:

* list of patches (according to files in debian/patches)
* which patches needs update (because dependency or upstream was updated)
* which patches don't have their commit objects in the object database
* which patches have not yet been merged in the build branch


==links==
* man git-quiltimport
* quilt headers http://dep.debian.net/deps/dep3/
 
Old 07-29-2011, 05:41 PM
Thomas Koch
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Thomas Koch:

> I've attached a draft of a description of the workflow.

now living at http://wiki.debian.org/ThomasKoch/GitPackagingWorkflow
with one correction:

* git-base-commit: SHA-1 value of the base commit of the patch branch (only
necessary if git-base-dependencies is not empty)

Thomas Koch, http://www.koch.ro


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201107291941.10598.thomas@koch.ro">http://lists.debian.org/201107291941.10598.thomas@koch.ro
 
Old 07-30-2011, 10:13 AM
martin f krafft
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

also sprach Thomas Koch <thomas@koch.ro> [2011.07.29.1613 +0200]:
> I've attached a draft of a description of the workflow.

It would be good to put this onto the vcs-pkg-discuss mailing list.

--
.'`. martin f. krafft <madduck@d.o> Related projects:
: :' : proud Debian developer http://debiansystem.info
`. `'` http://people.debian.org/~madduck http://vcs-pkg.org
`- Debian - when you have better things to do than fixing systems

"i believe that the moment is near when by a procedure
of active paranoiac thought, it will be possible
to systematise confusion and contribute to
the total discrediting of the world of reality."
-- salvador dali
 
Old 07-30-2011, 12:35 PM
Lucas Nussbaum
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

On 28/07/11 at 15:36 +0200, Lucas Nussbaum wrote:
> Hi,
>
> It seems that many different teams have recently switched to Git, or are
> considering switching. But each of them seems to re-implement the wheel
> by designing their own Git workflow.
>
> I think that it would be fantastic if we could collaborate on defining
> a common Git workflow.
>
> [...]
>
> If you are interested in participating (which doesn't mean that you
> commit on behalf of your team to switch to that workflow, just that you
> want to participate in the discussions), add yourself (and your team) to
> http://wiki.debian.org/GitPackagingWorkflow. If more than 3 teams are
> interested in a BOF at DC11, I'll try to organize one on saturday (all
> the video-broadcasted slots are taken, but we will make sure to take
> notes). If the BOF doesn't happen, I'll contact interested people to see
> how to continue.

Hi,

So a BOF happened at Debconf 11, and I've put the raw notes online at
http://wiki.debian.org/GitPackagingWorkflow/DebConf11BOF

My understanding of the situation is that we are ready to document a
workflow including:
- using git-buildpackage for basic stuff
- managing several repositories with mr
- managing and tracking the status of the repositories (e.g with PET)
- managing patches with quilt (the "nothing fancy" approach)
+ additionally, using a patch-queue system (but which one?) to
manage patches locally
- managing debian/changelog with git-dch

Some people expressed interest in starting a proper documentation on
that on the Debian wiki. I suggest to use the
http://wiki.debian.org/GitPackagingWorkflow (or maybe to extend the
http://wiki.debian.org/PackagingWithGit page).

We probably need more discussion:
- on patch queue managers
- on the "one-branch-per-patch" approaches

Also, Raphael Hertzog mentioned that the next version of dpkg will
unapply patches and remove .pc after the build, which should help with
managing patches in git.

- Lucas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110730123502.GA28972@xanadu.blop.info">http://lists.debian.org/20110730123502.GA28972@xanadu.blop.info
 
Old 07-30-2011, 03:26 PM
David Goodenough
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

On Saturday 30 Jul 2011, Lucas Nussbaum wrote:
> On 28/07/11 at 15:36 +0200, Lucas Nussbaum wrote:
> > Hi,
> >
> > It seems that many different teams have recently switched to Git, or are
> > considering switching. But each of them seems to re-implement the wheel
> > by designing their own Git workflow.
> >
> > I think that it would be fantastic if we could collaborate on defining
> > a common Git workflow.
> >
> > [...]
> >
> > If you are interested in participating (which doesn't mean that you
> > commit on behalf of your team to switch to that workflow, just that you
> > want to participate in the discussions), add yourself (and your team) to
> > http://wiki.debian.org/GitPackagingWorkflow. If more than 3 teams are
> > interested in a BOF at DC11, I'll try to organize one on saturday (all
> > the video-broadcasted slots are taken, but we will make sure to take
> > notes). If the BOF doesn't happen, I'll contact interested people to see
> > how to continue.
>
> Hi,
>
> So a BOF happened at Debconf 11, and I've put the raw notes online at
> http://wiki.debian.org/GitPackagingWorkflow/DebConf11BOF
>
> My understanding of the situation is that we are ready to document a
> workflow including:
> - using git-buildpackage for basic stuff
> - managing several repositories with mr
> - managing and tracking the status of the repositories (e.g with PET)
> - managing patches with quilt (the "nothing fancy" approach)
> + additionally, using a patch-queue system (but which one?) to
> manage patches locally
> - managing debian/changelog with git-dch
>
> Some people expressed interest in starting a proper documentation on
> that on the Debian wiki. I suggest to use the
> http://wiki.debian.org/GitPackagingWorkflow (or maybe to extend the
> http://wiki.debian.org/PackagingWithGit page).
>
> We probably need more discussion:
> - on patch queue managers
> - on the "one-branch-per-patch" approaches
Eclipse recently moved to one-branch-per-patch, or at least one branch
per issue being fixed. This seems to be the fashion.

David
>
> Also, Raphael Hertzog mentioned that the next version of dpkg will
> unapply patches and remove .pc after the build, which should help with
> managing patches in git.
>
> - Lucas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201107301626.30124.david.goodenough@btconnect.com" >http://lists.debian.org/201107301626.30124.david.goodenough@btconnect.com
 
Old 07-30-2011, 03:55 PM
Benjamin Drung
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

Am Samstag, den 30.07.2011, 16:26 +0100 schrieb David Goodenough:
> On Saturday 30 Jul 2011, Lucas Nussbaum wrote:
> > On 28/07/11 at 15:36 +0200, Lucas Nussbaum wrote:
> > > Hi,
> > >
> > > It seems that many different teams have recently switched to Git, or are
> > > considering switching. But each of them seems to re-implement the wheel
> > > by designing their own Git workflow.
> > >
> > > I think that it would be fantastic if we could collaborate on defining
> > > a common Git workflow.
> > >
> > > [...]
> > >
> > > If you are interested in participating (which doesn't mean that you
> > > commit on behalf of your team to switch to that workflow, just that you
> > > want to participate in the discussions), add yourself (and your team) to
> > > http://wiki.debian.org/GitPackagingWorkflow. If more than 3 teams are
> > > interested in a BOF at DC11, I'll try to organize one on saturday (all
> > > the video-broadcasted slots are taken, but we will make sure to take
> > > notes). If the BOF doesn't happen, I'll contact interested people to see
> > > how to continue.
> >
> > Hi,
> >
> > So a BOF happened at Debconf 11, and I've put the raw notes online at
> > http://wiki.debian.org/GitPackagingWorkflow/DebConf11BOF
> >
> > My understanding of the situation is that we are ready to document a
> > workflow including:
> > - using git-buildpackage for basic stuff
> > - managing several repositories with mr
> > - managing and tracking the status of the repositories (e.g with PET)
> > - managing patches with quilt (the "nothing fancy" approach)
> > + additionally, using a patch-queue system (but which one?) to
> > manage patches locally
> > - managing debian/changelog with git-dch
> >
> > Some people expressed interest in starting a proper documentation on
> > that on the Debian wiki. I suggest to use the
> > http://wiki.debian.org/GitPackagingWorkflow (or maybe to extend the
> > http://wiki.debian.org/PackagingWithGit page).
> >
> > We probably need more discussion:
> > - on patch queue managers
> > - on the "one-branch-per-patch" approaches
> Eclipse recently moved to one-branch-per-patch, or at least one branch
> per issue being fixed. This seems to be the fashion.

Did we?

--
Benjamin Drung
Debian & Ubuntu Developer
 
Old 07-30-2011, 04:54 PM
David Goodenough
 
Default Call for teams interested in collaborating on a 'standard' Git workflow

On Saturday 30 Jul 2011, Benjamin Drung wrote:
> Am Samstag, den 30.07.2011, 16:26 +0100 schrieb David Goodenough:
> > On Saturday 30 Jul 2011, Lucas Nussbaum wrote:
> > > On 28/07/11 at 15:36 +0200, Lucas Nussbaum wrote:
> > > > Hi,
> > > >
> > > > It seems that many different teams have recently switched to Git, or
> > > > are considering switching. But each of them seems to re-implement
> > > > the wheel by designing their own Git workflow.
> > > >
> > > > I think that it would be fantastic if we could collaborate on
> > > > defining a common Git workflow.
> > > >
> > > > [...]
> > > >
> > > > If you are interested in participating (which doesn't mean that you
> > > > commit on behalf of your team to switch to that workflow, just that
> > > > you want to participate in the discussions), add yourself (and your
> > > > team) to http://wiki.debian.org/GitPackagingWorkflow. If more than
> > > > 3 teams are interested in a BOF at DC11, I'll try to organize one on
> > > > saturday (all the video-broadcasted slots are taken, but we will
> > > > make sure to take notes). If the BOF doesn't happen, I'll contact
> > > > interested people to see how to continue.
> > >
> > > Hi,
> > >
> > > So a BOF happened at Debconf 11, and I've put the raw notes online at
> > > http://wiki.debian.org/GitPackagingWorkflow/DebConf11BOF
> > >
> > > My understanding of the situation is that we are ready to document a
> > > workflow including:
> > > - using git-buildpackage for basic stuff
> > > - managing several repositories with mr
> > > - managing and tracking the status of the repositories (e.g with PET)
> > > - managing patches with quilt (the "nothing fancy" approach)
> > >
> > > + additionally, using a patch-queue system (but which one?) to
> > >
> > > manage patches locally
> > >
> > > - managing debian/changelog with git-dch
> > >
> > > Some people expressed interest in starting a proper documentation on
> > > that on the Debian wiki. I suggest to use the
> > > http://wiki.debian.org/GitPackagingWorkflow (or maybe to extend the
> > > http://wiki.debian.org/PackagingWithGit page).
> > >
> > > We probably need more discussion:
> > > - on patch queue managers
> > > - on the "one-branch-per-patch" approaches
> >
> > Eclipse recently moved to one-branch-per-patch, or at least one branch
> > per issue being fixed. This seems to be the fashion.
>
> Did we?
If I remember correctly part of the Indego annoucement was for the Mylyn
Git support was to move to one branch per issue. When you open an issue to
work on it, it creates a branch, and then merges it back in when you close
the ticket.

David


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201107301754.41515.david.goodenough@btconnect.com" >http://lists.debian.org/201107301754.41515.david.goodenough@btconnect.com
 

Thread Tools




All times are GMT. The time now is 05:43 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org