FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 07-23-2008, 02:54 AM
Doug Ledford
 
Default RFC: Exploded source repo layouts

I've been working on getting this set up and functional. The first
thing that pops up as an issue is the fact that when dealing with
exploded source repos, you really want the topdir to be the same as
upstream. That means that given a normal upstream source layout, the
topdir is the same as the traditional RPM_BUILD_DIR.

This promotes the most seamless integration with upstream, and the
greatest ability to share things with upstream. It does, however,
present problems for our custom make targets. So, what I've been
working with is a sort of home grown standard. Obviously, no one to
date has really been trying to handle things the way I'm doing now, or
at least no one has attempted to even try and standardize it. We have
had our dist-cvs setup, and other distros have had their internal
setups, and the pkg-vcs people have been talking about other similar
systems to our dist system. What people haven't been talking about is
exploded source repos where the primary look and feel of the repo is
just like a bare upstream repo, and where our distro specific stuff is
an add-on to that upstream repo. That's what I'm aiming for.

Here's how I've been doing it so far.

First, we have to do away with a bunch of our make targets because they
are too commonly named and might conflict with legitimate upstream make
targets (well, more like some of them might conflict, some of them
*definitely* conflict). This is important because our Makefile.common
is included in upstream's toplevel Makefile.

Second, upstream probably doesn't want to deal with trying to
accommodate Fedora, Unbuntu, Gentoo, etc., so we need a simple way that
upstream can accommodate us without putting a bunch of stress on them.
My method for dealing with that is that everything Fedora specific (and
specific to any other distro) goes into $topdir/distropkg. That's where
you'll find a simple Makefile that includes our Makefile.common, that's
where our spec file goes, that's where any custom source files go, and
embedded in that simple distropkg/Makefile are any custom install or
make targets that we need for this specific package (there is an example
of this in the mdadm/distropkg/Makefile from the git repo below).

The standard, as far as upstream is concerned, is this: everything
distro specific goes into distropkg and the top level Makefile should
include distropkg/Makefile if it exists (or alternatively, upstream can
provide their own empty distropkg/Makefile and include it
unconditionally in the toplevel Makefile if they want to save the
overhead of testing for the distropkg/Makefile).

As far as we are concerned, we make the majority of our custom changes,
the stuff that we don't expect upstream to ever take because it simply
is our goo that we use when building packages, in the distropkg
directory and Makefile. Anything that upstream should take, get's done
in the main source code repository and pushed upstream (with a git repo
that's really easy...make the patch, commit, cherry pick to a
for-upstream branch, email upstream about changesets, done).

As far as Ubuntu, debian, others are concerned, they do the same thing.
We each use distropkg to our own ends, and changes in that area are
things we keep local and never send upstream.

If people would like to see some examples of what I'm talking about, you
can clone the following:

git://fedorapeople.org/~dledford/mdadm.git
git://fedorapeople.org/~dledford/common.git

and see what things look like so far.

Note: this is not yet a functional, building setup. For starters, I
still need to add some custom headers to rpm (and add an rpm repo to the
list above to other people can test with the modified rpm). Then I need
to make a modified koji so I can build this into a build system. The
above packages give a decent idea of what the repos would look like, and
some of the make targets actually work (like the make tarball target).
It's mainly there for people to take a look at things like the spec file
and the filesystem layout and to provide feedback based upon what they
see before things are so far along that it's hard to change anything.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 07:43 AM
Hans de Goede
 
Default RFC: Exploded source repo layouts

Doug Ledford wrote:

I've been working on getting this set up and functional.


<lots of complicated hacks and workarounds deleted>

So Far I've been quiet on this, sort of hoping it would go away by itself, but
as a contributor with quite a few packages let me say that I'm deeply worried
about this whole distributed VCS / exploded source idea floating around.


It seems there are a few people who are a big fan of this, and about as much
active opponents. I have no problems with adding the possibility to use a
distributed VCS with exploded trees to the mix of ways to maintain and build
packages, but this should not replace the current nice and simple setup we have.


First of all it does NOT match the way rpm was designed at all, rpm is about
pristine sources with _separate_ patches, but most importantly, this is rather
complicated making things unnecessary hard for people who don't want to do the
stuff some of the distributed VCS proponents want to do. This worries me, I'm
esp. worried that the barrier of entry to becoming a Fedora packager will be
raised significantly.


Also I even fail to see the claimed advantages in using a distributed VCS at
all, isn't our mantra upstream upstream upstream, well if this mantra is
properly followed and upstream is undergoing active development then most of
the time the pristine sources should be fine without any patches at all, since
all patches are integrated upstream in a timely manner. Also if someone wants
to do so much work on the upstrewam sources, then he/she should just become an
upstream developer. Really getting upstream cvs/svn/whatever access isn't that
hard, then one can directly commit one's changes in to upstreams VCS.


Regards,

Hans

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 08:16 AM
Andreas Ericsson
 
Default RFC: Exploded source repo layouts

Hans de Goede wrote:

Doug Ledford wrote:

I've been working on getting this set up and functional.


<lots of complicated hacks and workarounds deleted>

So Far I've been quiet on this, sort of hoping it would go away by
itself, but as a contributor with quite a few packages let me say that
I'm deeply worried about this whole distributed VCS / exploded source
idea floating around.




Does any of those upstream packages use a dvcs as their source control
tool?

It seems there are a few people who are a big fan of this, and about as
much active opponents. I have no problems with adding the possibility to
use a distributed VCS with exploded trees to the mix of ways to maintain
and build packages, but this should not replace the current nice and
simple setup we have.




Noone has proposed replacing the existing setup, so the rest of your
email can safely be delegated to /dev/null.

--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 08:39 AM
yersinia
 
Default RFC: Exploded source repo layouts

I am not think the debian borg are crazy because they* have svn-buildpackage, cvs-buildpackage, git-buildpackage
from years(http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html). Doesn't exists a rpm equivalent - or

better an rpmbuild integration with vcs - only because of rpm fragmentation and, i think, because every other distro want a
competive advantage in using the their "buildsystem". But, in 2008, it is time to gave to all the rpm packager an unified vcs integration with RPM


JMHO, YMMV



On Wed, Jul 23, 2008 at 9:43 AM, Hans de Goede <j.w.r.degoede@hhs.nl> wrote:

Doug Ledford wrote:


I've been working on getting this set up and functional.




<lots of complicated hacks and workarounds deleted>



So Far I've been quiet on this, sort of hoping it would go away by itself, but as a contributor with quite a few packages let me say that I'm deeply worried about this whole distributed VCS / exploded source idea floating around.




It seems there are a few people who are a big fan of this, and about as much active opponents. I have no problems with adding the possibility to use a distributed VCS with exploded trees to the mix of ways to maintain and build packages, but this should not replace the current nice and simple setup we have.




First of all it does NOT match the way rpm was designed at all, rpm is about pristine sources with _separate_ patches, but most importantly, this is rather complicated making things unnecessary hard for people who don't want to do the stuff some of the distributed VCS proponents want to do. This worries me, I'm esp. worried that the barrier of entry to becoming a Fedora packager will be raised significantly.




Also I even fail to see the claimed advantages in using a distributed VCS at all, isn't our mantra upstream upstream upstream, well if this mantra is properly followed and upstream is undergoing active development then most of the time the pristine sources should be fine without any patches at all, since all patches are integrated upstream in a timely manner. Also if someone wants to do so much work on the upstrewam sources, then he/she should just become an upstream developer. Really getting upstream cvs/svn/whatever access isn't that hard, then one can directly commit one's changes in to upstreams VCS.




Regards,



Hans



--

fedora-devel-list mailing list

fedora-devel-list@redhat.com

https://www.redhat.com/mailman/listinfo/fedora-devel-list



--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 02:51 PM
"Dominik 'Rathann' Mierzejewski"
 
Default RFC: Exploded source repo layouts

On Wednesday, 23 July 2008 at 09:43, Hans de Goede wrote:
> Doug Ledford wrote:
> >I've been working on getting this set up and functional.
>
> <lots of complicated hacks and workarounds deleted>
>
> So Far I've been quiet on this, sort of hoping it would go away by itself,
> but as a contributor with quite a few packages let me say that I'm deeply
> worried about this whole distributed VCS / exploded source idea floating
> around.
>
> It seems there are a few people who are a big fan of this, and about as
> much active opponents. I have no problems with adding the possibility to
> use a distributed VCS with exploded trees to the mix of ways to maintain
> and build packages, but this should not replace the current nice and simple
> setup we have.

Seconded. I'm pretty happy with our current workflow and the only things
I'm missing are "svn cp" and "svn mv".

[...]
> Also I even fail to see the claimed advantages in using a distributed VCS
> at all, isn't our mantra upstream upstream upstream, well if this mantra is
> properly followed and upstream is undergoing active development then most
> of the time the pristine sources should be fine without any patches at all,
> since all patches are integrated upstream in a timely manner. Also if
> someone wants to do so much work on the upstrewam sources, then he/she
> should just become an upstream developer. Really getting upstream
> cvs/svn/whatever access isn't that hard, then one can directly commit one's
> changes in to upstreams VCS.

Apparently a couple of packages (grub, kernel, few others?) maintain a massive
patch set over their respective upstream tarballs, so while this change might
make life easier for them, it is not necessarily so for the rest of us.

Make it an option, yes, but do not disturb the workflow of the majority
for the benefit of the few.

Regards,
R.

--
Fedora http://fedoraproject.org/wiki/User:Rathann
Livna http://rpm.livna.org | MPlayer http://mplayerhq.hu
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 02:55 PM
"Dominik 'Rathann' Mierzejewski"
 
Default RFC: Exploded source repo layouts

On Wednesday, 23 July 2008 at 10:16, Andreas Ericsson wrote:
> Hans de Goede wrote:
> >Doug Ledford wrote:
> >>I've been working on getting this set up and functional.
> >
> ><lots of complicated hacks and workarounds deleted>
[...]
> >It seems there are a few people who are a big fan of this, and about as
> >much active opponents. I have no problems with adding the possibility to
> >use a distributed VCS with exploded trees to the mix of ways to maintain
> >and build packages, but this should not replace the current nice and
> >simple setup we have.
> >
>
> Noone has proposed replacing the existing setup, so the rest of your
> email can safely be delegated to /dev/null.

Let me provide the missing quote then:

> >>First, we have to do away with a bunch of our make targets because they
> >>are too commonly named and might conflict with legitimate upstream make
> >>targets (well, more like some of them might conflict, some of them
> >>*definitely* conflict). This is important because our Makefile.common
> >>is included in upstream's toplevel Makefile.

If that doesn't touch our existing setup then fine, but I'm afraid
Doug really meant what he wrote.

Regards,
R.

--
Fedora http://fedoraproject.org/wiki/User:Rathann
Livna http://rpm.livna.org | MPlayer http://mplayerhq.hu
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 02:58 PM
Doug Ledford
 
Default RFC: Exploded source repo layouts

On Wed, 2008-07-23 at 16:55 +0200, Dominik 'Rathann' Mierzejewski wrote:
> On Wednesday, 23 July 2008 at 10:16, Andreas Ericsson wrote:
> > Hans de Goede wrote:
> > >Doug Ledford wrote:
> > >>I've been working on getting this set up and functional.
> > >
> > ><lots of complicated hacks and workarounds deleted>
> [...]
> > >It seems there are a few people who are a big fan of this, and about as
> > >much active opponents. I have no problems with adding the possibility to
> > >use a distributed VCS with exploded trees to the mix of ways to maintain
> > >and build packages, but this should not replace the current nice and
> > >simple setup we have.
> > >
> >
> > Noone has proposed replacing the existing setup, so the rest of your
> > email can safely be delegated to /dev/null.
>
> Let me provide the missing quote then:
>
> > >>First, we have to do away with a bunch of our make targets because they
> > >>are too commonly named and might conflict with legitimate upstream make
> > >>targets (well, more like some of them might conflict, some of them
> > >>*definitely* conflict). This is important because our Makefile.common
> > >>is included in upstream's toplevel Makefile.
>
> If that doesn't touch our existing setup then fine, but I'm afraid
> Doug really meant what he wrote.

Actually, no. For Makefile.dist-CVS, it is the same as it used to be.
Same make targets, same functionality, same, same, same. It's
Makefile.repo-git that gets included in the exploded source's top level
Makefile, and *that's* where you have new make targets and such.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 03:06 PM
"Dominik 'Rathann' Mierzejewski"
 
Default RFC: Exploded source repo layouts

On Wednesday, 23 July 2008 at 16:58, Doug Ledford wrote:
> On Wed, 2008-07-23 at 16:55 +0200, Dominik 'Rathann' Mierzejewski wrote:
> > On Wednesday, 23 July 2008 at 10:16, Andreas Ericsson wrote:
> > > Hans de Goede wrote:
> > > >Doug Ledford wrote:
> > > >>I've been working on getting this set up and functional.
> > > >
> > > ><lots of complicated hacks and workarounds deleted>
> > [...]
> > > >It seems there are a few people who are a big fan of this, and about as
> > > >much active opponents. I have no problems with adding the possibility to
> > > >use a distributed VCS with exploded trees to the mix of ways to maintain
> > > >and build packages, but this should not replace the current nice and
> > > >simple setup we have.
> > > >
> > >
> > > Noone has proposed replacing the existing setup, so the rest of your
> > > email can safely be delegated to /dev/null.
> >
> > Let me provide the missing quote then:
> >
> > > >>First, we have to do away with a bunch of our make targets because they
> > > >>are too commonly named and might conflict with legitimate upstream make
> > > >>targets (well, more like some of them might conflict, some of them
> > > >>*definitely* conflict). This is important because our Makefile.common
> > > >>is included in upstream's toplevel Makefile.
> >
> > If that doesn't touch our existing setup then fine, but I'm afraid
> > Doug really meant what he wrote.
>
> Actually, no. For Makefile.dist-CVS, it is the same as it used to be.
> Same make targets, same functionality, same, same, same. It's
> Makefile.repo-git that gets included in the exploded source's top level
> Makefile, and *that's* where you have new make targets and such.

So... am I going to have to type
make -f Makefile.dist-CVS targetfoo
after this change?

Regards,
R.

--
Fedora http://fedoraproject.org/wiki/User:Rathann
Livna http://rpm.livna.org | MPlayer http://mplayerhq.hu
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 03:22 PM
Doug Ledford
 
Default RFC: Exploded source repo layouts

On Wed, 2008-07-23 at 16:51 +0200, Dominik 'Rathann' Mierzejewski wrote:
> On Wednesday, 23 July 2008 at 09:43, Hans de Goede wrote:
> > Doug Ledford wrote:
> > >I've been working on getting this set up and functional.
> >
> > <lots of complicated hacks and workarounds deleted>
> >
> > So Far I've been quiet on this, sort of hoping it would go away by itself,
> > but as a contributor with quite a few packages let me say that I'm deeply
> > worried about this whole distributed VCS / exploded source idea floating
> > around.
> >
> > It seems there are a few people who are a big fan of this, and about as
> > much active opponents. I have no problems with adding the possibility to
> > use a distributed VCS with exploded trees to the mix of ways to maintain
> > and build packages, but this should not replace the current nice and simple
> > setup we have.
>
> Seconded. I'm pretty happy with our current workflow and the only things
> I'm missing are "svn cp" and "svn mv".
>
> [...]
> > Also I even fail to see the claimed advantages in using a distributed VCS
> > at all, isn't our mantra upstream upstream upstream, well if this mantra is
> > properly followed and upstream is undergoing active development then most
> > of the time the pristine sources should be fine without any patches at all,
> > since all patches are integrated upstream in a timely manner. Also if
> > someone wants to do so much work on the upstrewam sources, then he/she
> > should just become an upstream developer. Really getting upstream
> > cvs/svn/whatever access isn't that hard, then one can directly commit one's
> > changes in to upstreams VCS.
>
> Apparently a couple of packages (grub, kernel, few others?) maintain a massive
> patch set over their respective upstream tarballs, so while this change might
> make life easier for them, it is not necessarily so for the rest of us.

Actually, no. A massive patch set is no easier to maintain in a dvcs
than it is with patches. The problem is that with a massive patchset,
you get conflicts on update. This happens whether the patches are
applied in an rpm, or by the dvcs as part of an update process.

> Make it an option, yes, but do not disturb the workflow of the majority
> for the benefit of the few.

I won't force anyone to anything. Of course, a number of the benefits I
laid out in the rpm thread have nothing to do with packaging and have
everything to do with things like look aside cache growth, source
storage size, and limitations of Fedora's infrastructure. Those items,
obviously, are of benefit to *all* packages, not just ones those with
active developers handling the package. And in order for Fedora to even
consider the idea of hosting source for spin makers, something like this
would be a requirement. But, obviously, that would be disturbing the
majority for the benefit of the few.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 07-23-2008, 03:26 PM
Doug Ledford
 
Default RFC: Exploded source repo layouts

On Wed, 2008-07-23 at 17:06 +0200, Dominik 'Rathann' Mierzejewski wrote:
> On Wednesday, 23 July 2008 at 16:58, Doug Ledford wrote:
> > On Wed, 2008-07-23 at 16:55 +0200, Dominik 'Rathann' Mierzejewski wrote:
> > > On Wednesday, 23 July 2008 at 10:16, Andreas Ericsson wrote:
> > > > Hans de Goede wrote:
> > > > >Doug Ledford wrote:
> > > > >>I've been working on getting this set up and functional.
> > > > >
> > > > ><lots of complicated hacks and workarounds deleted>
> > > [...]
> > > > >It seems there are a few people who are a big fan of this, and about as
> > > > >much active opponents. I have no problems with adding the possibility to
> > > > >use a distributed VCS with exploded trees to the mix of ways to maintain
> > > > >and build packages, but this should not replace the current nice and
> > > > >simple setup we have.
> > > > >
> > > >
> > > > Noone has proposed replacing the existing setup, so the rest of your
> > > > email can safely be delegated to /dev/null.
> > >
> > > Let me provide the missing quote then:
> > >
> > > > >>First, we have to do away with a bunch of our make targets because they
> > > > >>are too commonly named and might conflict with legitimate upstream make
> > > > >>targets (well, more like some of them might conflict, some of them
> > > > >>*definitely* conflict). This is important because our Makefile.common
> > > > >>is included in upstream's toplevel Makefile.
> > >
> > > If that doesn't touch our existing setup then fine, but I'm afraid
> > > Doug really meant what he wrote.
> >
> > Actually, no. For Makefile.dist-CVS, it is the same as it used to be.
> > Same make targets, same functionality, same, same, same. It's
> > Makefile.repo-git that gets included in the exploded source's top level
> > Makefile, and *that's* where you have new make targets and such.
>
> So... am I going to have to type
> make -f Makefile.dist-CVS targetfoo
> after this change?

No, of course not. For those people using CVS, things will be totally
transparent and they won't have to so much as read a single line of a
README file.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 

Thread Tools




All times are GMT. The time now is 06:56 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org