> I do understand the response, because part of the strategy mentioned
> *is* not to provide plain-text patches, but instead manage them,
> possibly jointly with other distributions, in a midstream repository.
Exactly.
The point is: I have to maintain lots of packages for different distros,
as well as for my own build system. I cannot do this manually for each
single distro, so I prefer doing _generic_ fixes and let the distro
buildfiles only do the plain standard build process
(eg. ./autogen.sh && ./configure && make && make install).
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
On Sun, Jul 11, 2010 at 12:39 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> The point is: I have to maintain lots of packages for different distros,
> as well as for my own build system. I cannot do this manually for each
> single distro, so I prefer doing _generic_ fixes and let the distro
> buildfiles only do the plain standard build process
> (eg. ./autogen.sh && ./configure && make && make install).
>
>
I know the problem you are trying to solve is quite important.
Various distros apply their own patches ontop of products, which often
leads to duplicated work, regressions and even security bugs, all
because there aren't enough eyeballs. We need a solution for this.
Now, in a perfect world, we wouldn't need this, since upstream would
quickly merge bugfixes, and patches would only have to be kept till
the next release. However, we have seen that upstreams are often
stubborn, or don't agree with us. Some distros (like Gentoo,
Slackware, ArchLinux) prefer to keep patching to a *minimum*. A lot of
teams have a policy of "upstream first" before adding a patch to the
tree (unless it is small/hackish/temporary, very distro specific, and
has no place upstream).
On the other hand, Debian/Ubuntu like to carry a thousand lines of
patches on top of packages like Firefox, and Ubuntu likes to diverge
/majorly/ from upstream releases. Fedora also suffers from this
problem of not agreeing with upstream quite often[1]; but w.r.t.
GNOME/Xorg/udev they *are* upstream, so the effect is lessened. IIRC,
OpenSuse has had a good policy of upstreaming often.
If I understand your system correctly, you essentially maintain clones
of upstream repos, with all the various distro patches applied on top,
and release tarballs as well. I don't see how these various distros
can be made to agree with each other and I certainly can't see them
using a common tarball source. On a technical level, it's got serious
security, trust, and redundancy problems. It is extremely important
that distros collaborate in some form when it comes to patches that
*can* be shared, but the solution you have devised is fundamentally
flawed.
I can say today with extreme confidence that no major distros will use
your repositories and your tarballs, or work with you to collaborate
with each other. There is simply very little incentive to change the
way we work.
A practical solution to the problem of patch sharing is to have a
website with a search interface for upstream source tarballs, which
can display all the patches that various distros apply, as well as a
download link for the patchsets (hotlinked to the distro files where
possible). Currently, it is extremely painful to find out what patches
each distro has applied on top of a give source package, and as a
result, collaboration is minimal.
As the popularity of such a website increases (you'll need to
advertise it!), you'll find that at least the divergent/community
supported distros (debian/slackware/arch/gentoo/opensolaris) will
start using it heavily and sharing of patches (wherever feasible) will
occur automatically.
Distro packagers are much more comfortable with downloading patchsets
from a foreign source than complete tarballs. They already have a
trust-relationship with upstream, and they can verify the integrity of
a patchset much easier than an arbitrary tarball.
I know you have spent a lot of time on this already, but please
understand it from where we stand. We're short on manpower, and
there's no real benefits of shifting our tarball source; OTOH there
are major disadvantages too unless we pitch in with manpower
ourselves. And honestly speaking, that manpower is better spent making
stuff work locally.
Please consider the "patch-website" idea above. We definitely need
someone to code it up, gather the source-package to distro patches
mappings,
and advertise it. And this is a solution that can easily be created
and maintained by one man alone, and will solve the problem of distro
collaboration as well!
Best,
1. https://bugs.gentoo.org/291328
--
~Nirbheek Chauhan
Gentoo GNOME+Mozilla Team
07-11-2010, 09:51 AM
Duncan
Nirbheek Chauhan posted on Sun, 11 Jul 2010 14:28:35 +0530 as excerpted:
> ...
> Please consider the "patch-website" idea above. We definitely need
> someone to code it up, gather the source-package to distro patches
> mappings,
> ...
Thanks for writing this up. Well said. =:^)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
07-11-2010, 09:53 AM
Enrico Weigelt
* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
> If I understand your system correctly, you essentially maintain clones
> of upstream repos, with all the various distro patches applied on top,
> and release tarballs as well.
Yes. And if some upstream does not provide suitable vcs access
(or doesnt even have one), I make an pseudo-upstream branch by
committing the release source trees.
> I don't see how these various distros can be made to agree with
> each other and I certainly can't see them using a common tarball
> source.
Thats not even necessary. They just should use the infrastructure,
as described in my paper. So everyone can easily set up automatic
notifications, cherry-pick, etc, etc.
> On a technical level, it's got serious security, trust, and
> redundancy problems.
Git makes that very easy ;-p
> It is extremely important that distros collaborate in some form
> when it comes to patches that *can* be shared,
If we're doing a good job (my generic fixes instead of distro-
specfic dirty hacks) about 99% can be shared ;-p
> but the solution you have devised is fundamentally flawed.
If it's really flawed, then just for pure "social" reasons, no
serious technical ones.
> A practical solution to the problem of patch sharing is to
> have a website with a search interface for upstream source
> tarballs, which can display all the patches that various
> distros apply, as well as a download link for the patchsets
> (hotlinked to the distro files where possible).
Too complicated, and actually would not help me a single bit.
What I could offer is an (semi-)automatic import mechanism
(assuming certain package managers dont do such insane things
like directly sed'ing sources etc) - there's still a bunch of
work to do for that, but its possible.
> Distro packagers are much more comfortable with downloading
> patchsets from a foreign source than complete tarballs.
man git-format-patch ;-p
Maybe I could set up an git webfrontend (or automatically push
to some public service).
> I know you have spent a lot of time on this already, but please
> understand it from where we stand. We're short on manpower, and
> there's no real benefits of shifting our tarball source; OTOH there
> are major disadvantages too unless we pitch in with manpower
> ourselves. And honestly speaking, that manpower is better spent making
> stuff work locally.
Well, Gentoo is short of manpower ? hmm, perhaps some should think
about why so many folks are resigning and so few fresh coming in
(at least according to this lists traffic) ;-O
> Please consider the "patch-website" idea above. We definitely need
> someone to code it up, gather the source-package to distro patches
> mappings, and advertise it.
Actually, I once had somehing in that area, called "comprehensive
source database", but unfurtinately it got lost in an disk array
crash a few years ago, and I didnt find the time to rewrite it yet.
Meanwhile I dont need it anymore, since I gave up maintaining
plaintext patches in favour of git. And that makes my daily works
_much_ easier.
Oh, btw: I'm announcing my oss-qm releases via twitter:
http://twitter.com/oss_qm
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
On Sun, Jul 11, 2010 at 3:23 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>> I don't see how these various distros can be made to agree with
>> each other and I certainly can't see them using a common tarball
>> source.
>
> Thats not even necessary. They just should use the infrastructure,
> as described in my paper. So everyone can easily set up automatic
> notifications, cherry-pick, etc, etc.
>
Why should we? I am *yet* to see a single reason for us to change how
we work other than "please use this since I've been putting a lot of
effort into it".
>> On a technical level, it's got serious security, trust, and
>> redundancy problems.
>
> Git makes that very easy ;-p
>
No, it does not. The security problems come because you are the single
point of failure. The trust problems come because we have no reason to
trust you. The redundancy problems come because if your hosting goes
down or you lose interest, we're left high and dry. Git has nothing to
do with any of this.
>> It is extremely important that distros collaborate in some form
>> when it comes to patches that *can* be shared,
>
> If we're doing a good job (my generic fixes instead of distro-
> specfic dirty hacks) about 99% can be shared ;-p
>
I'd advise you to take a look at the sort of patching Ubuntu/Debian
does, and then revisit that figure. You'll find it more along the
lines of 30%.
>> A practical solution to the problem of patch sharing is to
>> have a website with a search interface for upstream source
>> tarballs, which can display all the patches that various
>> distros apply, as well as a download link for the patchsets
>> (hotlinked to the distro files where possible).
>
> Too complicated, and actually would not help me a single bit.
Help *you*? I thought this was about helping the distros. If your
proposal is not about making our work easier, please don't waste our
time.
>> Distro packagers are much more comfortable with downloading
>> patchsets from a foreign source than complete tarballs.
>
> man git-format-patch ;-p
>
So why don't you submit that to bugzilla?
>> I know you have spent a lot of time on this already, but please
>> understand it from where we stand. We're short on manpower, and
>> there's no real benefits of shifting our tarball source; OTOH there
>> are major disadvantages too unless we pitch in with manpower
>> ourselves. And honestly speaking, that manpower is better spent making
>> stuff work locally.
>
> Well, Gentoo is short of manpower ? hmm, perhaps some should think
> about why so many folks are resigning and so few fresh coming in
> (at least according to this lists traffic) ;-O
>
I'm beginning to think that you're not taking my honest advice very seriously.
>> Please consider the "patch-website" idea above. We definitely need
>> someone to code it up, gather the source-package to distro patches
>> mappings, and advertise it.
>
> Actually, I once had somehing in that area, called "comprehensive
> source database", but unfurtinately it got lost in an disk array
> crash a few years ago, and I didnt find the time to rewrite it yet.
>
> Meanwhile I dont need it anymore, since I gave up maintaining
> plaintext patches in favour of git. And that makes my daily works
> _much_ easier.
>
You don't need to maintain **anything** manually if you code the
website properly. That's the whole point. You get major benefits with
minimal long-term work which can be done by a single person in their
free time.
This job is easily automated to simply aggregate links to patches
which all the distros manually publish themselves. For Gentoo, it's
the ebuilds; for Debian/Ubuntu, they actually publish the diffs[1];
Fedora keeps its patches in a common CVS repo[2], etc etc. Once the
website is up and running, maintenance is minimal, and can be done by
a single person looking at it in their free time.
1. See packages.(debian|ubuntu).(org|com)
2. cvs.fedoraproject.org/viewvc/
--
~Nirbheek Chauhan
Gentoo GNOME+Mozilla Team
07-11-2010, 04:53 PM
"Jorge Manuel B. S. Vicetto"
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Nirbheek,
thanks for writing such a well thought-out and comprehensive reply to
Enrico. I agree with all the points you raised.
On 11-07-2010 10:28, Nirbheek Chauhan wrote:
> On Sun, Jul 11, 2010 at 3:23 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>>> I don't see how these various distros can be made to agree with
>>> each other and I certainly can't see them using a common tarball
>>> source.
>>
>> Thats not even necessary. They just should use the infrastructure,
>> as described in my paper. So everyone can easily set up automatic
>> notifications, cherry-pick, etc, etc.
>>
>
> Why should we? I am *yet* to see a single reason for us to change how
> we work other than "please use this since I've been putting a lot of
> effort into it".
Enrico,
just because you decided to "offer" some service, it in no way "forces
us" to accept it.
>>> On a technical level, it's got serious security, trust, and
>>> redundancy problems.
>>
>> Git makes that very easy ;-p
>>
>
> No, it does not. The security problems come because you are the single
> point of failure. The trust problems come because we have no reason to
> trust you. The redundancy problems come because if your hosting goes
> down or you lose interest, we're left high and dry. Git has nothing to
> do with any of this.
These 3 issues are in my view the most important issues regarding your
proposal and I agree with Nirbheek's reply.
With your proposal, you're expecting us (distro maintainers) to put our
trust and our users safety in you and your service - what made you think
we would or should do it? Besides, what significant gain would we have
to justify trusting your service?
>>> It is extremely important that distros collaborate in some form
>>> when it comes to patches that *can* be shared,
>>
>> If we're doing a good job (my generic fixes instead of distro-
>> specfic dirty hacks) about 99% can be shared ;-p
>>
>
> I'd advise you to take a look at the sort of patching Ubuntu/Debian
> does, and then revisit that figure. You'll find it more along the
> lines of 30%.
>
>
>>> A practical solution to the problem of patch sharing is to
>>> have a website with a search interface for upstream source
>>> tarballs, which can display all the patches that various
>>> distros apply, as well as a download link for the patchsets
>>> (hotlinked to the distro files where possible).
>>
>> Too complicated, and actually would not help me a single bit.
>
> Help *you*? I thought this was about helping the distros. If your
> proposal is not about making our work easier, please don't waste our
> time.
>
>>> Distro packagers are much more comfortable with downloading
>>> patchsets from a foreign source than complete tarballs.
>>
>> man git-format-patch ;-p
>>
>
> So why don't you submit that to bugzilla?
Please don't assume replies are based solely on people not knowing the
tools (git in this case).
>>> I know you have spent a lot of time on this already, but please
>>> understand it from where we stand. We're short on manpower, and
>>> there's no real benefits of shifting our tarball source; OTOH there
>>> are major disadvantages too unless we pitch in with manpower
>>> ourselves. And honestly speaking, that manpower is better spent making
>>> stuff work locally.
>>
>> Well, Gentoo is short of manpower ? hmm, perhaps some should think
>> about why so many folks are resigning and so few fresh coming in
>> (at least according to this lists traffic) ;-O
This type of argument when you're trying to convince others to use your
service is in the least a disservice to your purpose.
> I'm beginning to think that you're not taking my honest advice very seriously.
>
>>> Please consider the "patch-website" idea above. We definitely need
>>> someone to code it up, gather the source-package to distro patches
>>> mappings, and advertise it.
>>
>> Actually, I once had somehing in that area, called "comprehensive
>> source database", but unfurtinately it got lost in an disk array
>> crash a few years ago, and I didnt find the time to rewrite it yet.
>>
>> Meanwhile I dont need it anymore, since I gave up maintaining
>> plaintext patches in favour of git. And that makes my daily works
>> _much_ easier.
>>
>
> You don't need to maintain **anything** manually if you code the
> website properly. That's the whole point. You get major benefits with
> minimal long-term work which can be done by a single person in their
> free time.
>
> This job is easily automated to simply aggregate links to patches
> which all the distros manually publish themselves. For Gentoo, it's
> the ebuilds; for Debian/Ubuntu, they actually publish the diffs[1];
> Fedora keeps its patches in a common CVS repo[2], etc etc. Once the
> website is up and running, maintenance is minimal, and can be done by
> a single person looking at it in their free time.
>
> 1. See packages.(debian|ubuntu).(org|com)
> 2. cvs.fedoraproject.org/viewvc/
>
- --
Regards,
Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
I have managed to find a work-around to the latest kernel failing to boot on my
Toshiba laptop. Before downgrading, I tried chrooting the system and rebuilding
the initramfs, but no luck it stopped at the same place on boot -- Setting up
UTF-8 mode.
So I booted to the dual-install media, chrooted the system and downgraded to
kernel26-2.6.34-2. This kernel boots fine.
There is a problem with kernel26-2.6.34.1-1-x86_64. Since I can't boot it, I'm
not sure what testing I can do, but if you can think of one, I'm happy to run
it. This laptop is a Toshiba 205d. It hasn't ever had a problem with any other
Arch or SuSE kernel, so I'm not sure what to tell you.
I've posted the dmidecode information in case that will help. It is here:
--
David C. Rankin, J.D.,P.E.
Rankin Law Firm, PLLC
510 Ochiltree Street
Nacogdoches, Texas 75961
Telephone: (936) 715-9333
Facsimile: (936) 715-9339
www.rankinlawfirm.com
07-12-2010, 02:44 AM
Enrico Weigelt
* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
> > Thats not even necessary. They just should use the infrastructure,
> > as described in my paper. So everyone can easily set up automatic
> > notifications, cherry-pick, etc, etc.
>
> Why should we?
To make tracking and applying other's changes much easier.
Currently it's quite work intensive to do so.
If - hypothetically - everyone work within an git repo, using a
normalized naming/versioning scheme, it is trivial to set some
little tracking system, informing package maintainers if something
happens (new upstream, new patches from distro XYZ, etc). And
we can inspect that easily, take patches, etc, etc, using
standard git tools.
> I am *yet* to see a single reason for us to change how we work
> other than "please use this since I've been putting a lot of
> effort into it".
I have no intention to urge you to use my approach, I'm just
offering you my works, nothing more.
> >> On a technical level, it's got serious security, trust, and
> >> redundancy problems.
> >
> > Git makes that very easy ;-p
> >
>
> No, it does not. The security problems come because you are the single
> point of failure.
Which SPOF ?
man 1 git-cone ;-p
> The trust problems come because we have no reason to trust you.
You dont have to trust me. Just let me point out some possible
leak points (if I missed some, feel free to add them ...):
a) repository manipulation (directly changing objects): will
be found out by git-gc.
b) injecting changed upstream: you can easily compare my UPSTREAM.*
tags against the real upstream's tarballs or vcs tags.
BTW: as long as not all upstreams sign their releases, our trust
relies just relies on their server's integrity (and the connection
to them).
c) manipulated tags: someone, perhaps myself overwrites other's tags
use signed tags, and check the signatures (easily doable by a
little shell script
d) some vendor (possibly myself) adds crappy changes: you'll most
likely have a look at the changes before cherry-picking them.
> The redundancy problems come because if your hosting goes
> down or you lose interest, we're left high and dry.
That wouldn't affect your clones. You simply won't get anything
new from my site anymore.
> > If we're doing a good job (my generic fixes instead of distro-
> > specfic dirty hacks) about 99% can be shared ;-p
> >
>
> I'd advise you to take a look at the sort of patching Ubuntu/Debian
> does, and then revisit that figure. You'll find it more along the
> lines of 30%.
I never claimed that Ubuntu does clean and generic fixes.
BTW: Gentoo tends to have similar problems, just look at the zlib ebuild:
>> # trust exit status of the compiler rather than stderr #55434
>> # -if test "`(...) 2>&1`" = ""; then
>> # +if (...) 2>/dev/null; then
>> sed -i 's|<test "`([^"]*) 2>&1`" = ""|1 2>/dev/null|' configure || die
>> sed -i -e '/ldconfig/d' Makefile* || die
> >> A practical solution to the problem of patch sharing is to
> >> have a website with a search interface for upstream source
> >> tarballs, which can display all the patches that various
> >> distros apply, as well as a download link for the patchsets
> >> (hotlinked to the distro files where possible).
> >
> > Too complicated, and actually would not help me a single bit.
>
> Help *you*? I thought this was about helping the distros.
Yes, I first want to solve my daily business problems, but in a way
that other folks can benefit from my works.
> >> Distro packagers are much more comfortable with downloading
> >> patchsets from a foreign source than complete tarballs.
> >
> > man git-format-patch ;-p
> >
>
> So why don't you submit that to bugzilla?
Yet too complicated/work intensive to do this for each individual
distro. It would be a completely different issue, if there was
some robot interface for that.
On the other hand, if you pull from my repo, you can easily hack
up a little bot, which tells you when something happens (or
even send you patches).
> > Meanwhile I dont need it anymore, since I gave up maintaining
> > plaintext patches in favour of git. And that makes my daily works
> > _much_ easier.
> >
>
> You don't need to maintain **anything** manually if you code the
> website properly. That's the whole point. You get major benefits with
> minimal long-term work which can be done by a single person in their
> free time.
First, I have to build that website and maintain it over a long time.
Then I'll have to do a lot of advertisement work to get people to
actually push their patches there.
On the other hand, the git-based infrastructure is already there,
people can use it right now. And it gives my exactly what I need.
So I prefer spending my time in advocating this one.
> This job is easily automated to simply aggregate links to patches
> which all the distros manually publish themselves.
It's not that simple. Many distros don't even do proper patches,
instead wildly copy over or directly sed certain sourcesfiles,
or even (like Debian) use their own broken tarballs.
(the worst srcpkg I've ever seen is Debian's mysql-5.0.32 ...)
And even *if* we assume, that everyone's just doing patches, we
still need to transform the everbody's strange (often not even
linear) versioning schemes. One of OSS-QMs primary goals is to
use a strictly normalized/canonical naming/versioning scheme in
the primay source repository.
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
* Jorge Manuel B. S. Vicetto <jmbsvicetto@gentoo.org> schrieb:
> just because you decided to "offer" some service, it in no way "forces
> us" to accept it.
No, it does not. I never claimed that.
And so really I dont understand that fundamentalistic kind of
argumentation (which is really near to ranting).
> With your proposal, you're expecting us (distro maintainers) to put our
> trust and our users safety in you and your service - what made you think
> we would or should do it?
Not at all, please refer to my other answer (some minutes earlier).
> >> Well, Gentoo is short of manpower ? hmm, perhaps some should think
> >> about why so many folks are resigning and so few fresh coming in
> >> (at least according to this lists traffic) ;-O
>
> This type of argument when you're trying to convince others to use your
> service is in the least a disservice to your purpose.
That particular argument is not about the OSS-QM project, but the
general social climate @Gentoo. Actually, when talking to collegues,
most of them share this. The most commonly used term I've heared
from them about Gentoo devs is "total lack of social competence".
(to be fair: this also applies to lots of other OSS projects).
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
