Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Gentoo Development (http://www.linux-archive.org/gentoo-development/)
-   -   proxy maintainership and gentoo-x86 scm (http://www.linux-archive.org/gentoo-development/308799-proxy-maintainership-gentoo-x86-scm.html)

Mike Frysinger 01-13-2010 02:54 PM

proxy maintainership and gentoo-x86 scm
 
On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> > If you feel like it, become a proxy-maintainer and poke a developer to
> > put your ebuilds on tree. Have you ever heard of that ? :)
>
> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
> in my opinion. It is a virtually unknown option.

i think our current work flows also significantly impede the smooth running of
this. if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
their behalf.
-mike

Ben de Groot 01-13-2010 08:02 PM

proxy maintainership and gentoo-x86 scm
 
2010/1/13 Mike Frysinger <vapier@gentoo.org>:
> On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
>> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
>> > If you feel like it, become a proxy-maintainer and poke a developer to
>> > put your ebuilds on tree. Have you ever heard of that ? :)
>>
>> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
>> in my opinion. It is a virtually unknown option.
>
> i think our current work flows also significantly impede the smooth running of
> this. *if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> their behalf.
> -mike

I couldn't agree more!

--
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
__________________________________________________ ____

justin 01-13-2010 08:18 PM

proxy maintainership and gentoo-x86 scm
 
On 13/01/10 22:02, Ben de Groot wrote:
> 2010/1/13 Mike Frysinger <vapier@gentoo.org>:
>> On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
>>> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
>>>> If you feel like it, become a proxy-maintainer and poke a developer to
>>>> put your ebuilds on tree. Have you ever heard of that ? :)
>>>
>>> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
>>> in my opinion. It is a virtually unknown option.
>>
>> i think our current work flows also significantly impede the smooth running of
>> this. if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
>> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
>> their behalf.
>> -mike
>
> I couldn't agree more!
>

So what is the current mood to switch to git. As there is no traffic on
the scm-ml I assume that there is no progress. But shouldn' we start to
move on? Now!?

Christian Faulhammer 01-13-2010 09:03 PM

proxy maintainership and gentoo-x86 scm
 
Hi,

justin <jlec@j-schmitz.net>:
> So what is the current mood to switch to git. As there is no traffic
> on the scm-ml I assume that there is no progress. But shouldn' we
> start to move on? Now!?

Mood is good, but someone has to do the work. And robbat2 cannot do
all of it.

V-Li

--
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://gentoo.faulhammer.org/>

Nirbheek Chauhan 01-14-2010 11:49 AM

proxy maintainership and gentoo-x86 scm
 
On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> i think our current work flows also significantly impede the smooth running of
> this. *if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> their behalf.
>

In theory, yes. In practice, git is too slow to handle 30,000 files.
Even simple operations like git add become painful even if you put the
whole of portage on tmpfs since git does a stat() on every single file
in the repository with every operation.

Simple test: do a git init followed by git add && git commit -m
"Initial commit" in your portage dir (.gitignore packages/ and
distfiles/)

Once this is done, you can test how it'll feel like to use a DSCM on
portage (without history). Unless you have a really fast SSD and
processor, you'll want to go back to the good old days of CVS with its
network-bound latencies on just 5-6 files in the current dir.

Besides this, there is the problem of accommodating people who use a
subtree of gentoo-x86, and those who don't want the entire CVS history
on their hard drives. In summation, robbat2 needs *our* help in the
following:

a) Push functionality in shallow clones (patches exist upstream)
b) Partial-tree checkouts (patches exist upstream)
c) Optimize git so it can handle 30,000 files
- Maybe maintain a cache of directory timestamps and only stat()
directories?
- Implement recursive timestamps on directories in various
filesystems and then in git (via xattrs perhaps)? People want to do
this for things like Tracker too. Prelim patches might exist.
d) Implement scripts/infra for people to fetch repository (shallow and
deep) bundles to initialize their local git clones (similar to portage
snapshots)
- git clone from scratch taxes the server too much, just like
rsync from scratch
e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS
- We want this for overlays right now too.
f) (Optional) Fix http cloning in git to make it "smarter" to help
people behind firewalls get anonymous clones (patches exist upstream)

Did I miss something Robin?

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Markos Chandras 01-14-2010 12:30 PM

proxy maintainership and gentoo-x86 scm
 
On Wednesday 13 January 2010 23:02:17 Ben de Groot wrote:
> 2010/1/13 Mike Frysinger <vapier@gentoo.org>:
> > On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
> >> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> >> > If you feel like it, become a proxy-maintainer and poke a developer to
> >> > put your ebuilds on tree. Have you ever heard of that ? :)
> >>
> >> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
> >> in my opinion. It is a virtually unknown option.
> >
> > i think our current work flows also significantly impede the smooth
> > running of this. if we had were using a dscm (git) on gentoo-x86, i feel
> > like it'd be a much smoother ride for Gentoo devs to pull from a proxy
> > maintainer and push on their behalf.
> > -mike
>
> I couldn't agree more!
>
My experience as developer this years implies that decisions about these
things take way too long to get implemented ( or even discussed ). So imho,
the best way to promote the proxy-maintainer thing, is individually using our
blogs or any other means.
--
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

Nguyen Thai Ngoc Duy 01-14-2010 12:47 PM

proxy maintainership and gentoo-x86 scm
 
On 1/14/10, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> > i think our current work flows also significantly impede the smooth running of
> > this. if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> > much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> > their behalf.
> >
>
>
> In theory, yes. In practice, git is too slow to handle 30,000 files.
> Even simple operations like git add become painful even if you put the
> whole of portage on tmpfs since git does a stat() on every single file
> in the repository with every operation.

What you need is "git update-index --assume-unchanged". That feature
was introduced exactly to reduce stat().

BTW, if you know you only work in certain directories, doing "git diff
--stat <dir>", "git diff --cached --stat <dir>" instead of "git
status" would also help. Make aliases for them ("git dis" and "git
dics" in my ~/.gitconfig) so you don't have to type full command every
time.

"git commit <dir>" and "git status <dir>" still do full tree lstat().
I can try to make a patch or two to reduce lstat() in such cases.

Does that help?
--
Duy

Ben de Groot 01-14-2010 03:24 PM

proxy maintainership and gentoo-x86 scm
 
2010/1/14 Nirbheek Chauhan <nirbheek@gentoo.org>:
> In theory, yes. In practice, git is too slow to handle 30,000 files.
> Even simple operations like git add become painful even if you put the
> whole of portage on tmpfs since git does a stat() on every single file
> in the repository with every operation.
> [...]
> Besides this, there is the problem of accommodating people who use a
> subtree of gentoo-x86, and those who don't want the entire CVS history
> on their hard drives. In summation, robbat2 needs *our* help in the
> following:

Thanks! That was a nice overview of the remaining issues. Has anyone
tested Mercurial to see how it compares, especially with respect to
these issues?

Cheers,
--
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
__________________________________________________ ____

Ben de Groot 01-14-2010 03:35 PM

proxy maintainership and gentoo-x86 scm
 
2010/1/14 Markos Chandras <hwoarang@gentoo.org>:
> My experience as developer this years implies that decisions about these
> things take way too long to get implemented ( or even discussed ). So imho,
> the best way to promote the proxy-maintainer thing, is individually using our
> blogs or any other means.

I'm not saying we should wait for a move to a DVCS. That is obviously
going to take some time still. I think we should do both: promote the
proxy-maintenance possibility, and at the same time work on DVCS
migration, which will ultimately make such work easier.

Cheers,
--
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
__________________________________________________ ____

"PaweĊ‚ Hajdan, Jr." 01-14-2010 04:04 PM

proxy maintainership and gentoo-x86 scm
 
On 1/14/10 1:49 PM, Nirbheek Chauhan wrote:
> Besides this, there is the problem of accommodating people who use a
> subtree of gentoo-x86, and those who don't want the entire CVS history
> on their hard drives. In summation, robbat2 needs *our* help in the
> following:
>
> a) Push functionality in shallow clones (patches exist upstream)
> b) Partial-tree checkouts (patches exist upstream)
> c) Optimize git so it can handle 30,000 files
> - Maybe maintain a cache of directory timestamps and only stat()
> directories?
> - Implement recursive timestamps on directories in various
> filesystems and then in git (via xattrs perhaps)? People want to do
> this for things like Tracker too. Prelim patches might exist.
> d) Implement scripts/infra for people to fetch repository (shallow and
> deep) bundles to initialize their local git clones (similar to portage
> snapshots)
> - git clone from scratch taxes the server too much, just like
> rsync from scratch
> e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS
> - We want this for overlays right now too.
> f) (Optional) Fix http cloning in git to make it "smarter" to help
> people behind firewalls get anonymous clones (patches exist upstream)
>
> Did I miss something Robin?

It would be nice to post that info to a webpage. That could increase a
chance of a volunteer contributing some help.

Note in advance: I don't know git internals, so can't help at this moment.


All times are GMT. The time now is 07:25 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.