FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 05-17-2012, 08:40 AM
Michał Górny
 
Default RFC: git-2.eclass & fetching from multiple repos

Hello,

Sometimes it is necessary for a single package to pull from multiple
remote repositories. One case are broken packages like glibc (where
two repositories have to be merged for build-time), another are git
submodules (which don't cooperate well with bare checkouts).

Thus, I'm wondering about adding support for such a feature to
git-2.eclass. However, I don't have any concept for it yet so I'd
appreciate your ideas. What needs to be thought up is not only how to
do it but also what to do, exactly.

The simpler case would be only to provide support for fetching from
multiple repositories in git-2, and use it for submodules internally.
This would already fix glibc-9999 & smart-live-rebuild cooperation but
the glibc ebuild would still need to do a little hackery to checkout
the second repository.

The more complete case would be to completely support multiple
repostories but it would require a larger eclass rewrite, to support
hacking more of its variables.

Another question is how to implement it API-wide. The main problem here
is that we already use multiple values for EGIT_REPO_URI to support
fallback URIs, and I'd prefer supporting that for the additional repos
as well.


1/ Mike suggested something like:

EGIT_REPO_URI=( "repo1 fallback1" "repo2 fallback2" )

but IMO it hurts readability and makes the behavior a little surprising
(not that the fallback syntax isn't surprising already).


2/ Exherbo way would be doing something like:

EGIT_SECONDARY_REPOS='foo'
EGIT_foo_REPO_URI=...

but that would be real PITA for smart-live-rebuild, and I really hate
dynamically named variables.


3/ I'm wondering about something in-between those two, supposedly:

EGIT_SECONDARY_REPO_URIS=( "repo2 fallback2" ... )

Still painful for s-l-r but could be implemented without
eclass-specific hacks as array support. Of course, if we support this
not only for fetching but for checkouts as well, it's likely to get
unreadable as well...

What are your ideas?

--
Best regards,
Michał Górny
 
Old 05-19-2012, 07:28 PM
Luca Barbato
 
Default RFC: git-2.eclass & fetching from multiple repos

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 17/05/12 01:40, Michał Górny wrote:
>
>
> 1/ Mike suggested something like:
>
> EGIT_REPO_URI=( "repo1 fallback1" "repo2 fallback2" )
>
> but IMO it hurts readability and makes the behavior a little surprising
> (not that the fallback syntax isn't surprising already).

Does not look that bad, do you have an actual example so we can see how
bad could get?

lu

- --

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+39EIACgkQ6Ex4woTpDjQ7WgCdHXC8qOKxKG TkZg0UdlXr2BA0
VoEAoLmcUx/0TfsrtKf7eiOxrLQMcpBj
=MUjk
-----END PGP SIGNATURE-----
 
Old 05-19-2012, 08:18 PM
Michał Górny
 
Default RFC: git-2.eclass & fetching from multiple repos

On Sat, 19 May 2012 12:28:02 -0700
Luca Barbato <lu_zero@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 17/05/12 01:40, Michał Górny wrote:
> >
> >
> > 1/ Mike suggested something like:
> >
> > EGIT_REPO_URI=( "repo1 fallback1" "repo2 fallback2" )
> >
> > but IMO it hurts readability and makes the behavior a little
> > surprising (not that the fallback syntax isn't surprising already).
>
> Does not look that bad, do you have an actual example so we can see
> how bad could get?

Well, for example if the project was hosted on github, it would look
like:

EGIT_REPO_URI=(
"git://github.com/foo/foo.git
http://github.com/foo/foo.git"

"git://github.com/foo/bar.git
http://github.com/foo/bar.git"
)

Or something like that. What I am afraid of is that we use nested list
with tricky syntax, and one could miss the specifics of "" use.

--
Best regards,
Michał Górny
 
Old 05-19-2012, 08:38 PM
Luca Barbato
 
Default RFC: git-2.eclass & fetching from multiple repos

On 19/05/12 13:18, Michał Górny wrote:
> EGIT_REPO_URI=(
> "git://github.com/foo/foo.git
> http://github.com/foo/foo.git"
>
> "git://github.com/foo/bar.git
> http://github.com/foo/bar.git"
> )
>
> Or something like that. What I am afraid of is that we use nested list
> with tricky syntax, and one could miss the specifics of "" use.

it doesn't look that bad, the result would be getting

foo and bar in workdir I guess.

for github, sourceforge, bitbucket, gnome, etc, we might use something
like gitmirror://github/foo/bar

lu

--

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero
 
Old 05-20-2012, 11:40 AM
Steven J Long
 
Default RFC: git-2.eclass & fetching from multiple repos

Michał Górny wrote:
> Sometimes it is necessary for a single package to pull from multiple
> remote repositories.
<snip>
> Another question is how to implement it API-wide. The main problem here
> is that we already use multiple values for EGIT_REPO_URI to support
> fallback URIs, and I'd prefer supporting that for the additional repos
> as well.

So currently: EGIT_REPO_URI="uri fallbackA fallbackB fallbackC"
..where A, B and C are all fallbacks for uri.

> EGIT_REPO_URI=( "repo1 fallback1" "repo2 fallback2" )

Would require you to support both the above, and the new syntax if it's an
array. Since this is new functionality, it seems wiser to prevent confusion
between:
EGIT_REPO_URI=(uri fallbackA fallbackB fallbackC)
and:
EGIT_REPO_URI=(repo1 repo2 repo3 repo4)
..and also make the implementation simpler (ie quicker for metadata), by
using a new variable.

Semantically as well, it's the git repo for the package: in most cases
that's all that's required, so keep it simple for the majority usage (ie
separate.)

> 2/ Exherbo way would be doing something like:
>
EGIT_SECONDARY_REPOS='foo bar'
EGIT_foo_REPO_URI='repo1 fallback1'
EGIT_bar_REPO_URI='repo2 fallback2a fallback2b'
..
if use baz; then
EGIT_SECONDARY_REPOS+=' baz'
EGIT_baz_REPO_URI='repo3 fallback3'
fi

> but that would be real PITA for smart-live-rebuild, and I really hate
> dynamically named variables.
>
It's amateurishly-done as well, but there's no fixing the ugliness, nor a
requirement for duplication, even if you fix the fail. I'd go into it, but
technical posts I make, discussing implementation, seem to get taken as
political, so I'll stop here, given the provenance, and stick to the RFC.

> 3/ I'm wondering about something in-between those two, supposedly:
>
> EGIT_SECONDARY_REPO_URIS=( "repo2 fallback2" ... )
>
> Still painful for s-l-r but could be implemented without
> eclass-specific hacks as array support. Of course, if we support this
> not only for fetching but for checkouts as well, it's likely to get
> unreadable as well...
>
> What are your ideas?
>
Assuming an identifier/ tag as above is useful, since you can then
clone to a named subdirectory, or fetch from a specific repo, as directed by
the ebuild, or fetch all except the named ones, which might be fetched into
a different base-directory when the caller has cd'ed there, etc, extend your
last one with optional tags:

EGIT_SECONDARY=("foo@ repo1 fallback1" "repo2 fallback2a fallback2b")
..
use baz && EGIT_SECONDARY+=("baz@ repo3 fallback3")

So, fake associative arrays in a less cluttered fashion than 2/, that are
simple to lex in-script, and don't lead to visual or mental overload.

Since tags are optional, this accepts the same format, and extends it only
when the ebuild needs a tag, so simple builds stay simple. The @ symbol is
not valid in url's iirc, leave alone git uris, and draws attention to the
named items. It preserves ordering, which an associative array won't, and
you only ever have to worry about one variable name. Nor is there any issue
with making sure tags and variables are in sync. (DRY.)

The simple tag enables all the stuff I mentioned above, and by giving the
ebuild author a symbolic reference which can be passed around, it can be
extended ad-hoc.

--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
 

Thread Tools




All times are GMT. The time now is 03:29 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org