FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 09-05-2012, 11:25 AM
Michał Górny
 
Default src_fetch() phase function to support VCS fetching

On Wed, 5 Sep 2012 12:07:22 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Wed, 5 Sep 2012 13:00:05 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > > I guess that's a pretty comprehensive "we need to do this
> > > properly" then.
> >
> > Did I say we don't need to? We have the two eclasses which need to
> > do this properly, right? So what's your problem?
>
> The problem is that we need to work out how to deal with this at least
> for Subversion, and probably for CVS too... As much as we'd like to,
> we can't just roll out a src_fetch without eclass changes. This
> doesn't appear to be a trivial feature to provide.

First of all, subversion isn't a problem here. Subversion doesn't have
native branches. It's just directories in the tree, with tree having
linear history. In other words, it just works.

So it remains CVS and possibly darcs. But considering that the issue is
so unlikely and so small, we can as well assume that the affected
ebuilds should set ECVS_LOCALNAME to something with the branch. This is
in line with how we handle SRC_URI collisions.

And in any case, we may just decide to use a ECVS_LOCALNAME with branch
appended by default.

Finally, I don't think eclasses are really forced to use src_fetch()
from day one. src_unpack() will still work for them, and we can adjust
them gradually.

--
Best regards,
Michał Górny
 
Old 09-05-2012, 01:25 PM
Ian Stakenvicius
 
Default src_fetch() phase function to support VCS fetching

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 05/09/12 07:25 AM, Michał Górny wrote:
> Finally, I don't think eclasses are really forced to use
> src_fetch() from day one. src_unpack() will still work for them,
> and we can adjust them gradually.
>

...except for the fact that the whole point of this is so that live
ebuilds will download their sources via src_fetch() when emerge -f is
called...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBHUtcACgkQ2ugaI38ACPA7YAEAmuGb+BY67a dmkDhAAO5sLDvI
iChvTWdTFgRclPxeylEA/iUWFv3eCTzVBBhbvpGc44F2P8IO010OhEu1PrFj2mfC
=fO5h
-----END PGP SIGNATURE-----
 
Old 09-06-2012, 03:12 PM
James Cloos
 
Default src_fetch() phase function to support VCS fetching

>>>>> "CM" == Ciaran McCreesh <ciaran.mccreesh@googlemail.com> writes:

CM> This doesn't work if we have, for example, foo:1 and foo:2 both using
CM> the same SCM repository, but different branches.

The subversion eclass already handles that; it stores in $distfiles/$P/$branch.

The cvs eclass also could do so.

-JimC
--
James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6
 
Old 09-06-2012, 04:49 PM
Brian Harring
 
Default src_fetch() phase function to support VCS fetching

On Wed, Sep 05, 2012 at 12:07:22PM +0100, Ciaran McCreesh wrote:
> On Wed, 5 Sep 2012 13:00:05 +0200
> Micha?? G??rny <mgorny@gentoo.org> wrote:
> > > I guess that's a pretty comprehensive "we need to do this properly"
> > > then.
> >
> > Did I say we don't need to? We have the two eclasses which need to do
> > this properly, right? So what's your problem?
>
> The problem is that we need to work out how to deal with this at least
> for Subversion, and probably for CVS too... As much as we'd like to, we
> can't just roll out a src_fetch without eclass changes. This doesn't
> appear to be a trivial feature to provide.

You're conflating the required phase/hook w/ existing bad
implementations; they're separate.

The bad implementations can't use the hook till they sort out their
mess; straight forward enough.

Worth noting, last I looked, git eclass actually didn't do this right
either; basically merges the namespace of each remote/fetch source
into the local namespace, no prefixing. Fixable, but the issue mostly
comes about because of the fact we do *not* have a src_fetch in the
first place.

Either way, if the hook was in place, I expect the eclass level issues
to be sorted shortly after; right now there isn't a reason to do that
work (chicken/egg).

Consider that my +1 for src_fetch also, although FDEPEND is needed or
dependencies; I don't care which, getting a src_fetch has been on my
todo for chrome-os for a while now.

One additional thought- re: the scenarios where we don't fetch to an
intermediate location, then transfer that contents into ${WORKDIR},
while a better name is needed, something along the lines of
RESTRICT=fetches-into-workdir comes to mind.

Basically that restriction would be interpretted as "$WORKDIR must be
setup/preserved from invocation of src_fetch to actual building".

Via that restrict, both scenarios should be addressed in full.
~harring
 
Old 09-06-2012, 06:50 PM
Michał Górny
 
Default src_fetch() phase function to support VCS fetching

On Thu, 6 Sep 2012 09:49:13 -0700
Brian Harring <ferringb@gmail.com> wrote:

> One additional thought- re: the scenarios where we don't fetch to an
> intermediate location, then transfer that contents into ${WORKDIR},
> while a better name is needed, something along the lines of
> RESTRICT=fetches-into-workdir comes to mind.
>
> Basically that restriction would be interpretted as "$WORKDIR must be
> setup/preserved from invocation of src_fetch to actual building".
>
> Via that restrict, both scenarios should be addressed in full.

Does separate src_fetch() provide any benefit in that scenario?

--
Best regards,
Michał Górny
 
Old 09-06-2012, 07:10 PM
Brian Harring
 
Default src_fetch() phase function to support VCS fetching

Yes.* The manager can still parallelize prefetching, only consuming a build job slot post fetch.

On Sep 6, 2012 11:49 AM, "Michał Górny" <mgorny@gentoo.org> wrote:
On Thu, 6 Sep 2012 09:49:13 -0700

Brian Harring <ferringb@gmail.com> wrote:



> One additional thought- re: the scenarios where we don't fetch to an

> intermediate location, then transfer that contents into ${WORKDIR},

> while a better name is needed, something along the lines of

> RESTRICT=fetches-into-workdir comes to mind.

>

> Basically that restriction would be interpretted as "$WORKDIR must be

> setup/preserved from invocation of src_fetch to actual building".

>

> Via that restrict, both scenarios should be addressed in full.



Does separate src_fetch() provide any benefit in that scenario?



--

Best regards,

Michał Górny
 
Old 09-11-2012, 10:36 PM
"Rick "Zero_Chaos" Farina"
 
Default src_fetch() phase function to support VCS fetching

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/06/2012 02:50 PM, Michał Górny wrote:
> On Thu, 6 Sep 2012 09:49:13 -0700
> Brian Harring <ferringb@gmail.com> wrote:
>
>> One additional thought- re: the scenarios where we don't fetch to an
>> intermediate location, then transfer that contents into ${WORKDIR},
>> while a better name is needed, something along the lines of
>> RESTRICT=fetches-into-workdir comes to mind.

Realizing this is a late response I would like to add.... Um, what?
src_fetch should only be putting things into /usr/portage/distfiles,
never into ${WORKDIR}, that's for src_unpack to handle.

Am I missing something here?

- -Zero
>>
>> Basically that restriction would be interpretted as "$WORKDIR must be
>> setup/preserved from invocation of src_fetch to actual building".
>>
>> Via that restrict, both scenarios should be addressed in full.
>
> Does separate src_fetch() provide any benefit in that scenario?
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJQT7z+AAoJEKXdFCfdEflKVPQQALGcLGAgfo 8U+M6TdW5Edksf
oqaXE+NSTeFe2DE0G2mgKYSdSIZgMiFp5mLFwpdfAT1gjzFAc+ 34+5SY8X/0uaaG
OU7fafdUmOlqgD7rCvX56kSWZVPmTV3oZDghmwB1SUIQpL9PNS Zoz5uKoatt4UL8
mBMiTmnsYou8f+wDCJoN8eLVoQb/Hm2inobGUCozCsqU6ASgk1eVePpAmJNNVKp6
wrKuVbj4FeDS17Q6xc2g8exXlkxhGmdS1MmugKBR9csYC9P82f h4bXqVzxG15h9O
YZDU5nagJQ9fY6M7oeKg6etVe6PvwOd/FH0Z4wtQJ2NicOsu6DiBf7J+yLvadJdF
M5fE1kxjtR+rm+6bfNgBl2hP5DPdUYxPnZChftPNRpiVe0P0YZ GuRgy+GqfXSmMh
8Zf37hJylauBDy397yGapIJ4ergpYVb3Z1ZfU6uW7n8k0apqjq k+TYLEv2tajtTc
WBdRoBAT1LxvjFIHj2Bf5uzNtqev4l19vJv1AnALgs1v1Z8/TiSPB/B/2DvvIawR
Ys+mAEKzQOCezaCjVOpjq7pvi80/8PMcw6Txk9WpsAezrxdAj2X24kwsptVugCAO
lG3agqmOQgH+vCf0PkplxMSGBlofh4hZ3mNbuDaFlaRfiSlzFX//rkQTtOjK4YQu
6SiRnKxiDzJEn+Q1SNUr
=xgcT
-----END PGP SIGNATURE-----
 
Old 09-11-2012, 11:40 PM
Brian Harring
 
Default src_fetch() phase function to support VCS fetching

On Tue, Sep 11, 2012 at 06:36:46PM -0400, Rick "Zero_Chaos" Farina wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 09/06/2012 02:50 PM, Micha?? G??rny wrote:
> > On Thu, 6 Sep 2012 09:49:13 -0700
> > Brian Harring <ferringb@gmail.com> wrote:
> >
> >> One additional thought- re: the scenarios where we don't fetch to an
> >> intermediate location, then transfer that contents into ${WORKDIR},
> >> while a better name is needed, something along the lines of
> >> RESTRICT=fetches-into-workdir comes to mind.
>
> Realizing this is a late response I would like to add.... Um, what?
> src_fetch should only be putting things into /usr/portage/distfiles,
> never into ${WORKDIR}, that's for src_unpack to handle.
>
> Am I missing something here?

WORKDIR was an example; punting it directly into the pkgs distfiles is
also fine.

Basically, you're assuming that the content *is* cachable- cause if it
isn't, then dumping in ${DISTDIR} is wasteful (both since it'll
require a copy into the ebuild's workdir, and since it means
increasing crap accumulating in the workdir).

Further, there are cases where content *is* available, but is
fundamentally outside ${DISTDIR}- in cros, they have the
chrome/chromium source available in certain cases.

Now, either we can store 13GB int ${DISTDIR} that's a copy of that
external developer checkout, then copy that into ${WORKDIR}, or w/ a
restrict marker, copy that sucker into ${WORKDIR} directly.

Is it a corner case? Yep, but it's easy enough to deal w/- the only
constraint there is that the src_fetch's target dir isn't ${DISTDIR},
it's ${WORKDIR}, and the PM is required to preserve ${WORKDIR} from
src_fetch to the time of that pkg actually running.

~harring
 

Thread Tools




All times are GMT. The time now is 05:02 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org