FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 04-26-2012, 07:57 AM
Zac Medico
 
Default New third party mirrors

On 04/26/2012 12:30 AM, Corentin Chary wrote:
> On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> On Wed, 25 Apr 2012 09:16:05 +0200
>> Corentin Chary <corentin.chary@gmail.com> wrote:
>>
>>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
>>> wrote:
>>>> On Tue, 24 Apr 2012 16:19:11 +0000
>>>> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>>>>
>>>>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>>>>>>>> $ ./mirrors.py --all --count
>>>>>>>> 297 ?? ?? http://pear.php.net
>>>>>>>> 297 ?? ?? http://pear.php.net/get
>>>>>>>> 88 ?? ?? ??http://pecl.php.net
>>>>>>>> 88 ?? ?? ??http://pecl.php.net/get
>>>>>>> These are already mirror bouncers. If you visit the above,
>>>>>>> you'll get the closest mirror for downloading.
>>>>>> And since there is already ~10 "mirrors" with only one actual
>>>>>> backend, should they go to thirdpartymirrors or not ? If not,
>>>>>> what about this pseudo-mirrors already present in
>>>>>> thirdpartymirrors ?
>>>>> I think we should add the pseudo-mirrors, but explicitly mark them
>>>>> as such in the file, so that they don't get duplicate entries
>>>>> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>>>>> Should have just the bouncer).
>>>>
>>>> It'd be great if we could add some kind of additional mirror
>>>> entries, which would be used by repoman to signal missing mirror://
>>>> entries but won't be used for downloads.
>>>
>>> Yep, we could put that in it too:
>>> github http://github.com/downloads/
>>> https://github.com/downloads/
>>
>> Per spec, portage can choose a random mirror of the list. If we put
>> entries like that, these two will be equally possible as the preferred
>> cloud. URL -- while they redirect one to another.
>>
>> We might decide on some common syntax like preceding all extra entries
>> with '-' but I don't want to be the one deciding here.
>
> I checked, and current portage code already handle entries starting
> with a - gracefully thanks to stack_dictlist (removing them from the
> list of mirrors).

That means repoman will ignore them too. If you want existing versions
of repoman to check for those paths in SRC_URI, you can add a line like
this to thirdpartymirrors:

github-bad-urls http://github.com/downloads/ https://github.com/downloads/
--
Thanks,
Zac
 
Old 04-26-2012, 08:03 AM
Corentin Chary
 
Default New third party mirrors

On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico <zmedico@gentoo.org> wrote:
> On 04/26/2012 12:30 AM, Corentin Chary wrote:
>> On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>> On Wed, 25 Apr 2012 09:16:05 +0200
>>> Corentin Chary <corentin.chary@gmail.com> wrote:
>>>
>>>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
>>>> wrote:
>>>>> On Tue, 24 Apr 2012 16:19:11 +0000
>>>>> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>>>>>
>>>>>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>>>>>>>>> $ ./mirrors.py --all --count
>>>>>>>>> 297 ?? ?? http://pear.php.net
>>>>>>>>> 297 ?? ?? http://pear.php.net/get
>>>>>>>>> 88 ?? ?? ??http://pecl.php.net
>>>>>>>>> 88 ?? ?? ??http://pecl.php.net/get
>>>>>>>> These are already mirror bouncers. If you visit the above,
>>>>>>>> you'll get the closest mirror for downloading.
>>>>>>> And since there is already ~10 "mirrors" with only one actual
>>>>>>> backend, should they go to thirdpartymirrors or not ? If not,
>>>>>>> what about this pseudo-mirrors already present in
>>>>>>> thirdpartymirrors ?
>>>>>> I think we should add the pseudo-mirrors, but explicitly mark them
>>>>>> as such in the file, so that they don't get duplicate entries
>>>>>> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>>>>>> Should have just the bouncer).
>>>>>
>>>>> It'd be great if we could add some kind of additional mirror
>>>>> entries, which would be used by repoman to signal missing mirror://
>>>>> entries but won't be used for downloads.
>>>>
>>>> Yep, we could put that in it too:
>>>> github * * * * * * * *http://github.com/downloads/
>>>> https://github.com/downloads/
>>>
>>> Per spec, portage can choose a random mirror of the list. If we put
>>> entries like that, these two will be equally possible as the preferred
>>> cloud. URL -- while they redirect one to another.
>>>
>>> We might decide on some common syntax like preceding all extra entries
>>> with '-' but I don't want to be the one deciding here.
>>
>> I checked, and current portage code already handle entries starting
>> with a - gracefully thanks to stack_dictlist (removing them from the
>> list of mirrors).
>
> That means repoman will ignore them too. If you want existing versions
> of repoman to check for those paths in SRC_URI, you can add a line like
> this to thirdpartymirrors:
>
> github-bad-urls http://github.com/downloads/ https://github.com/downloads/

Hum, I checked repoman source code, and I didn't find where it checks
if SRC_URI matches something in thirdpartymirror. Any hint ?


--
Corentin Chary
http://xf.iksaif.net
 
Old 04-26-2012, 08:07 AM
Zac Medico
 
Default New third party mirrors

On 04/26/2012 01:03 AM, Corentin Chary wrote:
> On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico <zmedico@gentoo.org> wrote:
>> On 04/26/2012 12:30 AM, Corentin Chary wrote:
>>> On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>>> On Wed, 25 Apr 2012 09:16:05 +0200
>>>> Corentin Chary <corentin.chary@gmail.com> wrote:
>>>>
>>>>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
>>>>> wrote:
>>>>>> On Tue, 24 Apr 2012 16:19:11 +0000
>>>>>> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>>>>>>
>>>>>>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>>>>>>>>>> $ ./mirrors.py --all --count
>>>>>>>>>> 297 ?? ?? http://pear.php.net
>>>>>>>>>> 297 ?? ?? http://pear.php.net/get
>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net
>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net/get
>>>>>>>>> These are already mirror bouncers. If you visit the above,
>>>>>>>>> you'll get the closest mirror for downloading.
>>>>>>>> And since there is already ~10 "mirrors" with only one actual
>>>>>>>> backend, should they go to thirdpartymirrors or not ? If not,
>>>>>>>> what about this pseudo-mirrors already present in
>>>>>>>> thirdpartymirrors ?
>>>>>>> I think we should add the pseudo-mirrors, but explicitly mark them
>>>>>>> as such in the file, so that they don't get duplicate entries
>>>>>>> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>>>>>>> Should have just the bouncer).
>>>>>>
>>>>>> It'd be great if we could add some kind of additional mirror
>>>>>> entries, which would be used by repoman to signal missing mirror://
>>>>>> entries but won't be used for downloads.
>>>>>
>>>>> Yep, we could put that in it too:
>>>>> github http://github.com/downloads/
>>>>> https://github.com/downloads/
>>>>
>>>> Per spec, portage can choose a random mirror of the list. If we put
>>>> entries like that, these two will be equally possible as the preferred
>>>> cloud. URL -- while they redirect one to another.
>>>>
>>>> We might decide on some common syntax like preceding all extra entries
>>>> with '-' but I don't want to be the one deciding here.
>>>
>>> I checked, and current portage code already handle entries starting
>>> with a - gracefully thanks to stack_dictlist (removing them from the
>>> list of mirrors).
>>
>> That means repoman will ignore them too. If you want existing versions
>> of repoman to check for those paths in SRC_URI, you can add a line like
>> this to thirdpartymirrors:
>>
>> github-bad-urls http://github.com/downloads/ https://github.com/downloads/
>
> Hum, I checked repoman source code, and I didn't find where it checks
> if SRC_URI matches something in thirdpartymirror. Any hint ?

Search for SRC_URI.mirror in /usr/bin/repoman.
--
Thanks,
Zac
 
Old 04-26-2012, 08:21 AM
Corentin Chary
 
Default New third party mirrors

On Thu, Apr 26, 2012 at 10:07 AM, Zac Medico <zmedico@gentoo.org> wrote:
> On 04/26/2012 01:03 AM, Corentin Chary wrote:
>> On Thu, Apr 26, 2012 at 9:57 AM, Zac Medico <zmedico@gentoo.org> wrote:
>>> On 04/26/2012 12:30 AM, Corentin Chary wrote:
>>>> On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>>>> On Wed, 25 Apr 2012 09:16:05 +0200
>>>>> Corentin Chary <corentin.chary@gmail.com> wrote:
>>>>>
>>>>>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
>>>>>> wrote:
>>>>>>> On Tue, 24 Apr 2012 16:19:11 +0000
>>>>>>> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>>>>>>>
>>>>>>>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>>>>>>>>>>> $ ./mirrors.py --all --count
>>>>>>>>>>> 297 ?? ?? http://pear.php.net
>>>>>>>>>>> 297 ?? ?? http://pear.php.net/get
>>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net
>>>>>>>>>>> 88 ?? ?? ??http://pecl.php.net/get
>>>>>>>>>> These are already mirror bouncers. If you visit the above,
>>>>>>>>>> you'll get the closest mirror for downloading.
>>>>>>>>> And since there is already ~10 "mirrors" with only one actual
>>>>>>>>> backend, should they go to thirdpartymirrors or not ? If not,
>>>>>>>>> what about this pseudo-mirrors already present in
>>>>>>>>> thirdpartymirrors ?
>>>>>>>> I think we should add the pseudo-mirrors, but explicitly mark them
>>>>>>>> as such in the file, so that they don't get duplicate entries
>>>>>>>> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>>>>>>>> Should have just the bouncer).
>>>>>>>
>>>>>>> It'd be great if we could add some kind of additional mirror
>>>>>>> entries, which would be used by repoman to signal missing mirror://
>>>>>>> entries but won't be used for downloads.
>>>>>>
>>>>>> Yep, we could put that in it too:
>>>>>> github * * * * * * * *http://github.com/downloads/
>>>>>> https://github.com/downloads/
>>>>>
>>>>> Per spec, portage can choose a random mirror of the list. If we put
>>>>> entries like that, these two will be equally possible as the preferred
>>>>> cloud. URL -- while they redirect one to another.
>>>>>
>>>>> We might decide on some common syntax like preceding all extra entries
>>>>> with '-' but I don't want to be the one deciding here.
>>>>
>>>> I checked, and current portage code already handle entries starting
>>>> with a - gracefully thanks to stack_dictlist (removing them from the
>>>> list of mirrors).
>>>
>>> That means repoman will ignore them too. If you want existing versions
>>> of repoman to check for those paths in SRC_URI, you can add a line like
>>> this to thirdpartymirrors:
>>>
>>> github-bad-urls http://github.com/downloads/ https://github.com/downloads/
>>
>> Hum, I checked repoman source code, and I didn't find where it checks
>> if SRC_URI matches something in thirdpartymirror. Any hint ?
>
> Search for SRC_URI.mirror in /usr/bin/repoman.

Arg.. ok, I only looked in pym/repoman/.

So two solutions here:

First one:
github http://cloud.github.com/downloads -http://github.com/downloads/
-https://github.com/downloads/
+ a small patch that would allow repoman to do something like
settings.thirdpartymirrors(keep_bad_uris=True) in order to keep uris
starting with a - in the list.

Second solution:
github http://cloud.github.com/downloads
github-bad-uris -http://github.com/downloads/ -https://github.com/downloads/

The good thing with the first one is that it would allow repoman to
outputs something like "you should use 'mirror://github'".


--
Corentin Chary
http://xf.iksaif.net
 
Old 04-26-2012, 06:41 PM
Michał Górny
 
Default New third party mirrors

On Thu, 26 Apr 2012 10:21:36 +0200
Corentin Chary <corentin.chary@gmail.com> wrote:

> Second solution:
> github http://cloud.github.com/downloads
> github-bad-uris -http://github.com/downloads/
-https://github.com/downloads/
>
> The good thing with the first one is that it would allow repoman to
> outputs something like "you should use 'mirror://github'".

Well, we could decide on something common and special like:

github:bad-uris http://.....

And then let repoman suggest using mirror with ':bad-uris' stripped.

--
Best regards,
Michał Górny
 
Old 04-27-2012, 08:06 AM
Corentin Chary
 
Default New third party mirrors

On Thu, Apr 26, 2012 at 8:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 26 Apr 2012 10:21:36 +0200
> Corentin Chary <corentin.chary@gmail.com> wrote:
>
>> Second solution:
>> github http://cloud.github.com/downloads
>> github-bad-uris -http://github.com/downloads/
> -https://github.com/downloads/
>>
>> The good thing with the first one is that it would allow repoman to
>> outputs something like "you should use 'mirror://github'".
>
> Well, we could decide on something common and special like:
>
> github:bad-uris http://.....
>
> And then let repoman suggest using mirror with ':bad-uris' stripped.

Works for me. What would be the next step to push this ?


--
Corentin Chary
http://xf.iksaif.net
 
Old 04-27-2012, 12:37 PM
Michał Górny
 
Default New third party mirrors

On Fri, 27 Apr 2012 10:06:38 +0200
Corentin Chary <corentin.chary@gmail.com> wrote:

> On Thu, Apr 26, 2012 at 8:41 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
> > On Thu, 26 Apr 2012 10:21:36 +0200
> > Corentin Chary <corentin.chary@gmail.com> wrote:
> >
> >> Second solution:
> >> github http://cloud.github.com/downloads
> >> github-bad-uris -http://github.com/downloads/
> > -https://github.com/downloads/
> >>
> >> The good thing with the first one is that it would allow repoman to
> >> outputs something like "you should use 'mirror://github'".
> >
> > Well, we could decide on something common and special like:
> >
> > github:bad-uris http://.....
> >
> > And then let repoman suggest using mirror with ':bad-uris' stripped.
>
> Works for me. What would be the next step to push this ?

Next step would be to find a good name :P. I suggest going the Perl way
and prepending '~' ;P.

--
Best regards,
Michał Górny
 

Thread Tools




All times are GMT. The time now is 10:33 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org