As part of my portage-janitor scripts (https://github.com/iksaif/portage-janitor)
I created a tool that counts URIs prefixes and sort them.
Here is the result of this script on the gentoo-x86 tree:
That would affect ~1000 URIs.
Unfortunatly I didn't found any mirror for:
- pecl.php.net
- hackage.haskell.org
- get.qt.nokia.com
- launchpad
But maybe we should still add them to thirdpartymirrors ? Look at 3dgamers, beyondunreal, bitbucket, github, gnome, kernel, xfce, vdrfiles and vdr-developerorg: they all only have one mirror !
For those afraid to add these mirrors and patch all affected ebuilds: no worry, mirrors.py can do it for you, and I already generated splitted patches generated from various thirdpartymirrors here: http://xf.iksaif.net/dev/gentoo/
Note that as part of https://bugs.gentoo.org/show_bug.cgi?id=405533 mgorny already commited a massive amount of patches adding mirror:// when the URI prefix was already present in thirdpartymirrors.
Thanks,
--
Corentin Chary
http://xf.iksaif.net/
04-24-2012, 02:35 PM
"Robin H. Johnson"
New third party mirrors
On Tue, Apr 24, 2012 at 03:10:00PM +0200, Corentin Chary wrote:
> Hello,
>
> As part of my portage-janitor scripts (https://github.com/iksaif/portage-janitor)
> I created a tool that counts URIs prefixes and sort them.
> Here is the result of this script on the gentoo-x86 tree:
>
> $ ./mirrors.py --all --count
> 297 http://pear.php.net
> 297 http://pear.php.net/get
> 88 http://pecl.php.net
> 88 http://pecl.php.net/get
These are already mirror bouncers. If you visit the above, you'll get
the closest mirror for downloading.
--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
04-24-2012, 02:50 PM
Corentin Chary
New third party mirrors
On Tue, Apr 24, 2012 at 4:35 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Tue, Apr 24, 2012 at 03:10:00PM +0200, Corentin Chary wrote:
>> Hello,
>>
>> As part of my portage-janitor scripts (https://github.com/iksaif/portage-janitor)
>> I created a tool that counts URIs prefixes and sort them.
>> Here is the result of this script on the gentoo-x86 tree:
>>
>> $ ./mirrors.py --all --count
>> 297 * * http://pear.php.net
>> 297 * * http://pear.php.net/get
>> 88 * * *http://pecl.php.net
>> 88 * * *http://pecl.php.net/get
> These are already mirror bouncers. If you visit the above, you'll get
> the closest mirror for downloading.
And since there is already ~10 "mirrors" with only one actual backend,
should they go to thirdpartymirrors or not ? If not, what about this
pseudo-mirrors already present in thirdpartymirrors ?
--
Corentin Chary
http://xf.iksaif.net
04-24-2012, 04:19 PM
"Robin H. Johnson"
New third party mirrors
On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
> >> $ ./mirrors.py --all --count
> >> 297 ?? ?? http://pear.php.net
> >> 297 ?? ?? http://pear.php.net/get
> >> 88 ?? ?? ??http://pecl.php.net
> >> 88 ?? ?? ??http://pecl.php.net/get
> > These are already mirror bouncers. If you visit the above, you'll get
> > the closest mirror for downloading.
> And since there is already ~10 "mirrors" with only one actual backend,
> should they go to thirdpartymirrors or not ? If not, what about this
> pseudo-mirrors already present in thirdpartymirrors ?
I think we should add the pseudo-mirrors, but explicitly mark them as
such in the file, so that they don't get duplicate entries added (eg
adding us.pear, de.pear and the pear bouncer is bad. Should have just
the bouncer).
--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
04-24-2012, 04:38 PM
Michał Górny
New third party mirrors
On Tue, 24 Apr 2012 16:19:11 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
> > >> $ ./mirrors.py --all --count
> > >> 297 ?? ?? http://pear.php.net
> > >> 297 ?? ?? http://pear.php.net/get
> > >> 88 ?? ?? ??http://pecl.php.net
> > >> 88 ?? ?? ??http://pecl.php.net/get
> > > These are already mirror bouncers. If you visit the above, you'll
> > > get the closest mirror for downloading.
> > And since there is already ~10 "mirrors" with only one actual
> > backend, should they go to thirdpartymirrors or not ? If not, what
> > about this pseudo-mirrors already present in thirdpartymirrors ?
> I think we should add the pseudo-mirrors, but explicitly mark them as
> such in the file, so that they don't get duplicate entries added (eg
> adding us.pear, de.pear and the pear bouncer is bad. Should have just
> the bouncer).
It'd be great if we could add some kind of additional mirror entries,
which would be used by repoman to signal missing mirror:// entries but
won't be used for downloads.
--
Best regards,
Michał Górny
04-24-2012, 04:49 PM
Zac Medico
New third party mirrors
On 04/24/2012 09:38 AM, Michał Górny wrote:
On Tue, 24 Apr 2012 16:19:11 +0000
"Robin H. Johnson"<robbat2@gentoo.org> wrote:
On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
These are already mirror bouncers. If you visit the above, you'll
get the closest mirror for downloading.
And since there is already ~10 "mirrors" with only one actual
backend, should they go to thirdpartymirrors or not ? If not, what
about this pseudo-mirrors already present in thirdpartymirrors ?
I think we should add the pseudo-mirrors, but explicitly mark them as
such in the file, so that they don't get duplicate entries added (eg
adding us.pear, de.pear and the pear bouncer is bad. Should have just
the bouncer).
It'd be great if we could add some kind of additional mirror entries,
which would be used by repoman to signal missing mirror:// entries but
won't be used for downloads.
We could just add them in thirpartymirrors under fake names that nobody
will use.
--
Thanks,
Zac
04-24-2012, 05:54 PM
Hans de Graaff
New third party mirrors
On Tue, 2012-04-24 at 15:10 +0200, Corentin Chary wrote:
> And here are some mirrors that can be deduced from that (except freebsd-jp
> which I found reading the linphone ebuild):
>
> xemacs http://ftp.xemacs.org/pub/ ftp://ftp.sa.xemacs.org/pub/
I only consider the first site to be a reliable source for the packages,
and this is what the eclass provides (only 4 packages actually provide a
SRC_URI, all others are handled through the eclass).
Kind regards,
Hans
04-25-2012, 07:16 AM
Corentin Chary
New third party mirrors
On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Tue, 24 Apr 2012 16:19:11 +0000
> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>
>> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>> > >> $ ./mirrors.py --all --count
>> > >> 297 ?? ?? http://pear.php.net
>> > >> 297 ?? ?? http://pear.php.net/get
>> > >> 88 ?? ?? ??http://pecl.php.net
>> > >> 88 ?? ?? ??http://pecl.php.net/get
>> > > These are already mirror bouncers. If you visit the above, you'll
>> > > get the closest mirror for downloading.
>> > And since there is already ~10 "mirrors" with only one actual
>> > backend, should they go to thirdpartymirrors or not ? If not, what
>> > about this pseudo-mirrors already present in thirdpartymirrors ?
>> I think we should add the pseudo-mirrors, but explicitly mark them as
>> such in the file, so that they don't get duplicate entries added (eg
>> adding us.pear, de.pear and the pear bouncer is bad. Should have just
>> the bouncer).
>
> It'd be great if we could add some kind of additional mirror entries,
> which would be used by repoman to signal missing mirror:// entries but
> won't be used for downloads.
Yep, we could put that in it too:
github http://github.com/downloads/ https://github.com/downloads/
nongnu http://download.savannah.nongnu.org/releases/
> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
> > On Tue, 24 Apr 2012 16:19:11 +0000
> > "Robin H. Johnson" <robbat2@gentoo.org> wrote:
> >
> >> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
> >> > >> $ ./mirrors.py --all --count
> >> > >> 297 ?? ?? http://pear.php.net
> >> > >> 297 ?? ?? http://pear.php.net/get
> >> > >> 88 ?? ?? ??http://pecl.php.net
> >> > >> 88 ?? ?? ??http://pecl.php.net/get
> >> > > These are already mirror bouncers. If you visit the above,
> >> > > you'll get the closest mirror for downloading.
> >> > And since there is already ~10 "mirrors" with only one actual
> >> > backend, should they go to thirdpartymirrors or not ? If not,
> >> > what about this pseudo-mirrors already present in
> >> > thirdpartymirrors ?
> >> I think we should add the pseudo-mirrors, but explicitly mark them
> >> as such in the file, so that they don't get duplicate entries
> >> added (eg adding us.pear, de.pear and the pear bouncer is bad.
> >> Should have just the bouncer).
> >
> > It'd be great if we could add some kind of additional mirror
> > entries, which would be used by repoman to signal missing mirror://
> > entries but won't be used for downloads.
>
> Yep, we could put that in it too:
> github http://github.com/downloads/
> https://github.com/downloads/
Per spec, portage can choose a random mirror of the list. If we put
entries like that, these two will be equally possible as the preferred
cloud. URL -- while they redirect one to another.
We might decide on some common syntax like preceding all extra entries
with '-' but I don't want to be the one deciding here.
--
Best regards,
Michał Górny
04-26-2012, 07:30 AM
Corentin Chary
New third party mirrors
On Wed, Apr 25, 2012 at 6:41 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Wed, 25 Apr 2012 09:16:05 +0200
> Corentin Chary <corentin.chary@gmail.com> wrote:
>
>> On Tue, Apr 24, 2012 at 6:38 PM, Michał Górny <mgorny@gentoo.org>
>> wrote:
>> > On Tue, 24 Apr 2012 16:19:11 +0000
>> > "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>> >
>> >> On Tue, Apr 24, 2012 at 04:50:49PM +0200, Corentin Chary wrote:
>> >> > >> $ ./mirrors.py --all --count
>> >> > >> 297 ?? ?? http://pear.php.net
>> >> > >> 297 ?? ?? http://pear.php.net/get
>> >> > >> 88 ?? ?? ??http://pecl.php.net
>> >> > >> 88 ?? ?? ??http://pecl.php.net/get
>> >> > > These are already mirror bouncers. If you visit the above,
>> >> > > you'll get the closest mirror for downloading.
>> >> > And since there is already ~10 "mirrors" with only one actual
>> >> > backend, should they go to thirdpartymirrors or not ? If not,
>> >> > what about this pseudo-mirrors already present in
>> >> > thirdpartymirrors ?
>> >> I think we should add the pseudo-mirrors, but explicitly mark them
>> >> as such in the file, so that they don't get duplicate entries
>> >> added (eg adding us.pear, de.pear and the pear bouncer is bad.
>> >> Should have just the bouncer).
>> >
>> > It'd be great if we could add some kind of additional mirror
>> > entries, which would be used by repoman to signal missing mirror://
>> > entries but won't be used for downloads.
>>
>> Yep, we could put that in it too:
>> github Â* Â* Â* Â* Â* Â* Â* Â*http://github.com/downloads/
>> https://github.com/downloads/
>
> Per spec, portage can choose a random mirror of the list. If we put
> entries like that, these two will be equally possible as the preferred
> cloud. URL -- while they redirect one to another.
>
> We might decide on some common syntax like preceding all extra entries
> with '-' but I don't want to be the one deciding here.
I checked, and current portage code already handle entries starting
with a - gracefully thanks to stack_dictlist (removing them from the
list of mirrors).
New third party mirrors
