On Mon, Sep 22, 2008 at 4:59 PM, Kulbir Saini
<kulbirsaini@students.iiit.ac.in> wrote:
> Hi Mikesell,
>
> Les Mikesell wrote:
>>
>> seth vidal wrote:
>>>
>>> On Mon, 2008-09-22 at 10:15 -0600, Nathanael D. Noblet wrote:
>>>>
>>>> Hello,
>>>> I've become increasingly interested in a simple updates mirroring system
>>>> to proxy/store updates for a number of fedora machines on our network. I
>>>> searched google and found the instant mirror project, but it seemed somewhat
>>>> dead. Is it still alive? Did some other solution replace it?
>>>>
>>>
>>> Take a look at intelligent mirror. A GSoC project by Kulbir Saini.
>>>
>>> https://fedorahosted.org/intelligentmirror/
>>
>> Is there still no way to make yum default to behaving intelligently by
>> itself in the presence of a caching proxy? All it really needs to do is use
>> the same mirror as the last person chose so the URLs will match.
>
> I think its not possible. Especially in shared environments. For example in
> a university, machine A ran yum and downloaded foo.rpm via bar.com. Now if
> machine B runs yum to download the same foo.rpm, how will it know what
> mirror machine A used?

You make it sound like yum is incapable of persisting data. Can't it
just save the last used mirror?

--
Fedora 9 : sulphur is good for the skin
( www.pembo13.com )

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Mon, Sep 22, 2008 at 5:15 PM, Emmanuel Seyman
<emmanuel.seyman@club-internet.fr> wrote:
> * Gregory Maxwell [22/09/2008 20:04] :
>>
>> How would it know?
>
> And more importantly, if you want to always hit the same mirror, why don't
> you edit its configuration to reflect that ?


I am not sure that I would consider this "more importantly" but that
is exactly the approach that I use.



--
Fedora 9 : sulphur is good for the skin
( www.pembo13.com )

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Mon, 2008-09-22 at 18:50 -0500, Arthur Pemberton wrote:
>
> You make it sound like yum is incapable of persisting data. Can't it
> just save the last used mirror?


A single machine can continually hit the same mirror. However the
original question (at least as I read it) is why can't yum on client A
hit the same mirror that yum on client B just got done using, which
would be pretty difficult for yum on client A to know about.

--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

2008/9/22 Jesse Keating <jkeating@redhat.com>:
> On Mon, 2008-09-22 at 18:50 -0500, Arthur Pemberton wrote:
>>
>> You make it sound like yum is incapable of persisting data. Can't it
>> just save the last used mirror?
>
>
> A single machine can continually hit the same mirror. However the
> original question (at least as I read it) is why can't yum on client A
> hit the same mirror that yum on client B just got done using, which
> would be pretty difficult for yum on client A to know about.


Thanks for that clarification, I totally missed this interpretation.

I currently manually set the proxy for this on each machine. Would be
nice is there was a zero-conf way of doing this, especially on a large
scale.


--
Fedora 9 : sulphur is good for the skin
( www.pembo13.com )

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Mon, Sep 22, 2008 at 8:20 PM, Arthur Pemberton <pemboa@gmail.com> wrote:

> I currently manually set the proxy for this on each machine. Would be
> nice is there was a zero-conf way of doing this, especially on a large
> scale.

There is - if you set up a private mirror in MirrorManager, then all
requests from your netblock will be directed to said mirror. Or am I
missing something?

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Mon, Sep 22, 2008 at 9:42 PM, Jon Stanley <jonstanley@gmail.com> wrote:
> On Mon, Sep 22, 2008 at 8:20 PM, Arthur Pemberton <pemboa@gmail.com> wrote:
>
>> I currently manually set the proxy for this on each machine. Would be
>> nice is there was a zero-conf way of doing this, especially on a large
>> scale.
>
> There is - if you set up a private mirror in MirrorManager, then all
> requests from your netblock will be directed to said mirror. Or am I
> missing something?


What you are missing, due to my own ambiguity, is that I am currently
using Squid. Not nearly as elegant as it can be, but was very quick to
setup with my Centos server hosting the proxy.


--
Fedora 9 : sulphur is good for the skin
( www.pembo13.com )

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Jon Stanley wrote:

On Mon, Sep 22, 2008 at 8:20 PM, Arthur Pemberton <pemboa@gmail.com> wrote:


I currently manually set the proxy for this on each machine. Would be
nice is there was a zero-conf way of doing this, especially on a large
scale.


There is - if you set up a private mirror in MirrorManager, then all
requests from your netblock will be directed to said mirror. Or am I
missing something?


What you are missing is that the people sharing a proxy shouldn't have
to know about each other or set anything up. All they should have to do
is request the same URLs.


Why can't that happen automatically? That is, within whatever blocks
geoip might consider 'near', arbitrarily select the first-choice mirror
based on the source IP range in a repeatable way. That way everyone
behind the same proxy requests the same URL and the caching works
automatically.


--
Les Mikesell
lesmikesell@gmail.com

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Mon, 2008-09-22 at 22:51 -0500, Les Mikesell wrote:
> All they should have to do is request the same URLs.
>
> Why can't that happen automatically? That is, within whatever blocks
> geoip might consider 'near', arbitrarily select the first-choice mirror
> based on the source IP range in a repeatable way. That way everyone
> behind the same proxy requests the same URL and the caching works
> automatically.

MirrorManager gives the clients the URLs to try in a specific order
(and soon with even more data). Yum/urlgrabber will try the URLs in the
order it gets them.
So what you are saying essentially is: "Why can't MirrorManager decide
what the best URL is for a netblock/geoip and always list it first, just
to make the proxy problem zero-conf"
And I can guess that the answer to that is basically "Because it
doesn't work", feel free to send Matt patches though if you think
otherwise.

It is on my TODO to look at trying to get a usable version of
yum-avahi¹ that people can use, this should solve your proxy problem in
a generic way ... although it would do so by routing around it.
I also have a lot of things on my TODO list, so again patches
welcome

¹ http://users.ecs.soton.ac.uk/rds204/yum-avahi/

--
James Antill <james.antill@redhat.com>
Red Hat
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Tue, Sep 23, 2008 at 03:01:39AM -0400, James Antill wrote:
> On Mon, 2008-09-22 at 22:51 -0500, Les Mikesell wrote:
> > All they should have to do is request the same URLs.
> >
> > Why can't that happen automatically? That is, within whatever blocks
> > geoip might consider 'near', arbitrarily select the first-choice mirror
> > based on the source IP range in a repeatable way. That way everyone
> > behind the same proxy requests the same URL and the caching works
> > automatically.

GeoIP doesn't have this level of granularity. Best we can get from
that database "for free" is the country for a given address.

Furthermore, I absolutely don't want to return the same mirror at the
top of the list _for everyone_ in a given country. We want to load
balance the requests across the mirrors in a given country.

If your organization wants to ensure that everyone within it always is
directed to the same mirror first, get the mirror owner to add your
organization's netblock to the list of netblocks on their entry in
MM's database. Then MM will always hand out that mirror to your users
first.

> MirrorManager gives the clients the URLs to try in a specific order
> (and soon with even more data). Yum/urlgrabber will try the URLs in the
> order it gets them.
> So what you are saying essentially is: "Why can't MirrorManager decide
> what the best URL is for a netblock/geoip and always list it first, just
> to make the proxy problem zero-conf"
> And I can guess that the answer to that is basically "Because it
> doesn't work", feel free to send Matt patches though if you think
> otherwise.

If someone can give me a method of getting the entire Internet's BGP
routing table, and a good algorithm for finding the bandwidth-weighted
shortest path between two arbitrary points in that table, I'll gladly
include that in MM. Until then, MM simply can't know enough to always
give you the same (ideally correct) answer for any given set of
clients. The netblocks trick is already pretty slick, and "good
enough" for a lot of folks.

but yes, patches would be welcome!

Thanks,
Matt


--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

James Antill wrote:


So what you are saying essentially is: "Why can't MirrorManager decide
what the best URL is for a netblock/geoip and always list it first, just
to make the proxy problem zero-conf"
And I can guess that the answer to that is basically "Because it
doesn't work", feel free to send Matt patches though if you think
otherwise.


How can it be worse than whatever you are doing now which essentially
defeats any caching infrastructure that anyone has in place? That is, I
don't see how any attempt at ordering the list repeatably can 'not work'
or be worse than no attempt at all. What can break if you do something
simple like take the list you'd return after geoip calculations (if
any), divide the ip space up by the number of mirrors and rotate the
list to the corresponding starting point for the source IP? You'd still
be giving the same set of choices to the same recipients, but in a way
that will work if they have caching in place.


--
Les Mikesell
lesmikesell@gmail.com

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list