hi !
while trying to download F9 what I face is quite frustrating ..in fact
this can be faced by a lot of user in India .
while wbut mirror is down (temporarily ) and other cannot handle all
the load and bit torrent is quite slow .
clicking on i386 direct download always gives error 412 : too many
connected users .
any way of avoiding this .. I am currently downloading from a mirror
in USA and its quite slow.

--
Regards
Subhodip Biswas

GPG key : FAEA34AB
Server : pgp.mit.edu
http://subhodipbiswas.wordpress.com
http:/www.fedoraproject.org/wiki/SubhodipBiswas

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

subhodip biswas wrote:

hi !
while trying to download F9 what I face is quite frustrating ..in fact
this can be faced by a lot of user in India .
while wbut mirror is down (temporarily ) and other cannot handle all
the load and bit torrent is quite slow .
clicking on i386 direct download always gives error 412 : too many
connected users .
any way of avoiding this .. I am currently downloading from a mirror
in USA and its quite slow.



Hi,

Going from what I know (someone might want to update me here):
We currently only have 3 mirrors in India
(http://mirrors.fedoraproject.org/), one is on a 45mbps link, while the
other two are on 5mbps and 2mbps links.


From what I see, there isn't really much we can do apart from waiting
for more Indian mirrors to come aboard, ISPs/Universities are the best
bet and is actually where quite a few of our mirrors are located.


Another alternative is try an Asian or Europe mirror, China, Japan,
Italy, France etc, many of these mirrors are on 100Mbps+ links and
shouldn't be returning 421 errors. (I have no clue on the Indian
internet topology but generally geographically closer locations have
better links)


Noting the 421 errors, it's fairly hard to prevent them, the script (as
Matt mentioned on IRC) could be adapted to query the server if there are
slots open but thats not always possible.


- N.J.

Amendment: I noticed the two slower mirrors only allow ftp, try
http://ftp.iitm.ac.in/fedora/ which is the server on the 45Mbps link and
responds to me in NZ.


_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

On Thu, May 15, 2008 at 10:04:51PM +1200, Nigel Jones wrote:
> subhodip biswas wrote:
> >hi !
> >while trying to download F9 what I face is quite frustrating ..in fact
> >this can be faced by a lot of user in India .
> >while wbut mirror is down (temporarily ) and other cannot handle all
> >the load and bit torrent is quite slow .
> >clicking on i386 direct download always gives error 412 : too many
> >connected users .
> >any way of avoiding this .. I am currently downloading from a mirror
> >in USA and its quite slow.
> >
> >
> Hi,
>
> Going from what I know (someone might want to update me here):
> We currently only have 3 mirrors in India
> (http://mirrors.fedoraproject.org/), one is on a 45mbps link, while the
> other two are on 5mbps and 2mbps links.

One thing MM would benefit from would be a weighted sampler.

Right now, it collects various lists of Hosts to return (same
netblock, same country, same continent, global). It shuffles (python random.shufle()) each of
the lists, then concatenates them, and sends that result to the end
user. However, this does not take into account that one mirror has a
45Mbps link, and another a 2Mbps; they would each get chosen
"randomly".

What I need is a replacement for random.shuffle() that takes a list of
tuples: (something, weight). And returns a list of somethings that
was generated with a statistical sampling based on weight.

Now, this is _probably_ a simple thing to do; my college number theory
professor would mock me for having not "just done it" myself, but hey
- I took that class _twice_ before passing it - I don't trust my
skills in this area to get it right.

Volunteers?


>
> From what I see, there isn't really much we can do apart from waiting
> for more Indian mirrors to come aboard, ISPs/Universities are the best
> bet and is actually where quite a few of our mirrors are located.
>
> Another alternative is try an Asian or Europe mirror, China, Japan,
> Italy, France etc, many of these mirrors are on 100Mbps+ links and
> shouldn't be returning 421 errors. (I have no clue on the Indian
> internet topology but generally geographically closer locations have
> better links)

And please please, help find more mirrors in India.

>
> Noting the 421 errors, it's fairly hard to prevent them, the script (as
> Matt mentioned on IRC) could be adapted to query the server if there are
> slots open but thats not always possible.

yea, I like _this_ less and less the more I think about it.

>
> - N.J.
>
> Amendment: I noticed the two slower mirrors only allow ftp, try
> http://ftp.iitm.ac.in/fedora/ which is the server on the 45Mbps link and
> responds to me in NZ.

which just came online yesterday.

--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

> One thing MM would benefit from would be a weighted sampler.
>
> [...]
>
> What I need is a replacement for random.shuffle() that takes a list of
> tuples: (something, weight). And returns a list of somethings that
> was generated with a statistical sampling based on weight.
>
> Now, this is _probably_ a simple thing to do; my college number theory
> professor would mock me for having not "just done it" myself, but hey
> - I took that class _twice_ before passing it - I don't trust my
> skills in this area to get it right.

Here is a crude approach:

Say A, B and C are mirrors having bandwidth of x,y and z Mbps
respectively. So lets have an array which has: A, A, A... (x times),
B, B, B... (y times), C, C, C... (z times), and then generate a random
number within [0, x+y+z] and use that as an index to fetch a mirror
from the array.

Happy hacking,
Debarshi
--
"From what we get, we can make a living; what we give, however, makes a life."
-- Arthur Ashe

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

Debarshi Ray wrote:

Say A, B and C are mirrors having bandwidth of x,y and z Mbps
respectively. So lets have an array which has: A, A, A... (x times),
B, B, B... (y times), C, C, C... (z times), and then generate a random
number within [0, x+y+z] and use that as an index to fetch a mirror
from the array.


No, your array should be something like :
([A,x];[B,y];[C,z])
and when you pick your random number within [0,x+y+z], you parse the
array decreasing your random number until it is less than the current
server bandwidth (unless your array is kept static, it would be more
costy to generate it (its length will be the sum of server bandwidth)
than to parse a server list).


But as far as I am concerned, I would prefer giving back to the user a
list of nearby servers links (which may be picked bandwidth-randomly) in
which the user would choose a server by itself.


François

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

Do we really need an array? Lets see..

* Lets say we have three mirrors X (45 Mb), Y (5Mb), Z (3Mb)

* =>Total bandwidth = 53Mb

* Now lets say, at any given instance there are 100 requests.

* So proportionally, X should get (100/53)*45 = 85 requests.
Y should get 9 requests and Z, 6 requests only.

Now

1. Generate a random no r.
2. If r<85 route to X,
else if 85<r<95 route to Y
else route to Z.

This will do the job.


--
Regards,
Susmit.

=============================================
ssh
0x86DD170A
http://www.fedoraproject.org/wiki/SusmitShannigrahi
=============================================

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

On Thu, May 15, 2008 at 08:57:59PM +0530, susmit shannigrahi wrote:
> Do we really need an array? Lets see..
>
> * Lets say we have three mirrors X (45 Mb), Y (5Mb), Z (3Mb)
>
> * =>Total bandwidth = 53Mb
>
> * Now lets say, at any given instance there are 100 requests.

we don't know this number (100) in the app...

Hence it needs to be a probability chosen out of the 53 (which we do
know).

--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

susmit shannigrahi wrote:

Do we really need an array? Lets see..



2. If r<85 route to X,
else if 85<r<95 route to Y
else route to Z.


This is de facto an array... Your's is hardcoded (and you need a code
generator and you need to generate the code it for all GeoIP possible
result). Mine is build up using GeoIP results.


François

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

>
> we don't know this number (100) in the app...
> Hence it needs to be a probability chosen out of the 53 (which we do
> know).


No, Wait.

I can show if this is true for 100, this will be true for _any_ number.

* As we are generating a random number within 100, the probability of each
number being picked up is 1/100.

* So the probability of getting a number within 85 is always 85/100.


* Lets assume there are n requests to distribute.

* Whatever be the value of n, the random number's probability of being
in range <85
is 85/100.

* So out of n requests, n*85/100 requests will always be routed to X
irrespective of the value n.

* Lets say n=10..So X gets 10*85/100 = 8.5 requests.

* Lets say n=27 so X gets 27*85/100 = 22.95 requests.

* If n=1, then the probability that X will get that particular requests is 85%.

Correct?

--
Regards,
Susmit.

=============================================
ssh
0x86DD170A
http://www.fedoraproject.org/wiki/SusmitShannigrahi
=============================================

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

On Thu, May 15, 2008 at 9:31 PM, Francois Petillon <fantec@proxad.net> wrote:
> susmit shannigrahi wrote:
>>
>> Do we really need an array? Lets see..
>
>> 2. If r<85 route to X,
>> else if 85<r<95 route to Y
>> else route to Z.
>
> This is de facto an array...

Well, may be, but without any operation on stack, thereby reducing
time complexity
and also reducing the overhead of creating, maintaining or updating an array.
Is not it?




--
Regards,
Susmit.

=============================================
ssh
0x86DD170A
http://www.fedoraproject.org/wiki/SusmitShannigrahi
=============================================

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list