Desperately need help with multi-core NIC performance
 

Hi,

I am running a VOIP application on Centos 5.3. It is a 16 core box
with 12 G of mem and all what it does is passing packets.

What happens is that at around 2K channels running g711 ( 64k) codec,
all eth0 is used up and no more traffic can go through.

I have checked google and it talked about interrupt scheduler.

does anyone know how to configure the kernel to allow it to use all
CPSs for socket transmission of UDP packets?

Any pointer will be greatly appreciated.

pete
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

On Wed, Feb 24, 2010 at 2:30 PM, Pete Kay <petedao@gmail.com> wrote:
> Hi,
>
> I am running a VOIP application on Centos 5.3. *It is a 16 core box
> with 12 G of mem and all what it does is passing packets.
>
> What happens is that at around 2K channels running g711 ( 64k) codec,
> all eth0 is used up and no more traffic can go through.
>
> I have checked google and it talked about interrupt scheduler.
>
> does anyone know how to configure the kernel to allow it to use all
> CPSs for socket transmission of UDP packets?
>
> Any pointer will be greatly appreciated.

There are some configuration parameters in the xinetd.conf file that
relate to CPS for services handled through xinetd. However, it may be
worthwhile to configure the service to listen directly to the port in
very high usage cases. Not sure if this is what you mean..
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

Pete Kay wrote:
> What happens is that at around 2K channels running g711 ( 64k) codec,
> all eth0 is used up and no more traffic can go through.
>

2000 * 64kbit/sec is 128Mbit/sec, not counting any additional protocol
overhead are you sending and receiving this data on the same
ethernet port? then its double that total traffic.
what speed is your ethernet ?


if all this data is flowing through a single UDP port, then its pretty
hard to spread it across multiple CPUs as only one application thread is
reading or writing from that socket and doing whatever sorts of routing
you're doing. you may well be better off with a server that has fewer
faster cores.




_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

----- "Pete Kay" <petedao@gmail.com> wrote:
> Hi,
>
> I am running a VOIP application on Centos 5.3. It is a 16 core box
> with 12 G of mem and all what it does is passing packets.
>
> What happens is that at around 2K channels running g711 ( 64k) codec,
> all eth0 is used up and no more traffic can go through.
>
> I have checked google and it talked about interrupt scheduler.
>
> does anyone know how to configure the kernel to allow it to use all
> CPSs for socket transmission of UDP packets?
>
> Any pointer will be greatly appreciated.
>
> pete

What kind of NICs are you using?

--Tim
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

Pete Kay wrote:

Hi,

I am running a VOIP application on Centos 5.3. It is a 16 core box
with 12 G of mem and all what it does is passing packets.

What happens is that at around 2K channels running g711 ( 64k) codec,
all eth0 is used up and no more traffic can go through.


So that's about 128Mbits per second not counting UDP packet overheads
I guesstimate that you cannot get much more than that through a 1G
ethernet NIC - it is not CP bound, rather you are nearing the capacity
of a single NIC.

HTH Rob

I have checked google and it talked about interrupt scheduler.

does anyone know how to configure the kernel to allow it to use all
CPSs for socket transmission of UDP packets?

Any pointer will be greatly appreciated.

pete
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

On Wednesday 24 February 2010 14:55:22 John R Pierce wrote:
> Pete Kay wrote:
> > What happens is that at around 2K channels running g711 ( 64k) codec,
> > all eth0 is used up and no more traffic can go through.
>
> 2000 * 64kbit/sec is 128Mbit/sec, not counting any additional protocol

Actually if you are running g.711 over VoIP, the overhead brings it up to
something like 87Kbps per direction.

Bobby
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

Bobby wrote:
> On Wednesday 24 February 2010 14:55:22 John R Pierce wrote:
>
>> Pete Kay wrote:
>>
>>> What happens is that at around 2K channels running g711 ( 64k) codec,
>>> all eth0 is used up and no more traffic can go through.
>>>
>> 2000 * 64kbit/sec is 128Mbit/sec, not counting any additional protocol
>>
>
> Actually if you are running g.711 over VoIP, the overhead brings it up to
> something like 87Kbps per direction.
>

k, that puts it up around 175Mbit/sec. (and thats just one way...if
these are full duplex conversations, then there's that amount of data
being recieved AND transmitted from each end. now we're up to
700Mbit/sec if we're looking at transmit and recieve in and out...






_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Desperately need help with multi-core NIC performance
 

Hi

So is that the limit? I have heard people being able to run like 10K
call channels before max out CPU cap.

Is this only possible if multiple nics are being used?

Please help.

pete

On Wed, Feb 24, 2010 at 2:53 PM, John R Pierce <pierce@hogranch.com> wrote:
> Bobby wrote:
>> On Wednesday 24 February 2010 14:55:22 John R Pierce wrote:
>>
>>> Pete Kay wrote:
>>>
>>>> What happens is that at around 2K channels running g711 ( 64k) codec,
>>>> all eth0 is used up and no more traffic can go through.
>>>>
>>> 2000 * 64kbit/sec is 128Mbit/sec, not counting any additional protocol
>>>
>>
>> Actually if you are running g.711 over VoIP, the overhead brings it up to
>> something like 87Kbps per direction.
>>
>
> k, that puts it up around 175Mbit/sec. * (and thats just one way...if
> these are full duplex conversations, then there's that amount of data
> being recieved AND transmitted from each end. * *now we're up to
> 700Mbit/sec if we're looking at transmit and recieve in and out...
>
>
>
>
>
>
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos