We have a RHEL 5 box which requires frequent flushing of arp cache to stay up and running and not create network bottlenecks. Anyone have these issues? we have quite a few servers and have never faced an issue before this one. I thought it might have to do with the switch but the network folks say the switch is clear. Any thoughts??

cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth4
Currently Active Slave: eth4
MII Status: up
MII Polling Interval (ms): 500
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:37:c9:38:e3:3b

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:11:17:4c:ee:22


Thanks

Brian.

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

On 07/12/2011 09:46 PM, brian irvin wrote:

We have a RHEL 5 box which requires frequent flushing of arp cache to stay up and running and not create network bottlenecks. Anyone have these issues? we have quite a few servers and have never faced an issue before this one. I thought it might have to do with the switch but the network folks say the switch is clear. Any thoughts??

cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth4
Currently Active Slave: eth4
MII Status: up
MII Polling Interval (ms): 500
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:37:c9:38:e3:3b

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:11:17:4c:ee:22


Thanks

Brian.



Can you clarify a bit the staying up and running/bottleneck issue? What
happens if you do not flush the ARP table in terms of what you see in
the system and what do you see on the LAN side for the system? Also
which Ethernet driver module you use in the system?


I have seen issues with IP aliasing (not HA bonding) on very busy LANs,
but long time ago, not on recent 5 releases. As far as I know, unless
you use a load balancing mode bonding, the netwrk switch should not be
tweaked.



GM

--
--
George Magklaras PhD
RHCE no: 805008309135525

Senior Systems Engineer/IT Manager
Biotek Center, University of Oslo
EMBnet TMPC Chair

http://folk.uio.no/georgios

Tel: +47 22840535

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

We are using bnx2 driver. We are getting outages when arp table fills up and unless we flush the table, network connectivity is an issue.

more ifcfg-bond0
DEVICE=bond0
BONDING_OPTS="mode=1 miimon=500 primary=eth4"


Thanks

Brian


--- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no> wrote:

> From: Georgios Magklaras <georgios@biotek.uio.no>
> Subject: Re: Arp Cache issue
> To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> Date: Tuesday, July 12, 2011, 9:41 PM
> On 07/12/2011 09:46 PM, brian irvin
> wrote:
> > We have a RHEL 5 box which requires frequent flushing
> of arp cache to stay up and running and not create network
> bottlenecks. Anyone have these issues? we have quite a few
> servers and have never faced an issue before this one. I
> thought it might have to do with the switch but the network
> folks say the switch is clear. Any thoughts??
> >
> > cat /proc/net/bonding/bond0
> > Ethernet Channel Bonding Driver: v3.4.0 (October 7,
> 2008)
> >
> > Bonding Mode: fault-tolerance (active-backup)
> > Primary Slave: eth4
> > Currently Active Slave: eth4
> > MII Status: up
> > MII Polling Interval (ms): 500
> > Up Delay (ms): 0
> > Down Delay (ms): 0
> >
> > Slave Interface: eth0
> > MII Status: up
> > Link Failure Count: 0
> > Permanent HW addr: 00:37:c9:38:e3:3b
> >
> > Slave Interface: eth4
> > MII Status: up
> > Link Failure Count: 0
> > Permanent HW addr: 00:11:17:4c:ee:22
> >
> >
> > Thanks
> >
> > Brian.
> >
>
> Can you clarify a bit the staying up and running/bottleneck
> issue? What happens if you do not flush the ARP table in
> terms of what you see in the system and what do you see on
> the LAN side for the system? Also which Ethernet driver
> module you use in the system?
>
> I have seen issues with IP aliasing (not HA bonding) on
> very busy LANs, but long time ago, not on recent 5 releases.
> As far as I know, unless you use a load balancing mode
> bonding, the netwrk switch should not be tweaked.
>
>
> GM
>
> -- -- George Magklaras PhD
> RHCE no: 805008309135525
>
> Senior Systems Engineer/IT Manager
> Biotek Center, University of Oslo
> EMBnet TMPC Chair
>
> http://folk.uio.no/georgios
>
> Tel: +47 22840535
>
> -- redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

Can you post your /etc/sysctl.conf file? Maybe the answer resides in your arp cache size limits and garbage collection frequency?

Thanks!

Josh Richardson
General Dynamics AIS

-----Original Message-----
From: redhat-list-bounces@redhat.com [mailto:redhat-list-bounces@redhat.com] On Behalf Of brian irvin
Sent: Wednesday, July 13, 2011 9:34 AM
To: General Red Hat Linux discussion list
Subject: Re: Arp Cache issue

We are using bnx2 driver. We are getting outages when arp table fills up and unless we flush the table, network connectivity is an issue.

more ifcfg-bond0
DEVICE=bond0
BONDING_OPTS="mode=1 miimon=500 primary=eth4"


Thanks

Brian


--- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no> wrote:

> From: Georgios Magklaras <georgios@biotek.uio.no>
> Subject: Re: Arp Cache issue
> To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> Date: Tuesday, July 12, 2011, 9:41 PM
> On 07/12/2011 09:46 PM, brian irvin
> wrote:
> > We have a RHEL 5 box which requires frequent flushing
> of arp cache to stay up and running and not create network
> bottlenecks. Anyone have these issues? we have quite a few
> servers and have never faced an issue before this one. I
> thought it might have to do with the switch but the network
> folks say the switch is clear. Any thoughts??
> >
> > cat /proc/net/bonding/bond0
> > Ethernet Channel Bonding Driver: v3.4.0 (October 7,
> 2008)
> >
> > Bonding Mode: fault-tolerance (active-backup)
> > Primary Slave: eth4
> > Currently Active Slave: eth4
> > MII Status: up
> > MII Polling Interval (ms): 500
> > Up Delay (ms): 0
> > Down Delay (ms): 0
> >
> > Slave Interface: eth0
> > MII Status: up
> > Link Failure Count: 0
> > Permanent HW addr: 00:37:c9:38:e3:3b
> >
> > Slave Interface: eth4
> > MII Status: up
> > Link Failure Count: 0
> > Permanent HW addr: 00:11:17:4c:ee:22
> >
> >
> > Thanks
> >
> > Brian.
> >
>
> Can you clarify a bit the staying up and running/bottleneck
> issue? What happens if you do not flush the ARP table in
> terms of what you see in the system and what do you see on
> the LAN side for the system? Also which Ethernet driver
> module you use in the system?
>
> I have seen issues with IP aliasing (not HA bonding) on
> very busy LANs, but long time ago, not on recent 5 releases.
> As far as I know, unless you use a load balancing mode
> bonding, the netwrk switch should not be tweaked.
>
>
> GM
>
> -- -- George Magklaras PhD
> RHCE no: 805008309135525
>
> Senior Systems Engineer/IT Manager
> Biotek Center, University of Oslo
> EMBnet TMPC Chair
>
> http://folk.uio.no/georgios
>
> Tel: +47 22840535
>
> -- redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

> We are using bnx2 driver. We are getting outages when arp table fills up
> and unless we flush the table, network connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"

Has your network team actually configured your switch ports for bonding?
We've found, in our environment, significant issues can occur if you're
bonding ports on the server side but the network team has not configured
the switch ports for bonding.

--
Mike Burger
http://www.bubbanfriends.org

Visit the Dog Pound II BBS
telnet://dogpound2.citadel.org or http://dogpound2.citadel.org

To be notified of updates to the web site, visit:

https://www.bubbanfriends.org/mailman/listinfo/site-update

or send a blank email message to:

site-update-subscribe@bubbanfriends.org

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

> We are using bnx2 driver. We are getting outages when arp table fills up
> and unless we flush the table, network connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"

Has your network team actually configured your switch ports for bonding?
We've found, in our environment, significant issues can occur if you're
bonding ports on the server side but the network team has not configured
the switch ports for bonding.

--
Mike Burger
http://www.bubbanfriends.org

Visit the Dog Pound II BBS
telnet://dogpound2.citadel.org or http://dogpound2.citadel.org

To be notified of updates to the web site, visit:

https://www.bubbanfriends.org/mailman/listinfo/site-update

or send a blank email message to:

site-update-subscribe@bubbanfriends.org

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

Posting sysctl.conf

# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
kernel.sysrq = 1

#
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_orphan_retries = 3

Thanks

Brian

> -----Ursprüngliche Nachricht-----
> Von: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> Im Auftrag von Richardson, Joshua A.
> Gesendet: Mittwoch, 13. Juli 2011 16:18
> An: General Red Hat Linux discussion list
> Betreff: RE: Arp Cache issue
>
> Can you post your /etc/sysctl.conf file?* Maybe the
> answer resides in your arp cache size limits and garbage
> collection frequency?
>
> Thanks!
>
> Josh Richardson
> General Dynamics AIS
>
> -----Original Message-----
> From: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> On Behalf Of brian irvin
> Sent: Wednesday, July 13, 2011 9:34 AM
> To: General Red Hat Linux discussion list
> Subject: Re: Arp Cache issue
>
> We are using bnx2 driver. We are getting outages when arp
> table fills up and unless we flush the table, network
> connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"*
>
>
> Thanks
>
> Brian
>
>
> --- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no>
> wrote:
>
> > From: Georgios Magklaras <georgios@biotek.uio.no>
> > Subject: Re: Arp Cache issue
> > To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> > Date: Tuesday, July 12, 2011, 9:41 PM
> > On 07/12/2011 09:46 PM, brian irvin
> > wrote:
> > > We have a RHEL 5 box which requires frequent
> flushing
> > of arp cache to stay up and running and not create
> network
> > bottlenecks. Anyone have these issues? we have quite a
> few servers and
> > have never faced an issue before this one. I thought
> it might have to
> > do with the switch but the network folks say the
> switch is clear. Any
> > thoughts??
> > >
> > > cat /proc/net/bonding/bond0
> > > Ethernet Channel Bonding Driver: v3.4.0 (October
> 7,
> > 2008)
> > >
> > > Bonding Mode: fault-tolerance (active-backup)
> Primary Slave: eth4
> > > Currently Active Slave: eth4 MII Status: up MII
> Polling Interval
> > > (ms): 500 Up Delay (ms): 0 Down Delay (ms): 0
> > >
> > > Slave Interface: eth0
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:37:c9:38:e3:3b
> > >
> > > Slave Interface: eth4
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:11:17:4c:ee:22
> > >
> > >
> > > Thanks
> > >
> > > Brian.
> > >
> >
> > Can you clarify a bit the staying up and
> running/bottleneck issue?
> > What happens if you do not flush the ARP table in
> terms of what you
> > see in the system and what do you see on the LAN side
> for the system?
> > Also which Ethernet driver module you use in the
> system?
> >
> > I have seen issues with IP aliasing (not HA bonding)
> on very busy
> > LANs, but long time ago, not on recent 5 releases.
> > As far as I know, unless you use a load balancing mode
> bonding, the
> > netwrk switch should not be tweaked.
> >
> >
> > GM
> >
> > -- -- George Magklaras PhD
> > RHCE no: 805008309135525
> >
> > Senior Systems Engineer/IT Manager
> > Biotek Center, University of Oslo
> > EMBnet TMPC Chair
> >
> > http://folk.uio.no/georgios
> >
> > Tel: +47 22840535
> >
> > -- redhat-list mailing list
> > unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> > https://www.redhat.com/mailman/listinfo/redhat-list
> >
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

Posting sysctl.conf

# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
kernel.sysrq = 1

#
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_orphan_retries = 3

Thanks

Brian

> -----Ursprüngliche Nachricht-----
> Von: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> Im Auftrag von Richardson, Joshua A.
> Gesendet: Mittwoch, 13. Juli 2011 16:18
> An: General Red Hat Linux discussion list
> Betreff: RE: Arp Cache issue
>
> Can you post your /etc/sysctl.conf file?* Maybe the
> answer resides in your arp cache size limits and garbage
> collection frequency?
>
> Thanks!
>
> Josh Richardson
> General Dynamics AIS
>
> -----Original Message-----
> From: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> On Behalf Of brian irvin
> Sent: Wednesday, July 13, 2011 9:34 AM
> To: General Red Hat Linux discussion list
> Subject: Re: Arp Cache issue
>
> We are using bnx2 driver. We are getting outages when arp
> table fills up and unless we flush the table, network
> connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"*
>
>
> Thanks
>
> Brian
>
>
> --- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no>
> wrote:
>
> > From: Georgios Magklaras <georgios@biotek.uio.no>
> > Subject: Re: Arp Cache issue
> > To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> > Date: Tuesday, July 12, 2011, 9:41 PM
> > On 07/12/2011 09:46 PM, brian irvin
> > wrote:
> > > We have a RHEL 5 box which requires frequent
> flushing
> > of arp cache to stay up and running and not create
> network
> > bottlenecks. Anyone have these issues? we have quite a
> few servers and
> > have never faced an issue before this one. I thought
> it might have to
> > do with the switch but the network folks say the
> switch is clear. Any
> > thoughts??
> > >
> > > cat /proc/net/bonding/bond0
> > > Ethernet Channel Bonding Driver: v3.4.0 (October
> 7,
> > 2008)
> > >
> > > Bonding Mode: fault-tolerance (active-backup)
> Primary Slave: eth4
> > > Currently Active Slave: eth4 MII Status: up MII
> Polling Interval
> > > (ms): 500 Up Delay (ms): 0 Down Delay (ms): 0
> > >
> > > Slave Interface: eth0
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:37:c9:38:e3:3b
> > >
> > > Slave Interface: eth4
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:11:17:4c:ee:22
> > >
> > >
> > > Thanks
> > >
> > > Brian.
> > >
> >
> > Can you clarify a bit the staying up and
> running/bottleneck issue?
> > What happens if you do not flush the ARP table in
> terms of what you
> > see in the system and what do you see on the LAN side
> for the system?
> > Also which Ethernet driver module you use in the
> system?
> >
> > I have seen issues with IP aliasing (not HA bonding)
> on very busy
> > LANs, but long time ago, not on recent 5 releases.
> > As far as I know, unless you use a load balancing mode
> bonding, the
> > netwrk switch should not be tweaked.
> >
> >
> > GM
> >
> > -- -- George Magklaras PhD
> > RHCE no: 805008309135525
> >
> > Senior Systems Engineer/IT Manager
> > Biotek Center, University of Oslo
> > EMBnet TMPC Chair
> >
> > http://folk.uio.no/georgios
> >
> > Tel: +47 22840535
> >
> > -- redhat-list mailing list
> > unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> > https://www.redhat.com/mailman/listinfo/redhat-list
> >
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

It may not be answer actual solution as to why your network bottlenecks, but this document might help you to tune the arp cache on the machine so the table doesn't fill up as rapidly or up the default garbage collection time to clear out the table?

http://www.google.com/url?sa=t&source=web&cd=1&ved=0CBUQFjAA&url=http%3A %2F%2Fwww.networknuts.net%2FPerformance%2520Tuning-4.pdf&rct=j&q=redhat%20arp%20cache%20tuning%20sysc tl&ei=tKgdTvKkEcbSgQeq9Yj8CQ&usg=AFQjCNF_qyyhAoXKa fZUw4xQoISNsuYPZQ&cad=rja

Josh Richardson
General Dynamics AIS
Office: 703-272-1761
Cell: 202-400-1733


-----Original Message-----
From: redhat-list-bounces@redhat.com [mailto:redhat-list-bounces@redhat.com] On Behalf Of brian irvin
Sent: Wednesday, July 13, 2011 11:00 AM
To: General Red Hat Linux discussion list
Subject: Re:Arp Cache issue

Posting sysctl.conf

# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
kernel.sysrq = 1

#
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_orphan_retries = 3

Thanks

Brian

> -----Ursprüngliche Nachricht-----
> Von: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> Im Auftrag von Richardson, Joshua A.
> Gesendet: Mittwoch, 13. Juli 2011 16:18
> An: General Red Hat Linux discussion list
> Betreff: RE: Arp Cache issue
>
> Can you post your /etc/sysctl.conf file?* Maybe the
> answer resides in your arp cache size limits and garbage
> collection frequency?
>
> Thanks!
>
> Josh Richardson
> General Dynamics AIS
>
> -----Original Message-----
> From: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> On Behalf Of brian irvin
> Sent: Wednesday, July 13, 2011 9:34 AM
> To: General Red Hat Linux discussion list
> Subject: Re: Arp Cache issue
>
> We are using bnx2 driver. We are getting outages when arp
> table fills up and unless we flush the table, network
> connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"*
>
>
> Thanks
>
> Brian
>
>
> --- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no>
> wrote:
>
> > From: Georgios Magklaras <georgios@biotek.uio.no>
> > Subject: Re: Arp Cache issue
> > To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> > Date: Tuesday, July 12, 2011, 9:41 PM
> > On 07/12/2011 09:46 PM, brian irvin
> > wrote:
> > > We have a RHEL 5 box which requires frequent
> flushing
> > of arp cache to stay up and running and not create
> network
> > bottlenecks. Anyone have these issues? we have quite a
> few servers and
> > have never faced an issue before this one. I thought
> it might have to
> > do with the switch but the network folks say the
> switch is clear. Any
> > thoughts??
> > >
> > > cat /proc/net/bonding/bond0
> > > Ethernet Channel Bonding Driver: v3.4.0 (October
> 7,
> > 2008)
> > >
> > > Bonding Mode: fault-tolerance (active-backup)
> Primary Slave: eth4
> > > Currently Active Slave: eth4 MII Status: up MII
> Polling Interval
> > > (ms): 500 Up Delay (ms): 0 Down Delay (ms): 0
> > >
> > > Slave Interface: eth0
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:37:c9:38:e3:3b
> > >
> > > Slave Interface: eth4
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:11:17:4c:ee:22
> > >
> > >
> > > Thanks
> > >
> > > Brian.
> > >
> >
> > Can you clarify a bit the staying up and
> running/bottleneck issue?
> > What happens if you do not flush the ARP table in
> terms of what you
> > see in the system and what do you see on the LAN side
> for the system?
> > Also which Ethernet driver module you use in the
> system?
> >
> > I have seen issues with IP aliasing (not HA bonding)
> on very busy
> > LANs, but long time ago, not on recent 5 releases.
> > As far as I know, unless you use a load balancing mode
> bonding, the
> > netwrk switch should not be tweaked.
> >
> >
> > GM
> >
> > -- -- George Magklaras PhD
> > RHCE no: 805008309135525
> >
> > Senior Systems Engineer/IT Manager
> > Biotek Center, University of Oslo
> > EMBnet TMPC Chair
> >
> > http://folk.uio.no/georgios
> >
> > Tel: +47 22840535
> >
> > -- redhat-list mailing list
> > unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> > https://www.redhat.com/mailman/listinfo/redhat-list
> >
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

It may not be answer actual solution as to why your network bottlenecks, but this document might help you to tune the arp cache on the machine so the table doesn't fill up as rapidly or up the default garbage collection time to clear out the table?

http://www.google.com/url?sa=t&source=web&cd=1&ved=0CBUQFjAA&url=http%3A %2F%2Fwww.networknuts.net%2FPerformance%2520Tuning-4.pdf&rct=j&q=redhat%20arp%20cache%20tuning%20sysc tl&ei=tKgdTvKkEcbSgQeq9Yj8CQ&usg=AFQjCNF_qyyhAoXKa fZUw4xQoISNsuYPZQ&cad=rja

Josh Richardson
General Dynamics AIS
Office: 703-272-1761
Cell: 202-400-1733


-----Original Message-----
From: redhat-list-bounces@redhat.com [mailto:redhat-list-bounces@redhat.com] On Behalf Of brian irvin
Sent: Wednesday, July 13, 2011 11:00 AM
To: General Red Hat Linux discussion list
Subject: Re:Arp Cache issue

Posting sysctl.conf

# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
kernel.sysrq = 1

#
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_orphan_retries = 3

Thanks

Brian

> -----Ursprüngliche Nachricht-----
> Von: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> Im Auftrag von Richardson, Joshua A.
> Gesendet: Mittwoch, 13. Juli 2011 16:18
> An: General Red Hat Linux discussion list
> Betreff: RE: Arp Cache issue
>
> Can you post your /etc/sysctl.conf file?* Maybe the
> answer resides in your arp cache size limits and garbage
> collection frequency?
>
> Thanks!
>
> Josh Richardson
> General Dynamics AIS
>
> -----Original Message-----
> From: redhat-list-bounces@redhat.com
> [mailto:redhat-list-bounces@redhat.com]
> On Behalf Of brian irvin
> Sent: Wednesday, July 13, 2011 9:34 AM
> To: General Red Hat Linux discussion list
> Subject: Re: Arp Cache issue
>
> We are using bnx2 driver. We are getting outages when arp
> table fills up and unless we flush the table, network
> connectivity is an issue.
>
> more ifcfg-bond0
> DEVICE=bond0
> BONDING_OPTS="mode=1 miimon=500 primary=eth4"*
>
>
> Thanks
>
> Brian
>
>
> --- On Tue, 7/12/11, Georgios Magklaras <georgios@biotek.uio.no>
> wrote:
>
> > From: Georgios Magklaras <georgios@biotek.uio.no>
> > Subject: Re: Arp Cache issue
> > To: "General Red Hat Linux discussion list" <redhat-list@redhat.com>
> > Date: Tuesday, July 12, 2011, 9:41 PM
> > On 07/12/2011 09:46 PM, brian irvin
> > wrote:
> > > We have a RHEL 5 box which requires frequent
> flushing
> > of arp cache to stay up and running and not create
> network
> > bottlenecks. Anyone have these issues? we have quite a
> few servers and
> > have never faced an issue before this one. I thought
> it might have to
> > do with the switch but the network folks say the
> switch is clear. Any
> > thoughts??
> > >
> > > cat /proc/net/bonding/bond0
> > > Ethernet Channel Bonding Driver: v3.4.0 (October
> 7,
> > 2008)
> > >
> > > Bonding Mode: fault-tolerance (active-backup)
> Primary Slave: eth4
> > > Currently Active Slave: eth4 MII Status: up MII
> Polling Interval
> > > (ms): 500 Up Delay (ms): 0 Down Delay (ms): 0
> > >
> > > Slave Interface: eth0
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:37:c9:38:e3:3b
> > >
> > > Slave Interface: eth4
> > > MII Status: up
> > > Link Failure Count: 0
> > > Permanent HW addr: 00:11:17:4c:ee:22
> > >
> > >
> > > Thanks
> > >
> > > Brian.
> > >
> >
> > Can you clarify a bit the staying up and
> running/bottleneck issue?
> > What happens if you do not flush the ARP table in
> terms of what you
> > see in the system and what do you see on the LAN side
> for the system?
> > Also which Ethernet driver module you use in the
> system?
> >
> > I have seen issues with IP aliasing (not HA bonding)
> on very busy
> > LANs, but long time ago, not on recent 5 releases.
> > As far as I know, unless you use a load balancing mode
> bonding, the
> > netwrk switch should not be tweaked.
> >
> >
> > GM
> >
> > -- -- George Magklaras PhD
> > RHCE no: 805008309135525
> >
> > Senior Systems Engineer/IT Manager
> > Biotek Center, University of Oslo
> > EMBnet TMPC Chair
> >
> > http://folk.uio.no/georgios
> >
> > Tel: +47 22840535
> >
> > -- redhat-list mailing list
> > unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> > https://www.redhat.com/mailman/listinfo/redhat-list
> >
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>
> --
> redhat-list mailing list
> unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/redhat-list
>

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list