Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian User (http://www.linux-archive.org/debian-user/)
-   -   NetXtreme BCM5722 strangeness on Proliant ML115 (http://www.linux-archive.org/debian-user/680657-netxtreme-bcm5722-strangeness-proliant-ml115.html)

Berni Elbourn 07-06-2012 08:46 AM

NetXtreme BCM5722 strangeness on Proliant ML115
 
11:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722 Gigabit Ethernet PCI Express

With standard squeeze kernel seemingly once a day this nic stops working. Looking at the switch port it seems the nic is
transmitting (or retransmitting) something as fast as possible. A simple ifdown/ifup seems to cure for another day.


On latest backport kernel 3.2.20-1~bpo60+1 (installed today) the nic does seems to be stable. iperf reports 1gb
performance. However the count of dropped packets is slowly growing:


eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
inet addr:192.168.2.10 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::221:5aff:fed3:d00c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8801144 errors:0 dropped:1824 overruns:0 frame:0
TX packets:2263228 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1257

This issue is present in the firmware nonfree from stable, and backports, and the using the latest source compiled from
broadcom 3.122n. I have changed cables and switch ports. There is another other gigabit nic on the switch is nVidia
Corporation MCP77 and this has no errors or dropped packets.


Anyone else seeing this? How to progress? .. should I log a debian bug, or just go buy an Intel card? Or ? :-)

Berni

--
"Confidence is what you have before you understand a problem" - Woody Allen


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4FF6A5E9.90409@gmail.com">http://lists.debian.org/4FF6A5E9.90409@gmail.com

Camaleón 07-07-2012 02:18 PM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On Fri, 06 Jul 2012 09:46:33 +0100, Berni Elbourn wrote:

> 11:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722
> Gigabit Ethernet PCI Express
>
> With standard squeeze kernel seemingly once a day this nic stops
> working. Looking at the switch port it seems the nic is transmitting (or
> retransmitting) something as fast as possible. A simple ifdown/ifup
> seems to cure for another day.
>
> On latest backport kernel 3.2.20-1~bpo60+1 (installed today) the nic
> does seems to be stable. iperf reports 1gb performance. However the
> count of dropped packets is slowly growing:

(...)

> RX packets:8801144 errors:0 dropped:1824 overruns:0 frame:0

(...)

> This issue is present in the firmware nonfree from stable, and
> backports, and the using the latest source compiled from broadcom
> 3.122n. I have changed cables and switch ports. There is another other
> gigabit nic on the switch is nVidia Corporation MCP77 and this has no
> errors or dropped packets.

So you have tested with almost all of the possibilities (you've discarded
a hardware issue by replacing the patch cord and using a different switch
port and you've discarded a software/driver problem by installing a
different kernel and the latest available broadcom driver) yet still you
don't see a noticeable improvement on this, right? Then it can be
something specific to your setup/environment... I would start with the
ouput of "ethtool eth0" and "ethtool -k eth0" just in case.

Have you noted an increment of packets being dropped when the system is
running a concrete task or process that can exhaust the available memory?
I ask this because Google suggest that dropped packates can be related to
low memory situations :-?

> Anyone else seeing this? How to progress? .. should I log a debian bug,
> or just go buy an Intel card? Or ? :-)

In workstations and servers I always try to have at least a couple of
different NIC cards (from different manufacturers and models) just to
prevent these situations, because if you think about it, what's a server
with no network connection? Nowadays, close to nothing; a toaster is even
more useful :-)

Greetings,

--
Camaleón


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/jt9gfr$2af$10@dough.gmane.org

Berni Elbourn 07-09-2012 09:35 AM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On 07/07/12 15:18, Camaleón wrote:

On Fri, 06 Jul 2012 09:46:33 +0100, Berni Elbourn wrote:


11:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722
Gigabit Ethernet PCI Express

With standard squeeze kernel seemingly once a day this nic stops
working. Looking at the switch port it seems the nic is transmitting (or
retransmitting) something as fast as possible. A simple ifdown/ifup
seems to cure for another day.

On latest backport kernel 3.2.20-1~bpo60+1 (installed today) the nic
does seems to be stable. iperf reports 1gb performance. However the
count of dropped packets is slowly growing:


(...)


RX packets:8801144 errors:0 dropped:1824 overruns:0 frame:0


(...)


This issue is present in the firmware nonfree from stable, and
backports, and the using the latest source compiled from broadcom
3.122n. I have changed cables and switch ports. There is another other
gigabit nic on the switch is nVidia Corporation MCP77 and this has no
errors or dropped packets.


So you have tested with almost all of the possibilities (you've discarded
a hardware issue by replacing the patch cord and using a different switch
port and you've discarded a software/driver problem by installing a
different kernel and the latest available broadcom driver) yet still you
don't see a noticeable improvement on this, right? Then it can be
something specific to your setup/environment... I would start with the
ouput of "ethtool eth0" and "ethtool -k eth0" just in case.

Have you noted an increment of packets being dropped when the system is
running a concrete task or process that can exhaust the available memory?
I ask this because Google suggest that dropped packates can be related to
low memory situations :-?


Anyone else seeing this? How to progress? .. should I log a debian bug,
or just go buy an Intel card? Or ? :-)


In workstations and servers I always try to have at least a couple of
different NIC cards (from different manufacturers and models) just to
prevent these situations, because if you think about it, what's a server
with no network connection? Nowadays, close to nothing; a toaster is even
more useful :-)

Greetings,



Strangest thing. The dropped packets stopped at 3000 odd. After reboots (this machine is shutdown overnight) the number
of dropped packets seems to stop incrementing at 20 or 30 or so even after several gigabytes of transfer.


Performance however is fine with the backports kernel:

$ cat /proc/version
Linux version 3.2.0-0.bpo.2-amd64 (Debian 3.2.20-1~bpo60+1) (debian-kernel@lists.debian.org) (gcc version 4.4.5 (Debian
4.4.5-8) ) #1 SMP Fri Jun 29 20:42:29 UTC 2012


$ sudo ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
Link detected: yes

$ sudo ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
ntuple-filters: off
receive-hashing: off

$ sudo ethtool -i eth0
driver: tg3
version: 3.121
firmware-version: 5722-v3.07, ASFIPMI v6.02
bus-info: 0000:11:00.0

eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
inet addr:192.168.2.10 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::221:5aff:fed3:d00c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:123154 errors:0 *dropped:26* overruns:0 frame:0
TX packets:131936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15913111 (15.1 MiB) TX bytes:107913843 (102.9 MiB)
Interrupt:19



--
"Confidence is what you have before you understand a problem" - Woody Allen


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4FFAA5DC.5080702@gmail.com">http://lists.debian.org/4FFAA5DC.5080702@gmail.com

Camaleón 07-09-2012 02:14 PM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On Mon, 09 Jul 2012 10:35:24 +0100, Berni Elbourn wrote:

> On 07/07/12 15:18, Camaleón wrote:

(...)

>> Have you noted an increment of packets being dropped when the system is
>> running a concrete task or process that can exhaust the available
>> memory? I ask this because Google suggest that dropped packates can be
>> related to low memory situations :-?
>>
>>> Anyone else seeing this? How to progress? .. should I log a debian
>>> bug, or just go buy an Intel card? Or ? :-)
>>
>> In workstations and servers I always try to have at least a couple of
>> different NIC cards (from different manufacturers and models) just to
>> prevent these situations, because if you think about it, what's a
>> server with no network connection? Nowadays, close to nothing; a
>> toaster is even more useful :-)
>>
> Strangest thing. The dropped packets stopped at 3000 odd. After reboots
> (this machine is shutdown overnight) the number of dropped packets seems
> to stop incrementing at 20 or 30 or so even after several gigabytes of
> transfer.

You mean with Squeeze's stock kernel or the backported one? :-?

> Performance however is fine with the backports kernel:

If there's a noticeable difference between both kernels, I would report
it just in the event the problem can be addressed and patched for the
upcoming dot point releases.

> $ cat /proc/version
> Linux version 3.2.0-0.bpo.2-amd64 (Debian 3.2.20-1~bpo60+1)
> (debian-kernel@lists.debian.org) (gcc version 4.4.5 (Debian 4.4.5-8) )
> #1 SMP Fri Jun 29 20:42:29 UTC 2012
>
> $ sudo ethtool eth0

(...)

This output looks normal.

> $ sudo ethtool -k eth0

(...)

> $ sudo ethtool -i eth0
> driver: tg3
> version: 3.121
> firmware-version: 5722-v3.07, ASFIPMI v6.02 bus-info: 0000:11:00.0
^^^^^^^^^^

And also these.

Well, you can check if there's the possibility fo getting an updated
firmware but for NICs I never had to did an update before :-?


> eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
> inet addr:192.168.2.10 Bcast:192.168.2.255
> Mask:255.255.255.0 inet6 addr: fe80::221:5aff:fed3:d00c/64
> Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:123154 errors:0 *dropped:26* overruns:0 frame:0 TX
> packets:131936 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:15913111 (15.1 MiB) TX bytes:107913843 (102.9 MiB)
> Interrupt:19

Despite the small number of dropped packages (26) the total ammount of
received packages is also very low (15.1 MiB), there shouldn't be a
single drop.

Is "dmesg | grep -i eth0" showing any anomaly?

Greetings,

--
Camaleón


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/jtep03$sc9$10@dough.gmane.org

Berni Elbourn 07-09-2012 04:31 PM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On 09/07/12 15:14, Camaleón wrote:

On Mon, 09 Jul 2012 10:35:24 +0100, Berni Elbourn wrote:


On 07/07/12 15:18, Camaleón wrote:


(...)


Have you noted an increment of packets being dropped when the system is
running a concrete task or process that can exhaust the available
memory? I ask this because Google suggest that dropped packates can be
related to low memory situations :-?


Anyone else seeing this? How to progress? .. should I log a debian
bug, or just go buy an Intel card? Or ? :-)


In workstations and servers I always try to have at least a couple of
different NIC cards (from different manufacturers and models) just to
prevent these situations, because if you think about it, what's a
server with no network connection? Nowadays, close to nothing; a
toaster is even more useful :-)


Strangest thing. The dropped packets stopped at 3000 odd. After reboots
(this machine is shutdown overnight) the number of dropped packets seems
to stop incrementing at 20 or 30 or so even after several gigabytes of
transfer.


You mean with Squeeze's stock kernel or the backported one? :-?


Performance however is fine with the backports kernel:


If there's a noticeable difference between both kernels, I would report
it just in the event the problem can be addressed and patched for the
upcoming dot point releases.


$ cat /proc/version
Linux version 3.2.0-0.bpo.2-amd64 (Debian 3.2.20-1~bpo60+1)
(debian-kernel@lists.debian.org) (gcc version 4.4.5 (Debian 4.4.5-8) )
#1 SMP Fri Jun 29 20:42:29 UTC 2012

$ sudo ethtool eth0


(...)

This output looks normal.


$ sudo ethtool -k eth0


(...)


$ sudo ethtool -i eth0
driver: tg3
version: 3.121
firmware-version: 5722-v3.07, ASFIPMI v6.02 bus-info: 0000:11:00.0

^^^^^^^^^^

And also these.

Well, you can check if there's the possibility fo getting an updated
firmware but for NICs I never had to did an update before :-?



eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
inet addr:192.168.2.10 Bcast:192.168.2.255
Mask:255.255.255.0 inet6 addr: fe80::221:5aff:fed3:d00c/64
Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:123154 errors:0 *dropped:26* overruns:0 frame:0 TX
packets:131936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15913111 (15.1 MiB) TX bytes:107913843 (102.9 MiB)
Interrupt:19


Despite the small number of dropped packages (26) the total ammount of
received packages is also very low (15.1 MiB), there shouldn't be a
single drop.

Is "dmesg | grep -i eth0" showing any anomaly?

Greetings,



I'll give the code from the broadcom website a wiz...just for the record here is the status after a few hours of use:

# sudo ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
inet addr:192.168.2.10 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::221:5aff:fed3:d00c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14050436 errors:0 dropped:26 overruns:0 frame:2
TX packets:3730779 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20348704555 (18.9 GiB) TX bytes:614582976 (586.1 MiB)
Interrupt:19

# dmesg | grep eth0
[ 1.191804] e100 0000:02:04.0: eth0: addr 0xfafff000, irq 18, MAC addr 00:90:27:b0:0a:7d
[ 6.508590] udev[474]: renamed network interface eth1 to eth1-eth0
[ 6.532526] udev[470]: renamed network interface eth0 to eth1
[ 6.584538] udev[474]: renamed network interface eth1-eth0 to eth0
[ 11.677058] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 14.772314] tg3 0000:11:00.0: eth0: Link is up at 1000 Mbps, full duplex
[ 14.772319] tg3 0000:11:00.0: eth0: Flow control is on for TX and on for RX
[ 14.772698] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 25.520006] eth0: no IPv6 routers present


--
"Confidence is what you have before you understand a problem" - Woody Allen


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4FFB0772.6020208@gmail.com">http://lists.debian.org/4FFB0772.6020208@gmail.com

Camaleón 07-09-2012 04:44 PM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On Mon, 09 Jul 2012 17:31:46 +0100, Berni Elbourn wrote:

> On 09/07/12 15:14, Camaleón wrote:

(...)

>> Despite the small number of dropped packages (26) the total ammount of
>> received packages is also very low (15.1 MiB), there shouldn't be a
>> single drop.
>>
>> Is "dmesg | grep -i eth0" showing any anomaly?
>>
>>
>>
> I'll give the code from the broadcom website a wiz...just for the record
> here is the status after a few hours of use:
>
> # sudo ifconfig eth0
> eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
> inet addr:192.168.2.10 Bcast:192.168.2.255
> Mask:255.255.255.0 inet6 addr: fe80::221:5aff:fed3:d00c/64
> Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:14050436 errors:0 dropped:26 overruns:0 frame:2 TX
> packets:3730779 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:20348704555 (18.9 GiB) TX bytes:614582976 (586.1
> MiB) Interrupt:19

That's more reasonable as there are no additional dropped packages but
traffic has increased a lot.

> # dmesg | grep eth0
> [ 1.191804] e100 0000:02:04.0: eth0: addr 0xfafff000, irq 18, MAC addr 00:90:27:b0:0a:7d
> [ 6.508590] udev[474]: renamed network interface eth1 to eth1-eth0
> [ 6.532526] udev[470]: renamed network interface eth0 to eth1
> [ 6.584538] udev[474]: renamed network interface eth1-eth0 to eth0
> [ 11.677058] ADDRCONF(NETDEV_UP): eth0: link is not ready
> [ 14.772314] tg3 0000:11:00.0: eth0: Link is up at 1000 Mbps, full duplex
> [ 14.772319] tg3 0000:11:00.0: eth0: Flow control is on for TX and on for RX
> [ 14.772698] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [ 25.520006] eth0: no IPv6 routers present

Beyond the "swaping dance" (eth1 → eth0) the rest looks normal.

I would keep monitoring the interface and the number of dropped packages
for a while, but regardless the backported kernel is working fine, I'd
open a bug report so kernel developers review this.

Greetings,

--
Camaleón


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/jtf1p9$sc9$14@dough.gmane.org

Berni Elbourn 07-10-2012 03:07 PM

NetXtreme BCM5722 strangeness on Proliant ML115
 
On 09/07/12 17:44, Camaleón wrote:

On Mon, 09 Jul 2012 17:31:46 +0100, Berni Elbourn wrote:


On 09/07/12 15:14, Camaleón wrote:


(...)


Despite the small number of dropped packages (26) the total ammount of
received packages is also very low (15.1 MiB), there shouldn't be a
single drop.

Is "dmesg | grep -i eth0" showing any anomaly?




I'll give the code from the broadcom website a wiz...just for the record
here is the status after a few hours of use:

# sudo ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:21:5a:d3:d0:0c
inet addr:192.168.2.10 Bcast:192.168.2.255
Mask:255.255.255.0 inet6 addr: fe80::221:5aff:fed3:d00c/64
Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14050436 errors:0 dropped:26 overruns:0 frame:2 TX
packets:3730779 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20348704555 (18.9 GiB) TX bytes:614582976 (586.1
MiB) Interrupt:19


That's more reasonable as there are no additional dropped packages but
traffic has increased a lot.


# dmesg | grep eth0
[ 1.191804] e100 0000:02:04.0: eth0: addr 0xfafff000, irq 18, MAC addr 00:90:27:b0:0a:7d
[ 6.508590] udev[474]: renamed network interface eth1 to eth1-eth0
[ 6.532526] udev[470]: renamed network interface eth0 to eth1
[ 6.584538] udev[474]: renamed network interface eth1-eth0 to eth0
[ 11.677058] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 14.772314] tg3 0000:11:00.0: eth0: Link is up at 1000 Mbps, full duplex
[ 14.772319] tg3 0000:11:00.0: eth0: Flow control is on for TX and on for RX
[ 14.772698] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 25.520006] eth0: no IPv6 routers present


Beyond the "swaping dance" (eth1 → eth0) the rest looks normal.

I would keep monitoring the interface and the number of dropped packages
for a while, but regardless the backported kernel is working fine, I'd
open a bug report so kernel developers review this.

Greetings,



Here we go...

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=681089

--
"Confidence is what you have before you understand a problem" - Woody Allen


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4FFC4527.1000304@gmail.com">http://lists.debian.org/4FFC4527.1000304@gmail.com


All times are GMT. The time now is 08:27 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.