FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 08-20-2010, 07:36 AM
Timo Schoeler
 
Default Cannot set MTU != 1500 on Intel NIC

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

thus Timo Schoeler spake:
> Hi list,

Little followup,

> I have a *very* strange problem, unfortunately it's kind of a show
> stopper regarding the deployment of the machine.
>
> I have two Intel Gigabit Ethernet NICs on board (Supermicro-based
> Server), quoting lspci (full output see at the end of the email):
>
> 0d:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
> Controller (Copper) (rev 03)
> 0f:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet
> Controller

I tried with Intel's most recent driver, it shows the same behaviour:

[root@bla ~]# ip link set dev eth0 mtu 1576
SIOCSIFMTU: Invalid argument
[root@bla ~]# ip link set dev eth1 mtu 1576
[root@bla ~]# ip link show dev eth1
6: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1576 qdisc pfifo_fast
qlen 1000
link/ether 00:30:48:xx:xx:xy brd ff:ff:ff:ff:ff:ff

Timo

> While the first device (seen as eth0) does *not* allow to set a
> different MTU (tried with 1576 and 1546, both values I could use; I am
> forced to use values >1500 here due to protocol stuff), the second works
> without a problem.
>
> dmesg shows:
>
> eth0: Unsupported MTU setting
> ADDRCONF(NETDEV_UP): eth0: link is not ready
> eth1: changing MTU from 1500 to 1576
> ADDRCONF(NETDEV_UP): eth1: link is not ready
> 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
> All bugs added by David S. Miller <davem@redhat.com>
> ADDRCONF(NETDEV_UP): eth0.543: link is not ready
> ADDRCONF(NETDEV_UP): eth0.820: link is not ready
> e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
> ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> ADDRCONF(NETDEV_CHANGE): eth0.543: link becomes ready
> ADDRCONF(NETDEV_CHANGE): eth0.820: link becomes ready
> e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
> ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
>
> The MTU is set as usual in the ifcfg-ethN files as 'MTU=1576', e.g.
>
> I tried with kernel 2.6.18-194.8.1.el5.028stab070.2 (OpenVZ) as well as
> the vanilla CentOS kernels 2.6.18-194.8.1.el5 and 2.6.18-194.11.1.el5 --
> same problem.
>
> Has anybody an idea what goes wrong here?
>
> Thanks,
>
> Timo
>
> ---
>
> 0d:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
> Controller (Copper) (rev 03)
> Subsystem: Super Micro Computer Inc Unknown device 108c
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B-
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 90
> Region 0: Memory at d0200000 (32-bit, non-prefetchable) [size=128K]
> Region 2: I/O ports at 2000 [size=32]
> Capabilities: [c8] Power Management version 2
> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> PME(D0+,D1-,D2-,D3hot+,D3cold+)
> Status: D0 PME-Enable- DSel=0 DScale=1 PME-
> Capabilities: [d0] Message Signalled Interrupts: 64bit+
> Queue=0/0 Enable+
> Address: 00000000fee01000 Data: 405a
> Capabilities: [e0] Express Endpoint IRQ 0
> Device: Supported: MaxPayload 256 bytes, PhantFunc 0,
> ExtTag-
> Device: Latency L0s <512ns, L1 <64us
> Device: AtnBtn- AtnInd- PwrInd-
> Device: Errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> Device: MaxPayload 128 bytes, MaxReadReq 512 bytes
> Link: Supported Speed 2.5Gb/s, Width x1, ASPM unknown,
> Port 0
> Link: Latency L0s <128ns, L1 <64us
> Link: ASPM Disabled RCB 64 bytes CommClk+ ExtSynch-
> Link: Speed 2.5Gb/s, Width x1
> Capabilities: [100] Advanced Error Reporting
> Capabilities: [140] Device Serial Number xy-xx-xx-ff-ff-48-30-00
>
> 0f:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet
> Controller
> Subsystem: Super Micro Computer Inc Unknown device 109a
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B-
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 106
> Region 0: Memory at d0300000 (32-bit, non-prefetchable) [size=128K]
> Region 2: I/O ports at 3000 [size=32]
> Capabilities: [c8] Power Management version 2
> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> PME(D0+,D1-,D2-,D3hot+,D3cold+)
> Status: D0 PME-Enable- DSel=0 DScale=1 PME-
> Capabilities: [d0] Message Signalled Interrupts: 64bit+
> Queue=0/0 Enable+
> Address: 00000000fee01000 Data: 406a
> Capabilities: [e0] Express Endpoint IRQ 0
> Device: Supported: MaxPayload 256 bytes, PhantFunc 0,
> ExtTag-
> Device: Latency L0s <512ns, L1 <64us
> Device: AtnBtn- AtnInd- PwrInd-
> Device: Errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
> Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> Device: MaxPayload 128 bytes, MaxReadReq 512 bytes
> Link: Supported Speed 2.5Gb/s, Width x1, ASPM unknown,
> Port 0
> Link: Latency L0s <128ns, L1 <64us
> Link: ASPM Disabled RCB 64 bytes CommClk+ ExtSynch-
> Link: Speed 2.5Gb/s, Width x1
> Capabilities: [100] Advanced Error Reporting
> Capabilities: [140] Device Serial Number xx-xx-xx-ff-ff-48-30-00
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFMbjBjfg746kcGBOwRAvM3AKCbA5H7DHI04chYw2e9cY CLeERKxgCgpmWF
qfEwOY3zkMOJDlVpzmVW/rI=
=ZUPo
-----END PGP SIGNATURE-----
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-20-2010, 08:07 AM
JohnS
 
Default Cannot set MTU != 1500 on Intel NIC

On Fri, 2010-08-20 at 09:36 +0200, Timo Schoeler wrote:

>
> I tried with Intel's most recent driver, it shows the same behaviour:
>
> [root@bla ~]# ip link set dev eth0 mtu 1576
> SIOCSIFMTU: Invalid argument
> [root@bla ~]# ip link set dev eth1 mtu 1576
> [root@bla ~]# ip link show dev eth1
> 6: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1576 qdisc pfifo_fast
> qlen 1000
> link/ether 00:30:48:xx:xx:xy brd ff:ff:ff:ff:ff:ff

Try with "ethtool"

> > While the first device (seen as eth0) does *not* allow to set a
> > different MTU (tried with 1576 and 1546, both values I could use; I am
> > forced to use values >1500 here due to protocol stuff), the second works
> > without a problem.

Some e1000 cards do not even have support for jumbo frames. Just maybe
if yours is that case you may not be able to raise the mtu value no
higher than 1500.

John

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-20-2010, 08:17 AM
Timo Schoeler
 
Default Cannot set MTU != 1500 on Intel NIC

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

thus JohnS spake:
> On Fri, 2010-08-20 at 09:36 +0200, Timo Schoeler wrote:
>
>> I tried with Intel's most recent driver, it shows the same behaviour:
>>
>> [root@bla ~]# ip link set dev eth0 mtu 1576
>> SIOCSIFMTU: Invalid argument
>> [root@bla ~]# ip link set dev eth1 mtu 1576
>> [root@bla ~]# ip link show dev eth1
>> 6: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1576 qdisc pfifo_fast
>> qlen 1000
>> link/ether 00:30:48:xx:xx:xy brd ff:ff:ff:ff:ff:ff
>
> Try with "ethtool"

Doesn't work either.

>>> While the first device (seen as eth0) does *not* allow to set a
>>> different MTU (tried with 1576 and 1546, both values I could use; I am
>>> forced to use values >1500 here due to protocol stuff), the second works
>>> without a problem.
>
> Some e1000 cards do not even have support for jumbo frames. Just maybe
> if yours is that case you may not be able to raise the mtu value no
> higher than 1500.

I saw this:

82573(V/L/E) TX Unit Hang Messages
----------------------------------
Several adapters with the 82573 chipset display "TX unit hang" messages
during normal operation with the e1000e driver. The issue appears both
with
TSO enabled and disabled, and is caused by a power management function
that
is enabled in the EEPROM. Early releases of the chipsets to vendors
had the
EEPROM bit that enabled the feature. After the issue was discovered newer
adapters were released with the feature disabled in the EEPROM.

If you encounter the problem in an adapter, and the chipset is an
82573-based
one, you can verify that your adapter needs the fix by using ethtool:

# ethtool -e eth0
Offset Values
------ ------
0x0000 00 12 34 56 fe dc 30 0d 46 f7 f4 00 ff ff ff ff
0x0010 ff ff ff ff 6b 02 8c 10 d9 15 8c 10 86 80 de 83
^^
The value at offset 0x001e (de) has bit 0 unset. This enables the
problematic
power saving feature. In this case, the EEPROM needs to read "df" at
offset
0x001e.

A one-time EEPROM fix is available as a shell script. This script will
verify
that the adapter is applicable to the fix and if the fix is needed or
not. If
the fix is required, it applies the change to the EEPROM and updates the
checksum. The user must reboot the system after applying the fix if
changes
were made to the EEPROM.

(from http://downloadmirror.intel.com/15817/eng/README.txt)

However, the card is okay, I didn't need to fix it.

I also had a look at the driver's source code, what still puzzles me
after reading this is that the other card on board *works*:

(...)

case e1000_82573:
case e1000_82574:
case e1000_82583:
/* Disable ASPM L0s due to hardware errata */
e1000e_disable_aspm(adapter->pdev, PCIE_LINK_STATE_L0S);

if (pdev->device == E1000_DEV_ID_82573L) {
adapter->flags |= FLAG_HAS_JUMBO_FRAMES;
adapter->max_hw_frame_size = DEFAULT_JUMBO;
}
break;
default:
break;
}

return 0;
}

(...)

static struct e1000_info e1000_82573_info = {
.mac = e1000_82573,
.flags = FLAG_HAS_HW_VLAN_FILTER
| FLAG_HAS_WOL
| FLAG_APME_IN_CTRL3
| FLAG_RX_CSUM_ENABLED
| FLAG_HAS_SMART_POWER_DOWN
| FLAG_HAS_AMT
| FLAG_HAS_SWSM_ON_LOAD,
.pba = 20,
.max_hw_frame_size = ETH_FRAME_LEN + ETH_FCS_LEN,
.init_ops = e1000_init_function_pointers_82571,
.get_variants = e1000_get_variants_82571,
};

(...)

/* 82573 Errata 17 */
if (((adapter->hw.mac.type == e1000_82573) ||
(adapter->hw.mac.type == e1000_82574)) &&
(max_frame > ETH_FRAME_LEN + ETH_FCS_LEN)) {
adapter->flags2 |= FLAG2_DISABLE_ASPM_L1;
e1000e_disable_aspm(adapter->pdev, PCIE_LINK_STATE_L1);
}

(...)

> John

Timo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFMbjoXfg746kcGBOwRAtvNAJ9u8VSKuwNBUSWNzrhJbY GywJ5mLACePeEM
PDNR5rHikYMs0r8hWa2qZ4Y=
=dA9v
-----END PGP SIGNATURE-----
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-20-2010, 09:00 AM
JohnS
 
Default Cannot set MTU != 1500 on Intel NIC

On Fri, 2010-08-20 at 10:17 +0200, Timo Schoeler wrote:
> (from http://downloadmirror.intel.com/15817/eng/README.txt)
---
I read that also. Will they work without using the Intel Driver? As in
using the kernel driver in the kernel. Seems to me it should because
the e1000 is a fairly popular card.

Since you can't set the mtu you may benefit from setting the tcp values
in sysctl.conf.

John

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-20-2010, 09:03 AM
Timo Schoeler
 
Default Cannot set MTU != 1500 on Intel NIC

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

thus JohnS spake:
> On Fri, 2010-08-20 at 10:17 +0200, Timo Schoeler wrote:
>> (from http://downloadmirror.intel.com/15817/eng/README.txt)
> ---
> I read that also. Will they work without using the Intel Driver? As in
> using the kernel driver in the kernel. Seems to me it should because
> the e1000 is a fairly popular card.

Sure, first I tried with the CentOS driver.

> Since you can't set the mtu you may benefit from setting the tcp values
> in sysctl.conf.

Nope, unfortunately I'm forced to have that MTU. However, we just
swapped the switch port using the Switch's management so that the NIC
that's able to do >1500 MTU get's the traffic in question.

So, I have a solution, but it doesn't solve the problem (regarding the
NIC itself)...

> John

Timo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFMbkTkfg746kcGBOwRAg2xAJoCHcSTwxx03UsQPOQgLN/MAZpKWACgsP0I
dlZwmctxCIGOY0unQRxBoH8=
=SV2Q
-----END PGP SIGNATURE-----
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-20-2010, 04:10 PM
Gordon Messmer
 
Default Cannot set MTU != 1500 on Intel NIC

On 08/20/2010 12:15 AM, Timo Schoeler wrote:
> 0d:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
> Controller (Copper) (rev 03)

http://download.intel.com/design/network/specupdt/82573.pdf
17 ASPM/Jumbo Frames Disabled Due to Early Receive Threshold Overrun
Buffer
Status: Intel does not plan to resolve this erratum in the 82573
Gigabit Ethernet Controller. Jumbo frames is not supported in
82573E/V & is supported with the workaround above in 82573L.

You'll have to use different hardware. The chipset you've got has a
bug, and jumbo frame support is disabled as a result.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-23-2010, 04:22 PM
Timo Schoeler
 
Default Cannot set MTU != 1500 on Intel NIC

>> 0d:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
>> Controller (Copper) (rev 03)
>
> http://download.intel.com/design/network/specupdt/82573.pdf
> 17 ASPM/Jumbo Frames Disabled Due to Early Receive Threshold Overrun
> Buffer
> Status: Intel does not plan to resolve this erratum in the 82573
> Gigabit Ethernet Controller. Jumbo frames is not supported in
> 82573E/V& is supported with the workaround above in 82573L.

Interestingly, I somewhere got the same PDF (from the intel site) that
required a password -- and didn't investigate further on your link. Now,
out of curiosity, I did...

> You'll have to use different hardware. The chipset you've got has a
> bug, and jumbo frame support is disabled as a result.

...and yes, you're right.

I'm amused about PeeCee hardware (sorry, only half of a pun intended)...
There's those two NICs on board of a *server* grade machine, a 82573E
and a 82573L. One of them is just *broken* (see above).

Cheers,

Timo
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-23-2010, 06:35 PM
Gordon Messmer
 
Default Cannot set MTU != 1500 on Intel NIC

On 08/23/2010 09:22 AM, Timo Schoeler wrote:
> I'm amused about PeeCee hardware (sorry, only half of a pun intended)...
> There's those two NICs on board of a *server* grade machine, a 82573E
> and a 82573L. One of them is just *broken* (see above).

Actually, both of them are broken. One of them has a workaround
available for its brokenness.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-24-2010, 09:59 AM
Timo Schoeler
 
Default Cannot set MTU != 1500 on Intel NIC

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

thus Gordon Messmer spake:
> On 08/23/2010 09:22 AM, Timo Schoeler wrote:
>> I'm amused about PeeCee hardware (sorry, only half of a pun intended)...
>> There's those two NICs on board of a *server* grade machine, a 82573E
>> and a 82573L. One of them is just *broken* (see above).

Hi,

> Actually, both of them are broken. One of them has a workaround
> available for its brokenness.

yep, I read the driver's source (and hey, they do comment their code! ...

For me, it's dead silicon. I know, even CPUs may have hundreds of
errata, but shipping and selling crappy NIC chipsets is something that
collides with my universe.

YMMV, tho.

Timo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFMc5gYfg746kcGBOwRAkFxAJ0djqrQnNPAFOq2cbnTNw vqsEnQNQCggklF
gl0jqwNXvmybbAS7Qdf4tTs=
=KiQQ
-----END PGP SIGNATURE-----
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 06:33 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org