Neterion’s (Formerly S2io) Xframe I/II PCI-X 10GbE driver¶
Release notes for Neterion’s (Formerly S2io) Xframe I/II PCI-X 10GbE driver.
1. Introduction¶
This Linux driver supports Neterion’s Xframe I PCI-X 1.0 and Xframe II PCI-X 2.0 adapters. It supports several features such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. See below for complete list of features.
All features are supported for both IPv4 and IPv6.
2. Identifying the adapter/interface¶
- Insert the adapter(s) in your system. 
- Build and load driver: - # insmod s2io.ko 
- View log messages: - # dmesg | tail -40 
You will see messages similar to:
eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
eth4: Device is on 64 bit 133MHz PCIX(M1) bus
The above messages identify the adapter type(Xframe I/II), adapter revision, driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed as well.
To associate an interface with a physical adapter use “ethtool -p <ethX>”. The corresponding adapter’s LED will blink multiple times.
3. Features supported¶
- Jumbo frames. Xframe I/II supports MTU up to 9600 bytes, modifiable using ip command. 
- Offloads. Supports checksum offload(TCP/UDP/IP) on transmit and receive, TSO. 
- Multi-buffer receive mode. Scattering of packet across multiple buffers. Currently driver supports 2-buffer mode which yields significant performance improvement on certain platforms(SGI Altix, IBM xSeries). 
- MSI/MSI-X. Can be enabled on platforms which support this feature resulting in noticeable performance improvement (up to 7% on certain platforms). 
- Statistics. Comprehensive MAC-level and software statistics displayed using “ethtool -S” option. 
- Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, with multiple steering options. 
4. Command line parameters¶
- tx_fifo_num
- Number of transmit queues 
 
Valid range: 1-8
Default: 1
- rx_ring_num
- Number of receive rings 
 
Valid range: 1-8
Default: 1
- tx_fifo_len
- Size of each transmit queue 
 
Valid range: Total length of all queues should not exceed 8192
Default: 4096
- rx_ring_sz
- Size of each receive ring(in 4K blocks) 
 
Valid range: Limited by memory on system
Default: 30
- intr_type
- Specifies interrupt type. Possible values 0(INTA), 2(MSI-X) 
 
Valid values: 0, 2
Default: 2
5. Performance suggestions¶
General:
- Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) 
- Set TCP windows size to optimal value. 
For instance, for MTU=1500 a value of 210K has been observed to result in good performance:
# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
For MTU=9000, TCP window size of 10 MB is recommended:
# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
Transmit performance:
- By default, the driver respects BIOS settings for PCI bus parameters. However, you may want to experiment with PCI bus parameters max-split-transactions(MOST) and MMRBC (use setpci command). - A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. - It could be different for your hardware. - Set MMRBC to 4K**. - For example you can set - For opteron: - #setpci -d 17d5:* 62=1d - For Itanium: - #setpci -d 17d5:* 62=3d - For detailed description of the PCI registers, please see Xframe User Guide. 
- Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this parameter. 
- Turn on TSO(using “ethtool -K”): - # ethtool -K <ethX> tso on 
Receive performance:
- By default, the driver respects BIOS settings for PCI bus parameters. However, you may want to set PCI latency timer to 248: - #setpci -d 17d5:* LATENCY_TIMER=f8 - For detailed description of the PCI registers, please see Xframe User Guide. 
- Use 2-buffer mode. This results in large performance boost on certain platforms(eg. SGI Altix, IBM xSeries). 
- Ensure Receive Checksum offload is enabled. Use “ethtool -K ethX” command to set/verify this option. 
- Enable NAPI feature(in kernel configuration Device Drivers ---> Network device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to bring down CPU utilization. 
Note
For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are recommended as safe parameters.
For more information, please review the AMD8131 errata at http://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/ 26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf
6. Support¶
For further support please contact either your 10GbE Xframe NIC vendor (IBM, HP, SGI etc.)