Linux Tuning

This page contains a quick reference guide for Linux 2.6 tuning for hosts connected at speeds of 1Gbps or higher. For a detailed explanation of some of the advice on this page, see the Linux Tuning Expert page. Also see the tuning advice for test/measurement hosts page.

 General Approach

To check what setting your system is using, use 'sysctl name' (e.g.: 'sysctl net.ipv4.tcp_rmem'). To change a setting use 'sysctl -w'. To make the setting permanent add the setting to the file 'sysctl.conf'.

 TCP tuning

Like most modern OSes, Linux now does a good job of auto-tuning the TCP buffers, but the  default maximum Linux TCP buffer sizes are too small. The following settings are recommended:

# increase TCP max buffer size setable using setsockopt()
# 16 MB with a few parallel streams is recommended for most 10G paths
# 32 MB might be needed for some very long end-to-end 10G or 40G paths
net.core.rmem_max = 16777216 
net.core.wmem_max = 16777216 
# increase Linux autotuning TCP buffer limits 
# min, default, and max number of bytes to use
# (only change the 3rd value, and make it 16 MB or more)
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# recommended to increase this for 10G NICS
net.core.netdev_max_backlog = 30000

Note: you should leave net.tcp_mem alone. The defaults are fine.
Linux supports pluggable congestion control algorithms . To get a list of congestion control algorithms that are available in your kernel (kernal  2.6.20+), run:

sysctl net.ipv4.tcp_available_congestion_control

If cubic and/or htcp are not listed try the following, as most distributions include them as loadable kernel modules:

/sbin/modprobe tcp_htcp
/sbin/modprobe tcp_cubic

For long fast paths, we highly recommend using cubic or htcp. Cubic is the default for a number of Linux distributions, but if is not the default on your system, you can do the following:

 sysctl -w net.ipv4.tcp_congestion_control=cubic

NOTE: There seem to be bugs in both bic and cubic for a number of versions of the 2.6.18 kernel used by Redhat Enterprise Linux 5.3 - 5.5 and its variants (Centos, Scientific Linux, etc.) We recommend using htcp with a 2.6.18.x kernel to be safe.

 NIC Tuning

These can be added to /etc/rc.local to get run at boot time.
# increase txqueuelen for 10G NICS
/sbin/ifconfig eth2 txqueuelen 10000

http://fasterdata.es.net/fasterdata/host-tuning/linux/#t3