linux network tune

Don't try this yourself, you might break something. These recommendations were from Pete Vogel, who runs a bigger site than you do (can you do 85Mb/s on just one server? .. and he has many!!). Don't change these params unless you understand them. I am working on adding explainations. Amongst other things (like breaking your network connectivity), you might actually make your machine slower if you just blindly use them. In fact, I only set a couple of them. And always best to set one at a time to see if you notice the improvement you are looking for.

echo 65534 > /proc/sys/fs/file-max
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 0 > /proc/sys/net/ipv4/tcp_window_scaling
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 0 > /proc/sys/net/ipv4/tcp_sack
echo 0 > /proc/sys/net/ipv4/tcp_ecn
echo 2 > /proc/sys/net/ipv4/tcp_orphan_retries
echo 1 > /proc/sys/net/ipv4/tcp_syncookies

• /proc/sys/fs/file-max: This is basically the number of file descriptors available in the kernel. Which also effects the number of fd's a process can have open. For large sites you will definately need to upgrade this, and for some os'es you will need to use ulimit to increase the number of fd's available for that process. For red-hat (and other linux's) you can use:
  

  echo 32767 > /proc/sys/fs/file-max

  which I put into /etc/rc.d/rc.local (at/near the end). You can check your current usage using:

  [cameron@jack cameron]$ cat /proc/sys/fs/file-nr
  5941	2844	32767
  [cameron@jack cameron]$ 

  and what this means is that since the kernel has been running the max number of fd's in use was 5941, the current number is 2844, and the max available is 32767.
• /proc/sys/net/ipv4/tcp_tw_recycle:

  BOOLEAN
  	Enable fast recycling TIME-WAIT sockets. Default value is 1.
  	It should not be changed without advice/request of technical
  	experts.

  The definition of TIME_WAIT in "man netstat" is

         TIME_WAIT
                The socket is waiting after close to handle packets
                still in the network.

  On Red Hat, the default is 0, so I checked on one of the loaded machines to see 2500 sockets in the TIME_WAIT state

  [cameron@jack cameron]$ netstat -n | grep TIME_WAIT | wc
     2446   14676  198126
  [cameron@jack cameron]$ netstat -n | wc
     2847   17104  230009

  As you can see, for me, this takes up a lot of the network resources. By turning this on, I saw a slight drop in the number of connections (about 200), and a definate drop in the number of TIME_WAIT connections. I figured that this was good. I see that machine handling slightly more traffic, but my measuring mechanism is "poor". This is the next day after I made the change (at a busier time of the day).

  [cameron@jack httpd-2.0.35]$ netstat -n | grep TIME_WAIT | wc ; netstat -n | wc
     2291   13746  185571
     2905   17452  234696
  [cameron@jack httpd-2.0.35]$ 

  
  So by turning it on, you should see a reduction in the number of FD's used in the kernel, but you might get some extra junk traffic??
• /proc/sys/net/ipv4/tcp_window_scaling:

  BOOLEN
  	Enable window scaling as defined in RFC1323

  These are recommendations to make TCP/IP work over very high speeds. I left the default at 1.
• /proc/sys/net/ipv4/tcp_timestamps: Todo

  BOOLEAN
  	Enable timestamps as defined in RFC1323

  an except:

        On the other hand, a Timestamps option may appear in any data or
        ACK segment, adding 12 bytes to the 20-byte TCP header.  We
        believe that the bandwidth saved by reducing unnecessary
        retransmissions will more than pay for the extra header bandwidth.

• /proc/sys/net/ipv4/tcp_sack:

  BOOLEAN
  	Enable select acknowledgments (SACKS).

• /proc/sys/net/ipv4/tcp_ecn:

  BOOLEAN
  	Enable Explicit Congestion Notification in TCP.

• /proc/sys/net/ipv4/tcp_orphan_retries:

  INTEGER
  	How may times to retry before killing TCP connection, closed
  	by our side. Default value 7 corresponds to ~50sec-16min
  	depending on RTO. If you machine is loaded WEB server,
  	you should think about lowering this value, such sockets
  	may consume significant resources. Cf. tcp_max_orphans.

• /proc/sys/net/ipv4/tcp_syncookies:

  BOOLEAN
  	Only valid when the kernel was compiled with CONFIG_SYNCOOKIES
  	Send out syncookies when the syn backlog queue of a socket 
  	overflows. This is to prevent against the common 'syn flood attack'
  	Default: FALSE
  
  	Note, that syncookies is fallback facility.
  	It MUST NOT be used to help highly loaded servers to stand
  	against legal connection rate. If you see synflood warnings
  	in your logs, but investigation	shows that they occur
  	because of overload with legal connections, you should tune
  	another parameters until this warning disappear.
  	See: tcp_max_syn_backlog, tcp_synack_retries, tcp_abort_on_overflow.
  
  	syncookies seriously violate TCP protocol, do not allow
  	to use TCP extensions, can result in serious degradation
  	of some services (f.e. SMTP relaying), visible not by you,
  	but your clients and relays, contacting you. While you see
  	synflood warnings in logs not being really flooded, your server
  	is seriously misconfigured.

MTU on DSL lines
It is said that DSL wraps packets in it's own packet. So for some DSL connections decreasing your MTU (Maximum Transfer Unit), from 1500 to 1490 is advantagous. You can do this using:

 ifconfig eth0 mtu 1490

Which I then added to /etc/rc.d/rc.local (for red hat 7.2) Links:

• http://hr.uoregon.edu/davidrl/Documentation/networking/ip-sysctl.txt - local copy

Last Change: Sunday, 06-Dec-2009 20:07:29 EST
http://www.bloke.com/linux/kernel/network.html