The "three-way handshake" is the procedure used to establish a connection. This procedure normally is initiated by one TCP and responded to by another TCP. The procedure also works if two TCP simultaneously initiate the procedure. When simultaneous attempt occurs, each TCP receives a "SYN" segment which carries no acknowledgment after it has sent a "SYN". Of course, the arrival of an old duplicate "SYN" segment can potentially make it appear, to the recipient, that a simultaneous connection initiation is in progress. Proper use of "reset" segments can disambiguate these cases.
Several examples of connection initiation follow. Although these examples do not show connection synchronization using data-carrying segments, this is perfectly legitimate, so long as the receiving TCP doesn't deliver the data to the user until it is clear the data is valid (i.e., the data must be buffered at the receiver until the connection reaches the ESTABLISHED state). The three-way handshake reduces the possibility of false connections. It is the implementation of a trade-off between memory and messages to provide information for this checking.
The simplest three-way handshake is shown in figure 7 below. The figures should be interpreted in the following way. Each line is numbered for reference purposes. Right arrows (-->) indicate departure of a TCP segment from TCP A to TCP B, or arrival of a segment at B from A. Left arrows (<--), indicate the reverse. Ellipsis (...) indicates a segment which is still in the network (delayed). An "XXX" indicates a segment which is lost or rejected. Comments appear in parentheses. TCP states represent the state AFTER the departure or arrival of the segment (whose contents are shown in the center of each line). Segment contents are shown in abbreviated form, with sequence number, control flags, and ACK field. Other fields such as window, addresses, lengths, and text have been left out in the interest of clarity.
Showing posts with label tcp. Show all posts
Showing posts with label tcp. Show all posts
Friday, September 2, 2011
Transmission Control Protocol (TCP)
Transmission Control Protocol is a Transport Layer host-to-host protocol that provides reliable, connection-oriented communication over IP networks.
- Transmission Control Protocol Characteristics
- Transmission Control Protocol Operation
- TCP Segmentation
- TCP Connection Establishment
- Synchronization
- 3-Way Handshake
- TCP Reliable Transmission
- TCP Connection Teardown
- TCP Reassembly
- Transmission Control Protocol Packet Structure
- TCP Header
- Payload Carries Upper Layer Protocols
Tuesday, August 2, 2011
Finding the optimal TCPIP receive window size
What problem does the receive window solve?
TCPIP is a reliable network protocol where each packet sent is acknowledged by receiver, and if the sender doesn't get the acknowledge packet back within a certain timeout, then it retransmits the original packet. But it is inefficient to wait for the acknowledge packet from the receiver before sending the next packet. The Receive WINdow(RWIN) solves the problem of the sender constantly waiting for the receiver acknowledge.
Using router metric to specify primary gateway
If you are able to connect to several gateways/routers to other networks, it is possible to prioritize in which order the gateways should be searched for a certain address.
This is done by the Metric setting which is assigned to each gateway. The gateway with lowest Metric is the first to be searched(Default Gateway). Metric is the cost of using that gateway, and it will always use the one with the lowest cost first.
This is done by the Metric setting which is assigned to each gateway. The gateway with lowest Metric is the first to be searched(Default Gateway). Metric is the cost of using that gateway, and it will always use the one with the lowest cost first.
Configure TCPIP settings for better performance
Description of what TCP/IP settings to configure for better performance and quality connection. These settings are configured through the Windows registry, but there also exist utilities to easily configure these settings:
- DSLReport's DrTCP - Utility to configure the most common settings (Recommended).
- SpeedGuide's TCP Optimizer - Utility to configure almost everything and includes a connection speed tester to give recommendations
- If wanting to get your hands dirty, then here are details of where to go in the registry:
Saturday, June 11, 2011
Network Tool
Httcp
Flowgrind: TCP Performance Measurement Tool
Flowgrind is a tool similar to iperf, netperf to measure throughput and other metrics for TCP.
Features:
distributed architecture:
Flowgrind is split into two components: the flowgrind daemon and the flowgrind controller. Using the controller, flows between any two systems running the flowgrind daemon can be setup (third party tests). At regular intervals during the test the controller collects and displays the measured results from the daemons. It can run multiple flows at once with the same or different settings and individually schedule every one. Test and control connection can optionally be diverted to different interfaces.
Advanced metrics
besides goodput it measures application layer IAT and 2-way (RTT) delay, blockcount and Networktransactions/s
Stochastic Traffic Generation
bulk transfers, rate-limited flows, sophisticated request/response tests
Automatic Dump support
can use libpcap to automatically dump traffic for qualitative analysis
Special Linux TCP support
shows tcpi_info stats (RTT, RTO, CWND, SSTRESH, congestion control state etc), can set TCP options like TCP_NODELAY, set buffer size on per flow basis, set congestion control algorithm, DSCP field etc
Singlethreaded flow handling
handles all flow inside one thread to improve fairness between flows
Gnuplot compatible configureable output format
bwctl
NAME
bwctl - Client application to request throughput tests.
SYNOPSIS
bwctl [options] -c recvhost -s sendhost
bwctl [options] -c recvhost
bwctl [options] -s sendhost
DESCRIPTION
bwctl is a command line client application that is used to initiate throughput tests.
This version of bwctl is capable of initiating Iperf, Nuttcp, and Thrulay tests.
bwctl works by contacting a bwctld daemon on both the receiving host and the sending host. bwctld manages and schedules the resources of the host it runs on. In the case where only one of the receiving host or sending host is specified, bwctl assumes the local host is the other endpoint. bwctl will attempt to contact a local bwctld if it can. If there is no local bwctld running, bwctl assumes the local host does not require policy controls and will execute the bwctld functionality required to run the test directly.
If cases where bwctl is directly running the throughput test on the host, there are several configuration options that are shared with bwctld. Those configuration options can be set using the bwctlrc(5) configuration file in a way very similar to the way they are specified in the bwctld.conf(5) file.
The bwctl client is used to request the desired type of throughput test. Furthermore, it requests when the test is wanted. bwctld on each endpoint either responds with a tentative reservation or a test denied message. Once bwctl is able to get a matching reservation from both bwctld processes (one for each host involved in the test), it confirms the reservation. Then, the bwctld processes run the test and return the results. The results are returned to the client from both sides of the test from the respective bwctld processes. Additionally, the bwctld processes share the results from their respective side of the test with each other.
BWCTL (bwctl and bwctld) is used to enable non-specific throughput tests to hosts without having to give full user accounts on the given systems. Users want the ability to run throughput tests to determine the achievable or available bandwidth between a pair of hosts. It is often useful to test to multiple points along a network path to determine the network characteristics along that path. Typically, users who want to do this path decomposition have to directly contact the network/system administrators who control the hosts along the path. The administrator needs to either run half of the test for the user or give them a user account on the host. Also, network paths of interest are typically controlled by multiple administrators. These hurdles have made this kind of testing difficult in practice.
BWCTL was designed to help with this problem. It allows an administrator to configure a given host as an Iperf, Thrulay, or Nuttcp endpoint. The endpoint can be a packet sender (e.g. Iperf client) or a packet receiver (e.g. Iperf server). It can be shared by multiple users without concern that those users will interfere with each other. Specific policy limits can be applied to specific users, and individual tests are scheduled so they will not interfere with each other. Additionally, full user accounts are not required for the users running the tests.
BWCTL allows the administrator to classify incoming connections based upon a user name and AES key combination or, alternatively, based upon an IP/netmask. Once the connection is classified, the bwctld can determine the exact type and intensities of througput tests that will be allowed. More information on the policy controls can be found in the bwctld(8) man page.
BWCTL makes use of a distributed scheduling algorithm. Each host maintains a schedule independently. As a client requests a test, the two endpoints are contacted and each bwctld server responds with the first available open schedule slot. This enables on-demand tests to co-exist with regularly scheduled tests since regularly scheduled tests are implemented by having the client request tests on regular intervals. Different priorities can be implemented using the event_horizon configuration directive to bwctld. (By allowing clients that implement regularly scheduled tests to reserve their time slots further into the future.)
ARGUMENTS
Connection/Authentication Arguments:
-A authmethod
authmethod is used to specify the authentication method the bwctl client is willing to use for communication with the bwctld on the sendhost and recvhost. The authentication options of bwctl are intended to be extensible. The communication from the bwctl client to each bwctld server may take different options for different types of authentication. If the authmethod option is specified for either the -s, or the -c argument, it overrides the authmethod specified with the -A option for communication with that particular host. (Therefore, the -A argument is really only useful if the same authentication can be used with both hosts.)
Allowing different authentication methods for each connection should allow a client to use different authentication methods with different servers which should in turn allow cross-domain tests to occur more easily.
The format for authmethod is:
authmode [authscheme schemeopts]
authmode
Specifies the authentication mode the client is willing to speak with a server. It must be set as a character string with any or all of the characters "AEO". The modes are:
A
[A]uthenticated. This mode encrypts the control connection.
E
[E]ncrypted. This mode encrypts the control connection. If the test supports encryption, this mode will additionally encrypt the test stream. (Encryption of the test stream is not currently supported, so this mode is currently identical to authenticated.)
O
[O]pen. No encryption of any kind is done.
The client can specify all the modes with which it is willing to communicate. The most strict mode that both the server and the client are willing to use will be selected.
Default:
"AEO"
authscheme schemeopts
authscheme indicates the authentication scheme that should be used to achieve the authenticated or encrypted modes. schemeopts are a list of arguments specific to each particular authentication scheme. Supported authscheme values follow (listed with the schemeopts each scheme requires):
AESKEY userid [keyfile]
This is the initial "simple" shared secret (AES key) model. userid is required to identify which shared secret the server and client should use. keyfile optionally specifies a file to retrieve the AES key from. If keyfile is not specified, the user will be prompted for a passphrase. keyfile can be generated using the aespasswd(1) application.
Default:
Unauthenticated
authscheme and schemeopts are only needed if authenticated communication (A or E modes of authmode) is wanted with sendhost and recvhost.
-B srcaddr
Bind the local address of the client socket to srcaddr. srcaddr can be specified using a DNS name or using standard textual notations for the IP addresses.
Default:
Unspecified (wild-card address selection).
-c recvhost[:port] [authmethod]
Specifies the host that will run the Iperf, Thrulay or Nuttcp server. The :port suffix is optional and is only needed if bwctld is being run on a non-default port number. If an IPv6 address is being specified, note that the accepted format contains the recvhost portion of the specification in square brackets as: [fe80::fe9f:62d8]:4823. This ensures the port number is distinct from the address specification, and is not needed if the :port suffix is not being used.
At least one of the -c or -s options must be specified. If one of them is not specified, it is assumed to be the local host.
authmethod is a specifically ordered list of keywords that is only needed if authenticated communication is wanted with recvhost. These keywords are used to describe the type of communication and authentication that should be used to contact the recvhost. If recvhost and sendhost share the same authentication methods and identities, it is possible to specify the authmethod for both recvhost and sendhost using the -A argument. An authmethod specified with the -c option will override an authmethod specified with the -A argument for communication with the recvhost.
The format for authmethod and a description of the currently available authentication methods are described with the -A argument.
-k
This option has been deprecated. Originally, it was used to specify the keyfile for authentication. All authentication options can now be specified using the -A argument. For the next several versions this option will report an error. Eventually, it may be reclaimed for another purpose.
-s sendhost[:port] [authmethod]
Specifies the host that will run the Iperf, Thrulay or Nuttcp client. The :port suffix is optional and is only needed if bwctld is being run on a non-default port number. If an IPv6 address is being specified, note that the accepted format contains the sendhost portion of the specification in square brackets as: [fe80::fe9f:62d8]:4823. This ensures the port number is distinct from the address specification, and is not needed if the :port suffix is not being used.
At least one of the -c or -s options must be specified. If one of them is not specified, it is assumed to be the local.
authmethod is a specifically ordered list of keywords that is only needed if authenticated communication is wanted with sendhost. These keywords are used to describe the type of communication and authentication that should be used to contact the sendhost. If recvhost and sendhost share the same authentication methods and identities, it is possible to specify the authmethod for both recvhost and sendhost using the -A argument. An authmethod specified with the -s option will override an authmethod specified with the -A argument for communication with the sendhost.
The format for authmethod and a description of the currently available authentication methods are described with the -A argument.
-U
This option has been deprecated. Originally, it was used to specify the username to identify the AES key for authentication. All authentication options can now be specified using the -A argument. For the next several versions this option will report an error. Eventually, it may be reclaimed for another purpose.
Throughput Test Arguments:
The arguments were named to match their counterparts in Iperf as closely as possible.
Some of the options are not available for some of the throughput testers. BWCTL does not support UDP tests, changing the output format or changing the output units for either Nuttcp or Thrulay.
-T
Specify which throughput tester to use:
iperf
thrulay
nuttcp
Default:
None. Selects a tool that the client and server have in common
-b bandwidth
Limit UDP send rate to bandwidth (bits/sec).
Default:
1 Mb
-i interval
Report interval (seconds).
Default:
unset (no intervals reported)
-l len
length of read/write buffers (bytes).
Default:
8 KB TCP, 1470 bytes UDP
-P nStreams
Number of concurrent streams for the test. See the -P option of Iperf for details.
-S TOS
Set the TOS (See RFC 1349) byte in packets.
Default:
0 (not set)
-t time
Duration of test (seconds).
Default:
10
-u
UDP test.
Default:
TCP test
-W window
Same as the -w option, except that the value is advisory. bwctl will attempt to dynamically determine the appropriate TCP window, based upon RTT information gathered from the control socket. If bwctl is unable to dynamically determine a window, the value window will be used.
Default:
Unset (system defaults)
-w window
Socket buffer sizes (bytes). For TCP, this sets the TCP window size. For UDP, this sets the socket receive buffer size.
Default:
Unset (system defaults)
Scheduling Arguments:
-a syncfuzz
Allow bwctl to run without a synchronized system clock. Use this to specify how far off the local clock is from UTC. bwctl prefers to have an NTP synchronized system clock to ensure the two endpoints of the test are actually agreeing to the same scheduled time window for test execution.
If two systems do NOT have a close enough notion of time, then the throughput test will eventually fail because one endpoint of the test will attempt to run at a different time than the other.
If the operating system supports the NTP system calls, and the system clock is determined to be unsynchronized, error messages will still be reported depending upon the value of the -e flag.
When calculating the time errors, this value will be aded in to account for the difference. The maximum time offset can be bounded on the server side, using the max_time_error directive, to prevent a denial of service attack. If set, the server will reject any requests to test with a peer that has too high a timestamp error.
Default:
Unset (Defaults to Set for systems without the NTP system calls)
-I interval
Specifies that bwctl should attempt to run a throughput test every interval seconds.
Default:
Unset. If it is unset, bwctl only runs the test once.
-L longest
Specifies the longest amount of time the client is willing to wait for a reservation window. When bwctl requests a test from the bwctld server, it specifies the earliest time and the latest time it is willing to accept. The latest time is determined by adding this longest option to the earliest time. The earliest time is essentially 'now'. The longest time is specified as a number of seconds.
Default:
If interval is set, the default is 50% of interval. Otherwise, the default is twice the test duration time but no smaller than 5 minutes. (See -t.)
-n nIntervals
Number of tests to perform if the -I option is set.
Default:
Continuous
-R alpha
Randomize the start time of the test within this alpha percent of the interval. Valid values for alpha are from 0-50. bwctl will attempt to run the test every interval +/- alpha percent. For example, if the interval is 300 seconds and alpha is set to 10 percent, then bwctl will attempt to run a test every 270-330 seconds. This option is only useful with the -I option.
Default:
0 (no randomness)
Output Arguments:
-d dir
Specifies directory for results files if the -p option is set.
-e facility
Syslog facility to log messages to.
Default:
LOG_USER
-f units
Specify the units for the tool to use when displaying the results. The accepted values for units are tool specific.
Iperf:
k
Kilobits per second
K
Kilobytes per second
m
Megabits per second
M
Megabytes per second
-h
Print a help message.
-p
Place test results in files. Print the filenames to stdout when results are complete.
-q
Quiet output. Output as little as possible.
-r
Send syslog messages to stderr. This is the default unless the -q option is specified so this option is only useful with the -q option.
-V
Print version information and exit.
-v
Verbose output. Specifying additional -v's increases the verbosity.
-x
Output sender (client) results as well as receiver results. By default, sender results are not output. If the -p option is specified, the sender results are placed in an additional file.
-y format
Specify the output format of the tool. The accepted values for format are tool specific.
Iperf:
c
[c]omma-separated output
ENVIRONMENT VARIABLES
bwctl Environment Variable use default
BWCTLRC Config file ~/.bwctlrc
BWCTL_DEBUG_TIMEOFFSET Offset 0.0(seconds)
LIMITATIONS
Only tested with versions 1.7.0 and 2.0.b of Iperf.
EXAMPLES
bwctl -c somehost.example.com
Run a default 10 second TCP test as soon as possible with local as the sender and somehost.example.com as the receiver, using whichever tools they have in common. Return the results from the receive side of the test.
bwctl -x -c somehost.example.com
Like the previous test, but also return the results from the sender side of the test.
bwctl -x -c somehost.example.com -s otherhost.example.com
Like the previous test, but with otherhost.example.com as the sender instead of local.
bwctl -t 30 -T iperf -s somehost.example.com
Run a 30 second TCP Iperf test with somehost.example.com as the sender and local as the receiver.
bwctl -I 3600 -R 10 -t 10 -u -b 10m -s somehost.example.com
Run a 10 second UDP test about every hour (3600 +/- 360 seconds) with the sender rate limited to 10 Mbits per second from somehost.example.com to local.
bwctl -s somehost.example.com AE AESKEY someuser
Run the default 10 second TCP test. Authenticate using the identity someuser. bwctl will prompt for a passphrase that will be used to create an AES key.
SEE ALSO
bwctld(8) and the http://e2epi.internet2.edu/bwctl/ web site.
For details on Iperf, see the http://sourceforge.net/projects/iperf web site.
For details on Nuttcp, see the http://www.wcisd.hpc.mil/nuttcp/Nuttcp-HOWTO.html web site.
For details on Thrulay, see the http://e2epi.internet2.edu/thrulay/ web site.
ACKNOWLEDGMENTS
This material is based in part on work supported by the National Science Foundation (NSF) under Grant No. ANI-0314723. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.
Index
NAME
SYNOPSIS
DESCRIPTION
ARGUMENTS
Connection/Authentication Arguments:
Throughput Test Arguments:
Scheduling Arguments:
Output Arguments:
ENVIRONMENT VARIABLES
LIMITATIONS
EXAMPLES
SEE ALSO
ACKNOWLEDGMENTS
https://launchpad.net/flowgrind
http://www.internet2.edu/performance/bwctl/bwctl.man.html
Flowgrind: TCP Performance Measurement Tool
Flowgrind is a tool similar to iperf, netperf to measure throughput and other metrics for TCP.
Features:
distributed architecture:
Flowgrind is split into two components: the flowgrind daemon and the flowgrind controller. Using the controller, flows between any two systems running the flowgrind daemon can be setup (third party tests). At regular intervals during the test the controller collects and displays the measured results from the daemons. It can run multiple flows at once with the same or different settings and individually schedule every one. Test and control connection can optionally be diverted to different interfaces.
Advanced metrics
besides goodput it measures application layer IAT and 2-way (RTT) delay, blockcount and Networktransactions/s
Stochastic Traffic Generation
bulk transfers, rate-limited flows, sophisticated request/response tests
Automatic Dump support
can use libpcap to automatically dump traffic for qualitative analysis
Special Linux TCP support
shows tcpi_info stats (RTT, RTO, CWND, SSTRESH, congestion control state etc), can set TCP options like TCP_NODELAY, set buffer size on per flow basis, set congestion control algorithm, DSCP field etc
Singlethreaded flow handling
handles all flow inside one thread to improve fairness between flows
Gnuplot compatible configureable output format
bwctl
NAME
bwctl - Client application to request throughput tests.
SYNOPSIS
bwctl [options] -c recvhost -s sendhost
bwctl [options] -c recvhost
bwctl [options] -s sendhost
DESCRIPTION
bwctl is a command line client application that is used to initiate throughput tests.
This version of bwctl is capable of initiating Iperf, Nuttcp, and Thrulay tests.
bwctl works by contacting a bwctld daemon on both the receiving host and the sending host. bwctld manages and schedules the resources of the host it runs on. In the case where only one of the receiving host or sending host is specified, bwctl assumes the local host is the other endpoint. bwctl will attempt to contact a local bwctld if it can. If there is no local bwctld running, bwctl assumes the local host does not require policy controls and will execute the bwctld functionality required to run the test directly.
If cases where bwctl is directly running the throughput test on the host, there are several configuration options that are shared with bwctld. Those configuration options can be set using the bwctlrc(5) configuration file in a way very similar to the way they are specified in the bwctld.conf(5) file.
The bwctl client is used to request the desired type of throughput test. Furthermore, it requests when the test is wanted. bwctld on each endpoint either responds with a tentative reservation or a test denied message. Once bwctl is able to get a matching reservation from both bwctld processes (one for each host involved in the test), it confirms the reservation. Then, the bwctld processes run the test and return the results. The results are returned to the client from both sides of the test from the respective bwctld processes. Additionally, the bwctld processes share the results from their respective side of the test with each other.
BWCTL (bwctl and bwctld) is used to enable non-specific throughput tests to hosts without having to give full user accounts on the given systems. Users want the ability to run throughput tests to determine the achievable or available bandwidth between a pair of hosts. It is often useful to test to multiple points along a network path to determine the network characteristics along that path. Typically, users who want to do this path decomposition have to directly contact the network/system administrators who control the hosts along the path. The administrator needs to either run half of the test for the user or give them a user account on the host. Also, network paths of interest are typically controlled by multiple administrators. These hurdles have made this kind of testing difficult in practice.
BWCTL was designed to help with this problem. It allows an administrator to configure a given host as an Iperf, Thrulay, or Nuttcp endpoint. The endpoint can be a packet sender (e.g. Iperf client) or a packet receiver (e.g. Iperf server). It can be shared by multiple users without concern that those users will interfere with each other. Specific policy limits can be applied to specific users, and individual tests are scheduled so they will not interfere with each other. Additionally, full user accounts are not required for the users running the tests.
BWCTL allows the administrator to classify incoming connections based upon a user name and AES key combination or, alternatively, based upon an IP/netmask. Once the connection is classified, the bwctld can determine the exact type and intensities of througput tests that will be allowed. More information on the policy controls can be found in the bwctld(8) man page.
BWCTL makes use of a distributed scheduling algorithm. Each host maintains a schedule independently. As a client requests a test, the two endpoints are contacted and each bwctld server responds with the first available open schedule slot. This enables on-demand tests to co-exist with regularly scheduled tests since regularly scheduled tests are implemented by having the client request tests on regular intervals. Different priorities can be implemented using the event_horizon configuration directive to bwctld. (By allowing clients that implement regularly scheduled tests to reserve their time slots further into the future.)
ARGUMENTS
Connection/Authentication Arguments:
-A authmethod
authmethod is used to specify the authentication method the bwctl client is willing to use for communication with the bwctld on the sendhost and recvhost. The authentication options of bwctl are intended to be extensible. The communication from the bwctl client to each bwctld server may take different options for different types of authentication. If the authmethod option is specified for either the -s, or the -c argument, it overrides the authmethod specified with the -A option for communication with that particular host. (Therefore, the -A argument is really only useful if the same authentication can be used with both hosts.)
Allowing different authentication methods for each connection should allow a client to use different authentication methods with different servers which should in turn allow cross-domain tests to occur more easily.
The format for authmethod is:
authmode [authscheme schemeopts]
authmode
Specifies the authentication mode the client is willing to speak with a server. It must be set as a character string with any or all of the characters "AEO". The modes are:
A
[A]uthenticated. This mode encrypts the control connection.
E
[E]ncrypted. This mode encrypts the control connection. If the test supports encryption, this mode will additionally encrypt the test stream. (Encryption of the test stream is not currently supported, so this mode is currently identical to authenticated.)
O
[O]pen. No encryption of any kind is done.
The client can specify all the modes with which it is willing to communicate. The most strict mode that both the server and the client are willing to use will be selected.
Default:
"AEO"
authscheme schemeopts
authscheme indicates the authentication scheme that should be used to achieve the authenticated or encrypted modes. schemeopts are a list of arguments specific to each particular authentication scheme. Supported authscheme values follow (listed with the schemeopts each scheme requires):
AESKEY userid [keyfile]
This is the initial "simple" shared secret (AES key) model. userid is required to identify which shared secret the server and client should use. keyfile optionally specifies a file to retrieve the AES key from. If keyfile is not specified, the user will be prompted for a passphrase. keyfile can be generated using the aespasswd(1) application.
Default:
Unauthenticated
authscheme and schemeopts are only needed if authenticated communication (A or E modes of authmode) is wanted with sendhost and recvhost.
-B srcaddr
Bind the local address of the client socket to srcaddr. srcaddr can be specified using a DNS name or using standard textual notations for the IP addresses.
Default:
Unspecified (wild-card address selection).
-c recvhost[:port] [authmethod]
Specifies the host that will run the Iperf, Thrulay or Nuttcp server. The :port suffix is optional and is only needed if bwctld is being run on a non-default port number. If an IPv6 address is being specified, note that the accepted format contains the recvhost portion of the specification in square brackets as: [fe80::fe9f:62d8]:4823. This ensures the port number is distinct from the address specification, and is not needed if the :port suffix is not being used.
At least one of the -c or -s options must be specified. If one of them is not specified, it is assumed to be the local host.
authmethod is a specifically ordered list of keywords that is only needed if authenticated communication is wanted with recvhost. These keywords are used to describe the type of communication and authentication that should be used to contact the recvhost. If recvhost and sendhost share the same authentication methods and identities, it is possible to specify the authmethod for both recvhost and sendhost using the -A argument. An authmethod specified with the -c option will override an authmethod specified with the -A argument for communication with the recvhost.
The format for authmethod and a description of the currently available authentication methods are described with the -A argument.
-k
This option has been deprecated. Originally, it was used to specify the keyfile for authentication. All authentication options can now be specified using the -A argument. For the next several versions this option will report an error. Eventually, it may be reclaimed for another purpose.
-s sendhost[:port] [authmethod]
Specifies the host that will run the Iperf, Thrulay or Nuttcp client. The :port suffix is optional and is only needed if bwctld is being run on a non-default port number. If an IPv6 address is being specified, note that the accepted format contains the sendhost portion of the specification in square brackets as: [fe80::fe9f:62d8]:4823. This ensures the port number is distinct from the address specification, and is not needed if the :port suffix is not being used.
At least one of the -c or -s options must be specified. If one of them is not specified, it is assumed to be the local.
authmethod is a specifically ordered list of keywords that is only needed if authenticated communication is wanted with sendhost. These keywords are used to describe the type of communication and authentication that should be used to contact the sendhost. If recvhost and sendhost share the same authentication methods and identities, it is possible to specify the authmethod for both recvhost and sendhost using the -A argument. An authmethod specified with the -s option will override an authmethod specified with the -A argument for communication with the sendhost.
The format for authmethod and a description of the currently available authentication methods are described with the -A argument.
-U
This option has been deprecated. Originally, it was used to specify the username to identify the AES key for authentication. All authentication options can now be specified using the -A argument. For the next several versions this option will report an error. Eventually, it may be reclaimed for another purpose.
Throughput Test Arguments:
The arguments were named to match their counterparts in Iperf as closely as possible.
Some of the options are not available for some of the throughput testers. BWCTL does not support UDP tests, changing the output format or changing the output units for either Nuttcp or Thrulay.
-T
Specify which throughput tester to use:
iperf
thrulay
nuttcp
Default:
None. Selects a tool that the client and server have in common
-b bandwidth
Limit UDP send rate to bandwidth (bits/sec).
Default:
1 Mb
-i interval
Report interval (seconds).
Default:
unset (no intervals reported)
-l len
length of read/write buffers (bytes).
Default:
8 KB TCP, 1470 bytes UDP
-P nStreams
Number of concurrent streams for the test. See the -P option of Iperf for details.
-S TOS
Set the TOS (See RFC 1349) byte in packets.
Default:
0 (not set)
-t time
Duration of test (seconds).
Default:
10
-u
UDP test.
Default:
TCP test
-W window
Same as the -w option, except that the value is advisory. bwctl will attempt to dynamically determine the appropriate TCP window, based upon RTT information gathered from the control socket. If bwctl is unable to dynamically determine a window, the value window will be used.
Default:
Unset (system defaults)
-w window
Socket buffer sizes (bytes). For TCP, this sets the TCP window size. For UDP, this sets the socket receive buffer size.
Default:
Unset (system defaults)
Scheduling Arguments:
-a syncfuzz
Allow bwctl to run without a synchronized system clock. Use this to specify how far off the local clock is from UTC. bwctl prefers to have an NTP synchronized system clock to ensure the two endpoints of the test are actually agreeing to the same scheduled time window for test execution.
If two systems do NOT have a close enough notion of time, then the throughput test will eventually fail because one endpoint of the test will attempt to run at a different time than the other.
If the operating system supports the NTP system calls, and the system clock is determined to be unsynchronized, error messages will still be reported depending upon the value of the -e flag.
When calculating the time errors, this value will be aded in to account for the difference. The maximum time offset can be bounded on the server side, using the max_time_error directive, to prevent a denial of service attack. If set, the server will reject any requests to test with a peer that has too high a timestamp error.
Default:
Unset (Defaults to Set for systems without the NTP system calls)
-I interval
Specifies that bwctl should attempt to run a throughput test every interval seconds.
Default:
Unset. If it is unset, bwctl only runs the test once.
-L longest
Specifies the longest amount of time the client is willing to wait for a reservation window. When bwctl requests a test from the bwctld server, it specifies the earliest time and the latest time it is willing to accept. The latest time is determined by adding this longest option to the earliest time. The earliest time is essentially 'now'. The longest time is specified as a number of seconds.
Default:
If interval is set, the default is 50% of interval. Otherwise, the default is twice the test duration time but no smaller than 5 minutes. (See -t.)
-n nIntervals
Number of tests to perform if the -I option is set.
Default:
Continuous
-R alpha
Randomize the start time of the test within this alpha percent of the interval. Valid values for alpha are from 0-50. bwctl will attempt to run the test every interval +/- alpha percent. For example, if the interval is 300 seconds and alpha is set to 10 percent, then bwctl will attempt to run a test every 270-330 seconds. This option is only useful with the -I option.
Default:
0 (no randomness)
Output Arguments:
-d dir
Specifies directory for results files if the -p option is set.
-e facility
Syslog facility to log messages to.
Default:
LOG_USER
-f units
Specify the units for the tool to use when displaying the results. The accepted values for units are tool specific.
Iperf:
k
Kilobits per second
K
Kilobytes per second
m
Megabits per second
M
Megabytes per second
-h
Print a help message.
-p
Place test results in files. Print the filenames to stdout when results are complete.
-q
Quiet output. Output as little as possible.
-r
Send syslog messages to stderr. This is the default unless the -q option is specified so this option is only useful with the -q option.
-V
Print version information and exit.
-v
Verbose output. Specifying additional -v's increases the verbosity.
-x
Output sender (client) results as well as receiver results. By default, sender results are not output. If the -p option is specified, the sender results are placed in an additional file.
-y format
Specify the output format of the tool. The accepted values for format are tool specific.
Iperf:
c
[c]omma-separated output
ENVIRONMENT VARIABLES
bwctl Environment Variable use default
BWCTLRC Config file ~/.bwctlrc
BWCTL_DEBUG_TIMEOFFSET Offset 0.0(seconds)
LIMITATIONS
Only tested with versions 1.7.0 and 2.0.b of Iperf.
EXAMPLES
bwctl -c somehost.example.com
Run a default 10 second TCP test as soon as possible with local as the sender and somehost.example.com as the receiver, using whichever tools they have in common. Return the results from the receive side of the test.
bwctl -x -c somehost.example.com
Like the previous test, but also return the results from the sender side of the test.
bwctl -x -c somehost.example.com -s otherhost.example.com
Like the previous test, but with otherhost.example.com as the sender instead of local.
bwctl -t 30 -T iperf -s somehost.example.com
Run a 30 second TCP Iperf test with somehost.example.com as the sender and local as the receiver.
bwctl -I 3600 -R 10 -t 10 -u -b 10m -s somehost.example.com
Run a 10 second UDP test about every hour (3600 +/- 360 seconds) with the sender rate limited to 10 Mbits per second from somehost.example.com to local.
bwctl -s somehost.example.com AE AESKEY someuser
Run the default 10 second TCP test. Authenticate using the identity someuser. bwctl will prompt for a passphrase that will be used to create an AES key.
SEE ALSO
bwctld(8) and the http://e2epi.internet2.edu/bwctl/ web site.
For details on Iperf, see the http://sourceforge.net/projects/iperf web site.
For details on Nuttcp, see the http://www.wcisd.hpc.mil/nuttcp/Nuttcp-HOWTO.html web site.
For details on Thrulay, see the http://e2epi.internet2.edu/thrulay/ web site.
ACKNOWLEDGMENTS
This material is based in part on work supported by the National Science Foundation (NSF) under Grant No. ANI-0314723. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.
Index
NAME
SYNOPSIS
DESCRIPTION
ARGUMENTS
Connection/Authentication Arguments:
Throughput Test Arguments:
Scheduling Arguments:
Output Arguments:
ENVIRONMENT VARIABLES
LIMITATIONS
EXAMPLES
SEE ALSO
ACKNOWLEDGMENTS
https://launchpad.net/flowgrind
http://www.internet2.edu/performance/bwctl/bwctl.man.html
Friday, April 15, 2011
TCP/IP Connection cutting on Linux Firewalls and Routers
- TCP/IP Connection cutting on Linux Firewalls and Routers(Introducing "Cutter 1.03")
Summary
Network security administrators sometimes need to be able to abort TCP/IP connections routed over their firewalls on demand. This would allow them to terminate connections such as SSH tunnels or VPNs left in place by employees over night, abort hacker attacks when they are detected, stop high bandwidth consuming downloads - etc. There are many potential applications.This article describes how a Linux IPTables based firewall/router can be used to send the right combination of TCP/IP packets to both ends of a connection to cause them to abort the conversation. It describes the steps required to perform this task, and introduces a new open-source utility called "cutter" that automates the process.
Important Warning
The technique documented here, and the software referred to are designed for "legal" and "appropriate" use by network security administrators and the like. It has been written as part of a larger Linux firewall project, targetting at controlling traffic from peer-to-peer software such as Kazaa, iMesh and others into and out of a private network. It is not designed as a tool for malicious use and the author in no way sanctions such use.Users of the software should be aware that it's actions are easily detectable using a number of readily available network monitoring tools, and it makes no attempt to disguise it's actions. Malicious use of "cutter" could result in a jail sentance in a number of countries around the world.
The author cannot be held responsible for inapropriate use of the documented technique or software (whether in it's original form, or modified).
Introduction
The use of linux systems as IP network firewalls and routers is becoming increasingly popular. The cheapness of the software and hardware combine with the flexibility and reliability of Linux's networking support to make such a solution highly attractive. It is often possible to deliver routing and fire walling facilities at a fraction of the cost associated with systems provided by industrial heavy-weights such as Cisco, Nortel and others.For the knowledgeable, an out-of-the-box linux distribution such as "RedHat" has many of the features required to build highly personalized firewalls. For the less adventurous; there are cut-down distributions available that are designed specifically for this task. The UK based "SmoothWall" and it's clone "IPCop" are good examples of such an approach; they are highly optimized distributions that include a tiny subset of the software commonly installed by the likes of RedHat, but add a powerful web-based front end for the tasks of configuring and managing the system. These solutions are ideal for small office or home networks.
One advantage of using a Linux system in this way is the ease with which it can be extended or modified. Software can be downloaded from the Internet for free, compiled and installed onto the system to add features such as web proxying (Smoothwall and IPCop already have this), content filtering, anti-virus measures or any other feature you desire.
I have recently been working on a solution to the problems of peer-to-peer traffic filtering on a Linux firewall, and have had to develop a "connection cutter" as part of the system, and it is this tool that is described in this page.
What is a "TCP/IP connection cutter"?
Consider a "client workstation" system in a private network, connected to a server on the public Internet, as in the picture below. The connection passes through the firewall which has two network cards - one connected to each network.The job of the firewall is typically to ensure the safety of the systems on the private network, and the security of any information held on them. It is common for firewalls to be set up to allow the hosts on the private network more or less unrestricted use of services on the public Internet while setting very tight limits on what private systems can be accessed from the outside world.
Firewalls often use a network address mapping technique ("NAT") in order to allow the private addresses to be hidden from the outside world. For example; when the user of the client at 10.0.0.2 accesses the server at 100.1.2.3, the messages that the server receives lead it to believe that it is being accessed by a host with address 200.4.5.6 - ie the public side of the firewall. The firewall "messes about with" (technical term!) the network traffic that passes through it in order to allow this little deception to work.
A TCP/IP connection cutter is a software tool that can be run on the firewall to forcibly abort the connection between the server and the client. This is done in a way that leaves both ends believing that it was the other that initiated the abort. Only a device that sits in the path of the connection (such as the firewall in our example) can do this.
The ability to abort a connection in this way can be useful to firewall administrators for any number of reasons. For example...
- A firewall administrator identifies that a workstation on his network is using a service on the public network that should not be permitted. He can force the closure of the connection. This might be because of the network bandwidth being used, or the nature of the service or some other reason that fits the organization's security policy.
- Or: a firewall administrator can forcibly close SSH tunnels or VPNs that rogue employees leave open over night between their office desktops and home networks. This can be a real problem, and it is a well known access route into private networks.
- Or a web server administrator can request that a rogue incoming connection is terminated without having to "kill" the web server process on the server.
How to kill a TCP/IP connection using "IPTables REJECT with RESET"
Simply put: Fire the "RST" bullet.
At the most basic level, it's dead easy - just send an "RST" packet to both ends of the connection.The TCP/IP protocol defines a packet called "RST", which is short for "Reset". When an RST packet arrives on a connection it is understood to mean that something has gone badly wrong and that the application should abort the conversation. "Badly wrong" means something like a protocol error, unrecoverable corruption, breaks in the circuit etc. The RST packet does not include information about the nature of the problem but is simply a kind of "Panic button" that tells it's recipient to stop talking.
But we cant take the first shot, or shoot on someone else's behalf
Since RST packets present such a golden opportunity for attack, most systems will insist that the sequence number information contained in the packet, and the address from which it comes contain good "expected" values. Some systems even check the layer 2 MAC address before agreeing to act on the instruction to reset. These measures make RSTs harder to "spoof". In fact, the normal RST-sending logic embedded in the Linux kernel (and, I imagine: others) is to send RSTs only in response to incoming packets - that way, the sequence number can be guaranteed to be correct.IPTables as a possible "gun" (but not a perfect one)
One solution for a Linux router using IPTables is to define a rule that says "reject packets sent from address 100.1.2.3 to 10.0.0.2 (and visa versa) with an RST".To use this technique, we must..
- Discover which IP addresses and port numbers refer to both ends of the conversation, and the "NAT"ed addresses that are used to present them over the subnets. All the relevant information is held in the pseudo file /proc/net/ip_conntrack.
- Build an IPTables rule that rejects relevant outbound forwarded traffic by responding to it with an RST. For our example above, the iptables command will look something like the following (depending on how your tables are named, and the port numbers used)..
- iptables -A FORWARD -p tcp -s 10.0.0.2 --sport 3245 -d 100.1.2.3 -dport 22 -j REJECT --reject-with tcp-reset
-
- Build another rule that does the same for inbound traffic.
- iptables -A FORWARD -p tcp -s 100.1.2.3 --sport 22 -d 10.0.0.2 --dport 3245 -j REJECT -reject-with tcp-reset
-
- Watch the network traffic using something like tcpdump to wait for the next incoming and outgoing packets from both ends of the connection, and observe that IPTables does indeed send the RST packets back, or keep a track of the iptables counters for the rules we have created.
- Finally; remove the IPTables rules so that they don't interfere with future traffic or clog up precious table space.
- iptables -D FORWARD -p tcp -s 10.0.0.2 --sport 3245 -d 100.1.2.3 -dport 22 -j REJECT --reject-with tcp-reset
- iptables -D FORWARD -p tcp -s 100.1.2.3 --sport 22 -d 10.0.0.2 --dport 3245 -j REJECT -reject-with tcp-reset
-
However, there are a couple of issues with this approach.
First: IPTables will not send an RST until an incoming or outgoing packet arrives from the client or server for it to reject. If the application is connected but idle - this may take many minutes or hours to occur. An example is an SSH or telnet session where the user is typing nothing. The connection stays "up" until the user hits the first keystroke on the keyboard, at which point a packet is sent to the server (via the firewall), and firewall can respond to it with an RST.
Second; we cant abort both ends simultaneously. To take the SSH or telnet example already sited - the client will abort the connection when it gets it's RST after the user presses a key, but the server will know nothing about what has happened, and will still believe the connection to be open. This situation will persist until the server sends a packet - which it may never do.
It would be nice to be able to force the transmission of RST packets immediately, so closing the connection even when there is no application data being sent.
In effect, the firewall cant "shoot" it's RST bullet until client and/or server have sent something through the firewall that allows it to determine the sequence number that should be included in the RST packet. Simply guessing a sequence number wont work - the client and server will (if they are sensible) ignore the packet.
Better: How to cut a TCP/IP connection by sending a FIN
The idea of sending an "RST" packet to both ends is good, but we could do with a better way of getting it sent than turning on an IPtables "reject" rule and waiting for something to arrive that the firewall can reject. What is needed is some way of forcing the two ends of the connection to send a packet to the firewall that IPtables can then respond to.There is a feature of the TCP/IP protocol that we could use to good effect here :- If a packet (other than an RST) is received on a connection that has the wrong sequence number, then the host responds by sending a corrective "ACK" packet back. This "ACK" reply is designed to put the sequence numbers at both ends back into step - and allows the protocol to retransmit packets that got lost. This is very nice for our needs - if the firewall sends a packet that is "correct" in all respects except for the sequence number, then the host very helpfully tells us what should have been used. We can then use this information to build the RST packet that will abort the connection.
Following this idea, we can do the following..
- Identify the IP addresses and port numbers of the hosts at both ends of the connection, and any NATted address that the firewall is using to present the private host to the public network by inspecting the /proc/net/ip_conntrack file.
- Build the IPtables rules that will send RST packets to both ends of the connection when an incoming packet arrives (the rules have already been described).
- Send a packet to both ends with bad sequence numbers.
- use sequence_number = zero - it's pretty much bound to be wrong (which is what we want).
- use a FIN packet - they fairly easy to put together and don't need to contain any "data" payload.
- Send the packet using Layer 2 raw Ethernet access - so that the IP layers don't mess around with the headers.
- We need the MAC address of the next hop to the target machine for this - which we can determine by inspecting the firewall's ARP and routing tables.
-
-
- Wait for the hosts to respond with their ACK, and for Iptables to send it's RST packet back.
- Remove the newly added Iptables rules.
But - we still have to have a process for determining when the FIN - ACK - RST packets have done their stuff so that we can clean up the iptables rules. Various options for doing this are possible. For example: the rules can be left in place, while a cron job does a periodic cleanup. Or we can "listen in" to the packet flow using Linux's raw packet mode. I'm sure there are any number of other "clean up" solutions that could be devised.
Better still: Without IPTables rules
It turns out that we can produce the same FIN -> ACK -> RST sequence of packets without using Iptables to generate the RST at all - so there are no IPTables rules to create or clean up.The "raw packet socket" technique we have already suggested for the sending of the FIN packet can be used to listen for the incoming ACK, and send the resulting RST. This is how the "cutter" program works.
Cutter 1.03
Description and Synopsis
"Cutter" is an open source program that uses the FIN-ACK-RST packet technique described above to abort TCP/IP connections routed over the firewall or router on which it is run. It can be called using one of the following four syntaxes..cutter ip-address
Example: "cutter 10.10.0.45"
Cuts all connections passing through the firewall between any ports on the specified ip-address (either a "private" or "public" address) and any other hosts. This can be used to close down all incoming connections to a particular server, all outgoing connections from a particular client or all outgoing connections to a server.
cutter ip-address port
Example: "cutter 200.1.2.3 80"
Cuts all connections to or from the specified ip-address/port pair. This allows the user to be a little more specific than the previous example and allows targetting of specific services on specific hosts.
cutter ip-address-1 port-1 ip-address-2
Example "cutter 200.1.2.3 22 10.10.0.45"
Cuts all connections between ip-address-2 and ip-address-1/port-1. This allows the user to cut connections between a specified "client" and a particular service on a specified host. Our example closes host 10.10.0.45's SSH connection to server 200.1.2.3.
cutter ip-address-1 port-1 ip-address-2 port-2
Example: "cutter 200.1.2.3 22 10.10.0.45 32451"
Cuts the specific connection between the two ip/port number pairs given.
License
Cutter 1.03 is released under the terms of GNU GENERAL PUBLIC LICENSE Version 2 (June 1991) and comes with all the usual freedom, openness and disclaimers associated with that license.Status
Cutter 1.03 should be considered experimental. The author is releasing a tool that works on the systems he has access to (namely: IPCop and RedHat Linux), and he is seeking input on it's use on other systems, ideas for improvement, offers of sponsorship - etc.Courtesy : http://www.lowth.com/cutter/
Tuesday, April 12, 2011
The TCPIP nagle algorithm can slow down network
When a file is sent over the network the file is chopped up in small packets, which are then sent to the destination. The TCPIP protocol is a secure protocol and demands that each packet is acknowledged, so the receiver has to respond with a matching ACK packet. It is important for the sender not to flood the receiver and for the receiver not to flood the sender.
The Nagle algorithm (by John Nagle) is a method for congestion control (RFC 896), so the sender won't flood the receiver with data. When the sender sends a packet to the receiver, then the sender will wait for an ACK from the receiver before sending the following packets.
The delayed ACK algorithm is also a method for congestion control (RFC 2581), so the receiver won't flood the network with ACK packets. When the receiver has to sent an ACK in response to a packet, then it waits a little (200 ms or until it has 2 outstanding ACKs) to see if more packets should arrive that it can acknowledge with that single ACK.
The delayed ACK algorithm can slow down the network, if the sender doesn't send the next packet before receiving the ACK of the previous packet (200 ms delay).
The delayed ACK algorithm can get even worse when combined with the Nagle algorithm, if the sender sends two packets and waits for ACK, as nagling will hold back the second packet until recieving the delayed ACK for the first packet (400 ms delay).
The solution to such slow downs is to disable or lower the delayed ACK timeout.
To configure the max outstanding ACKs in Windows XP/2003/Vista/2008:
Note before trying to disable ACK delay (RFC 1122) one should at least consider the following:
Note SMB Signing requires that SMB commands are processed synchronously, so a client is only allowed to send the next SMB command when it receives ACK of the previous one (Only one outstanding). This means that a client can max sent 5 SMB Commands/sec, as it has to wait for the Server's 200 ms ACK delay before it is allowed to sent the next SMB Command. This can cause very low performance when copying small files to a Server with SMB signing enabled (Imagine copying 1000 files of 1 Kbyte).
Note if a computer's only job is to receive large files or streaming data, one can increase performance by increasing the number of outstanding ACKs before it sends an ACK (TcpAckFrequency). It will allow acknowledgment of large chunks of data with a single ACK packet instead of sending ACK for every 2 packet. Make sure that the TCPIP RWIN is larger than TcpAckFrequency*MTU, as the sender will stop sending data if it fills the TCPIP RWIN without getting an ACK. Recommended values:
The Nagle algorithm (by John Nagle) is a method for congestion control (RFC 896), so the sender won't flood the receiver with data. When the sender sends a packet to the receiver, then the sender will wait for an ACK from the receiver before sending the following packets.
The delayed ACK algorithm is also a method for congestion control (RFC 2581), so the receiver won't flood the network with ACK packets. When the receiver has to sent an ACK in response to a packet, then it waits a little (200 ms or until it has 2 outstanding ACKs) to see if more packets should arrive that it can acknowledge with that single ACK.
The delayed ACK algorithm can slow down the network, if the sender doesn't send the next packet before receiving the ACK of the previous packet (200 ms delay).
The delayed ACK algorithm can get even worse when combined with the Nagle algorithm, if the sender sends two packets and waits for ACK, as nagling will hold back the second packet until recieving the delayed ACK for the first packet (400 ms delay).
The solution to such slow downs is to disable or lower the delayed ACK timeout.
To configure the max outstanding ACKs in Windows XP/2003/Vista/2008:
[HKEY_LOCAL_MACHINE \SYSTEM \CurrentControlSet \Services \Tcpip \Parameters \Interfaces \{Adapter-id}]To configure the interval timeout in Win2000 SP3+:
TcpAckFrequency = 2 (Default=2, 1=Disables delayed ACK, 2-n = If n outstanding ACKs before timed interval, sent ACK)
More Info MS KB Q328890
More Info MS KB 815230 (XP/2003 needs hotfix or SP2 for it to work)
More Info MS KB 935458 (Vista needs hotfix or SP1 for it to work)
[HKEY_LOCAL_MACHINE \SYSTEM \CurrentControlSet \Services \Tcpip \Parameters \Interfaces \{Adapter-id}]To configure the interval timeout in WinNT SP4 (Go to the Services-key and do a search for "TCPIP" to find the different adapters using TCPIP):
TcpDelAckTicks = 1 (Default=2, 0=Disables delayed ACK, 1-6 = 100-600 ms)
More Info MS KB Q311833
More Info MS KB Q321098
More Info MS KB Q321169
[HKEY_LOCAL_MACHINE \SYSTEM \CurrentControlSet \Services \{Adapter-Name} \Parameters \Tcpip]Note if disabling or shortening delayed ACK on a few machines (Like a file-server or domain-controller), then it will probably result in greater network performance for those machines. If on large corporate network and disabling delayed ACK for all computers, then it will most likely lower the available bandwidth for actual filetransfer as more of the bandwidth is used for sending ACKs.
TcpDelAckTicks = 1 (Default=2, 0=Disables nagling, 1-6 = 100-600 ms)
Note before trying to disable ACK delay (RFC 1122) one should at least consider the following:
- Increased performance will only be seen if requests are sent to your machine, and the requesters doesn't request anything else before your machine replies back(ACK) to the first request.
- Some additions to the above statement:
- If the application doing socket communication uses the socket option TCP_NODELAY, then it will disable the nagle algorithm but not the delayed ACK.
- If all of the upload bandwidth is already used (easy if slow connection), then then disabling delayed ACK will lower performance because it will generate even more upload traffic.
- If on a half duplex connection, then disabling delayed ACK will lower performance because only one party can sent at a time (Receiver will block the sender when sending ACK).
- If on a ethernet hub with other computers(Instead of a switch), then disabling delayed ACK will lower performance because the increased traffic will increase chance of collision and require retransmissions.
Note SMB Signing requires that SMB commands are processed synchronously, so a client is only allowed to send the next SMB command when it receives ACK of the previous one (Only one outstanding). This means that a client can max sent 5 SMB Commands/sec, as it has to wait for the Server's 200 ms ACK delay before it is allowed to sent the next SMB Command. This can cause very low performance when copying small files to a Server with SMB signing enabled (Imagine copying 1000 files of 1 Kbyte).
Note if a computer's only job is to receive large files or streaming data, one can increase performance by increasing the number of outstanding ACKs before it sends an ACK (TcpAckFrequency). It will allow acknowledgment of large chunks of data with a single ACK packet instead of sending ACK for every 2 packet. Make sure that the TCPIP RWIN is larger than TcpAckFrequency*MTU, as the sender will stop sending data if it fills the TCPIP RWIN without getting an ACK. Recommended values:
- 1 GigaBit: TcpAckFrequency = 13 (RWIN = 64 KByte)
- 100 MegaBit: TcpAckFrequency = 5 (RWIN = 17 KByte)
- 10 MegaBit: TcpAckFrequency = 2 (RWIN = 8 KByte)
Subscribe to:
Posts (Atom)