Summary:

This article describes how to tune TCP/IP for maximum performance with MassTransit.

Description:

Explanation & Overview

TCP/IP Settings

As network speeds have increased some of the default configuration settings for TCP/IP no longer result in maximum performance. In particular, networks that have high bandwidth and long round trip times, e.g. DSL, need special attention. These particular types of networks have large "bandwidth delay product" ¹, or BDP, which means that they can have substantial amounts of unacknowledged data in the network. In many of today's networks, the BDP has increased beyond what was imagined when the TCP protocol was designed. In order to allow for this increase in BDP several extensions to TCP have been created and most modern TCP/IP stacks implement these features. However, these features are not always enabled by default and need to be explicitly turned on by system administrators.

MassTransit provides solutions to enable the TCP/IP features described above on both Macintosh and Windows. On Macintosh [OS X] TCP/IP features are enabled in a small application named "mtstacktuner" that is described below. On Windows MassTransit ships with several .reg files that can be used to modify or add the required registry entries.

Large Windows (RFC 1323)

The initial design of TCP limits send and receive buffers to 64K. If the network BDP is small [either because the link is slow or the RTT is small, e.g. LAN] then these buffer sizes are adequate. However, on networks with a large BDP larger buffers are needed. RFC1323 describes a mechanism to allow for buffers larger than 64K and most modern TCP/IP stacks have this enabled by default. E.g. on Mac OS X, typing the following command in a Terminal window and scanning the output will reveal that the RFC1323 option is enabled:

sysctl net.inet.tcp.rfc1323

On Windows XP the RFC1323 option is enabled by default but for Windows 2000 a registry change is needed in the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

Add the following name/value pair. Note that the name is case sensitive.

Tcp1323Opts

REG_DWORD

3

Note the TCP1323Opts entry is not relevant for Windows NT.

Large Buffers

Once the TCP/IP stack can support larger buffers, the stack must be configured to use them.

On Windows larger buffers are enabled by making more registry edits. Like the RFC1323 option, Windows XP already uses larger buffers when necessary. For Windows 2000 systems add the following name/value pairs to the same key as before:

GlobalMaxTcpWindowSize	REG_DWORD	262144
TcpWindowSize	REG_DWORD	262144

For Mac OS X the default buffer size is 32K which is far too low for high BDP networks. Unfortunately, to change the default buffer size you must execute the "sysctl" commands as root. In addition, these changes must be made each time the computer is restarted. To solve this problem MassTransit provides a utility component called "mtstacktuner" that makes these changes automatically when the computer is booted. The component is provided in the MassTransit Web Client installer that prompts the user whether or not to install the component. To allow MassTransit server computers to be tuned, Group Logic also provides the mtstacktuner component in a standalone installer. Both installers place the component in the /Library/StartupItems/ folder. Once installed, the computer must be restarted before the changes will take effect. The component makes the following changes:

Name	Old Value	New Value
net.inet.tcp.sendspace	32768	262144
net.inet.tcp.recvspace	32768	262144
net.inet.tcp.delayed_ack	1	0
kern.ipc.maxsockbuf	262144	2097152

More Sources of Information

"Enabling High Performance Data Transfers"
http://www.psc.edu/networking/perf_tune.html

Microsoft Windows 2000 and Windows XP:
http://rdweb.cns.vt.edu/public/notes/win2k-tcpip.htm

Navas Cable Modem/DSL Tuning Guide
http://cable-dsl.navasgroup.com/

For Further Reading

• RFC 1323: TCP Extensions for High Performance
• TCP/IP Illustrated, Volume 1 W. Richard Stevens. Sections 20.3, 20.4, and 24.4.

The bandwidth delay product is simply the bandwidth of the slowest link multiplied by the round trip time.

Tuning TCP/IP in Windows NT and Windows 2000

 

As explained in the EXPLANATION & OVERVIEW section, in Windows the changes must be made via registry entries. Rather than having the user manually add these entries and value, Group Logic provides a .reg file that will make the additions automatically. In addition Uninstallers are also provided. These are *.inf files that will remove the registry entries made by the installers. Also note that these changes only need to be made in Windows 2000 and Windows NT 4 versions. Windows XP already has the tweaks. Download the installers and uninstallers here: w2ktuning.zip 

To use the registry file, simply double-click on the file. In Windows 2000 a dialog will prompt to confirm adding the registry entries, and that the entries have been added. In Windows NT 4 only a dialog confirming the entries have been added will be displayed. Note: A reboot is not required with these registry entries.

If you want to make the changes manually listed below are the locations, keys, and values that need to be added (Note all items below are case sensitive):

Key	Type	Value
Tcp1323Opts	REG_DWORD	3
GlobalMaxTcpWindowSize	REG_DWORD	262144
TcpWindowSize	REG_DWORD	262144

To use the uninstallers, save the *.inf file to your desktop. Right-click on it, and choose "Install" to run it. Reboot your system for the change to take effect. You may delete the *.inf file after installing it.

Tuning TCP/IP in Mac OS X

Mac OS X tuning is performed through the MassTransit utility component called "mtstacktuner". By default this utility is installed with the OS X native Web Client. If a Mac OS X server requires this utility and does not have the native Web Client, a separate installer for the utility only is available through Group Logic Technical Support. The changes made by this utility are more thoroughly explained in the EXPLANATION & OVERVIEW section.