TCP Stream Connection and Handshaking

Introduction

TCP stream connections are established between Gnutella2 nodes when they elect to form a permeant link, creating the fundamental network topology of a highly interconnected hub network serving dense clusters of leaf nodes.

Initiation

TCP connections are initiated by leaf or hub nodes in an attempt to gain a connection to a hub node. Leaf nodes are never the target of an outbound connection. The TCP port number is not standardised, and must be stored with the IP address.

Handshaking

Upon the establishment of a TCP stream connection between two Gnutella2 nodes, a handshaking phase must be completed to negotiate the nature of the link and exchange other necessary information.

This handshaking phase is the only part of the communication which remains compatible with the old Gnutella network, allowing new connections to be negotiated without fore-knowledge the capabilities of the other node. The handshaking process has been well documented elsewhere, however, a short summary is provided here.

Handshake Stages

The Gnutella handshake process consists of three header blocks. The node which initiated the connection sends an initial header block, of the form:

GNUTELLA CONNECT/0.6
Listen-IP: 1.2.3.4:6346
Remote-IP: 6.7.8.9
User-Agent: Shareaza 1.8.2.0
Accept: application/x-gnutella2
X-Ultrapeer: False

The receiver then responds with its own header block:

GNUTELLA/0.6 200 OK
Listen-IP: 6.7.8.9:6346
Remote-IP: 1.2.3.4
User-Agent: Shareaza 1.8.2.0
Content-Type: application/x-gnutella2
Accept: application/x-gnutella2
X-Ultrapeer: True
X-Ultrapeer-Needed: False

Finally, the initiator accepts the receiver's header block, and provides any final information:

GNUTELLA/0.6 200 OK
Content-Type: application/x-gnutella2
X-Ultrapeer: False
The latter two stages may be replaced with an error condition if the connection is
being rejected. Appropriate error status codes are returned in this case, for example:
GNUTELLA/0.6 503 Too many connections
(more headers)

Note that only the HTTP-style error code should be interpreted by the software: any descriptive text provided is for display purposes only and is not standardised.

Headers

Important headers which are required or strongly recommended are detailed in the following sections.

Addressing Headers

Two important headers to send on all connections are "Remote-IP" and "Listen-IP". Both of these headers should be sent on the first transmission, meaning in the first and second header blocks in the three block exchange.

The Remote-IP header contains the IP address from which the remote host is connecting. This allows a remote host operating through some kind of network address translation system to learn its effective external address.

The Listen-IP header contains the IP address and port number that the local host is listening for inbound TCP connections on. It should be listening for UDP datagrams on the same port. The format of this header is "IP:PORT", eg "1.2.3.4:6346".

Identification

The User-Agent header is used to identify the client software operating on the sending node. It should be sent on the first transmission, meaning in the first and second header blocks in the three block exchange. Note that this is a descriptive string that often includes a version number, and is not a "vendor code" as described elsewhere.

Content Type (Protocol)

The Accept and Content-Type header exchange is used to negotiate the data protocol that will be used in the connection, in this case Gnutella2. The Gnutella2 content type is "application/x-gnutella2", and this exchange follows standard HTTP rules for negotiating content type.

The first step is to advertise support for the content type (Gnutella2) in the first header block with "Accept: application/x-gnutella2". The responding node will then indicate that it will send Gnutella2 content with "Content-Type: application/xgnutella2", and that it also supports Gnutella2 with "Accept: application/x-gnutella2". The initiating host then confirms that it will be sending Gnutella2 with "Content- Type: application/x-gnutella2" in the third header block. For more information on the Accept/Content-Type exchange, consult a HTTP reference.

Note that the content type negotiation process is designed to be a "one way" process, i.e. a different content type can be negotiated for sending and receiving. However when the gnutella2 protocol is negotiated, both channels must use the same content type. This means that a receiving node must not accept Gnutella2 if the initiator did not advertise support for it, and if at the end of the handshake bidirectional Gnutella2 was not negotiated, the connection should be terminated.

Node State Negotiation

There are two node types in a Gnutella2 peer to peer network, a hub and a leaf as described in Node Types and Responsibilities. During the initial handshake the two parties must exchange their current node type and advise of their capabilities, negotiating the node types they will adopt when the connection completes, and indeed whether it should complete at all.

As the handshake sequence is compatible with Gnutella1, the headers involved in negotiating node types are identical to those used to negotiate Gnutella1 "Ultrapeer" states:

X-Ultrapeer: [True|False]
X-Ultrapeer-Needed: [True|False]

Both headers contain a Boolean value, "true" or "false", case insensitive.

The X-Ultrapeer header indicates whether the transmitting node is currently operating as a hub. Hub nodes will send "X-Ultrapeer: True" while leaf nodes will send "X-Ultrapeer: False".

The X-Ultrapeer-Needed header indicates whether the transmitting node would like (and allow) the receiver to be a hub. A hub which sees no need for additional hubs in its area of the network will send "X-Ultrapeer-Needed: False", indicating to the connecting node that it must operate in leaf mode if it wishes to connect. A hub which sees a need for additional hubs will send "X-Ultrapeer-Needed: True", indicating that the receiving node should become a hub if it is capable of doing so. A leaf may send "X-Ultrapeer-Needed: True" to indicate that it is seeking a connection to a hub.

The X-Ultrapeer header should be sent on all three of the header blocks to indicate the current intended state of the node, while the X-Ultrapeer-Needed header should be sent in the first two header blocks only to indicate the desired status of the receiver. If the nodes cannot agree on a satisfactory arrangement, the connection will be terminated at or prior to the third header block with an appropriate message, for example:

GNUTELLA/0.6 503 Too many hub connections
GNUTELLA/0.6 503 Too many leaves
GNUTELLA/0.6 503 I have leaves, can't downgrade to leaf mode
GNUTELLA/0.6 503 Leaf mode disabled

Hub Address Exchange

It is desirable for connecting nodes to exchange the node addresses of other hubs on the network to facilitate rapid connection. The Gnutella2 protocol includes highly efficient methods to share hub addresses with peers once connected, but for a node trying to connect and encountering only "full" hubs, learning new hubs to try is helpful.

The X-Try-Ultrapeers header was developed for this purpose, and like the rest of the handshake is also semi-compatible with Gnutella1. It contains a comma separated list of hub node addresses and ports, along with a timestamp recording the time the hub was last seen. For example:

X-Try-Ultrapeers: 1.2.3.4:6346 2003-03-25T23:59Z, [..more..]

Hub addresses should not be sent unless the transmitter has reasonable knowledge of the hub's existence and the timestamp is not too old. The content type negotiation headers should be used to verify that the listed hub addresses are indeed Gnutella2 hubs.

Compression Negotiation and Encoding

The Gnutella2 architecture makes widespread use of "deflate" compression, due to its high availability and ease of integration. Support of compressed TCP links is not a requirement in the Gnutella2 standard, however it is strongly recommended.

Deflate compression of a TCP link is negotiated with the pair standard HTTP headers "Accept-Encoding" and "Content-Encoding". For example:

Header block one (initiator):

Accept-Encoding: deflate

Header block two (receiver):

Accept-Encoding: deflate
Content-Encoding: deflate

Header block three (initiator):

Content-Encoding: deflate

In this example the initiator advertises support for receiving a deflated connection. The receiver then indicates that it too supports receiving a deflated connection, and that it intends to transmit deflated data. Finally, the initiator upon noting that the receiver supports deflate indicates that it too will transmit deflated data.

Note that unlike the content-type / protocol negotiation, deflated encoding can be applied on either incoming, outgoing or both channels of a connection.

For performance reasons, nodes should consider whether they can afford to support additional deflated connections before advertising support for them, or agreeing to provide a deflated data stream. In the Gnutella2 network topology, all links benefit from compression except the leaf to hub channel of the leaf/hub link. Exempting this channel from compression saves the leaf and more importantly the hub a considerable CPU and RAM investment.

Post Handshake Communication

After the third and final header block has been received by the initiator, subsequent communication over the TCP stream occurs in the negotiated protocol, with the negotiated encoding. This means that while the handshake sequence was backwards compatible with Gnutella1, after Gnutella2 support has been negotiated all subsequent communication occurs in the Gnutella2 common protocol - an entirely new system not backwards compatible with any other protocol.