TCP Stream Connection and Handshaking: Difference between revisions

From Gnutella2
Jump to navigation Jump to search
No edit summary
 
 
(11 intermediate revisions by 5 users not shown)
Line 1: Line 1:
[[UDP_Transceiver|UDP Transceiver >>]] | [[Main_Page|Main Page]]
== Introduction ==
== Introduction ==


TCP stream connections are established between Gnutella2 nodes when they elect to
TCP stream connections are established between Gnutella2 nodes when they elect to
form a permeant link, creating the fundamental network topology of a highly
form a permanent link, creating the fundamental network topology of a highly
interconnected hub network serving dense clusters of leaf nodes.
interconnected hub network serving dense clusters of leaf nodes.


Line 20: Line 23:
compatible with the old Gnutella network, allowing new connections to be negotiated
compatible with the old Gnutella network, allowing new connections to be negotiated
without fore-knowledge the capabilities of the other node. The handshaking process
without fore-knowledge the capabilities of the other node. The handshaking process
has been well documented elsewhere, however a short summary is provided here.
has been well documented elsewhere, however, a short summary is provided here.


=== Handshake Stages ===
=== Handshake Stages ===
Line 34: Line 37:
User-Agent: Shareaza 1.8.2.0
User-Agent: Shareaza 1.8.2.0
Accept: application/x-gnutella2
Accept: application/x-gnutella2
X-Ultrapeer: False
X-Hub: False
</nowiki>
</nowiki>
</pre>
</pre>
Line 48: Line 51:
Content-Type: application/x-gnutella2
Content-Type: application/x-gnutella2
Accept: application/x-gnutella2
Accept: application/x-gnutella2
X-Ultrapeer: True
X-Hub: True
X-Ultrapeer-Needed: False
X-Hub-Needed: False
</nowiki>
</nowiki>
</pre>
</pre>
Line 60: Line 63:
GNUTELLA/0.6 200 OK
GNUTELLA/0.6 200 OK
Content-Type: application/x-gnutella2
Content-Type: application/x-gnutella2
X-Ultrapeer: False
X-Hub: False
</nowiki>
</pre>
 
The latter two stages may be replaced with an error condition if the connection is
The latter two stages may be replaced with an error condition if the connection is
being rejected. Appropriate error status codes are returned in this case, for example:
being rejected. Appropriate error status codes are returned in this case, for example:
<pre>
<nowiki>
GNUTELLA/0.6 503 Too many connections
GNUTELLA/0.6 503 Too many connections
(more headers)
(more headers)
Line 84: Line 93:
The Remote-IP header contains the IP address from which the remote host is
The Remote-IP header contains the IP address from which the remote host is
connecting. This allows a remote host operating through some kind of network
connecting. This allows a remote host operating through some kind of network
address translation system to learn its effective external address.
address translation system, to learn its effective external address.


The Listen-IP header contains the IP address and port number that the local host is
The Listen-IP header contains the IP address and port number that the local host is
Line 107: Line 116:
The first step is to advertise support for the content type (Gnutella2) in the first
The first step is to advertise support for the content type (Gnutella2) in the first
header block with "Accept: application/x-gnutella2". The responding node will then
header block with "Accept: application/x-gnutella2". The responding node will then
indicate that it will send Gnutella2 content with "Content-Type: application/xgnutella2",
indicate that it will send Gnutella2 content with "Content-Type: application/x-gnutella2",
and that it also supports Gnutella2 with "Accept: application/x-gnutella2".
and that it also supports Gnutella2 with "Accept: application/x-gnutella2".
The initiating host then confirms that it will be sending Gnutella2 with "Content-
The initiating host then confirms that it will be sending Gnutella2 with "Content-
Line 113: Line 122:
Accept/Content-Type exchange, consult a HTTP reference.
Accept/Content-Type exchange, consult a HTTP reference.


Note that the content type negotiation process is designed to be a "one way"
Note that the content type negotiation process is designed to be a "one-way"
process, i.e. a different content type can be negotiated for sending and receiving.
process, i.e. a different content type can be negotiated for sending and receiving.
However when the gnutella2 protocol is negotiated, both channels must use the
However, when the gnutella2 protocol is negotiated, both channels must use the
same content type. This means that a receiving node must not accept Gnutella2 if
same content type. This means that a receiving node must not accept Gnutella2 if
the initiator did not advertise support for it, and if at the end of the handshake
the initiator did not advertise support for it, and if at the end of the handshake,
bidirectional Gnutella2 was not negotiated, the connection should be terminated.
bidirectional Gnutella2 was not negotiated, the connection should be terminated.


=== Node State Negotiation ===
=== Node State Negotiation ===


There are two node types in a Gnutella2 peer to peer network, a hub and a leaf as
There are two node types in a Gnutella2 peer to peer network, a hub and a leaf, as
described in [[Node Types and Responsibilities]]. During the initial handshake
described in [[Node Types and Responsibilities]]. During the initial handshake,
the two parties must exchange their current node type and advise of their
the two parties must exchange their current node type and advise of their
capabilities, negotiating the node types they will adopt when the connection
capabilities, negotiating the node types they will adopt when the connection
Line 129: Line 138:


As the handshake sequence is compatible with Gnutella1, the headers involved in
As the handshake sequence is compatible with Gnutella1, the headers involved in
negotiating node types are identical to those used to negotiate Gnutella1 "Ultrapeer"
negotiating node types should not be identical to those used to negotiate Gnutella1 to minimize network pollution.
states:
states:


<pre>
<pre>
<nowiki>
<nowiki>
X-Ultrapeer: [True|False]
X-Hub: [True|False]
X-Ultrapeer-Needed: [True|False]
X-Hub-Needed: [True|False]
</nowiki>
</nowiki>
</pre>
</pre>
Line 141: Line 150:
Both headers contain a Boolean value, "true" or "false", case insensitive.
Both headers contain a Boolean value, "true" or "false", case insensitive.


The X-Ultrapeer header indicates whether the transmitting node is currently
The X-Hub header indicates whether the transmitting node is currently
operating as a hub. Hub nodes will send "X-Ultrapeer: True" while leaf nodes will
operating as a hub. Hub nodes will send "X-Hub: True" while leaf nodes will
send "X-Ultrapeer: False".
send "X-Hub: False".


The X-Ultrapeer-Needed header indicates whether the transmitting node would like
The X-Hub-Needed header indicates whether the transmitting node would like
(and allow) the receiver to be a hub. A hub which sees no need for additional hubs in
(and allow) the receiver to be a hub. A hub which sees no need for additional hubs in
its area of the network will send "X-Ultrapeer-Needed: False", indicating to the
its area of the network, will send "X-Hub-Needed: False", indicating to the
connecting node that it must operate in leaf mode if it wishes to connect. A hub
connecting node that it must operate in leaf mode if it wishes to connect. A hub
which sees a need for additional hubs will send "X-Ultrapeer-Needed: True",
which sees a need for additional hubs, will send "X-Hub-Needed: True",
indicating that the receiving node should become a hub if it is capable of doing so. A
indicating that the receiving node should become a hub, if it is capable of doing so. A
leaf may send "X-Ultrapeer-Needed: True" to indicate that it is seeking a connection
leaf may send "X-Hub-Needed: True" to indicate that it is seeking a connection
to a hub.
to a hub.


The X-Ultrapeer header should be sent on all three of the header blocks to indicate
The X-Hub header should be sent on all three of the header blocks to indicate
the current intended state of the node, while the X-Ultrapeer-Needed header should
the current intended state of the node, while the X-Hub-Needed header should
be sent in the first two header blocks only to indicate the desired status of the
be sent in the first two header blocks only, to indicate the desired status of the
receiver. If the nodes cannot agree on a satisfactory arrangement, the connection
receiver. If the nodes cannot agree on a satisfactory arrangement, the connection
will be terminated at or prior to the third header block with an appropriate message,
will be terminated at or prior to the third header block with an appropriate message,
Line 169: Line 178:
</nowiki>
</nowiki>
</pre>
</pre>
- NOTE: because X-Ultrapeer and X-Ultrapeer-Needed has been used on Gnutella2 for a longtime, there should be some backward compatibility code to comunicate with older nodes until they disappear.


=== Hub Address Exchange ===
=== Hub Address Exchange ===
Line 178: Line 189:
helpful.
helpful.


The X-Try-Ultrapeers header was developed for this purpose, and like the rest of
The X-Try-Hubs header was developed for this purpose. It contains a comma
the handshake is also semi-compatible with Gnutella1. It contains a comma
separated list of hub node addresses and ports, along with a timestamp recording
separated list of hub node addresses and ports, along with a timestamp recording
the time the hub was last seen. For example:
the time the hub was last seen. For example:
Line 185: Line 195:
<pre>
<pre>
<nowiki>
<nowiki>
X-Try-Ultrapeers: 1.2.3.4:6346 2003-03-25T23:59Z, [..more..]
X-Try-Hubs: 1.2.3.4:6346 2007-01-10T23:59Z, [..more..]
</nowiki>
</nowiki>
</pre>
</pre>


Hub addresses should not be sent unless the transmitter has reasonable knowledge
Hub addresses should not be sent unless the transmitter has reasonable knowledge
of the hub's existence and the timestamp is not too old. The content type negotiation
of the hub's existence and the timestamp is not too old.
headers should be used to verify that the listed hub addresses are indeed Gnutella2
hubs.


=== Compression Negotiation and Encoding ===
=== Compression Negotiation and Encoding ===
Line 198: Line 206:
The Gnutella2 architecture makes widespread use of "deflate" compression, due to
The Gnutella2 architecture makes widespread use of "deflate" compression, due to
its high availability and ease of integration. Support of compressed TCP links is not a
its high availability and ease of integration. Support of compressed TCP links is not a
requirement in the Gnutella2 standard, however it is strongly recommended.
requirement in the Gnutella2 standard, however, it is strongly recommended.


Deflate compression of a TCP link is negotiated with the pair standard HTTP headers
Deflate compression of a TCP link is negotiated with the pair standard HTTP headers
Line 228: Line 236:
</pre>
</pre>


In this example the initiator advertises support for receiving a deflated connection.
In this example, the initiator advertises support for receiving a deflated connection.
The receiver then indicates that it too supports receiving a deflated connection, and
The receiver then indicates that it too supports receiving a deflated connection, and
that it intends to transmit deflated data. Finally, the initiator upon noting that the
that it intends to transmit deflated data. Finally, the initiator upon noting that the
receiver supports deflate indicates that it too will transmit deflated data.
receiver supports deflate, indicates that it too will transmit deflated data.


Note that unlike the content-type / protocol negotiation, deflated encoding can be
Note that unlike the content-type / protocol negotiation, deflated encoding can be
applied on either incoming, outgoing or both channels of a connection.
applied on either incoming, outgoing, or both channels of a connection.


For performance reasons, nodes should consider whether they can afford to support
For performance reasons, nodes should consider whether they can afford to support
additional deflated connections before advertising support for them, or agreeing to
additional deflated connections before advertising support for them, or agreeing to
provide a deflated data stream. In the Gnutella2 network topology, all links benefit
provide a deflated data stream. In the Gnutella2 network topology, all links benefit
from compression except the leaf to hub channel of the leaf/hub link. Exempting this
from compression, except the leaf to hub channel of the leaf/hub link. Exempting this
channel from compression saves the leaf and more importantly the hub a
channel from compression, saves the leaf and more importantly, the hub, a
considerable CPU and RAM investment.
considerable CPU and RAM investment.


Line 248: Line 256:
communication over the TCP stream occurs in the negotiated protocol, with the
communication over the TCP stream occurs in the negotiated protocol, with the
negotiated encoding. This means that while the handshake sequence was backwards
negotiated encoding. This means that while the handshake sequence was backwards
compatible with Gnutella1, after Gnutella2 support has been negotiated all
compatible with Gnutella1, after Gnutella2 support has been negotiated, all
subsequent communication occurs in the Gnutella2 common protocol - an entirely
subsequent communication occurs in the Gnutella2 common protocol - an entirely
new system not backwards compatible with any other protocol.
new system not backwards compatible with any other protocol.

Latest revision as of 14:57, 31 December 2008

UDP Transceiver >> | Main Page


Introduction

TCP stream connections are established between Gnutella2 nodes when they elect to form a permanent link, creating the fundamental network topology of a highly interconnected hub network serving dense clusters of leaf nodes.

Initiation

TCP connections are initiated by leaf or hub nodes in an attempt to gain a connection to a hub node. Leaf nodes are never the target of an outbound connection. The TCP port number is not standardised, and must be stored with the IP address.

Handshaking

Upon the establishment of a TCP stream connection between two Gnutella2 nodes, a handshaking phase must be completed to negotiate the nature of the link and exchange other necessary information.

This handshaking phase is the only part of the communication which remains compatible with the old Gnutella network, allowing new connections to be negotiated without fore-knowledge the capabilities of the other node. The handshaking process has been well documented elsewhere, however, a short summary is provided here.

Handshake Stages

The Gnutella handshake process consists of three header blocks. The node which initiated the connection sends an initial header block, of the form:


GNUTELLA CONNECT/0.6
Listen-IP: 1.2.3.4:6346
Remote-IP: 6.7.8.9
User-Agent: Shareaza 1.8.2.0
Accept: application/x-gnutella2
X-Hub: False

The receiver then responds with its own header block:


GNUTELLA/0.6 200 OK
Listen-IP: 6.7.8.9:6346
Remote-IP: 1.2.3.4
User-Agent: Shareaza 1.8.2.0
Content-Type: application/x-gnutella2
Accept: application/x-gnutella2
X-Hub: True
X-Hub-Needed: False

Finally, the initiator accepts the receiver's header block, and provides any final information:


GNUTELLA/0.6 200 OK
Content-Type: application/x-gnutella2
X-Hub: False

The latter two stages may be replaced with an error condition if the connection is being rejected. Appropriate error status codes are returned in this case, for example:


GNUTELLA/0.6 503 Too many connections
(more headers)

Note that only the HTTP-style error code should be interpreted by the software: any descriptive text provided is for display purposes only and is not standardised.

Headers

Important headers which are required or strongly recommended are detailed in the following sections.

Addressing Headers

Two important headers to send on all connections are "Remote-IP" and "Listen-IP". Both of these headers should be sent on the first transmission, meaning in the first and second header blocks in the three block exchange.

The Remote-IP header contains the IP address from which the remote host is connecting. This allows a remote host operating through some kind of network address translation system, to learn its effective external address.

The Listen-IP header contains the IP address and port number that the local host is listening for inbound TCP connections on. It should be listening for UDP datagrams on the same port. The format of this header is "IP:PORT", eg "1.2.3.4:6346".

Identification

The User-Agent header is used to identify the client software operating on the sending node. It should be sent on the first transmission, meaning in the first and second header blocks in the three block exchange. Note that this is a descriptive string that often includes a version number, and is not a "vendor code" as described elsewhere.

Content Type (Protocol)

The Accept and Content-Type header exchange is used to negotiate the data protocol that will be used in the connection, in this case Gnutella2. The Gnutella2 content type is "application/x-gnutella2", and this exchange follows standard HTTP rules for negotiating content type.

The first step is to advertise support for the content type (Gnutella2) in the first header block with "Accept: application/x-gnutella2". The responding node will then indicate that it will send Gnutella2 content with "Content-Type: application/x-gnutella2", and that it also supports Gnutella2 with "Accept: application/x-gnutella2". The initiating host then confirms that it will be sending Gnutella2 with "Content- Type: application/x-gnutella2" in the third header block. For more information on the Accept/Content-Type exchange, consult a HTTP reference.

Note that the content type negotiation process is designed to be a "one-way" process, i.e. a different content type can be negotiated for sending and receiving. However, when the gnutella2 protocol is negotiated, both channels must use the same content type. This means that a receiving node must not accept Gnutella2 if the initiator did not advertise support for it, and if at the end of the handshake, bidirectional Gnutella2 was not negotiated, the connection should be terminated.

Node State Negotiation

There are two node types in a Gnutella2 peer to peer network, a hub and a leaf, as described in Node Types and Responsibilities. During the initial handshake, the two parties must exchange their current node type and advise of their capabilities, negotiating the node types they will adopt when the connection completes, and indeed whether it should complete at all.

As the handshake sequence is compatible with Gnutella1, the headers involved in negotiating node types should not be identical to those used to negotiate Gnutella1 to minimize network pollution. states:


X-Hub: [True|False]
X-Hub-Needed: [True|False]

Both headers contain a Boolean value, "true" or "false", case insensitive.

The X-Hub header indicates whether the transmitting node is currently operating as a hub. Hub nodes will send "X-Hub: True" while leaf nodes will send "X-Hub: False".

The X-Hub-Needed header indicates whether the transmitting node would like (and allow) the receiver to be a hub. A hub which sees no need for additional hubs in its area of the network, will send "X-Hub-Needed: False", indicating to the connecting node that it must operate in leaf mode if it wishes to connect. A hub which sees a need for additional hubs, will send "X-Hub-Needed: True", indicating that the receiving node should become a hub, if it is capable of doing so. A leaf may send "X-Hub-Needed: True" to indicate that it is seeking a connection to a hub.

The X-Hub header should be sent on all three of the header blocks to indicate the current intended state of the node, while the X-Hub-Needed header should be sent in the first two header blocks only, to indicate the desired status of the receiver. If the nodes cannot agree on a satisfactory arrangement, the connection will be terminated at or prior to the third header block with an appropriate message, for example:


GNUTELLA/0.6 503 Too many hub connections
GNUTELLA/0.6 503 Too many leaves
GNUTELLA/0.6 503 I have leaves, can't downgrade to leaf mode
GNUTELLA/0.6 503 Leaf mode disabled

- NOTE: because X-Ultrapeer and X-Ultrapeer-Needed has been used on Gnutella2 for a longtime, there should be some backward compatibility code to comunicate with older nodes until they disappear.

Hub Address Exchange

It is desirable for connecting nodes to exchange the node addresses of other hubs on the network to facilitate rapid connection. The Gnutella2 protocol includes highly efficient methods to share hub addresses with peers once connected, but for a node trying to connect and encountering only "full" hubs, learning new hubs to try is helpful.

The X-Try-Hubs header was developed for this purpose. It contains a comma separated list of hub node addresses and ports, along with a timestamp recording the time the hub was last seen. For example:


X-Try-Hubs: 1.2.3.4:6346 2007-01-10T23:59Z, [..more..]

Hub addresses should not be sent unless the transmitter has reasonable knowledge of the hub's existence and the timestamp is not too old.

Compression Negotiation and Encoding

The Gnutella2 architecture makes widespread use of "deflate" compression, due to its high availability and ease of integration. Support of compressed TCP links is not a requirement in the Gnutella2 standard, however, it is strongly recommended.

Deflate compression of a TCP link is negotiated with the pair standard HTTP headers "Accept-Encoding" and "Content-Encoding". For example:

Header block one (initiator):


Accept-Encoding: deflate

Header block two (receiver):


Accept-Encoding: deflate
Content-Encoding: deflate

Header block three (initiator):


Content-Encoding: deflate

In this example, the initiator advertises support for receiving a deflated connection. The receiver then indicates that it too supports receiving a deflated connection, and that it intends to transmit deflated data. Finally, the initiator upon noting that the receiver supports deflate, indicates that it too will transmit deflated data.

Note that unlike the content-type / protocol negotiation, deflated encoding can be applied on either incoming, outgoing, or both channels of a connection.

For performance reasons, nodes should consider whether they can afford to support additional deflated connections before advertising support for them, or agreeing to provide a deflated data stream. In the Gnutella2 network topology, all links benefit from compression, except the leaf to hub channel of the leaf/hub link. Exempting this channel from compression, saves the leaf and more importantly, the hub, a considerable CPU and RAM investment.

Post Handshake Communication

After the third and final header block has been received by the initiator, subsequent communication over the TCP stream occurs in the negotiated protocol, with the negotiated encoding. This means that while the handshake sequence was backwards compatible with Gnutella1, after Gnutella2 support has been negotiated, all subsequent communication occurs in the Gnutella2 common protocol - an entirely new system not backwards compatible with any other protocol.