Gnutella2 - User contributions [en]

CRAWLR

2005-09-08T10:38:28Z

Spooky: /* Receiving */

[[CRAWLA|<< CRAWLA]] | [[HAW|HAW >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLR - Crawler Request =

The crawler request packet is sent by a Gnutella2 crawler in order to gather information about the Gnutella2 network.

== Sending ==

The /CRAWLR packet ... TODO

== Receiving ==

Upon receiving a /CRAWLR packet a [[CRAWLA|/CAWLA]] packet containing the requested information should be sent.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLR packet has the following child packets defined:

* /CRAWLR/RLEAF - Request leaf information
* /CRAWLR/RNAME - Request Nickname
* /CRAWLR/RGPS - Request GPS data
* /CRAWLR/REXT - Request version information

CRAWLA

2005-09-07T12:21:19Z

Spooky: /* Children */

[[CRAWLR|CRAWLR >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLA - Crawler Answer =

The crawl answer packet is the response to a [[CRAWLR|crawl request packet]]. It contains information about the current node for a Gnutella2 crawler.

== Sending ==

The /CRAWLA packet ... TODO

== Receiving ==

A /CRAWLA packet should only be received by a crawler in response to a [[CRAWLR|/CRAWLR]] packet, which should use it to gather information about the Gnutella2 network.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLA packet has the following child packets defined:

* /CRAWLA/SELF - Vendor Code and Version
* /CRAWLA/NA - Network Address
* /CRAWLA/HS - Hub Status
* /CRAWLA/HUB - Only present if the host is a hub
* /CRAWLA/LEAF - Only present if the host is a leaf
* /CRAWLA/GPS - GPS information
* /CRAWLA/NH - Neighbouring Hub

KHL

2005-09-07T09:24:56Z

Spooky: navi

[[HAW|<< HAW]] | [[LNI|LNI >>]] | [[Root_packets_index|Packet Index]]

= /KHL =

The known hub list packet is used to exchange a list of known Gnutella2 hubs with
neighbouring nodes. It is exchanged regularly over all TCP connections, between
hubs and between hubs and leaves (and vice versa). It enables a completely up to
date picture of the local hub cluster, and the gradual propagation of cached hub
records.

== Sending ==

This packet should be sent through TCP links at regular intervals. One minute is
suggested for busy links, leaves may send to hubs less often. Note that unlike /LNI,
/KHL must be sent regularly even if the information does not change in order to
refresh timestamps and keep time-sorted caches in the correct order.

== Receiving ==

Upon receiving a known hub list, the detailed hubs should be stored in appropriate
data structures. Neighbouring hubs must be retained in the hub cluster records,
while cached hubs should be merged with the local accessible hub cache.

== Payload ==

This packet has no payload at the current time.

== Children ==

This packet has several child packet types defined at the current time:

* /KHL/TS - Timestamp
* /KHL/NH - Neighbouring Hub
* /KHL/CH - Cached Hub

== /KHL/TS - Timestamp ==

The /KHL/TS child provides a timestamp representing the current universal time at
the sending node. This can be used as a reference when considering other
timestamps in the packet, allowing them to be adjusted to eliminate differences
between the time setting on the local and remote node.

=== Sending ===

This child is optional but recommended.

=== Payload ===

A 32-bit integer representing the current UNIX time, or time (NULL).

=== Children ===

This packet has no known children at the current time.

== /KHL/NH - Neighbouring Hub ==

This child packet represents a neighbouring hub.

=== Sending ===

One should be included for each Gnutella2 neighbour hub, with the possible
exception of the one to which this packet is being sent.

=== Payload ===

The node address of the hub. See datatypes for more information.

=== Children ===

This packet has several child packet types defined at the current time:

* /KHL/NH/GU - GUID
* /KHL/NH/V - Vendor Code
* /KHL/NH/LS - Library Statistics
* /KHL/NH/HS - Hub Status

=== /KHL/NH/GU - GUID ===

The /KHL/NH/GU child packet specifies the globally unique identifier of the hub.

==== Sending ====

This child is optional.

==== Payload ====

The 16 byte globally unique node identifier of the hub.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/V - Vendor Code ===

The /KHL/NH/V child packet specifies the vendor code of the software operating the
hub.

==== Sending ====

This child is optional.

==== Payload ====

A four byte vendor code.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/LS - Library Statistics ===

The /KHL/NH/LS packet provides information about the content library of the hub
and its connected leaves.

==== Sending ====

This child is optional.

==== Payload ====

Two 32-bit integers representing the number of files, and the total KB of available
content in the hub's own library and the libraries of its connected leaves. The
payload may grow beyond 8 bytes in the future.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/HS - Hub Status ===

The /KHL/NH/HS packet contains the status of the hub.

==== Sending ====

This child is optional.

==== Payload ====

Two 16-bit integers representing the current leaf count and the maximum leaf count
respectively. This packet may grow beyond 4 bytes in the future.

==== Children ====

This packet has no known children at the current time.

== /KHL/CH - Cached Hub ==

This child packet represents a hub from the hub cache.

=== Sending ===

An arbitrary number of these children should be included, constrained by space and
the number of available cached hubs. A selection of freshly time stamped but
previously unknown hubs should be included, and the list should not include hubs in
the local cluster.

=== Payload ===

The node address of the hub, followed by the time this hub was last seen as a 32-bit
timestamp. The last seen time can be adjusted based on the difference between the
/KHL/TS timestamp and the local time. Note that the node address is of variable
length, so it is important to consider the last 4 bytes as the timestamp rather than
assuming a fixed length node address.

=== Children ===

This packet has several child packet types defined at the current time:

* /KHL/CH/GU - GUID
* /KHL/CH/V - Vendor Code
* /KHL/CH/LS - Library Statistics
* /KHL/CH/HS - Hub Status

This extra information is considered less relevant for a cached hub because it is
unlikely to be up to date enough to be useful.

=== /KHL/CH/GU - GUID ===

The /KHL/CH/GU child packet specifies the globally unique identifier of the hub.

==== Sending ====

This child is optional.

==== Payload ====

The 16 byte globally unique node identifier of the hub.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/V - Vendor Code ===

The /KHL/CH/V child packet specifies the vendor code of the software operating the
hub.

==== Sending ====

This child is optional.

==== Payload ====

A four byte vendor code.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/LS - Library Statistics ===

The /KHL/NH/LS packet provides information about the content library of the hub
and its connected leaves.

==== Sending ====

This child is optional.

==== Payload ====

Two 32-bit integers representing the number of files, and the total KB of available
content in the hub's own library and the libraries of its connected leaves. The
payload may grow beyond 8 bytes in the future.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/HS - Hub Status ===

The /KHL/CH/HS packet contains the status of the hub.

==== Sending ====

This child is optional.

==== Payload ====

Two 16-bit integers representing the current leaf count and the maximum leaf count
respectively. This packet may grow beyond 4 bytes in the future.

==== Children ====

This packet has no known children at the current time.

HAW

2005-09-06T01:01:57Z

Spooky: initial version

[[CRAWLR|<< CRAWLR]] | [[KHL|KHL >>]] | [[Root_packets_index|Packet Index]]

= /HAW - Hub Advertising Walker =

The HAW packet ...

== Sending ==

The /HAW packet ... TODO

== Receiving ==

...

== Payload ==

A 16 bit GUID

== Children ==

The /HAW packet has the following child packets defined:

* /HAW/NA - Network Address
* /HAW/HS - Hub Status
* /HAW/V - Vendor Code

CRAWLA

2005-09-06T00:47:07Z

Spooky: /* Children */

[[CRAWLR|CRAWLR >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLA - Crawler Answer =

The crawl answer packet is the response to a [[CRAWLR|crawl request packet]]. It contains information about the current node for a Gnutella2 crawler.

== Sending ==

The /CRAWLA packet ... TODO

== Receiving ==

A /CRAWLA packet should only be received by a crawler in response to a [[CRAWLR|/CRAWLR]] packet, which should use it to gather information about the Gnutella2 network.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLA packet has the following child packets defined:

* /CRAWLA/SELF - Vendor Code and Version
* /CRAWLA/NA - Network Address
* /CRAWLA/HS - Hub Status
* /CRAWLA/NAME - User Nickname
* /CRAWLA/V - Vendor Code
* /CRAWLA/CV - Current Version
* /CRAWLA/HUB - Only present if the host is a hub
* /CRAWLA/LEAF - Only present if the host is a leaf
* /CRAWLA/GPS - GPS information
* /CRAWLA/NH - Neighbouring Hub

CRAWLR

2005-09-06T00:46:12Z

Spooky: initial version

[[CRAWLA|<< CRAWLA]] | [[HAW|HAW >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLR - Crawler Request =

The crawler request packet is sent by a Gnutella2 crawler in order to gather information about the Gnutella2 network.

== Sending ==

The /CRAWLR packet ... TODO

== Receiving ==

Upon receiving a /CRAWLR packet a [[CRAWLA|/CAWLA]] packet conating the requested information should be sent.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLR packet has the following child packets defined:

* /CRAWLR/RLEAF - Request leaf information
* /CRAWLR/RNAME - Request Nickname
* /CRAWLR/RGPS - Request GPS data
* /CRAWLR/REXT - Request version information

CRAWLA

2005-09-06T00:40:06Z

Spooky:

[[CRAWLR|CRAWLR >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLA - Crawler Answer =

The crawl answer packet is the response to a [[CRAWLR|crawl request packet]]. It contains information about the current node for a Gnutella2 crawler.

== Sending ==

The /CRAWLA packet ... TODO

== Receiving ==

A /CRAWLA packet should only be received by a crawler in response to a [[CRAWLR|/CRAWLR]] packet, which should use it to gather information about the Gnutella2 network.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLA packet has the following child packets defined:

* /CRAWLA/SELF - Vendor Code and Version
* /CRAWLA/NA - Network Address
* /CRAWLA/HS - Hub Status
* /CRAWLA/NAME - User Nickname
* /CRAWLA/V - Vendor Code
* /CRAWLA/CV - Current Version
* /CRAWLA/HUB - Only present if the host is a hub
* /CRAWLA/LEAF - Only present if the host is a leaf
* /CRAWLA/GPS -
* /CRAWLA/NH - Neighbour Host

CRAWLA

2005-09-06T00:37:50Z

Spooky: /* Sending */

[[CRAWLR|CRAWLR >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLA - Local Node Information =

The crawl answer packet is the response to a [[CRAWLR|crawl request packet]]. It contains information about the current node for a Gnutella2 crawler.

== Sending ==

The /CRAWLA packet ... TODO

== Receiving ==

A /CRAWLA packet should only be received by a crawler in response to a [[CRAWLR|/CRAWLR]] packet, which should use it to gather information about the Gnutella2 network.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLA packet has the following child packets defined:

* /CRAWLA/SELF - Vendor Code and Version
* /CRAWLA/NA - Network Address
* /CRAWLA/HS - Hub Status
* /CRAWLA/NAME - User Nickname
* /CRAWLA/V - Vendor Code
* /CRAWLA/CV - Current Version
* /CRAWLA/HUB - Only present if the host is a hub
* /CRAWLA/LEAF - Only present if the host is a leaf
* /CRAWLA/GPS -
* /CRAWLA/NH - Neighbour Host

CRAWLA

2005-09-06T00:36:24Z

Spooky: initial version

[[CRAWLR|CRAWLR >>]] | [[Root_packets_index|Packet Index]]

= /CRAWLA - Local Node Information =

The crawl answer packet is the response to a [[CRAWLR|crawl request packet]]. It contains information about the current node for a Gnutella2 crawler.

== Sending ==

The /LNI packet should be sent to a TCP neighbour upon connection, and at regular
intervals following that if the information within it has changed. A minimum update
time of one minute is recommended.

== Receiving ==

A /CRAWLA packet should only be received by a crawler in response to a [[CRAWLR|/CRAWLR]] packet, which should use it to gather information about the Gnutella2 network.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /CRAWLA packet has the following child packets defined:

* /CRAWLA/SELF - Vendor Code and Version
* /CRAWLA/NA - Network Address
* /CRAWLA/HS - Hub Status
* /CRAWLA/NAME - User Nickname
* /CRAWLA/V - Vendor Code
* /CRAWLA/CV - Current Version
* /CRAWLA/HUB - Only present if the host is a hub
* /CRAWLA/LEAF - Only present if the host is a leaf
* /CRAWLA/GPS -
* /CRAWLA/NH - Neighbour Host

Template:Root packet list

2005-09-06T00:15:37Z

Spooky: haw packet added

* /[[CRAWLA]] - Crawler Answer
* /[[CRAWLR]] - Crawler Request
* /[[HAW]] - Hub Advertisement Walker
* /[[KHL]] - Known Hub List
* /[[LNI]] - Local Node Information
* /[[PI]] - Ping
* /[[PO]] - Pong
* /[[PUSH]] - Push
* /[[QKA]] - Query Key Acknowledgement
* /[[QKR]] - Query Key Request
* /[[Q2]] - Gnutella2 Query
* /[[QA]] - Query Acknowledgement
* /[[QH2]] - Gnutella2 Query Hit
* /[[QHT]] - Query Hash Table
* /[[UPROC]] - User Profile Challenge
* /[[UPROD]] - User Profile Delivery

Template:Root packet list

2005-09-06T00:08:23Z

Spooky: crawler packets added to list

* /[[CRAWLA]] - Crawler Answer
* /[[CRAWLR]] - Crawler Request
* /[[KHL]] - Known Hub List
* /[[LNI]] - Local Node Information
* /[[PI]] - Ping
* /[[PO]] - Pong
* /[[PUSH]] - Push
* /[[QKA]] - Query Key Acknowledgement
* /[[QKR]] - Query Key Request
* /[[Q2]] - Gnutella2 Query
* /[[QA]] - Query Acknowledgement
* /[[QH2]] - Gnutella2 Query Hit
* /[[QHT]] - Query Hash Table
* /[[UPROC]] - User Profile Challenge
* /[[UPROD]] - User Profile Delivery

QKR

2005-09-05T23:40:25Z

Spooky: navi

[[QKA|<< QKA]] | [[Q2|Q2 >>]] | [[Root_packets_index|Packet Index]]

= /QKR - Query Key Request =

The query key request asks a destination hub to provide a query key that the
requesting node can use to securely perform queries. This mechanism ensures that
the network cannot be used to direct unsolicited packets to external hosts.

== Sending ==

A query key request should be sent to a hub via the connectionless transport if a key
is not already available for that hub for the local node address, or if a previously
acquired key no longer works or has expired. There is no reason to request a query
key via a TCP link.

== Receiving ==

Upon receiving a query key request, a hub should respond with a query key answer
(/[[QKA]]) packet if it is willing to supply a query key. The answer packet should be
dispatched to the return address indicated in the request packet only, and the
allocated key should be locked to that address.

== Payload ==

This packet has no payload at the current time.

== Children ==

The QKR packet has one child packet defined at the current time:

* /QKR/RNA

== /QKR/RNA - Requesting Node Address ==

A query key may be requested for a node address other than that of the sender, for
example when a UDP-firewalled leaf node wishes to acquire a query key for a remote
hub that it will route through a locally connected hub. In this case it must request a
query key for the node address of a connected hub.

=== Sending ===

This child is required to ensure that the sender knows its address.

=== Payload ===

The network address of the requesting node. See [[Datatypes]] for more information.

=== Children ===

This packet has no known children at the current time.

QKA

2005-09-05T23:39:24Z

Spooky: navi

[[PUSH|<< PUSH]] | [[QKR|QKR >>]] | [[Root_packets_index|Packet Index]]

= /QKA - Query Key Answer =

The query key answer packet is sent in response to a [[QKR|query key request]] or in the
event that a query with an incorrect or omitted query key is received. It advises the
receiver of the correct query key to be used when contacting the sending hub to
execute a remote cluster query.

== Sending ==

The query key answer packet should be sent by a hub when a query key has been
requested. It should contain a key unique to the local hub and the requesting node
address supplied in the request, and should be sent to that address only.
The query key answer packet may also be sent upon receiving a query with a
missing or invalid query key. In this case no query should be performed.

== Receiving ==

Upon receiving a query key answer packet, the supplied query key should be stored
for later use. If the answer packet contains a sending node address (/QKA/SNA) child
which does not match the local node address, it may be forwarded to a connected
leaf node via TCP with a matching address. In the forwarding case, a query node
address (/QKA/QNA) child is added prior to forwarding to retain the remote (queried)
hub's address.

== Payload ==

This packet has no payload at the current time.

== Children ==

The QKA packet has three child packets defined at the current time:

* /QKA/QK
* /QKA/SNA
* /QKA/QNA

== /QKA/QK - Query Key ==

The /QKA/QK child contains the query key that was requested.

=== Sending ===

This child is required if a key is being issued. Sending a /QKA without a /QKA/QK
indicates that the hub does not wish to authorise the requesting node to query it or
its cluster.

=== Payload ===

A 32-bit query key which is unique to the hub and the sending node address.

=== Children ===

This packet has no known children at the current time.

== /QKA/SNA - Sending Node Address ==

This child specifies the node address for which the key has been issued. This is not
necessarily the node that physically transmitted the key request or the query.

=== Sending ===

This child is required.

=== Payload ===

The node address (with port number optional) of the node which requested the
query key.

=== Children ===

This packet has no known children at the current time.

== /QKA/QNA - Queried Node Address ==

If this query key answer packet has been forwarded to a firewalled leaf from a
connected hub, a /QKA/QNA child will have been added to indicate the address of
the remote hub.

=== Sending ===

This child should be added by a hub when relaying a received query key answer on
to a leaf via TCP.

=== Receiving ===

If this child is present, the query key answer should be treated as if it originated
from the stored address rather than the TCP neighbour.

=== Payload ===

The node address (with port number optional) of the remote hub.

=== Children ===

This packet has no known children at the current time.

PUSH

2005-09-05T23:38:34Z

Spooky: navi

[[PO|<< PO]] | [[QKA|QKA >>]] | [[Root_packets_index|Packet Index]]

= /PUSH =

The push packet is used to request that a node initiate a TCP connection to a
specified node address. It is useful where the receiving node is unable to accept TCP
connections naturally.

For more information about when to use PUSH packets: [[Node_Route_Cache_and_Addressed_Packet_Forwarding#Reverse_Connection_.28Push.29_Requests|Reverse Connection (Push) Requests]]

== Sending ==

An addressed /PUSH packet should be sent to a hub with direct or proxy access to
the destination leaf node. It can be sent directly by a hub to one of its leaves without
the need for addressing.

== Receiving ==

Upon receiving a /PUSH packet, the local node should initiate a TCP connection to
the elected node address and issue a "push handshake", documented below.

== Payload ==

The network address of the node to contact, including of course the port number.

== Children ==

This packet has no known children at the current time, however it is often used with
the generic addressing child packet "/?/TO" to indicate that it should be forwarded to
a destination.

PO

2005-09-05T23:37:53Z

Spooky: navi

[[PI|<< PI]] | [[PUSH|PUSH >>]] | [[Root_packets_index|Packet Index]]

= /PO - Pong =

The pong packet serves as a reply to a one or two hop ping (/PI) packet. It indicates
that the pinged node received and processed the ping, and in the case of UDP, that
the local node was able to received UDP from the sender.

== Sending ==

Send a pong packet in response to a direct or relayed ping request.

== Receiving ==

No action is required in response to a pong packet. If the packet contains a
/PO/RELAY marker child, test the sender's network address against the list of active
TCP connections and if there is no match, consider the local node able to receive
UDP.

== Payload ==

This packet has no payload at the current time.

== Children ==

The ping packet has one child packet type defined at the current time:
* /PO/RELAY

== /PO/RELAY - Relayed Pong Marker ==

If a /PO packet contains a /PO/RELAY child marker, the pong was sent in response to
a relayed ping.

=== Sending ===

The /PO/RELAY child marker should be added to a /PO if the pong is being sent in
response to a relayed ping.

=== Receiving ===

No specific action is required for a relayed pong.

=== Payload ===

This packet has no payload at the current time.

=== Children ===

This packet has no known children at the current time.

PI

2005-09-05T23:37:02Z

Spooky: nav

[[LNI|<< LNI]] | [[PO|PO >>]] | [[Root_packets_index|Packet Index]]

= /PI - Ping =

The ping packet is used to verify the presence of the addressed node, testing that:

* The destination node is online
* The destination node can receive the transmission
* The local node can receive the reply

Aside from performing a "keep-alive" function on TCP links, pings can be used to
solicit inbound UDP datagrams to test the local node's ability to receive UDP. Many
NAT systems will route UDP traffic from nodes with whom a persistent TCP stream
has been established, so it is necessary to test inbound traffic from nodes with which
there is no TCP connection. This is achieved by sending a ping with a /PI/UDP packet
to a connected hub via TCP, which will forward the request to its neighbours who
then reply to the originator via UDP.

== Sending ==

On a TCP stream connection, if one end has not received a valid packet from the
other after an internally determined period of time, it has the option of sending a
keep-alive ping to verify the other end's presence. If a pong is not received in a
timeout period it may close the connection.

If a node needs to verify that it is able to receive UDP datagrams from nodes with
which it does not have a TCP stream connection, it may send a ping packet with a
/PI/UDP child packet. If it then receives a /PO (pong) packet with a /PO/RELAY child
tag and there is no TCP connection to the sender, it can assume it is able to receive
UDP.

== Receiving ==

Upon receiving a keep-alive ping, a node should respond with a keep-alive pong
immediately.

If the ping was received from a TCP neighbour and contains a /PI/UDP child packet,
but no /PI/RELAY tag, a /PI/RELAY tag should be added and the packet forwarded to
all neighbouring hubs. If a /PI/RELAY tag was present, the node should send a /PO
packet with a /PO/RELAY tag to the UDP address identified in the /PI/UDP child
packet.

== Payload ==

This packet has no payload at the current time.

== Children ==

The ping packet has two child packets defined at the current time:

* /PI/RELAY
* /PI/UDP

== /PI/RELAY - Relayed Ping Marker ==

If a /PI packet contains a /PI/RELAY child marker, the ping has been relayed for a
third party. The receiving node should reply to the original party rather than the
sender. The original party should be contacted via UDP with the address contained in
the /PI/UDP child. If the /PI/UDP child is absent, the packet is invalid.

=== Sending ===

The /PI/RELAY child should be added to a /PI packet when relaying a ping to hub
neighbours at the request of a leaf.

=== Receiving ===

Upon receiving a /PI packet with a /PI/RELAY tag, a /PO (pong) packet should be
sent via UDP to the node identified in the /PI/UDP child.

=== Payload ===

This packet has no payload at the current time.

=== Children ===

This packet has no known children at the current time.

== /PI/UDP - Ping Response Address ==

The /PI/UDP child packet specifies the return address for a relay able or relayed
ping.

=== Sending ===

Add a /PI/UDP child packet to a /PI packet to request a relayed "two hop" ping.

=== Receiving ===

The /PI/UDP child packet indicates that a /PI packet needs to be relayed.

=== Payload ===

The /PI/UDP packet contains an endpoint address.

=== Children ===

This packet has no known children at the current time.

Main Page

2005-07-04T00:30:53Z

Spooky: link changed

<table width="100%" ><tr>
<td align="left">
== Gnutella2 Developer Network ==
Welcome to the Gnutella2 Developer Network.
The Gnutella2 Developer Network (or G2DN) is a convenient central location for discussing, developing and documenting all aspects of Gnutella2. Like the Gnutella2 platform itself, this website is designed to evolve through community support and participation. Everyone is free to contribute discussion, proposal documents, developer resources or any other relevant material.

<p>Although the G2DN is primarily geared toward developers and technical enthusiasts, users of Gnutella2-enabled software are also welcome to participate. Those who are new to Gnutella2 may wish to check out some of the [[Gnutella2-powered software]].</p></td>
<td width="20px"></td>
<td align="right" valign="bottom" width="200px">
__TOC__
</td></tr>
</table>

==News==
* ''10th, April, 2005'' - '''Gnutella2 website re-opened'''
<p>After server problems we're proud to re-launch the Gnutella2 website. It is now based on MediaWiki (like Wikipedia). The forums are gone, but we now have a [[mailing list]] for Gnutella2 related discussion. Feel free to contribute to the wiki after reading [[Gnutella2:Community Portal]] to learn about editing policy.</p>

==Mailing List==
To allow communication among developers of Gnutella2 clients, there exists a [[mailing list]]. You can access it too via regular news reader at the news-to-mail gateway [http://www.gmane.org Gmane] (list page [http://dir.gmane.org/gmane.network.gnutella2.devel here]).

== The Gnutella 2 Network==
# [[Gnutella2|What is Gnutella2]]
# [[Gnutella2 Standard]]
# [[Network Architecture]]
# [[Node Types and Responsibilities]]
# [[Background|Background / Archive]]

== The Protocol==
# [[TCP Stream Connection and Handshaking]]
# [[UDP Transceiver]]
# [[Packet Structure]]
# [[Datatypes]]
# [[Basic Network Maintenance]]
# [[Known Hub Cache and Hub Cluster Cache]]
# [[Node Route Cache and Addressed Packet Forwarding]]
# [[Query Hash Tables]]
# [[Object Search Mechanism]]
# [[Search Security]]
# [[Search Description]]
# [[Search Acknowledgement]]
# [[Search Results]]
# [[Simple Query Language and Metadata]]
# [[Server for Uploading]]
# [[Client for Downloading]]
# [[User Profile Challenge and Delivery]]

== Root Packets ==
{{Root packets}}

== Proposals==
* [[Active Source Exchange]] (ASE) by Kevin O'Toole

*HTTPCache and G3Net proposal (Open for [[User_talk:F5inet|Discussion]])

== Additional Information==
Since Gnutella2 somehow evolved from the original Gnutella protocol you might find the following links useful:
* [http://www.the-gdf.org The GDF Wiki]
* [http://groups.yahoo.com/group/the_gdf/ Gnutella Developer forum]
* [http://rfc-gnutella.sourceforge.net/ RFC Gnutella]
* [http://draketo.de/inhalt/krude-ideen/gnufu_en.html Gnutella for Users (GnuFU)]
* [http://crawler.instantnetworks.net/ Gnutella2 Crawler]

Mailing list

2005-05-20T21:15:53Z

Spooky: Added Gmane info

==G2DN Mailing List==

This is the mailing list for communication between Gnutella2 developers and other interested people. This is not for help on how to use Gnutella2 applications.

'''Subscribe''' to the list by sending an email to g2-dev-request@gnutella2.com with 'subscribe' in the subject line. Alternatively subscribe here: http://www.gnutella2.com/mailman/listinfo/g2-dev_gnutella2.com

To post on the mailinglist send an email to g2-dev@gnutella2.com

Message archives: http://www.gnutella2.com/pipermail/g2-dev_gnutella2.com/

===Gmane===
The list can also be accessed through the news-to-mail gateway [http://www.gmane.org Gmane]:<br>
[http://dir.gmane.org/gmane.network.gnutella2.devel G2DN at Gmane]

Talk:Gnutella2 Standard

2005-05-13T16:31:20Z

Spooky:

May 12, 2005:

The article says:

All applications making use of Gnutella2 technology for any application class MUST IMPLEMENT the following core features:

Bidirectional TCP stream connections (stream compression OPTIONAL)
.
.
.
Gnutella2 Standard for File Sharing

--

I don't think every gnutella2 application ought to necessarily implement the Gnutella2 Standard for file sharing. Only G2 filesharing applications must implement the Gnutella2 Standard for file sharing. The way it is worded, every G2 application must also be a file sharing app.

Perhaps someone could edit this page to fix this apparent mistake.

Hm you are probably right. So feel free to edit the page yourself. That's how a wiki works.

QA

2005-04-05T19:11:39Z

Spooky: typo

= /QA - Query Acknowledgement =

The query acknowledgement packet is used to inform a search client that a target
hub has received its [[Q2|query]] and is processing it. It also provides information for the
search client's hub cache, expanding knowledge of the network and ensuring hubs
are not searched more than once in a given query.

== Sending ==

Send a query acknowledgement packet only when operating in hub mode, and only
when the query was not received from a hub via a TCP link. Acknowledgements
should only be sent after verifying the query key authentication.

== Receiving ==

When a query acknowledgement is received, update the local hub cache as
necessary:

* Hubs which have been searched should not be sent this query again
* New suggested hubs should be added and refreshed
* Any "back-off" time should be recorded and honoured.

== Payload ==

A GUID identifying the query being acknowledged.

== Children ==

This packet has several child packet types defined at the current time:

* /QA/TS - Timestamp
* /QA/D - "Done" or Completed Hub
* /QA/S - "Search" or New Hub
* /QA/RA - Retry After
* /QA/FR - From Address

== /QA/TS - Timestamp ==

The /QA/TS child provides a timestamp representing the current universal time at the
sending node. This can be used as a reference when considering other timestamps in
the packet, allowing them to be adjusted to eliminate differences between the time
setting on the local and remote node.

=== Sending ===

This child is optional but recommended.

=== Payload ===

A 32-bit integer representing the current UNIX time, or time(NULL).

=== Children ===

This packet has no known children at the current time.

== /QA/D - Done Hub ==

References a hub which has now been queried, and should not be queried again.

=== Sending ===

Send a /QA/D child for the local hub and each neighbouring hub, whether or not the
hub has actually forwarded the query. If a hub has not forwarded the query, it was
still effectively searched as the QHT indicated it could not field a matching object.

=== Receiving ===

Upon receiving a /QA/D child, add the hub to the cache, freshen its timestamp and
mark it as queried in this search. Do not query this node again for several minutes.

=== Payload ===

A network/node address followed by a 16-bit leaf count.

=== Children ===

This packet has no known children at the current time.

== /QA/S - Search Hub ==

References a hub which was not searched in this operation, but could be searched
later if it has not already been touched.

=== Sending ===

Send a /QA/S child for every unique hub in the local hub cluster which is not a direct
neighbour. Optionally select some recent cached hubs and send /QA/S children for
them also, to make up at least 10 in total.

=== Receiving ===

Upon receiving a /QA/S child, add the hub to the cache and/or freshen its
timestamp.

=== Payload ===

A network/node address, optionally followed by a 32-bit last-seen timestamp. This
makes detecting the address family slightly harder, but the compactness is
worthwhile. Timestamps are omitted for hubs in the local hub cluster, which should
make up the majority.

=== Children ===

This packet has no known children at the current time.

== /QA/RA - Retry After ==

Advises the search client that the hub does not wish to be contacted again for a
specified period of time. Queries earlier than this time may result in a longer-term
ban.

=== Sending ===

Send this child, with or without executing the search if the hub's resources are
becoming saturated. It provides an indirect means of flow control.

=== Receiving ===

Upon receiving a /QA/RA child, record the retry after interval and do not send any
requests to the hub until the period has expired.

=== Payload ===

A 16 bit or 32 bit integer specifying the delay in seconds until the hub may be
contacted again.

=== Children ===

This packet has no known children at the current time.

== /QA/FR - From Address ==

This child is used when a query acknowledgement is forwarded from a proxy hub to
a firewalled leaf node. It advises the leaf of the original sender.

=== Sending ===

When a hub receives a /QA packet for a search originated by one of its leaves, it
should forward the /QA to the appropriate leaf after adding a /QA/FR child with the
remote hub's network address.

=== Receiving ===

If this child is present and the /QA packet was received via TCP from a local hub, the
node address within should be used as the remote hub's node address. This
information is often reflected in the first /QA/D child, however this is not always
present.

=== Payload ===

A network address which may omit the port number.

=== Children ===

This packet has no known children at the current time.

Node Route Cache and Addressed Packet Forwarding

2005-04-05T18:39:07Z

Spooky:

[[Known Hub Cache and Hub Cluster Cache|<< Known Hub Cache and Hub Cluster Cache]] | [[Query Hash Tables|Query Hash Tables >>]] | [[Main_Page|Main Page]]

__TOC__
== Introduction ==

Each Gnutella2 node must maintain a dynamic node route cache to map node GUIDs
to appropriate destinations. The route cache is consulted when a packet needs to be
dispatched to, or toward a GUID-addressed node. GUID-addressing is used over
network-addressing in a number of situations.

== Data ==

Each entry in a node route cache will have:

* The GUID of the target node
* The local TCP connection providing the best route to the node, or
* A UDP endpoint for the node or the best route to the node

== Performance ==

A route cache needs to be addressable by GUID, and must implement a refresh
mechanism to store routes for an appropriate amount of time based upon route hits.
Many schemes exist to engineer efficient lookup systems, such as hash tables, twotable
exchanges and balanced trees.

The route cache is similar to the several GUID mapping caches involved in old
Gnutella applications, with two key differences:

* Each entry may map to a local address or a UDP endpoint
* There is only one unified route cache for all purposes

== Applications ==

GUIDs are used to identify virtual entities within the network, such as nodes (node
GUIDs), searches (search GUIDs) and other transactional requests. The GUIDs
associated with these entities can then be committed to the route cache and mapped
to reflect the easiest route back to the owner of the object.

All nodes should be aware of their directly connected neighbours, and treat these
node GUIDs as a special case that need never expire.

== Addressed Packet Forwarding ==

Any Gnutella2 packet may be addressed to a particular destination node by GUID.
Upon receiving an addressed packet, it should be immediately forwarded either to
the destination, or toward it. Addressed packets should not be interpreted locally,
unless the destination address matches the local GUID.

Loops are avoided by placing restrictions upon forwarding:

* If received from a leaf via TCP, a packet may be forwarded anywhere
* If received from a hub, a packet may only be forwarded to a leaf
* If received via UDP, a packet may not be forwarded via UDP

Note that these restrictions apply only to generically addressed packets. Some packet
types have specific forwarding rules which override these. These rules allow any
node to be reached in two hops.

Packets are addressed by including a special child packet as the first child of the root
packet. The child packet is named "TO", so its full name would be "/?/TO" where ? is
the root packet name. The address packet's payload consists of a 16 byte GUID, and
it has no children defined at the current time.

== Reverse Connection (Push) Requests ==

Addressed packet forwarding can be used to deliver any valid Gnutella2
communications to leaf nodes that are unable to accept TCP or UDP traffic directly.
The packet is simply sent to one of the hubs to which the target leaf is attached, or a
hub in the same hub cluster, and the forwarding rules will allow the packet to reach
its destination.

This mechanism is used to request that a "firewalled" leaf initiate a "reverse
connection" or "call-back" to an elected address. The root packet type "/[[PUSH]]" is
used.

== Push Handshaking ==

The Gnutella2 push handshake is very simple - it simply identifies the connection as
originating from a push, and provides the GUID of the initiating node:

<pre>
<nowiki>
PUSH guid:GUIDinHEXguidINhexGUIDinHEXguidI (\r\n)
(\r\n)
</nowiki>
</pre>

Note that unlike the Gnutella1 case, no purpose-specific information is provided. The
pushed connection can now be used for any purpose a normally established TCP link
could adopt, including, but not limited to:

* Gnutella2 network connection
* Data transfer
* Personal communications
* Etc

Gnutella2 implementations are however, strongly advised to provide backward
support for the Gnutella1 push handshake, even if not supporting Gnutella1 directly.
This is because applications supporting both protocols may be unable to determine in
advance, whether the host they are pushing to is Gnutella1 or Gnutella2.
The legacy-style push handshake looks like:

<pre>
<nowiki>
GIV 0:GUIDinHEXguidINhexGUIDinHEXguidI/ (\n\n)
</nowiki>
</pre>

The leading zero and trailing slash have task-specific meanings in Gnutella1;
however Gnutella2 applications can safely ignore them and consider only the GUID.
Be sure to allow for variable length handshakes, however.

Known Hub Cache and Hub Cluster Cache

2005-04-05T18:35:12Z

Spooky:

[[Basic Network Maintenance|<< Basic Network Maintenance]] | [[Node Route Cache and Addressed Packet Forwarding|Node Route Cache and Addressed Packet Forwarding >>]] | [[Main_Page|Main Page]]

== Introduction ==

Each Gnutella2 node must maintain a non-exhaustive cache of known hubs at the
global level, and an exhaustive cache of hubs within the neighbouring hub cluster(s).

== The Known Hub Cache ==

The known hub list is used to provide connection hints to the local connection
manager, and other nodes which connect permanently or transiently. The most
recent portion of it is exchanged with neighbours regularly.

It is also used when executing a query on the network, which involves iteratively
contacting hubs representing each hub cluster and simultaneously recording the
hubs which have been accounted for and adding those newly discovered.

The known hub cache should be highly efficient, addressable by node address and
timestamp, and sorted by the last seen timestamp of each hub record. Adding fresh
hubs should push the oldest hubs from the bottom of the cache.

It is suggested that each cached hub entry store:

* Node address
* Last seen time
* Query key, if available
* [[Datatypes|GUID]], if available
* Vendor information, if desired and available
* Throttling timing for last query key request, last query, etc

Hubs whose last seen timestamp is too old should be removed from the cache, and
should certainly never be sent to other nodes.

== The Hub Cluster Cache ==

The hub cluster cache is used to maintain an up-to-date list of neighbouring hubs,
and the neighbouring hubs of neighbouring hubs. Thinking in terms of hops, it is an
exhaustive list of every hub which is 1 or 2 hops away from the local node (be the
local node a hub or a leaf). The cluster cache is really only important when operating
in hub mode, however, it can be maintained in leaf mode for informational purposes.

The hub cluster cache is used by a hub when responding to a keyed remote query
request, in the generation of a query acknowledgement packet (/[[QA]]). The /[[QA]]
packet contains a list of neighbouring hubs and a selection of second degree
neighbours.

The cluster cache is updated using information from /[[LNI]] and /[[KHL]] packets.

Search Description

2005-04-05T18:30:49Z

Spooky: Query throttle chapter added,s hould there be a detailed description of how a search works on this page ?

== Introduction ==

The Gnutella2 object search mechanism defines a powerful query descriptor capable
of specifying a number of different extensible search criteria, and requesting a subset
of information about matching objects.

/[[Q2]] - Gnutella2 Query

== Query Throttle ==
Querying the Gnutella2 network can lead to '''a lot''' of results, putting a resource strain on both the network as a whole nad the querying party itself. Therfore a client SHOULD NOT send any more queries after he has received about 100 results. The client should also not send too many query packets at the same time

Active Source Exchange

2005-04-05T18:05:16Z

Spooky:

==Active Source Exchange (ASE) 1.0==

<table width="100%" ><tr>
<td align="left">
<small>Written By: Kevin O'Toole<br>
kevo@napjunk.net<br>
Shareaza Alpha Team<br>
2003.06.14

Edited By: Jonathan C. Nilson<br>
JCNilson@myrealbox.com<br>
Shareaza Alpha Team<br>
2003.06.16</small>
</td>

<td align="right">__TOC__</td>
</tr>
</table>

===Status===
This proosal is currently not implemented by any client.

===Introduction===
This proposal addresses the current flaws in the alternate source mesh by not passing on sources that are "old" and client identification for whom to receive sources from. This does not eliminate alternate sources but should make the current system better because it will help keep the mesh (exchange) of sources clean of dead hosts. The proposal has the added benefit of reducing the effects of a Gnutella-style Denial-of-Service attack by reducing the number of bad sources a Shareaza node has, thus keeping that Shareaza node from pestering as many dead sources.

===Why is it needed?===
ASE removes the need to requery frequently to gather a list of new sources. By sharing only current sources, users will not ...

* get flooded with more dead hosts than live ones
* receive connection attempts for a long time after
* have to clean out a long list of bad hosts
* expericence hangs in the client as frequently
* attempt as many connections on a random port/dynamic IP user
* participte unwittingly in a DoS attack on another user
* requery the network as frequently
* pass a firewalled source to a firewalled client

There are still problems ASE does address but cannot completely solve, which include ...

* hostile clients passing on fake or dead hosts
* hostile clients passing on expired sources

===What's different?===
ASE changes the way a client chooses which alternate sources to pass and how long each will be active. In effect, a complying client would pass on only active sources in the mesh, each with an expiry time which should be between 10 and 15 minutes. Active sources are defined as sources which have successfully been Queued or Uploading at some time in the last expiry time block (10~15 minutes). This timeout is critical. If clients simply maintain a list, the dead hosts will take longer to populate the ASE, but will still eventually spread and the same problems will occur.

To avoid this fiasco, a client in the ASE passes the address and the timeout (at its current value) to the recipient. This timeout is only reset if the address is successfully contacted and Queued or Uploading before the timeout value expires. Therefore, unless a client sees the source in a successful queue or upload, the address will expire (no matter how many clients have received the source) at the same time. If one client does reset the expiry counter, only sources which receive the source in the ASE subsequently will still have the source. No messages are sent back to clients who have an entry that is about to expire.

If a client wishes to maintain a list of sources that have expired from the ASE, these sources shall not be passed on to subsequent hosts, with or without a timeout. At this point, the sources shall remain knowlegable to the local host alone.

===Firewalls===
To avoid the propogation of firewalled sources to firewalled clients, each source shall have a header identification indicating the source as a firewalled client which would be sent with the address and timeout. The uploader sends any sources to normal clients, while push sources are not sent to firewalled clients. IANA Reserved IP ranges will not be passed along.

===Security===
If every client played by these rules, security would be no problem, unfortunately we have to plan for the contingency that we may have hostile, buggy, or poorly-programmed clients. Keeping this system secure would perhaps be a more difficult task than implementing it. Options for achieving this would be either client identification (which can easily be spoofed) or client-to-client authentication. Without client identification or authentication, users run the same risks involved in the alternate-source mesh now.

With client identification security, a user would only receive hosts from chosen vendors (filtered by reported user-agent strings) but still send hosts to any client. This would be a simple solution to the problem if everyone obeyed the rules. Unfortunately, hostile clients such as Qtrax change their user-agent identification to match that of their downloader, which would make the system useless against such clients (fortunately, few exist).

A more complex (perhaps neglible) solution would be client-to-client authentication. Although this should completely solve the problem, it would require a large amount of work for minimal gain, and some valuable alternate sources from unsupporting clients would be lost. This authentication method would, by definition, be closed to only trusted clients, which also defeats the spirit of the Gnutella community. While it solves some problems very well, this course of action is not recommended.

===Scenario===
To properly illustrate this new system, consider the following scenario. We will assume for the sake of illustration that the expiry threshold is at 15 minutes.

ClientA, ClientB, and Client C all support ASE in its current form. ClientA and ClientB are both downloading File1. Client A knows of the following sources for File1:

<pre><nowiki>111.111.111.1 (Queued 3 of 10 Now)
111.111.111.2 (Downloading Now)
111.111.111.3 (Queued 10 minutes ago)
111.111.111.5 (Downloading Now)
192.168.0.1 (Push) (Never contacted)
0.0.0.0 (Push) (Downloaded 5 minutes ago)
</nowiki></pre>

ClientB knows of the following sources for File1:

<pre><nowiki>222.222.222.4 (Queued 20 minutes ago)
ClientA (Never contacted)</nowiki></pre>

As ClientB contacts ClientA for the file in question, it refuses to transmit the single known source it has because that source has expired from the ASE. While ClientB may choose to keep that source for its own use, it will not transmit that source to other clients, with or without an expiry timeout.

When ClientA returns the handshake, it transmits a list of ASE sources to ClientB:

<pre><nowiki>111.111.111.1; Timeout=15m
111.111.111.2; Timeout=15m
111.111.111.3; Timeout=5m
111.111.111.5; Timeout=15m; Push</nowiki></pre>

Since 0.0.0.0 is an IANA Private Range, the address is not passed to ClientB, although it is a valid source with a timeout at 10m. 192.168.0.1 is not passed because it has never been contacted.

After ClientB receives this list of sources and is sent a Busy reply by ClientA, ClientB reattempts a connection with 222.222.222.4. This connection results in a successful download. One minute has passed since the ASE list from ClientA was received by ClientB.

ClientC is firewalled, and after searching and finding ClientB as a source for File1, it attempts a connection. Since it has no sources, none are passed to ClientB, however, ClientB passes the same list of known sources to ClientC as ClientA did to ClientB. ClientB also adds its own experience by transimtting 222.222.222.4 because that source is now active and omiting 111.111.111.5 because that is a firewalled source and ClientC is firewalled, so the final list transmitted from ClientB to ClientC is as follows:

<pre><nowiki>111.111.111.1; Timeout=14m
111.111.111.2; Timeout=14m
111.111.111.3; Timeout=4m
222.222.222.4; Timeout=15m</nowiki></pre>

Please note that Timeout values are listed in minutes for the sake of human understanding and will probably be coded in seconds instead.

===Conclusion===
While this does not solve every problem of the current alternate source mesh, it would be beneficial for anyone who currently uses this system. ASE offers a simple solution to the problems with the alternate source mesh. If implemented on a wide-scale, it could reduce traffic substantially and increase file transfer efficiency tenfold.

Main Page

2005-04-05T18:02:51Z

Spooky: News added

<table width="100%" ><tr>
<td align="left">
== Gnutella2 Developer Network ==
Welcome to the Gnutella2 Developer Network
The Gnutella2 Developers' Network (or G2DN) is a convenient central location for discussing, developing and documenting all aspects of Gnutella2. Like the Gnutella2 platform itself, this website is designed to evolve through community support and participation. Everyone is free to contibute discussion, proposal documents, developer resources or any other relevant material.

Although the G2DN is primarily geared toward developers and technical enthusiests, users of Gnutella2-enabled software are also welcome to participate. Those who are new to Gnutella2 may wish to check out some of the [[Gnutella2-powered software]].</td>
<td width="20px"></td>
<td align="right" valign="bottom" width="200px">
__TOC__
</td></tr>
</table>

==News==
* ''5th, April, 2005'' - '''Gnutella2 website re-opened'''<br>After server problems we're proud to re-launch the Gnutella2 website. It is now based on MediaWiki (like Wikipedia). The forums are gone, but we now have a [[mailing list]] for Gnutella2 related discussion. Feel free to contribute to the wiki.

==Mailing List==
To allow communication among developers of Gnutella2 clients, there exists a [[mailing list]]

== The Gnutella 2 Network==
# [[Gnutella2|What is Gnutella2]]
# [[Gnutella2 Standard]]
# [[Network Architecture]]
# [[Node Types and Responsibilities]]
# [[Background|Background / Archive]]

== The Protocol==
# [[TCP Stream Connection and Handshaking]]
# [[UDP Transceiver]]
# [[Packet Structure]]
# [[Datatypes]]
# [[Basic Network Maintenance]]
# [[Known Hub Cache and Hub Cluster Cache]]
# [[Node Route Cache and Addressed Packet Forwarding]]
# [[Query Hash Tables]]
# [[Object Search Mechanism]]
# [[Search Security]]
# [[Search Description]]
# [[Search Acknowledgement]]
# [[Search Results]]
# [[Simple Query Language and Metadata]]
# [[Server for Uploading]]
# [[Client for Downloading]]
# [[User Profile Challenge and Delivery]]

== Root Packets ==
{{Root packets}}

== Proposals==
* [[Active Source Exchange]] (ASE) by Kevin O'Toole

== Additional Information==
Since Gnutella2 somehow evolved from the original Gnutella protocol you might find the following links useful:
* [http://www.the-gdf.org The GDF Wiki]
* [http://groups.yahoo.com/group/the_gdf/ Gnutella Developer forum]
* [http://rfc-gnutella.sourceforge.net/ RFC Gnutella]
* [http://draketo.de/inhalt/krude-ideen/gnufu_en.html Gnutella for Users (GnuFU)]
* [http://crawler.instantnetworks.net/ Gnutella2 Crawler]

Gnutella2-powered software

2005-04-05T17:45:35Z

Spooky: Updated

==Gnutella2 Clients==

The following clients support Gnutella2:

<table width="100%" border="0" cellpadding="6" cellspacing="1" bgcolor="#A5CAD2">
<tr>
<th width="55%">Client Name</th>
<th width="23%">Platform</th>
<th width="22%">Support</th>
</tr>
<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://www.shareaza.com Shareaza]<br>
Shareaza pioneered the Gnutella2 platform and continues to play an active role in its development. Connects to several additional networks as well.</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Windows</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Full 1.0 Support</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#F2F2F2">[http://www.mldonkey.net/ MLDonkey]<br>
An open source deamon client. Has many different GUI's you can use. Also connects to many other networks in addition to Gnutella2.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Cross-Platform</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">1.0 Support (Beta)</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://caribou.epitech.net/ Caribou]<br>
Caribou is a peer to peer software design for general purpose p2p communication (file sharing, news, messaging), based on Gnutella2 network</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Windows, Linux, Mac OS X, etc.</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">pre-alpha</td>
</tr>

<tr align="left" valign="top">
<td bgcolor="#F2F2F2">[http://www.filescope.com/ FileScope]<br>
FileScope is a file sharing program that connects to multiple peer-to-peer (P2P) networks and allows for finding and downloading all types of files. Written in C#.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Windows</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">N/A</td>
</tr>

<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://sourceforge.net/projects/agio/ Adagio]<br>
Adagio is a Gnutella2 server which aims to provide a very configurable, highly reliable cross-platform daemon.</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Cross-Platform</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">1.0 Support</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#F2F2F2">[http://www.gnucleus.com/ Gnucleus]<br>
Gnucleus is one of the first Gnutella clients and is an open source Gnutella2 client. It is based on GnucDNA, a Gnutella2 library.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Windows</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Full 1.0 support</td>
</tr>

<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://www.kiwialpha.com/ Kiwi]<br>
Lightweight client, based on GnucDNA</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Windows</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Uses GnucDNA</td>
</tr>

<tr align="left" valign="top" class="normal">
<td bgcolor="#F2F2F2">[http://www.morpheus.com/ Morpheus]<br>
Morpheus is a mainstram P2P client which switched from the FastTrack network to the Gnutella network and now to the Gnutella2 network.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Windows</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Uses GnucDNA</td>
</tr>
</table>

Mailing list

2005-04-05T17:31:34Z

Spooky: dcat: feel free to put info about the real mailinglist on this page

==G2DN Mailing List==

This is the mailing list for communication between Gnutella2 developers and other interested people. This is not for help on how to use Gnutella2 applications.

To '''subscribe''' to this list send an e-mail to g2dn-request@freelists.org with 'subscribe' in the subject line.

To post on this mailinglist send an e-mail to g2dn@freelists.org

This mailinglist is powered by FreeLists.org and runs on Ecartis. You can login to your account at http://www.freelists.org

For help on how to use advanced features of this list refer to http://www.freelists.org/help/index.html . You can also send an e-mail with 'help' in the subject line to g2dn-request@freelists.org

You can find the message archives at http://www.freelists.org/archives/g2dn/

If you want to '''unsubscribe''' from the list you can do so through the web interface, or by sending email to the following address: g2dn-request@freelists.org with 'unsubscribe' in the Subject line.

Active Source Exchange

2005-04-05T17:19:39Z

Spooky:

==Active Source Exchange (ASE) 1.0==

<table width="100%" ><tr>
<td allign="right">
<small>Written By: Kevin O'Toole<br>
kevo@napjunk.net<br>
Shareaza Alpha Team<br>
2003.06.14

Edited By: Jonathan C. Nilson<br>
JCNilson@myrealbox.com<br>
Shareaza Alpha Team<br>
2003.06.16</small>
</td>

<td allign="right">__TOC__</td>
</tr>
</table>

===Status===
This proosal is currently not implemented by any client.

===Introduction===
This proposal addresses the current flaws in the alternate source mesh by not passing on sources that are "old" and client identification for whom to receive sources from. This does not eliminate alternate sources but should make the current system better because it will help keep the mesh (exchange) of sources clean of dead hosts. The proposal has the added benefit of reducing the effects of a Gnutella-style Denial-of-Service attack by reducing the number of bad sources a Shareaza node has, thus keeping that Shareaza node from pestering as many dead sources.

===Why is it needed?===
ASE removes the need to requery frequently to gather a list of new sources. By sharing only current sources, users will not ...

* get flooded with more dead hosts than live ones
* receive connection attempts for a long time after
* have to clean out a long list of bad hosts
* expericence hangs in the client as frequently
* attempt as many connections on a random port/dynamic IP user
* participte unwittingly in a DoS attack on another user
* requery the network as frequently
* pass a firewalled source to a firewalled client

There are still problems ASE does address but cannot completely solve, which include ...

* hostile clients passing on fake or dead hosts
* hostile clients passing on expired sources

===What's different?===
ASE changes the way a client chooses which alternate sources to pass and how long each will be active. In effect, a complying client would pass on only active sources in the mesh, each with an expiry time which should be between 10 and 15 minutes. Active sources are defined as sources which have successfully been Queued or Uploading at some time in the last expiry time block (10~15 minutes). This timeout is critical. If clients simply maintain a list, the dead hosts will take longer to populate the ASE, but will still eventually spread and the same problems will occur.

To avoid this fiasco, a client in the ASE passes the address and the timeout (at its current value) to the recipient. This timeout is only reset if the address is successfully contacted and Queued or Uploading before the timeout value expires. Therefore, unless a client sees the source in a successful queue or upload, the address will expire (no matter how many clients have received the source) at the same time. If one client does reset the expiry counter, only sources which receive the source in the ASE subsequently will still have the source. No messages are sent back to clients who have an entry that is about to expire.

If a client wishes to maintain a list of sources that have expired from the ASE, these sources shall not be passed on to subsequent hosts, with or without a timeout. At this point, the sources shall remain knowlegable to the local host alone.

===Firewalls===
To avoid the propogation of firewalled sources to firewalled clients, each source shall have a header identification indicating the source as a firewalled client which would be sent with the address and timeout. The uploader sends any sources to normal clients, while push sources are not sent to firewalled clients. IANA Reserved IP ranges will not be passed along.

===Security===
If every client played by these rules, security would be no problem, unfortunately we have to plan for the contingency that we may have hostile, buggy, or poorly-programmed clients. Keeping this system secure would perhaps be a more difficult task than implementing it. Options for achieving this would be either client identification (which can easily be spoofed) or client-to-client authentication. Without client identification or authentication, users run the same risks involved in the alternate-source mesh now.

With client identification security, a user would only receive hosts from chosen vendors (filtered by reported user-agent strings) but still send hosts to any client. This would be a simple solution to the problem if everyone obeyed the rules. Unfortunately, hostile clients such as Qtrax change their user-agent identification to match that of their downloader, which would make the system useless against such clients (fortunately, few exist).

A more complex (perhaps neglible) solution would be client-to-client authentication. Although this should completely solve the problem, it would require a large amount of work for minimal gain, and some valuable alternate sources from unsupporting clients would be lost. This authentication method would, by definition, be closed to only trusted clients, which also defeats the spirit of the Gnutella community. While it solves some problems very well, this course of action is not recommended.

===Scenario===
To properly illustrate this new system, consider the following scenario. We will assume for the sake of illustration that the expiry threshold is at 15 minutes.

ClientA, ClientB, and Client C all support ASE in its current form. ClientA and ClientB are both downloading File1. Client A knows of the following sources for File1:

<pre><nowiki>111.111.111.1 (Queued 3 of 10 Now)
111.111.111.2 (Downloading Now)
111.111.111.3 (Queued 10 minutes ago)
111.111.111.5 (Downloading Now)
192.168.0.1 (Push) (Never contacted)
0.0.0.0 (Push) (Downloaded 5 minutes ago)
</nowiki></pre>

ClientB knows of the following sources for File1:

<pre><nowiki>222.222.222.4 (Queued 20 minutes ago)
ClientA (Never contacted)</nowiki></pre>

As ClientB contacts ClientA for the file in question, it refuses to transmit the single known source it has because that source has expired from the ASE. While ClientB may choose to keep that source for its own use, it will not transmit that source to other clients, with or without an expiry timeout.

When ClientA returns the handshake, it transmits a list of ASE sources to ClientB:

<pre><nowiki>111.111.111.1; Timeout=15m
111.111.111.2; Timeout=15m
111.111.111.3; Timeout=5m
111.111.111.5; Timeout=15m; Push</nowiki></pre>

Since 0.0.0.0 is an IANA Private Range, the address is not passed to ClientB, although it is a valid source with a timeout at 10m. 192.168.0.1 is not passed because it has never been contacted.

After ClientB receives this list of sources and is sent a Busy reply by ClientA, ClientB reattempts a connection with 222.222.222.4. This connection results in a successful download. One minute has passed since the ASE list from ClientA was received by ClientB.

ClientC is firewalled, and after searching and finding ClientB as a source for File1, it attempts a connection. Since it has no sources, none are passed to ClientB, however, ClientB passes the same list of known sources to ClientC as ClientA did to ClientB. ClientB also adds its own experience by transimtting 222.222.222.4 because that source is now active and omiting 111.111.111.5 because that is a firewalled source and ClientC is firewalled, so the final list transmitted from ClientB to ClientC is as follows:

<pre><nowiki>111.111.111.1; Timeout=14m
111.111.111.2; Timeout=14m
111.111.111.3; Timeout=4m
222.222.222.4; Timeout=15m</nowiki></pre>

Please note that Timeout values are listed in minutes for the sake of human understanding and will probably be coded in seconds instead.

===Conclusion===
While this does not solve every problem of the current alternate source mesh, it would be beneficial for anyone who currently uses this system. ASE offers a simple solution to the problems with the alternate source mesh. If implemented on a wide-scale, it could reduce traffic substantially and increase file transfer efficiency tenfold.

Active Source Exchange

2005-04-05T17:17:24Z

Spooky: first version

==Active Source Exchange (ASE) 1.0==

<table width="100%" ><tr>
<td allign="right">
<small>Written By: Kevin O'Toole<br>
kevo@napjunk.net<br>
Shareaza Alpha Team<br>
2003.06.14

Edited By: Jonathan C. Nilson<br>
JCNilson@myrealbox.com<br>
Shareaza Alpha Team<br>
2003.06.16</small>
</td>

<td allign="right">__TOC__</td>
</tr>
</table>

===1. Introduction===
This proposal addresses the current flaws in the alternate source mesh by not passing on sources that are "old" and client identification for whom to receive sources from. This does not eliminate alternate sources but should make the current system better because it will help keep the mesh (exchange) of sources clean of dead hosts. The proposal has the added benefit of reducing the effects of a Gnutella-style Denial-of-Service attack by reducing the number of bad sources a Shareaza node has, thus keeping that Shareaza node from pestering as many dead sources.

===2. Why is it needed?===
ASE removes the need to requery frequently to gather a list of new sources. By sharing only current sources, users will not ...

* get flooded with more dead hosts than live ones
* receive connection attempts for a long time after
* have to clean out a long list of bad hosts
* expericence hangs in the client as frequently
* attempt as many connections on a random port/dynamic IP user
* participte unwittingly in a DoS attack on another user
* requery the network as frequently
* pass a firewalled source to a firewalled client

There are still problems ASE does address but cannot completely solve, which include ...

* hostile clients passing on fake or dead hosts
* hostile clients passing on expired sources

===3. What's different?===
ASE changes the way a client chooses which alternate sources to pass and how long each will be active. In effect, a complying client would pass on only active sources in the mesh, each with an expiry time which should be between 10 and 15 minutes. Active sources are defined as sources which have successfully been Queued or Uploading at some time in the last expiry time block (10~15 minutes). This timeout is critical. If clients simply maintain a list, the dead hosts will take longer to populate the ASE, but will still eventually spread and the same problems will occur.

To avoid this fiasco, a client in the ASE passes the address and the timeout (at its current value) to the recipient. This timeout is only reset if the address is successfully contacted and Queued or Uploading before the timeout value expires. Therefore, unless a client sees the source in a successful queue or upload, the address will expire (no matter how many clients have received the source) at the same time. If one client does reset the expiry counter, only sources which receive the source in the ASE subsequently will still have the source. No messages are sent back to clients who have an entry that is about to expire.

If a client wishes to maintain a list of sources that have expired from the ASE, these sources shall not be passed on to subsequent hosts, with or without a timeout. At this point, the sources shall remain knowlegable to the local host alone.

===4. Firewalls===
To avoid the propogation of firewalled sources to firewalled clients, each source shall have a header identification indicating the source as a firewalled client which would be sent with the address and timeout. The uploader sends any sources to normal clients, while push sources are not sent to firewalled clients. IANA Reserved IP ranges will not be passed along.

===5. Security===
If every client played by these rules, security would be no problem, unfortunately we have to plan for the contingency that we may have hostile, buggy, or poorly-programmed clients. Keeping this system secure would perhaps be a more difficult task than implementing it. Options for achieving this would be either client identification (which can easily be spoofed) or client-to-client authentication. Without client identification or authentication, users run the same risks involved in the alternate-source mesh now.

With client identification security, a user would only receive hosts from chosen vendors (filtered by reported user-agent strings) but still send hosts to any client. This would be a simple solution to the problem if everyone obeyed the rules. Unfortunately, hostile clients such as Qtrax change their user-agent identification to match that of their downloader, which would make the system useless against such clients (fortunately, few exist).

A more complex (perhaps neglible) solution would be client-to-client authentication. Although this should completely solve the problem, it would require a large amount of work for minimal gain, and some valuable alternate sources from unsupporting clients would be lost. This authentication method would, by definition, be closed to only trusted clients, which also defeats the spirit of the Gnutella community. While it solves some problems very well, this course of action is not recommended.

===6. Scenario===
To properly illustrate this new system, consider the following scenario. We will assume for the sake of illustration that the expiry threshold is at 15 minutes.

ClientA, ClientB, and Client C all support ASE in its current form. ClientA and ClientB are both downloading File1. Client A knows of the following sources for File1:

<pre><nowiki>111.111.111.1 (Queued 3 of 10 Now)
111.111.111.2 (Downloading Now)
111.111.111.3 (Queued 10 minutes ago)
111.111.111.5 (Downloading Now)
192.168.0.1 (Push) (Never contacted)
0.0.0.0 (Push) (Downloaded 5 minutes ago)
</nowiki></pre>

ClientB knows of the following sources for File1:

<pre><nowiki>222.222.222.4 (Queued 20 minutes ago)
ClientA (Never contacted)</nowiki></pre>

As ClientB contacts ClientA for the file in question, it refuses to transmit the single known source it has because that source has expired from the ASE. While ClientB may choose to keep that source for its own use, it will not transmit that source to other clients, with or without an expiry timeout.

When ClientA returns the handshake, it transmits a list of ASE sources to ClientB:

<pre><nowiki>111.111.111.1; Timeout=15m
111.111.111.2; Timeout=15m
111.111.111.3; Timeout=5m
111.111.111.5; Timeout=15m; Push</nowiki></pre>

Since 0.0.0.0 is an IANA Private Range, the address is not passed to ClientB, although it is a valid source with a timeout at 10m. 192.168.0.1 is not passed because it has never been contacted.

After ClientB receives this list of sources and is sent a Busy reply by ClientA, ClientB reattempts a connection with 222.222.222.4. This connection results in a successful download. One minute has passed since the ASE list from ClientA was received by ClientB.

ClientC is firewalled, and after searching and finding ClientB as a source for File1, it attempts a connection. Since it has no sources, none are passed to ClientB, however, ClientB passes the same list of known sources to ClientC as ClientA did to ClientB. ClientB also adds its own experience by transimtting 222.222.222.4 because that source is now active and omiting 111.111.111.5 because that is a firewalled source and ClientC is firewalled, so the final list transmitted from ClientB to ClientC is as follows:

<pre><nowiki>111.111.111.1; Timeout=14m
111.111.111.2; Timeout=14m
111.111.111.3; Timeout=4m
222.222.222.4; Timeout=15m</nowiki></pre>

Please note that Timeout values are listed in minutes for the sake of human understanding and will probably be coded in seconds instead.

===7. Conclusion===
While this does not solve every problem of the current alternate source mesh, it would be beneficial for anyone who currently uses this system. ASE offers a simple solution to the problems with the alternate source mesh. If implemented on a wide-scale, it could reduce traffic substantially and increase file transfer efficiency tenfold.

Main Page

2005-04-05T17:02:20Z

Spooky: /* Proposals */

== Gnutella2 Developer Network ==
Welcome to the Gnutella2 Developer Network
The Gnutella2 Developers' Network (or G2DN) is a convenient central location for discussing, developing and documenting all aspects of Gnutella2. Like the Gnutella2 platform itself, this website is designed to evolve through community support and participation. Everyone is free to contibute discussion, proposal documents, developer resources or any other relevant material.

Although the G2DN is primarily geared toward developers and technical enthusiests, users of Gnutella2-enabled software are also welcome to participate. Those who are new to Gnutella2 may wish to check out some of the [[Gnutella2-powered software]].

===Mailing List===
To allow communication among developers of Gnutella2 clients, there exists a [[mailing list]]

== The Gnutella 2 Network==
# [[Gnutella2|What is Gnutella2]]
# [[Gnutella2 Standard]]
# [[Network Architecture]]
# [[Node Types and Responsibilities]]
# [[Background|Background / Archive]]

== The Protocol==
# [[TCP Stream Connection and Handshaking]]
# [[UDP Transceiver]]
# [[Packet Structure]]
# [[Datatypes]]
# [[Basic Network Maintenance]]
# [[Known Hub Cache and Hub Cluster Cache]]
# [[Node Route Cache and Addressed Packet Forwarding]]
# [[Query Hash Tables]]
# [[Object Search Mechanism]]
# [[Search Security]]
# [[Search Description]]
# [[Search Acknowledgement]]
# [[Search Results]]
# [[Simple Query Language and Metadata]]
# [[Server for Uploading]]
# [[Client for Downloading]]
# [[User Profile Challenge and Delivery]]

== Root Packets ==
{{Root packets}}

== Proposals==
* [[Active Source Exchange]] (ASE) by Kevin O'Toole

== Additional Information==
Since Gnutella2 somehow evolved from the original Gnutella protocol you might find the following links useful:
* [http://www.the-gdf.org The GDF Wiki]
* [http://groups.yahoo.com/group/the_gdf/ Gnutella Developer forum]
* [http://rfc-gnutella.sourceforge.net/ RFC Gnutella]
* [http://draketo.de/inhalt/krude-ideen/gnufu_en.html Gnutella for Users (GnuFU)]
* [http://crawler.instantnetworks.net/ Gnutella2 Crawler]

Main Page

2005-04-05T17:01:37Z

Spooky: ASE added

== Gnutella2 Developer Network ==
Welcome to the Gnutella2 Developer Network
The Gnutella2 Developers' Network (or G2DN) is a convenient central location for discussing, developing and documenting all aspects of Gnutella2. Like the Gnutella2 platform itself, this website is designed to evolve through community support and participation. Everyone is free to contibute discussion, proposal documents, developer resources or any other relevant material.

Although the G2DN is primarily geared toward developers and technical enthusiests, users of Gnutella2-enabled software are also welcome to participate. Those who are new to Gnutella2 may wish to check out some of the [[Gnutella2-powered software]].

===Mailing List===
To allow communication among developers of Gnutella2 clients, there exists a [[mailing list]]

== The Gnutella 2 Network==
# [[Gnutella2|What is Gnutella2]]
# [[Gnutella2 Standard]]
# [[Network Architecture]]
# [[Node Types and Responsibilities]]
# [[Background|Background / Archive]]

== The Protocol==
# [[TCP Stream Connection and Handshaking]]
# [[UDP Transceiver]]
# [[Packet Structure]]
# [[Datatypes]]
# [[Basic Network Maintenance]]
# [[Known Hub Cache and Hub Cluster Cache]]
# [[Node Route Cache and Addressed Packet Forwarding]]
# [[Query Hash Tables]]
# [[Object Search Mechanism]]
# [[Search Security]]
# [[Search Description]]
# [[Search Acknowledgement]]
# [[Search Results]]
# [[Simple Query Language and Metadata]]
# [[Server for Uploading]]
# [[Client for Downloading]]
# [[User Profile Challenge and Delivery]]

== Root Packets ==
{{Root packets}}

== Proposals==
* Active Source Exchange (ASE) by Kevin O'Toole

== Additional Information==
Since Gnutella2 somehow evolved from the original Gnutella protocol you might find the following links useful:
* [http://www.the-gdf.org The GDF Wiki]
* [http://groups.yahoo.com/group/the_gdf/ Gnutella Developer forum]
* [http://rfc-gnutella.sourceforge.net/ RFC Gnutella]
* [http://draketo.de/inhalt/krude-ideen/gnufu_en.html Gnutella for Users (GnuFU)]
* [http://crawler.instantnetworks.net/ Gnutella2 Crawler]

Main Page

2005-04-03T02:10:56Z

Spooky: Crawler link added

== Gnutella2 Developer Network ==
Welcome to the Gnutella2 Developer Network
The Gnutella2 Developers' Network (or G2DN) is a convenient central location for discussing, developing and documenting all aspects of Gnutella2. Like the Gnutella2 platform itself, this website is designed to evolve through community support and participation. Everyone is free to contibute discussion, proposal documents, developer resources or any other relevant material.

Although the G2DN is primarily geared toward developers and technical enthusiests, users of Gnutella2-enabled software are also welcome to participate. Those who are new to Gnutella2 may wish to check out some of the [[Gnutella2-powered software]].

===Mailing List===
To allow communication among develoeprs of Gnutella2 clients, there exists a [[mailing list]]

==1. The Gnutella 2 Network==
# [[Gnutella2|What is Gnutella2]]
# [[Gnutella2 Standard]]
# [[Network Architecture]]
# [[Node Types and Responsibilities]]
# [[Background|Background / Archive]]

==2. The Protocol==
# [[TCP Stream Connection and Handshaking]]
# [[UDP Transceiver]]
# [[Packet Structure]]
# [[Datatypes]]
# [[Basic Network Maintenance]]
# [[Known Hub Cache and Hub Cluster Cache]]
# [[Node Route Cache and Addressed Packet Forwarding]]
# [[Query Hash Tables]]
# [[Object Search Mechanism]]
# [[Search Security]]
# [[Search Description]]
# [[Search Acknowledgement]]
# [[Search Results]]
# [[Simple Query Language and Metadata]]
# [[Server for Uploading]]
# [[Client for Downloading]]
# [[User Profile Challenge and Delivery]]

==3. Proposals==

==4. Additional Information==
Since Gnutella2 somehow evolved from the original Gnutella protocol you might find the following links useful:
* [http://www.the-gdf.org The GDF Wiki]
* [http://groups.yahoo.com/group/the_gdf/ Gnutella Developer forum]
* [http://rfc-gnutella.sourceforge.net/ RFC Gnutella]
* [http://draketo.de/inhalt/krude-ideen/gnufu_en.html Gnutella for Users (GnuFU)]
* [http://crawler.instantnetworks.net/ Gnutella2 Crawler]

== Indices ==

* [[Main index]]
* [[Root packets index]]

Background

2005-04-03T02:03:37Z

Spooky: /* History */

==History==
The Gnutella2 protocol has been developed by [[Wikipedia:Michael_Stokes|Michael Stokes]] in the year 2002. It has been discussed quite controversial and some people prefer to refer to the Gnutella2 protocal by the name ''Mike's Protocol'' (MP). You might want to read the [[Wikipedia:Gnutella2|Wikipedia article]] on Gnutella2.

==Old Publications==
* [[G2_specs_part1|Gnutella2 Specifications part ONE]]
* [[G2_specs_part2|Gnutella2 Specifications part TWO]]

Gnutella2-powered software

2005-04-03T02:02:16Z

Spooky: First version

==Gnutella2 Clients==

The following clients support Gnutella2:

<table width="100%" border="0" cellpadding="6" cellspacing="1" bgcolor="#A5CAD2">
<tr>
<th width="55%">Client Name</th>
<th width="23%">Platform</th>
<th width="22%">Support</th>
</tr>
<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://www.shareaza.com Shareaza]<br>
Shareaza pioneered the Gnutella2 platform and continues to play an active role in its development. Connects to several additional networks as well.</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Windows</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Full 1.0 Support</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#F2F2F2">[http://www.mldonkey.net/ MLDonkey]<br>
An open source deamon client. Has many different GUI's you can use. Also connects to many other networks in addition to Gnutella2.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Cross-Platform</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">1.0 Support (Beta)</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://shinobu.sourceforge.net/ Shinobu]<br>
Shinobu is the first Linux-only Gnutella2 client. It's written in C, uses GTK 2.0, and attempts to be easy-to-use without using a large amount of memory</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Linux</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">N/A</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#F2F2F2">[http://sourceforge.net/projects/agio/ Adagio]<br>
Adagio is a Gnutella2 server which aims to provide a very configurable, highly reliable cross-platform daemon.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Cross-Platform</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">1.0 Support</td>
</tr>
<tr align="left" valign="top">
<td bgcolor="#FFFFFF">[http://www.gnucleus.com/ Gnucleus]<br>
Gnucleus is one of the first Gnutella clients and is an open source Gnutella2 client. It is based on GnucDNA, a Gnutella2 library.</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Windows</td>
<td align="center" valign="middle" bgcolor="#FFFFFF">Full 1.0 support</td>
</tr>

<tr align="left" valign="top" class="normal">
<td bgcolor="#F2F2F2">[http://www.morpheus.com/ Morpheus]<br>
Morpheus is a mainstram P2P client which switched from the FastTrack network to the Gnutella network and now to the Gnutella2 network.</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Windows</td>
<td align="center" valign="middle" bgcolor="#F2F2F2">Uses GnucDNA</td>
</tr>
</table>

LNI

2005-04-02T22:32:30Z

Spooky:

[[KHL|<< KHL]] | [[PI|PI >>]] | [[Root_packets_index|Packet Index]]

= /LNI - Local Node Information =

The local node information packet is used to convey essential information about the
node on either end of a TCP stream connection.

== Sending ==

The /LNI packet should be sent to a TCP neighbour upon connection, and at regular
intervals following that if the information within it has changed. A minimum update
time of one minute is recommended.

== Receiving ==

Upon receiving a /LNI packet, the information within should be stored for subsequent
use.

== Payload ==

This packet has no payload at the current time.

== Children ==

The /LNI packet has many child packet types defined at the current time:

* /LNI/NA - Node Address
* /LNI/GU - GUID
* /LNI/V - Vendor Code
* /LNI/LS - Library Statistics
* /LNI/HS - Hub Status

== /LNI/NA - Node Address ==

The /LNI/NA child packet specifies the node or network address of the sending node.

=== Sending ===

This child is required.

=== Payload ===

The physical network address of the sending node. See datatypes for more
information.

=== Children ===

This packet has no known children at the current time.

== /LNI/GU - GUID ==

The /LNI/GU child packet specifies the globally unique identifier of the sending node.

=== Sending ===

This child is required.

=== Payload ===

The 16 byte globally unique node identifier of the sending node.

=== Children ===

This packet has no known children at the current time.

== /LNI/V - Vendor Code ==

The /LNI/V child packet specifies the vendor code of the software operating the
sending node.

=== Sending ===

This child is optional.

=== Payload ===

A four byte vendor code.

=== Children ===

This packet has no known children at the current time.

== /LNI/LS - Library Statistics ==

The /LNI/LS packet provides information about the content library of the sending
node.

=== Sending ===

This child is optional.

=== Payload ===

Two 32-bit integers representing the number of files in the local library, and the total
KB in the local library respectively. If the sending node is a hub these statistics are
combined with the known statistics from all connected leaves. The payload may grow
beyond 8 bytes in the future.

=== Children ===

This packet has no known children at the current time.

== /LNI/HS - Hub Status ==

The /LNI/HS packet is included only if the sending node is a hub, and contains the
status of the hub.

=== Sending ===

This child packet should only be sent by hub nodes. Leaf nodes should not transmit
it, and if it is received from a leaf node, it should be ignored.

=== Payload ===

Two 16-bit integers representing the current leaf count and the maximum leaf count
respectively. This packet may grow beyond 4 bytes in the future.

=== Children ===

This packet has no known children at the current time.

KHL

2005-04-02T22:30:47Z

Spooky:

[[LNI|LNI >>]] | [[Root_packets_index|Packet Index]]

= /KHL =

The known hub list packet is used to exchange a list of known Gnutella2 hubs with
neighbouring nodes. It is exchanged regularly over all TCP connections, between
hubs and between hubs and leaves (and vice versa). It enables a completely up to
date picture of the local hub cluster, and the gradual propagation of cached hub
records.

== Sending ==

This packet should be sent through TCP links at regular intervals. One minute is
suggested for busy links, leaves may send to hubs less often. Note that unlike /LNI,
/KHL must be sent regularly even if the information does not change in order to
refresh timestamps and keep time-sorted caches in the correct order.

== Receiving ==

Upon receiving a known hub list, the detailed hubs should be stored in appropriate
data structures. Neighbouring hubs must be retained in the hub cluster records,
while cached hubs should be merged with the local accessible hub cache.

== Payload ==

This packet has no payload at the current time.

== Children ==

This packet has several child packet types defined at the current time:

* /KHL/TS - Timestamp
* /KHL/NH - Neighbouring Hub
* /KHL/CH - Cached Hub

== /KHL/TS - Timestamp ==

The /KHL/TS child provides a timestamp representing the current universal time at
the sending node. This can be used as a reference when considering other
timestamps in the packet, allowing them to be adjusted to eliminate differences
between the time setting on the local and remote node.

=== Sending ===

This child is optional but recommended.

=== Payload ===

A 32-bit integer representing the current UNIX time, or time (NULL).

=== Children ===

This packet has no known children at the current time.

== /KHL/NH - Neighbouring Hub ==

This child packet represents a neighbouring hub.

=== Sending ===

One should be included for each Gnutella2 neighbour hub, with the possible
exception of the one to which this packet is being sent.

=== Payload ===

The node address of the hub. See datatypes for more information.

=== Children ===

This packet has several child packet types defined at the current time:

* /KHL/NH/GU - GUID
* /KHL/NH/V - Vendor Code
* /KHL/NH/LS - Library Statistics
* /KHL/NH/HS - Hub Status

=== /KHL/NH/GU - GUID ===

The /KHL/NH/GU child packet specifies the globally unique identifier of the hub.

==== Sending ====

This child is optional.

==== Payload ====

The 16 byte globally unique node identifier of the hub.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/V - Vendor Code ===

The /KHL/NH/V child packet specifies the vendor code of the software operating the
hub.

==== Sending ====

This child is optional.

==== Payload ====

A four byte vendor code.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/LS - Library Statistics ===

The /KHL/NH/LS packet provides information about the content library of the hub
and its connected leaves.

==== Sending ====

This child is optional.

==== Payload ====

Two 32-bit integers representing the number of files, and the total KB of available
content in the hub's own library and the libraries of its connected leaves. The
payload may grow beyond 8 bytes in the future.

==== Children ====

This packet has no known children at the current time.

=== /KHL/NH/HS - Hub Status ===

The /KHL/NH/HS packet contains the status of the hub.

==== Sending ====

This child is optional.

==== Payload ====

Two 16-bit integers representing the current leaf count and the maximum leaf count
respectively. This packet may grow beyond 4 bytes in the future.

==== Children ====

This packet has no known children at the current time.

== /KHL/CH - Cached Hub ==

This child packet represents a hub from the hub cache.

=== Sending ===

An arbitrary number of these children should be included, constrained by space and
the number of available cached hubs. A selection of freshly time stamped but
previously unknown hubs should be included, and the list should not include hubs in
the local cluster.

=== Payload ===

The node address of the hub, followed by the time this hub was last seen as a 32-bit
timestamp. The last seen time can be adjusted based on the difference between the
/KHL/TS timestamp and the local time. Note that the node address is of variable
length, so it is important to consider the last 4 bytes as the timestamp rather than
assuming a fixed length node address.

=== Children ===

This packet has several child packet types defined at the current time:

* /KHL/CH/GU - GUID
* /KHL/CH/V - Vendor Code
* /KHL/CH/LS - Library Statistics
* /KHL/CH/HS - Hub Status

This extra information is considered less relevant for a cached hub because it is
unlikely to be up to date enough to be useful.

=== /KHL/CH/GU - GUID ===

The /KHL/CH/GU child packet specifies the globally unique identifier of the hub.

==== Sending ====

This child is optional.

==== Payload ====

The 16 byte globally unique node identifier of the hub.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/V - Vendor Code ===

The /KHL/CH/V child packet specifies the vendor code of the software operating the
hub.

==== Sending ====

This child is optional.

==== Payload ====

A four byte vendor code.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/LS - Library Statistics ===

The /KHL/NH/LS packet provides information about the content library of the hub
and its connected leaves.

==== Sending ====

This child is optional.

==== Payload ====

Two 32-bit integers representing the number of files, and the total KB of available
content in the hub's own library and the libraries of its connected leaves. The
payload may grow beyond 8 bytes in the future.

==== Children ====

This packet has no known children at the current time.

=== /KHL/CH/HS - Hub Status ===

The /KHL/CH/HS packet contains the status of the hub.

==== Sending ====

This child is optional.

==== Payload ====

Two 16-bit integers representing the current leaf count and the maximum leaf count
respectively. This packet may grow beyond 4 bytes in the future.

==== Children ====

This packet has no known children at the current time.

Basic Network Maintenance

2005-04-02T22:28:46Z

Spooky:

[[Datatypes|<< Datatypes]] | [[Known_Hub_Cache_and_Hub_Cluster_Cache|Known Hub Cache and Hub Cluster Cache >>]] | [[Main_Page|Main Page]]

== Introduction ==
Once TCP stream connections have been established forming the overall network
topology, a set of basic packet types and behaviours are used to keep the network
together and perform maintenance functions.
These maintenance packets are documented in this section, each with name,
payload, children and behaviours.

== Packet Types ==

/[[PI]] - Ping

/[[PO]] - Pong

/[[LNI]] - Local Node Information

Datatypes

2005-04-02T22:26:52Z

Spooky:

[[Packet_Structure|<< Packet Structure]] | [[Basic_Network_Maintenance|Basic Network Maintenance >>]] | [[Main_Page|Main Page]]

== Introduction ==

The format of a packet payload is defined by the packet type and can consist of any
binary data; however, there are a number of conventions in place for serializing
common datatypes.

== Multi-Byte Integers ==

Multi-byte integers are serialized in the byte-order of the topmost packet. Little endian
is the default byte-order; however, big-endian byte order can be selected for
those who want it.

== Network/Node Addresses ==

A network or node address consists of a physical address and a port number, and are
of variable length, depending on the address family.
In IPv4, a network/node address is six bytes long: 4 bytes for an IP address and 2
bytes for a port number as follows:

<pre>
<nowiki>
typedef struct
{
BYTE ip[4];
SHORT port;
} IPV4_ENDPOINT;
</nowiki>
</pre>

Note that this is considered an array of 4 8-bit integers (bytes), followed by a 16-bit
integer (short). Byte order does not affect bytes, but it will affect the 16-bit port
number.

IPv6 addresses are longer and are not yet defined within the scope of Gnutella2,
however, applications should be aware that if the node address is not 6 bytes, it is of
a different address family.

== GUIDs ==

Globally unique identifiers (GUIDs) are used to identify nodes on the network. GUIDs
are serialized as an array of 16 bytes.

== Strings ==

Strings are encoded with UTF-8 encoding and serialized as a zero-terminated
sequence of 8 bit integers.

A zero character (0x00) marks the end of the string, however, if the string data meets
the end of the packet (or child packet) payload, the terminator is not required. This
means that packets whose payload consists of a string, do not need to include a zero
string terminator and their payload length will be the byte length of the encoded
string exactly.

UTF-8 encoding is required for all strings present in the packet payload. This means
that 7-bit characters may be passed as-is, while extended characters are encoded
with multi-byte sequences.

All applications must be able to parse UTF-8 encoded strings, however, it is up to the
individual application whether to store the string in Unicode, or convert it to the
local code page for processing. In situations where a packet must be processed 'and'
forwarded, the original packet must be forwarded rather than a regenerated version.
This ensures that both locally unsupported encodings and packet extensions are
preserved.

Applications should never send ANSI strings directly if they contain extended
characters with the MSB set. These should be encoded with UTF-8. If this is not
done, the decoding process may fail and the packet will be discarded or contain
bogus information.

Packet Structure

2005-04-02T22:25:21Z

Spooky:

[[UDP_Transceiver |<< UDP Transceiver]] | [[Datatypes|Datatypes >>]] | [[Main_Page|Main Page]]

== Introduction ==

All Gnutella2 communications are represented with Gnutella2 lightweight tree
packets. This applies everywhere from TCP stream communications to reliable UDP
transmissions to HTTP packet exchanges (where protocol data has been negotiated).
Each tree packet may contain meaningful payload data and/or one or more child
packets, allowing complex document structures to be created and extended in a
backward compatible manner.

The concept can be compared to an XML document tree. The "packets" are
elements, which can in turn contain zero or more child elements (packets). The
payload of a packet is like the attributes of an XML element. However, serializing XML
has a lot of overhead due to all the naming, even in a compact binary form. The
Gnutella2 packet structure makes a compromise: it names elements (packets),
allowing them to be globally recognized and understood, without knowledge of their
format - and stores attributes as binary payloads, requiring knowledge of their
content to parse them.

Thus the element (packet or child packet) is the finite unit of comprehension. This
system provides an excellent trade-off between format transparency and
compactness.

== Fictitious Visual Example ==

<pre>
<nowiki>
+ Query Hit Packet
|
|-+ Node ID (standard)
|
|-+ Server Status (standard)
| \-+ Shareaza Server Status (private extension)
|
|-+ Hit Object
| |-+ URN (standard)
| |-+ Descriptive name (standard)
| | \-+ Alternate name list (extension)
| |-+ URL (standard)
| |-+ Priority indicator (private extension)
| | \-+ Digital signature (private)
| |-+ Alternate source summary (standard)
| \-+ Available ranges (standard)
| . \-+ Estimated completion time (private extension)
|
|-+ Selective digital signature (private)
|
\-+ Routing tags
</nowiki>
</pre>

== Contents ==

Each Gnutella2 packet contains:

* Control flags
* A type name meaningful in the namespace of the packet's parent or context
* A length (or implied length)
* Payload data of a format specific to the packet type name and namespace
* Child packets existing in the namespace of this packet

== Namespace Considerations ==

Each packet contains a relative type name of up to 8 bytes in length, which are case
sensitive. The packet type name is meaningful only in the namespace of the packet's
parent, or in the absence of a parent, the context of the packet (e.g. root level TCP
stream).

This means that, for example a packet "A" inside packet "X" is different to a packet
"A" inside packet "Y". Packets are of the same type only if their fully qualified
absolute type names are equal.

As a convention, when discussing packet type names, they will be noted in their
absolute form with a URL style slash (/) separating each level. In the above example,
the first packet is "/X/A" while the second is "/Y/A". It is clear now that the packets
are of different types.

Packet type names can contain from 1 to 8 bytes inclusive, and none of these bytes
may be a null (0). Community approved packets are by convention named with
uppercase characters and digits, for example "PUSH". Private packet types are by
convention named with lowercase characters and digits, prefixed with the vendor
code of the owner, for example "RAZAclr2".

== Framing ==

Packets are encoded with a single leading control byte, followed by one or more
bytes of packet length, followed by one or more bytes of packet name/ID, followed
by zero or more child packets (framed the same way), followed by zero or more
bytes of payload:

<pre>
<nowiki>
| Control | Length_| Name___ | children and/or payload |
</nowiki>
</pre>

All packets can contain a payload only, children and a payload, children only, or
nothing at all. The total length of the packet header (control, length and type name)
cannot exceed 12 bytes and cannot be less than 2 bytes.

=== The Control Byte ===

The control byte is always non-zero. A zero control byte identifies the end of a
stream of packets, and thus has special meaning. It should not be used in root
packet streams (which do not end).
Control bytes have the following format:

<pre>
<nowiki>
Bit 7 Bit 0
| Len_Len | Name_Len - 1 | CF | BE | // |
</nowiki>
</pre>

* Len_Len is the number of bytes in the length field of the packet, which immediately follows the control byte. There are two bits here which means the length field can be up to 3 bytes long. Len_Len? can be zero if the packet has zero length (no children and no payload), in which case there is no need to encode the length.
* Name_Len is the number of bytes in the packet name field MINUS ONE, which follows the packet length field. There are three bits here which means that packet names can be 1 to 8 bytes long inclusive. Because a 0 here equates to one byte of name, unnamed packets are not possible.
* The three least significant bits of the control byte are reserved for flags. They have the following meanings:
* CF is the compound packet flag. If this bit is set, the packet contains one or more child packets. If not set, the packet does not contain any child packets. If the packet is of zero length, this flag is ignored.
* BE is the big-endian packet flag. If set, all multi-byte values encoded in the packet and its children are encoded in big-endian byte order - including the length in the packet header.
* Other bits are reserved.

=== The Length Field ===

The length field immediately follows the control byte, and can be 0 to 3 bytes long.
Length bytes are stored in the byte order of the packet.

The length value includes the payload of this packet AND any child packets in their
entirety. This is obviously needed so that the entire packet can be detected and
acquired from a stream. The length does not include the header (control byte,
length, and name). The length field precedes the name field to allow it to be read
faster from a stream when acquiring packets.

The length field is in the byte order of the root packet.

=== The Type Name Field ===

The type name field immediately follows the length bytes, and can be from 1 to 8
bytes long. Its format is detailed in the previous section entitled "Namespace
Considerations".

=== Child Packets ===

Child packets are only present if the "compound packet bit" is set in the control byte.
If set, there is one or more child packet immediately following the end of the header.
These child packets are included in the total length of their parent (along with the
payload, which follows the child packets after a packet stream terminator).

Child packets are framed exactly the same way, with a control byte, length, name,
children and/or payload. When the compound bit is set and the packet is not of zero
length, the first child packet must exist. Subsequent child packets may also exist, and
are read in sequentially in the same way that they are read from a root packet
stream. The end of the child packet stream is signalled by the presence of a zero
control byte, OR the end of the parent packet's length (in which case there is no
payload). Including a terminating zero control byte when there is no payload is still
valid, but unnecessary.

=== Payload ===

Payload may exist whenever the length field is non-zero. However, if the compound
bit is set, one or more child packets must be read before the payload is reached. If
there is no packet left after the end of the last child, there is no payload.

=== Notes on the Control Byte ===
Note that there are a number of "marker packet types", which have no children or
payload. It is desirable to encode these in as small a space as possible, which means
omitting the length field and setting the len_len bits to zero in the control byte. This
creates a potential conflict, as the control byte itself may be zero if the type name is
one byte long - and as noted above, a zero control byte has special meaning (end of
packet stream). This must be avoided; luckily it is perfectly legal to set the
compound packet flag (CF) on zero length packets, thus producing a non-zero
control byte and the most compact packet possible.

The compound packet bit MUST be checked when decoding every packet. It should
be done in low-level decoding code to avoid accidental omission. Do not assume that
a packet will not have children - it might not now, but no packets are sterile.
Anything could be augmented or extended in some unknown way in the future. If
you are not interested in children, skip them (which is easy, you don't even need to
recurs through their children).

=== Simple Packet Decoder in C ===

<pre>
<nowiki>
BYTE nInput = ReadNextByte();

if ( nInput == 0 ) return S_NO_MORE_CHILDREN;

BYTE nLenLen = ( nInput & 0xC0 ) >> 6;
BYTE nTypeLen = ( nInput & 0x38 ) >> 3;
BYTE nFlags = ( nInput & 0x07 );

BOOL bBigEndian = ( nFlags & 0x02 ) ? TRUE : FALSE;
BOOL bIsCompound = ( nFlags & 0x04 ) ? TRUE : FALSE;

ASSERT( ! bBigEndian );

DWORD nPacketLength = 0;

ReadBytes( (BYTE*)&nPacketLength, nLenLen );

CHAR szType[9];
ReadBytes( (BYTE*)szType, nTypeLen + 1 );
szType[ nTypeLen + 1 ] = 0;
</nowiki>
</pre>

UDP Transceiver

2005-04-02T22:23:07Z

Spooky:

[[TCP_Stream_Connection_and_Handshaking|<< TCP Stream Connection and Handshaking]] | [[Packet_Structure|Packet Structure >>]] | [[Main_Page|Main Page]]

== Introduction ==

The User Datagram Protocol (UDP) is invaluable in peer to peer systems because it
provides a relatively low-cost (low-overhead) method of sending short, irregular
messages to a very large number of peers on demand. Establishing a TCP stream
connection to a peer simply to deliver a single packet of information is wasteful in
data volume and time for the peers involved and state-aware network devices along
the route, for example, network address translation facilities. When dealing with a
large number of peers quickly, these costs become unbearable. UDP provides a
solution and makes this kind of interaction possible.

However, the delivery of UDP packets is not reliable: packets may be lost en-route for
a number of reasons. Often this behaviour is desirable, for example when the
destination node's connection is highly congested, UDP packets are likely to be
discarded. If the content was not critical, this loss is appropriate as the host's
resources are effectively unavailable. In other scenarios involving critical payloads,
UDP's lack of reliability is a problem: either the sender needs to make sure the
receiver gets the payload, or it needs to know definitively that the receiver was
unavailable.

The Gnutella2 network solves this problem by implementing a selectively engaged
reliability layer on top of the basic UDP protocol. This reliability layer shares some
common functionality with TCP, but importantly does not provide any connection
state and thus retains the efficiency originally sought in UDP.

This allows Gnutella2 to select the most optimal communication medium for each
and every transmission it needs to perform:

* If a significant volume of data is to be exchanged, or subsequent data will be exchanged with the same destination, a TCP connection is established
* If a small volume of important data is to be exchanged in a once-off operation or irregularly, reliable UDP is used
* If a small volume of unimportant data is to be exchanged in a once-off operation or irregularly, unreliable UDP is used

== UDP ==

Gnutella2 semi-reliable communication is transported using the UDP protocol. The
port number for receiving UDP is the same as the port number listening for TCP
connections.

== Encoding ==

Implementing an additional reliable protocol within UDP requires a small control
header before the payload itself. This header is put to good use:

A small signature identifies the packet as a Gnutella2 semi-reliable UDP datagram.
This allows the same port to be used for receiving UDP traffic for other protocols if
desired, and offers some simple protection against random, unexpected traffic.
A content code identifies the payload as a Gnutella2 packet stream, allowing future
protocols to be added within the same reliability layer if desired.
Flags allow additional attributes to be specified, such as inline stateless compression
of the payload (which is a required feature).
The header has a fixed size of 8 bytes, and is represented by the following C
structure:

<pre>
<nowiki>
#pragma pack(1)
typedef struct
{
CHAR szTag3;
BYTE nFlags;
WORD nSequence;
BYTE nPart;
BYTE nCount;
} GND_HEADER;
</nowiki>
</pre>

The members of the structure are detailed below:

* '''szTag''' - contains a three byte encoding protocol identifier, in this case "GND" for "GNutella Datagram". If this signature is not present the packet should not be decoded as a Gnutella2 reliability layer transmission.
* '''nFlags''' - contains flags which modify the content of the packet. The low-order nibble is reserved for critical flags: if one of these bits is set but the decoding software does not understand the meaning, the packet must be discarded. The high-order nibble is reserved for non-critical flags: when set these bits may be interpreted, but an inability to interpret a bit does not cause the packet to be discarded. Currently defined flags are:
:* '''0x01''' - Deflate
:: When the deflate bit is set, the entire payload is compressed with the deflate algorithm. The compression method used is the Deflate Compression Data Format (RFC 1951). On top of this compression a ZLIB ‘wrapper’ is applied (RFC 1950, ZLIB Compressed Data Format). The ZLIB wrapper ensures packet integrity, among other things. Note that the entire payload must be reassembled in the correct order before it can be deflated if the packet was fragmented. Fragments are not compressed separately!
:* '''0x02''' - Acknowledge Me
:: When the acknowledge me bit is set, the sender is expecting an acknowledgement for this packet.

* '''nSequence''' - contains the sequence number of the packet. This sequence number is unique to the sending host only. It is not unique to the pair of the sending host and receiving host as in TCP, as there is no concept of connection state. Sequence numbers on consecutive packets need not be increasing (although that is convenient) â€“ they must only be different. If a packet is fragmented, all of its fragments will have the same sequence number. Byte order is unimportant here.
* '''nPart''' - contains the fragment part number (1 <= nPart <= nCount)
* '''nCount''' - contains the number of fragment parts in this packet. On a transmission, this value will be non-zero (all packets must have at least one fragment). If nCount is zero, this is an acknowledgement (see below).

== Fragmentation ==

Large packets must be fragmented before they can be sent through most network
interfaces. Different network media have different MTUs, and it is difficult to predict
what the lowest common size will be. Fragmentation and reassembly is performed by
the existing Internet protocols, however, there are two important reasons why the
reliability layer performs its own fragmentation:

* Sockets implementations specify a maximum datagram size. This is adequate for the vast majority of transmissions, but it is desirable to have the transparent ability to send larger packets without worrying about the host implementation.
* When the Internet protocols fragment, a packet and one or more fragments are lost, it may decide to discard the whole packet in an unreliable datagram protocol. The Gnutella2 reliability layer can compensate by retransmitting the whole packet, which would then be re-fragmented and each fragment resent - however, this wastes the fragments that were successfully received before. Managing fragmentation natively allows this optimisation.

Each node determines its own MTU, often based on a best guess combined with
information from the host's sockets implementation. Packets exceeding this size are
fragmented into multiple datagrams of the appropriate size. Each datagram has the
same sequence number and the same fragment count (nCount), but a different
fragment number (nPart).

== Transmission Process ==

When a packet is to be transmitted, the network layer must:

* Cache the payload
* Allocate a new locally and temporally unique sequence number
* Derive the appropriate number of fragments
* Queue the fragments for dispatch
* If the fragments do not need to be acknowledged, the packet can be flushed now

The payload will generally be cached for an appropriate timeout period, or until the
data cache becomes full, at which time older payloads can be discarded. Fragments
are dispatched according to the dispatch algorithm of choice, and the sender listens
for acknowledgements. When an acknowledgement is received:

* Lookup the sent packet by sequence number
* Mark the nPart fragment as received and cancel any retransmissions of this part
* If all fragments have been acknowledged, flush this packet from the cache

If a fragment has been transmitted but has not been acknowledged within the
timeout, it should be retransmitted. A finite number of retransmissions are allowed
before the packet as a whole expires, at which time it is assumed that the packet
was not received.

== Receive Process ==

When a new datagram is received, the network layer must:

* If the acknowledge bit was set, send an acknowledge packet for this sequence number and part number, with nCount set to zero (ack)
* Lookup any existing packet by the sending IP and sequence number
* If there is no existing packet, create a new packet entry with the IP, sequence number, fragment count and flags
* If there was an existing packet, make sure it is not marked as done - if so, abort
* Add the transmitted fragment to the (new or old) packet entry
* If the packet now has all fragments, mark it as done and decode it and pass it up to the application layer
* Leave the packet on the cache even if it was finished, in case any parts are retransmitted
* Expire old packets from the receive buffer after a timeout, or if the buffer is full

== Dispatch Algorithm ==

Fragment datagrams need to be dispatched intelligently, to spread the load on
network resources and maximise the chance that the receiver will get the message.
To do this, the dispatch algorithm should take into account several points:

* Prioritize acknowledgements.
* If fragments are waiting to be sent to a number of hosts, do not send to the same host twice in a row. Alternating, or looping through the target hosts achieves the same data rate locally, but spreads out the load over downstream links.
* Do not exceed or approach the capacity of the local network connection. If a host has a 128 kb/s outbound bandwidth, dispatching 32 KB of data in one second will likely cause massive packet loss, leading to a retransmission.
* After considering the above points, prefer fragments that were queued recently to older packets. A LIFO or stack type approach means that even if a transmitter is becoming backed up, some fragments will get there on time, while others will be delayed. A FIFO approach would mean that a backed up host delivers every fragment late.

== Parameters ==
The recommended parameters for the reliability layer are as follows:

<pre>
<nowiki>
MTU = 500 bytes
Transmit Retransmit Interval = 10 seconds
Transmit Packet Timeout / Expire = 26 seconds (allows for two retransmissions before expiry)
Receive Packet Expiry = 30 seconds (allows 10 seconds beyond final retransmission)
</nowiki>
</pre>

== Performance Considerations ==

Relatively low-level network implementations such as this are reasonably
complicated, but must operate fast. It is desirable to avoid runtime memory
allocations in network code as much as possible, and particularly at this level.

It should be noted that in almost all cases, transmissions to "untested" nodes are
single fragment. Replies on the other hand are often larger, and may be deflated in
many fragments. This is optimal, because attempting to contact a node which may be
unavailable, involves a retransmission of only a single fragment.

Flow control is an important topic, however, it is handled at a higher layer. The UDP
reliability layer is only responsible for guaranteeing delivery of selected datagrams.

Only critical transmissions whose reception cannot otherwise be inferred, should have
the acknowledge request bit set.

TCP Stream Connection and Handshaking

2005-04-02T22:21:33Z

Spooky: Nav element added

[[UDP_Transceiver|UDP Transceiver >>]] | [[Main_Page|Main Page]]

== Introduction ==

TCP stream connections are established between Gnutella2 nodes when they elect to
form a permeant link, creating the fundamental network topology of a highly
interconnected hub network serving dense clusters of leaf nodes.

== Initiation ==

TCP connections are initiated by leaf or hub nodes in an attempt to gain a connection
to a hub node. Leaf nodes are never the target of an outbound connection. The TCP
port number is not standardised, and must be stored with the IP address.

== Handshaking ==

Upon the establishment of a TCP stream connection between two Gnutella2 nodes, a
handshaking phase must be completed to negotiate the nature of the link and
exchange other necessary information.

This handshaking phase is the only part of the communication which remains
compatible with the old Gnutella network, allowing new connections to be negotiated
without fore-knowledge the capabilities of the other node. The handshaking process
has been well documented elsewhere, however, a short summary is provided here.

=== Handshake Stages ===

The Gnutella handshake process consists of three header blocks. The node which
initiated the connection sends an initial header block, of the form:

<pre>
<nowiki>
GNUTELLA CONNECT/0.6
Listen-IP: 1.2.3.4:6346
Remote-IP: 6.7.8.9
User-Agent: Shareaza 1.8.2.0
Accept: application/x-gnutella2
X-Ultrapeer: False
</nowiki>
</pre>

The receiver then responds with its own header block:

<pre>
<nowiki>
GNUTELLA/0.6 200 OK
Listen-IP: 6.7.8.9:6346
Remote-IP: 1.2.3.4
User-Agent: Shareaza 1.8.2.0
Content-Type: application/x-gnutella2
Accept: application/x-gnutella2
X-Ultrapeer: True
X-Ultrapeer-Needed: False
</nowiki>
</pre>

Finally, the initiator accepts the receiver's header block, and provides any final
information:

<pre>
<nowiki>
GNUTELLA/0.6 200 OK
Content-Type: application/x-gnutella2
X-Ultrapeer: False
The latter two stages may be replaced with an error condition if the connection is
being rejected. Appropriate error status codes are returned in this case, for example:
GNUTELLA/0.6 503 Too many connections
(more headers)
</nowiki>
</pre>

Note that only the HTTP-style error code should be interpreted by the software: any
descriptive text provided is for display purposes only and is not standardised.

== Headers ==

Important headers which are required or strongly recommended are detailed in the
following sections.

=== Addressing Headers ===

Two important headers to send on all connections are "Remote-IP" and "Listen-IP".
Both of these headers should be sent on the first transmission, meaning in the first
and second header blocks in the three block exchange.

The Remote-IP header contains the IP address from which the remote host is
connecting. This allows a remote host operating through some kind of network
address translation system, to learn its effective external address.

The Listen-IP header contains the IP address and port number that the local host is
listening for inbound TCP connections on. It should be listening for UDP datagrams
on the same port. The format of this header is "IP:PORT", eg "1.2.3.4:6346".

=== Identification ===

The User-Agent header is used to identify the client software operating on the
sending node. It should be sent on the first transmission, meaning in the first and
second header blocks in the three block exchange. Note that this is a descriptive
string that often includes a version number, and is not a "vendor code" as described
elsewhere.

=== Content Type (Protocol) ===

The Accept and Content-Type header exchange is used to negotiate the data
protocol that will be used in the connection, in this case Gnutella2. The Gnutella2
content type is "application/x-gnutella2", and this exchange follows standard HTTP
rules for negotiating content type.

The first step is to advertise support for the content type (Gnutella2) in the first
header block with "Accept: application/x-gnutella2". The responding node will then
indicate that it will send Gnutella2 content with "Content-Type: application/xgnutella2",
and that it also supports Gnutella2 with "Accept: application/x-gnutella2".
The initiating host then confirms that it will be sending Gnutella2 with "Content-
Type: application/x-gnutella2" in the third header block. For more information on the
Accept/Content-Type exchange, consult a HTTP reference.

Note that the content type negotiation process is designed to be a "one-way"
process, i.e. a different content type can be negotiated for sending and receiving.
However, when the gnutella2 protocol is negotiated, both channels must use the
same content type. This means that a receiving node must not accept Gnutella2 if
the initiator did not advertise support for it, and if at the end of the handshake,
bidirectional Gnutella2 was not negotiated, the connection should be terminated.

=== Node State Negotiation ===

There are two node types in a Gnutella2 peer to peer network, a hub and a leaf, as
described in [[Node Types and Responsibilities]]. During the initial handshake,
the two parties must exchange their current node type and advise of their
capabilities, negotiating the node types they will adopt when the connection
completes, and indeed whether it should complete at all.

As the handshake sequence is compatible with Gnutella1, the headers involved in
negotiating node types are identical to those used to negotiate Gnutella1 "Ultrapeer"
states:

<pre>
<nowiki>
X-Ultrapeer: [True|False]
X-Ultrapeer-Needed: [True|False]
</nowiki>
</pre>

Both headers contain a Boolean value, "true" or "false", case insensitive.

The X-Ultrapeer header indicates whether the transmitting node is currently
operating as a hub. Hub nodes will send "X-Ultrapeer: True" while leaf nodes will
send "X-Ultrapeer: False".

The X-Ultrapeer-Needed header indicates whether the transmitting node would like
(and allow) the receiver to be a hub. A hub which sees no need for additional hubs in
its area of the network, will send "X-Ultrapeer-Needed: False", indicating to the
connecting node that it must operate in leaf mode if it wishes to connect. A hub
which sees a need for additional hubs, will send "X-Ultrapeer-Needed: True",
indicating that the receiving node should become a hub, if it is capable of doing so. A
leaf may send "X-Ultrapeer-Needed: True" to indicate that it is seeking a connection
to a hub.

The X-Ultrapeer header should be sent on all three of the header blocks to indicate
the current intended state of the node, while the X-Ultrapeer-Needed header should
be sent in the first two header blocks only, to indicate the desired status of the
receiver. If the nodes cannot agree on a satisfactory arrangement, the connection
will be terminated at or prior to the third header block with an appropriate message,
for example:

<pre>
<nowiki>
GNUTELLA/0.6 503 Too many hub connections
GNUTELLA/0.6 503 Too many leaves
GNUTELLA/0.6 503 I have leaves, can't downgrade to leaf mode
GNUTELLA/0.6 503 Leaf mode disabled
</nowiki>
</pre>

=== Hub Address Exchange ===

It is desirable for connecting nodes to exchange the node addresses of other hubs on
the network to facilitate rapid connection. The Gnutella2 protocol includes highly
efficient methods to share hub addresses with peers once connected, but for a node
trying to connect and encountering only "full" hubs, learning new hubs to try is
helpful.

The X-Try-Ultrapeers header was developed for this purpose, and like the rest of
the handshake is also semi-compatible with Gnutella1. It contains a comma
separated list of hub node addresses and ports, along with a timestamp recording
the time the hub was last seen. For example:

<pre>
<nowiki>
X-Try-Ultrapeers: 1.2.3.4:6346 2003-03-25T23:59Z, [..more..]
</nowiki>
</pre>

Hub addresses should not be sent unless the transmitter has reasonable knowledge
of the hub's existence and the timestamp is not too old. The content type negotiation
headers should be used to verify that the listed hub addresses are indeed Gnutella2
hubs.

=== Compression Negotiation and Encoding ===

The Gnutella2 architecture makes widespread use of "deflate" compression, due to
its high availability and ease of integration. Support of compressed TCP links is not a
requirement in the Gnutella2 standard, however, it is strongly recommended.

Deflate compression of a TCP link is negotiated with the pair standard HTTP headers
"Accept-Encoding" and "Content-Encoding". For example:

Header block one (initiator):

<pre>
<nowiki>
Accept-Encoding: deflate
</nowiki>
</pre>

Header block two (receiver):

<pre>
<nowiki>
Accept-Encoding: deflate
Content-Encoding: deflate
</nowiki>
</pre>

Header block three (initiator):

<pre>
<nowiki>
Content-Encoding: deflate
</nowiki>
</pre>

In this example, the initiator advertises support for receiving a deflated connection.
The receiver then indicates that it too supports receiving a deflated connection, and
that it intends to transmit deflated data. Finally, the initiator upon noting that the
receiver supports deflate, indicates that it too will transmit deflated data.

Note that unlike the content-type / protocol negotiation, deflated encoding can be
applied on either incoming, outgoing, or both channels of a connection.

For performance reasons, nodes should consider whether they can afford to support
additional deflated connections before advertising support for them, or agreeing to
provide a deflated data stream. In the Gnutella2 network topology, all links benefit
from compression, except the leaf to hub channel of the leaf/hub link. Exempting this
channel from compression, saves the leaf and more importantly, the hub, a
considerable CPU and RAM investment.

== Post Handshake Communication ==

After the third and final header block has been received by the initiator, subsequent
communication over the TCP stream occurs in the negotiated protocol, with the
negotiated encoding. This means that while the handshake sequence was backwards
compatible with Gnutella1, after Gnutella2 support has been negotiated, all
subsequent communication occurs in the Gnutella2 common protocol - an entirely
new system not backwards compatible with any other protocol.

Server for Uploading

2005-04-02T22:17:35Z

Spooky: typo

This is not a core part of the Gnutella2 architecture, and is discussed in depth
elsewhere. {TODO: it should still be documented here eventually or link to appropriate documents}

Filetransfer is basically the same as in the original Gnutella protocol. See the documentation [http://www.the-gdf.org/wiki/index.php?title=File_Transfer_%28was:_Requesting_a_download%29 here]

[[X-Queue|Gnutella2's Queue System (X-Queue)]]

Client for Downloading

2005-04-02T22:17:03Z

Spooky: typo

Client for Downloading

2005-04-02T22:16:39Z

Spooky: Added link to GDF docs

This is not a core part of the Gnutella2 architecture, and is discussed in depth
elsewhere. {TODO: it should still be documented here eventually or link to appropriate documents}

Filetransfer is basically the same as in the original Gnutella protocol. See the ducumentation [http://www.the-gdf.org/wiki/index.php?title=File_Transfer_%28was:_Requesting_a_download%29 here]

Server for Uploading

2005-04-02T22:15:47Z

Spooky: Link to GDF docs added

This is not a core part of the Gnutella2 architecture, and is discussed in depth
elsewhere. {TODO: it should still be documented here eventually or link to appropriate documents}

Filetransfer is basically the same as in the original Gnutella protocol. See the ducumentation [http://www.the-gdf.org/wiki/index.php?title=File_Transfer_%28was:_Requesting_a_download%29 here]

[[X-Queue|Gnutella2's Queue System (X-Queue)]]

Help:Editing

2005-04-02T22:13:45Z

Spooky: typo

This is a copy of the How to edit pages page from Wikipedia. Since both are based on [[Wikipedia:MediaWiki|MediaWiki]] it also applies to the Gnutella2 Developer Network wiki.

[[Main_Page|Gnutella 2 Developer Network]] is a [[Wikipedia:WikiWiki|Wiki]], which means that anyone can easily edit any [[Wikipedia:Protected page|unprotected]] [[Wikipedia:What is an article|article]] and have those changes posted immediately to that page.

Editing a Wiki page is very easy. Simply click on the "'''Edit this page'''" tab at the top (or the edit link on the right or bottom) of a Wiki page. This will bring you to a page with a text box containing the editable text of that page. '''If you just want to experiment, please do so in the [[Wikipedia:sandbox|sandbox]]''', not here. You should write a short [[Wikipedia:Edit summary|edit summary]] in the small field below the edit-box. You may use shorthand to describe your changes, as described in the [[Wikipedia:Edit summary legend|legend]], and when you've finished, press [[Wikipedia:Show preview|preview]] to see how your changes will look. If you're happy with what you see, then press "Save" and your changes will be immediately applied to the article.

You can also click on the "'''Discussion'''" tab to see the corresponding [[Wikipedia:Talk page|talk page]], which contains comments about the page from other Wikipedia users. Click on the "'''+'''" tab to add a new section, or edit the page in the same way as an article page.

== Minor edits ==
''See also [[Wikipedia:Minor edit]]''

When editing a page, a [[Wikipedia:How to log in|logged-in]] user can mark that edit as being "minor". Minor edits generally mean spelling corrections, formatting, and minor rearrangement of text. It is possible to ''hide'' minor edits when viewing [[Wikipedia:Recent Changes]]. Marking a significant change as a minor edit is considered bad behavior, and even more so if it involves the deletion of some text. If one has accidentally marked an edit as minor, the person should edit the source once more, mark it major (or, rather, ensure that the check-box for "This is a minor edit" is not checked), and, in the summary, state that the previous change was a major one.

== Wiki markup ==

The '''wiki markup''' is the syntax system you can use to format a Wikipedia page.

In the left column of the table below, you can see what effects are possible. In the right column, you can see how those effects were achieved. In other words, to make text look like it looks in the left column, type it in the format you see in the right column.

You may want to keep this page open in a separate browser window for reference. '''If you want to try out things without danger of doing any harm, you can do so in the [[Wikipedia:Sandbox|Sandbox]]'''. Try opening the Sandbox in a separate window or tab and keeping this page open for reference.

----

=== Sections, paragraphs, lists and lines ===
{| border="1" cellpadding="2" cellspacing="0"
|-
! What it looks like
! What you type
|- valign="top"
|
Start your [[Wikipedia:Manual of Style (headings)|sections]] as follows:




'''<font style="font-size:120%">New section</font>'''

'''<font style="font-size:110%">Subsection</font>'''

'''<font style="font-size:100%">Sub-subsection</font>'''

* Start with a second-level heading (<tt><nowiki>==</nowiki></tt>); do not use first-level headings (=).
* Do not skip levels (e.g., second-level followed by fourth-level).
* A [[#Placement_of_the_Table_of_Contents_.28TOC.29|Table of Contents]] will automatically be added to an article that has four or more sections.
|
<pre><nowiki>
==New section==

===Subsection===

====Sub-subsection====
</nowiki></pre>
|- valign="top"
|
A single [[newline]]
generally has no effect on the layout.
These can be used to separate
sentences within a paragraph.
Some editors find that this aids editing
and improves the function ''diff''
(used internally to compare
different versions of a page).

But an empty line
starts a new paragraph.

* When used in a list, a newline ''does'' affect the layout ([[#lists|see below]]).
|
<pre><nowiki>
A single [[newline]]
generally has no effect on the layout.
These can be used to separate
sentences within a paragraph.
Some editors find that this aids editing
and improves the function ''diff''
(used internally to compare
different versions of a page).

But an empty line
starts a new paragraph.
</nowiki></pre>
|- valign="top"
|
You can break lines<br>
without starting a new paragraph.

* Please use this sparingly.
* Close markup between lines, don't start a [[link]] or ''italics'' or '''bold''' on one line and close it on the next.
|
<pre><nowiki>
You can break lines<br>
without starting a new paragraph.
</nowiki></pre>
|- id="lists" valign="top"
|
* Lists are easy to do:
** Start every line with a star (= [[asterisk]]).
*** More stars means deeper levels.
**** A newline in a list
marks the end of a list item.

* An empty line starts a new list.
|
<pre><nowiki>
* Lists are easy to do:
** Start every line with a star.
*** More stars means deeper levels.
**** A newline in a list
marks the end of a list item.

* An empty line starts a new list.
</nowiki></pre>
|- valign="top"
|
# Numbered lists are also good
## very organized
## easy to follow
### easier still
|
<pre><nowiki>
# Numbered lists are also good
## very organized
## easy to follow
### easier still
</nowiki></pre>
|- valign="top"
|
* You can even do mixed lists
*# and nest them
*#* like this
|
<pre><nowiki>
* You can even do mixed lists
*# and nest them
*#* like this
</nowiki></pre>
|- valign="top"
|
; Definition list : list of definitions
; item : the item's definition
; another item
: the other item's definition

* One item per line; a newline can appear before the colon, but using a space before the colon improves parsing.
|
<pre><nowiki>
; Definition list : list of definitions
; item : the item's definition
; another item
: the other item's definition
</nowiki></pre>
|- valign="top"
|
: A colon indents a line or paragraph.
A manual newline starts a new paragraph.

* This is primarily for displayed material, but is also used for discussion on [[Wikipedia:Talk page|Talk page]]s.
|
<pre><nowiki>
: A colon indents a line or paragraph.
A manual newline starts a new paragraph.
</nowiki></pre>
|- valign=top
|
IF a line starts with a space THEN
it will be formatted exactly
as typed;
in a fixed-width font;
lines won't wrap;
ENDIF

*This is useful for:
** pasting preformatted text;
** algorithm descriptions;
** program source code;
** [[ASCII art]];
** chemical structures;
* '''WARNING''': If you make it wide, you [[page widening|force the whole page to be wide]] and hence less readable, especially for people who use lower resolutions. Never start ordinary lines with spaces.
|
<pre><nowiki>
IF a line starts with a space THEN
it will be formatted exactly
as typed;
in a fixed-width font;
lines won't wrap;
ENDIF
</nowiki></pre>
|- valign="top"
|
<center>Centered text.</center>

* Note the American spelling of "center".
|
<pre><nowiki>
<center>Centered text.</center>
</nowiki></pre>
|- valign="top"
|
A [[horizontal dividing line]]:
this is above it
----
and this is below it.

* Mainly useful for separating threads on Talk pages.
* Also used to [[Wikipedia:Disambiguation|disambiguate]] within an article without creating a separate page.
|
<pre><nowiki>
A [[horizontal dividing line]]:
this is above it
----
and this is below it.
</nowiki></pre>
|}

=== Links and URLs ===
{| border="1" cellpadding="2" cellspacing="0"
|-
! What it looks like
! What you type
|- valign="top"
|
London has [[public transport]].

* A link to another Wikipedia article.
* Internally, the first letter of the target page is automatically capitalized and spaces are represented as underscores (typing an underscore in the link has the same effect as typing a space, but is not recommended).
* Thus the link above is to the [[URL]] http://www.wikipedia.org/wiki/Public_transport, which is the Wikipedia article with the name "Public transport". See also [[Wikipedia:Canonicalization]].
|
<pre><nowiki>
London has [[public transport]].
</nowiki></pre>
|- valign="top"
|
San Francisco also has
[[public transport|public transportation]].

* Same target, different name.
* This is a [[piped link]].
|
<pre><nowiki>
San Francisco also has
[[public transport|public transportation]].
</nowiki></pre>
|- valign="top"
|
San Francisco also has
[[public transport]]ation.

Examples include [[bus]]es, [[taxi]]s
and [[streetcar]]s.

* Endings are blended into the link.
* Preferred style is to use this instead of a piped link, if possible.
|
<pre><nowiki>
San Francisco also has
[[public transport]]ation.

Examples include [[bus]]es, [[taxi]]s
and [[streetcar]]s.
</nowiki></pre>
|- valign="top"
|
See the [[Wikipedia:Manual of Style]].

* A link to another [[Wikipedia:namespace|namespace]].
|
<pre><nowiki>
See the [[Wikipedia:Manual of Style]].
</nowiki></pre>
|- id="link-to-section" valign="top"
|
[[Economics#See also]] is a link
to a section within another page.

[[#Links and URLs]] is a link
to a section on the current page.

[[#example]] is a link to an
anchor that was created using
<div id="example">an id attribute</div>

* The part after the number sign (#) must match a section heading on the page. Matches must be exact in terms of spelling, case and punctuation. Links to non-existent sections aren't broken; they are treated as links to the top of the page.
* Identifiers may be created by attaching an <code>id="..."></code> attribute to almost any HTML element.
|
<pre><nowiki>
[[Economics#See also]] is a link
to a section within another page.

[[#Links and URLs]] is a link
to a section on the current page.

[[#example]] is a link to an
anchor that was created using
<div id="example">an id attribute</div>
</nowiki></pre>
|- valign="top"
|
Automatically hide stuff in parentheses:
[[kingdom (biology)|kingdom]].

Automatically hide namespace:
[[Wikipedia:Village Pump|Village Pump]].

Or both:
[[Wikipedia:Manual of Style (headings)|Manual of Style]]

But not:
[[Wikipedia:Manual of Style#Links|]]

* The server fills in the part after the pipe character (|) when you save the page. The next time you open the edit box you will see the expanded piped link. When [[Wikipedia:Show preview|preview]]ing your edits, you will not see the expanded form until you press '''Save''' and '''Edit''' again. The same applies to links to sections within the same page ([[#link-to-section|see previous entry]]).
|
<pre><nowiki>
Automatically hide stuff in parentheses:
[[kingdom (biology)|]].

Automatically hide namespace:
[[Wikipedia:Village Pump|]].

Or both:
[[Wikipedia:Manual of Style (headings)|]]

But not:
[[Wikipedia:Manual of Style#Links|]]
</nowiki></pre>
|- valign="top"
|
[[The weather in London]] is a page
that doesn't exist yet.

* You can create it by clicking on the link (but please don't do so with this particular link).
* To create a new page:
*# Create a link to it on some other (related) page.
*# Save that page.
*# Click on the link you just made. The new page will open for editing.
* For more information, see [[Wikipedia:How to start a page|How to start a page]] and check out Wikipedia's [[Wikipedia:Naming conventions|naming conventions]].
* Please do not create a new article without linking to it from at least one other article.
|
<pre><nowiki>
[[The weather in London]] is a page
that doesn't exist yet.
</nowiki></pre>
|- valign="top"
|
[[Wikipedia:How to edit a page]] is this page.

* [[Self link]]s appear as bold text when the article is viewed.
* Do not use this technique to make the article name bold in the first paragraph; see the [[Wikipedia:Manual of Style#Article titles|Manual of Style]].
|
<pre><nowiki>
[[Wikipedia:How to edit a page]] is this page.
</nowiki></pre>
|- valign="top"
|
When adding a comment to a Talk page,
you should sign it by adding
three tildes to add your user name:
: [[User:Brockert|Ben Brockert]]
or four to add user name plus date/time:
: [[User:Brockert|Ben Brockert]] 00:18, Nov 19, 2004 (UTC)
Five tildes gives the date/time alone:
: 00:18, Nov 19, 2004 (UTC)

* The first two both provide a link to your [[Wikipedia:user page|user page]].
|
<pre><nowiki>
When adding a comment to a Talk page,
you should sign it by adding
three tildes to add your user name:
: ~~~
or four for user name plus date/time:
: ~~~~
Five tildes gives the date/time alone:
: ~~~~~
</nowiki></pre>
|- valign="top"
|
* [[Wikipedia:Redirect|Redirect]] one article title to another by placing a directive like the one shown to the right on the ''first'' line of the article (such as at a page titled "[[USA]]").
* Note that, while it is possible to link to a section, it is not possible to redirect to a section. For example, "<nowiki>#REDIRECT [[United States#History]]</nowiki>" will redirect to the [[United States]] page, but not to any particular section on it. This feature '''will not''' be implemented in the future, so such redirects should not be used.
|
<pre><nowiki>
#REDIRECT [[United States]]
</nowiki></pre>
|- valign="top"
|
* Link to a page on the same subject in another language by using a link of the form: <nowiki>[[language code:Title]]</nowiki>.
* It does not matter where you put these links while editing as they will always show up in the same place when you save the page, but placement at the end of the edit box is recommended.
* Please see [[Wikipedia:Interlanguage links]] and the [[Wikipedia:Complete list of language wikis available|list of languages and codes]].
|
<pre><nowiki>
[[fr:Wikipédia:Aide]]
</nowiki></pre>
|- valign="top"
|
'''What links here''' and '''Related changes'''
pages can be linked as:
[[Special:Whatlinkshere/Wikipedia:How to edit a page]]
and
[[Special:Recentchangeslinked/Wikipedia:How to edit a page]]

|
<pre><nowiki>
'''What links here''' and '''Related changes'''
pages can be linked as:
[[Special:Whatlinkshere/Wikipedia:How to edit a page]]
and
[[Special:Recentchangeslinked/Wikipedia:How to edit a page]]
</nowiki></pre>
|- valign="top"
|
A user's '''Contributions''' page can be linked as:
[[Special:Contributions/UserName]]
or
[[Special:Contributions/192.0.2.0]]
|
<pre><nowiki>
A user's '''Contributions''' page can be linked as:
[[Special:Contributions/UserName]]
or
[[Special:Contributions/192.0.2.0]]
</nowiki></pre>
|- valign="top"
|
* To put an article in a [[Wikipedia:Category]], place a link like the one to the right anywhere in the article. As with interlanguage links, it does not matter where you put these links while editing as they will always show up in the same place when you save the page, but placement at the end of the edit box is recommended.
|
<pre><nowiki>
[[Category:Character sets]]
</nowiki></pre>
|- valign="top"
|
* To ''link'' to a [[Wikipedia:Category]] page without putting the article into the category, use an initial colon (:) in the link.
|
<pre><nowiki>
[[:Category:Character sets]]
</nowiki></pre>
|- id="link-external" valign="top"
|
Three ways to link to external (non-wiki) sources:
# Bare URL: http://www.nupedia.com/ (bad style)
# Unnamed link: [http://www.nupedia.com/] ('''very bad style''')
# Named link: [http://www.nupedia.com Nupedia]

:See [[MetaWikiPedia:Interwiki_map]] for the list of shortcuts.

* Square brackets indicate an external link. Note the use of a ''space'' (not a pipe) to separate the URL from the link text in the "named" version.
* In the [[URL]], all symbols must be among:<br/>'''A-Z a-z 0-9 . _ \ / ~ % - + & # ? ! = ( ) @ \x80-\xFF'''
* If a URL contains a character not in this list, it should be encoded by using a percent sign (%) followed by the [[hexadecimal|hex]] code of the character, which can be found in the table of [[ASCII#ASCII printable characters|ASCII printable characters]]. For example, the caret character (^) would be encoded in a URL as '''%5E'''.
* See [[Wikipedia:External links]] for style issues.
|
<pre><nowiki>
Three ways to link to external (non-wiki) sources:
# Bare URL: http://www.nupedia.com/ (bad style)
# Unnamed link: [http://www.nupedia.com/] ('''very bad style''')
# Named link: [http://www.nupedia.com Nupedia]
</nowiki></pre>
|- valign="top"
|
Linking to other wikis:
# [[Interwiki]] link: [[Wiktionary:Hello]]
# Named interwiki link: [[Wiktionary:Hello|Hello]]
# Interwiki link without prefix: [[Wiktionary:Hello|Hello]]

* All of these forms lead to the URL http://en.wiktionary.org/wiki/Hello
* Note that interwiki links use the ''internal'' link style.
* See [[MetaWikiPedia:Interwiki_map]] for the list of shortcuts; if the site you want to link to isn't on the list, use an external link ([[#link-external|see above]]).
* See also [[Wikipedia:How to link to Wikimedia projects]].

Linking to another language's wiktionary:
# [[Wiktionary:fr:bonjour]]
# [[Wiktionary:fr:bonjour|bonjour]]
# [[Wiktionary:fr:bonjour|fr:bonjour]]

* All of these forms lead to the URL http://fr.wiktionary.org/wiki/bonjour
|
<pre><nowiki>
Linking to other wikis:
# [[Interwiki]] link: [[Wiktionary:Hello]]
# Named interwiki link: [[Wiktionary:Hello|Hello]]
# Interwiki link without prefix: [[Wiktionary:Hello|]]

Linking to another language's wiktionary:
# [[Wiktionary:fr:bonjour]]
# [[Wiktionary:fr:bonjour|bonjour]]
# [[Wiktionary:fr:bonjour|]]
</nowiki></pre>
|- valign="top"
|
ISBN 012345678X

ISBN 0-123-45678-X

* Link to books using their [[Wikipedia:ISBN|ISBN]] numbers. This is preferred to linking to a specific online bookstore, because it gives the reader a choice of vendors.
* ISBN links do not need any extra markup, provided you use one of the indicated formats.
|
<pre><nowiki>
ISBN 012345678X

ISBN 0-123-45678-X
</nowiki></pre>
|- valign=top
|
Date formats:
# [[July 20]], [[1969]]
# [[20 July]] [[1969]]
# [[1969]]-[[07-20]]

* Link dates in one of the above formats, so that everyone can set their own display order. If [[Special:Userlogin|logged in]], you can use [[Special:Preferences]] to change your own date display setting.
* All of the above dates will appear as "[[20 July|20 July]] [[1969|1969]]" if you set your date display preference to "15 January 2001", but as "[[20 July|July 20]], [[1969|1969]]" if you set it to "January 15, 2001".
|
<pre><nowiki>
Date formats:
# [[July 20]], [[1969]]
# [[20 July]] [[1969]]
# [[1969]]-[[07-20]]
</nowiki></pre>
|- valign="top"
|
[[media:Sg_mrob.ogg|Sound]]

*To include links to non-image uploads such as sounds, use a "media" link. For images, [[#Images|see next section]].

Some uploaded sounds are listed at [[Wikipedia:Sound]].
|
<pre><nowiki>
[[media:Sg_mrob.ogg|Sound]]
</nowiki></pre>
|}

=== Images ===
{| border="1" cellpadding="2" cellspacing="0"
|-
! What it looks like
! What you type
|- valign="top"
| A picture: [[Image:Wiki.png]]

or, with alternative text:
[[Image:Wiki.png|jigsaw globe]]

or, floating to the right side of the page and with a caption:
[[Image:Wiki.png|frame|Wikipedia Encyclopedia]]<br clear=all>

or, floating to the right side of the page ''without'' a caption:
[[Image:Wiki.png|right|Wikipedia Encyclopedia]]<br clear=all>

* Only images that have been uploaded to Wikipedia can be used. To upload images, use the [[Special:Upload|upload page]]. You can find the uploaded image on the [[Special:Imagelist|image list]].
* See the [[Wikipedia:Image use policy|image use policy]] and [[Wikipedia:Image markup|extended image markup/syntax]] pages for more hints.
* Alternative text, used when the image isn't loaded, in a text-only browser, or when spoken aloud, is '''strongly''' encouraged. See [[Wikipedia:Alternate text for images|Alternate text for images]] for help on choosing it.
* The frame tag automatically floats the image right.
| <pre><nowiki>
A picture: [[Image:Wiki.png]]</nowiki>

or, with alternative text:
<nowiki>[[Image:Wiki.png|jigsaw globe]]</nowiki>

or, floating to the right side of the page and with a caption:
<nowiki>[[Image:Wiki.png|frame|Wikipedia Encyclopedia]]</nowiki>

or, floating to the right side of the page ''without'' a caption:
<nowiki>[[Image:Wiki.png|right|Wikipedia Encyclopedia]]</nowiki></pre>
|-
|
Clicking on an uploaded image displays a description page, which you can also link directly to: [[:Image:Wiki.png]]
| <pre><nowiki>

[[:Image:Wiki.png]]
</nowiki></pre>
|-
|
To include links to images shown as links instead of drawn on the page, use a "media" link.

<br/>[[media:Tornado.jpg|Image of a Tornado]]
|
<pre>
<nowiki>

[[media:Tornado.jpg|Image of a Tornado]]

</nowiki></pre>
|}

=== Character formatting ===
{| border="1" cellpadding="2" cellspacing="0"
|- valign="top"
! What it looks like
! What you type
|- id="emph" valign="top"
|
''Emphasize'', '''strongly''', '''''very strongly'''''.
* These are double, triple, and quintuple apostrophes (single-quote marks), not double-quote marks.
|
<pre><nowiki>
''Emphasize'', '''strongly''', '''''very strongly'''''.
</nowiki></pre>
|- valign="top"
|
<math>\sin x + \ln y</math><br>
sin''x'' + ln''y''


<math>\mathbf{x} = 0</math><br>
'''x''' = 0

Ordinary text should use [[#emph|wiki markup for emphasis]], and should not use <code><i></code> or <code><b></code>. However, mathematical formulas often use italics, and sometimes use bold, for reasons unrelated to emphasis. Complex formulas should use [[Help:Formula|<code><math></code> markup]], and simple formulas may use <code><math></code>; or <code><i></code> and <code><b></code>; or <code><nowiki>''</nowiki></code> and <code><nowiki>'''</nowiki></code>. According to [[Wikipedia:WikiProject Mathematics#Italicization and bolding|WikiProject Mathematics]], wiki markup is preferred over HTML markup like <code><i></code> and <code><b></code>.
|
<pre><nowiki>
<math>\sin x + \ln y</math>
sin''x'' + ln''y''

<math>\mathbf{x} = 0</math>
'''x''' = 0
</nowiki></pre>
|- valign="top"
|
A typewriter font for <tt>monospace text</tt>
or for computer code: <code>int main()</code>

* For semantic reasons, using <code><code></code> where applicable is preferable to using <code><tt></code>.
|
<pre><nowiki>
A typewriter font for <tt>monospace text</tt>
or for computer code: <code>int main()</code>
</nowiki></pre>
|- valign="top"
|
You can use <small>small text</small> for captions.
|
<pre><nowiki>
You can use <small>small text</small> for captions.
</nowiki></pre>
|- valign="top"
|
You can <s>strike out deleted material</s>
and <u>underline new material</u>.

You can also mark <del>deleted material</del> and
<ins>inserted material</ins> using logical markup
rather than visual markup.

* When editing regular Wikipedia articles, just make your changes and don't mark them up in any special way.
* When editing your own previous remarks in talk pages, it is sometimes appropriate to mark up deleted or inserted material.
|
<pre><nowiki>
You can <s>strike out deleted material</s>
and <u>underline new material</u>.

You can also mark <del>deleted material</del> and
<ins>inserted material</ins> using logical markup
rather than visual markup.
</nowiki></pre>
|- valign="top"
|
'''Diacritical marks:'''
<br/>
À Á Â Ã Ä Å <br/>
Æ Ç È É Ê Ë <br/>
Ì Í
Î Ï Ñ Ò <br/>
Ó Ô Õ
Ö Ø Ù <br/>
Ú Û Ü ß
à á <br/>
â ã ä å æ
ç <br/>
è é ê ë ì í<br/>
î ï ñ ò ó ô <br/>
&oelig; õ
ö ø ù ú <br/>
û ü ÿ

* See [[meta:Help:Special characters|special characters]].
|
<br/>
<pre><nowiki>
&Agrave; &Aacute; &Acirc; &Atilde; &Auml; &Aring;
&AElig; &Ccedil; &Egrave; &Eacute; &Ecirc; &Euml;
&Igrave; &Iacute; &Icirc; &Iuml; &Ntilde; &Ograve;
&Oacute; &Ocirc; &Otilde; &Ouml; &Oslash; &Ugrave;
&Uacute; &Ucirc; &Uuml; &szlig; &agrave; &aacute;
&acirc; &atilde; &auml; &aring; &aelig; &ccedil;
&egrave; &eacute; &ecirc; &euml; &igrave; &iacute;
&icirc; &iuml; &ntilde; &ograve; &oacute; &ocirc;
&oelig; &otilde; &ouml; &oslash; &ugrave; &uacute;
&ucirc; &uuml; &yuml;
</nowiki></pre>
|- valign="top"
|
'''Punctuation:'''
<br/>
¿ ¡ § ¶<br/>
&dagger; &Dagger; • – —<br/>
&lsaquo; &rsaquo; « »<br/>
‘ ’ “ ”
|
<br/>
<pre><nowiki>
&iquest; &iexcl; &sect; &para;
&dagger; &Dagger; &bull; &ndash; &mdash;
&lsaquo; &rsaquo; &laquo; &raquo;
&lsquo; &rsquo; &ldquo; &rdquo;
</nowiki></pre>
|- valign="top"
|
'''Commercial symbols:'''
<br/>
™ © ® ¢ € ¥<br/>
£ ¤
|
<br/>
<pre><nowiki>
&trade; &copy; &reg; &cent; &euro; &yen;
&pound; &curren;
</nowiki></pre>
|- valign="top"
|
'''Subscripts:'''
<br/>
x<sub>1</sub> x<sub>2</sub> x<sub>3</sub> or
<br/>
x₀ x₁ x₂ x₃ x₄
<br/>
x₅ x₆ x₇ x₈ x₉

'''Superscripts:'''
<br/>
x<sup>1</sup> x<sup>2</sup> x<sup>3</sup> or
<br/>
x⁰ x¹ x² x³ x⁴
<br/>
x⁵ x⁶ x⁷ x⁸ x⁹

*The latter methods of sub/superscripting can't be used in the most general context, as they rely on Unicode support which may not be present on all users' machines. For the 1-2-3 superscripts, it is nevertheless preferred when possible (as with units of measurement) because most browsers have an easier time formatting lines with it.

ε<sub>0</sub> =
8.85 × 10<sup>−12</sup>
C² / J m.

1 [[hectare]] = [[1 E4 m²]]
|
<br/>
<pre><nowiki>
x<sub>1</sub> x<sub>2</sub> x<sub>3</sub> or
<br/>
x&#8320; x&#8321; x&#8322; x&#8323; x&#8324;
<br/>
x&#8325; x&#8326; x&#8327; x&#8328; x&#8329;
</nowiki></pre>

<pre><nowiki>
x<sup>1</sup> x<sup>2</sup> x<sup>3</sup> or
<br/>
x&#8304; x&sup1; x&sup2; x&sup3; x&#8308;
<br/>
x&#8309; x&#8310; x&#8311; x&#8312; x&#8313;

&epsilon;<sub>0</sub> =
8.85 &times; 10<sup>&minus;12</sup>
C&sup2; / J m.

1 [[hectare]] = [[1 E4 m&sup2;]]
</nowiki></pre>
|- valign="top"
|
'''Greek characters:'''
<br/>
α β γ δ ε ζ<br/>
η θ ι κ λ μ ν<br/>
ξ ο π ρ σ &sigmaf;<br/>
τ υ φ χ ψ ω<br/>
Γ Δ Θ Λ Ξ Π<br/>
Σ Φ Ψ Ω
|
<br/>
<pre><nowiki>
&alpha; &beta; &gamma; &delta; &epsilon; &zeta;
&eta; &theta; &iota; &kappa; &lambda; &mu; &nu;
&xi; &omicron; &pi; &rho; &sigma; &sigmaf;
&tau; &upsilon; &phi; &chi; &psi; &omega;
&Gamma; &Delta; &Theta; &Lambda; &Xi; &Pi;
&Sigma; &Phi; &Psi; &Omega;
</nowiki></pre>
|- valign="top"
|
'''Mathematical characters:'''
<br/>
∫ ∑ ∏ √ − ± ∞<br/>
≈ &prop; &equiv; ≠ ≤ ≥<br/>
× · ÷ ∂ ′ ″<br/>
∇ &permil; ° &there4; &alefsym; ø<br/>
∈ ∉ ∩ ∪ ⊂ ⊃ &sube; &supe;<br/>
¬ &and; &or; ∃ ∀ ⇒ ⇔<br/>
→ ↔<br/>
* See also [[Wikipedia:WikiProject Mathematics|WikiProject Mathematics]].
|
<br/>
<pre><nowiki>
&int; &sum; &prod; &radic; &minus; &plusmn; &infin;
&asymp; &prop; &equiv; &ne; &le; &ge;
&times; &middot; &divide; &part; &prime; &Prime;
&nabla; &permil; &deg; &there4; &alefsym; &oslash;
&isin; &notin; &cap; &cup; &sub; &sup; &sube; &supe;
&not; &and; &or; &exist; &forall; &rArr; &hArr;
&rarr; &harr;
</nowiki></pre>
|- valign="top"
|
'''Spacing in simple math formulas:'''
<br/>
Obviously, ''x''² ≥ 0 is true.
*To space things out without allowing line breaks to interrupt the formula, use non-breaking spaces: <tt>&nbsp;</tt>.
|
<br/>
<pre><nowiki>
Obviously, ''x''&sup2;&nbsp;&ge;&nbsp;0 is true.
</nowiki></pre>
|- valign="top"
|
'''Complicated formulas:'''
<br/>
: <math>\sum_{n=0}^\infty \frac{x^n}{n!}</math>
* See [[Help:Formula]] for how to use <tt><math></tt>.
* A formula displayed on a line by itself should probably be indented by using the colon (:) character.
|
<br/>
<pre><nowiki>
: <math>\sum_{n=0}^\infty \frac{x^n}{n!}</math>
</nowiki></pre>
|- valign="top"
|
'''Suppressing interpretation of markup:'''
<br/>
<nowiki>Link → (''to'') the [[Wikipedia FAQ]]</nowiki>
* Used to show literal data that would otherwise have special meaning.
* Escape all wiki markup, including that which looks like HTML tags.
* Does not escape HTML character references.
* To escape HTML character references such as <tt>&rarr;</tt> use <tt>&amp;rarr;</tt>
|
<br/>
<pre><nowiki>
<nowiki>Link &rarr; (''to'')
the [[Wikipedia FAQ]]</nowiki>
</nowiki></pre>
|- valign="top"
|
'''Commenting page source:'''
<br/>
''not shown when viewing page''
* Used to leave comments in a page for future editors.
* Note that most comments should go on the appropriate [[Wikipedia:Talk page|Talk page]].
|
<br/>
<pre><nowiki>

</nowiki></pre>
|}
''(see also: [[Chess symbols in Unicode]])''

=== Table of Contents ===
==== Placement of the Table of Contents (TOC) ====
At the current status of the wiki markup language, having at least four headers on a page triggers the TOC to appear in front of the first header (or after introductory sections). Putting <nowiki>__TOC__</nowiki> anywhere forces the TOC to appear at that point (instead of just before the first header). Putting <nowiki>__NOTOC__</nowiki> anywhere forces the TOC to disappear. See also [[Wikipedia:Section#Compact_TOC|compact TOC]] for alphabet and year headings.

==== Keeping headings out of the Table of Contents ====

If you want some subheadings to not appear in the Table of Contents, then make the following replacements.

Replace <nowiki> == Header 2 == with <h2> Header 2 </h2> </nowiki>

Replace <nowiki> === Header 3 === with <h3> Header 3 </h3> </nowiki>

And so forth.

For example, notice that the following header has the same font as the other subheaders to this "Tables" section, but the following header does not appear in the Table of Contents for this page.

<h4> This header has the h4 font, but is NOT in the Table of Contents </h4>

This effect is obtained by the following line of code.

<code><nowiki><h4> This header has the h4 font, but is NOT in the Table of Contents </h4></nowiki></code>

=== Tables ===
There are two ways to build tables:
*in special Wiki-markup (see [[Help:Table]])
*with the usual HTML elements: <table>, <tr>, <td> or <th>.

For the latter, and a discussion on when tables are appropriate, see [[Wikipedia:How to use tables]].

===Variables===

''(See also [[Help:Variable]])''
{|
|-
! Code
! Effect
|-
| <nowiki>{{CURRENTMONTH}}</nowiki> || {{CURRENTMONTH}}
|-
| <nowiki>{{CURRENTMONTHNAME}}</nowiki>
| {{CURRENTMONTHNAME}}
|-
| <nowiki>{{CURRENTMONTHNAMEGEN}}</nowiki>
| {{CURRENTMONTHNAMEGEN}}
|-
| <nowiki>{{CURRENTDAY}}</nowiki> || {{CURRENTDAY}}
|-
| <nowiki>{{CURRENTDAYNAME}}</nowiki> || {{CURRENTDAYNAME}}
|-
| <nowiki>{{CURRENTYEAR}}</nowiki> || {{CURRENTYEAR}}
|-
| <nowiki>{{CURRENTTIME}}</nowiki> || {{CURRENTTIME}}
|-
| <nowiki>{{NUMBEROFARTICLES}}</nowiki>
| {{NUMBEROFARTICLES}}
|-
| <nowiki>{{PAGENAME}}</nowiki> || {{PAGENAME}}
|-
| <nowiki>{{NAMESPACE}}</nowiki> || {{NAMESPACE}}
|-
| <nowiki>{{localurl:pagename}}</nowiki>
| {{localurl:pagename}}
|-
| <nowiki>{{localurl:</nowiki>''Wikipedia:Sandbox''<nowiki>|action=edit}}</nowiki>
| {{localurl:Wikipedia:Sandbox|action=edit}}
|-
| <nowiki>{{SERVER}}</nowiki> || {{SERVER}}
|-
| <nowiki>{{ns:1}}</nowiki> || {{ns:1}}
|-
| <nowiki>{{ns:2}}</nowiki> || {{ns:2}}
|-
| <nowiki>{{ns:3}}</nowiki> || {{ns:3}}
|-
| <nowiki>{{ns:4}}</nowiki> || {{ns:4}}
|-
| <nowiki>{{ns:5}}</nowiki> || {{ns:5}}
|-
| <nowiki>{{ns:6}}</nowiki> || {{ns:6}}
|-
| <nowiki>{{ns:7}}</nowiki> || {{ns:7}}
|-
| <nowiki>{{ns:8}}</nowiki> || {{ns:8}}
|-
| <nowiki>{{ns:9}}</nowiki> || {{ns:9}}
|-
| <nowiki>{{ns:10}}</nowiki> || {{ns:10}}
|-
| <nowiki>{{ns:11}}</nowiki> || {{ns:11}}
|-
| <nowiki>{{ns:12}}</nowiki> || {{ns:12}}
|-
| <nowiki>{{ns:13}}</nowiki> || {{ns:13}}
|-
| <nowiki>{{ns:14}}</nowiki> || {{ns:14}}
|-
| <nowiki>{{ns:15}}</nowiki> || {{ns:15}}
|-
| <nowiki>{{SITENAME}}</nowiki> || {{SITENAME}}
|}

'''NUMBEROFARTICLES''' is the number of pages in the main namespace which contain a link and are not a redirect, i.e. number of articles, stubs containing a link, and disambiguation pages.

'''CURRENTMONTHNAMEGEN''' is the genitive (possessive) grammatical form of the month name, as used in some languages; '''CURRENTMONTHNAME''' is the nominative (subject) form, as usually seen in English.

In languages where it makes a difference, you can use constructs like <nowiki>{{grammar:case|word}}</nowiki> to convert a word from the nominative case to some other case. For example, <nowiki>{{grammar:genitive|{{CURRENTMONTHNAME}}}}</nowiki> means the same as <nowiki>{{CURRENTMONTHNAMEGEN}}</nowiki>. 

===Templates===

The [[Wikipedia:MediaWiki|MediaWiki]] software used by Wikipedia has support for templates. This means standardized text chunks (such as [[Wikipedia:Template messages|boilerplate]] text) can be inserted into articles. For example, typing <nowiki>{{stub}}</nowiki> will appear as "''This article is a [[Wikipedia:The perfect stub article|stub]]. You can help Wikipedia by [[Wikipedia:Find or fix a stub|expanding it]].''" when the page is saved. See [[Wikipedia:Template messages]] for the complete list. Other commonly used ones are: <nowiki>{{disambig}}</nowiki> for disambiguation pages, <nowiki>{{spoiler}}</nowiki> for spoiler warnings and <nowiki>{{sectstub}}</nowiki> like an article stub but for a section. There are many subject-specific stubs e.g.: <nowiki> {{Geo-stub}}, {{Hist-stub}} and {{Linux-stub}}</nowiki>. For a complete list of stubs see [[Wikipedia:Template messages/Stubs]].

===Hiding the edit links===

Insert '''<nowiki>__NOEDITSECTION__</nowiki>''' into the document to suppress the edit links that appear next to every section header.

G2 specs part1

2005-04-02T22:03:31Z

Spooky:

<h1>Gnutella2 Specifications part one</h1>

I realise this has been quite a long time in coming, however I'm a
strong believer in getting some field experience with new ideas
before asserting that they will work (especially in a distributed
environment). This has been productive and as a direct result of
the experience gained, several significant changes have been evolved
into the proposal over the past few weeks. That process of testing
is not yet complete, however I'm now confident that the basic ideas
are relatively sound.

I'm also very interested in the thoughts of everyone here in the
GDF. I tend to have strongly held views of how things should work,
but the problem with a lone designer is that you rarely cover all
the possible angles. It's only by working with a group of
appropriately qualified people that there is any hope of exploring a
problem fully. So, I invite everyone to comment, criticise and
discuss.

Finally, to reiterate some of the stuff I posted last week: the
entire Gnutella2 concept is intended to be an open advancement in
the Gnutella world. I recognise that the GDF is not a standards
body and our purpose here is not to flesh out a new specification,
however I hope that this effort will spark some intelligent
discussion of the ideas contained within.

The Gnutella2 concept contains four parts:

* A new protocol
* A new data transport architecture
* New base services, including search
* An implementation standard

While I feel that all four parts are important to move forward with,
I suspect that the third part, the new base services and in
particular the new search mechanism is likely to be the area of
greatest interest for everyone here. Thus I'll focus on the
searching aspect of Gnutella2 in this post, and save discussion of
the other items for later.

(A final word on the first two points in this post: I do acknowledge
that it is entirely possible to implement most if not all new
features within the framework of the original Gnutella spec, working
around its limitations with things like GGEP. However I would argue
that in doing that we are limiting ourselves unnecessarily, not to
mention producing less than elegant designs and increasing overall
complexity. The original specification united us as developers
however if we are unable to move logically beyond it now, I think
that reflects an inability on our part to work together.)

In any case, this post is focused on the search architecture of
Gnutella2. The search framework is built upon the previous two
levels of implementation, however it is possible (and more
meaningful) to discuss it at a higher level now. I'll detail the
specifics of my implementation later.

__TOC__

== Background ==

It's important to take a step back and analyse exactly what a search
system should be able to do. In an ideal world of unlimited
bandwidth and CPU resources, it should be possible to execute a
search and locate every matching object present on the network
instantly. In the real world compromises must be made, but the
choice of compromise is important.

=== An Analysis of Gnutella1 ===

In the original (Gnutella1) architecture, the compromise is one of
search scope. Performing a search has a more or less fixed cost,
occurs over a more or less fixed time interval and covers an area of
more or less fixed size (but which is randomly selected).
Unfortunately, when mapped from the technical view to the outcome
view, it is evident that this compromise is not very desirable: In
an ideal scenario, Gnutella1 searches tend to find every matching
result within a fixed (but unknown) area of the network. There is
little or no opportunity to scale the search to match the rarity of
the target object (TTL and number of injection points can be scaled,
but have a high cost).

=== An Optimal Search System ===

It is my view that an optimal search system, under real world
limitations should have the following properties:

* It should be able to guarantee that if a matching object exists somewhere, it can be found
* It should have a means of scaling to match the rarity of the object that is not associated with a high cost to the network
* In a situation of limited resources, the ability to find at least some instances of a rare object should be of higher importance than the ability to find every instance of a common object.

In other words, the principal difference between my "optimal" search
objectives and the Gnutella1 model is that I am placing a higher
priority on the ability to find an object anywhere, than the ability
to find every object in a limited range. My rationale for this is
that:

* The ability to locate at least one instance of an object is very important, because without it no further action is possible
* Having located one instance of an object, there are other means of finding additional locations (if desired), such as a download mesh
* In a distributed hash table environment, a general search system only needs to locate a URN matching a search, after which the DHT can efficiently provide additional locations where that object may be found.
* Last but not least, the inability to find an object is demoralising for users

The third point there is quite important – I'm very keen on the use
of a large scale distributed hash table within Gnutella in the near
future (some work is going on there already). Thus the design of
this search system is such that it is not superseded, but rather
forms a part of, a DHT-enhanced search solution.

=== A search model meeting these objectives ===

It is well understood that in order to design a distributed search
system which has at least one scalable variable, there needs to be a
means of controlling the progress of the search. (Gnutella1 of
course fails on this principle as it involves "unleashing" a search
and waiting for the results) Controlled searches thus lend
themselves to an iterative, rather than recursive, approach. There
are three commonly known approaches:

* The random walker approach, in which the search maintains periodic contact with its originator, allowing the originator to continue or abort progress.
* The random or crawling mesh approach, in which the originator iteratively contacts nodes on the network until all nodes have been contacted or a desired condition has been reached.
* The guided mesh approach, in which the originator iteratively contacts nodes seeking a particular node which is responsible for a particular URN (DHT).

At the most simplistic level, the Gnutella2 network search model
takes the second approach: an iterative crawl of all nodes on the
network until a desired condition is reached. It also makes use of
the random walker model for another, non-search function, however
that is outside the scope of this discussion. It is envisioned that
the third DHT approach will be incorporated in the future as a more
efficient way of locating nodes with an instance of a URN obtained
from the current search model.

In this respect, the use of an iterative crawl is similar to the
GUESS proposal. The differences between this implementation and
GUESS become apparent in the process and optimisations described
below.

== Real World Considerations ==

=== Generic Viability ===

Implementing a node crawl directly is not viable for a public, large
scale network for several reasons:

* There are too many nodes to contact
* The cost of contacting a node is too high for the expected number of matching objects on that node (the probability of one match is often a lot lower than 1%)
* Not all nodes can be contacted directly (NATs, firewalls)
* Most nodes do not have the resources to handle and respond to queries from every other node

=== Increasing Network Density ===

One of the solutions is of course something we have been using for
some time in the Gnutella world – organising nodes into a two level
hierarchy, in which "high capacity" nodes shield "low capacity"
nodes. High capacity nodes have been called Ultrapeers
or "Supernodes" in the past, although I have used the term "Hubs" as
it seems to make the purpose much more clear to the general user
population.

Hubs solve several issues:

* Hubs can effectively answer queries for their leaves, so:
* Only hubs need to be contacted in a mesh crawl, reducing the number of nodes to contact
* Contacting one hub allows a large number of nodes to be searched, increasing the probability of locating an object and thus reducing the relative cost of contact
* Hubs must be able to be contacted directly, allowing un-contactable nodes to exist and participate as leaves
* Hubs can shield their leaves from the bulk of the bandwidth and CPU requirements

These facts are not new, and many of them apply to traditional
Gnutella1 as well. However in the Gnutella2 world, the reduced cost
of contact becomes a positive factor where it previously was not.
Similarly, the total number of nodes (now hubs) which could be
contacted for a complete network-wide search is reduced.

=== Problems with Gnutella Hubs ===

It is interesting to note that several other successful distributed
networks use a similar model, grouping a large number of leaf nodes
under a hub node, although different terminology is often used. The
server-based model is obviously the extreme version of this, where
all participants in the searchable area are connected to a common
server. Moving away from this to a more distributed model,
additional servers (-> hubs) are employed, with searches being
processed by each.

What these other networks tend to rely upon, however, is having a
very large number of leaf nodes per hub. This very high leaf
density makes searching very efficient, because the relative cost of
contacting each hub and the total number of hubs are both low. In
the Gnutella world, our ultrapeers/hubs tend to have far fewer
leaves. This presents a problem: there are too many hubs to
contact, and the relative cost per hub is too high.

These problems must be solved, or the time necessary to execute a
search will be far too high for reasonably rare and very rare
objects.

== Gnutella2 Solutions ==

The Gnutella2 model solves the "too many hubs" problem in two ways.

=== Optimising Hub to Leaf Communication ===

The first solution tackles the problem at its cause – to reduce the
number of hubs; the number of leaves per hub must be increased. The
limiting factor on the number of leaves per hub is the capacity of a
particular hub to support a large number of leaves. Some aspects of
this limitation are fixed, for example the amount of CPU power and
bandwidth available to a particular hub. This can only be addressed
by choosing better hubs (which is also important)

However, other aspects are highly variable and good targets for
optimisation: the cost of each additional leaf can be reduced.
Gnutella2 does this by:

* Leaves send their hub an extremely large query hash table which applies to all query traffic. This results in a substantially lower bandwidth requirement for each leaf, at the cost of additional memory on the hub. The latter is reduced by storing the tables in compressed hierarchical form to balance memory and CPU requirements. Patch tables are sent in simple 1-bit packed form, and the necessary levels are generated and compressed by the hub upon receipt of new or patch data.

* All hub to leaf traffic is compressed using the deflate algorithm. This provides a significant reduction in bandwidth requirement for each leaf, once again at the cost of state memory and some CPU on the hub. The CPU requirement is not as significant as might be expected, as the large query hash tables filter the majority of traffic leaving only a vast minority for compression. The combination of these two systems really squeezes the absolute most out of hub to leaf bandwidth. The memory requirement can be halved by not compressing leaf to hub bandwidth, which is almost non-existent in Gnutella2.

* Leaves are given the option of uploading a compressed snapshot of their library to their hub. I have not finished experimenting with this yet, but to spell out the obvious advantages and disadvantages: hub to leaf traffic is eliminated, memory and CPU requirements on the hub increase. By using a similar compressed hierarchical storage and access system I expect to be able to make the impact manageable.

The savings in bandwidth requirement per leaf provided by these
optimisations, at minimal extra cost to the hub in terms of CPU and
memory requirements, allow a typical hub to serve a much larger
number of leaves that it would normally be able to. This helps to
increase the average leaves per hub, and with appropriate hub
promotion control, reduce the total number of hubs. To map that
back to the original assertion, it reduces the relative cost of
contacting each hub and therefore also the amount of time taken to
perform a search.

=== Leveraging Inter-Hub Communication ===

Gnutella2 also makes another key optimisation. Assuming the total
number of hubs cannot easily be reduced any further, there are other
ways to reduce the relative cost of contacting each hub and reduce
the effective number of hubs which actually need to be contacted.

In the Gnutella2 search model, when a particular hub receives a
query from a query originator, it not only processes it locally and
forwards it to its leaves (where relevant), it also forwards it to
its neighbouring hubs. These hubs in turn process the query locally
and forward it to their leaves (where relevant), however they do not
forward it any further. To make a comparison with the Gnutella1
broadcast model, this is similar to executing a query with TTL=2.
In Gnutella2 there are no TTLs involved, rather fixed routing rules
are used to ensure that any hub receiving a query will always
forward it to its leaves, and may forward it to its neighbouring
hubs if the origin was not already a neighbouring hub. The "two
level" approach is not arbitrarily chosen, and its significance is
detailed later.

This behaviour means that for the cost of sending a query to a
single hub, a small "cluster" of connected hubs are actually
searched. For a typical hub with 5 neighbours, 6 hubs and their
leaves are searched at once. In essence, "hub clustering" offsets
Gnutella's lower average leaves per hub. The elimination of
broadcast "flood" traffic also means that hubs could actually have a
higher number of neighbour hubs, further increasing the cluster size
and lowering the cost per contact.

At first examination the cluster effect would appear only to benefit
the search originator by reducing the amount of outbound query
traffic and the time taken to contact every node (if required). In
the unoptimised case, every hub would still need to receive every
query, even if it was forwarded by an intermediary hub. The
benefits to the search originator are important, however further
optimisations allow the cluster effect to benefit hubs as well.

At the first level, hub to hub links are compressed using the
deflate algorithm in both directions. This is an important effect,
as the nature of UDP query traffic is that it cannot generally be
compressed (it's too small and there is no reliable context). Thus
receiving a query via UDP is much less efficient in terms of
bandwidth than receiving one via TCP. But as we know, UDP has other
benefits. A compromise must be struck, and the retransmission of
query packets within the hub cluster provides an excellent
opportunity. A single inbound UDP (inefficient) query is received
by one hub, which is then forwarded to the other 4+ hubs in the
cluster via TCP (efficient). One (random) hub takes the extra load;
the other (majority) receive the benefit.

At the next level up, query retransmission within the hub cluster is
filtered by very large query hash tables. Gnutella2 hubs regularly
exchange query hash table data, which is a composite of the hub's
own available objects and the hash tables of its leaves. Because
retransmission is restricted to a neighbour and its leaves, the use
of filtering tables at this level is perfectly acceptable (and
accurate). The hash table associated with a particular hub at an
instant in time is the same no matter which neighbour views it.

The use of query hash tables in hub to hub links makes it possible
to filter out query traffic which would not result in a hit,
reducing the hub to hub bandwidth requirements. This not only makes
the hub cluster concept more efficient, but also allows hubs to have
more leaves. Through clever hash table tricks, the size of the hash
tables employed on hub to hub links can be larger than that retained
on hub to leaf links (to offset the higher density). Methods of
patching hash tables as they change to reduce update bandwidth are
well documented also.

=== Summary ===

The primary focus of these solutions has been to increase the
efficiency of the system, in a situation of limited resources.
Decreasing the number of hubs to be contacted and decreasing the
effective cost of each contact (which includes both reducing the
cost of sending the query, and the cost of receiving and processing
it).

== Calculations ==

While the optimisations described previously provide significant
performance improvements, they also add complexity. For a small
network this complexity is not particularly necessary. However as
the number of nodes on the network grows, the benefits of higher
leaf density and lower cost of contact become much greater.

For a 500,000 node network with presently loaded ultrapeers
averaging about 150 leaves each, there will be approximately 3333
ultrapeers. Every one of these 3333 ultrapeers must be contacted to
achieve a global search, incurring the cost of transmission on the
search originator and the cost of receiving on each ultrapeer (plus
leaf costs depending on how that is set up). None of these queries
can be compressed.

If the same network is built with optimised hubs averaging 500 nodes
each, there will be approximately 1000 hubs. If each hub is
connected to an average of 5 neighbours, less than 170 hubs need to
be contacted by the search originator via UDP. The originator can
do that faster and more reliably. The other approximately 830 hubs
and their leaves are still effectively searched, but incur a reduced
(compressed) penalty, if any at all (QHT filtering).

== Hub Management ==

This section deals with some of the more mundane but important
aspects of operation.

=== Crawling the Network ===

It is not viable for every node to maintain an accurate table of
every hub on the network, if the network is of any reasonable size.
The storage requirement is not significant, but the overhead in
maintaining it is. It makes more sense to progressively discover
the extent of the network as it is traversed. This provides a sweep
of the current extent over a period of time (i.e. not a snapshot),
but its good enough.

To achieve this, each hub which is contacted must provide additional
information about where the search originator might look next. In a
simple system this is quite straight forward, however the use of the
hub clustering technique (while vastly increasing efficiency)
complicates matters somewhat.

=== Query Acknowledgements ===

Upon receiving a valid query from a search originator, a hub
immediately returns a query acknowledgement. The acknowledgement is
tagged for the specific search ID and includes two lists of
information: the hubs which have received the query (including the
current hub itself), and a selection of hubs to try.

The list of hubs which received the query will be every neighbouring
Gnutella2 hub, including hubs for which the query failed the QHT
test. The query originator adds these hubs to the "done list", and
avoids sending them direct queries. Note that additional
information is also returned, such as the leaf count for each hub,
so that the originator can display search statistics.

The list of other hubs to try is more complex. The hubs in the list
must be accurate, or time and resources are wasted on a large
scale. The only hubs which the hub can be 100% sure of are its
neighbours, but they are not eligible because they have already been
queried. To solve this problem, every Gnutella2 node knows every
neighbour, and every neighbour's neighbour. This applies partially
to leaves too, in that leaves must know their neighbours and their
neighbours' neighbours; however only hubs are included in the data
(leaf addresses are of no use). This information is kept up to date
quite stringently.

As a result, a hub sending a query acknowledgement packet can
include its neighbours as "searched", and its neighbours' neighbours
as "try these". This allows a logical crawling process to occur
unimpeded by hub clusters.

=== General Hub Discovery ===

Crawling through the network logically has some benefits; however a
more random approach can also be beneficial. To aid this, a few
random hubs from the active hub cache are also returned in query
acknowledgements. Random cached hubs are exchanged between all
nodes on the network, much like the "X-Try-Ultrapeers" header,
except as a continual process. Finally, hub walker advertisements
use a random walker approach to advertise long-running hubs to
distant points in the web. All indirect hub references are covered
by a last seen time allowing more recently confirmed hubs to be
prioritised and limiting the impact of lost hubs. (Connection
attempts on former ultrapeer IPs has been a significant problem in
Gnutella1).

== Security ==

The term "security" can have many meanings in a peer-to-peer context:

* Preventing denial of service attack upon the network
* Preventing the network from being used to launch an attack on other hosts
* Preventing the originator of a search from being identified
* Preventing the originator of a shared object from being identified
* Preventing false shared objects from being propagated

The first two points relate to securing the network structure
against misuse. The next two relate to anonymity, while the last is
more difficult to classify. As a developer I am most interested in
the first two items, and also to some extent the last item.

=== Risks Associated With UDP ===

Any distributed network (and particularly those which involve the
trading of host addresses) must take special consideration of
possible misuse of its nodes. In the original Gnutella network
there is already a significant potential to launch denial of service
attacks upon third party hosts by misrepresenting host addresses,
which then propagate through the network, resulting in thousands of
unsolicited connection attempts. Gnutella2 attempts to address this
somewhat by keeping track of last seen times for all hub addresses,
however there is a larger security concern: UDP.

At the lowest level, TCP connections are reasonably protected by the
need to establish a stream between two hosts. Forging the return
address of a connection establishment packet results only in a
single (small) response packet being directed at the target host.
Once the stream is established, traffic cannot be directed to other
hosts.

UDP on the other hand does not benefit from the security of a stream
connection. A received datagram may not necessarily have originated
from where its header would suggest. In a distributed search
scenario this can have a very serious consequence: forging a query
packet for a common object and sending it to a hub could result in a
large volume of search results being delivered to the forged, third
party host address. The more common the query term and the more
leaves attached to the hubs (including the effects of hub
clustering) and the larger the volume of return traffic. In this
situation the network is acting as a traffic amplifier, allowing the
attacker to anonymously transmit a small volume, but hit their
target with a large and untraceable amount of traffic.

Similarly, attacks can be directed upon the network itself. A
malicious host could flood one or more hubs with query traffic using
random return addresses to thwart any flood control mechanism,
causing an overload. While nothing can be done about unsolicited
traffic to a node on the network (which is completely independent
from the peer to peer system anyway), the use of UDP can eliminate
the important ability to detect and ignore such abuse.

=== Gnutella2 Solution – Query Keys ===

A solution is built into the Gnutella2 query model. Before a search
originator can transmit a query to a particular hub, it must obtain
a "query key" for that hub and include it in the transmission.
Query keys are unique to the hub generating them and the return
address with which they are associated and generally expire after a
relatively long period of time.

Originators request a query key by sending a query key request to a
hub, which includes the intended return address for the key. The
hub generates a key unique to that return address and dispatches it
there. This makes it impossible to get a query key for an IP which
you do not control, and prevents keys from being shared between two
nodes (key stealing).

When receiving a query from a foreign node, Gntuella2 hubs check the
query key against the one they have issued for the query's requested
return address and proceed only if there is a match. If and only if
the query key matches will the query acknowledgement be sent, or the
search processed locally and forwarded to local leaves and
neighbouring hubs.

This has several positive effects. If a query does not have a valid
key for the receiving hub, it will not be processed and will thus
not generate a traffic amplification effect which may be used in a
denial of service attack on a third party host. Secondly, the query
key mechanism ensures that queries are only processed if they were
transmitted from a host which has control over the host in the
return address (in the normal case, this is the same host). This
means that flood control mechanisms can remain just as effective as
in the TCP case. Similarly, host blocking is possible and more
viable than in Gnutella1 (where the source of a query could not be
verified).

== Conclusion ==

This document has covered the high level search model used in
Gnutella2, as currently deployed in the Shareaza 1.7 betas. I've
focused purely on search related issues here, as I think that
searching is probably the area of most interest to everyone at the
moment, and is definitely the area where there is likely to be some
interesting discussion.

Gnutella2 consists of a lot more than just a revised search system,
which I will cover soon. The actual implementation of the query
model allows for a lot more flexibility than we have in the
Gnutella1 framework, in terms of the way information is requested
and the way it is returned, as well as what kinda of information is
dealt with. Also:

* A new generic and extensible packet format, on which all of the
new functionality is built
* The unreliable and semi-reliable UDP transports, including encapsulation, fragmentation, retransmission, etc
* The specific packets involved in the search system
* Other base services

Michael Stokes<br>
mike@shareaza.com<br>
November 17, 2002

G2 specs part2

2005-04-02T22:01:40Z

Spooky: First version

<h1>Gnutella2 Specifications Part Two</h1>

===Background===

There has been some interesting discussion about the merits of moving to a fresh new protocol vs. grafting things onto the original beta protocol through encoding extensions. I’m glad the issue is getting some consideration, because at times in the past it has almost seemed like a taboo subject. I firmly believe that a new protocol the right thing to do at this time, and have made a number of arguments for that case recently which I will summarise below. Of course there are also good arguments against it, so it’s a question of weighing everything up.

One thing I’m not questioning is that you don’t NEED a new protocol to do new things. The major changes that fall under the “Gnutella2” umbrella are fundamental architectural changes relating to search and other services, and the specifics of how you express them in bytes are of somewhat less importance. But when creating a whole new “high level” system, why would you want to express it using an old, limited format? Redesigning the “top” of Gnutella provides the perfect and logical opportunity to redesign the “bottom” too. And doing so allows us to break free from the strange idiosyncrasies we had to work with in the Gnutella1 beta protocol.

===What This Document Covers===

This document discusses the base-level packet structure upon which higher level services are built. It details the limitations of Gnutella1, the goals of a flexible new base protocol, the way Gnutella2 packets are structured and the way these structures are serialized in binary streams.

That’s the first half of the picture. The second half (which will follow soon) documents a set of base packet types using the new extensible tree structure. These are used to deliver distributed host discovery and object search services, and a few other interesting things. The general operation of the search algorithm itself has been set out in the previous document; however the new packet tree structures which implement it also contain a number of key optimisations and improvements over Gnutella1.

==The Gnutella1 Protocol==

The original Gnutella beta protocol is not the worst protocol in the world by any stretch, but it is far from optimal. It is, understandably, the type of protocol that software engineers create when they want to test an idea. The fact that it has been so widely adopted despite that fact reflects very well upon its many good points.

However, the purpose for which it was designed is quite different from the realities of Gnutella today. I’m referring not so much to its use of a broadcast architecture (as that’s a higher level construct), but more toward the changes in “Gnutella politics”. The original protocol was designed to communicate between distributed applications with a common origin. There was no need for interoperability between independent vendors doing slightly different things, only a theoretical need for backward compatibility within the one development stream.

When you have complete control over a protocol, things are obviously a lot simpler. You decide when and how new things are coded, you (should) know exactly how various changes or improvements you make will affect everyone else, because you coded the software everyone else is using. That’s the kind of scene the Gnutella protocol was designed for. But as we know, that’s not what happened – it became an open protocol. And problems arose.

The fact that these problems were largely overcome is a testament to the efforts of the GDF. Establishing reasonably standard ways of tacking new data onto old constructs without disturbing the existing parsing code of many different vendors in an environment lacking strong version control was a very impressive achievement. But the point is this – with an appropriately designed base protocol, these problems would never have arisen.

Granted, issues of incompatibility have been largely solved or avoided through the development of new extension encoding techniques. These new techniques are great, but they’re tacking their new stuff on to the old structures, sometimes going to unnecessary lengths to do so. Why don’t we just lose the old stuff and keep the new, well thought out stuff?

I believe that Gnutella is an ideal and a community, not a protocol. If the protocol should be better, we should change it. That doesn’t destroy what this community is; it just helps it get better.

==Gnutella2 Protocol Objectives==

So, if there was an ideal protocol for a distributed network of nodes running software developed by many different vendors, what would it be like? I drew up a list of key objectives (please feel free to add to them!):

===It Should Not Be Too Specific===

G1 was designed to support a broadcast and narrow return architecture, which is a specific model we are now departing from. Every packet has a GUID, a TTL and a hop count, which are all useful in that model and useless ballast in most other models.

An ideal protocol should focus on a standard way of expressing information, rather than worrying about what kind of information. If a GUID is needed in a particular packet type, put it in that packet, not in the header of every packet. In other words, it should be minimal and compact.

===It Should Involve a Namespace===

In a closed protocol with one controlled implementer, numeric op-codes or function numbers are meaningful. In an open environment with many different implementers working independently, namespaces become necessary. This is one of the key innovations in things like GGEP and vendor messages.

===It Should Be Hierarchical===

Hierarchical document structures are extremely powerful (eg XML), and if you can make them compact enough, make ideal packets. Why? Because they allow any piece of information – anywhere – to be extended or duplicated without disturbing its value. That means that any implementer can extend the work of any other implementer without compromising it. Take the following example:

<pre><nowiki>
+ Query Hit Packet

|

|-+ Node ID (standard)

|

|-+ Server Status (standard)
| \-+ Shareaza Server Status (private extension)

|

|-+ Hit Object

| |-+ URN (standard)

| |-+ Descriptive name (standard)

| | \-+ Alternate name list (extension)

| |-+ URL (standard)

| |-+ Priority indicator (private extension)

| | \-+ Digital signature (private)

| |-+ Alternate source summary (standard)

| \-+ Available ranges (standard)

| . \-+ Estimated completion time (private extension)

|

|-+ Selective digital signature (private?)

|

\-+ Routing tags
</nowiki></pre>

I made that example up (clearly), but it shows how a hierarchical/tree model is perfectly suited to structured expressions of information, such as those we find in P2P (collections of query results, for example). You can approximate the same effect with fixed structures, but a true tree allows you to add anything anywhere anytime, enabling types of extensions that were not anticipated.

===It Should Be Independently Extensible===

If developer A creates a child packet which expresses support for peer to peer chat, developer B may want to support it. But developer B has a voice chat feature too, and wants to be able to detect it. Does developer B create a whole new packet to advertise their specific type of chat, even though it’s compatible? Why not re-use developer A’s, but modify it? Modifying someone else’s packet endangers compatibility. The solution is to create a “voice chat capability” child node and re-use A’s “chat capability” packet.

The tree structure allows the format of individual packets (or nodes in tree-speak) to be owned by developers, while allowing other developers to transparently extend them. Any party can invent new concepts and share them with other developers without (necessarily) losing control over them.

===It Should Be International===

It makes sense to design for international character set support from the beginning, even if not all clients currently support it. Shareaza 1.7 for example is still ANSI, but supports Unicode in the protocol and performs the necessary conversion. If Unicode protocol support is required from the word go, we’re free to implement it independently at leisure with no incompatibility problems.

===It Should Probably Support IPv6===

This is a new one based on the discussion the other day – I don’t know much about IPv6 but this is a good opportunity to do something about it.

==Proposed Gnutella2 Protocol==

===Overview===

The low-level Gnutella2 protocol used by the current Shareaza 1.7 betas is a cross between GGEP and vendor messages, without the (now unnecessary) overheads, and in a freeform tree structure.

Every Gnutella2 packet has a qualified name, length and other flags. It can contain a payload and/or one or more child packets. Child packets are identical to their parents in capability, i.e. you can nest packets to infinity. Multiple instances of a packet type can occur at any level (for example, lists of nodes, queries, hits, people, etc).

The concept can be compared to an XML document tree. The “packets” are elements, which can in turn contain zero or more child elements (packets). The payload of a packet is like the attributes of an XML element. However as we know, serializing XML has a lot of overhead due to all the naming, even in a compact binary form. The Gnutella2 packet structure makes a compromise: it names elements (packets), allowing them to be globally recognized and understood without knowledge of their format – and stores attributes as binary payloads, requiring knowledge of their content to parse them.

Thus the element (packet or child packet) is the finite unit of comprehension. This system provides an excellent tradeoff between format transparency and compactness.

===Packet Framing===

Packets are encoded with a single leading control byte, followed by one or more bytes of packet length, followed by one or more bytes of packet name/ID, followed by zero or more child packets (framed the same way), followed by zero or more bytes of payload:

<pre><nowiki>
+---------+----/----+----/----+---

| Control | Length_ | Name___ | children and/or payload

+---------+----/----+----/----+---
</nowiki></pre>

All packets can contain a payload only, children and a payload, children only, or nothing at all. The total length of the packet header cannot exceed 12 bytes and cannot be less than 3 bytes.

===The Control Byte===

The control byte is always non-zero. A zero control byte identifies the end of a stream of packets, and thus has special meaning. It should not be used in root packet streams (which do not end).

Control bytes have the following format:

<pre><nowiki>
Bit 7 < - > Bit 0

+----+----+----+----+----+----+----+----+

| Len_Len | Name_Len – 1 | // | // | CF |

+----+----+----+----+----+----+----+----+
</nowiki></pre>

'''Len_Len''' is the number of bytes in the length field of the packet, which immediately follows the control byte. There are two bits here which means the length field can be up to 3 bytes long. Note that Len_Len cannot be zero, which protects the control byte from ever being zero.

'''Name_Len''' is the number of bytes in the packet name field MINUS ONE, which follows the packet length field. There are three bits here which means that packet names can be 1 to 8 bytes long inclusive.

The three least significant bits of the control byte are reserved for flags. The LSB is defined as the compound packet flag. If this bit is set, the packet contains one or more child packets. If not set, the packet contains only its own payload (or nothing).

I am considering using the next bit to flag packets included in the digital signature, allowing selective signing of some parts of a packet tree – i.e. one signature which covers certain parts only. This allows other parts to be changed, added and removed without disturbing the integrity of the signature, which allows for some neat routing tricks.

===The Length Field===

The length field immediately follows the control byte, and can be 1 to 3 bytes long. Length bytes are stored least significant first, which allows them to be read very easily on little-endian machines.

The length value includes the payload of this packet AND any child packets in their entirety. This is obviously needed so that the entire packet can be detected and acquired from a stream. The length does not include the header (control byte, length, and name). The length field precedes the name field to allow it to be read faster from a stream when acquiring packets.

===The Name Field===

The name field immediately follows the length bytes, and can be from 1 to 8 bytes long. It follows a similar convention to the GGEP naming standard – bytes can be any value, but are generally alphanumeric.

Public packet types which form the core of important services should be agreed upon, and can use short names consisting of capital letters. For example general-purpose keep-alive ping/pongs use the names “PI” and “PO” (note that these are very different to G1 ping/pongs).

Private packet types or extensions should be prefixed with a namespace equal to the four letter vendor code, followed by lowercase. For example “RAZAgrp4”.

Naming convention is most important in the root namespace, and at other common points in common public packet types.

===Child Packets===

Child packets are only present if the “compound packet bit” is set in the control byte. If set, there is one or more child packet immediately following the end of the header. These child packets are included in the total length of their parent (along with the payload, which follows the child packets).

Child packets are framed exactly the same way, with a control byte, length, name, children and/or payload. When the compound bit is set, the first child packet must exist. Existing child packets may also exist, and are read in sequentially in the same way that they are read from a root packet stream. The end of the child packet stream is signaled by the presence of a zero control byte, OR the end of the parent packet’s length (in which case there is no payload). Including a terminating zero control byte when there is no payload is still valid, but unnecessary.

===Payload===

Payload may exist whenever the length field is non-zero. However, if the compound bit is set, one or more child packets must be read before the payload is reached. If there is no packet left after the end of the last child, there is no payload.

VERY IMPORTANT: The compound packet bit MUST be checked on every packet. It should be done in low-level decoding code to avoid accidental omission. Do not assume that a packet will not have children – it might not now, but no packets are sterile. Anything could be augmented. If you are not interested in children, skip them (which is easy, you don’t even need to recurse through their children).

===Payload Format===

The format of the payload is defined by the packet type, and may include various types of binary data. Payloads may or may not be a fixed length for a given packet type.

Multi-byte integers should always be expressed little-endian first, as the vast majority of users are using little-endian systems, and that is unlikely to change.

Strings can be expressed in Gnutella1 style, i.e. with a terminating null character. Alternatively the string is said to terminate if it reaches the end of the payload, avoiding an unnecessary null.

ALL STRINGS may be ANSI or Unicode. All software supporting the protocol must be prepared to accept either, performing appropriate conversion if necessary. Unicode software everywhere would be nice, but that’s easier said than done. Note that if you convert Unicode back to local ANSI, don’t retransmit it. Let the Unicode pass through.

ANSI strings are stored as consecutive bytes followed by a null byte (or end of packet). Unicode strings are stored as a 0xFF (255) marker byte, followed by 16-bit Unicode characters in little-endian form. The string is terminated by 0x0000 or the end of the payload. ALWAYS check for 0xFF and convert if not using Unicode strings internally.

In generating strings it is acceptable to check for extended characters and fall back to ANSI if none are present, to conserve space.

==Samples==

===Example Packets===

A simple ping (“PI”) packet with no payload:

<pre><nowiki>
Packet Length: 0
Packet Name: PI
Control Byte: 01 001 000 = 0x48 (lenlen=1,namelen=2,cmp=0)
Packet: 48 00 ‘P’ ‘I’
</nowiki></pre>

A pong packet containing a ping child (nonsensical but good example):

<pre><nowiki>
Packet Length: 4
Packet Name: PO
Control Byte: 01 001 001 = 0x49 (lenlen=1,namelen=2,cmp=1)
Packet: 49 04 ‘P’ ‘O’ 48 00 ‘P’ ‘I’
</nowiki></pre>

Now, what if it contains two ping children:

<pre><nowiki>
Packet: 49 08 ‘P’ ‘O’ 48 00 ‘P’ ‘I’ 48 00 ‘P’ ‘I’
</nowiki></pre>

Add a payload of “test” to the pong:

<pre><nowiki>
Packet: 49 0D ‘P’ ‘O’ 48 00 ‘P’ ‘I’ 48 00 ‘P’ ‘I’ 00 ‘t’ ‘e’ ‘s’ ‘t’
</nowiki></pre>

(Note the 00 terminating control byte after the children but before the payload)

===Simple Packet Decoder Logic===

The following is a simple packet header decoder in C that will work on any root or child packet. It assumes that the packet is present or that its input functions will block. In a production situation there would of course need to be more failure cases (data not available, malformatted packet). In particular, child packet bounds must not exceed the extent of the parent.

<pre><nowiki>
BYTE nInput = ReadNextByte();
if ( nInput == 0 ) return S_NO_MORE_CHILDREN;

BYTE nLenLen = ( nInput & 0xC0 ) >> 6;
BYTE nTypeLen = ( nInput & 0x38 ) >> 3;
BYTE nFlags = ( nInput & 0x07 );

DWORD nPacketLength = 0;
ReadBytes( (BYTE*)&nPacketLength, nLenLen );

CHAR szType[9];
ReadBytes( (BYTE*)szType, nTypeLen + 1 );
szType[ nTypeLen + 1 ] = 0;

BOOL bIsCompound = ( nFlags & 0x01 ) ? TRUE : FALSE;
</nowiki></pre>

==Higher Level Structures==

This document has covered the basic packet framing and structuring stuff only, without going into the higher level constructs to support discovery, identification, searching, etc. I will document that soon for further discussion, as there is definitely some cool stuff in there which takes advantage of the tree packet structures to deliver flexible and extensible services.

Michael Stokes<br>
mike@shareaza.com<br>
November 20, 2002

G2 specs part1

2005-04-02T21:35:30Z

Spooky: First version

Background

2005-04-02T21:20:49Z

Spooky: /* History */

==History==
The Gnutella2 protocol has been developed by [[Wikipedia:Michael_Stokes|Michael Stokes]] in the year 2002. It has been discussed quite controversial and some people prefer to refer to the Gnutella2 protocal by the name ''Mike's Protoco'' (MP). You might want to read the [[Wikipedia:Gnutella2|Wikipedia article]] on Gnutella2.

==Old Publications==
* [[G2_specs_part1|Gnutella2 Specifications part ONE]]
* [[G2_specs_part2|Gnutella2 Specifications part TWO]]