UDP Transceiver - Revision history

Ram: /* Negotiation */ Clarified processing contract for improved acknowledgments

2012-09-29T15:58:22Z

Negotiation: Clarified processing contract for improved acknowledgments

← Older revision		Revision as of 15:58, 29 September 2012
Line 245:		Line 245:

	When the ''Improved Acknowledgments'' flag is '''missing''', the receiver '''must fall back''' to the ''legacy'' acknowledgment protocol. Since '''0x10''' is in the high-nibble part of the flag field, no legacy implementation will choke upon seeing it, and they can safely ignore it. This guarantees the sender that the message will be processed correctly by legacy implementations.		When the ''Improved Acknowledgments'' flag is '''missing''', the receiver '''must fall back''' to the ''legacy'' acknowledgment protocol. Since '''0x10''' is in the high-nibble part of the flag field, no legacy implementation will choke upon seeing it, and they can safely ignore it. This guarantees the sender that the message will be processed correctly by legacy implementations.

			When the ''Improved Acknowledgment'' bit is seen, the receiver is naturally free to ignore it altogether (as legacy servents will do) or chose to reply only with ''Cumulative Acknowledgements'' and never send ''Extended Acknowledgments'', for instance.

			However, when the sender sets the ''Improved Acknowledgment'' bit, it '''must''' be prepared to receive both ''Cumulative Acknowledgments'' '''and''' ''Extended Acknowledgments''. It is safe to ignore the extension part of the latter though, but the former must be fully handled.

	=== Delayed Acknowledgments ===		=== Delayed Acknowledgments ===

Ram: /* Negotiation */

2012-09-28T10:28:46Z

Negotiation

← Older revision		Revision as of 10:28, 28 September 2012
Line 244:		Line 244:
	* '''Improved Acknowledgements''': when the flag '''0x10''' is set in emitted fragments, it tells the other party that it can reply with the ''Cumulative Acknowledgments'' or the ''Extended Acknowledgment'' bits in its acknowledgment messages.		* '''Improved Acknowledgements''': when the flag '''0x10''' is set in emitted fragments, it tells the other party that it can reply with the ''Cumulative Acknowledgments'' or the ''Extended Acknowledgment'' bits in its acknowledgment messages.

	When the ''Improved Acknowledgments'' flag is '''missing''', the receiver '''must fall back''' to the ''legacy'' acknowledgment protocol. Since '''0x10''' is in the high-nibble part of the flag field, no legacy implementation will choke upon seeing it, and can safely ignore it. This guarantees the sender that the message will be processed correctly by legacy implementations.		When the ''Improved Acknowledgments'' flag is '''missing''', the receiver '''must fall back''' to the ''legacy'' acknowledgment protocol. Since '''0x10''' is in the high-nibble part of the flag field, no legacy implementation will choke upon seeing it, and they can safely ignore it. This guarantees the sender that the message will be processed correctly by legacy implementations.

	=== Delayed Acknowledgments ===		=== Delayed Acknowledgments ===

Ram: /* Delayed Acknowledgments */

2012-09-27T07:11:05Z

Delayed Acknowledgments

← Older revision		Revision as of 07:11, 27 September 2012
Line 250:		Line 250:
	To maximize the usefulness of ''Cumulative Acknowledgments'', the receiver can delay its acknowledgments for a small period of time (100 ms, say) in the hope that more fragments of the message will arrive meanwhile, provided the message has more than 1 fragment naturally. When the 100 ms delay is expired, it can then acknowledge everything it got so far. Performance will not be impacted by this small delaying, but it can save a few acknowledgments for larger messages.		To maximize the usefulness of ''Cumulative Acknowledgments'', the receiver can delay its acknowledgments for a small period of time (100 ms, say) in the hope that more fragments of the message will arrive meanwhile, provided the message has more than 1 fragment naturally. When the 100 ms delay is expired, it can then acknowledge everything it got so far. Performance will not be impacted by this small delaying, but it can save a few acknowledgments for larger messages.

	This means a sender must use a transmission policy whereby it first sends '''all the fragments''' of each message at once ~~the first time~~, then ~~wait~~ for acknowledgments or timeouts before retransmitting the ~~missing~~ fragments. In other words, the sender should not send a fragment, wait for its ~~acknowledge~~ before sending the next one: this ~~delays~~ the reception of the entire message and it ~~prevents~~ using the optimizations that this extension proposal wants to promote!		This means a sender must use a transmission policy whereby it first sends '''all the fragments''' of each message at once, then waits for acknowledgments or timeouts before retransmitting the um-acknowledged fragments. In other words, the sender should not send a fragment, wait for its acknowledgement before sending the next one: this would delay the reception of the entire message and it would prevent using the optimizations that this extension proposal wants to promote!

	=== Author ===		=== Author ===

	This proposal was created by Raphael Manfredi, last modified on 01:03, 26 September 2012 (PDT).		This proposal was created by Raphael Manfredi, last modified on 01:03, 26 September 2012 (PDT).

Ram: /* Improved Acknowledgments */ Really emphasize that missingBits is big-endian

2012-09-26T21:45:21Z

Improved Acknowledgments: Really emphasize that missingBits is big-endian

← Older revision		Revision as of 21:45, 26 September 2012
Line 224:		Line 224:
	In the unlikely case where missingBits is too small to hold all the missing parts, only the ones that can be represented are included, the '''nReceived''' field being there to provide additional information.		In the unlikely case where missingBits is too small to hold all the missing parts, only the ones that can be represented are included, the '''nReceived''' field being there to provide additional information.

	Proper generation and parsing of the missingBits field is crucial, so to remove any ambiguity, it is best to interpret missingBits as a number. Then bit 0 is the bit corresponding to 2^0, bit ''n'' is the bit corresponding to 2^n.		Proper generation and parsing of the '''missingBits''' field is crucial, so to remove any ambiguity, it is best to interpret missingBits as a number. Then bit 0 is the bit corresponding to 2^0, bit ''n'' is the bit corresponding to 2^n. In the above framing pictogram, bit 0 is the 7th bit of the 3rd byte (because the number in '''missingBits''' is encoded in ''big-endian'').

	Bit 0 corresponds to fragment #1, unless the '''0x10''' flag was set. In that case, if for instance '''nPart''' is 3, then it means fragments #1, #2 and #3 were already received. The base is therefore 3, and if bit 0 is set in '''missingBits''', it means fragment #4 (0 + 3 + 1) is still missing.		Bit 0 corresponds to fragment #1, unless the '''0x10''' flag was set. In that case, if for instance '''nPart''' is 3, then it means fragments #1, #2 and #3 were already received. The base is therefore 3, and if bit 0 is set in '''missingBits''', it means fragment #4 (0 + 3 + 1) is still missing.

Ram: /* Improved Acknowledgments */

2012-09-26T21:42:04Z

Improved Acknowledgments

← Older revision		Revision as of 21:42, 26 September 2012
Line 220:		Line 220:
	'''nReceived''' is the total amount of parts successfully received so far.		'''nReceived''' is the total amount of parts successfully received so far.

	'''missingBits''' is a bitfield that is read as a big-endian number and which contains set bits for the parts still missing. The ''base'' part number is normally 0, unless flag '''0x10''' was set, in which case the '''nPart''' value indicates the base. The rule is then that if bit ''b'' is set, then the fragment number ''b'' + ''base'' + 1 is still missing.		'''missingBits''' is a bitfield that is read as a '''big-endian number''' and which contains set bits for the parts still missing. The ''base'' part number is normally 0, unless flag '''0x10''' was set, in which case the '''nPart''' value indicates the base. The rule is then that if bit ''b'' is set, then the fragment number ''b'' + ''base'' + 1 is still missing.

	In the unlikely case where missingBits is too small to hold all the missing parts, only the ones that can be represented are included, the '''nReceived''' field being there to provide additional information.		In the unlikely case where missingBits is too small to hold all the missing parts, only the ones that can be represented are included, the '''nReceived''' field being there to provide additional information.

Ram: /* Extension Proposal */ Added delayed acknowledgments + negotiation sub-sections

2012-09-26T08:03:22Z

Extension Proposal: Added delayed acknowledgments + negotiation sub-sections

← Older revision		Revision as of 08:03, 26 September 2012
Line 198:		Line 198:
	This extension ''should'' be '''totally backward compatible''' with existing implementations provided the above page specifications were strictly adhered to. Here it goes:		This extension ''should'' be '''totally backward compatible''' with existing implementations provided the above page specifications were strictly adhered to. Here it goes:

	Because acknowledgment messages can be lost in the way or arrive out of		=== Improved Acknowledgments ===
	order, it is best to include as much of the reception state as possible
	so that the sending party can optimize retransmissions.		Because acknowledgment messages can be lost in the way or arrive out of order, it is best to include as much of the reception state as possible so that the sending party can optimize retransmissions.

	In order to do that, the following extensions to the original specifications		In order to do that, the following extensions to the original specifications has been added by gtk-gnutella:
	has been added by gtk-gnutella:

	* '''Cumulative Acknowledgements''': when the flag '''0x10''' is set, it tells the other party that '''all''' the fragments up to '''nPart''' have been received.		* '''Cumulative Acknowledgements''': when the flag '''0x10''' is set, it tells the other party that '''all''' the fragments up to '''nPart''' have been received.
Line 221:		Line 220:
	'''nReceived''' is the total amount of parts successfully received so far.		'''nReceived''' is the total amount of parts successfully received so far.

	'''missingBits''' is a bitfield that is read as a big-endian number and		'''missingBits''' is a bitfield that is read as a big-endian number and which contains set bits for the parts still missing. The ''base'' part number is normally 0, unless flag '''0x10''' was set, in which case the '''nPart''' value indicates the base. The rule is then that if bit ''b'' is set, then the fragment number ''b'' + ''base'' + 1 is still missing.
	which contains set bits for the parts still missing. The ''base'' part number
	is normally 0, unless flag '''0x10''' was set, in which case the '''nPart'''
	value indicates the base. The rule is then that if bit ''b'' is set,
	then the fragment number ''b'' + ''base'' + 1 is still missing.

	In the unlikely case where missingBits is too small to hold all the		In the unlikely case where missingBits is too small to hold all the missing parts, only the ones that can be represented are included, the '''nReceived''' field being there to provide additional information.
	missing parts, only the ones that can be represented are included, the
	'''nReceived''' field being there to provide additional information.

	Proper generation and parsing of the missingBits field is crucial, so to		Proper generation and parsing of the missingBits field is crucial, so to remove any ambiguity, it is best to interpret missingBits as a number. Then bit 0 is the bit corresponding to 2^0, bit ''n'' is the bit corresponding to 2^n.
	remove any ambiguity, it is best to interpret missingBits as a number.
	Then bit 0 is the bit corresponding to 2^0, bit ''n'' is the bit
	corresponding to 2^n.

	Bit 0 corresponds to fragment #1, unless the '''0x10''' flag was set. In that		Bit 0 corresponds to fragment #1, unless the '''0x10''' flag was set. In that case, if for instance '''nPart''' is 3, then it means fragments #1, #2 and #3 were already received. The base is therefore 3, and if bit 0 is set in '''missingBits''', it means fragment #4 (0 + 3 + 1) is still missing.
	case, if for instance '''nPart''' is 3, then it means fragments #1, #2 and #3
	were already received. The base is therefore 3, and if bit 0 is set
	in '''missingBits''', it means fragment #4 (0 + 3 + 1) is still missing.

	This extended acknowledgment lets the sending party optimize its		This extended acknowledgment lets the sending party optimize its retransmissions even when some acknowledgments are lost.
	retransmissions even when some acknowledgments are lost.

	''Extended ~~Acknowlegments~~'' are only useful when the total amount of fragments		''Extended Acknowledgments'' are only useful when the total amount of fragments
	is 3 or above. Indeed, with only 2 fragments, the ''Cumulative Acknowledgment''		is 3 or above. Indeed, with only 2 fragments, the ''Cumulative Acknowledgment''
	lets the receiving party know about the whole reception state.		lets the receiving party know about the whole reception state.

	When the amount of fragments is 3 or more and only a		When the amount of fragments is 3 or more and only a ''Cumulative Acknowledgment'' is sent out, it '''implicitly denies reception''' of any other fragments. This optimizes bandwidth since the 4 extra bytes sent out will only be required for large messages (more than 2 fragments) in case fragments are received out-of-order.
	''Cumulative Acknowledgment''
	is sent out, it '''implicitly denies reception''' of any other		=== Negotiation ===
	fragments. This optimizes bandwidth since the 4 extra bytes sent out will
	only be required for large messages (more than 2 fragments) in case fragments		Before the recipient can use this extension, it must be certain that the sender will understand its semantics. Indeed, imagine the case where a 3-fragment message is sent and completely received, and therefore the recipient only sends a ''Cumulative Acknowledge'' for fragment #3. A legacy sender would think fragments #1 and #2 are still missing.
	are received out-of-order.
			To signify to the recipient that the sender will understand the new acknowledgments, it '''must set the flag 0x10''' in all the fragments it sends. This new flag is therefore defined as:

			* '''Improved Acknowledgements''': when the flag '''0x10''' is set in emitted fragments, it tells the other party that it can reply with the ''Cumulative Acknowledgments'' or the ''Extended Acknowledgment'' bits in its acknowledgment messages.

			When the ''Improved Acknowledgments'' flag is '''missing''', the receiver '''must fall back''' to the ''legacy'' acknowledgment protocol. Since '''0x10''' is in the high-nibble part of the flag field, no legacy implementation will choke upon seeing it, and can safely ignore it. This guarantees the sender that the message will be processed correctly by legacy implementations.

			=== Delayed Acknowledgments ===

			To maximize the usefulness of ''Cumulative Acknowledgments'', the receiver can delay its acknowledgments for a small period of time (100 ms, say) in the hope that more fragments of the message will arrive meanwhile, provided the message has more than 1 fragment naturally. When the 100 ms delay is expired, it can then acknowledge everything it got so far. Performance will not be impacted by this small delaying, but it can save a few acknowledgments for larger messages.

			This means a sender must use a transmission policy whereby it first sends '''all the fragments''' of each message at once the first time, then wait for acknowledgments or timeouts before retransmitting the missing fragments. In other words, the sender should not send a fragment, wait for its acknowledge before sending the next one: this delays the reception of the entire message and it prevents using the optimizations that this extension proposal wants to promote!

			=== Author ===

			This proposal was created by Raphael Manfredi, last modified on 01:03, 26 September 2012 (PDT).

Ram: /* Encoding */ Fixed szTag item in structure (was missing [])

2012-09-23T12:55:35Z

Encoding: Fixed szTag item in structure (was missing [])

← Older revision		Revision as of 12:55, 23 September 2012
Line 60:		Line 60:
	typedef struct		typedef struct
	{		{
	CHAR ~~szTag3~~;		CHAR szTag[3];
	BYTE nFlags;		BYTE nFlags;
	WORD nSequence;		WORD nSequence;

Ram: /* Parameters */ MTU should really be 476 bytes, not 500

2012-09-23T12:19:25Z

Parameters: MTU should really be 476 bytes, not 500

← Older revision		Revision as of 12:19, 23 September 2012
Line 172:		Line 172:
	</nowiki>		</nowiki>
	</pre>		</pre>

			The UDP/IP header being typically 28 bytes, it is best to limit the payload of the messages to '''476 bytes'''. That way, with the 8-byte header we're topping plus the UDP/IP header, the whole message will not be greater than 512 bytes. The recommended MTU should therefore be 476 rather than the legacy 500. --RAM, 05:19, 23 September 2012 (PDT)

	== Performance Considerations ==		== Performance Considerations ==

Ram: Added proposed acknowledgment extension

2012-09-23T12:07:27Z

Added proposed acknowledgment extension

← Older revision		Revision as of 12:07, 23 September 2012
Line 189:		Line 189:
	Only critical transmissions whose reception cannot otherwise be inferred, should have		Only critical transmissions whose reception cannot otherwise be inferred, should have
	the acknowledge request bit set.		the acknowledge request bit set.

			== Extension Proposal ==

			The following is still a proposoal, but since there is no G2 Development Forum where this can be discussed, I thought I would document it here. 05:07, 23 September 2012 (PDT)

			This extension ''should'' be '''totally backward compatible''' with existing implementations provided the above page specifications were strictly adhered to. Here it goes:

			Because acknowledgment messages can be lost in the way or arrive out of
			order, it is best to include as much of the reception state as possible
			so that the sending party can optimize retransmissions.

			In order to do that, the following extensions to the original specifications
			has been added by gtk-gnutella:

			* '''Cumulative Acknowledgements''': when the flag '''0x10''' is set, it tells the other party that '''all''' the fragments up to '''nPart''' have been received.

			* '''Extended Acknowledgments''': when the flag '''0x20''' is set, it tells the other party that an acknowledgment payload is present. It immediately follows the header and is architected thusly:

			<pre>
			<nowiki>
			0 1 2 3
			0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7\| Byte
			+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
			\| nReceived \| missingBits \| 0-3
			+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
			</nowiki>
			</pre>

			'''nReceived''' is the total amount of parts successfully received so far.

			'''missingBits''' is a bitfield that is read as a big-endian number and
			which contains set bits for the parts still missing. The ''base'' part number
			is normally 0, unless flag '''0x10''' was set, in which case the '''nPart'''
			value indicates the base. The rule is then that if bit ''b'' is set,
			then the fragment number ''b'' + ''base'' + 1 is still missing.

			In the unlikely case where missingBits is too small to hold all the
			missing parts, only the ones that can be represented are included, the
			'''nReceived''' field being there to provide additional information.

			Proper generation and parsing of the missingBits field is crucial, so to
			remove any ambiguity, it is best to interpret missingBits as a number.
			Then bit 0 is the bit corresponding to 2^0, bit ''n'' is the bit
			corresponding to 2^n.

			Bit 0 corresponds to fragment #1, unless the '''0x10''' flag was set. In that
			case, if for instance '''nPart''' is 3, then it means fragments #1, #2 and #3
			were already received. The base is therefore 3, and if bit 0 is set
			in '''missingBits''', it means fragment #4 (0 + 3 + 1) is still missing.

			This extended acknowledgment lets the sending party optimize its
			retransmissions even when some acknowledgments are lost.

			''Extended Acknowlegments'' are only useful when the total amount of fragments
			is 3 or above. Indeed, with only 2 fragments, the ''Cumulative Acknowledgment''
			lets the receiving party know about the whole reception state.

			When the amount of fragments is 3 or more and only a
			''Cumulative Acknowledgment''
			is sent out, it '''implicitly denies reception''' of any other
			fragments. This optimizes bandwidth since the 4 extra bytes sent out will
			only be required for large messages (more than 2 fragments) in case fragments
			are received out-of-order.

Dcat: /* Encoding */ added byte format for header

2005-05-01T02:47:01Z

Encoding: added byte format for header

@@ Line 66: / Line 66: @@
     BYTE nCount;
 } GND_HEADER;
 </nowiki>
 </pre>