Subject: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-01.txt and draft-kuehlewind-tcpm-accurate-ecn-option-01.txt

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 16 Jul 2012 14:54:44 +0200

Hi everybody,

we updated the two drafts describing more accurate ECN feedback:

- draft-kuehlewind-tcpm-accurate-ecn-option-01.txt 
specifies an additional TCP option

- draft-kuehlewind-tcpm-accurate-ecn-01.txt
specifies the re-use of the ECN/NS TCP header bits

Mirja

----------  Forwarded Message  ----------

Subject: New Version Notification for 
draft-kuehlewind-tcpm-accurate-ecn-option-01.txt
Date: Friday 13 July 2012, 16:25:43
From: [email-masked]
To: [email-masked]
CC: [email-masked]

A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-option-01.txt
has been successfully submitted by Mirja Kuehlewind and posted to the
IETF repository.

Filename:	 draft-kuehlewind-tcpm-accurate-ecn-option
Revision:	 01
Title:		 Accurate ECN Feedback Option in TCP
Creation date:	 2012-07-13
WG ID:		 Individual Submission
Number of pages: 7
URL:             
http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-option-01.txt
Status:          
http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn-option
Htmlized:        
http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-option-01
Diff:            
http://tools.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-option-01

Abstract:
   This document specifies an TCP option to get accurate Explicit
   Congestion Notification (ECN) feedback from the receiver.  ECN is an
   IP/TCP mechanism where network nodes can mark IP packets instead of
   dropping them to indicate congestion to the end-points.  An ECN-
   capable receiver will feedback this information to the sender.  ECN
   is specified for TCP in such a way that only one feedback signal can
   be transmitted per Round-Trip Time (RTT).  Recently new TCP
   mechanisms like ConEx or DCTCP need more accurate feedback
   information in the case where more than one marking is received in
   one RTT.  This TCP extension can be used in addition to the classic
   ECN as well as with a more accurate ECN scheme recently proposed
   which reuses the ECN bit in the TCP header.

The IETF Secretariat

-------------------------------------------------------

----------  Forwarded Message  ----------

Subject: New Version Notification for 
draft-kuehlewind-tcpm-accurate-ecn-01.txt
Date: Monday 16 July 2012, 14:50:02
From: [email-masked]
To: [email-masked]
CC: [email-masked]

A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-01.txt
has been successfully submitted by Mirja Kuehlewind and posted to the
IETF repository.

Filename:	 draft-kuehlewind-tcpm-accurate-ecn
Revision:	 01
Title:		 More Accurate ECN Feedback in TCP
Creation date:	 2012-07-16
WG ID:		 Individual Submission
Number of pages: 19
URL:             
http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-01.txt
Status:          
http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn
Htmlized:        
http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-01
Diff:            
http://tools.ietf.org/rfcdiff?url2=draft-kuehlewind-tcpm-accurate-ecn-01

Abstract:
   Explicit Congestion Notification (ECN) is an IP/TCP mechanism where
   network nodes can mark IP packets instead of dropping them to
   indicate congestion to the end-points.  An ECN-capable receiver will
   feedback this information to the sender.  ECN is specified for TCP in
   such a way that only one feedback signal can be transmitted per
   Round-Trip Time (RTT).  Recently, new TCP mechanisms like ConEx or
   DCTCP need more accurate ECN feedback information in the case where
   more than one marking is received in one RTT.  This documents
   specifies a different scheme for the ECN feedback in the TCP header
   to provide more than one feedback signal per RTT.

The IETF Secretariat

-------------------------------------------------------

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 23 Jul 2012 04:01:24 +0100

Mirja, Richard,

I have the following top level comments about this draft.
(I'll send another posting with all the nits later.)

I believe the problem this draft addresses ought 
to be adopted as TCPM WG business.
I agree it should be Experimental.

I am particularly concerned to ensure that DCTCP 
uses a standardised feedback scheme, even if the 
DCTCP algorithm as a whole isn't brought to the 
IETF. Otherwise, with DCTCP already in Win8 
Server beta, we will start to see a new 
proprietary variant of the TCP wire protocol, 
where the DCTCP receiver won't interwork properly 
with senders that do not use the same proprietary 
DCTCP capability negotiation and the same 
proprietary extension to TCP feedback (that 
wouldn't pass standardisation, because it isn't robust to ACK losses).

However, the draft needs some work to make it 
clear what the spec is asking for. The mechanism 
is fairly clear, but not the roadmap in relation to other RFCs.

1/ Needs to be clearer what an implementer must do.

At the end of "1.1 Use Cases" it says it is best 
to have one generic feedback mechanism, whether 
it is used or not. I agree that feedback about 
every congestion signal (rather than just one per 
RTT) is certainly more generic, because a source 
can pick out one per RTT if it needs to (as in 
one of the example use-cases you give).

However, this section could confuse implementers: 
it needs to say that, in order to be compliant 
with this spec, a receiver will still need to 
implement classic ECN feedback as well.

In the capability negotiation section, it says a 
receiver might be negotiating with any of four 
different types of ECN sender (altho it doesn't 
use the 'classic' terminology introduced at the 
start for any of them), and the result could be 
one of four modes. So it needs to says which of 
these modes a compliant receiver must implement.

It also needs to explain what mode means: is the 
receiver in this mode? or the connection? or the half-connection?

2/ Relation to RFC3168?

You're certainly correct that this doesn't 
obsolete RFC3168. But what does it do to RFC3168? 
At the end of "1.4. Design Choices" it says that 
this proposed scheme will only be used instead of 
classic ECN feedback [RFC3168], not in parallel. 
I think that implies that this draft is intended 
to update RFC3168 (say this in the header block 
and in the abstract & introduction). It is best 
to avoid a design fork (this draft is an 
evolution of 3168, not an alternative, but it can 
interwork with an RFC3168 TCP sender if necessary).

You also ought to make it clear in the 
introduction that this scheme manages to give 
much more feedback information without using any 
new bits on the wire relative to the classic scheme.

3a/ Single Repeat Unnecessary?

You propose that the receiver must repeat each 
count of CE that it feeds back exactly once, even 
if its internal count has already increased again 
by the next ACK. I don't understand why you need 
this, and it introduces considerable complexity.

I think you are assuming that this will improve 
resilience against lost pure ACKs. But re-stating 
a counter in every packet is already resilient to 
loss. For instance if the receiver's internal 
count of CE marks is 3 then 4, when it feeds back 
3 then 4, if the ACK with 3 is lost, the 4 
recovers the lost information without having to 
repeat the 3. IOW, the ack count is cumulative.

This criticism also applies to the repetition of 
the other (nonce) counter in Appendix B.

Where we do have a problem is in detecting wrap 
(every 5 increments in your proposal). However, 
repetition doesn't help with that. For instance, 
if the receiver feeds back 3 twice, the second 3 
could represent 5 increments on from the first 
(or zero increments, or any multiple of 5), 
because a string of pure ACKS may have been lost 
between the two ACKs received. Similarly, 4 could 
represent an increment of 5i+1, where i is an integer.

3b/ Better solution?:
I proposed (in the re-ECN protcol, which was very 
similar to this proposal - thanks for the 
acknowledgement of that BTW) a way for the sender 
to calculate whether wrap could have happened, 
and to calculate the worst-case count of CE marks 
that the receiver might have fed back, given 
worst-case losses of pure ACKs (assuming regular sized data packets).

The sender calculates:
         n = how many new packets the latest ACK acknowledges.
         x = (AcE_latest - AcE_previous) % b,
where AcE is the congestion ack field and b is its base (5 in your proposal).
Then the sender can calculate the worst case CE 
marks signalled by the receiver as:
         D = n - (n-x)%b

For the sender to get a good calculation of the 
number of new packets acknowledged, SACK would be recommended.

In all the following examples, the base of AcE field is
         b = 5

#1 example:
Sender sees ACK of 1 new packet, and AcE rises 
from 2 to 3. This can only mean feedback of 1 CE mark. In maths:
         n = 1
         AcE_latest = 3
         AcE_previous = 2
Then
         x = (3-2)%5 = 1
         D = 1 - (1-1)%5 = 1

#2 example:
Sender sees ACK of 7 new packets (some ACKs must 
have got lost), and AcE rises from 2 to 3. This 
could mean feedback of 1 or 6 CE marks. In maths:
         n = 7
         AcE_latest = 3
         AcE_previous = 2
Then
         x = (3-2)%5 = 1
         D = 7 - (7-1)%5 = 6     # The formula 
automatically takes the worst-case

#3 example:
Sender sees delayed ACK of 2 new packets, and AcE 
wraps from 4 to 1 (base 5). This can only mean feedback of 2 CE mark. In maths:
         n = 2
         AcE_latest = 1
         AcE_previous = 4
Then
         x = (1-4)%5 = 2
         D = 2 - (2-2)%5 = 2

4/ ECT in the IP header of the SYN-ACK
You need to say whether this is optional or 
mandatory for compliance with the accurate-ecn spec.

5/ ECT in the IP header of the SYN
You say this is optional. I think a better way to 
say this is that the accurate-ecn spec does not 
preclude turning on ECN support in a SYN, but 
RFC3168 advises against it. If some future scheme 
uses ECN on the SYN, accurate-ecn will be able to 
support it, but it can also support RFC3168 not using ECN on the SYN.

6/ ECN Nonce support needs to be mandatory (only for receiver feedback).
I suggest the receiver MUST feedback a count of 
ECT(1), but make it clear that this doesn't mean 
the sender has to use the ECN nonce. In a 
connection where the sender doesn't use the 
nonce, the receiver's count will always be zero.

Reasoning: The nonce is a security mechanism for 
the sender to detect misbehaving receivers. If 
it's optional for receivers to do the feedback, 
then lots of receivers (good and bad) won't 
support it. Then any misbehaving receivers will 
just have to say they don't support it, and they 
can avoid their misbehaviour being detected.

7/ Advanced Compatibility Mode
I'm dubious about this part. It seems a nice 
idea, but it is quite unsafe, in that a lot of 
congestion indications can get lost without detection.

8/ I have an alternative proposal that you might like (see a following mail)

9/ And, as already promised, nits will follow 
too. It needs a fair amount of word-smithing.

HTH

Bob

At 13:54 16/07/2012, Mirja Kuehlewind wrote:
>_______________________________________________
>tcpm mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/tcpm

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 25 Jul 2012 15:15:08 +0000

Hi Bob,

thank you for your valuable feedback!

> I am particularly concerned to ensure that DCTCP uses a standardised
> feedback scheme, even if the DCTCP algorithm as a whole isn't brought to
> the IETF. Otherwise, with DCTCP already in Win8 Server beta, we will start
> to see a new proprietary variant of the TCP wire protocol, where the DCTCP
> receiver won't interwork properly with senders that do not use the same
> proprietary DCTCP capability negotiation and the same proprietary
> extension to TCP feedback (that wouldn't pass standardisation, because it
> isn't robust to ACK losses).

Agreed. Since DCTCP isn't really disclosed, particular the handling of CWR on the sender side, it's interaction with legacy RFC3168 receivers and senders can lead to "interesting" effects...

Ad 2) Yes, the draft should say that it updates (but does not obsolete!) RFC3168.

Ad 3a/b) The latest draft has a pseudo-code example, which uses the heuristic you describe. Also, I need to go back to my simulation - the repetition of each signaled value was intended to primarily address counter wrap when multiple ACKs are lost. We did some simulation runs with burst losses, and mean burst length=2, where the repetition scheme fared slightly better in not overestimating the true CE marks, IIRC.

However, some complexity remains, as a new ACK must be sent whenever the counter increases by (b-1). With b=3 (ECT1 counter signal), delayed ACK m must not exceed 2 (when a ECT(1) is set on a number of consecutive Packets...). This is the main driver for having a counter (the last Base(b)-digit to be signaled) and a Gauge. Without repeating each signaled codepoint twice, there is more complexity on the sender side to detect counter wraps, as you mention (average segment size, or knowing the exact sequence number boundaries/number of packets between two sequence numbers, etc). Having a single bit (per counter) to facilitate the repetition seemed rather small in comparison.

I'll try to run my simulations with and without the repetition of the same codepoint, and post the results to the WG.

Ad 4/5) A reference to RFC5562 may be in order...

Ad 6) Very reasonable argument; the idea was to allow some codepoints to remain reserved for other future uses. But your point is absolutely valid, and a security mechanism must be mandatory.

Best regards,

Richard Scheffenegger

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Bob Briscoe
> Sent: Montag, 23. Juli 2012 05:01
> To: Mirja Kuehlewind; Scheffenegger, Richard
> Cc: [email-masked]
> Subject: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt
> 
> Mirja, Richard,
> 
> I believe the problem this draft addresses ought to be adopted as TCPM WG
> business.
> I am particularly concerned to ensure that DCTCP uses a standardised
> feedback scheme, even if the DCTCP algorithm as a whole isn't brought to
> the IETF. Otherwise, with DCTCP already in Win8 Server beta, we will start
> to see a new proprietary variant of the TCP wire protocol, where the DCTCP
> receiver won't interwork properly with senders that do not use the same
> proprietary DCTCP capability negotiation and the same proprietary
> extension to TCP feedback (that wouldn't pass standardisation, because it
> isn't robust to ACK losses).
> 
> 
> However, the draft needs some work to make it clear what the spec is
> asking for. The mechanism is fairly clear, but not the roadmap in relation
> to other RFCs.
> 
> At the end of "1.1 Use Cases" it says it is best to have one generic
> feedback mechanism, whether it is used or not. I agree that feedback about
> RTT) is certainly more generic, because a source can pick out one per RTT
> if it needs to (as in one of the example use-cases you give).
> 
> it needs to say that, in order to be compliant with this spec, a receiver
> will still need to implement classic ECN feedback as well.
> 
> In the capability negotiation section, it says a receiver might be
> negotiating with any of four different types of ECN sender (altho it
> doesn't use the 'classic' terminology introduced at the start for any of
> them), and the result could be one of four modes. So it needs to says
> which of these modes a compliant receiver must implement.
> 
> It also needs to explain what mode means: is the receiver in this mode? or
> the connection? or the half-connection?
> 
> 2/ Relation to RFC3168?
> 
> You're certainly correct that this doesn't obsolete RFC3168. But what does
> it do to RFC3168?
> At the end of "1.4. Design Choices" it says that this proposed scheme will
> only be used instead of classic ECN feedback [RFC3168], not in parallel.
> I think that implies that this draft is intended to update RFC3168 (say
> this in the header block and in the abstract & introduction). It is best
> to avoid a design fork (this draft is an evolution of 3168, not an
> alternative, but it can interwork with an RFC3168 TCP sender if
> necessary).
> 
> introduction that this scheme manages to give much more feedback
> information without using any new bits on the wire relative to the classic
> scheme.
> 
> You propose that the receiver must repeat each count of CE that it feeds
> back exactly once, even if its internal count has already increased again
> by the next ACK. I don't understand why you need this, and it introduces
> considerable complexity.
> 
> I think you are assuming that this will improve resilience against lost
> pure ACKs. But re-stating a counter in every packet is already resilient
> to loss. For instance if the receiver's internal count of CE marks is 3
> then 4, when it feeds back
> 3 then 4, if the ACK with 3 is lost, the 4 recovers the lost information
> without having to repeat the 3. IOW, the ack count is cumulative.
> 
> This criticism also applies to the repetition of the other (nonce) counter
> in Appendix B.
> 
> Where we do have a problem is in detecting wrap (every 5 increments in
> your proposal). However, repetition doesn't help with that. For instance,
> if the receiver feeds back 3 twice, the second 3 could represent 5
> increments on from the first (or zero increments, or any multiple of 5),
> because a string of pure ACKS may have been lost between the two ACKs
> received. Similarly, 4 could represent an increment of 5i+1, where i is an
> integer.
> 
> 3b/ Better solution?:
> I proposed (in the re-ECN protcol, which was very similar to this proposal
> - thanks for the acknowledgement of that BTW) a way for the sender to
> calculate whether wrap could have happened, and to calculate the worst-
> case count of CE marks that the receiver might have fed back, given worst-
> case losses of pure ACKs (assuming regular sized data packets).
> 
> The sender calculates:
>          x = (AcE_latest - AcE_previous) % b, where AcE is the congestion
> ack field and b is its base (5 in your proposal).
> Then the sender can calculate the worst case CE marks signalled by the
> receiver as:
>          D = n - (n-x)%b
> 
> For the sender to get a good calculation of the number of new packets
> acknowledged, SACK would be recommended.
> 
> Sender sees ACK of 1 new packet, and AcE rises from 2 to 3. This can only
> mean feedback of 1 CE mark. In maths:
>          n = 1
>          AcE_latest = 3
>          AcE_previous = 2
> Then
>          x = (3-2)%5 = 1
>          D = 1 - (1-1)%5 = 1
> 
> #2 example:
> Sender sees ACK of 7 new packets (some ACKs must have got lost), and AcE
> rises from 2 to 3. This could mean feedback of 1 or 6 CE marks. In maths:
>          n = 7
>          AcE_latest = 3
>          AcE_previous = 2
> Then
>          x = (3-2)%5 = 1
> Sender sees delayed ACK of 2 new packets, and AcE wraps from 4 to 1 (base
> 5). This can only mean feedback of 2 CE mark. In maths:
>          n = 2
>          AcE_latest = 1
>          AcE_previous = 4
> Then
>          x = (1-4)%5 = 2
>          D = 2 - (2-2)%5 = 2
> 
> You need to say whether this is optional or mandatory for compliance with
> the accurate-ecn spec.
> 
> You say this is optional. I think a better way to say this is that the
> accurate-ecn spec does not preclude turning on ECN support in a SYN, but
> RFC3168 advises against it. If some future scheme uses ECN on the SYN,
> accurate-ecn will be able to support it, but it can also support RFC3168
> not using ECN on the SYN.
> 
> I suggest the receiver MUST feedback a count of ECT(1), but make it clear
> that this doesn't mean the sender has to use the ECN nonce. In a
> connection where the sender doesn't use the nonce, the receiver's count
> will always be zero.
> 
> Reasoning: The nonce is a security mechanism for the sender to detect
> misbehaving receivers. If it's optional for receivers to do the feedback,
> then lots of receivers (good and bad) won't support it. Then any
> misbehaving receivers will just have to say they don't support it, and
> they can avoid their misbehaviour being detected.
> 
> I'm dubious about this part. It seems a nice idea, but it is quite unsafe,
> in that a lot of congestion indications can get lost without detection.
> 
> 9/ And, as already promised, nits will follow too. It needs a fair amount
> of word-smithing.
> 
> HTH
> 
> 
> Bob
> 
> 
> 
> >University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart
> >
> >tel: +49(0)711/685-67973

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sun, 29 Jul 2012 23:08:07 +0000

Hi Bob,

Regarding this observation:

#########

You propose that the receiver must repeat each count of CE that it feeds back exactly once, even if its internal count has already increased again by the next ACK. I don't understand why you need this, and it introduces considerable complexity.

I think you are assuming that this will improve resilience against lost pure ACKs. But re-stating a counter in every packet is already resilient to loss. For instance if the receiver's internal count of CE marks is 3 then 4, when it feeds back
3 then 4, if the ACK with 3 is lost, the 4 recovers the lost information without having to repeat the 3. IOW, the ack count is cumulative.

This criticism also applies to the repetition of the other (nonce) counter in Appendix B.

Where we do have a problem is in detecting wrap (every 5 increments in your proposal). However, repetition doesn't help with that. For instance, if the receiver feeds back 3 twice, the second 3 could represent 5 increments on from the first (or zero increments, or any multiple of 5), because a string of pure ACKS may have been lost between the two ACKs received. Similarly, 4 could represent an increment of 5i+1, where i is an integer.

3b/ Better solution?:
I proposed (in the re-ECN protcol, which was very similar to this proposal - thanks for the acknowledgement of that BTW) a way for the sender to calculate whether wrap could have happened, and to calculate the worst-case count of CE marks that the receiver might have fed back, given worst-case losses of pure ACKs (assuming regular sized data packets).

#########

After having run the simulations more, I agree with critism 3a - the added value of the repetition only shows under specific circumstances. In the majority of cases, the difference between a direct encoding of the current counter value, vs. repeating the value two times is negligible...

Nevertheless, with delayed ACKs there should still be a skew rate limit, to have at least a minimum number of ACKs in between two counter wraps (e.g. whenever the counter has changed by <= 1/2 of the available codepoints, an ACK should be triggered... That would equate to sending an ACK for every other data packet, when every data packet is marked; for lower CE marking rates, the delAck rate can be significantly larger than 2...).

However, the conservative approach you describe in 3b has a major drawback, by assuming that whenever enough ACKs got lost that the counters could have wraped, that the sender needs to assume that they indeed have wraped, you end up with the sender overestimating the congestion of the network...

You mention that when there is congestion on the ACK path, the sensible thing to do would be to throttle the sending rate. However, when the delACK rate is larger than 6, this would need to be interpreted by the sender as 5 consecutive CE-marks (out of 6 data packets). Two consecutive lost ACKs would already be enough to cause that effect... With receive side coalescing, however, I don't think it is too uncommon for the receiver to skip such a small number of ACKs...

I don't yet know how to properly address this.

Attached are 4 graphs, showing these effects; All are for burst loss simulations (with the burst size increasing in the X-axis for the marked data packets, Y-axis in lost ACKs). The colors run from black = 100% too few marked noted on the receiver side, to white (1% underestimation), yellow (0.1% underestimation), green (perfect tracking of the CE marks), cyan (1% overestimation), blue (10% overestimate), magenta (100% overestimate or more).

Note that the yellow color in the simple graphs comes from a 1% background, random loss rate in both forward and return path, independent of the burst loss size; 

The cons(ervative) does address that case (the line along the x-axis remains green), but overestimates the CE marks extremely in most other instances...

Note that in these simulations, the (data) CE mark / ACK loss rate are modeled independent of any reaction of the sender...

One mechanism to limit the overestimating of the conservative algorithm (as laid out in 3b) might be to track the recent marking rate, and limit the skew during ack loss to some value related to the recently seen marking rate? (Obviously this wouldn't work well with step-function marking, though).

Guidance from you or the group on these topics would be highly appreciated.

But I think 3a (and the complexity due to that) can be removed from the draft...

Best regards,

Richard Scheffenegger