Subject: [conex] Comments on ConEx TCP Modifications

From: Nandita Dukkipati <[email-masked]>
Date: Wed, 28 Mar 2012 14:07:24 +0200

Following are a comments as individual contributor on
http://www.ietf.org/id/draft-ietf-conex-tcp-modifications-01.txt and one
comment as chair.

Nandita

1. What changes, if any, are required to accounting when ConEx marked
packets get lost?

2. Sec 3
"A sender which sends different sized packets with unequally distributed
packet sizes should know about reason to do so and thus may be able to
reconstruct the exact number of headers based on this information."
Sentence doesn't read well; please reword.

3. Sections 3.1.1 and 3.1.2
By acked_bytes I assume you mean the cumulatively acked bytes or the change
in snd.una. What happens when there is loss and snd.una does not advance?
Instead, I suggest that you consider using DeliveredData as defined in
http://datatracker.ietf.org/doc/draft-ietf-tcpm-proportional-rate-reduction/?include_text=1
-
it tracks precisely that amount of data delivered to the receiver for each
ACK even in the loss case.

4. Section 3.1.2
"Unfortunately, in a high congestion situation where all packets are CE
marled..."
marled -> marked

5. Section 3.1.2
On the use of 'M'
You won't require to use any variable 'M' when you track precisely the
bytes delivered per ACK using the DeliveredData that I mentioned above.

6. Sec 3.1.2
"In average the sender will sent..." -> "On average the sender will send..."

7. Advanced Compatibility mode
Not clear to me at all what this text is trying to say.

8. Sec 3.2
"Note that the above heuristics delays the ConEx signal by one segment, and
also..."
You mean delay by 1 RTT?

Comment as chair

Section 3 Accounting Congestion
bytes vs. packets: I would like to see consistency across the two drafts
(abstract mechanism, TCP modifications) on the issue of whether ConEx
marking and accounting should be in bytes or in packets. What are the
pros/cons and corner cases for each? What is the final recommendation that
you make? Which of these drafts should make this recommendation - I would
think it's the TCP modifications draft.

If you haven't already, could you please read up the latest Concepts and
Abstract Mechanism draft and communicate with the authors of that document
and make sure we have consistency on bytes vs. packets across the two I-Ds.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Fri, 30 Mar 2012 12:47:56 +0000

Hi Nandita,

thank you for your comments!

I think using the definition of DeliveredData instead of acked_bytes makes sense.

However, in section 3.1.2, simple and advanced compatibility mode, we run into problems one way or the other...  We will - on average - miss up to (M-1)/M CE marks (M ... delayed ACK count, ie. 2) for every CE received. The obvious and preferred approach is to have a accurate ECN feedback on TCP here, but for some time we will need to consider these compatibility modes. For them, knowning the ACK ratio (M) is definitely necessary. However, a proper definition of M (and making it uppercase everywhere) needs to be added.

Advanced Compatibility mode would be when the sender selectively sets CWR only on those packets, that will trigger an ACK on a legacy ECN receiver. That would, to a large extent, address the concern we have around missing a CE mark (which would result in a reduction of the TCP window). The basic idea around this we came up after some brainstorming (thanks Andrew and Jana). However, the complexity of implementing this (phase locked loops to figure out which data packets trigger ACKs, and the dead-time compensation) is probably too complex for the gains here, compared with the two other variants (simple, legacy ECN feedback, and basic compatibility by keeping CWR set constantly).

After some discussions with Alfons yesterday, there is actually one point that needs further discussion / clearification. During the presentation of the simulation results, two variants of policers were presented. The strict variant would preferably drop conex-marked packets, the normal should drop any conex-enabled packet with equal probability. (Also, in "inverse" of the strict variant can be envisioned). Depending on what the proper way is to implement the drop policies in there, our current conex TCP mod draft needs to revisit when to set the L mark actually.

Richard Scheffenegger

5. Section 3.1.2
On the use of 'M'
You won't require to use any variable 'M' when you track precisely the bytes delivered per ACK using the DeliveredData that I mentioned above.

8. Sec 3.2
"Note that the above heuristics delays the ConEx signal by one segment, and also..."
You mean delay by 1 RTT?

Comment as chair

Section 3 Accounting Congestion
bytes vs. packets: I would like to see consistency across the two drafts (abstract mechanism, TCP modifications) on the issue of whether ConEx marking and accounting should be in bytes or in packets. What are the pros/cons and corner cases for each? What is the final recommendation that you make? Which of these drafts should make this recommendation - I would think it's the TCP modifications draft.

If you haven't already, could you please read up the latest Concepts and Abstract Mechanism draft and communicate with the authors of that document and make sure we have consistency on bytes vs. packets across the two I-Ds.

---

From: Nandita Dukkipati <[email-masked]>
Date: Fri, 30 Mar 2012 23:55:13 +0200

On Fri, Mar 30, 2012 at 2:47 PM, Scheffenegger, Richard <[email-masked]>wrote:

>  Hi Nandita,****
>
> ** **
>
> thank you for your comments!****
>
> ** **
>
> I think using the definition of DeliveredData instead of acked_bytes makes
> sense.****
>
> ** **
>
> However, in section 3.1.2, simple and advanced compatibility mode, we run
> into problems one way or the other…  We will – on average – miss up to
> (M-1)/M CE marks (M … delayed ACK count, ie. 2) for every CE received. The
> obvious and preferred approach is to have a accurate ECN feedback on TCP
> here, but for some time we will need to consider these compatibility modes.
> For them, knowning the ACK ratio (M) is definitely necessary.
>

> However, a proper definition of M (and making it uppercase everywhere)
> needs to be added.
>

keep it simple and easy to reason about. The CUBIC CC algo. in Linux impl.
has one way to track delayed ack ratio - take a look to see if you can make
it better than what there.

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 30 Mar 2012 20:30:42 +0100

Richard,

At 13:47 30/03/2012, Scheffenegger, Richard wrote:
>two variants of policers were presented. The strict variant would 
>preferably drop conex-marked packets, the normal should drop any 
>conex-enabled packet with equal probability. (Also, in "inverse" of 
>the strict variant can be envisioned). Depending on what the proper 
>way is to implement the drop policies in there, our current conex 
>TCP mod draft needs to revisit when to set the L mark actually.

I know Credit marking and *audit* are mutually dependent (Matt's 
talk). But I don't see how TCP's ConEx marking depends on *policer* 
design (or the other way round for that matter).

I imagine people will design policers to try not to be firm but not 
pathological to typical transport *responses* to loss, but I don't 
see how ConEx-marking behaviour will matter to a policer.

What are you thinking of here?

1/ 'Best' treatment of L-markings
I have always been worried that dropping L packets could lead to 
problems, by creating a "black hole", as follows:
1) Bucket is empty, which must have been due to too many L packets.
2) Policer drop causes sender to slow-down but also insert more L packets.
3) Bucket empties faster.
4) Goto 2)

I suspected this might cause the sender to stall, so we always tried 
to avoid dropping L-marked packets. I was interested to see that this 
didn't happen in Alfons/Michael's simulations, so I would like to 
know the details of their policer algo.

Perhaps it's because the NewReno rate-reduction is sufficient to jump 
out of the "black hole". It would be interesting to see what happens 
with an unresponsive flow, and with DCTCP, which responds in more 
smaller steps.

2/ Policer diversity
One of the outcomes of ConEx experiments will be to see the breadth 
of designs people come up with. Michael's experiments are a welcome 
start to that.

We should maintain a list of these designs and experiments. In S.11.2 
of my PhD thesis "Policer Diversity" 
<http://bobbriscoe.net/projects/refb/#refb-dis>, I listed all the 
designs of policer I had published or knew of at the time. Since then 
I know of one more publication of a policer design, with summary 
simulation results:

Raiciu, C. (Ed.), "Progress on resource control," Trilogy EU 7th 
Framework Project ICT-216372 Deliverable 9 (December 2009)

Bob

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sat, 31 Mar 2012 17:56:45 +0000

Hi Bob,

Richard Scheffenegger

> I imagine people will design policers to try not to be firm but not pathological
> to typical transport *responses* to loss, but I don't see how ConEx-marking
> behaviour will matter to a policer.
> 
> I have always been worried that dropping L packets could lead to problems,
> by creating a "black hole", as follows:
> I suspected this might cause the sender to stall, so we always tried to avoid
> dropping L-marked packets. I was interested to see that this didn't happen in
> Alfons/Michael's simulations, so I would like to know the details of their
> policer algo.

In our current draft, we have effectively a SHOULD to set the L bit not with the retransmission itself, but with the next following (data) packet - at least the text can be interpreted that way. The intention was actually to prevent the interpretation of  L-bit == retransmission. But if the WG feels that explicitly marking only the actual retransmissions with L, to allow additional uses of the conex bits, we should spell that out more clearly:

"When a
   data segment is retransmitted, LEG will be increased by the size of
   the TCP payload packet containing the retransmission, assuming equal
   sized segments such that the retransmitted packet will have the same
   number of header as the original ones.  When sending subsequent
   segments, the ConEx L bit is set as long as LEG is positive, and LEG
   is decreased by the size of the sent TCP payload with the ConEx L bit
   set."

However, I think you have one assumption in your 4 steps above that doesn't hold. The statement "bucket empties faster". I was under the impression, that the refill ratio of the buckts is time- and packet-rate invariant, not? With other words, within a given time t, the bucket fills up with b credits; if the packet rate during t, with conex marks exceeds b, the policer drops; this causes the sender to reduce the packet rate - so in a given time period t, fewer packets are seen by the policier (unless the sender is not reactive, of course). Even when all packets sent by the sender are marked with conex bits, eventually the packet rate will drop below b/t, and the policer won't drop any more packets... 

Or am I missing something there?

But my issue is with the probability of dropping a packet; when the retransmitted data packets are dropped with a higher likelihood (and this depends on which packets carry the L-bit, and if the way the policer implements a "variable" drop probability depending on the conex marks), this may result in a higher probability of a retransmission get dropped, which will inevitably lead to a RTO stall of the TCP session... Which IMHO is undesirable (loss of ACK clock, significant latency etc).

For that reason, I believe we must have understanding which packets should carry the L-bits (only retransmissions, regular data and retransmissions), and if packets carrying conex marks should be treated any different from packets being merely conex-enabled...

Best regards,
   Richard Scheffenegger