Subject: Re: [tcpm] A new ID on TCP option space extension.

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sun, 1 Apr 2012 19:38:29 +0000

Hi Anantha

I beliefe delayed negotiation for timestamps would be comparatively easy, compared to dynamically disabling it at some time later during the session. I have the stromg suspicion that some stack may already do this - but most likely due to a buggy 1323 implementation more than anything... At least when i did some tests for my ts nego draft, some stacks reacted like that when temporarily disabling ts. But in my tests,  ts were negotiated for during syn.

Properly disabling ts will be an order of magnitude more involved i guess.

But just delaying ts nego would free precious syn option space, which has become increasingly important.

Best regards,
  Richard

Am 01.04.2012 um 04:12 schrieb "Anantha Ramaiah (ananth)" <[email-masked]>:

> <just got back to SFO. Catching up...>
> 
> RTTM has been touted to be the main use for timestamps which includes
> solution to the Karn's ambiguity problem. The other one is PAWS and
> w.r.t I agree it is wastage of TCP option bytes and the idea of turning
> on PAWS detection dynamically (using timestamps) makes sense to me.
> Changing the timestamp semantics means change to many TCP/IP stacks
> (variables like ts_val, ts_recent would not be interpreted as before,
> all other debugging tools etc., may need to understand this change)
> 
> Even if we are planning to do so, I thought you meant to negotiate the
> capability during SYN (don't send the timestamp value) and send
> timestamps when needed. Or are we saying that we are trying to negotiate
> the option when needed, this sounds like a bigger change in semantics
> since every TCP option is only negotiated during SYN as it stands today.
> To make this more useful, you would want to not only turn on this
> capability when needed, also turn off when not needed.
> 
> Didn't we already have a RFC 1323bis document? Is that expired or did it
> make it to the RFC 1323 update ? If not, you could always fold this idea
> (assuming there is a consensus) into that document. Not sure whether we
> would want multiple RFC 1323bis documents...
> 
> Just my 2 cents..
> 
> -Anantha
> 
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]]
>> Sent: Saturday, March 31, 2012 7:03 AM
>> To: Scheffenegger, Richard
>> Cc: David Borman; [email-masked]; Anantha Ramaiah (ananth)
>> Subject: Re: [tcpm] A new ID on TCP option space extension.
>> 
>> 
>>> Actually, allowing for delay-based congestion algorithms like
> LEDBET,
>>> CTCP may be a stronger use case for timestamps.
>> 
>> That is absolutely fine.  I am not saying we should rid the world of
>> timestamps.  I am saying that we should re-arrange the world such that
>> TCP connections do not have to use timestamps because they *might*
> need
>> them later.  If there is some good reason to use them from the onset
> of
>> the connection, great, use them.  But,
>> 
>> - If RTTM is the reason to use the TS option then I haven't seen the
>>   empirical evidence that says this is at all helpful and so I am
>>   pretty dubious.
>> 
>> - If PAWS is the reason to use the TS option then I don't see why
>> we'd
>>   waste the bytes on the TS option for connections that send less
>> than
>>   4GB (i.e., that will not wrap the sequence space).  So, being able
>>   to turn on the TS option mid-connection when PAWS is required
> makes
>>   good sense to me.
>> 
>> So, I am voting for an RFC 1323bis that calls for negotiating
>> timestamps only when they are needed.  And, if some new use needs to
> do
>> so at the beginning of the connection and use them throughout then we
>> should absolutely not preclude that.
>> 
>> allman
>> 
>> 
>

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Sun, 1 Apr 2012 17:47:20 -0700

Agreed. All I was trying to say is that when we are planning to change,
we could always cover the solution from all angles as much as possible.
Of course, there are these middleboxes (firewalls) which can come in the
way of delayed negotiation.

-Anantha

> From: Scheffenegger, Richard [mailto:[email-masked]]
> Sent: Sunday, April 01, 2012 12:38 PM
> To: Anantha Ramaiah (ananth)
> Cc: [email-masked]; David Borman; [email-masked]
> I beliefe delayed negotiation for timestamps would be comparatively
> easy, compared to dynamically disabling it at some time later during
> the session. I have the stromg suspicion that some stack may already
do
> this - but most likely due to a buggy 1323 implementation more than
> anything... At least when i did some tests for my ts nego draft, some
> stacks reacted like that when temporarily disabling ts. But in my
> tests,  ts were negotiated for during syn.
> 
> > w.r.t I agree it is wastage of TCP option bytes and the idea of
> > turning on PAWS detection dynamically (using timestamps) makes sense
> to me.
> > timestamps when needed. Or are we saying that we are trying to
> > negotiate the option when needed, this sounds like a bigger change
in
> > semantics since every TCP option is only negotiated during SYN as it
> stands today.
> > Didn't we already have a RFC 1323bis document? Is that expired or
did
> > it make it to the RFC 1323 update ? If not, you could always fold
> this
> > idea (assuming there is a consensus) into that document. Not sure
> > whether we would want multiple RFC 1323bis documents...
> >
> > Just my 2 cents..
> >
> > -Anantha
> >
> >> timestamps.  I am saying that we should re-arrange the world such
> >> that TCP connections do not have to use timestamps because they
> >> *might*
> > need

---

From: Mark Allman <[email-masked]>
Date: Sun, 01 Apr 2012 21:53:06 -0400

> capability when needed, also turn off when not needed.

I am with Richard in thinking turning it off is a lot more complicated
than turning it on.

Can you sketch why this would be more useful?

allman

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Sun, 1 Apr 2012 19:17:38 -0700

Hi Mark,

> Sent: Sunday, April 01, 2012 6:53 PM
> Cc: Scheffenegger, Richard; David Borman; [email-masked]

A clarification :  What I meant is that you can turn it on, but sending
timestamps option every TCP packet (segment) is optional.  I was
thinking along the following lines.

a) during 3 way handshake we could just negotiate the option itself.
This is going to be a new TCP option (if we do not want to change the
semantics of the original TCP option and get caught with the sanity
checking of firewalls + the difficulty of reliably negotiating the
option in the midway of a TCP connection) This would be an option-kind
(just like SACK permitted) so 2 bytes.
 
b) whenever needed send timestamps during the lifetime of the TCP
connection. 

c) the catch is whether you want to use the *same* option kind which
would imply that the timestamps option would be a variable length option
either 2 bytes or 10 bytes. Or else have another option just like SACK
option to send the timestamp values, but this would mean that you cannot
send timestamp value during SYN using the same option code.

IMO, this would be a cleaner approach than negotiating the TCP option
during the midway of the connection, which is a change in semantics for
TCP option and it may not go well with middleboxes etc., Stacks can also
migrate to using the newer timestamp option eventually and eventually
deprecate the old timestamp option.

I hope it makes sense or did I miss something here?

Thanks,
-Anantha

---

From: Mark Allman <[email-masked]>
Date: Sun, 01 Apr 2012 22:31:45 -0400

> during the midway of the connection, which is a change in semantics
> for TCP option and it may not go well with middleboxes etc., Stacks
> can also migrate to using the newer timestamp option eventually and
> eventually deprecate the old timestamp option.

I liked David's sketch of how to negotiate when you wanted to start
using timestamps and not in the 3WHS.  That made pretty good sense to me
and I bet has a very high value of Just Works.  I don't see defining a
whole new timestamp option with dramatically different properties just
to change a single property (i.e., when to start using them).

OKing options in the SYN is---IMHO---just historic artifact.  I am sure
someone can remind us of the machine that crashed or caught fire or
something when an unknown option was received after the 3WHS.  This was
bad and made us then go an negotiate everything in the 3WHS.  But, I am
not aware of this being a hazard we should worry about in 2012.

And, defined and undefined options after the 3WHS that are not
negotiated during the 3WHS worked in a large fraction of the cases for
us when we studied it a while back.  So, this doesn't worry me too much.
It is also a comment that at the time middleboxes just ignored unknown
options in the middle of connections (see out CCR 2005 paper;
http://www.icir.org/mallman/papers/tcp-evo-ccr05.ps).  I guess who knows
if that has changed.  My gut wonders why it would have---i.e., why would
middleboxes much care?  Clearly there are middleboxes that care about
the sequence space and so options like SACK get munged.  But, since this
isn't a sequence space option I am not hugely worried about this case.
It'd be nice to see some more recent data.

allman

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Sun, 1 Apr 2012 21:49:10 -0700

> Sent: Sunday, April 01, 2012 7:32 PM
> using timestamps and not in the 3WHS.  That made pretty good sense to
> me and I bet has a very high value of Just Works.  I don't see
defining
> a whole new timestamp option with dramatically different properties
> just to change a single property (i.e., when to start using them).

Ok, maybe having a new TCP option for this purpose is height of
paranoia. RFC 1323 is very common and implementations have followed the
wordings of RFC verbatim esp, the place where it says :-

      We must pay careful attention to compatibility, i.e., to
      interoperation with existing implementations.  The only TCP option
      defined previously, MSS, may appear only on a SYN segment.  Every
      implementation should (and we expect that most will) ignore
      unknown options on SYN segments.  However, some buggy TCP
      implementation might be crashed by the first appearance of an
      option on a non-SYN segment.  Therefore, for each of the
      extensions defined below, TCP options will be sent on non-SYN
      segments only when an exchange of options on the SYN segments has
      indicated that both sides understand the extension.  Furthermore,
      an extension option will be sent in a <SYN,ACK> segment only if
      the corresponding option was received in the initial <SYN>
      segment.

So, some over zealous firewalls, would interpret this to mean an attack
(since it would crash the end system, if it is buggy) may strip this
option OR would drop the packet. Sorry if I am sounding like
hand-waving, we should get some recent data as you say below. [I am also
planning to check our firewall implementations and get back...]

In any case, that should not prevent one from writing an ID which
updates RFC 1323. I fully support it for obvious reasons :-) I have 2
questions :-

- Are we only going to target timestamps Or make it generic looking to
say other TCP options can also be negotiated mid-way? Or are we going to
say that only timestamp would be an exception? For example I would think
negotiating a window scale in the middle of the TCP connection may be
complicated.

- I asked this before. There was already a rfc 1323 bis document before.
If we are only going to focus on timestamps, then why not revive that
document?. Not sure why that document did not make through.

-Anantha

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 2 Apr 2012 09:38:02 +0000

Anantha,

There are a bunch of mid-stream, temporary options for MP-TCP; Actually, I'm more concerned because TS are not a "unknown" option for middleboxes, but middleboxes may have code to interpret these (check the semantics etc).

During the MP-TCP option testing, late negotiation of TS was obviously not tested.

On the other hand, any implementation that wants to go with late TS negotiation must be prepared that Timestamps may not be negotiated / reflected then - and treat this in a graceful way. If the reason for late TS negotiation failing is the other TCP end not supporting it, or any random middlebox removing the option because of RFC1323 semantic checks, doesn't make any difference.

The only thing that we really should verify is, if there are middleboxes that will actually terminate a TCP session, when all of a sudden the TS option appears... I hope this won't be the case, but it needs to be verified.

So, if you can verify the firewall code and it's interactions, that would be a very good starting point, thanks a lot!

For a 1323bis, I think we should limit this late negotiation to timestamps for now; other options are really beneficial from the start (SACK, MPTCP,MSS), while stil others would require some definition of new semantics if late negotiation would be used (WS).

I have been asked to look at 1323bis and try to revive it - currently in the process of updating the formatting, references and boilerplate to the current standard; 

This is a short list of the issues that were raised during RFC1323bis discussions (and that I was able to dig up; if anyone has more info, please let me know):

* Window reduction during recovery bug - Matt
* RTO adjustments (no need to use every sample, one update per RTT enough) - Mark
* Enabling TS (PAWS) midstream - David, Matt, and others
* PAWS and non-DF segments (incorrectly reassembled packets, not captured by TCP checksum) [?]
* References / interaction with SACK (TS check does not look if SACK is present - to cover certain cased of lost ACKs that inflate RTO) [?]
* Updating TS.recent - Kacheon Poon

However, it appears that the majority of these points is already addressed in the last 1323bis draft. Perhaps some more explicit text is needed there though (i.e. the use of the maximum seen Rcv.Window when sending retransmissions instead of the current advertised Rcv.Window; currently there is only a pointer to RFC1122 and as an implementer, extrapolating the last sentence from there is not necessarily intuitive).
  
Best regards,

Richard Scheffenegger

> From: Anantha Ramaiah (ananth) [mailto:[email-masked]]
> Sent: Montag, 02. April 2012 06:49
> To: [email-masked]
> Subject: RE: [tcpm] A new ID on TCP option space extension.
> 
> 
> 
> Ok, maybe having a new TCP option for this purpose is height of paranoia.
> RFC 1323 is very common and implementations have followed the wordings
> of RFC verbatim esp, the place where it says :-
> 
> So, some over zealous firewalls, would interpret this to mean an attack (since
> it would crash the end system, if it is buggy) may strip this option OR would
> drop the packet. Sorry if I am sounding like hand-waving, we should get
> some recent data as you say below. [I am also planning to check our firewall
> implementations and get back...]
> 
> In any case, that should not prevent one from writing an ID which updates
> RFC 1323. I fully support it for obvious reasons :-) I have 2 questions :-
> 
> - Are we only going to target timestamps Or make it generic looking to say
> other TCP options can also be negotiated mid-way? Or are we going to say
> that only timestamp would be an exception? For example I would think

---

From: Andre Oppermann <[email-masked]>
Date: Mon, 02 Apr 2012 19:16:50 +0200

On 02.04.2012 11:38, Scheffenegger, Richard wrote:
>
> Anantha,
>
> There are a bunch of mid-stream, temporary options for MP-TCP; Actually, I'm more concerned
> because TS are not a "unknown" option for middleboxes, but middleboxes may have code to interpret
> these (check the semantics etc).
>
> On the other hand, any implementation that wants to go with late TS negotiation must be prepared
> that Timestamps may not be negotiated / reflected then - and treat this in a graceful way. If the
> reason for late TS negotiation failing is the other TCP end not supporting it, or any random
> middlebox removing the option because of RFC1323 semantic checks, doesn't make any difference.
>
> The only thing that we really should verify is, if there are middleboxes that will actually
> terminate a TCP session, when all of a sudden the TS option appears... I hope this won't be the
> case, but it needs to be verified.

It's really difficult to change the behavior of a system that was very
clearly spelled out 20 years ago.  It's too ingrained everywhere, too
many (rightful) assumptions have been made and any change will just
open a new can of worms.

> So, if you can verify the firewall code and it's interactions, that would be a very good starting

We're even having trouble with ordering of options.  Don't walk into
the fragile lands of obscure errors.

> For a 1323bis, I think we should limit this late negotiation to timestamps for now; other options
> are really beneficial from the start (SACK, MPTCP,MSS), while stil others would require some
> definition of new semantics if late negotiation would be used (WS).

I strongly recommend against changing RFC1323 timestamps in any way.
Make it new options but leave the current behavior as it is.

> I have been asked to look at 1323bis and try to revive it - currently in the process of updating
> the formatting, references and boilerplate to the current standard;
>
> This is a short list of the issues that were raised during RFC1323bis discussions (and that I was
> able to dig up; if anyone has more info, please let me know):
>

?

 > * RTO adjustments (no need to use every sample, on update per RTT enough) - Mark

Agreed as MAY.

> * Enabling TS (PAWS) midstream - David, Matt, and others

Disagreed due to interop issues.  The current behavior is engraved in
stone in 20 years worth of implementations of any kind.  We have enough
trouble keeping TCP fully interoperable as it is.  Don't add yet another
fuzzy failure case.  Even worse that connections may suddenly fail mid-
transfer without anyone knowing why (because firefall gets tripped up).

And don't forget that timestamps add some entropy to the already limited
32^2 sequence space.

> * PAWS and non-DF segments (incorrectly reassembled packets, not captured by TCP checksum) [?]

Shit happens... What can you do short of TCP-AO?

 > * References / interaction with SACK (TS check does not look if SACK is
 > present - to cover certain cased of lost ACKs that inflate RTO) [?]

The presence of SACK removes ambiguity from timestamps and also from
the somewhat brittle window update logic (see my long post from some
two years ago).

 > * Updating TS.recent - Kacheon Poon

Updating TS.recent with SACK would be nice.

> However, it appears that the majority of these points is already addressed in the last 1323bis
> draft. Perhaps some more explicit text is needed there though (i.e. the use of the maximum seen
> Rcv.Window when sending retransmissions instead of the current advertised Rcv.Window; currently
> there is only a pointer to RFC1122 and as an implementer, extrapolating the last sentence from
> there is not necessarily intuitive).

-- 
Andre