Subject: Re: [tcpm] A new ID on TCP option space extension.

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Tue, 20 Mar 2012 00:28:41 -0700

Forgot to add :-

I am thankful to the chairs to squeeze this item for the last 10 mins
left on the agenda. Of course if things take more time for other
allotted slots etc.,,  this slot has the risk of getting shunted out.  I
hope we get  some time  to discuss this during the meeting, if not we
can always continue the discussion on the mailing list.

Thanks,

-Anantha

From: Anantha Ramaiah (ananth) 
Sent: Monday, March 19, 2012 7:50 PM
To: [email-masked]
Subject: A new ID on TCP option space extension.

Hi,

      I have a new ID on TCP option space extension. Basically, sometime
back I had volunteered  to  write up a draft which talks about the TCP
option space issue, motivation, current list of proposals , other ideas
if any and what we should moving forward  on this.

     Here is the link :- (I could not meet the draft -00- cut off and
hence posting the link)

ftp://ftpeng.cisco.com/ananth/draft-ananth-tcpm-tcpopt.txt

Please review and provide feedback.  In the upcoming TCPM meeting there
is a slot allotted to go over this, but I would appreciate if there is
earlier feedback.  (At least this issue is well known and has been
discussed several times in the past, in some sense nothing new)

Note :  there is some scope to improve the structure of the draft, like
when comparing each of the proposals one could chose a tabular form OR
at least do comparisons in an orderly fashion (like M1, M2 to M6)
instead of the way currently it is. 

Thanks,

-Anantha

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Fri, 30 Mar 2012 14:45:47 +0000

Hi Anantha,

After a hint by Matt, I dug out this mail; my comments follow afterwards.

*  To: TCP Maintenance and Minor Extensions WG <tcpm at ietf.org<mailto:[email-masked]>>
*  Subject: [tcpm] RFC 1323: Timestamps option
*  From: David Borman <david.borman at windriver.com<mailto:[email-masked]>>
*  Date: Fri, 26 Jan 2007 12:33:29 -0600

One of the topics that has been discussed in the past for the revision of RFC 1323 is to relax the requirement on when to send the Timestamps option. I'd like to come to some resolution on this issue.

Current:
The Timestamps option is negotiated during the initial SYN exchange. If both sides support it, then every packet in the connection has to have the Timestamps option.

Option #1
---------
The proposed change is to relax the requirement that Timestamps have to be negotiated in the initial SYN, and instead allow them to be enabled later on in the connection. At the time 1323 was originally written, there was a big concern for legacy TCP stacks that didn't handle unknown options in the middle of the connection, but were able to properly ignore unknown options during the SYN exchange. But that was a long time ago, and unknown options in the middle of a TCP connection *shouldn't* be as big of an issue anymore.

The rules for enabling Timestamps mid-stream would be:
1) If you receive a Timestamps option mid-stream, you can enable Timestamps. From then on, every packet sent will need to contain a Timestamps option.
2) If you want to enable Timestamps mid-stream, you can only do so on a data packet. You record the sequence number of that data packet, and include the Timestamps option on every packet from then on, until you receive an ACK for that sequence number. If at any time you receive a packet with the Timestamps option, then they are enabled for the rest of the connection. If you do not receive a packet with a Timestamps option by the time you get the ACK for that initial sequence number, then the other side has not enabled Timestamps and you must stop sending Timestamps for the duration of the connection.

The restriction on only being able to initiate Timestamps on a data packet is that they are the only packets that are delivered reliably, so determining whether or not the other side supports them is deterministic by getting the ACK for that data.

Option #2
---------
This is similar to Option #1, but not quite the same. In this case, the Timestamps would be negotiated in the initial SYN as before, but then no additional packets would have Timestamps options. At any point during the connection, either side can enable the use of Timestamps just by starting to send them. Since we already know that both sides support them, once you start sending them you never stop. And if you every receive one, you are then required to also send them for the duration of the connection.

The question then is, how do you negotiate Timestamps without turning them on? My suggestion would be to send a Timestamps option with the special format TSval=0,TSecr=0. If you receive this in a SYN, then the request is to negotiate knowledge of the Timestamps option. The returning SYN,ACK would also need to contain TSval=0,TSecr=0 to complete the negotiation. If you are connecting to a site that doesn't support this feature but does support Timestamps, it would respond with TSval=xxx,TSecr=0. At that point, Timestamps are now enabled. The downside of this is that now the originator has started their Timestamps values at 0, and must continue from there with their timestamps; they can't just suddenly jump to some arbitrary value because that could break PAWs.

Option #3
---------
Do nothing, and leave Timestamps alone as they are currently defined. You negotiate them in the SYN, and then include them in every packet from then on.

One of the issues with Timestamps is that the TSecr value is defined to be valid anytime the ACK bit is set. This works well with the initial SYN negotiation, but when you enable Timestamps midstream, the initiator has to send a Timestamps option with an invalid TSecr value, but the ACK bit will be set. That's the case for both of the previous options.

So why should we change things? My recollection was that the primary motivation was to conserve option space, to avoid having to put in the extra 12 bytes of TCP options on every packet until we get to the point where the sequence space might wrap, and then turn them on so that we can get PAWS to protect the rest of the connection. But I'm not sure I buy that argument. One of the things that you lose by deferring enabling Timestamps is that you don't get the RTT measurements that Timestamps provide. Without Timestamps, using the typical BSD algorithm of timing one segment and waiting for an ACK, and adding in the Karn algorithm that you have to invalidate any measurements if you had to retransmit in that window, means that at best, you'll get one measurement per RTT. At worst, you can get in a state where you never get a valid RTT measurement. When you use the Timestamps option, the worst case is that you'll only get one measurement per RTT, and the best case is that you'll get multiple valid samples,

My viewpoint
------------
Right now, I'm inclined to leave the Timestamps option alone; you negotiate it during the SYN exchange, and then include it on every packet. I'm willing to be convinced otherwise, but now I've placed a stake in the ground, and the mailing list will have to decide if there is sufficient reason to change the behavior.
                       -David Borman

Perhaps we should review some of the options that  are large and fairly typical. The mentioned timestamp option is  prime candidate to try and shift that to somewhere later in the stream.

Timestamps really are only useful and necessary (in their current form) for "high-speed" links (>100 Mbit), and then also only when there are many packets in flight. During the initial few (data) packets, initial RTO and the limited initial window, a session could do without timestamps...

However, not having timestamps in the SYN - as a virtual extention of the sequence number space, assuming the timestamps between two hosts are monotonically increasing also across sessions - would impact certain ad-hoc features (i.e. reducing the 2MSL port reuse policy).

>From the above options, Option #1 seems to be most workable - with minimal code chance. I even suspect, that some stacks implicitly support that already from my timestamp tests some time ago (where I stopped sending timestamps, after they were negotiated for, and then sent them again).

Sending the first timestamp only with the first data packet (and maintaining sending timestamps thereafter for one full window/RTT) could be useful. Perhaps also allowing a passive sender (web server) to initiate this, when the first data is sent from that side. This may mean that a  negotiation is started only once a client request has made it in full to the server. After all, moving these 10 bytes / 80 bits away from the SYN would yield just as many bits as what Tim today mentioned in his comment to your presentation....

Another approach (Option #4) would be to define a new option, without the actual fields - much alike how it is done with SACK - to prime for timestamps in the SYN. That would yield only 8 bytes. And it could be done even without asking for a new tcp option codepoint, if the tuple <len,option> is used. Len=2, option=timestamp is currently invalid, but could be used to signal the delayed use of timestamps...

Are there any other options, that would allow for lazy negotiation (with a data segment), without breaking things?

Best regards,

Richard Scheffenegger

Anantha Ramaiah (ananth)
Sent: Dienstag, 20. März 2012 08:29
To: [email-masked]
Subject: Re: [tcpm] A new ID on TCP option space extension.

Forgot to add :-

I am thankful to the chairs to squeeze this item for the last 10 mins left on the agenda. Of course if things take more time for other allotted slots etc.,,  this slot has the risk of getting shunted out.  I hope we get  some time  to discuss this during the meeting, if not we can always continue the discussion on the mailing list.

Thanks,
-Anantha

From: Anantha Ramaiah (ananth)
Sent: Monday, March 19, 2012 7:50 PM
To: [email-masked]<mailto:[email-masked]>
Subject: A new ID on TCP option space extension.

Hi,
      I have a new ID on TCP option space extension. Basically, sometime back I had volunteered  to  write up a draft which talks about the TCP option space issue, motivation, current list of proposals , other ideas if any and what we should moving forward  on this.
     Here is the link :- (I could not meet the draft -00- cut off and hence posting the link)

Please review and provide feedback.  In the upcoming TCPM meeting there is a slot allotted to go over this, but I would appreciate if there is earlier feedback.  (At least this issue is well known and has been discussed several times in the past, in some sense nothing new)

Note :  there is some scope to improve the structure of the draft, like when comparing each of the proposals one could chose a tabular form OR at least do comparisons in an orderly fashion (like M1, M2 to M6) instead of the way currently it is.

Thanks,
-Anantha

---

From: Mark Allman <[email-masked]>
Date: Fri, 30 Mar 2012 11:20:57 -0400

David-

> So why should we change things? My recollection was that the primary
> motivation was to conserve option space, to avoid having to put in the
> extra 12 bytes of TCP options on every packet until we get to the
> point where the sequence space might wrap, and then turn them on so
> that we can get PAWS to protect the rest of the connection. 

I think that is the right motivation.  Why send timestamps until they
are actually useful for something?  Until you need PAWS to kick in
you're just wasting space.

> One of the things that you lose by deferring enabling Timestamps is
> that you don't get the RTT measurements that Timestamps
> provide. Without Timestamps, using the typical BSD algorithm of timing
> one segment and waiting for an ACK, and adding in the Karn algorithm
> that you have to invalidate any measurements if you had to retransmit
> in that window, means that at best, you'll get one measurement per
> RTT. At worst, you can get in a state where you never get a valid RTT
> measurement. When you use the Timestamps option, the worst case is
> that you'll only get one measurement per RTT, and the best case is
> that you'll get multiple valid samples,

(1) Multiple RTT samples per RTT is not useful.

    (a) It was conjectured in the original RFC that multiple samples per
        RTT was useful, but never shown.

    (b) There is empirical evidence that suggests multiple samples give
        no benefit.  (See Allman/Paxson, SIGCOMM 1999.)

    Or, put another way: Please show us *evidence* that indicates
    multiple RTT samples per RTT are useful.

(2) If you can't get an RTT sample because of Karn's algorithm you're
    essentially hosed anyway.  

    (a) Lots of the traffic doesn't use timestamps now and so regardless
        of what we do with the option going forward lots of connections
        won't be helped by the TS option.

    (b) It seems a steep price that *everyone* should *always* use the
        TS option on *every* packet for the *rare* cases when the
        network is *pathologically* wonky enough to prevent a valid RTT
        sample from being taken.

I think the evidence points to enabling the TS option only when it is
necessary for PAWS being the right approach.  

(And, in fact, ideally just getting rid of the TS option and using a
counter option for PAWS that would be smaller in size would be better.
But, leveraging the installed base of TS-savvy systems seems useful.)

allman

---

From: David Borman <[email-masked]>
Date: Fri, 30 Mar 2012 11:31:27 -0500

> necessary for PAWS being the right approach.

While multiple samples per RTT may not be useful, as you have alluded
to, with the Karn algorithm at *best* you'll get one sample per RTT,
whereas with timestamps, at *worst* you'll get one time sample per RTT.
The Karn algorithm can only get an RTT sample when a new ACK arrives and
there haven't been any retransmissions.  Using timestamps you don't have
that ambiguity, and you can calculate the RTT in every new ACK.  Maybe
you only use one sample per RTT, but at least you are guaranteed to get
at least one sample.

			-David Borman

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Mark Allman <[email-masked]>
Date: Fri, 30 Mar 2012 12:47:04 -0400

> to,

(I didn't allude to it.  I brought *data* to the table.  Just FYI.)

> with the Karn algorithm at *best* you'll get one sample per RTT,
> whereas with timestamps, at *worst* you'll get one time sample per
> RTT.  The Karn algorithm can only get an RTT sample when a new ACK
> arrives and there haven't been any retransmissions.  Using timestamps
> you don't have that ambiguity, and you can calculate the RTT in every
> new ACK.  Maybe you only use one sample per RTT, but at least you are
> guaranteed to get at least one sample.

Right - I fully understand that.  And, if all other things were equal
we'd clearly prefer "at worst one sample" to "at best one sample".  But,
all other things are not equal.

Two things in addition to my (2) above:

  - If that happens for an RTT or two then Who Cares?  It is quite
    unlikely to have much impact on the estimator---which is the thing
    we really care about.

  - The only case I can see really caring about in terms of Karn's
    algorithm is the case you sketched where you couldn't get *any*
    sample.  That is bad.  But, also pretty pathological.  I don't see
    engineering every packet on the Internet for this pathology.  

    (Not to mention if you know you'll be in an environment where this
    is likely you can just use TS from the SYN.)

If getting rid of the need for Karn's algorithm is the entire reason to
keep timestamps in every packet I think that is very, very thin.

allman

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sat, 31 Mar 2012 08:13:23 +0000

Hi Mark,

> If getting rid of the need for Karn's algorithm is the entire reason to keep
> timestamps in every packet I think that is very, very thin.

Actually, allowing for delay-based congestion algorithms like LEDBET, CTCP may be a stronger use case for timestamps. However, the way they are signaled currently (opaque to the receiver), these controllers all rely on more or less good heuristics to interpret the timestamp.

I'll rewrite my draft to emphasize on this in the next couple weeks.. 

Richard Scheffenegger

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Mark Allman
> Sent: Freitag, 30. März 2012 18:47
> To: David Borman
> Cc: [email-masked]; Anantha Ramaiah (ananth)
> > > I think the evidence points to enabling the TS option only when it
> > > is necessary for PAWS being the right approach.
> >
> Right - I fully understand that.  And, if all other things were equal we'd clearly
> prefer "at worst one sample" to "at best one sample".  But, all other things
> are not equal.
> 
> If getting rid of the need for Karn's algorithm is the entire reason to keep
> timestamps in every packet I think that is very, very thin.
> 
> allman
> 
>

---

From: Mark Allman <[email-masked]>
Date: Sat, 31 Mar 2012 10:02:58 -0400

> Actually, allowing for delay-based congestion algorithms like LEDBET,
> CTCP may be a stronger use case for timestamps. 

That is absolutely fine.  I am not saying we should rid the world of
timestamps.  I am saying that we should re-arrange the world such that
TCP connections do not have to use timestamps because they *might* need
them later.  If there is some good reason to use them from the onset of
the connection, great, use them.  But, 

  - If RTTM is the reason to use the TS option then I haven't seen the
    empirical evidence that says this is at all helpful and so I am
    pretty dubious.

  - If PAWS is the reason to use the TS option then I don't see why we'd
    waste the bytes on the TS option for connections that send less than
    4GB (i.e., that will not wrap the sequence space).  So, being able
    to turn on the TS option mid-connection when PAWS is required makes
    good sense to me.  

So, I am voting for an RFC 1323bis that calls for negotiating timestamps
only when they are needed.  And, if some new use needs to do so at the
beginning of the connection and use them throughout then we should
absolutely not preclude that.

allman

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Sat, 31 Mar 2012 19:12:32 -0700

<just got back to SFO. Catching up...>

RTTM has been touted to be the main use for timestamps which includes
solution to the Karn's ambiguity problem. The other one is PAWS and
w.r.t I agree it is wastage of TCP option bytes and the idea of turning
on PAWS detection dynamically (using timestamps) makes sense to me.
Changing the timestamp semantics means change to many TCP/IP stacks
(variables like ts_val, ts_recent would not be interpreted as before,
all other debugging tools etc., may need to understand this change)

Even if we are planning to do so, I thought you meant to negotiate the
capability during SYN (don't send the timestamp value) and send
timestamps when needed. Or are we saying that we are trying to negotiate
the option when needed, this sounds like a bigger change in semantics
since every TCP option is only negotiated during SYN as it stands today.
To make this more useful, you would want to not only turn on this
capability when needed, also turn off when not needed.

Didn't we already have a RFC 1323bis document? Is that expired or did it
make it to the RFC 1323 update ? If not, you could always fold this idea
(assuming there is a consensus) into that document. Not sure whether we
would want multiple RFC 1323bis documents...

Just my 2 cents..

-Anantha

> From: [email-masked] [mailto:[email-masked]]
> Sent: Saturday, March 31, 2012 7:03 AM
> To: Scheffenegger, Richard
> Cc: David Borman; [email-masked]; Anantha Ramaiah (ananth)
> So, I am voting for an RFC 1323bis that calls for negotiating
> timestamps only when they are needed.  And, if some new use needs to
do
> so at the beginning of the connection and use them throughout then we
> should absolutely not preclude that.
> 
> allman
> 
>