Subject: [tcpm] 64-bit sequence numbers

From: Jonathan Looney <[email-masked]>
Date: Wed, 15 Mar 2017 09:17:08 -0400

Hi all,

At the Seoul meeting, we saw two proposals for making TCP perform better on
high-speed networks: increasing the maximum window size to 2**31 bits and
increasing the precision of timestamps to the microsecond or nanosecond
level. As I said in the meeting at Seoul, the combination of these two
proposals led me to believe that we need to begin planning for 64-bit
sequence numbers.

Given that it takes years to rollout new TCP features (especially something
as fundamental as changing sequence numbers), it seems like it is time to
begin thinking about how we will do this.

I submitted the below I-D with a proposal for 64-bit sequence numbers. I'm
sure the proposal isn't perfect. The draft itself highlights some
challenges. I asked the chairs to allocate a few minutes in Chicago to
discuss this topic. Even if we decide to go in a different direction than
this draft, I think it is important that we begin to think about 64-bit
sequence numbers.

Jonathan

---------- Forwarded message ----------
From: <[email-masked]>
Date: Mon, Mar 13, 2017 at 7:32 PM
Subject: New Version Notification for draft-looney-tcpm-64-bit-seqnos-00.txt
To: Jonathan Looney <[email-masked]>

A new version of I-D, draft-looney-tcpm-64-bit-seqnos-00.txt
has been successfully submitted by Jonathan Looney and posted to the
IETF repository.

Name:           draft-looney-tcpm-64-bit-seqnos
Revision:       00
Title:          64-bit Sequence Numbers for TCP
Document date:  2017-03-13
Group:          Individual Submission
Pages:          17
URL:            https://www.ietf.org/internet-drafts/draft-looney-tcpm-64-
bit-seqnos-00.txt
Status:         https://datatracker.ietf.org/doc/draft-looney-tcpm-64-bit-
seqnos/
Htmlized:       https://tools.ietf.org/html/draft-looney-tcpm-64-bit-
seqnos-00

Abstract:
   This draft updates RFC 793 to allow the optional use of 64-bit
   sequence numbers.  It also updates other standards to support the
   extended sequence number space.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Mark Allman <[email-masked]>
Date: Wed, 15 Mar 2017 09:32:00 -0400

I haven't read your document (although I'm putting it on my list of
things to look at).  But, I am going to leave this 11 year old piece
of tripe here ...

    https://www.icir.org/mallman/share/draft-allman-tcpx2-hack-00.txt

(At one point I had a student implement this and it isn't all that
difficult and can in fact co-exist with current TCP and largely use
the same code path for both "versions" of TCP.  Alas, it wasn't
properly finished and written up / code released / etc.---which is
clearly my fault.)

allman

--
https://www.icir.org/mallman/
@mallman_icsi

---

From: Joe Touch <[email-masked]>
Date: Wed, 15 Mar 2017 13:25:27 -0700

Hi, Jonathan (et al.),

A few issues:

1) why do you want to use all 64 bits for the TCP-AO MAC, but only the
low 32 bits for the KDF? The latter won't help with backward
compatibility, so why do this? FWIW, one mode of TCP-AO ignores options;
I don't see how you can say "ignore options" (experimental - RFC6978).
This doc probably needs to explain what happens when "ignore options" is
set (for NAT traversal), i.e., the sequence number extensions will be
ignored completely for TCP-AO in that case, AFAICT.

2) the doc should address how to handle RSTs, esp. if the state goes
away. It seems like a connection negotiated with 64-bit sequence numbers
can't ever be reset if the connection state is lost (all the RSTs will
be treated as out of window

3) there are more things than just TCP-AO that use ISNs as input; it
seems like you need a blanket statement on how they're handled. This
also then seems to kill automatic fall-back, AFAICT.

4) the doc needs to explain the impact of using this feature on each
segment on the TCP option space

5) the doc should probably address the impact of odd middleboxes, e.g.,
that split or splice segments or change sequence numbers

That's just scratching the surface; I fear this is a Gordian knot of
issues for which there is no simple sword.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 15 Mar 2017 13:30:56 -0700

Hi, Mark (et al.),

I was thinking about this too, but as I recall the challenge was legacy
fallback.

I'm not convinced the proposed approach has safe fallback either, though.

Joe

On 3/15/2017 6:32 AM, Mark Allman wrote:
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Jonathan Looney <[email-masked]>
Date: Mon, 20 Mar 2017 14:12:30 -0400

Dear Joe,

Thanks for taking the time to provide feedback!

> ignored completely for TCP-AO in that case, AFAICT.

Because the KDFs are expected to take 32-bit sequence numbers, it seemed to
make sense to only use 32-bit sequence numbers as input to these functions.
If there is a desire to take 64-bit sequence numbers as input to the KDFs,
we could certainly specify this; however, I believe this would require
modifying the specifications for KDFs, and I wanted to avoid going farther
than necessary in this draft.

The MAC only takes 64-bit sequence numbers once we are reasonably sure that
the session supports 64-bit sequence numbers. Because the MAC already used
a 64-bit sequence number, there were not the same concerns with MACs as the
KDF.

I don't see an automatic conflict between ignoring TCP options while
constructing the MAC on the one hand, and using 64-bit sequence numbers in
the MAC on the other hand. Can you explain why you think this is a conflict?

> be treated as out of window

I'm not sure I understand this comment. If the host sending the RST
understands 64-bit sequence numbers, it can use full 64-bit sequence
numbers in the RST. If the host doesn't understand 64-bit sequence numbers,
it will use 32-bit sequence numbers and (after the three-way handshake)
that RST will be ignored. While that *may* result in ignoring "legitimate"
RSTs from middleboxes that don't understand 64-bit sequence numbers, I'm
not sure this is a fatal flaw. After all, in today's Internet, it seems
better to err on the side of ignoring legitimate RSTs rather than accepting
illegitimate RSTs.

> also then seems to kill automatic fall-back, AFAICT.

I'm not sure which features you are referring to. If you can provide
specifics, I can try to provide a more specific answer.

The general principal is to use 32-bit sequence numbers until you are sure
you can successfully negotiate 64-bit sequence numbers. So, if a feature
needs to use ISNs in the three-way handshake, it will need to use 32-bit
sequence numbers. If a feature uses ISNs after that point, it can use
64-bit sequence numbers.

More details may be needed. (e.g. Does a feature switch to using 64-bit
sequence numbers as soon as it detects support?) A general rule will need
to be conservative (if you need to use ISNs in the three-way handshake, you
need to always use 32-bit sequence numbers); however, specific rules can be
more tailored to the specific feature. If there are features that would
benefit from the use of 64-bit sequence numbers, we should be sure to
specify how they can use them.

If this is not explained thoroughly enough in the document, I would be
happy to modify it to explain the general rule in more detail. (Again, more
specific examples will help, since there are limits to how specific you can
be in a "general rule".)

> segment on the TCP option space

I'm not sure what you mean by this. Can you elaborate?

> that split or splice segments or change sequence numbers

Fair enough. IMHO, such middleboxes are already broken, since they can't
possibly know how to properly modify the TCP options. However, it is a good
point that the document should consider this.

> issues for which there is no simple sword.

It may be that there are challenges to using 64-bit sequence numbers.
However, I think it is important to find a way forward on this. Otherwise,
I fear we will "soon" (in TCP modification timelines) run into performance
limitations.

Thanks!

Jonathan

---

From: Joe Touch <[email-masked]>
Date: Mon, 20 Mar 2017 11:30:38 -0700

See below...

Joe

On 3/20/2017 11:12 AM, Jonathan Looney wrote:
> On Wed, Mar 15, 2017 at 4:25 PM, Joe Touch <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi, Jonathan (et al.),
>
>     A few issues:
>
> Because the KDFs are expected to take 32-bit sequence numbers, it
> seemed to make sense to only use 32-bit sequence numbers as input to
> these functions. If there is a desire to take 64-bit sequence numbers
> as input to the KDFs, we could certainly specify this; however, I
> believe this would require modifying the specifications for KDFs, and
> I wanted to avoid going farther than necessary in this draft.
I don't see a rationale for treating the two differently... (see the
next note)

> The MAC only takes 64-bit sequence numbers once we are reasonably sure
> that the session supports 64-bit sequence numbers. Because the MAC
> already used a 64-bit sequence number, there were not the same
> concerns with MACs as the KDF.
The MAC uses an *internal* SNE; you're overriding that by using the
extension in the option. That's a change too.

> constructing the MAC on the one hand, and using 64-bit sequence
> numbers in the MAC on the other hand. Can you explain why you think
> this is a conflict?

I don't think you should be say two conflicting things to the protocol -
"ignore options" means ignore them all, not "ignore all except this one".

> numbers in the RST. If the host doesn't understand 64-bit sequence
> numbers, it will use 32-bit sequence numbers and (after the three-way
> handshake) that RST will be ignored. While that *may* result in
> ignoring "legitimate" RSTs from middleboxes that don't understand
> 64-bit sequence numbers, I'm not sure this is a fatal flaw.

It means you can't reset a connection. Remember that TCP state doesn't
go away by itself. So you'll accumulate dead connections that will
interfere with new ones - and the new one won't reset the old one away
either. This can be handled, but requires timeouts to cleanup dead
connections, which are not part of the TCP spec. This was discussed in
TCP-AO.

> After all, in today's Internet, it seems better to err on the side of
> ignoring legitimate RSTs rather than accepting illegitimate RSTs.

Agreed, but you need to discuss it and find a way to reset connections
when state is lost.

> specifics, I can try to provide a more specific answer.
Search the RFCs for ISNs. Some crypto algs use ISNs as a source or
"randomness" (incorrectly, of course). There are also RFCs that talk
about generating ISNs. A scrub of these will give you a list of what to
address.

>  
> The general principal is to use 32-bit sequence numbers until you are
> sure you can successfully negotiate 64-bit sequence numbers. So, if a
> feature needs to use ISNs in the three-way handshake, it will need to
> use 32-bit sequence numbers. If a feature uses ISNs after that point,
> it can use 64-bit sequence numbers.

If that's true, then TCP-AO MAC cannot use the 64-bit ISNs in MAC
calculations until after the handshake completes! Otherwise, you can't
fall-back.

>
> More details may be needed. (e.g. Does a feature switch to using
> 64-bit sequence numbers as soon as it detects support?) A general rule
> will need to be conservative (if you need to use ISNs in the three-way
> handshake, you need to always use 32-bit sequence numbers); however,
> specific rules can be more tailored to the specific feature. If there
> are features that would benefit from the use of 64-bit sequence
> numbers, we should be sure to specify how they can use them.

This is a dangerous pit. TCP should have one set of assumptions at the
start of a connection, and another after the TWHS completes. Doing
changes at any other time is perilous and ill-defined.

> happy to modify it to explain the general rule in more detail. (Again,
> more specific examples will help, since there are limits to how
> specific you can be in a "general rule".)
>
>

You're using bytes for this option, and option space is very limited.
See the discussion of option space in TCP-AO.

> Fair enough. IMHO, such middleboxes are already broken, since they
> can't possibly know how to properly modify the TCP options. However,
> it is a good point that the document should consider this.

Right - the issue is whether you give false positives and whether you
can detect these and fall back (or should).

> However, I think it is important to find a way forward on this.
> Otherwise, I fear we will "soon" (in TCP modification timelines) run
> into performance limitations.

"Important" doesn't mean "possible". Don't get ahead of that issue, please.

Joe