Subject: [tcpm] comment about draft-nishida-tcpm-maxwin-02

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 19 Jan 2017 08:30:51 +0100

Hi,

Thanks for writing this draft, seems useful to me.

A couple of minor comments:

- I understand that this extension MUST be used with PAWS, correct? I 
mean, using a larger RCVWND without PAWS is likely to result in 
inability to distinguish old duplicate packets from new ones, correct? 
Should this requirement of this being used in conjunction with PAWS be 
explicitly stated?

- If an endpoint that does support this extension communicates with 
endpoints that do not support this extension, it will result in the 
endpoint that does support the extension to be allocating the double 
buffer size for every connection with a legacy endpoint that it should. 
I agree that is not fatal, but wouldnt this be a problem in terms of 
resources? Would this excess of demand in resource usage would make 
adoption of this extension not very attractive? Maybe it would be better 
to use the 15 shift count when the other endpoint also replies with a 15 
shift count or something like this?

- The second paragraph in section 4, it talks about "performance 
degradation caused by misinterpretation of the shift count" I dont 
understand what you are referring to, can you clarify?

thanks, marcelo

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 19 Jan 2017 13:01:41 -0800

Hi Marcelo,

Thanks for reading the draft! Sorry, I forgot to CC to tcpm..So, I resent
it.

On Wed, Jan 18, 2017 at 11:30 PM, marcelo bagnulo braun <[email-masked]>
wrote:

> mean, using a larger RCVWND without PAWS is likely to result in inability
> to distinguish old duplicate packets from new ones, correct? Should this
> requirement of this being used in conjunction with PAWS be explicitly
> stated?
>

That's right. We presume the use of PAWS here. We can add some texts.

> endpoint that does support the extension to be allocating the double buffer
> size for every connection with a legacy endpoint that it should. I agree
> that is not fatal, but wouldnt this be a problem in terms of resources?
> Would this excess of demand in resource usage would make adoption of this
> extension not very attractive? Maybe it would be better to use the 15 shift
> count when the other endpoint also replies with a 15 shift count or
> something like this?
>

Yes, we have thought about it before. One potential issue of the approach
is it will require other endpoint to use 15 shift count, although it might
want to use smaller shift count. WS option exchange is one way
notification. So, I just didn't want to use it as a negotiation mechanism
to activate the feature.
On the other hand, we presumed not so many TCP connections want to use this
feature, such as at most 10 connections at one time.
Also, adding a certain negotiation mechanism to activate the feature could
be another way.  (01 version has described the mechanism, which is deleted
in 02 version) We've compared these points and have chosen the current
mechanism.

> understand what you are referring to, can you clarify?

When an endpoint A offers shift count 15 but a conventional endpoint B
regards it as 14, we could have performance degradation issue.
Because when A says "I have X<<15 buffer", B will think "A has X<<14
buffer".

But, when X=65535, we shouldn't see performance degradation issue because
65535<<14 is the max window size that B can support.
So, if A allocates 65535<<15 + 65535<<14 buffer, A can always send X=65535
because B's buffer is at most 65535<<14, it cannot consume the buffer at A
more than that.

Thanks,
--
Yoshi

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 20 Jan 2017 08:27:19 +0100

below...

El 19/01/17 a las 22:01, Yoshifumi Nishida escribió:
> Hi Marcelo,
>
> Thanks for reading the draft! Sorry, I forgot to CC to tcpm..So, I 
> resent it.
>
> On Wed, Jan 18, 2017 at 11:30 PM, marcelo bagnulo braun 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hi,
>
>     - I understand that this extension MUST be used with PAWS,
>     correct? I mean, using a larger RCVWND without PAWS is likely to
>     result in inability to distinguish old duplicate packets from new
>     ones, correct? Should this requirement of this being used in
>     conjunction with PAWS be explicitly stated?
>
>

I understand it is not safe to use the increased window without paws, 
so, i would think the draft should sate that PAWS MUST be used with the 
increased window.

>
>     - If an endpoint that does support this extension communicates
>     with endpoints that do not support this extension, it will result
>     in the endpoint that does support the extension to be allocating
>     the double buffer size for every connection with a legacy endpoint
>     that it should. I agree that is not fatal, but wouldnt this be a
>     problem in terms of resources? Would this excess of demand in
>     resource usage would make adoption of this extension not very
>     attractive? Maybe it would be better to use the 15 shift count
>     when the other endpoint also replies with a 15 shift count or
>     something like this?
>
>
> Yes, we have thought about it before. One potential issue of the 
> approach is it will require other endpoint to use 15 shift count, 
> although it might want to use smaller shift count. WS option exchange 
> is one way notification. So, I just didn't want to use it as a 
> negotiation mechanism to activate the feature.

Right, imposing the use of 15 shift count in a connection endpoint that 
doesnt need it is also a waste of resources, so depending which endpoint 
is more constrained in resources this may or may not be a good approach 
(and we dont know which one is the bottleneck in advance).

Moreover, i would think that supporting very different buffers at the 
endpoints of a connection is very important. I mean, in a connection 
where a server providing content to a client, the client needs the big 
rcv buffer and the server doesnt and it is the server who has the 
greater demand if it is serving a large number of clients.

> On the other hand, we presumed not so many TCP connections want to use 
> this feature, such as at most 10 connections at one time.

I am uncertain about this argument. I would assume that the bandwidth 
will increase (for links and for connections) and hopefully latency will 
decrease in the future (eliminating bufferbloat and with mechanisms like 
l4s) so i think we should design this to be widely used, dont you think?

> Also, adding a certain negotiation mechanism to activate the feature 
> could be another way.  (01 version has described the mechanism, which 
> is deleted in 02 version) We've compared these points and have chosen 
> the current mechanism.

Right, just checked this. I am not sure it is worth consuming a new 
option codepoint for this, and especially not the use of the new option 
in the SYN packet, which is already crowded.

What about this:
The shift count is expressed in a byte, but only the values between 1 
and 14 are acceptable as per RFC 7323.
With this draft, the 15 shift count value would become acceptable, but 
it seems we will never need more than 16 values in the shift count 
(unless we increase the TCP seq number).
So, this means that we have 4 bits to play with in the shift count field.
One way of using it would be the following:
We divide the shift count filed in two fields of 4 bits. The first 4 
bits are used to express the shift count for legacy (RFC7323) endpoints.
The last 4 bits are used to express the shift count for endpoints that 
support this specification.

So, this would result in the following combinations:
- updated client talks to updated server: it expresses its own shift 
count using the last 4 bits, the server replies expressing its own shift 
count in the last 4 bits. Both endpoints understand that they are 
talking to updated endpoints, so they use the new values.
- updated client talks to legacy server. It expresses its own shift 
count using the last bits. The legacy server receives the option, checks 
that the value is larger than 14 and then uses 14 as shift count for 
this client. The server replies with its own shift count encoded in the 
first 4 bits. The client understands that the server is legacy and uses 
a 14 shift count for its own shift count.
- legacy client talking to legacy or updated server, behaves as 
described in RFC7323

If we do this, then when both endpoints are updated, one of the 
endpoints can use a smaller shift count even if the other endpoint uses 
a 15 shift count. This doesnt help in the situation where the server is 
legacy, but at least in the long run if this extension is widely 
deployed, the behaviour is the desired one.

what do you think?

> But, when X=65535, we shouldn't see performance degradation issue 
> because 65535<<14 is the max window size that B can support.
> So, if A allocates 65535<<15 + 65535<<14 buffer, A can always send 
> X=65535 because B's buffer is at most 65535<<14, it cannot consume the 
> buffer at A more than that.
>

So, there is no performance degradation issue :-)
I guess we agree, i guess just find it confusing when the draft talks 
about a performance degradation issue where this is not one.

Regards, marcelo

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 20 Jan 2017 11:32:33 -0800

Hi Marcelo,

Thanks for the comments.

On Thu, Jan 19, 2017 at 11:27 PM, marcelo bagnulo braun <[email-masked]>
wrote:

>> On Wed, Jan 18, 2017 at 11:30 PM, marcelo bagnulo braun <
>> [email-masked] <mailto:[email-masked]>> wrote:
>>
>>     Hi,
>>
> I understand it is not safe to use the increased window without paws, so,
> i would think the draft should sate that PAWS MUST be used with the
> increased window.

Yes, We'll state this point.

> doesnt need it is also a waste of resources, so depending which endpoint is
> more constrained in resources this may or may not be a good approach (and
> we dont know which one is the bottleneck in advance).
>
> endpoints of a connection is very important. I mean, in a connection where
> a server providing content to a client, the client needs the big rcv buffer
> and the server doesnt and it is the server who has the greater demand if it
> is serving a large number of clients.

Yes, I agree.

> I am uncertain about this argument. I would assume that the bandwidth will
> increase (for links and for connections) and hopefully latency will

I actually didn't think about this point very much.
But, if we can have nice design that can support wider use, yes, it's of
course better.

> Right, just checked this. I am not sure it is worth consuming a new option
> codepoint for this, and especially not the use of the new option in the SYN
> packet, which is already crowded.
>

Yep. That's why we didn't include it in the current version.

> What about this:
> The shift count is expressed in a byte, but only the values between 1 and
> 14 are acceptable as per RFC 7323.
> With this draft, the 15 shift count value would become acceptable, but it
> seems we will never need more than 16 values in the shift count (unless we
> increase the TCP seq number).
> We divide the shift count filed in two fields of 4 bits. The first 4 bits
> are used to express the shift count for legacy (RFC7323) endpoints.
> - updated client talks to updated server: it expresses its own shift count
> using the last 4 bits, the server replies expressing its own shift count in
> the last 4 bits. Both endpoints understand that they are talking to updated
> endpoints, so they use the new values.
> - updated client talks to legacy server. It expresses its own shift count
> using the last bits. The legacy server receives the option, checks that the
> value is larger than 14 and then uses 14 as shift count for this client.
> The server replies with its own shift count encoded in the first 4 bits.
> The client understands that the server is legacy and uses a 14 shift count
> for its own shift count.
> - legacy client talking to legacy or updated server, behaves as described
> in RFC7323
>
> If we do this, then when both endpoints are updated, one of the endpoints
> can use a smaller shift count even if the other endpoint uses a 15 shift
> count. This doesnt help in the situation where the server is legacy, but at
> least in the long run if this extension is widely deployed, the behaviour
> is the desired one.
>
> what do you think?

I've thought about it and I think it's a pretty good idea. It's more
explicit than the current scheme, but doesn't consume extra option space.
In this scheme, when an updated endpoint talks to a non-updated endpoint,
the shift count will automatically 14. But, I guess this cannot be solved
without adding extra negotiation scheme. So, I think it's acceptable.
One question I have is if we need to use 4bits for this. It might be ok to
use just one bit in the last 4 bits to indicate it supports new version.
But, this might be a very minor point.

> about a performance degradation issue where this is not one.

Got it. I might think about finding less confusing texts.
Or, with your proposed scheme, we don't need this part.

Thanks,
--
Yoshi

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 20 Jan 2017 21:28:12 +0100

below...

El 20/01/17 a las 20:32, Yoshifumi Nishida escribió:
>
>     What about this:
>     The shift count is expressed in a byte, but only the values
>     between 1 and 14 are acceptable as per RFC 7323.
>     With this draft, the 15 shift count value would become acceptable,
>     but it seems we will never need more than 16 values in the shift
>     count (unless we increase the TCP seq number).
>     We divide the shift count filed in two fields of 4 bits. The first
>     4 bits are used to express the shift count for legacy (RFC7323)
>     endpoints.
>     The last 4 bits are used to express the shift count for endpoints
>     that support this specification.
>
>     - updated client talks to updated server: it expresses its own
>     shift count using the last 4 bits, the server replies expressing
>     its own shift count in the last 4 bits. Both endpoints understand
>     that they are talking to updated endpoints, so they use the new
>     values.
>     - updated client talks to legacy server. It expresses its own
>     shift count using the last bits. The legacy server receives the
>     option, checks that the value is larger than 14 and then uses 14
>     as shift count for this client. The server replies with its own
>     shift count encoded in the first 4 bits. The client understands
>     that the server is legacy and uses a 14 shift count for its own
>     shift count.
>     endpoints can use a smaller shift count even if the other endpoint
>     uses a 15 shift count. This doesnt help in the situation where the
>     server is legacy, but at least in the long run if this extension
>     is widely deployed, the behaviour is the desired one.
>
>     what do you think?
>
>
> In this scheme, when an updated endpoint talks to a non-updated 
> endpoint, the shift count will automatically 14. But, I guess this 
> cannot be solved without adding extra negotiation scheme. So, I think 
> it's acceptable.
> One question I have is if we need to use 4bits for this. It might be 
> ok to use just one bit in the last 4 bits to indicate it supports new 
> version.

agree.
So, the shift count field in the WSO could then look like this:

+----------+
|SSSS|L|RRR|
+----------+

With SSSS being 4 bits used to express the shift count, L, being the bit 
to express the Large window support and RRR 3 reserved bits.

Makes sense?

Regards, marcelo

>         When an endpoint A offers shift count 15 but a conventional
>         endpoint B regards it as 14, we could have performance
>         degradation issue.
>         Because when A says "I have X<<15 buffer", B will think "A has
>         X<<14 buffer".
>
>         But, when X=65535, we shouldn't see performance degradation
>         issue because 65535<<14 is the max window size that B can support.
>         So, if A allocates 65535<<15 + 65535<<14 buffer, A can always
>         send X=65535 because B's buffer is at most 65535<<14, it
>         cannot consume the buffer at A more than that.
>
>
>     I guess we agree, i guess just find it confusing when the draft
>     talks about a performance degradation issue where this is not one.
>
>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 20 Jan 2017 13:14:45 -0800

Hi Marcelo,

On Fri, Jan 20, 2017 at 12:28 PM, marcelo bagnulo braun <[email-masked]>
wrote:

>> One question I have is if we need to use 4bits for this. It might be ok
>> to use just one bit in the last 4 bits to indicate it supports new version.
>>
>
> agree.

Yes, make sense to me. Thanks!
--
Yoshi