Subject: [quicwg/base-drafts] Connection ID Garbage Collection (#1729)

From: Christian Huitema <[email-masked]>
Date: Sun, 09 Sep 2018 09:57:48 -0700

Nodes use the NEW CONNECTION ID frames to provide peers with lists of connection Ids that can be assigned to new paths during migration. If everything goes well, the probing for the new path succeeds, and the peer receives packet bearing the previously communicated connection ID. But things don't always go well. In some cases, the probes will fail. The node that announced the probed connection ID will not now that it has been "consumed". The node will only be able to free the associated resource at the end of the connection. It will not know that the peer needs new connection ID to replace the one that was previously consumed, and the peer will not be able to undertake another migration.

Should we add some signalling to deal with the problem? Maybe a "CONSUMED CONNECTION ID" frame?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729

---

From: Martin Thomson <[email-masked]>
Date: Sun, 09 Sep 2018 17:42:37 -0700

Please see https://github.com/erickinnear/base-drafts/pull/4  - that address this problem (and some others).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-419757689

---

From: Christian Huitema <[email-masked]>
Date: Sun, 09 Sep 2018 21:43:08 -0700

Yes, the "connection ID finished" frame would solve the problem. But how am I supposed to find out a PR in Eric's private repo?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-419785607

---

From: Martin Thomson <[email-masked]>
Date: Sun, 09 Sep 2018 23:06:14 -0700

It was supposed to be shared weeks ago.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-419797983

---

From: erickinnear <[email-masked]>
Date: Mon, 10 Sep 2018 16:39:19 -0700

Yup, CONNECTION_ID_FINISHED should address this, which will be in a PR against the main repo shortly, it's waiting on a final read-through from the members of the CID DT (as I mentioned, hopefully tonight). 

I don't think anyone's expecting you to find that until it's a "real" PR. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420096247

---

From: erickinnear <[email-masked]>
Date: Mon, 10 Sep 2018 16:40:21 -0700

I think sending this frame will not only clean up those that are "burned" by probes that fail, but it will also request new CIDs to replace those that were used by probing, which should cover the other half of the issue above? 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420096432

---

From: Christian Huitema <[email-masked]>
Date: Mon, 10 Sep 2018 19:36:21 -0700

@erickinnear Yes. I would like to keep it very simple: just say "I have burned this connection ID", without specifying how and why. Then, the peer can do its own accounting of how many connection ID are outstanding, etc.

There is a parallel issue, how does a node delete a path? How does it say that "I used to send packets with sourceCID=X and destCID = Y, but I am not going to send anymore packets to destCID=Y, and I would like you to stop sending anything to the CID=X" ?

And yet another complication. Suppose that a node actually removes all resource attached to that old path. That would include the capability to send a packet to CID=X in the example above. Now suppose that despite timers and best efforts an errant packet arrives after that deletion. Since X cannot be routed, this is going to trigger a stateless reset. Does this reset cause the entire connection to crash?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420126670

---

From: Martin Thomson <[email-masked]>
Date: Mon, 10 Sep 2018 19:47:06 -0700

I have a better question: Why wasn't Christian involved with the design team work?

> [...] I would like you to stop sending anything to the CID=X"

The design team concluded that this wasn't a requirement we would address.  If you advertise a connection ID, then you have to be prepared to accept packets with that as a destination for the lifetime of the connection.  That doesn't mean that you have to sit quietly as your peer provides perfect linkability, you can definitely kill a connection if the peer acts irresponsibly, but we concluded that influencing the choice of connection ID from a peer would add too much complication.

> Does this reset cause the entire connection to crash?

Yes.  The expectation is that you can't remove any knowledge of an old connection ID until after some time has passed, even after receiving an indication that your peer has stopped using that value.  In the rare event that you spend that time and you still receive a delayed packet, then yes, you will send a stateless reset.  The hope here is that the peer has forgotten that particular stateless reset token.  That's why we have a recommendation for a timer.  We probably need text in the PR explaining that.  (@erickinnear, if you don't feel inclined to add that, please open an issue.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420128333

---

From: Christian Huitema <[email-masked]>
Date: Mon, 10 Sep 2018 22:12:12 -0700

Martin, I understand that design teams have to be kept rather small in order to be effective and I certainly do not want to be involved in each and everyone of them. But the expectation is that once the initial design is done, we revert to open discussion in the working group. Otherwise, we have something rather different from an IETF process. I believed that the design team completed their work in June, and transferred it to the with the publication of draft-13. I am surprised that the team is still operating today as a design team.

You say that "design team concluded that this wasn't a requirement we would address". Who is we, there? It is fine for the design team to say that they don't want to address a specific requirement, but it does not bind the working group. The product of the team there ought to be the reasoning and the arguments, which may or may not convince the working group.

On the specific issue, I completely agree that we don't want nodes to unilaterally remove support for a particular connection ID, and then send a stateless reset that kills the connection if a laggard packet happens. In fact, that would open a DOS avenue. Attackers could record old connection IDs, wait until after a migration happens, and then send a fake "laggard" to kill the connection. But then, if we support migration, we have to also support frequent migration. And we don't want successive migrations to continuously inflate the memory footprint of a connection.

I think that a garbage collection frame like CONNECTION_ID_FINISHED will solve that problem. If a node says, "that connection ID that you announced, I will never use it anymore", then it is almost safe to remove the associated resource. With a little twist: the message should also say, "... and I have destroyed the associated reset secret". 

For obvious reason, this requires that nodes do not pick the same reset secret for every connection ID. I don't know whether we have that requirement. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420149034

---

From: Mike Bishop <[email-masked]>
Date: Tue, 11 Sep 2018 15:36:31 -0700

@huitema, the design team first met in late July while draft -13 was published in late June.  So, no.  And nothing in the -14 changelog deals with CIDs.

The design team met, agreed on a path, and asked @erickinnear to write the text.  Assembling the design team's feedback into a real PR has been slow, and I think we're all mildly frustrated by that.  My apologies for not pushing the process more quickly.  We should see the text in a public repo soon.  (I actually didn't know that his repo was ACL'd.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420449443

---

From: erickinnear <[email-masked]>
Date: Tue, 11 Sep 2018 17:42:16 -0700

I don't believe the repo has any access control applied to it, so I'm not sure what's happening there? Should be open to everyone. Also shouldn't matter at this point, but I don't think there's any ACLs applied. 

There've been quite a few roundtrips on getting everything ironed out, and it's taken longer than I think any of us would have liked. A lot of that can be attributed to my being slow about updating the actual text in the PR. 

@huitema, it seems like you may be referring to the Stream 0 design team, which wrapped up in June and has output included in -13. That is a completely different thing from the CID DT that started after Montreal in late July/early August and should hopefully be wrapping up about now. 

It seems like a lot of the claims and discussion are around that assumption, so perhaps we should re-evaluate and move onwards in light of (potentially) new information. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420472907

---

From: erickinnear <[email-masked]>
Date: Tue, 11 Sep 2018 17:43:03 -0700

To the topic at hand, sending CONNECTION_ID_FINISHED should do exactly as described above: 
- Say I will never use this CID again
and
- Invalidate the associated SR token, saying "I promise I won't use this either" 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420473033

---

From: Christian Huitema <[email-masked]>
Date: Tue, 11 Sep 2018 20:41:43 -0700

Oh, yes, I completely missed "_the CID DT that started after Montreal_". My bad for not being in Montreal, I guess. But then it would be good if people stopped dying so you don't have to go organize funerals...

The access issue happened when I read the PR in your repo. There were points I wanted to comment on. So I started a review, wrote a couple of comments, and then github sent me a bizarre error message about commits not present.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420501879

---

From: MikkelFJ <[email-masked]>
Date: Tue, 11 Sep 2018 23:23:27 -0700

There is also the following: Disagreement about CID Liveness https://github.com/quicwg/base-drafts/issues/1495

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1729#issuecomment-420526514