Subject: Re: [rtcweb] JSEP-03: O/A state machine and trickle ICE with forking

From: Christer Holmberg <[email-masked]>
Date: Fri, 1 Mar 2013 08:51:59 +0000

Hi Justin,

No new questions in this e-mail, but comments on the previous ones ☺

------------------

>>> Q_OA_1: As I have commented earlier, the state machine does not support setting of remote offer when a
>>> remote pranswer has been set. A JS SIP app may e.g. receive an SDP answer in a reliable 18x response,
>>> call setPranswer, and then receive an UPDATE on the same leg with a new offer.
>>>
>>> Section 3.5.2, paragraph 2, attempts to provide a workaround for this issue.  
>>Section 3.5.2 talks about parallel forking, and how it can be solved by creating a new PC, and then send an UPDATE to provide the new PC information to the remote entity.
>>
>>But, my issues is when the remote entity sends an UPDATE, with a new offer. It's not even related to forking. See my example below:
>>
>>
>>BROWSER                         JS APP                                    REMOTE ENITTY
>>
>>    ------ create offer ----------
>>                                             -------- SDP Offer 1----------------->
>>                                            <------- SDP Answer 1 ----------------
>>    ------ set pranswer ---------
>>                                           <------- SDP Offer 2 -------------------
>>    ------ ??? ----------------------
>
> Right. This is clearly a problem, but the reason it is a problem is because the two sides don't agree on the state 
> of the negotiation. In a forking case, it would be handled via the mechanism I mentioned above. In a non-forking 
> case, this is not solvable even with cloning, because the two sides are out of sync. The browser has to handle this 
> case by rejecting SDP Offer 2. 

I don't understand what you mean by "out of sync". Once the remote entity has sent an SDP Answer, it can send a new SDP Offer.

Keep in mind that the remote entity does not know that the JS APP has mapped SDP Answer 1 to a "pranswer". It could as well have mapped it to "answer" (in which case the current state machine would allow SDP Offer 2).

------------------

> >Q_OA_3: When a pranswer has been set, the state machine does not allow creating a new local offer.
>>
>> So, if the JS APP needs to send a new offer, e.g. because of BUNDLE, it would have to set answer first. But, that would again more or less prevent serial forking (at least on the same PeerConnection).
>
> Can you explain the BUNDLE case you have in mind? 

See example below, where the first offer contains different ports, and the second identical ports, according to BUNDLE.

BROWSER                         JS APP                                                             REMOTE ENITTY

    ------ create offer ----------
                                             -------- SDP Offer 1 (Different ports) ----------------->
                                            <------- SDP Answer 1 (Same/Diff ports)----------------
   <---- set pranswer ----------
    ----------------- ???? ---------
                                            -------- SDP Offer 2 (Same ports) ----------------------->

> The BUNDLE renegotiation to a single port can't happen at this point in the negotiation, because the caller may receive another answer that doesn't want to use BUNDLE.

Well, then the BROWSER has to "fallback" to different ports (per the original offer). Isn't that what "pranswer" is all about - having all resources available in case there is going to be a different answer?

Now, for BUNDLE, we can discuss when the same-port offer is sent, but there may be many OTHER reasons why SDP Offer 2 has to be sent, so the problem is not BUNDLE-specific.

------------------

>>Q_TI_1:
>>
>>Assume the JS APP receives an SDP answer from remote entity A, and provides it to the browser as a pranswer. Then the JS APP receives an SDP answer from remote entity B, and provides it to the browser as a pranswer.
>>
>>Then, the JS APP receives trickle ICE candidates from both entities A and B. What will happen to the trickle ICE candidates from remote entity A? The same question applies if the browser receives peer reflexive candidates from A.
>>
>>The draft says that trickle ICE candidates will be provided to the ICE Agent, that will then perform connectivity checks etc. But, does that mean that the ICE Agent will perform connectivity checks with both A and B.
>>
>>This could of course be handled by always creating separate PeerConnections (and hence separate ICE agents) for A and B, even in the case of serial forking.
>>
>>But, in any case I think the draft also needs to describe the ICE impacts of forking, because currently I don't think there is any text.
>>
>>Thanks for pointing this out. I discussed this at the IETF 83.5 interim, but forgot to put this into the draft. The answer is that when B's pranswer is applied,
>>and the received ICE credentials are different than the previous answer (as they would be in this case), the existing candidates are discarded.
>>But, A doesn't know that, so it may keep sending STUN requests, trickle ICE candidates etc.
>
> The app should send a termination message to A when it accepts the pranswer from B.

Such procedure/restriction would have to be clearly documented in JSEP.

> Even if it didn'tm the STUN requests would be ignored by the impl, and the ICE candidates would be ignored by the app, so I don't think there is a problem.

True - assuming the app is never going to "switch back" to remote entity A.

>>But, assume that the JS APP discards whatever comes from A. Then, if the session will eventually be established with A (i.e. the SDP 
>>answer from A is provided to the browser as "answer"), A needs to be informed to re-send all trickle ICE candidates etc again,  because 
>>the previously sent ones were discarded (since the JS APP provided the SDP answer from B to the browser)...
>
> The app can't keep the session alive simultaneously with both A and B. It needs to pick one or the other. The last time this was 
> discussed, this was felt to be a reasonable solution. Perhaps this needs to be codified in the JSEP draft.

It's not only about not being able to keep the session alive simultaneously with both A and B - it's also about not being able to go back to A. Maybe we can live with such restriction, but as I said earlier: it needs to be described in the draft.

------------------

>> I actually think that PeerConnection cloning would solve *ALL* of the issues above: there would be no need for pranswer, which means that 
>> the JSEP O/A procedures would be fully aligned with RFC 3264. And, ICE procedures could take place with multiple remote entities simultaneously.
>
> I considered that in the past, but PRANSWER is not just for forking; it's also useful in the non-forking case  to get media flowing as fast as possible in a single O/A sequence. 

In a non-forking case, how does it make things faster than simply using ANSWER?

> The overall opinion of the WG was that PRANSWER solves the problem above and provides a fairly good treatment of forking without 
> introducing significant complexity. Adding cloning would allow a more comprehensive treatment of forking, but at the cost of significant 
> additional complexity. 

I was a big supporter of PRANSWER myself, because of the way we would support forking with a single PeerConnection.

But, we DO need to be able to support the use-cases in Q_OA_1 and Q_OA_3.

> Also, I have yet to have any application builder ask me for cloning support.

I don't know what your definition of application builder is, but I did my first Windows8 "Hello world" app last weekend, so I guess I count as a builder - and I think we should re-consider cloning :)

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 01 Mar 2013 11:53:13 +0100

On 02/28/2013 11:37 PM, Justin Uberti wrote:
>
>
>
> On Wed, Feb 27, 2013 at 10:57 PM, Christer Holmberg 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     Hi Justin,
>
>     Thanks for your response!
>
>     Comments inline, with an additional state machine question (Q_OA_3).
>
>
>     ------------------
>
>
>     Section 3.5.2 talks about parallel forking, and how it can be
>     solved by creating a new PC, and then send an UPDATE to provide
>     the new PC information to the remote entity.
>
>     But, my issues is when the remote entity sends an UPDATE, with a
>     new offer. It's not even related to forking. See my example below:
>
>
> Right. This is clearly a problem, but the reason it is a problem is 
> because the two sides don't agree on the state of the negotiation. In 
> a forking case, it would be handled via the mechanism I mentioned 
> above. In a non-forking case, this is not solvable even with cloning, 
> because the two sides are out of sync. The browser has to handle this 

My understanding is that the mismatch is that the JS APP isn't giving a 
consistent story to the browser on whether the answer was final or not.
This sequence would work:

     ------ create offer ----------
                                              -------- SDP Offer 
1----------------->
                                             <------- SDP Answer 1 
----------------
     ------ set remote pranswer(1) ---------
                                             <------- SDP Offer 2 
-------------------
     ------ set remote answer(1) -----
     ------ set remote offer(2) -----

that is, when the JS app gets the information that there's a new offer 
incoming, it realizes that Answer 1 needs to be treated as a final 
answer, and finalizes it before passing in the new offer.

(I'll leave it to others to argue about whether or not there needs to be 
a stable state between the set remote answer(1) and set remote offer(2))

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 1 Mar 2013 11:24:22 +0000

Hi Harald,

>My understanding is that the mismatch is that the JS APP isn't giving a consistent story to the browser on whether the answer was final or not.
>that is, when the JS app gets the information that there's a new offer incoming, it realizes that Answer 1 needs to be treated as a final answer, and finalizes it before passing in the new offer.

The problem with such approach is that it will terminate forking, i.e. it will not be possible to pass an SDP Answer from ANOTHER remote entity to the browser.

BROWSER                         JS APP                                    REMOTE ENTITY A         REMOTE ENTITY B

    ------ create offer ----------
                                             -------- SDP Offer 1----------------->
                                            ------- SDP Offer 1 ---------------------------------------------------->
                                            <------- SDP Answer 1 (A) -----------
    ------ set remote pranswer(1) ---------
                                            <------- SDP Offer 2 (A) --------------
    ------ set remote answer(1) -----
    ------ set remote offer(2) --------
                                            <------ SDP Answer 1 (B) ---------------------------------------------
    ------ ???? ----------------------------

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 01 Mar 2013 13:45:27 +0100

>                                              <------ SDP Answer 1 (B) ---------------------------------------------

Right. I don't think that's avoidable; after a second offer/answer 
exchange, you can't go back and act as if you were still in the first 
offer/answer exchange.

Is there a sequence of SIP operations described in a spec that would 
allow this to happen?

I don't even think it's logically possible in all cases.

For example, if Offer 1 (and answer 1a and 1b) have the form

m=audio

and offer 2 is

m=audio
m=video

then it's not possible to handle Answer 1(b) without dropping the video 
m-line - and in SDP offer/answer, you aren't allowed to drop m-lines, 
ever. (Right?)

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 1 Mar 2013 13:33:24 +0000

Hi,

> Right. I don't think that's avoidable; after a second offer/answer exchange, you can't go back and act as if you were still in the first offer/answer exchange.
>
> Is there a sequence of SIP operations described in a spec that would allow this to happen?

It's not that common for the offerer (the BROWSER in this case) to RECEIVE an offer in the backward direction before the call has been established, but I HAVE seen such flow(s) somewhere.

However, it is more common for the offerer to SEND a new offer (as I showed in Q_OA_3) before the call has been established. It's used e.g. for negotiation down codecs, changing the media direction attribute, changing pre-conditions status - and, for BUNDLE.

>then it's not possible to handle Answer 1(b) without dropping the video m-line - and in SDP offer/answer, you aren't allowed to drop m-lines, ever. (Right?)

Please keep in mind that, in SIP terms, we are talking about two separate early SIP dialogs, and the SDP offer/answer state machines are independent from each other. So, it would be allowed.

PeerConnection cloning would introduce the same concept in browsers, as each clone could represent a separate SIP early dialog. 

Regards,

Christer

---

From: Justin Uberti <[email-masked]>
Date: Fri, 1 Mar 2013 08:32:41 -0800

On Fri, Mar 1, 2013 at 12:51 AM, Christer Holmberg <
[email-masked]> wrote:

> >>Section 3.5.2 talks about parallel forking, and how it can be solved by
> creating a new PC, and then send an UPDATE to provide the new PC
> information to the remote entity.
> >>
> >>But, my issues is when the remote entity sends an UPDATE, with a new
> offer. It's not even related to forking. See my example below:
> >>
> >>
> >>BROWSER                         JS APP
>  REMOTE ENITTY
> >>
> because the two sides don't agree on the state
> > of the negotiation. In a forking case, it would be handled via the
> mechanism I mentioned above. In a non-forking
> > case, this is not solvable even with cloning, because the two sides are
> out of sync. The browser has to handle this
> I don't understand what you mean by "out of sync". Once the remote entity
> has sent an SDP Answer, it can send a new SDP Offer.
>
> Keep in mind that the remote entity does not know that the JS APP has
> mapped SDP Answer 1 to a "pranswer". It could as well have mapped it to
> "answer" (in which case the current state machine would allow SDP Offer 2).
>
>
> ------------------
>
>
> >> So, if the JS APP needs to send a new offer, e.g. because of BUNDLE, it
> would have to set answer first. But, that would again more or less prevent
> serial forking (at least on the same PeerConnection).
> >
> See example below, where the first offer contains different ports, and the
> second identical ports, according to BUNDLE.
>
> BROWSER                         JS APP
>                         REMOTE ENITTY
>
> (Same/Diff ports)----------------
> > The BUNDLE renegotiation to a single port can't happen at this point in
> the negotiation, because the caller may receive another answer that doesn't
> want to use BUNDLE.
>
> Well, then the BROWSER has to "fallback" to different ports (per the
> original offer). Isn't that what "pranswer" is all about - having all
> resources available in case there is going to be a different answer?
>
> Now, for BUNDLE, we can discuss when the same-port offer is sent, but
> there may be many OTHER reasons why SDP Offer 2 has to be sent, so the
> problem is not BUNDLE-specific.
>
>
> ------------------
>
>
> >>Q_TI_1:
> >>
> >>Assume the JS APP receives an SDP answer from remote entity A, and
> provides it to the browser as a pranswer. Then the JS APP receives an SDP
> answer from remote entity B, and provides it to the browser as a pranswer.
> >>
> >>Then, the JS APP receives trickle ICE candidates from both entities A
> and B. What will happen to the trickle ICE candidates from remote entity A?
> The same question applies if the browser receives peer reflexive candidates
> from A.
> >>
> >>The draft says that trickle ICE candidates will be provided to the ICE
> Agent, that will then perform connectivity checks etc. But, does that mean
> that the ICE Agent will perform connectivity checks with both A and B.
> >>
> >>This could of course be handled by always creating separate
> PeerConnections (and hence separate ICE agents) for A and B, even in the
> case of serial forking.
> >>
> >>But, in any case I think the draft also needs to describe the ICE
> impacts of forking, because currently I don't think there is any text.
> >>
> >>Thanks for pointing this out. I discussed this at the IETF 83.5 interim,
> but forgot to put this into the draft. The answer is that when B's pranswer
> is applied,
> >>and the received ICE credentials are different than the previous answer
> (as they would be in this case), the existing candidates are discarded.
> > Even if it didn'tm the STUN requests would be ignored by the impl, and
> the ICE candidates would be ignored by the app, so I don't think there is a
> problem.
>
> >>But, assume that the JS APP discards whatever comes from A. Then, if the
> session will eventually be established with A (i.e. the SDP
> >>answer from A is provided to the browser as "answer"), A needs to be
> informed to re-send all trickle ICE candidates etc again,  because
> > The app can't keep the session alive simultaneously with both A and B.
> It needs to pick one or the other. The last time this was
> It's not only about not being able to keep the session alive
> simultaneously with both A and B - it's also about not being able to go
> back to A. Maybe we can live with such restriction, but as I said earlier:
> it needs to be described in the draft.
>
>
> ------------------
>
> >> I actually think that PeerConnection cloning would solve *ALL* of the
> issues above: there would be no need for pranswer, which means that
> >> the JSEP O/A procedures would be fully aligned with RFC 3264. And, ICE
> procedures could take place with multiple remote entities simultaneously.
> >
> > I considered that in the past, but PRANSWER is not just for forking;
> it's also useful in the non-forking case  to get media flowing as fast as
> possible in a single O/A sequence.
>

Assuming for now that the ANSWER corresponds to the remote side physically
answering the call (i.e. media transmission starts), sending the PRANSWER
first allows the ICE and DTLS handshakes to complete while we are waiting
for the final ANSWER, preventing media clipping from the callee.

>
> > The overall opinion of the WG was that PRANSWER solves the problem above
> and provides a fairly good treatment of forking without
> > introducing significant complexity. Adding cloning would allow a more
> comprehensive treatment of forking, but at the cost of significant
> > additional complexity.
>
> I was a big supporter of PRANSWER myself, because of the way we would
> support forking with a single PeerConnection.
>
> I don't know what your definition of application builder is, but I did my
> first Windows8 "Hello world" app last weekend, so I guess I count as a
> builder - and I think we should re-consider cloning :)
>
> Regards,
>
> Christer
>
>