Subject: [rtcweb] Minimal SDP negotiation mechanism

From: Harald Alvestrand <[email-masked]>
Date: Mon, 19 Sep 2011 18:59:44 +0200

I am looking at the WEBRTC API spec, which specifies a rudimentary 
negotiation framework: SDP objects prefixed by the string "SDP".

It seems clear to me that this needs at least information about whether 
something is an offer or an answer, and some way to complete the 
transaction when an offer is sent and something prevents it from completing.
Until we know we need more, what about the following, to be specified in 
the WEBRTC API?

SDP objects are sent through the API, prefixed with either of

SDP OFFER
SDP ANSWER

Alternatively, one can pass

SDP ERROR

to reply to an SDP OFFER when something goes wrong.

If one gets an OFFER and sends out an ANSWER, state changes.
If OFFER gets an ANSWER back, state changes.
In all other cases, state is as before.

We need to handle glare - when one sends an OFFER and gets back an 
OFFER, reply with SDP ERROR, enter a "glare" state, wait a bit, and send 
out an offer again.

Do we really have to have anything else?

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 19 Sep 2011 13:23:05 -0400

Harald,

I really appreciate your addressing the need to explicitly distinguish 
offers from answers - it is important. SIP did a really bad job of that. 
In SIP one just sends the SDP and the determination of whether it is an 
offer or answer is contextual. That was a bad decision.

> Do we really have to have anything else?

Since, afaik, this is async messaging, I think you should have something 
used to identify an offer and then to refer back to it when sending the 
answer, to ensure that a received answer was indeed an answer to the 
offer you thought it was.

And that is part of the rtcweb-o/a protocol you are defining. It will 
also need some rules to prevent sending two offers without getting an 
answer first, or at least define what you do if that happens. (If one 
side sends an offer and fails to get an answer back, what should it do?)

	Thanks,
	Paul

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 20 Sep 2011 01:39:43 +0000

In SIP it's possible to get two or more different answers back for one offer, due to forking.  I'm not sure you want/need to handle that case, but it can and does in-practice happen in SIP.

Also, in SIP it's technically possible to generate SDP which is neither an offer nor an answer, but is rather "capabilities".  This is described in RFC 3264 section 9, and used in responses to SIP OPTIONS requests by some devices.  As Cullen noted in some earlier email in the mailing list, this could probably be hacked around with a fake SDP answer or error, but ya may want to keep things clean and just handle this case explicitly instead.

-hadriel

On Sep 19, 2011, at 12:59 PM, Harald Alvestrand wrote:

> I am looking at the WEBRTC API spec, which specifies a rudimentary negotiation framework: SDP objects prefixed by the string "SDP".
> 
> It seems clear to me that this needs at least information about whether something is an offer or an answer, and some way to complete the transaction when an offer is sent and something prevents it from completing.
> Until we know we need more, what about the following, to be specified in the WEBRTC API?
> 
> We need to handle glare - when one sends an OFFER and gets back an OFFER, reply with SDP ERROR, enter a "glare" state, wait a bit, and send out an offer again.
>

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 20 Sep 2011 14:08:46 +0200

Hi, 

>In SIP it's possible to get two or more different answers 
>back for one offer, due to forking.  I'm not sure you 
>want/need to handle that case, but it can and does 
>in-practice happen in SIP.

Whether we want the browser to support forking is one thing, and I guess it much depends on whether we want to be able to do things on the media plane during the early phase of a session establishment.

But, at least the API needs to allow a JS SIP app to "replace" a previously received SDP with a new one (if the SIP app, in the forking case, for example chooses to always use the latest received SDP answer).

Regards,

Christer

> 
> Also, in SIP it's technically possible to generate SDP which 
> is neither an offer nor an answer, but is rather 
> "capabilities".  This is described in RFC 3264 section 9, and 
> used in responses to SIP OPTIONS requests by some devices.  
> As Cullen noted in some earlier email in the mailing list, 
> this could probably be hacked around with a fake SDP answer 
> or error, but ya may want to keep things clean and just 
> handle this case explicitly instead.
> 
> -hadriel
> 
> 
> > I am looking at the WEBRTC API spec, which specifies a 
> rudimentary negotiation framework: SDP objects prefixed by 
> the string "SDP".
> > 
> > It seems clear to me that this needs at least information 
> about whether something is an offer or an answer, and some 
> way to complete the transaction when an offer is sent and 
> something prevents it from completing.
> > We need to handle glare - when one sends an OFFER and gets 
> back an OFFER, reply with SDP ERROR, enter a "glare" state, 
> wait a bit, and send out an offer again.
> >

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 20 Sep 2011 14:16:08 +0200

> Whether we want the browser to support forking is one thing, and I guess it much depends on whether we want to be able to do things on the media plane during the early phase of a session establishm…

I hope. If not, let's forget about integrating WebRTC in any telephony
environment.

The point here is that *media sessions* are "independant" of
signaling. If we make WebRTC just to deal with media plane, the
signaling (coded in JavaScript by a custom or native library) could
handle early media sessions with no problem.

But if we impose a signaling protocol (built-in within the browser
WebRTC stack) then we'll be limited to adopted design constrains,
which is not good.

> But, at least the API needs to allow a JS SIP app to "replace" a previously received SDP with a new one (if the SIP app, in the forking case, for example chooses to always use the latest received S…

There is no a magic solution for that case: which media to render when
parallel forking occurs and more than one branch replies a 180/183
with SDP?
Note that "199" response tries to "help" a bit:

  "SIP Response Code for Indication of Terminated Dialog"
  http://tools.ietf.org/html/rfc6228

If we let it at the JS application level, the developer could handle
those cases and decide which media session to render.

BTW: would be possible to render various at the same time by mixing
the incoming audio?.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 20 Sep 2011 14:17:28 +0200

> Whether we want the browser to support forking is one thing, and I guess it much depends on whether we want to be able to do things on the media plane during the early phase of a session establishm…
I think we need to do all the transforms we need to do. It would be 
great if there was a consistent set of things we are required to be able 
to do.
If a JS app wants to support forking, I think it's reasonable to leave 
it to the JS which answer it wants to pass on to the PeerConnection 
object. When we get multiple ANSWERs like this, is there any information 
from the first ANSWER that needs to be preserved after the second ANSWER 
has arrived? (I really hope that the answer is a really clear NO...)
> But, at least the API needs to allow a JS SIP app to "replace" a previously received SDP with a new one (if the SIP app, in the forking case, for example chooses to always use the latest received S…
We might get SIP compatibility if we decide that SDP ANSWER can be 
received at any time, changes the state of the connection, and never 
generates a response. SDP ANSWER is now an one-way offer/answer.

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 20 Sep 2011 14:36:24 +0200

Hi, 

>> Whether we want the browser to support forking is one 
>> thing, and I guess it much depends on whether we want to be 
>> able to do things on the media plane during the early phase 
>> of a session establishment.
> 
> I hope. If not, let's forget about integrating WebRTC in any 
> telephony environment.
> 
> signaling. If we make WebRTC just to deal with media plane, 
> the signaling (coded in JavaScript by a custom or native 
> library) could handle early media sessions with no problem.
> 
> But if we impose a signaling protocol (built-in within the 
> browser WebRTC stack) then we'll be limited to adopted design 
> constrains, which is not good.

That's not what I am suggesting. But, forking also have impacts on the media plane.

>>But, at least the API needs to allow a JS SIP app to 
>>"replace" a previously received SDP with a new one (if the 
>>SIP app, in the forking case, for example chooses to always 
>>use the latest received SDP answer).
> 
>There is no a magic solution for that case: which media to 
>render when parallel forking occurs and more than one branch 
>replies a 180/183 with SDP?

The JS SIP app can decide that, but my point was that the JS SIP app needs to be able to "replace" the SDP answer with anohter one (received on another early dialog) - *IF* it chooses to do so.

>   http://tools.ietf.org/html/rfc6228

That's such a great piece of RFC :)

And, it does help the JS SIP app to make a decission, but the API still needs to allow to "replace" the SDP answer.

>If we let it at the JS application level, the developer could 
>handle those cases and decide which media session to render.

Yes, the JS app can make the decision - but, again, the API still needs to allow the JS app to enforce that decision.
 
>BTW: would be possible to render various at the same time by 
>mixing the incoming audio?.

Well, that is what my first question was more or less about.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 20 Sep 2011 14:42:55 +0200

Hi, 

>> In SIP it's possible to get two or more different answers back for 
>> one offer, due to forking.  I'm not sure you want/need to 
>> handle that 
>> case, but it can and does in-practice happen in SIP.
> I think we need to do all the transforms we need to do. It 
> would be great if there was a consistent set of things we are 
> required to be able to do.
> If a JS app wants to support forking, I think it's reasonable 
> to leave it to the JS which answer it wants to pass on to the 
> PeerConnection object. When we get multiple ANSWERs like 
> this, is there any information from the first ANSWER that 
> needs to be preserved after the second ANSWER has arrived? (I 
> really hope that the answer is a really clear NO...)

The answers as such are independent from each other, so from that perspective the answer is NO :)

The question is whether we want to allow actions to to take place on the media plane before we know which early dialog will be part of the established call.

For example, you won't be able to perform resource reservation procedures for multiple early dialog if the browser only deals with one early dialog at any given time.

> We might get SIP compatibility if we decide that SDP ANSWER 
> can be received at any time, changes the state of the 
> connection, and never generates a response. SDP ANSWER is now 
> an one-way offer/answer.

Doesn't the browser still need to know with which offer the answer is associated?

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 20 Sep 2011 15:00:12 +0200

> For example, you won't be able to perform resource reservation procedures for multiple early dialog if the browser only deals with one early dialog at any given time.
I get a steady increase in my uneasiness here. A PeerConnection should 
(in my opinion) be about the connection between the browser and a single 
other endpoint, and the media transmitted between those two. Once we 
start requiring that the PeerConnection know the difference between 
"early" media and "late" media, it seems to me we're slipping down a 
slippery slope.

 From the standpoint of PeerConnections, it may make sense to treat a 
forked call as two PeerConnections rather than one - but in that case, 
we may have the need to be able to create a PeerConnection from another 
PeerConnection (duplicating state, and maybe sharing resources, which is 
ugly).

Forking is hard, it seems.
> Doesn't the browser still need to know with which offer the answer is associated?
The browser (PeerConnection implementation) or the Javascript?

That's a question I don't have an answer to.

Is there any information in there where the browser would act 
differently if it knew which offer the answer was associatied to?

I went over the offer/answer with Per Kjellander today; if we follow RFC 
3264's recommendation of only having one outstanding offer at a time, we 
did not find any need to identify offers ("there can be only one") - if 
we can get into the situation where we get an answer after we've sent 
out a subsequent offer and gotten that answered, we may get a need for it.

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 15:07:48 +0200

20 sep 2011 kl. 15:00 skrev Harald Alvestrand:

> Once we start requiring that the PeerConnection know the difference between "early" media and "late" media, it seems to me we're slipping down a slippery slope.

The difference between early and late media is purely a billing decision in PSTN. I don't think we should separate these on the rtcweb side. It's a PSTN gateway issue, not something to be bothered with in rtcweb.

"The campaign to simplify call states"
/O

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 20 Sep 2011 15:15:56 +0200

Hi, 

>> Once we start requiring that the PeerConnection know the 
>> difference between "early" media and "late" media, it seems 
>> to me we're slipping down a slippery slope.
> 
>The difference between early and late media is purely a 
>billing decision in PSTN. I don't think we should separate 
>these on the rtcweb side. It's a PSTN gateway issue, not 
>something to be bothered with in rtcweb.

It's not about knowing the difference between "early" and "late" media - it's about whether the API and browser need to support multiple SIMULTANOUS SDP answers - or whether we assume that the JS SIP app will always, at any given time, only provide ONE SDP answer to the API and browser.

Regards,

Christer

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 20 Sep 2011 15:23:43 +0200

> The JS SIP app can decide that, but my point was that the JS SIP app needs to be able to "replace" the SDP answer with anohter one (received on another early dialog) - *IF* it chooses to do so.

Sure. That must be designed by the signaling layer. What I hope is
that such signaling layer is up to the JS code (optionally in form of
custom code or a given WebRTC JS API).

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 20 Sep 2011 15:24:28 +0200

> OFFER, reply with SDP ERROR, enter a "glare" state, wait a bit, and send 

I think that is a bad design for glare handling. I agree it needs to be
handled. I think the ICE version of glare handling could work pretty
well here. Any offer needs to include a random 32 bit number. If the
end-point receive an offer in relation to a peer-connection while it has
an outstanding offer then it compares these codes. Who ever has the
greatest numerical value wins and that offer is handled the other is
responded with an error stating GLARE. Thus there is no extra delay and
more clear resolution of the glare case.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 15:40:01 +0200

> It's not about knowing the difference between "early" and "late" media - it's about whether the API and browser need to support multiple SIMULTANOUS SDP answers - or whether we assume that the JS S…

I just wanted to get rid of the early/late media discussion. As you state, the forking issue with getting multiple responses is a separate issue.

Do we have any use cases using forking? Is forking a desired feature or something that SIP brought in?

To give an example to those of you with no SIP history:

In SIP you can send one OFFER and get multiple ANSWERs in 200 OK from different devices. This means that you actually have multiple calls and need to hang up all of those that you do not want, or just mix them somehow and live with it. 

With early media, it gets even more confusing, because you will get an SDP answer before the call is in up state (in 180/183 response) then another SDP answer from another device at 200 OK. In this case you have an early dialog with early media with the device sending the first answer, a dialog that terminates at 200 OK. 

Supporting this behaviour adds complexity. And not all SIP devices support these situations properly, as we've proven at SIPit testing. 

Which leads to an unshameful plug: SIPit in Monaco is now open for registration - for all SIP developers to attend :-)

/O

---

From: Roman Shpount <[email-masked]>
Date: Tue, 20 Sep 2011 10:53:10 -0400

There are actually three related issues here:

1. As mentioned earlier, a single offer in SIP can create multiple dialogs
with independent answers. Each dialog is independent from each other and it
is up to end user device to choose how to render it (mix audio, play the
audio from the latest, display multiple videos side by side). These dialogs
can be early (created by a provisional response) or final (created by a
success response), but this distinction typically does not affect the media
plane. There are a number of common use cases for forking with multiple
answers such as service announcements (play some announcement from a media
server using an early dialog then start playing audio from the call using
another dialog), color ring back (play custom music while dialing the user),
find me/follow me (where call can be answered by the desk and cell phone
creating two final dialogs). One of the biggest design issues with such
multiple answer fork scenarios is that there is no way to map received media
to the actual answer, since there is nothing in the answer SDP which
identifies the response RTP stream.

2. The second scenario is non-standard, but very widely used -- in the same
dialog, different responses are send in the provisional and final SIP
responses. This is usually a result of forking being masked by a B2BUA or
SBC. Even though this is non-standard this is usually trivial to implement,
since all that needs to be done is to reinitialize the media stream based on
the new answer.

3. Even when multiple answers are not used, multiple different media streams
can be sent based on the offer. First of all, it is common to receive media
before the signaling response is received. Second, standard requires that
offerer should be ready to receive media once the offer is sent and there is
nothing that prevents multiple media streams from being sent to it.
Furthermore, multiple answers only define what media and where should be
sent to the answerer, but in no way specifies how many remote parties, from
which addresses, and what type of media will send the media to the offerer.
New media streams with new SSRC from new addresses with new media types
present in the offer can be added at any time without the offer/answer
exchange. Typically this is supported in such a way that only the newest
media stream, after probation interval, is played and older media streams
are ignored, but other more robust solutions are possible.

If we plan to interop with existing VoIP solutions, we will need to support
1 and 2, as well as define the expected behavior for 3.
_____________
Roman Shpount

On Tue, Sep 20, 2011 at 9:40 AM, Olle E. Johansson <[email-masked]> wrote:

> > It's not about knowing the difference between "early" and "late" media -
> it's about whether the API and browser need to support multiple SIMULTANOUS
> SDP answers - or whether we assume that the JS SIP app will always, at any
> given time, only provide ONE SDP answer to the API and browser.
>
> I just wanted to get rid of the early/late media discussion. As you state,
> the forking issue with getting multiple responses is a separate issue.
>
> In SIP you can send one OFFER and get multiple ANSWERs in 200 OK from
> different devices. This means that you actually have multiple calls and need
> to hang up all of those that you do not want, or just mix them somehow and
> live with it.
>
> With early media, it gets even more confusing, because you will get an SDP
> answer before the call is in up state (in 180/183 response) then another SDP
> answer from another device at 200 OK. In this case you have an early dialog
> with early media with the device sending the first answer, a dialog that
> terminates at 200 OK.
>
> Supporting this behaviour adds complexity. And not all SIP devices support
> these situations properly, as we've proven at SIPit testing.
>
> Which leads to an unshameful plug: SIPit in Monaco is now open for
> registration - for all SIP developers to attend :-)
>
> /O
>
>

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 18:09:39 +0200

20 sep 2011 kl. 16:53 skrev Roman Shpount:

> 1. As mentioned earlier, a single offer in SIP can create multiple dialogs with independent answers. Each dialog is independent from each other and it is up to end user device to choose how to render it (mix audio, play the audio from the latest, display multiple videos side by side). These dialogs can be early (created by a provisional response) or final (created by a success response), but this distinction typically does not affect the media plane. There are a number of common use cases for forking with multiple answers such as service announcements (play some announcement from a media server using an early dialog then start playing audio from the call using another dialog), color ring back (play custom music while dialing the user), find me/follow me (where call can be answered by the desk and cell phone creating two final dialogs). One of the biggest design issues with such multiple answer fork scenarios is that there is no way to map received media to the actual answer, since there is nothing in the answer SDP which identifies the response RTP stream.
> 
> 2. The second scenario is non-standard, but very widely used -- in the same dialog, different responses are send in the provisional and final SIP responses. This is usually a result of forking being masked by a B2BUA or SBC. Even though this is non-standard this is usually trivial to implement, since all that needs to be done is to reinitialize the media stream based on the new answer.
> 
> 3. Even when multiple answers are not used, multiple different media streams can be sent based on the offer. First of all, it is common to receive media before the signaling response is received. Second, standard requires that offerer should be ready to receive media once the offer is sent and there is nothing that prevents multiple media streams from being sent to it. Furthermore, multiple answers only define what media and where should be sent to the answerer, but in no way specifies how many remote parties, from which addresses, and what type of media will send the media to the offerer. New media streams with new SSRC from new addresses with new media types present in the offer can be added at any time without the offer/answer exchange. Typically this is supported in such a way that only the newest media stream, after probation interval, is played and older media streams are ignored, but other more robust solutions are possible.
> 
> If we plan to interop with existing VoIP solutions, we will need to support 1 and 2, as well as define the expected behavior for 3.

Agree. I think that would be bad. What do you think?

/O

---
* Olle E Johansson - [email-masked]
* Cell phone +46 70 593 68 51, Office +46 8 96 40 20, Sweden

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 18:09:39 +0200

/O

---

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 20 Sep 2011 13:58:28 -0600

> more clear resolution of the glare case.

I think it has to be possible to gateway what happens here to what happens in SIP with glare - I suspect that more or less puts us on using the SIP glare mechanism.

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 20 Sep 2011 14:07:14 -0600

> "The campaign to simplify call states"

Simplification is good - just make sure that it is possible to gateway to a SIP call to 1-800-go-fedex. A simplification that broke that would, in my personal opinion, would not be acceptable. (and I think that early media is a bit more than just a billing artifact but that's a longer story). You agree?

That said, I think that doing both forking and early media is hard. Lets assume we are using a signaling gateway that is not a media gateway to translate between a SIP call on one side and whatever is happening over on the browser side. The basic issue is the browser initiating the communications needs to be able to start receiving multiple RTP streams before it even has signaling information to tell it how many it might receive. To simplify this problem, Cary and my draft proposes not allowing forking on the SIP side of the signaling gateway but still allowing early media. If you wanted to do do forking in this case, one would need a SBC that processed media and turned the forked medial legs into one media leg. 

What I'm trying to say is I agree that early media + forking is hard, and the way I would like to simplify that is not allow forking. Note that none of this has much to do with if the PeerConnection object knows if media is early or late - what it needs to deal with is receiving RTP regardless os if the SIP signaling GW had sent it any signaling that indicated what had been in the SIP answer of the SIP side of a signaling GW. 

Cullen

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 20 Sep 2011 14:08:34 -0600

I think this sounds about right to me - I have been thinking about a draft along the lines of this and showing how it maps to SIP. Keep in mind there are also cases where one sides sends an OFFER, gets an ANSWER, but that ANSWER has an error. I think another key component is that a signaling GW needs to pass along an opaque blob that gets passed back to it so you can signaling GW that are only transaction state full and don't have to retain state for the duration of the communications. You also need some identifier information so that a JS application participating in multiple sessions can know what session a given piece of SDP goes with.

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 22:17:48 +0200

> Simplification is good - just make sure that it is possible to gateway to a SIP call to 1-800-go-fedex. A simplification that broke that would, in my personal opinion, would not be acceptable. (and…
Absolutely, but if the call is in UP state on the rtcweb side as soon as the PSTN/SIP gateway receives early media we will have no issue with sending DTMF in early media, which is what you are trying to trick me with :-)

Now, if we force early media which is supposed to be one-way, not many implementations would support fedex. I don't see this as a problem if we just forget the early-media and late-media difference and just talk about media and if it's up or not up. That would handle ringback as well. 

In my head, rtcweb is only the handset of a phone. The phone knows about early media, but the handset just has a mike and a speaker and it's simply on or off.

> That said, I think that doing both forking and early media is hard. Lets assume we are using a signaling gateway that is not a media gateway to translate between a SIP call on one side and whatever…
SBS or b2bua - in this case rtcweb to SIP ua. Yep, that's my way of thinking. 
> What I'm trying to say is I agree that early media + forking is hard, and the way I would like to simplify that is not allow forking. Note that none of this has much to do with if the PeerConnectio…

Agree. 

/O

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 20 Sep 2011 14:39:53 -0600

> Absolutely, but if the call is in UP state on the rtcweb side as soon as the PSTN/SIP gateway receives early media we will have no issue with sending DTMF in early media, which is what you are tryi…

really I was not going to mention DTMF :-) 

I was thinking a a call where the browser initiates the call, sends something to the web2sip signaling gateway that sends a SIP invite to the PSTN GW. Now the PSTN gateway sends a 180 and starts ICE. Then the PSTN GW starts sending RTP with the fedex prompts in them and then it eventually sends a 200.  The key thing is the browser needed to be able to receive RTP before the signaling gateway received the 200. As long as we are on the same page there, it's good with me. The PSTN GW sends the RTP directly to the the browser but sends the SIP to the signaling gateway. 

> Now, if we force early media which is supposed to be one-way, not many implementations would support fedex. I don't see this as a problem if we just forget the early-media and late-media difference…

I could care less about if the media is 1 way or two way. I just care about hearing the prompts when I call 1-800-fedex. (I care about early media, not one way media :-) 

> In my head, rtcweb is only the handset of a phone. The phone knows about early media, but the handset just has a mike and a speaker and it's simply on or off.

sure - not problem with that

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 20 Sep 2011 23:19:30 +0200

> really I was not going to mention DTMF :-) 

Background to the fedex/DTMF hints:

Fedex for some reason got a deal not only to send early media from a commercial company, but also have an IVR server and request DTMF while still not answering the call. This forced a code change in quite a lot of SIP servers... ;-)

/O

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 20 Sep 2011 21:45:29 +0000

Yup, that pretty much sums it up. :)
Some minor comments inline...

> 1. As mentioned earlier, a single offer in SIP can create multiple dialogs with independent answers. Each dialog is independent from each other and it is up to end user device to choose how to rend…

So far the discussion in rtcweb has been that ICE will be required.  As such, you can actually correlate received media with the SDP, because the username in the STUN connectivity checks from the remote peers will be uniquely indicated in the SDP from them, so media received on the same 5-tuple is from that peer. (once you get the SDP answer(s) of course, which per ICE should be rather quickly)

If we don't require ICE, we're still gonna require symmetric RTP, so the SDP answer's c/m-line info could be used to correlate the media in many cases.

---

From: Dzonatas Sol <[email-masked]>
Date: Tue, 20 Sep 2011 14:53:29 -0700

On 09/20/2011 01:39 PM, Cullen Jennings wrote:
> I was thinking a a call where the browser initiates the call, sends something to the web2sip signaling gateway that sends a SIP invite to the PSTN GW. Now the PSTN gateway sends a 180 and starts ICE.

The PSTN GW sounds like a room pressure aware device that may encounter 
said vacuum space on return. That resolves the potential error on GLARE 
exceptions.

Instead of "stable", the mobile units wants ubiquity with real-time 
GLARE-to-FIXATED states, smoothly.

Or; in return, the display unit and mobile unit become actors in 
use-cases (and used-cases), so we clarified (the two) for rtp-usage.

JIT repository is FIXATED when there is no HTTP sent/delivered due to 
non-expired certificates. Browsers that have no encryption provided by 
the certificate is not obligated to store such bytecode in non-obvious 
encryption; thus, raises the bar a little on reason to federate to 
FIXATED-known headers; no X-, no 0day.

JIT repository that stay FIXATED for more than one day are then 
valuable. The sites' bytecodes simply tune-out the origin servers that 
deal only with unified user-agents, as the FIXATED state cares more 
about the agent than the user (note-well vs. E911 recommendations).

SIP presents RTCWeb with many ways to undo, untwist, or simply focus on 
the user more, yet professionally there is no FIXATED state, or we have 
no professional reason why such FIXATED state of JIT does not exist and 
displays continue to consume power while on-the-... federated signal.

Elevators that just want the SSRC for MMUSIC, there are professionally 
known geolocations for that range and length of signal. We think of 
treats, not tricks.

>   Then the PSTN GW starts sending RTP with the fedex prompts in them and then it eventually sends a 200.  The key thing is the browser needed to be able to receive RTP before the signaling gateway received the 200. As long as we are on the same page there, it's good with me. The PSTN GW sends the RTP directly to the the browser but sends the SIP to the signaling gateway.
>
>
>    

-- 

---
<i>The wheel.</i metro-link=t dzonatasolyndra>

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 20 Sep 2011 22:17:27 +0000

> Fedex for some reason got a deal not only to send early media from a commercial company, but also have an IVR server and request DTMF while still not answering the call. This forced a code change i…

And it's a lot bigger list than just FedEx.  American Airlines, United Airlines, the IRS, 1-800-CALL-ATT, etc.  (American Airlines is usually the canonical example people refer to)

-hadriel

---

From: Hadriel Kaplan <[email-masked]>
Date: Tue, 20 Sep 2011 22:36:21 +0000

On Sep 20, 2011, at 4:07 PM, Cullen Jennings wrote:

> That said, I think that doing both forking and early media is hard. Lets assume we are using a signaling gateway that is not a media gateway to translate between a SIP call on one side and whatever is happening over on the browser side. The basic issue is the browser initiating the communications needs to be able to start receiving multiple RTP streams before it even has signaling information to tell it how many it might receive.

Not really - there will be signaling, because there has to be SDP answers even just to get ICE to work before the media starts flowing in many NAT cases.  And even in practice in SIP there're usually SDP answers in 18x to open "gates", and to get upstream DTMF.  So if the concern is just that there's no signaling to tell the browser there are multiple RTP streams coming, I think that can be allayed.  

The really hard part is knowing which stream to use/render/send-to, imho.  And putting that decision in the gateway isn't good - the best decider of that is probably the JS in the browser.

> To simplify this problem, Cary and my draft proposes not allowing forking on the SIP side of the signaling gateway but still allowing early media. If you wanted to do do forking in this case, one would need a SBC that processed media and turned the forked medial legs into one media leg. 

Obviously you can request that a request not be forked, using caller-prefs, but you can't "not allow" forking on the SIP side.  That would make it not SIP.  I know forking is hard, but that's life.  It's not appropriate for this WG to make fundamental changes/limitations to the SIP protocol, just because some of it's "hard" for a browser. 

-hadriel

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Wed, 21 Sep 2011 07:41:08 +0530

Forking in SIP does not apply in the literal sense to lot of SIP
applications (ISDN-SIP gateway, End-point which can't perform mixing).
In case of ISDN-SIP gateway, SIP callleg has to handle all forked
dialogs till 200 OK is received from anyone of the UAS and reject all
other dialogs with CANCEL, the media plane update is depend upon the
implementation whether to override the last SDP in media plane in case
mixing is not possible. I'm saying in this mail thread to highlight
forking handling in browser (as a SIP UA application) is not an
exception and it is the decision which has to be taken by any SIP
application development (and not SIP framework) for that matter. 

SIP application forking behavior depends upon RTP model (endpoint or
mixer). In case browser acts only as endpoint, I agree with Cullen that
forking shall be handled by browser without application aware and no
need of API or callback. 

The counter argument may be that my innovative mixing application in
browser is stopped by this API model.  In the generic SIP framework, the
callbacks are provided to handle this situation, default callback
function (browser as endpoint) are provided to reduce the application
awareness. From the API perspective, offer & answer state machine is not
required to be handled in application but we required to know whether
the application prefers which media model whether as end-point or mixer
and let end-point model be default. IMO, it is browser API design which
belongs to W3C. Please correct me here in case this API design is
somehow related to IETF.

Thanks
Partha

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On
Behalf
>Of Hadriel Kaplan
>Sent: Wednesday, September 21, 2011 4:06 AM
>To: Cullen Jennings
>Cc: [email-masked]
>Subject: Re: [rtcweb] Forking & Early Media - Was Re: Minimal SDP
>negotiation mechanism
>
>
>> That said, I think that doing both forking and early media is hard.
>Lets assume we are using a signaling gateway that is not a media
gateway
>to translate between a SIP call on one side and whatever is happening
>over on the browser side. The basic issue is the browser initiating the
>communications needs to be able to start receiving multiple RTP streams
>before it even has signaling information to tell it how many it might
>receive.
>
>Not really - there will be signaling, because there has to be SDP
>answers even just to get ICE to work before the media starts flowing in
>many NAT cases.  And even in practice in SIP there're usually SDP
>answers in 18x to open "gates", and to get upstream DTMF.  So if the
>concern is just that there's no signaling to tell the browser there are
>multiple RTP streams coming, I think that can be allayed.
>
>The really hard part is knowing which stream to use/render/send-to,
>imho.  And putting that decision in the gateway isn't good - the best
>decider of that is probably the JS in the browser.
>
>
>> To simplify this problem, Cary and my draft proposes not allowing
>forking on the SIP side of the signaling gateway but still allowing
>early media. If you wanted to do do forking in this case, one would
need
>a SBC that processed media and turned the forked medial legs into one
>media leg.
>
>Obviously you can request that a request not be forked, using caller-
>prefs, but you can't "not allow" forking on the SIP side.  That would
>make it not SIP.  I know forking is hard, but that's life.  It's not
>appropriate for this WG to make fundamental changes/limitations to the
>SIP protocol, just because some of it's "hard" for a browser.
>
>-hadriel
>

---

From: Roman Shpount <[email-masked]>
Date: Tue, 20 Sep 2011 22:20:40 -0400

Small correction to what you are saying -- new dialogs cannot be stopped
using CANCEL (this will stop all dialogs). They can only be stopped using
BYE.

I would consider it very unfortunate if we decide not to support forking in
RTC. It needs to be handled, and not in signaling, but by defining what to
do with multiple incoming media RTP streams and multiple answers. The
simplest model would be ignore everything except the RTP media defined in
the last answer and have an ability to update the answer to the last offer.
Alternative would be to create a new PeerConnection based on a new answer
(this is API change so should probably go to W3C).
_____________
Roman Shpount

---

From: "Olle E. Johansson" <[email-masked]>
Date: Wed, 21 Sep 2011 08:18:39 +0200

> The really hard part is knowing which stream to use/render/send-to, imho.  And putting that decision in the gateway isn't good - the best decider of that is probably the JS in the browser.
Food for thought.
> Obviously you can request that a request not be forked, using caller-prefs, but you can't "not allow" forking on the SIP side.  That would make it not SIP.  I know forking is hard, but that's life.…

But it will be appropriate to avoid SIP as a requirement within the RTCweb architecture because of that. I'm not saying that a new SIP dialect can't be built - but I think it's another WG's task. 

/O

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Wed, 21 Sep 2011 11:55:45 +0530

Hi Roman,

Thanks for the correction. I agree with you that there is no need to
mention that forking is not support in RTCWeb. Your simple model works
for browser as RTP End-point. One problem with the generic statement of
multiple answer is that it may violate offer/answer model itself. For
example: 18x and 200 having different answer for INVITE offer (standard
implementation violation of offer/answer ;-))

Hope we are in the same page that forking related API and its parameters
shall be decided by W3C

Thanks

Partha

From: Roman Shpount [mailto:[email-masked]] 
Sent: Wednesday, September 21, 2011 7:51 AM
To: Ravindran Parthasarathi
Cc: Hadriel Kaplan; Cullen Jennings; [email-masked]
Subject: Re: [rtcweb] Forking & Early Media - Was Re: Minimal SDP
negotiation mechanism

Small correction to what you are saying -- new dialogs cannot be stopped
using CANCEL (this will stop all dialogs). They can only be stopped
using BYE.

I would consider it very unfortunate if we decide not to support forking
in RTC. It needs to be handled, and not in signaling, but by defining
what to do with multiple incoming media RTP streams and multiple
answers. The simplest model would be ignore everything except the RTP
media defined in the last answer and have an ability to update the
answer to the last offer. Alternative would be to create a new
PeerConnection based on a new answer (this is API change so should
probably go to W3C).
_____________
Roman Shpount

On Tue, Sep 20, 2011 at 10:11 PM, Ravindran Parthasarathi
<[email-masked]> wrote:

Thanks
Partha

Behalf
gateway
need

---

From: Saúl Ibarra Corretgé <[email-masked]>
Date: Wed, 21 Sep 2011 09:00:09 +0200

On Sep 21, 2011, at 8:25 AM, Ravindran Parthasarathi wrote:

> Hi Roman,
>  
> Thanks for the correction. I agree with you that there is no need to mention that forking is not support in RTCWeb. Your simple model works for browser as RTP End-point. One problem with the generic statement of multiple answer is that it may violate offer/answer model itself. For example: 18x and 200 having different answer for INVITE offer (standard implementation violation of offer/answer ;-))
>  

I think that the case discussed before was that you may receive several 18x responses because the request forked in the SIP side. So, if more than one of these 18x responses do have an SDP, do we create a PeerConnection for each? One for the first? The last?

--
Saúl Ibarra Corretgé
AG Projects

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 21 Sep 2011 09:02:08 +0200

On 09/20/2011 10:08 PM, Cullen Jennings wrote:
> I think this sounds about right to me - I have been thinking about a draft along the lines of this and showing how it maps to SIP. Keep in mind there are also cases where one sides sends an OFFER, gets an ANSWER, but that ANSWER has an error.
Yes, I discussed this at our team yesterday - it seems to me that the 
offer/answer model only gives you 2 options here:
- Send another OFFER  and hope for a more appropriate ANSWER
- Give up and signal that the connection is closed

We don't have a message that fits in the SDP offer/answer model that 
allows you to inform the correspondent that the ANSWER was erroneous; 
this may argue that the PeerConnection needs to inform the Javascript 
layer and let that layer decide what to do.
>   I think another key component is that a signaling GW needs to pass along an opaque blob that gets passed back to it so you can signaling GW that are only transaction state full and don't have to retain state for the duration of the communications.
Certainly desirable (for gateways that don't want to mess with blob 
internals). So far, it has seemed to us that the operation line + the 
SDP blob forms exactly that type of blob.
>   You also need some identifier information so that a JS application participating in multiple sessions can know what session a given piece of SDP goes with.
When the message goes between JS and PeerConnection, there is such an 
identifier: The PeerConnection object itself. Do we need more?
If the ID needs to be passed outside of the JS execution context, we 
need a globally unique identifier. But I'm not sure we need one all the 
time, and if a specific application needs it, it can generate one.

---

From: Randell Jesup <[email-masked]>
Date: Wed, 21 Sep 2011 03:07:51 -0400

NOTE: Attached below is a proposed set of forking/early-media
and clipping-avoidance rules, so don't glance and delete!  :-)

Also note: I started writing this earlier today, so it was largely
done before much of today's discussion on forking.  I'll note that
I include in this a method to minimize chances of answer-time
clipping.  (For any who don't know (if there are any), this is
where the first fraction of a second after pickup is lost while
answering, starting codecs, doing ICE, etc.)

>  Do we have any use cases using forking? Is forking a desired feature or something that SIP brought in?

No, this is something inherent in a person you want to converse with
possibly being in different places.  Different phones in a home,
different computers in a home or out of it (your desktop, your laptop,
your tablet, your work computer, your Android phone) - when someone
wants to talk to you on Skype or what have you, often the service will
want to offer the connection to any and all devices you're logged into
the service from.  So, it forks the request.  We'd have this issue
even if we totally disallow SIP and disallow PSTN connectivity.  If
you require that the website/server handle this and only provide one
answer, you're much more likely to clip the answer (lose audio right
after accept while the channels are being opened).

Two things in particular appear here.  One is early media (I want to
send media to you but no one has accepted).  I do not propose that
rtcweb generate early media; some sort of "alerting" notification is
enough (equivalent to 180).  (Realize that means no custom callback
tones or video, or weird cases like sitting on hold or in an IVR while
not actually "in" a call).  If so, we only have to worry about interop
cases - calling out to legacy, or *maybe* a call forked in rtcweb
where one of the forks goes to a legacy device or gateway that sends
early media.

The other is choosing which answer to accept if multiple arrive; that
can be up to the application I think (though 99% likely the app will
want to use the first answer).  I don't think we have to *mandate*
that the first answer is the one we use though I can't think of any
cases where we wouldn't, but I'm pretty sure they exist and I wouldn't
want to outlaw them for no reason).  If it makes any use-cases easier
to mandate the first answer, that may change my opinion.  If you're
using SIP (JS or not) that might affect the answer, of course.

While waiting for an acceptance, it makes *lots* of sense to "warm up"
the connection(s) so that when the call is accepted there's minimal
delay or pickup loss.  "warm up" means to do an ICE exchange and
possibly even instantiate codecs, etc.  This is complicated by not
knowing the final answer until the user decides how to answer, but you
could warm up the likely streams/codecs in most cases, and drop some
if needed on ACCEPT.  In the forking case, you could warm up
connections to some or all possible answers.  (Pacing may be an issue
here, but often there are 5-20 seconds to do it in.)

Implicit in this is separating ANSWERs from "acceptance", and
verifying on "acceptance" that the correct ANSWER is used (for
example, we warm up audio and video, and the person answers
audio-only, or for some reason chooses a different codec).

So, to summarize in psuedo-spec language:

0)   I'm assuming an Offer-Answer model here, though not assuming SDP.
      If you want, read "SDP ANSWER" for "ANSWER", etc to map to Harald's
      proposals.  Note that I add "ACCEPT".
0.1) Rough mapping to SIP:
      a) INVITE ->  OFFER
      b) 183 ->  ANSWER
      c) 180 ->  ANSWER-with-no-media-streams
      c) 200 ->  ANSWER (may be suppressed) + ACCEPT
0.2) I'm assuming OFFERs and ANSWERs and ACCEPTs are delivered on
      a reliable, in-order channel.

1) webrtc clients WILL NOT send early media
    [See below; I see no real need for webrtc<->webrtc client connections
     to send early media, but SIP/PSTN interop cases may require it, so
     I have an alternative below]
2) when a webrtc client receives a OFFER, it MAY generate a speculative
    ANSWER in order to allow pre-starting the PeerConnection in a disabled
    state.  If pre-started, NO media shall be sent until the call has been
    ACCEPTED.  Note that the OFFERer may receive data before seeing
    the ACCEPT.
3) if the ANSWERer generated a speculative ANSWER, it may replace that
    with an alternative ANSWER before sending ACCEPT.  This alternative
    SHOULD use the same connection address as the original, and if so
    the existing PeerConnection established or being established SHOULD
    be retained, but the mediastream configuration changed to match
    the new ANSWER.
4) the OFFERer SHOULD pre-start PeerConnections on a speculative ANSWER, or
    they MAY wait until an ACCEPT and then start the last ANSWER from that
    source.  If multiple sources supply speculative ANSWERs, the OFFERer
    MAY pre-start some, none or all of them as it wishes.
    [Open question: do we pre-start MediaStreams in each pre-starting
     PeerConnection, or do (can) we defer this until ACCEPT?]
5) when the OFFERer receives an ACCEPT, it MAY close other PeerConnections
    opened speculatively.
6) when an ANSWERer sends an accept, it MAY begin sending media immediately
    if the PeerConnection was pre-started.  It SHOULD be ready to receive
    media before sending the ACCEPT.
7) servers handling signalling for webrtc clients MAY fork a call offer
    to multiple webrtc clients
8) if a call is forked, the webrtc client MAY receive either a single
    ANSWER and ACCEPT, or MAY receive multiple ANSWERs with one or more
    ACCEPTs, depending on how the server works.

The provides a way to minimize the chances of start-of-call clipping,
and handles forking with minimal clipping (with cooperation of the
app).  Note that there may be a implementation limit on the number of
PeerConnections that can be "warmed up" before an ACCEPT.

Yes, if we remove 1) and replace it with (probably lower down)

N) webrtc clients MAY send "early media" on a pre-started PeerConnection
    but MUST NOT send any media without explicit action or consent of the
    user.  webrtc clients MAY play the early media.

or

N) webrtc media gateways MAY send "early media" on a pre-started PeerConnection,
    and webrtc clients receiving "early media" MAY play it, and MAY send
    media (such as DTMF) but MUST NOT send any media without explicit
    action or consent of the user.

   (and you have to change 2) above)

you get something that is pretty interoperable with legacy SIP devices
and especially PSTN gateways or border controllers, including the infamous
American Airlines DTMF trick.  This assumes a WebRTC<->legacy media gateway
is in use (note that all the above is about PeerConnections).  I have not
tried to figure out how non-gatewayed legacy would work into this, but it
should be doable.

-- 
Randell Jesup
[email-masked]

---

From: "Olle E. Johansson" <[email-masked]>
Date: Wed, 21 Sep 2011 09:26:11 +0200

Randell,

If you want to go down this whole route and include all this application stuff in rtcweb, we might as well include all of SIP, because all of these issues have been worked on in the SIP world for a long time. 

Personally, I see rtcweb as a much lower layer than you do, which in my world excludes all this complexity.

SIP has been around for many years, and some of the discussion you bring to the table hasn't been solved yet or is poorly implemented in end points since they don't understand the complexity. I see no point in trying to solve all of that again with yet another signalling protocol.

We keep going back and forth on the signaling issue here - is it part of RTCweb? If yes, then we'll have forking, early media, DTMF and all that luggage including complex ISDN signalling scenarios that are just part of the German PSTN network and a requirement to have in RTCweb since it works in the ISDN. You'll have reqiuirements that the browse just HAS to support five different SIP Subscribe event packages - or the rtcweb version.

If we put signalling out of scope and let the application builder handle signaling and/or start a separate working group that can define "SIP lite" I believe we can make progress in rtcweb by focusing on a limited set of issues.

Darn. I feel like the old employee saying "we tried that years ago, it did not work." to kill all the inspired people that belive they can make a change. My apologies. Prove me wrong :-)

/O

---

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 21 Sep 2011 10:22:36 +0200

On 2011-09-20 21:58, Cullen Jennings wrote:
>>> We need to handle glare - when one sends an OFFER and gets back
>>> an OFFER, reply with SDP ERROR, enter a "glare" state, wait a
>>> bit, and send out an offer again.
>>> 
>> 
>> I think that is a bad design for glare handling. I agree it needs
>> to be handled. I think the ICE version of glare handling could work
>> pretty well here. Any offer needs to include a random 32 bit
>> number. If the end-point receive an offer in relation to a
>> peer-connection while it has an outstanding offer then it compares
>> these codes. Who ever has the greatest numerical value wins and
>> that offer is handled the other is responded with an error stating
>> GLARE. Thus there is no extra delay and more clear resolution of
>> the glare case.
>> 
>> Cheers
>> 
>> Magnus Westerlund
> 
> I think it has to be possible to gateway what happens here to what
> happens in SIP with glare - I suspect that more or less puts us on
> using the SIP glare mechanism.

I agree with that it needs to be possible to gateway with SIP. I will
think about if my proposal really is that difficult to interact with SIP.

And why, did you design such a poor glare handling algorithm for SIP? ;-)

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 21 Sep 2011 11:51:25 +0200

> more clear resolution of the glare case.
Hm... thinking about it, I think this can be incorporated.
The reason it works is that the signalling channel is supposed to 
guarantee reliable, ordered delivery, so we know that both parties agree 
that a glare condition exists. (actually, that condition doesn't hold 
for ICE, so I'm a bit surprised it works there.)

If we specify that the random number is 1 < N < MAXINT, we can even 
incorporate the SIP interworking case by saying that a gateway can force 
the party into GLARE by specifying one of the edge cases - whatever 
makes interworking happy.

---

From: "Muthu Arul Mozhi Perumal (mperumal)" <[email-masked]>
Date: Wed, 21 Sep 2011 17:04:37 +0530

|Not really - there will be signaling, because there has to 
|be SDP answers even just to get ICE to work before the media 
|starts flowing in many NAT cases.  And even in practice in 
|SIP there're usually SDP answers in 18x to open "gates", and 
|to get upstream DTMF.  So if the concern is just that there's
|no signaling to tell the browser there are multiple RTP streams
|coming, I think that can be allayed.

I think it is further allayed with ICE because ICE requires that early
media be not sent until the ICE connectivity check is done and a valid
candidate pair is selected -- it requires implementations to delay
alerting the called party until this point (in the case of a PSTN
gateway, this would mean that the setup message into the PSTN is delayed
until this point). So, the RTCWeb app would have to just create a single
PeerConnection object for receiving/transmitting a media stream once
this procedure is completed.
 
Of course there is complexity in the SIP/PSTN interworking gateway, but
it doesn't look the RTCWeb app would have worry about it.

Muthu

|-----Original Message-----
|From: [email-masked] [mailto:[email-masked]] On
Behalf Of Hadriel Kaplan
|Sent: Wednesday, September 21, 2011 4:06 AM
|To: Cullen Jennings (fluffy)
|Cc: [email-masked]
|Subject: Re: [rtcweb] Forking & Early Media - Was Re: Minimal SDP
negotiation mechanism
|
|
|On Sep 20, 2011, at 4:07 PM, Cullen Jennings wrote:
|
|> That said, I think that doing both forking and early media is hard.
Lets assume we are using a
|signaling gateway that is not a media gateway to translate between a
SIP call on one side and whatever
|is happening over on the browser side. The basic issue is the browser
initiating the communications
|needs to be able to start receiving multiple RTP streams before it even
has signaling information to
|tell it how many it might receive.
|
|Not really - there will be signaling, because there has to be SDP
answers even just to get ICE to work
|before the media starts flowing in many NAT cases.  And even in
practice in SIP there're usually SDP
|answers in 18x to open "gates", and to get upstream DTMF.  So if the
concern is just that there's no
|signaling to tell the browser there are multiple RTP streams coming, I
think that can be allayed.
|
|The really hard part is knowing which stream to use/render/send-to,
imho.  And putting that decision
|in the gateway isn't good - the best decider of that is probably the JS
in the browser.
|
|
|> To simplify this problem, Cary and my draft proposes not allowing
forking on the SIP side of the
|signaling gateway but still allowing early media. If you wanted to do
do forking in this case, one
|would need a SBC that processed media and turned the forked medial legs
into one media leg.
|
|Obviously you can request that a request not be forked, using
caller-prefs, but you can't "not allow"
|forking on the SIP side.  That would make it not SIP.  I know forking
is hard, but that's life.  It's
|not appropriate for this WG to make fundamental changes/limitations to
the SIP protocol, just because
|some of it's "hard" for a browser.
|
|-hadriel
|
|_______________________________________________
|rtcweb mailing list
[email-masked]
|https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Roman Shpount <[email-masked]>
Date: Wed, 21 Sep 2011 10:45:31 -0400

We should have an option to clone (or conditionally clone) the
PeerConnection based on the new answer as well as provide an answer update
to the exiting PeerConnection. In the first case, application can create a
separate media connection to a forked response and render it separately. In
the second, application can start rendering input from a new answer as well
as sending media to it. This way we can address all the possible scenarios
and leave control in the application developer's hands.

As far as offer/answer is concerned RTC should be able to interop with it,
but I would assume we should be able to add new features and functionality
if they would enable us to build new applications or interop with existing
deployments. Processing answer updates or offer failures is not described in
offer.answer but it will not break interoperability of RTC with any
offer/answer solutions
_____________
Roman Shpount

On Wed, Sep 21, 2011 at 3:00 AM, Saúl Ibarra Corretgé
<[email-masked]>wrote:

> mention that forking is not support in RTCWeb. Your simple model works for
> browser as RTP End-point. One problem with the generic statement of multiple
> answer is that it may violate offer/answer model itself. For example: 18x
> and 200 having different answer for INVITE offer (standard implementation
> violation of offer/answer ;-))
> >
>
> I think that the case discussed before was that you may receive several 18x
> responses because the request forked in the SIP side. So, if more than one
> of these 18x responses do have an SDP, do we create a PeerConnection for
> each? One for the first? The last?
>
> --
> Saúl Ibarra Corretgé
> AG Projects
>
>
>
>

---

From: Roman Shpount <[email-masked]>
Date: Wed, 21 Sep 2011 11:00:11 -0400

Randell,

Are we going to create multiple PeerConnections in case of forking when
multiple ANSWER + ACCEPT are received?
_____________
Roman Shpount

On Wed, Sep 21, 2011 at 3:07 AM, Randell Jesup <[email-masked]>wrote:

>>  I just wanted to get rid of the early/late media discussion. As you
>> state, the forking issue with getting multiple responses is a separate
>> issue.
>>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Randell Jesup <[email-masked]>
Date: Wed, 21 Sep 2011 08:37:53 -0700

On 9/21/2011 8:00 AM, Roman Shpount wrote:
> Randell,
>
> Are we going to create multiple PeerConnections in case of forking 
> when multiple ANSWER + ACCEPT are received?

That would be up to the application in my proposal.  I stated that on 
ACCEPT the OFFERer MAY close any 'warm' PeerConnections:

I didn't specify, but there's no requirement on multiple ACCEPT to close 
the other connections, so it's up to the app.

-- 
Randell Jesup
[email-masked]

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 07:17:23 -0600

> Obviously you can request that a request not be forked, using caller-prefs, but you can't "not allow" forking on the SIP side.  That would make it not SIP.  I know forking is hard, but that's life.…

I'll note that Cary and my draft has been putting serious limitation on SIP since the beginning and we have yet to receive a comment on that. You do realize the outcome of this decision would be that you needed an SBC between the SIP to WEB Gateway and any SIP network that forked so that SBC could isolate the other side from the forks. As full disclosure, really, I don't own any ACME stock :-) 

Clearly one could support forking in the browsers and equally obviously, that complicates things fairly significantly. The question will be if the complexity is worth the end user gain in functionality.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 07:44:43 -0600

> And why, did you design such a poor glare handling algorithm for SIP? ;-)

So, the SIP glare algorithm leaves much to be desired, no questions about it. I'm going to claim I was not there at the time until someone proves otherwise then switch my story to I was drunk not stupid. (in fairness we were trying to be backwards compatible with 2543 equipment).

Anyways, I've been bouncing ideas around with folks on how we might improve the situation. Here is a proposal that might improve it for SIP and could also be leveraged by RTCWeb. 

First, to review the current situation, the current algorithm is:

When two sides detect glare, they each pick a random integer from 1 to 10 and tell the other side to retry in that many seconds. two obvious problems with this: 1) it introduces, on average, over 7 seconds of delay which is a long time for a user 2) it has a non trivial chance of having glare a second time and needing to be repeat

I'm proposing an extension that would go something like this:

Both sides always include a random tie breaker number in their SDP that is used for glare resolution. If both sides support this and have the number, then glare is resolved by whoever has the lowest number. 

If neither side supports this, obviously the current things is what happens. 

if one side supports it but the other does not, the side that supports it tells the other side to retry after 0 seconds. So effectively it causes the other side to always "win" the tie and retry first. This reduces the latency of the glare resolution to the end user. The other side that does not support the new algorithm will still send a retry after some random number form 1 to 10 seconds. However, the side that supports this new mechanisms might want to decide that under some conditions, ti can short circuit that timer and retry sooner than that. It's not exactly clear to me when to short circuit this. Could it short circuit after receiving the retied transaction from the other side plus a one RTT delay? Could we just retry after 3 RTT or so knowing the other side won the race and was retying immediately? 

Even if the UA just runs out the full retry time, this algorithm still reduces the average delay significantly for cases where only one side supports it and gets really fast when both sides support it. 

Thoughts?

I suspect this would need an SDP extension to carry the tie breaker number or even if we used an existing number, it seems like we still need something that indicates support for this new extension.

---

From: Tim Panton <[email-masked]>
Date: Thu, 22 Sep 2011 06:57:36 -0700

> So, the SIP glare algorithm leaves much to be desired, no questions about it. I'm going to claim I was not there at the time until someone proves otherwise then switch my story to I was drunk not s…

Which is an important lesson for rtcweb, backward compatibility has longterm costs.

At the risk of showing my ignorance - why are we expecting glare to be a problem? 
To my mind glare only happens when you have a locked resource e.g. a busy line
or number . Rtcweb does not contain either of those concepts, what's the resource
that is being competed for here ?

Tim (speaking entirely for his own ignorance)

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 22 Sep 2011 14:04:45 +0000

> I'll note that Cary and my draft has been putting serious limitation on SIP since the beginning and we have yet to receive a comment on that. You do realize the outcome of this decision would be th…

Oh, I think we'll be busy enough. ;)

Choosing the right fork is really hard. (or so it seems when I sit down at fancy restaurants)

> Clearly one could support forking in the browsers and equally obviously, that complicates things fairly significantly. The question will be if the complexity is worth the end user gain in functiona…

Ignoring the issue of not being SIP anymore, I think you'll want forking in the end.  Even in XMPP people have asked for it now and then.

But anyway sorry I didn't read Cary and your draft before.  Would you like comments now?  :)
First, I assume you mean draft-cbran-rtcweb-protocols-00.

You have the following as optional:
   o  INVITEs without an offer
   o  re-INVITEs
   o  forking
   o  S/MIME
   o  SIPS URI Scheme

My comments:
S/MIME has never seen the light of day and would be removed if we ever raised 3261 to the soon-to-be-second-and-final level of maturity, so sure.
INVITEs without offers can be ignored in terms of generating them from rtcweb, but rtcweb has to be able to receive them. (I would think Cisco in particular would demand this ;)
Re-INVITEs again may not need to be generated by rtcweb, but have to be able to be received. (it's not hard is it?)
Forking is needed to eat our own dogfood with.
SIPS is sadly rare, but ironically I would think the rtcweb model is one we'd actually want and use it in (because we could require HTTPS be used, and it probably would even happen!).

Last comment: you gave me a good chuckle, making 4474/4916 required.  :)

-hadriel

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 22 Sep 2011 14:40:33 +0000

> that is being competed for here ?

The resource being competed for is SDP state/media info of the same session.  If both sides send a new SDP offer during the session at the same time, one side has to win.

One would think this would be ultra-rare, but in SIP it happens a lot in some deployments. 

-hadriel

---

From: Roman Shpount <[email-masked]>
Date: Thu, 22 Sep 2011 10:49:39 -0400

Since we are commenting on this draft, I, as I've previously stated on this
list, really would not like to see SIP as something directly supported by
Web RTC, especially a version of SIP lite described in this draft.

If we decide to build SIP into RTC, I would assume we would need a secure
channel for signaling, so we will need TLS transport. Also, if we plan to
use ICE, SIP messages will be larger then the SIP standard limit set for UDP
transport, so even in the cases of non secure connections we will need TCP
for signaling, not UDP with STUN keep alives. If we decide to implement SIP
over TCP from behind NAT we will need to fix RFC 5626 to provide a contact
for calls and transactions that can be recovered after a TCP connection
reset. As far as I know this standard assumes that GRUU is used for call
contacts, and is recovered using new registrations, but does nor provide any
guidance on what suppose to happen with transaction responses if connection
is reset. After we fix this spec, industry will have to go and implement
signaling servers that support such functionality (ASFAK there are none in
existence) and proxy it to normal SIP. I believe, at this point it would
actually be easier to build a JavaScript library + HTTP(S) to SIP proxy.
_____________
Roman Shpount

On Thu, Sep 22, 2011 at 10:04 AM, Hadriel Kaplan <[email-masked]>wrote:

> > I'll note that Cary and my draft has been putting serious limitation on
> SIP since the beginning and we have yet to receive a comment on that. You do
> realize the outcome of this decision would be that you needed an SBC between
> the SIP to WEB Gateway and any SIP network that forked so that SBC could
> isolate the other side from the forks. As full disclosure, really, I don't
> own any ACME stock :-)
>
> > Clearly one could support forking in the browsers and equally obviously,
> that complicates things fairly significantly. The question will be if the
> complexity is worth the end user gain in functionality.
>
> Ignoring the issue of not being SIP anymore, I think you'll want forking in
> the end.  Even in XMPP people have asked for it now and then.
>
> S/MIME has never seen the light of day and would be removed if we ever
> raised 3261 to the soon-to-be-second-and-final level of maturity, so sure.
> INVITEs without offers can be ignored in terms of generating them from
> rtcweb, but rtcweb has to be able to receive them. (I would think Cisco in
> particular would demand this ;)
> Re-INVITEs again may not need to be generated by rtcweb, but have to be
> able to be received. (it's not hard is it?)
> SIPS is sadly rare, but ironically I would think the rtcweb model is one
> we'd actually want and use it in (because we could require HTTPS be used,
> and it probably would even happen!).
>

---

From: Roman Shpount <[email-masked]>
Date: Thu, 22 Sep 2011 10:55:27 -0400

If we are planning to support SIP, then standard way of dealing with glare
actually works fairly well. We would probably not get as much glare even
with SIP if we would not use re-INVITEs for keep alive messages (SIP session
timers). If we do not plan to support SIP, all we need is an ability in the
API to cancel an offer and leave glare resolution to signaling. In general
glare is unavoidable, for instance in case were both parties in the call
decide for some reason to switch the video on.
_____________
Roman Shpount

On Thu, Sep 22, 2011 at 10:40 AM, Hadriel Kaplan <[email-masked]>wrote:

> The resource being competed for is SDP state/media info of the same
> session.  If both sides send a new SDP offer during the session at the same
> time, one side has to win.
>

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 22 Sep 2011 17:11:34 +0200

Hi,

Here is an analysis of what happens including Cullen's SIP side
optimization if one uses a random value attached to the offer as
tiebreaker and interop with SIP.

Lets start with the general scenario.

A <-RTCWEB-> WS/SGW <-SIP-> B

So lets look at the glare conditions that can occur.

a) Initially in regards to the creation of a PeerConnection/call where
both sides attempt to create a session with its peer.

b) During a re-Offer/re-inivite in an existing session.

However, I don't think this really matters for the glare resolution
mechanism. It might effect which SIP messages are sent in which case.

Secondly the glare condition will be detected in various cases depending
on in what state of signalling the glare occurs.

1) If A sends an offer that left A, but not arrived at the GW, while B's
are on its way between the GW and A.
2) IF A sends an offer that has passed the GW, while B's are on its way
between B and the GW.
3) Both A and Bs Offers are at the GW while neither has been forwarded.
4) Like 2) but B supports Cullen's behavior.

In case 1, entity A (a RTCWEB browser) will receive the Offer from B
when itself has an outstanding offer. It perform the local tie-breaking
of the SDP and determine that either A or Bs Offer is the winning. If A
wins it just discards B, alternatively send an error (but that doesn't
appear necessary). If B wins it abandons its Offer and process B's.

In case 1, the GW when A's Offer arrive at the GW, it has already sent
B's offer with its tie-breaking value. It immediately determine who won
the tie-breaking and then continue acting according to it. If A's won
then it sends a 491 with a significant large delay value. Then it sends
A's offer with some small delay just as it got lucky with a low value.
If B won, then it just discards A's Offer and waits for A's answer to
B's offer.

In case 2) the GW could look for a tie-breaker value according to
Cullen's proposal and if present then determine the winner. Considering
that the quickest way to resolve the tie-breaking towards the RTCWEB
client if the value isn't present is to fold for the RTCWEB client's
behalf. Thus set B's tie-breker value to max. Then respond with a 491
with delay 0 to B. When B come back with an new offer it assigns this
the highest tie-breaker value and send it to A. A will then abort its
offer and accept B's.

In case 2) in B, it will be completely in the SIP domain and it will
respond with a 491 with a random delay value back to the GW. The GW will
consume this message when it arrives.

In case 3) the GW can avoid having the SIP UA (B) know there was any
glare at all. This by setting the tie-breaking value for B's offer to
MAX assuring it wins over A, and then send it to A to replace the offer.

In case 4) the GW will have sent a tie-breaker value with A's offer to B
in the SDP. B's offer with a tie-breaker value arrives at the GW. The GW
determine who wins. If B wins it forwards B's offer to A. If A wins it
will do something to make B's offer being canceled. Maybe a 491 answer
is the simplest. Except that it will not be retired.

In case 4) the UA (B) will have sent an invite with a tie-breaker value.
When A's Invite with a tie-breaking value arrive it can perform the
tie-breaking. If A wins then it immediately process it as if no
outstanding Invite exist. (Question: how will the proxies react on that
B responds immediately?). If B wins it sends a 491 for A's Invite. Then
it continues waiting for the response.

So independent of the optimization this can work as long as a GW is
allowed to override a random selection mechanism in cases where the
other protocol will resolve quicker if it does.

I think the optimization makes sense, but is more a question of fixing
SIP than really being needed for RTCWEB. I also wonder why not a SIP
header for the tie-breaking? There appear also to be some more cases to
analyze in detail and to figure out if the scheme breaks with some
middleboxes.

Cheers

Magnus Westerlund

---

From: "Elwell, John" <[email-masked]>
Date: Thu, 22 Sep 2011 17:40:28 +0200

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Olle E. Johansson
> Sent: 21 September 2011 08:26
> To: Randell Jesup
> Cc: [email-masked]
> Subject: Re: [rtcweb] Forking & Early Media - Proposal
> 
> Randell,
> 
> If you want to go down this whole route and include all this 
> application stuff in rtcweb, we might as well include all of 
> SIP, because all of these issues have been worked on in the 
> SIP world for a long time. 
> 
> Personally, I see rtcweb as a much lower layer than you do, 
> which in my world excludes all this complexity.
[JRE] I think the point that Randell was trying to make was that forking might be a requirement even for browser-to-browser calls. If so, what he proposes makes sense. If not, SIP entities (e.g., gateways) need to hide forking, so that the RTC-Web environment doesn't need to take it into account. Even then, I rather like Randell's idea of a speculative answer and separating answer from accept, because the speculative answer can help to avoid clipping, which can give rise to user complaints.

John

John Elwell
Tel: +44 1908 817801 (office and mobile)
Email: [email-masked]
http://www.siemens-enterprise.com/uk/

Siemens Enterprise Communications Limited.
Registered office: Brickhill Street, Willen Lake, Milton Keynes, MK15 0DJ.
Registered No: 5903714, England.

Siemens Enterprise Communications Limited is a Trademark Licensee of Siemens AG.

> 
> SIP has been around for many years, and some of the 
> discussion you bring to the table hasn't been solved yet or 
> is poorly implemented in end points since they don't 
> understand the complexity. I see no point in trying to solve 
> all of that again with yet another signalling protocol.
> 
> We keep going back and forth on the signaling issue here - is 
> it part of RTCweb? If yes, then we'll have forking, early 
> media, DTMF and all that luggage including complex ISDN 
> signalling scenarios that are just part of the German PSTN 
> network and a requirement to have in RTCweb since it works in 
> the ISDN. You'll have reqiuirements that the browse just HAS 
> to support five different SIP Subscribe event packages - or 
> the rtcweb version.
> 
> If we put signalling out of scope and let the application 
> builder handle signaling and/or start a separate working 
> group that can define "SIP lite" I believe we can make 
> progress in rtcweb by focusing on a limited set of issues.
> 
> Darn. I feel like the old employee saying "we tried that 
> years ago, it did not work." to kill all the inspired people 
> that belive they can make a change. My apologies. Prove me wrong :-)
> 
> /O
> 
> 
> 
> >>> It's not about knowing the difference between "early" and 
> "late" media - it's about whether the API and browser need to 
> support multiple SIMULTANOUS SDP answers - or whether we 
> assume that the JS SIP app will always, at any given time, 
> only provide ONE SDP answer to the API and browser.
> >> I just wanted to get rid of the early/late media 
> discussion. As you state, the forking issue with getting 
> multiple responses is a separate issue.
> >> 
> >> Do we have any use cases using forking? Is forking a 
> desired feature or something that SIP brought in?
> >

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 22 Sep 2011 19:37:54 +0200

> I suspect this would need an SDP extension to carry the tie breaker number or even if we used an existing number, it seems like we still need something that indicates support for this new extension.
Seems like it could be a really short I-D, but definitely needs to be 
written up.

Magnus' analysis worries me a bit, because it seems to assume specific 
functionality in the gateway (tracking of state and ability to generate 
SIP messages depending on state).
It seems reasonably simple to build a gateway, but we quickly get to the 
point where we have to write standards for the gateway function .... 
which could lead us down rather deep ratholes.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 22 Sep 2011 20:20:52 +0200

2011/9/22 Harald Alvestrand <[email-masked]>:
> functionality in the gateway (tracking of state and ability to generate SIP
> messages depending on state).
> point where we have to write standards for the gateway function .... which
> could lead us down rather deep ratholes.

Are we assuming that a media gateway will be required for RTP/media
communication between a WebRTC client (web browser) and a SIP node?
If such decision is taken IMHO it's sad.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 12:56:59 -0600

> Tim (speaking entirely for his own ignorance)

Tim, it happens a fair amount in our playing around with browser deployments. Imagine you and I both using browsers and have an audio call set up.  We are talking an I say "hey, lets, add video", then we both go and push our "enable video" buttons at roughly the same time. In this case we often get glare. 

Part of the reasons is that the signaling path is not really fast as it involves waiting for web servers to deal with sending notifications down to other side. It's hard to guess what the signaling delay will be but it easy to imagine that it will be over 200 ms in many cases  and pretty easy to imagine that both of use would would click the add video button within 1/4 of second of each other. 

The glare we are talking about here is any time browser A has sent an SDP offer to the other side, but before A has received an SDP answer, the other side sends A and SDP offer.

---

From: Dzonatas Sol <[email-masked]>
Date: Thu, 22 Sep 2011 12:11:40 -0700

On 09/22/2011 10:37 AM, Harald Alvestrand wrote:
>> So, the SIP glare algorithm leaves much to be desired, no questions 
>> about it. I'm going to claim I was not there at the time until 
>> someone proves otherwise then switch my story to I was drunk not 
>> stupid. (in fairness we were trying to be backwards compatible with 
>> 2543 equipment).
>>
>> Anyways, I've been bouncing ideas around with folks on how we might 
>> improve the situation. Here is a proposal that might improve it for 
>> SIP and could also be leveraged by RTCWeb.
>>
>> When two sides detect glare, they each pick a random integer from 1 
>> to 10 and tell the other side to retry in that many seconds. two 
>> obvious problems with this: 1) it introduces, on average, over 7 
>> seconds of delay which is a long time for a user 2) it has a non 
>> trivial chance of having glare a second time and needing to be repeat
>>
>> Both sides always include a random tie breaker number in their SDP 
>> that is used for glare resolution. If both sides support this and 
>> have the number, then glare is resolved by whoever has the lowest 
>> number.
>>
>> if one side supports it but the other does not, the side that 
>> supports it tells the other side to retry after 0 seconds. So 
>> effectively it causes the other side to always "win" the tie and 
>> retry first. This reduces the latency of the glare resolution to the 
>> end user. The other side that does not support the new algorithm will 
>> still send a retry after some random number form 1 to 10 seconds. 
>> However, the side that supports this new mechanisms might want to 
>> decide that under some conditions, ti can short circuit that timer 
>> and retry sooner than that. It's not exactly clear to me when to 
>> short circuit this. Could it short circuit after receiving the retied 
>> transaction from the other side plus a one RTT delay? Could we just 
>> retry after 3 RTT or so knowing the other side won the race and was 
>> retying immediately?
>>
>> Even if the UA just runs out the full retry time, this algorithm 
>> still reduces the average delay significantly for cases where only 
>> one side supports it and gets really fast when both sides support it.
>>
>> Thoughts?
>>
>> I suspect this would need an SDP extension to carry the tie breaker 
>> number or even if we used an existing number, it seems like we still 
>> need something that indicates support for this new extension.
>>
> functionality in the gateway (tracking of state and ability to 
> generate SIP messages depending on state).
> It seems reasonably simple to build a gateway, but we quickly get to 
> the point where we have to write standards for the gateway function 
> .... which could lead us down rather deep ratholes.

May seem like that for mobiles, and even for mobile applications even if 
the unit is stationary. I think we can talk about gateways easier than 
blackboxes, especially the kind that are mostly like on-board each 
future car and have relatively stationary apps (from concept car to 
"printed" driverless vehicles that rolls itself to destination of the 
build order).

The initial roll-off: best done with nothing explosive, so we get into 
suspend-states and solid-states.

Both lead to gateways, so my perspective is the browser on the gateway 
rather than the browser on the mobile. One can "key-in" much like 
"dial-in", so we can limit ICE here in this model within range of the 
gateway (on the wire).

Capture the moment. If we clone that moment unto the next vehicle then 
do we worry if the same random number appears the same after suspend? 
Remember that the "power supply is suspended" yet there is energy from 
somewhere that completes this initial roll-off.

Win? Maybe we can show the fuel icon with an estimated time until 
refueled again or... until next cybernetic "prime" because you know 
the-new-owner can than innovate rather than think about scalable 
self-assembly (i.e. ag), or at least on the run-way this looks more out 
in the open... for "observation" (as over 200,000 of these clones 
roll-off each day).

Maybe the one ahead should help power the build for the one behind, the 
digital divide.

-- 

---

---

From: Matthew Kaufman <[email-masked]>
Date: Thu, 22 Sep 2011 12:33:49 -0700

On 9/22/11 11:56 AM, Cullen Jennings wrote:
> Tim, it happens a fair amount in our playing around with browser 
> deployments.

You make my own arguments for me.

> Imagine you and I both using browsers and have an audio call set up. 

Sounds great. And the web server has sent down some javascript that was 
able to query the capabilities and so it knows which audio and video 
codecs are supported at each end, even before the audio call started.

> We are talking an I say "hey, lets, add video", then we both go and 
> push our "enable video" buttons at roughly the same time.

Also good.

> In this case we often get glare. 

You shouldn't.

Each side is asking the server to tell the other end to start sending 
video, and the server knows what codecs and parameters will work, so 
there's no overlap at all.

> Part of the reasons is that the signaling path is not really fast as 
> it involves waiting for web servers to deal with sending notifications 
> down to other side. It's hard to guess what the signaling delay will 
> be but it easy to imagine that it will be over 200 ms in many cases 
> and pretty easy to imagine that both of use would would click the add 
> video button within 1/4 of second of each other. The glare we are 
> talking about here is any time browser A has sent an SDP offer to the 
> other side, but before A has received an SDP answer, the other side 
> sends A and SDP offer.

See, that's the problem.

IF there are SDP offers and answers, they shouldn't be tied to the peer 
connection object, they should be tied to the objects that represent the 
devices... that way you would have two separate offer/answer happening, 
one for each direction.

And if you *weren't* using offer/answer, but had instead collected the 
capabilities ahead of time, then the server could just immediately tell 
each end to start sending.

So instead of a 200 msec delay *and* the risk of glare, you'd have zero 
delay *and* zero chance of glare.

Matthew Kaufman

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 22 Sep 2011 21:41:15 +0200

Hi, 

>> That said, I think that doing both forking and early media is hard. Lets assume we are using a signaling gateway that is not a media gateway to translate between a SIP call on one side and whatever is happening over on the browser 
>>de. The basic issue is the browser initiating the communications needs to be able to start receiving multiple RTP streams before it even has signaling information to tell it how many it might receive.
>
>Not really - there will be signaling, because there has to be SDP answers even just to get ICE to work before the media starts flowing in many NAT cases.  And even in practice in SIP there're usually SDP answers in 18x to open 
>"gates", and to get upstream DTMF.  So if the concern is just that there's no signaling to tell the browser there are multiple RTP streams coming, I think that can be allayed.  
>

I am not sure whether the JS app would be able to make better decisions than the gateway - and I've never seen a SIP client which would allow the user to choose which early media to listen to. 

And, sometimes SBGs, eventhough they allow multiple early dialogs to pass towards the user, still only allow media associted with one of the early dialogs to pass.

In my opinion the best thing would be if we could come up with some common BCP, e.g. saying that media associated with the early dialog on which the latest SDP answer was received is considered "active", and will be presented to the user etc. 

Or something like that...

The network will then have to make sure that SDP answers are forwarded towards the user in a way that the "correct" media will be accepted and presented.

That was, once again, my input for the annual how-to-support-early-media debate :)

>>To simplify this problem, Cary and my draft proposes not allowing forking on the SIP side of the signaling gateway but still allowing early media. If you wanted to do do forking in this case, one would need a SBC that processed 
>>media and turned the forked medial legs into one media leg. 
>
>Obviously you can request that a request not be forked, using caller-prefs, but you can't "not allow" forking on the SIP side.  That would make it not SIP.  I know forking is hard, but that's life.  It's not appropriate for this WG 
>to make fundamental changes/limitations to the SIP protocol, just because some of it's "hard" for a browser. 

We need to keep in mind that supporting forking and supporting media from multiple locations are two different things, and I think we need to be very clear about what we mean when discussing.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 22 Sep 2011 21:48:00 +0200

Hi,

>I would consider it very unfortunate if we decide not to support forking in RTC. It needs to be handled, and not in signaling, but by defining what to do with multiple incoming media RTP streams and multiple answers. The simplest model would be ignore everything except the RTP media defined in the last
>answer and have an ability to update the answer to the last offer.

Correct.

>Alternative would be to create a new PeerConnection based on a new answer (this is API change so should probably go to W3C).

This is one of the things that maybe should be put in the "nice to have in a later release" bucket :)

Regards,

Christer

On Tue, Sep 20, 2011 at 10:11 PM, Ravindran Parthasarathi <[email-masked]<mailto:[email-masked]>> wrote:

Thanks
Partha

>From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On
Behalf
>Cc: [email-masked]<mailto:[email-masked]>
gateway
need
>[email-masked]<mailto:[email-masked]>
_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 13:51:13 -0600

> If such decision is taken IMHO it's sad.

My take is that most people want to make sure that a translator between SIP and web stuff would only need to look at signalling - it would not touch the media. I'm working on the asumption that is what any solution will look like. So, no media GW.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 22 Sep 2011 21:53:02 +0200

> My take is that most people want to make sure that a translator between SIP and web stuff would only need to look at signalling - it would not touch the media. I'm working on the asumption that is…

Yes please :)

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 13:53:51 -0600

Could you'd like to elaborate on how it works when one of the sides is an SIP endpoint, or we the two endpoint are at different domains that are federated?

On Sep 22, 2011, at 1:33 PM, Matthew Kaufman wrote:

>> Tim, it happens a fair amount in our playing around with browser deployments.
> 
> Sounds great. And the web server has sent down some javascript that was able to query the capabilities and so it knows which audio and video codecs are supported at each end, even before the audio call started.
> 
>> We are talking an I say "hey, lets, add video", then we both go and push our "enable video" buttons at roughly the same time.
> 
> Also good.
> 
> Each side is asking the server to tell the other end to start sending video, and the server knows what codecs and parameters will work, so there's no overlap at all.
> 
>> Part of the reasons is that the signaling path is not really fast as it involves waiting for web servers to deal with sending notifications down to other side. It's hard to guess what the signaling delay will be but it easy to imagine that it will be over 200 ms in many cases and pretty easy to imagine that both of use would would click the add video button within 1/4 of second of each other. The glare we are talking about here is any time browser A has sent an SDP offer to the other side, but before A has received an SDP answer, the other side sends A and SDP offer.
> 
> See, that's the problem.
> 
> IF there are SDP offers and answers, they shouldn't be tied to the peer connection object, they should be tied to the objects that represent the devices... that way you would have two separate offer/answer happening, one for each direction.
> 
> And if you *weren't* using offer/answer, but had instead collected the capabilities ahead of time, then the server could just immediately tell each end to start sending.
> 
> So instead of a 200 msec delay *and* the risk of glare, you'd have zero delay *and* zero chance of glare.
> 
> Matthew Kaufman
>

---

From: Matthew Kaufman <[email-masked]>
Date: Thu, 22 Sep 2011 13:00:03 -0700

> Could you'd like to elaborate on how it works when one of the sides is an SIP endpoint, or we the two endpoint are at different domains that are federated?

The server uses its knowledge of the web browser capabilities to 
generate an SDP offer and/or reply with an SDP answer as appropriate and 
then requests that the browser begin sending and/or prepare for 
receiving media as appropriate.

This is exactly the same answer as "how does it work when you have an 
MGCP (or Skinny) device and you want to talk to a SIP endpoint or the 
other endpoint is at the far side of a SIP trunk"

Matthew Kaufman

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 22 Sep 2011 22:04:34 +0200

Hi Cullen, 

>>> Magnus' analysis worries me a bit, because it seems to assume 
>>> specific functionality in the gateway (tracking of state and ability 
>>> to generate SIP messages depending on state).

If so, what is your assumption then regarding ICE? That the SIP nodes will support ICE, or that the browser will be allowed to communicate with the SIP nodes without enabling ICE?

Regards,

Christer

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 22 Sep 2011 14:37:56 -0600

> If so, what is your assumption then regarding ICE? That the SIP nodes will support ICE, or that the browser will be allowed to communicate with the SIP nodes without enabling ICE?

I see no way of solving the security problems without having ICE or something more or less like it. Therefore, I'm working on the assumption that it will only work if the SIP side supports ICE, or is front ended by a SBC with media GW that does ICE. In the short term, there will be some devices that don't do ICE but SIP devices are increasingly having ICE added. Particularly SIP devices that are internet facing because the need for NAT traversal. 

I find requiring ICE to be a very unfortunate assumption to have to make - obviously it reduces the number of legacy voip devices WebRTC devices can talk to without an SBC but I don't see any way around this limitation. Allowing web browsers inside the firewall to send packets to an arbitrary address that is inside the firewall with no validation that address speaks RTP is not acceptable.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 22 Sep 2011 22:57:33 +0200

> I find requiring ICE to be a very unfortunate assumption to have to make - obviously it reduces the number of legacy voip devices WebRTC devices can talk to without an SBC but I don't see any way a…

Hi Cullen, if that is not so a problem in current SIP networks (most
of the devices don't support ICE), why should it be so hard and
critical for WebRTC browsers? (sorry if I miss something).

Also there are cases in which the provider (the SIP provider) does
some rewrite in both the SDP offer and answer in order to force the
media path through a media gateway (RTP proxy). Would that be not
feasible for WebRTC clients?

Thanks a lot.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 22 Sep 2011 22:55:24 +0000

> I am not sure whether the JS app would be able to make better decisions than the gateway - and I've never seen a SIP client which would allow the user to choose which early media to listen to. 

But these aren't phone-call SIP clients - they're browsers.  They can show the multiple video streams in separate windows, for example.  They can show multiple chat sessions per fork too.  Or they may just pick the most recent one and only display it.  That's the nice thing about JS: the JS developer decides what the user experience model is like, based on what type of application/service they're providing; as opposed to the IETF/W3C choosing for them.

> In my opinion the best thing would be if we could come up with some common BCP, e.g. saying that media associated with the early dialog on which the latest SDP answer was received is considered "ac…

That's not the whole problem though.  Suppose it only forked to two agents: Robert and Bob.  Robert sent a 18x with SDP first and then Bob.  So the rtcweb browser starts off using Robert's, does ICE with Robert, etc., and then get's Bob's and starts using Bob's because it's fresher.  Now Robert accepts the connection and/or Bob rejects it; how does the rtcweb browser revert to using Robert?  

If the answer is "well the rtcweb browser had to keep state information about Robert's SDP/media info as well as about Bob's", then we're back to the tricky part about PeerConnection having multiple media peers when it only expected one; if we can solve that, then we've solved the basic issue at hand. (I think... or I need more coffee)

-hadriel

---

From: Roman Shpount <[email-masked]>
Date: Thu, 22 Sep 2011 19:24:34 -0400

> That's not the whole problem though.  Suppose it only forked to two
agents: Robert and Bob.  Robert sent a 18x with SDP first and then Bob.  So
the rtcweb browser starts off using Robert's, does ICE with Robert, etc.,
and then get's Bob's and starts using Bob's because it's fresher.  Now
Robert accepts the connection and/or Bob rejects it; how does the rtcweb
browser revert to using Robert?

The rtc browser will update the connection with the answer it got in SIP 200
OK when Robert accepts the connection. There is nothing that prevents RTC
from doing update to the final accepted party. This is not ideal but works
most of the time.

Alternative is when the second answer is received, update the stream with
hold and start playing ring back locally. Once call is connected, update to
the final answer.
_____________
Roman Shpount

On Thu, Sep 22, 2011 at 6:55 PM, Hadriel Kaplan <[email-masked]>wrote:

> > I am not sure whether the JS app would be able to make better decisions
> than the gateway - and I've never seen a SIP client which would allow the
> user to choose which early media to listen to.
>
>
> But these aren't phone-call SIP clients - they're browsers.  They can show
> the multiple video streams in separate windows, for example.  They can show
> multiple chat sessions per fork too.  Or they may just pick the most recent
> one and only display it.  That's the nice thing about JS: the JS developer
> decides what the user experience model is like, based on what type of
> application/service they're providing; as opposed to the IETF/W3C choosing
> for them.
>
>
> > In my opinion the best thing would be if we could come up with some
> common BCP, e.g. saying that media associated with the early dialog on which
> the latest SDP answer was received is considered "active", and will be
> presented to the user etc.
>
> That's not the whole problem though.  Suppose it only forked to two agents:
> Robert and Bob.  Robert sent a 18x with SDP first and then Bob.  So the
> rtcweb browser starts off using Robert's, does ICE with Robert, etc., and
> then get's Bob's and starts using Bob's because it's fresher.  Now Robert
> accepts the connection and/or Bob rejects it; how does the rtcweb browser
> revert to using Robert?
>
> If the answer is "well the rtcweb browser had to keep state information
> about Robert's SDP/media info as well as about Bob's", then we're back to
> the tricky part about PeerConnection having multiple media peers when it
> only expected one; if we can solve that, then we've solved the basic issue
> at hand. (I think... or I need more coffee)
>
> -hadriel
>

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 23 Sep 2011 07:06:55 +0200

Hi, 

>>> The really hard part is knowing which stream to 
>>> use/render/send-to, imho.  And putting that decision in the 
>>> gateway isn't good - the best decider of that is probably the 
>>> JS in the browser.
>> 
>> I am not sure whether the JS app would be able to make 
>> better decisions than the gateway - and I've never seen a SIP 
>> client which would allow the user to choose which early media 
>> to listen to. 
> 
> 
> But these aren't phone-call SIP clients - they're browsers.  
> They can show the multiple video streams in separate windows, 
> for example.  They can show multiple chat sessions per fork 
> too.  Or they may just pick the most recent one and only 
> display it.  That's the nice thing about JS: the JS developer 
> decides what the user experience model is like, based on what 
> type of application/service they're providing; as opposed to 
> the IETF/W3C choosing for them.

Sure, but it would be good if network applications, that provide the early media, could make assumptions on how the end user is going to process it. It would make life easier both for the network application designer and the client application designer - AND the SBC designer, btw, if the SBC will have to make decisions on which media to forward towards the user :)

>> In my opinion the best thing would be if we could come up 
>> with some common BCP, e.g. saying that media associated with 
>> the early dialog on which the latest SDP answer was received 
>> is considered "active", and will be presented to the user etc. 
> 
> That's not the whole problem though.  Suppose it only forked 
> to two agents: Robert and Bob.  Robert sent a 18x with SDP 
> first and then Bob.  So the rtcweb browser starts off using 
> Robert's, does ICE with Robert, etc., and then get's Bob's 
> and starts using Bob's because it's fresher.  Now Robert 
> accepts the connection and/or Bob rejects it; how does the 
> rtcweb browser revert to using Robert?  
> 
> If the answer is "well the rtcweb browser had to keep state 
> information about Robert's SDP/media info as well as about 
> Bob's", then we're back to the tricky part about 
> PeerConnection having multiple media peers when it only 
> expected one; if we can solve that, then we've solved the 
> basic issue at hand. (I think... or I need more coffee)

The browser doesn't need to keep state - it only does what the JS app tells it to do. 

But, the JS app needs to keep state, so that it can then provide the SDP answer associated with Bob to the browser when the 200 OK comes.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 23 Sep 2011 07:09:27 +0200

Hi,	

>> That's not the whole problem though.  Suppose it only forked to two agents: Robert and Bob.  Robert sent a 18x with SDP first and then Bob.  So the rtcweb browser starts off using 
>>>Robert's, does ICE with Robert, etc., and then get's Bob's and starts using Bob's because it's fresher.  Now Robert accepts the connection and/or Bob rejects it; how does the rtcweb 
>The rtc browser will update the connection with the answer it got in SIP 200 OK when Robert accepts the connection. There is nothing that prevents RTC from doing update to the final accepted 
>party. This is not ideal but works most of the time. 

The 200 OK may not contain the answer, so the JS app would need to store it somewhere. But, the browser doesn't need to do it (unless the application is built into the browser, of course).

Regards,

Christer

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 23 Sep 2011 06:24:21 +0000

> Sure, but it would be good if network applications, that provide the early media, could make assumptions on how the end user is going to process it. It would make life easier both for the network a…

If the "network application" you're talking about is provided by the same domain that runs the rtcweb server, then obviously they can "control" it, since they wrote the JS script.

If you're talking about when rtcweb domain A calls SIP/rtcweb domain B, and domain B wants to control it, then obviously they can because domain B can use an SBC, or control the offer/answer and media on their side some similar way, and we don't need to say anything about that.

But that wasn't my point... the discussion was around whether the rtcweb system as a whole (browser or browser+JS) had to be able to support media forking cases, or not.  And my point was yes, they do, because if they talk SIP to domain B then they cannot *mandate* B not media fork calls, because that's like mandating B not send re-INVITEs.  It's part of basic SIP, part of its base spec.  Even from a practical standpoint it's not something imaginary like S/MIME; media forking happens in practice all the time.

Obviously my employer would be better off if SBCs were mandated for all SIP communications, but it's not right to require them in an IETF spec/architecture.  People should use them if they have a need to, not just because browsers are too lazy to handle real SIP scenarios. 

Imagine if Google deployed rtcweb, and Columbia University deployed a SIP service for all their students, with multiple contacts, forking, etc.  Columbia goes to Google and says "can we peer using SIP"?  And Google says "sure, but you'll have to deploy something to hide media forking, re-INVITEs, etc."  Columbia says "but I thought you guys peer with SIP?" and Google says "oh we do, but it's a lighter SIP granted to us by the IETF, because we use browsers".  Do you see how crazy that would be?

> But, the JS app needs to keep state, so that it can then provide the SDP answer associated with Bob to the browser when the 200 OK comes.

I'm with you there, but methinks we've lost that battle already.  The browser's going to be the SDP offer/answer engine, so now I'm at least hoping it's a real/full one and not some hybrid.

As a side note, I'm not actually sure what would happen to ICE processing with peers if the browser only has one media state at a given time, and switches from Robert's SDP to Bob's and back to Robert's.  Someone should try it with the chromium prototype, in different timing scenarios.  Like what happens if the browser's ICE to Robert gets to a completed state, then the browser switches to Bob, and later back to Robert and the browser thinks ICE starts again with Robert but Robert thinks it's over.

-hadriel

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 23 Sep 2011 09:08:31 +0200

> which could lead us down rather deep ratholes.

My intention was to show that the general functionality do work when
combing an RTCWEB end-point and a SIP end-point. Yes, you need some
functionality on the signaling plane to adopt the two worlds. I don't
think it is that complex. The SIP GW may in fact be the Webapp itself
that has a JS SIP stack and a bit of logic around that. It can also be
in the webserver, or third signalling GW node that are connected to the
webserver for the RTCWEB service. I think all of these cases works. Thus
no need for standardizing the gateway function.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 23 Sep 2011 11:47:29 +0200

On 09/23/11 07:06, Christer Holmberg wrote:
> Sure, but it would be good if network applications, that provide the early media, could make assumptions on how the end user is going to process it.
If the network application (whatever that is) delivered the JS to the 
end-user that does the presentation, the network application knows 
exactly how the end user is going to process it (modulo various forms of 
JS local override tricks that browsers and users play).

If the network application didn't, then we're dealing with a proposal 
for an application that may be implemented on top of RTCWEB, but in my 
opinion, we shouldn't encode that into the browser specs.

In other words - "all bets are off".

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 23 Sep 2011 12:50:14 +0200

Hi, 

>> Sure, but it would be good if network applications, that 
>> provide the early media, could make assumptions on how the end user is going to 
>> process it. It would make life easier both for the network 
>> application designer and the client application designer - AND the SBC 
>> designer, btw, if the SBC will have to make decisions on which media 
>> to forward towards the user :)
> 
> If the "network application" you're talking about is provided 
> by the same domain that runs the rtcweb server, then 
> obviously they can "control" it, since they wrote the JS script.
>
> If you're talking about when rtcweb domain A calls SIP/rtcweb 
> domain B, and domain B wants to control it, then obviously 
> they can because domain B can use an SBC, or control the 
> offer/answer and media on their side some similar way, and we 
> don't need to say anything about that.

Sure, you can control the offer/answer. My point was that it would be good if everyone had a common understanding on how the offer/answers will be controlled.

But, of course, as long as the JS app and the network application is provided by the same provider, things would work.

> But that wasn't my point... the discussion was around whether 
> the rtcweb system as a whole (browser or browser+JS) had to 
> be able to support media forking cases, or not.  And my point 
> was yes, they do, because if they talk SIP to domain B then 
> they cannot *mandate* B not media fork calls, because that's 
> like mandating B not send re-INVITEs.  It's part of basic 
> SIP, part of its base spec.  Even from a practical standpoint 
> it's not something imaginary like S/MIME; media forking 
> happens in practice all the time.

I agree that we somehow need to support forking, meaning that we on the signalling level need to be able to handle multiple early dialogs.

On the media plane, we also need to be able to switch from one remote media source to another, but the question is still whether the browser, and the PeerConnection API, needs to support *simultanous* media from multiple remote sources.

> Obviously my employer would be better off if SBCs were 
> mandated for all SIP communications, but it's not right to 
> require them in an IETF spec/architecture.  People should use 
> them if they have a need to, not just because browsers are 
> too lazy to handle real SIP scenarios. 
> 
> Imagine if Google deployed rtcweb, and Columbia University 
> deployed a SIP service for all their students, with multiple 
> contacts, forking, etc.  Columbia goes to Google and says 
> "can we peer using SIP"?  And Google says "sure, but you'll 
> have to deploy something to hide media forking, re-INVITEs, 
> etc."  Columbia says "but I thought you guys peer with SIP?" 
> and Google says "oh we do, but it's a lighter SIP granted to 
> us by the IETF, because we use browsers".  Do you see how 
> crazy that would be?

Yes, and I am not suggesting anything like that :)

>> But, the JS app needs to keep state, so that it can then 
>> provide the SDP answer associated with Bob to the browser 
>> when the 200 OK comes.
> 
> I'm with you there, but methinks we've lost that battle 
> already.  The browser's going to be the SDP offer/answer 
> engine, so now I'm at least hoping it's a real/full one and 
> not some hybrid.

I am not sure I understand what you mean by the browser being the SDP offer/answer engine.

Just because we would use offer/answer between the browser and the JS app, it doesn't mean we have to support multiple dialogs on that interface.

>As a side note, I'm not actually sure what would happen to 
>ICE processing with peers if the browser only has one media 
>state at a given time, and switches from Robert's SDP to 
>Bob's and back to Robert's.  Someone should try it with the 
>chromium prototype, in different timing scenarios.  Like what 
>happens if the browser's ICE to Robert gets to a completed 
>state, then the browser switches to Bob, and later back to 
>Robert and the browser thinks ICE starts again with Robert 
>but Robert thinks it's over.

ICE is one of the impacts that we would need to consider, because what might happen is that when you switch from one remote media source to another everything (ICE, media security etc) is lost.

...unless, of course, the gateway relays all media, and handles the switching between remote media sources.

Regards,

Christer

---

From: Randell Jesup <[email-masked]>
Date: Mon, 26 Sep 2011 01:23:51 -0400

On 9/22/2011 4:37 PM, Cullen Jennings wrote:
>> If so, what is your assumption then regarding ICE? That the SIP nodes 
>> will support ICE, or that the browser will be allowed to communicate 
>> with the SIP nodes without enabling ICE? 

I agree we can't solve the security issue with permission to send with the
current threat model without ICE or some equivalent.

There is another option that may help with some of the use cases (I've mentioned
this before in the discussion on screensharing, among others).  For a number
of the use cases security is an impassible problem with the current threat model.
Those use cases generally involve replacing cases where an existing desktop
install or plugin was used (webex, screensharing, vnc, SIP softclient, Skype, etc).
Those cases all currently involve the user implicitly giving these apps total
or close to code that could do pretty much anything on the user's computer,
and are also often the "ongoing usage" authentication cases.

The only mitigating safety of the external app/plugin model is that they're typically
signed and go through the platforms software-install procedure, cert-showing, UACs, etc.

Currently people are trying to work out the HTML5 "installed" webapp security model;
if that's far enough along we may be able to piggyback off that.   I'm looking into it.

-- 
Randell Jesup
[email-masked]