Subject: [Sframe] Negotiating SFrame/RTP in SDP

From: Richard Barnes <[email-masked]>
Date: Thu, 2 Sep 2021 16:23:12 -0400

Hi folks,

One of the major work items left around SFrame is the work to specify how
SFrame-encrypted media are carried in RTP, and how that is signaled in
SDP.  The finer points might need to be worked out in AVTCORE / MMUSIC, but
since this group knows the intricacies better, it seems like having some
agreement here on general approach would be good.

Sergio put together a first stab at this in
draft-murillo-avtcore-multi-codec-payload-format [1].  He presented this at
AVTCORE @ IETF 111 [2].  Unfortunately, the discussion there doesn't seem
to have provided a clear path forward, thus this thread.

Personally, I think the proposal in Sergio's draft is a little wide of the
mark.  ISTM that specifying a "generic" media format and recovering the
specificity in an RTP header is more complex than is needed. Worse, it sets
up implementations to send the same media both encrypted and unencrypted --
obviously not great for security.

I'd like to propose a simpler approach, drawing inspiration from cryptex
[3]:

a=sframe:<frame|packet> [<keying>]

Basically, just a new SDP attribute per m-line, that specifies that SFrame
encryption is being done and how.  The desired semantic is clear for
SPacket: Just add an SFrame protect/unprotect before you do other
processing on the media payload.  Then you will have a payload that is as
described in the SDP.

You can *almost* describe SFrame in this way as well.  Assemble all the RTP
packets in a frame, then do SFrame across all of their payloads (in
particular, adding the header to the first packet and the tag to the
last).  If you constrained the sender to construct pre-SFrame packets  that
were properly packetized according to the rest of the SDP, then this would
integrate cleanly with the rest of the SDP/RTP model.  The problem is that
that constraint is incompatible with Insertable Streams.  So we need to
decide how much we care about compatibility with IS as it is.  Personally,
it seems like WebRTC stacks and web apps are going to have to upgrade
*anyway* if they're going to support SDP-based negotiation of SFrame, so if
they have to make this one extra change, it shouldn't be a big deal.

How does this approach strike folks?

--Richard

[1]
https://www.ietf.org/archive/id/draft-murillo-avtcore-multi-codec-payload-format-01.html
[2] https://youtu.be/S-JR4Fa8VFY?t=3510
[3]
https://www.ietf.org/archive/id/draft-ietf-avtcore-cryptex-02.html#name-signaling

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 2 Sep 2021 14:11:31 -0700

On Thu, Sep 2, 2021 at 13:23 Richard Barnes <[email-masked]> wrote:

> encryption is being done and how.  The desired semantic is clear for SPacket
>

[BA] My assumption is that an a= line in the Offer means “I desire to send
encrypted media, and can also receive it”. Is that your understanding?  If
a corresponding a=line is provided in the Answer does this mean “I will
send encrypted media and will expect to receive encrypted media” so that
clear text media won’t be sent in either direction?  Is there an
expectation that all BUNDLED m lines have the same settings, or can they be
different?

The desired semantic is clear for SPacket: Just add an SFrame
> protect/unprotect

[BA] In the case of BUNDLED media, it may not be quite that simple (e.g.
you may have to figure out which packets are supposed to be protected
first).

The problem is that that constraint is incompatible with Insertable
> Streams.  So we need to decide how much we care about compatibility with IS
> as it is.
>

[BA] If we can get clarity on the architecture and the protocol as well as
the use cases, then the API(s) will follow. But requiring backwards
compatibility with existing Web APIs could be overly constraining (and
might yield a null solution set).

  Personally, it seems like WebRTC stacks and web apps are going to have to
> upgrade *anyway* if they're going to support SDP-based negotiation of
> SFrame, so if they have to make this one extra change, it shouldn't be a
> big deal.
>

[BA] I would like to suggest broadening the mindset beyond WebRTC.  One of
the major advantages of SFRAME is that it is transport agnostic, so that it
can be transported over WebRTC data channel, WebTransport, or even HTTPS.
As a result, WebRTC Encoded Transform is not the only relevant Web API and
W3C WEBRTC WG is not the only relevant W3C WG.

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 6 Sep 2021 21:14:00 +0200

>From my point of view, all the mechanisms described on the multi codec
payload format are required:

The ability to differentiate a RTP packet encrypted end to end (either by
SFrame or SPacket) is a must, both from a formal point of view about how
RTP works, but also to ensure that the media servers does not perform any
payload inspection that will cause malfunction (metadata gathering, layer
switching, recording..)

The optionality of the encryption also comes from various factors. First,
end to end encryption is an "endpoint" decision, the SDP negotiation with
the media server should not be the place to enforce it or not. Also, not
always the media server and the endpoints are provided by the same company
(i.e. CPaaS) and they will have to support both end to end encryption and
non end to end encryption calls. Note that it is quite a common practice in
SFUs to send an SDP offer from the media server to the endpoint, in which
case, the media server will have to offer e2e encryption as optional.

As an example, in Millicast we support both e2ee and non-e2ee streams today
(with the insertable stream per codec hacks), it is a completely optional
feature up to each customer to implement and to decide when and if it is
used. We are just carriers, and we do not want the customer to have to
configure if the stream is e2ee or not. We are not (and must not be)
involved in enforcing e2ee in any way.

Best regards
Sergio

On Thu, Sep 2, 2021 at 10:23 PM Richard Barnes <[email-masked]> wrote:

> You can *almost* describe SFrame in this way as well.  Assemble all the
> RTP packets in a frame, then do SFrame across all of their payloads (in
> --
> Sframe mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sframe
>

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 6 Sep 2021 16:39:30 -0700

Richard said:

"Just add an SFrame protect/unprotect before you do other processing on the
media payload.  Then you will have a payload that is as described in the
SDP."

[BA]  If the SFU is untrusted to have cleartext access to the payload, why
would the endpoint use SDP to negotiate with the SFU whether to use E2E
encryption?  This potentially enables a downgrade attack.

Such a negotiation only makes sense in a use case where there are some
clients that support E2E encryption and others that do not, and the SFU is
looking to gracefully keep the non-E2E endpoints out of the conference
(e.g. by failing the O/A negotiation, instead of succeeding and then having
the non-E2E endpoints being unable to parse the codec payloads).

Sergio said:

"Also, not always the media server and the endpoints are provided by the
same company (i.e. CPaaS) and they will have to support both end to end
encryption and non end to end encryption calls. "

[BA]  SDP negotiation between the endpoint and SFU is useful in negotiating
hop-by-hop parameters, such as the RTP header extensions needed by the SFU
to forward packets without inspecting the RTP payload.

But in a CPaaS scenario, I agree that the application will determine
whether to use E2E encryption or not and the SFU need not know or care,
since it isn't parsing the codec payloads anyway.  In that scenario you
won't have a mixture of E2E capable and non-capable endpoints, so the SDP
negotiation would serve no useful purpose.

On Mon, Sep 6, 2021 at 12:14 PM Sergio Garcia Murillo <
[email-masked]> wrote:

> As an example, in Millicast we support both e2ee and non-e2ee streams
> today (with the insertable stream per codec hacks), it is a
> completely optional feature up to each customer to implement and to decide
> when and if it is used. We are just carriers, and we do not want the
> customer to have to configure if the stream is e2ee or not. We are not (and
> must not be) involved in enforcing e2ee in any way.
>
> Best regards
> Sergio
>
>> Personally, I think the proposal in Sergio's draft is a little wide of
>> the mark.  ISTM that specifying a "generic" media format and recovering the
>> Basically, just a new SDP attribute per m-line, that specifies that
>> SFrame encryption is being done and how.  The desired semantic is clear for

---

From: Richard Barnes <[email-masked]>
Date: Thu, 9 Sep 2021 14:49:31 -0400

Hi Bernard,

On the question of why we're doing SDP here at all, the main reason that
comes to my mind is: Giving SFUs fair notice of what's going on so that
they can act intelligently on E2E-encrypted streams.  There might be some
secondary benefits to setting up endpoints.

It is worth noting that the current round of implementations (both IS-based
ones and Webex) demonstrate that E2EE calls can be done without any changes
to SDP, as long as SFUs don't get upset when they can't interpret the
media.  The potential benefits of having some standardization of signaling
would be (1) to let SFUs react more intelligently in these situations, say
by requesting different RTP extensions, and (2) simplifying some of the
endpoint configuration that is needed to make SFrame work (or at least
letting an endpoint fail faster, say if it doesn't have any keys for
SFrame).

With regard to downgrade: First, in some applications, there is a signaling
level above SDP (like SIP, or some HTTP thing for WebRTC), that is
independent of the SFU and could thus distribute an E2EE signal to allow
detection of downgrade.  Second, if all the infrastructure is adversarial,
then it's up to the clients to signal somehow whether a given call is E2EE.

--RLB

On Mon, Sep 6, 2021 at 7:39 PM Bernard Aboba <[email-masked]>
wrote:

> Richard said:
>
> "Just add an SFrame protect/unprotect before you do other processing on
> the media payload.  Then you will have a payload that is as described in
> the SDP."
>
> [BA]  SDP negotiation between the endpoint and SFU is useful in
> negotiating hop-by-hop parameters, such as the RTP header extensions needed
> by the SFU to forward packets without inspecting the RTP payload.
>
>>> One of the major work items left around SFrame is the work to specify
>>> how SFrame-encrypted media are carried in RTP, and how that is signaled in

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 9 Sep 2021 13:51:35 -0700

"On the question of why we're doing SDP here at all, the main reason that
comes to my mind is: Giving SFUs fair notice of what's going on so that
they can act intelligently on E2E-encrypted streams.  There might be some
secondary benefits to setting up endpoints.

media. "

[BA] By negotiating RTP header extensions designed to enable
payload-agnostic forwarding (e.g. framemarking, DD, etc.), the SFU can
obtain the information it needs to forward packets without having to parse
the payload.

The question is whether this is sufficient.

It seems to me that SDP can enable a compatible combination of RTP header
extension and codecs to be determined, but there can be complications.

For example, the endpoint might support framemarking with the H.264 codec
but not with VP8 or VP9, which are also supported. Unfortunately, the SFU
only supports forwarding VP8, VP9 and AV1 as well as the Dependency
Descriptor RTP header extension but not framemarking.

In this situation, O/A negotiation will result in the endpoint and SFU
negotiating VP8 or VP9 without a forwarding RTP header extension. If the
endpoint then attempts to encrypt E2E, the SFU will "get upset", because it
will attempt to parse the payload, having no other means available to
forward the packets.

If E2E encryption is an endpoint requirement, the endpoint can refuse to
proceed with the call.  That would probably be the appropriate outcome
since it knows that the SFU doesn't support the combination of codecs and
RTP header extensions necessary to make E2E encryption work.

Richard also said:

"The potential benefits of having some standardization of signaling would
be (1) to let SFUs react more intelligently in these situations, say by
requesting different RTP extensions, and (2) simplifying some of the
endpoint configuration that is needed to make SFrame work (or at least
letting an endpoint fail faster, say if it doesn't have any keys for
SFrame)."

[BA] SDP allows the SFU to tell the endpoint what codecs and RTP header
extensions it supports, and for the endpoint to do the same. So the basic
info can be exchanged in a single found of Offer/Answer.  Is it possible to
implement "Intelligence" at a higher layer, within the application?  That
would simplify things.

Richard also said:

[BA] Could the client "signal" be provided in the media?  Remember that
SFRAME isn't tied to RTP necessarily (e.g. SFRAMEs can be transmitted over
WebRTC data channel, WebTransport, etc.).  So you have a lot of freedom
here.

On Thu, Sep 9, 2021 at 11:49 AM Richard Barnes <[email-masked]> wrote:

> It is worth noting that the current round of implementations (both
> IS-based ones and Webex) demonstrate that E2EE calls can be done without
> any changes to SDP, as long as SFUs don't get upset when they can't
> interpret the media.  The potential benefits of having some standardization
> of signaling would be (1) to let SFUs react more intelligently in these
> situations, say by requesting different RTP extensions, and (2) simplifying
> some of the endpoint configuration that is needed to make SFrame work (or
> at least letting an endpoint fail faster, say if it doesn't have any keys
> for SFrame).
>
> With regard to downgrade: First, in some applications, there is a
> signaling level above SDP (like SIP, or some HTTP thing for WebRTC), that
> is independent of the SFU and could thus distribute an E2EE signal to allow
>> [BA]  If the SFU is untrusted to have cleartext access to the payload,
>> why would the endpoint use SDP to negotiate with the SFU whether to use E2E
>>> The ability to differentiate a RTP packet encrypted end to end (either
>>> by SFrame or SPacket) is a must, both from a formal point of view about how
>>> The optionality of the encryption also comes from various factors.
>>> First, end to end encryption is an "endpoint" decision, the SDP negotiation
>>> with the media server should not be the place to enforce it or not. Also,
>>> not always the media server and the endpoints are provided by the same
>>> company (i.e. CPaaS) and they will have to support both end to end
>>> encryption and non end to end encryption calls. Note that it is quite a
>>> common practice in SFUs to send an SDP offer from the media server to the
>>> endpoint, in which case, the media server will have to offer e2e encryption
>>> as optional.
>>>
>>>> I'd like to propose a simpler approach, drawing inspiration from
>>>> cryptex [3]:
>>>>