Subject: [Sframe] Status of the SFRAME WG

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 27 Aug 2021 21:40:15 -0700

Colleagues,

I'm concerned with what appears to be rather low energy in this working
group.  There's been almost no activity on the list since IETF 110, and it
looks like there's been little interest shown in an interim meeting despite
efforts by the chairs to re-establish momentum.  It also looks like some of
the work expected to be done in SFRAME is actually happening in AVTCORE.

Is there still work to be done here and the energy and interest to do it?
Should we propose moving the remaining document to AVTCORE or some other
venue?  It had a milestone of June 2021, but that's now past without
adopting any document to satisfy it.

-MSK, ART AD

---

From: Richard Barnes <[email-masked]>
Date: Sat, 28 Aug 2021 07:50:24 -1000

Hi Murray,

Thanks for checking in.  To be clear, work is still happening.  See the
SFrame vs SPacket discussion at AVTCORE at IETF 111.  There have been some
discussions among a few of us following on that, sporadically because of
summer holidays.  I’ll try to gather up some notes from that and start some
conversations on the mailing list.

My read on the state of things is:

1. We should probably go ahead and adopt draft-omara-sframe and
draft-barnes-sframe-mls.   There is shipping code using both of them, and
pretty broad agreement on the contents.  In fact, I don’t expect a ton of
changes needed before WGLC; but of course, other folks in the WG should
have their say.

2. We need to figure out what the story is with regard to carrying SFrame
in RTP and signaling it in SDP.  Sergio made a proposal in AVTCORE; I have
some different ideas that I need to write up for the group.  We should
probably have a virtual interim to get aligned on a path.

3. As background for that discussion, folks should be aware that the W3C
WebRTC WG is considering taking up work that would leverage the RTP/SDP
work, so there is a consumer lined up to add some shipping pressure.

https://lists.w3.org/Archives/Public/public-webrtc/2021Aug/0038.html

End of brain dump.  Interested if others have a different read.

Cheers,
—Richard

On Fri, Aug 27, 2021 at 18:40 Murray S. Kucherawy <[email-masked]>
wrote:

> --
> Sframe mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sframe
>

---

From: Richard Barnes <[email-masked]>
Date: Sat, 28 Aug 2021 07:53:15 -1000

Sorry, just realized I blew past the AVTCORE question.

This group does have work to do, in terms of setting the base encapsulation
format and keying (draft-omara and draft-barnes).

The details of the RTP/SDP stuff might be worked out in AVTCORE, but I
think that will work best if this group aligns on a starting point first.

—RLB

On Sat, Aug 28, 2021 at 07:50 Richard Barnes <[email-masked]> wrote:

>> Is there still work to be done here and the energy and interest to do
>> it?  Should we propose moving the remaining document to AVTCORE or some
>> other venue?  It had a milestone of June 2021, but that's now past without

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 28 Aug 2021 11:24:00 -0700

Murray --

Standards work often slows down in August as people take vacation. So it is
not surprising to experience difficulty in getting a WG meeting together in
early September.  In the past I've had to cancel WG interims scheduled in
August or early September, and just last week we had to extend a WGLC due
to lack of responses. So some of what you are seeing may be the general
slowdown in August.

Aside from that though, I do think there is reason to take another look at
the problem definition in SFRAME WG.  During the pandemic the disparate
worlds of streaming and realtime communications have begun to merge, with
the new category of "low latency streaming" emerging. We've seen developers
taking a fresh look at problems such as video ingestion (e.g. RUSH, WISH)
and low latency streaming (e.g. media over QUIC or WebRTC data channel).
In the new architectures that are emerging, some of the traditional
streaming architectural elements are being rethought. As one example,
protocols such as WISH enable endpoints to upload simulcast or scalable
video coding, rather than requiring a transcoding service.  Also, APIs such
as WebCodecs make it possible to decode and render video without
containerization (as had been required when using MSE API).

As these new ideas take shape in newly chartered WGs or BOFs, it may be
appropriate to consider whether the SFRAME WG is targeting the right
problems.  Traditionally, streaming architectures have focused on the
generalized transport of frames whereas RTP has forsaken generic
encapsulations, choosing instead to customize packetization to each codec.
As a result, at the IETF 111 AVTCORE meeting, Sergio presented SPacket,
which while having quite a bit in common with SFrame, has security services
occurring after packetization rather than before.

However, even if SFrame turns out not to be suitable for use with RTP, it
still may prove useful in the new architectures.  As an example, in a low
latency streaming use case requiring support for content protection (e.g.
sports, concerts, etc.) it may be desirable for the downstream leg to
bypass traditional content protection schemes based on containerization,
just as next generation ingestion protocols may make it possible to bypass
transcoding services.  Could SFRAME play a role here? It may be worth
considering.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 28 Aug 2021 13:46:53 -0700

Hi Bernard,

On Sat, Aug 28, 2021 at 11:24 AM Bernard Aboba <[email-masked]>
wrote:

> Murray --
>
> Standards work often slows down in August as people take vacation. So it
> is not surprising to experience difficulty in getting a WG meeting together
> in early September.  In the past I've had to cancel WG interims scheduled
> in August or early September, and just last week we had to extend a WGLC
> due to lack of responses. So some of what you are seeing may be the general
> slowdown in August.
>

Sure, if I were only talking about August/September.  But the last real
chatter on the WG list trailed off in March.

> transcoding services.  Could SFRAME play a role here? It may be worth

If there's a rechartering to be considered to (ahem) frame the working
group more appropriately, let's have that discussion.  I'm happy to support
whatever path will enable meaningful work as the output.  But in
particular, I'm interested in supporting the chairs to restore the WG's
momentum.

-MSK

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 28 Aug 2021 16:18:15 -0700

"But in particular, I'm interested in supporting the chairs to restore the
WG's momentum."

[BA] Momentum comes from having an interesting problem to solve, as well as
a path to solving it.

SFrame promises to provide efficient E2E security over arbitrary transports
(e.g. not just RTP).  If it can achieve those goals, it can be an important
part of the post-pandemic communications architecture.

However, there are some basic gaps in the story.

For example, how does a Web application decode and render an SFrame that is
received over an arbitrary transport (e.g. HTTPS, WebTransport, WebRTC data
channel, etc.)?

Can't pass the SFrame to the MSE API, because it's not containerized.

Can't decode the SFrame within WebCodecs API without decrypting it first
(which today would require Javascript to have possession of the key, and
also would make the decoded VideoFrame accessible to JS where it can be
arbitrarily modified).  If Javascript is untrusted (or if access to content
is to be restricted) then that is problematic. (See:
https://github.com/w3c/webcodecs/issues/41)

One of the challenges of the post-pandemic world (in which streaming and
RTC technologies are merging) is getting historically disparate protocols
and APIs to work together.