Subject: [dispatch] Updated Telepresence charter

From: Mary Barnes <[email-masked]>
Date: Thu, 19 Aug 2010 11:41:47 -0500

Per the discussion at IETF-78, an action was taken to update the
telepresence charter to be more specific and more tightly scoped.  To that
end, the updated charter is below.

Please review and provide any additional feedback and comments to the list.
In addition, if you think the charter is ready to move forward in terms of
creating a new WG, please respond to the mailing list.  In the latter case,
if you could also please let us know (offlist if you'd prefer) if you are
willing to technically contribute to this work,  we can guage whether there
will be sufficient contributors to support a WG.

Best Regards,
Mary.
DISPATCH WG co-chair

================================================================

CLUE ControLling mUltiple streams for TElepresence
or
MUSCAT: MUlti-Stream Control Applied to Telepresence

In the context of this WG, the term telepresence is used in a general manner
to describe systems that provide high definition, high quality audio
enabling a "being-there" experience.  One example is an immersive
telepresence system using specially designed and special purpose rooms with
multiple displays permitting life size image reproduction using multiple
cameras, encoders, decoders and microphones.

Current telepresence systems are based on open standards such as RTP, SIP,
H.264, the H.323 suite, however, they cannot easily interoperate with each
other without operator assistance and expensive additional equipment which
translates from one vendor to another. A major factor in the inability of
telepresence systems to interwork is that there is no standard description
of the multiple streams of audio and video that comprise the media flows.
 In addition,  there is  no standardized way  to exchange semantic
information about what each media stream represents.

The WG will create specification to enable communication of enough
information about each media stream so that each receiving system or bridge
system can make reasonable decisions about selecting and displaying media
streams. This enables systems to make display choices that optimize the
"just like being there" experience.

This working group is chartered to specify the information about media
streams from one endpoint to another endpoint or conference bridge
including:

* Spatial relationships of cameras, displays, microphones, and
  speakers in relation to each other and to likely positions of
  participants

* Specific characteristics such as viewpoint, field of view/capture
  for camera/microphone/display/speaker so that senders and
  middleboxes can understand how best to compose streams for
  receivers and the receivers will know the characteristics of its
  received streams

* Aspect ratio of cameras and displays

Information between sources and sinks about media stream capabilities will
be exchanged.

The working group will define the semantics, language and transport
mechanism necessary for communicating the necessary information. It will
consider whether the existing signaling mechanisms (SDP, BFCP) can be
extended, or another messaging method should be used.

The scope of the work includes describing relatively static relations
between entities (participants and devices). It also includes handling more
dynamic relationships, such as identifying the audio and video streams for
the current speaker. The scope includes both systems that provide a fully
immersive experience, and systems that interwork with them and therefore
need to understand the same multiple stream semantics.

The focus of this work is on multiple audio and video streams.  Other media
types may be considered, however development of methodologies for them is
not within the scope of this work.

Interoperation with standards compliant systems is required, such as
SIP-based video conferencing systems.  However, backwards compatibility with
existing non-standards compliant telepresence systems is not required.

This working group is not currently chartered to work on issues of
continuous conference control including: far end camera control, indication
of fast frame update for video codecs or other rapid switches, floor
control, conference roster. Also not in scope is signaling to tell the
originator to which receivers the media streams should be sent.

Reuse of existing protocols and backwards compatibility with existing
systems is an important factor for the working group to consider. The work
will closely coordinate with the appropriate areas and working groups
including OPS Area, AVT, MMUSIC, MEDIACTRL, XCON, and SIPCORE.

 Milestones

 Nov 2010 Submit information draft to IESG on use cases and requirements

 Nov 2011 Submit standards track specification to IESG on indicating
          spatial relationships of screens, cameras (including variable
          field of view and orientation), speakers and microphones. This
          includes, semantics, language and transport.

 Apr 2011 Submit standards track specification to IESG on indicating the
          "usage" of a stream as define in charter.

---

From: Peter Musgrave <[email-masked]>
Date: Thu, 19 Aug 2010 13:13:07 -0400

Hi,

I have no issues with the charter as posed.

I am willing to make technical contributions to this work.

Cheers,

Peter Musgrave

On Thu, Aug 19, 2010 at 12:41 PM, Mary Barnes
<[email-masked]> wrote:
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch
>
>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 19 Aug 2010 13:23:38 -0400

This seems good, but I've indicated a few nits inline.

	Thanks,
	Paul

Mary Barnes wrote:
> telepresence charter to be more specific and more tightly scoped.  To 
> that end, the updated charter is below.  
> 
> Please review and provide any additional feedback and comments to the 
> list. In addition, if you think the charter is ready to move forward in 
> terms of creating a new WG, please respond to the mailing list.  In the 
> latter case, if you could also please let us know (offlist if you'd 
> prefer) if you are willing to technically contribute to this work,  we 
> can guage whether there will be sufficient contributors to support a WG. 
> 
> Best Regards,
> Mary.
> DISPATCH WG co-chair
> 
> In the context of this WG, the term telepresence is used in a general 
> manner to describe systems that provide high definition, high quality 
> audio enabling a "being-there" experience.  One example is an immersive 

*audio*???

s:audio:video: or s:audio:audio/video:

> telepresence system using specially designed and special purpose rooms 
> with multiple displays permitting life size image reproduction using 
> multiple cameras, encoders, decoders and microphones. 
> 
> Current telepresence systems are based on open standards such as RTP, 
> SIP, H.264, the H.323 suite, however, they cannot easily interoperate 
> with each other without operator assistance and expensive additional 

s/with each other//
(implied by "interoperate")

> equipment which translates from one vendor to another. A major factor in 
> the inability of telepresence systems to interwork is that there is no 
> standard description of the multiple streams of audio and video that 
> comprise the media flows.  In addition,  there is  no standardized way 
>  to exchange semantic information about what each media stream represents.  
> 
> information about each media stream so that each receiving system or 
> bridge system can make reasonable decisions about selecting and 
> displaying media streams. This enables systems to make display choices 

s/displaying/rendering/
(both audio and video being discussed, but you don't *display* audio.)

> that optimize the "just like being there" experience. 
> 
> Information between sources and sinks about media stream capabilities 
> will be exchanged. 
> 

I don't think SDP and BFCP are the only existing mechanisms. So

s/(/(e.g. /

> between entities (participants and devices). It also includes handling 
> more dynamic relationships, such as identifying the audio and video 
> streams for the current speaker. The scope includes both systems that 
> provide a fully immersive experience, and systems that interwork with 
> them and therefore need to understand the same multiple stream semantics.  
> 
> The focus of this work is on multiple audio and video streams.  Other 
> media types may be considered, however development of methodologies for 
> them is not within the scope of this work.
> 
> SIP-based video conferencing systems.  However, backwards compatibility 
> with existing non-standards compliant telepresence systems is not required.
> 
> continuous conference control including: far end camera control, 
> indication of fast frame update for video codecs or other rapid 
> switches, floor control, conference roster. Also not in scope is 
> signaling to tell the originator to which receivers the media streams 
> should be sent.
> 
> systems is an important factor for the working group to consider. The 
> work will closely coordinate with the appropriate areas and working 
> groups including OPS Area, AVT, MMUSIC, MEDIACTRL, XCON, and SIPCORE.
> 
>  Milestones  
> 
> ------------------------------------------------------------------------
>

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 19 Aug 2010 14:04:18 -0400

These changes all WFM.

Marshall

On Aug 19, 2010, at 1:23 PM, Paul Kyzivat wrote:

>> Per the discussion at IETF-78, an action was taken to update the telepresence charter to be more specific and more tightly scoped.  To that end, the updated charter is below.  Please review and provide any additional feedback and comments to the list. In addition, if you think the charter is ready to move forward in terms of creating a new WG, please respond to the mailing list.  In the latter case, if you could also please let us know (offlist if you'd prefer) if you are willing to technically contribute to this work,  we can guage whether there will be sufficient contributors to support a WG. Best Regards,
>> Mary.
>> DISPATCH WG co-chair
>> In the context of this WG, the term telepresence is used in a general manner to describe systems that provide high definition, high quality audio enabling a "being-there" experience.  One example is an immersive 
> 
> *audio*???
> 
>> telepresence system using specially designed and special purpose rooms with multiple displays permitting life size image reproduction using multiple cameras, encoders, decoders and microphones. Current telepresence systems are based on open standards such as RTP, SIP, H.264, the H.323 suite, however, they cannot easily interoperate with each other without operator assistance and expensive additional 
> 
> s/with each other//
>> equipment which translates from one vendor to another. A major factor in the inability of telepresence systems to interwork is that there is no standard description of the multiple streams of audio and video that comprise the media flows.  In addition,  there is  no standardized way  to exchange semantic information about what each media stream represents.  The WG will create specification to enable communication of enough information about each media stream so that each receiving system or bridge system can make reasonable decisions about selecting and displaying media streams. This enables systems to make display choices 
> 
> s/displaying/rendering/
>> that optimize the "just like being there" experience. This working group is chartered to specify the information about media streams from one endpoint to another endpoint or conference bridge including:
>>  for camera/microphone/display/speaker so that senders and                 middleboxes can understand how best to compose streams for
>>  receivers and the receivers will know the characteristics of its   received streams
>> Information between sources and sinks about media stream capabilities will be exchanged. The working group will define the semantics, language and transport mechanism necessary for communicating the necessary information. It will consider whether the existing signaling mechanisms (SDP, BFCP) can be 
> 
>> extended, or another messaging method should be used.  The scope of the work includes describing relatively static relations between entities (participants and devices). It also includes handling more dynamic relationships, such as identifying the audio and video streams for the current speaker. The scope includes both systems that provide a fully immersive experience, and systems that interwork with them and therefore need to understand the same multiple stream semantics.  The focus of this work is on multiple audio and video streams.  Other media types may be considered, however development of methodologies for them is not within the scope of this work.
>> Interoperation with standards compliant systems is required, such as SIP-based video conferencing systems.  However, backwards compatibility with existing non-standards compliant telepresence systems is not required.
>> This working group is not currently chartered to work on issues of continuous conference control including: far end camera control, indication of fast frame update for video codecs or other rapid switches, floor control, conference roster. Also not in scope is signaling to tell the originator to which receivers the media streams should be sent.
>> Reuse of existing protocols and backwards compatibility with existing systems is an important factor for the working group to consider. The work will closely coordinate with the appropriate areas and working groups including OPS Area, AVT, MMUSIC, MEDIACTRL, XCON, and SIPCORE.
>> Milestones   Nov 2010 Submit information draft to IESG on use cases and requirements

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Thu, 19 Aug 2010 12:03:25 -0700

Looks good to me, with Paul's improvements.

Allyn 

From: [email-masked] [mailto:[email-masked]] On
Behalf Of Mary Barnes
Sent: Thursday, August 19, 2010 9:42 AM
To: DISPATCH
Subject: [dispatch] Updated Telepresence charter

 

Best Regards,

Mary.

DISPATCH WG co-chair

or

 

to exchange semantic information about what each media stream
represents.  

  participants

  received streams

them and therefore need to understand the same multiple stream
semantics.  

Interoperation with standards compliant systems is required, such as
SIP-based video conferencing systems.  However, backwards compatibility
with existing non-standards compliant telepresence systems is not
required.

 Milestones

---

From: Alex Eleftheriadis <[email-masked]>
Date: Fri, 20 Aug 2010 05:27:27 -0400

It looks good, and I will be happy to contribute. I only have a clarification question.

Can you elaborate on the sentence: "Also not in scope is signaling to tell the originator to which receivers the media streams should be sent." Specifically, is it within scope for a single receiver to tell the originator which of the multiple streams should be sent to itself?

--Alex

On Aug 19, 2010, at 10:03 PM, Allyn Romanow (allyn) wrote:

Looks good to me, with Paul’s improvements.

Allyn

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Mary Barnes
Sent: Thursday, August 19, 2010 9:42 AM
To: DISPATCH
Subject: [dispatch] Updated Telepresence charter

Per the discussion at IETF-78, an action was taken to update the telepresence charter to be more specific and more tightly scoped.  To that end, the updated charter is below.

Please review and provide any additional feedback and comments to the list. In addition, if you think the charter is ready to move forward in terms of creating a new WG, please respond to the mailing list.  In the latter case, if you could also please let us know (offlist if you'd prefer) if you are willing to technically contribute to this work,  we can guage whether there will be sufficient contributors to support a WG.

Best Regards,
Mary.
DISPATCH WG co-chair

In the context of this WG, the term telepresence is used in a general manner to describe systems that provide high definition, high quality audio enabling a "being-there" experience.  One example is an immersive telepresence system using specially designed and special purpose rooms with multiple displays permitting life size image reproduction using multiple cameras, encoders, decoders and microphones.

Current telepresence systems are based on open standards such as RTP, SIP, H.264, the H.323 suite, however, they cannot easily interoperate with each other without operator assistance and expensive additional equipment which translates from one vendor to another. A major factor in the inability of telepresence systems to interwork is that there is no standard description of the multiple streams of audio and video that comprise the media flows.  In addition,  there is  no standardized way  to exchange semantic information about what each media stream represents.

The WG will create specification to enable communication of enough information about each media stream so that each receiving system or bridge system can make reasonable decisions about selecting and displaying media streams. This enables systems to make display choices that optimize the "just like being there" experience.

This working group is chartered to specify the information about media streams from one endpoint to another endpoint or conference bridge including:

Information between sources and sinks about media stream capabilities will be exchanged.

The working group will define the semantics, language and transport mechanism necessary for communicating the necessary information. It will consider whether the existing signaling mechanisms (SDP, BFCP) can be extended, or another messaging method should be used.

The scope of the work includes describing relatively static relations between entities (participants and devices). It also includes handling more dynamic relationships, such as identifying the audio and video streams for the current speaker. The scope includes both systems that provide a fully immersive experience, and systems that interwork with them and therefore need to understand the same multiple stream semantics.

The focus of this work is on multiple audio and video streams.  Other media types may be considered, however development of methodologies for them is not within the scope of this work.

 Milestones

_______________________________________________
dispatch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dispatch

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 20 Aug 2010 06:24:38 -0400

> Can you elaborate on the sentence: "Also not in scope is signaling to tell the originator to which receivers the media streams should be sent." Specifically, is it within scope for a single receive…

I interpret this as meaning 

- there are existing, adequate, means to signal the existence and location of receivers to senders, so such signaling is out of scope.

There are _not_ existing, adequate, means to tell which streams of a set of multiple streams should be sent where, so IMO the answer to your question is yes, such signaling is within scope. 

If we want to tweak that text, here is a suggestion :

As there are existing, adequate, means to signal the existence and location of receivers to senders, such signaling is also not in scope.

Regards
Marshall 

>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Mary Barnes

---

From: stephen botzko <[email-masked]>
Date: Fri, 20 Aug 2010 06:35:08 -0400

Specifically, is it within scope for a single receiver to tell the
originator which of the multiple streams should be sent to itself?
I think this needs to be in scope.

It looks to me like the intent of the sentence is to exclude signaling
identifying which receivers are in the conference.  IMHO it should not
exclude signaling on which media streams should be sent to a given receiver.

--Stephen Botzko

On Fri, Aug 20, 2010 at 5:27 AM, Alex Eleftheriadis <[email-masked]> wrote:

> Can you elaborate on the sentence: "Also not in scope is signaling to tell
> the originator to which receivers the media streams should be sent."
> Specifically, is it within scope for a single receiver to tell the
> originator which of the multiple streams should be sent to itself?
>
> --Alex
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Mary Barnes
> *Sent:* Thursday, August 19, 2010 9:42 AM
> *To:* DISPATCH
> *Subject:* [dispatch] Updated Telepresence charter
>
> manner to describe systems that provide high definition, high quality audio

---

From: Peter Musgrave <[email-masked]>
Date: Fri, 20 Aug 2010 06:42:05 -0400

Not to go too far down the rat-hole here - but I would assume this
could be done simply with a reINVITE with new SDP which indicated what
IP:port the stream should go to (and sending inactive to the streams
it does not want to receive).

I do not see this as requiring specific work from the new WG -
existing SIP signalling will meet this need.

Peter Musgrave

On Fri, Aug 20, 2010 at 6:35 AM, stephen botzko
<[email-masked]> wrote:
>> latter case, if you could also please let us know (offlist if you'd prefer)
>> if you are willing to technically contribute to this work,  we can guage
>> whether there will be sufficient contributors to support a WG.
>>
>> Best Regards,
>> Mary.
>> DISPATCH WG co-chair
>>
>> media types may be considered, however development of methodologies for them
>> is not within the scope of this work.
>>

---

From: stephen botzko <[email-masked]>
Date: Fri, 20 Aug 2010 07:03:20 -0400

What if the receiver wants to tell the sender that it should receive the
video stream of whomever is speaking?

The desired behavior is that if a 3-screen system is sending video stream X,
and someone at that site speaks (but is sent on Video Stream Y), then video
stream Y would automatically be forwarded to the receiver instead of X.

That cannot be done easily with a reINVITE (at least not w/o incurring at
least a RTT delay).

Another complication with this scenario - even if you are willing to accept
the RTT delay, in the multipoint case the various receivers will be doing
the reINVITE at different times.  Each reINVITE forces an intra video
refresh, which will result in visible artifacts for all receivers for
several seconds.

-- Stephen Botzko

On Fri, Aug 20, 2010 at 6:42 AM, Peter Musgrave <
[email-masked]> wrote:

> prefer)
> audio
> with
> SIP,
> bridge
> will
> more
> with
> indication
> work

---

From: Peter Musgrave <[email-masked]>
Date: Fri, 20 Aug 2010 07:17:21 -0400

Hi Stephen,

Sorry, I misunderstood your use case.

If this is a call configuration issue (i.e. at negotiation RX intends
"please send video of loudest speaker to me at IP:port" then I think
this becomes an out-of-band meta-info issue (and not the SDP mechanism
I had suggested).

In this case I think the part of the charter that refers to other RAI
groups covers the work e.g. a SIP TP event package might be defined.

I'm ok if the charter is changed to state this case explicitly - but
I'd be surprised if there was not a way to do within the existing
protocols. I'd suggest defining the need and assuming it can be done
within the protocols.

Peter Musgrave

On Fri, Aug 20, 2010 at 7:03 AM, stephen botzko
<[email-masked]> wrote:
> On Fri, Aug 20, 2010 at 6:42 AM, Peter Musgrave
> <[email-masked]> wrote:
>>

---

From: Peter Musgrave <[email-masked]>
Date: Fri, 20 Aug 2010 07:17:37 -0400

Hi Stephen,

I'm ok if the charter is changed to state this case explicitly - but
I'd be surprised if there was not a way to do within the existing
protocols. I'd suggest defining the need and assuming it can be done
within the protocols [and if we're wrong we'll need to a

Peter Musgrave

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Fri, 20 Aug 2010 09:02:05 -0500

Perhaps it should just say to not duplicate the work of SDP.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Peter Musgrave
Sent: Friday, August 20, 2010 7:17 AM
To: stephen botzko
Cc: DISPATCH
Subject: Re: [dispatch] Updated Telepresence charter

Hi Stephen,

Peter Musgrave

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Fri, 20 Aug 2010 08:44:04 -0700

Hi Mary,

I agree with Paul's comments, and added a few minor suggestions as well
(inline). I think the charter is ready to move forward in terms of
creating a new WG, and I am looking forward to contributing to this
work.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Paul Kyzivat
> (pkyzivat)
> Sent: Thursday, August 19, 2010 10:24 AM
> To: Mary Barnes
> Cc: DISPATCH

Add speakers.

> > standard description of the multiple streams of audio and video that

s/standard description/standardized way to describe and negotiate the
use/

represents.
> > The working group will define the semantics, language and transport

s/language/syntax

semantics.
required.