Subject: Re: [dispatch] Updated charter/work description for telepresencemulti-streams

From: Håkon Dahle <[email-masked]>
Date: 3 Jun 2010 02:01:35 +0200

Agree with Alex to a large extent in that  scalability, simulcast, publish/subscribe may all be applicable to the problem at hand.

However I do not think the Charter should jump to conclusions on what happens inside "the server" that Alex is referring to in his email - at this point I suggest leaving these details out of the charter, to make sure all options can be investigated.

-h

----- Reply message -----
From: "Alex Eleftheriadis" <[email-masked]>
Date: Wed, Jun 2, 2010 16:20
Subject: [dispatch] Updated charter/work description for telepresencemulti-streams
To: "Håkon Dahle" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>

Inline.

On Jun 2, 2010, at 9:51 PM, Håkon Dahle wrote:

A comment on the following sentence in the updated charter:

“This requires considering current widely deployed use cases, such as single and multiple screens, multi-point, Scalable Video Coding (SVC), as well as cases that are expected to be implemented using the protocol framework produced by this work item.  ”

Addressing use cases such as single and multiple screen systems, as well as both point-to-point and multi-point calling is great.  However I do not understand why a specific video codec or coding technique has been added to the same context.  The charter shouldn’t recommend or limit the work for any specific codec (it should be codec agnostic),  and I suggest the charter should be changed to comply with this.

Right. I believe that the correct reference is to alternative architectures for the server in both point-to-point and multi-point calls, where scalability (which includes simulcasting, pyramidal coding such as SVC, and multiple description coding) lends itself to VRUs (Video Routing Units) as opposed to MCUs. The VRU employs a "subscribe" model, made possible by scalability, and which works quite differently than an MCU.

--Alex

-h

________________________________
Fra: [email-masked]<mailto:[email-masked]> på vegne av Allyn Romanow (allyn)
Sendt: on 02.06.2010 18:11
Til: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Cullen Jennings (fluffy)
Emne: [dispatch] Updated charter/work description for telepresencemulti-streams
Folks,
Here is a version of the work description that addresses the comments made on the first draft.
The changes include:
1.       Charter section, qualification that we are working on SIP-based systems
2.       Charter section, minor word smithing
3.       Charter section, reference to RAI WG with relevant work
4.       Scope section, clarification of interoperability requirements
5.       Scope section, replacement of second paragraph regarding treatment of non-video non-audio media types
6.       Goals and Milestones – addition of a Problem Statement draft

Comments welcome

Allyn

Multi-streams for Telepresence Description of Work

Background
One branch of video conferencing has evolved that is focused on immersive “being there” experience.  Referred to in various ways such as virtual conferencing, telepresence or media spaces, early systems were mainly research projects or business systems with limited deployments.  In recent years telepresence systems have seen considerable market success.  Following the model of early systems, the first wave of commercial systems have been typically located in specially designed single-purpose rooms with multiple relatively large displays permitting life size image reproduction, multiple cameras, encoders, decoders and microphones.  These systems have several important characteristics that are different from more traditional video conferencing systems.

The first difference concerns controlling the visual viewpoint in order to improve participant nonverbal communication. These systems preserve essential group meeting characteristics such as eye contact, group gestures, seating order and spatial audio by carefully orchestrating the miking and camera angles at each of the sites . This is distinct from the more traditional approach where the geometric relationship between media streams is not used to preserve inter-stream communication aspects such as eye contact and group dynamics.

A second difference is manipulation of the environment to improve immersion.  With telepresence systems, cinematographic aspects of the local environment reproduction are carefully planned including color, table shape, seating and lighting so that when combined with large high quality displays, a strong sense of a "trompe l'oeil" or "being there" immersive experience is created.  Typical video conference systems do not include these considerations.

As telepresence video systems have become successful in the market, manufacturers have started exploring delivery of the nonverbal communication and immersive values of telepresence via smaller, less expensive and more flexible video conferencing systems for a variety of venues, such as individual offices, homes and kiosks. These are also  telepresence systems, since the audio and video quality is high enough to allow clear image reproduction for nonverbal communication, they are able to send and receive multiple media streams, and large coordinated multi image displays are available for immersive installations.   As the industry develops, the line between telepresence and video conferencing may become blurred as nonverbal communication and immersive installations become broadly available.

Problem
Although telepresence systems are based on open standards such as RTP, SIP, H.264, H.323 suite, they cannot easily interoperate with each other without operator assistance and expensive additional equipment that translates from one vendor to another. It would be like having to make sure all parties are on the same equipment (and network) when making a telephone call.  A major factor in the inability of Telepresence systems to work with each other is that there is no standard description of the multiple streams that comprise the media flows.

For example, in a multiple screen conference, the video and audio streams sent from remote participants must be understood by receivers so that they can be presented in a coherent and life-like manner. This includes the ability to present the remote participants at their true size for their apparent distance, while maintaining correct eye contact, gesticular cues, and simultaneously providing a spatial audio sound stage consistent with the video presentation.  The receiving device that decides how to display the incoming information needs to understand a number of variables such as the spatial position of the speaker, the field of view of the cameras, the camera zoom, which media stream is related to each of the displays, etc.

Charter
The Telepresence Multi-Streams work item in DISPATCH is chartered to define and specify for SIP-based systems the content of media multi-stream messages and the way these will be transported.

This work will provide a standard for the exchange of media semantic information that will foster interoperable end stations and conference bridges. It will specify  variables that describe the semantics of the media streams and the recommended behavior to achieve interoperability.

This requires considering current widely deployed use cases, such as single and multiple screens, multi-point, Scalable Video Coding (SVC), as well as cases that are expected to be implemented using the protocol framework produced by this work item.  The methodology for describing the variables must allow extensibility of the variables, since telepresence is still a young technology and may have use cases that are not currently considered.

The work item will identify use cases, define requirements, and define a method for describing and transporting information about multiple media streams, including a specification of variables required to support the use cases. This work item will consider the reuse of existing IETF protocols and produce an architecture/protocol framework document describing the protocols required to be implemented to support this functionality.  The document will identify any enhancements required to existing protocols as well as describing new protocol(s) for interoperable multi-streams negotiation that may be required.

Relevant work to be drawn upon has been done in XCON, MMUSIC, AVT, and FECframe.

Scope
The scope includes both systems that provide a fully immersive experience, and systems that interwork with them and therefore need to understand the same multiple stream semantics.

The focus of this work is on audio and video multiple streams.  Other media types may be considered, however development of methodologies for them is not within the scope of this work.

Interoperation with standards compliant systems is required, such as SIP-based video conferencing systems.  However, backwards compatibility with existing non-standards compliant telepresence systems is not required.

The group will produce
- Requirements and use cases

- Architectural Framework describing the protocols required to be implemented to support this functionality and identifying existing protocol enhancements and new protocol functionality required

- Specification of a new protocol to support telepresence multi-streams [if required]

Goals and Milestones
Nov 2010

Use Cases and Requirements to IESG as Informational RFC

Nov 2010
March 2011

Problem Statement to IESG as Informational RFC
Architecture to IESG as Informational RFC

March 2011

Revise Charter [IF new protocol is not required]

Nov 2011

Submit protocol draft to IESG as Proposed Standard RFC

_______________________________________________
dispatch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dispatch

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Wed, 2 Jun 2010 17:53:34 -0700

Hi Alex and Hakon, 

I think the intent here is to say that we want to cover current use cases and be extensible enough to cover new ones that may arise. I don't think it valuable to exhaustively enumerate relevant dimensions of use cases in this charter statement - that is better left to the use case document. 

How would it be to say,

"This requires considering current widely deployed use cases, as well as cases that are expected to be implemented using the protocol framework produced by this work item.  The methodology for describing the variables must allow extensibility of the variables, since telepresence is still a young technology and may have use cases that are not currently considered."

Allyn

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Håkon Dahle
Sent: Wednesday, June 02, 2010 5:02 PM
To: [email-masked]
Subject: Re: [dispatch] Updated charter/work description fortelepresencemulti-streams

-h

----- Reply message -----
From: "Alex Eleftheriadis" <[email-masked]>
Date: Wed, Jun 2, 2010 16:20
Subject: [dispatch] Updated charter/work description for telepresencemulti-streams
To: "Håkon Dahle" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>

Inline.

--Alex

-h

Comments welcome

Allyn

Background

Problem

Charter

Scope

The group will produce

Goals and Milestones
Nov 2010

Nov 2010
March 2011

March 2011

Nov 2011

[email-masked]

---

From: Alex Eleftheriadis <[email-masked]>
Date: Wed, 9 Jun 2010 03:10:15 -0400

Sure, that's fine. 

--Alex