Subject: Re: [dispatch] Multi-streams for telepresence description of work

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Wed, 2 Jun 2010 09:15:41 -0700

Hi Gao,

I had a chance to talk with Mary about your idea to split up the work, and she confirmed that it is good practice in general to do so, but we thought it is as yet too early to split up the multi-stream work.  As we progress with use cases, problem statement and requirements, we will be able to see better if the topic should be split up.

Best regards,

Allyn

From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, May 18, 2010 10:33 PM
To: Allyn Romanow (allyn)
Cc: IETF DISPATCH list
Subject: Re: [dispatch] Multi-streams for telepresence description of work

Hi, 

I have two level of comments, 

1. Feeling for the topic 
Technically, I am quit glad to see such discussion about telepresence, which is trailblazer for VR(Virtual Reality) production. 

2. About the charter WG 
One revolutionary application, such telepresence, would cover basic protocol level, application control level, and interworking issue. 

If the work is just focused on pure interworking issue, then the topics can be divided into smaller one, which might be in other basic protocols' WG's scope. But if the application itself is new, and has some new characteristic, more work should be proposed. 

If the work is also focused on telepresence application, which is a conferencing system, the xcon WG might be proper one for application level definition. 

If the charter also want to talk the esential aspect of telepresence(or more abstractly, about VR), such as how to using SDP, or relative protocol to describe the streams and it's collaboration, it would be quite new topic. One new WG might be better. 

I find the charter seems covers all the three thing. So, I suggest you classify such different level requirement. Then, we may find which part is proper for which WG, and which part should be covered by new WG. 

Thanks, 

Gao 

===================================
Zip    : 210012
Tel    : 87211
Tel2   :(+86)-025-52877211
e_mail : [email-masked]
=================================== 

"Allyn Romanow (allyn)" <[email-masked]> 
发件人:  [email-masked] 

2010-05-19 02:30 

收件人

"IETF DISPATCH list" <[email-masked]> 

抄送

主题

[dispatch] Multi-streams for telepresence description of work

Hi Folks, 
  
Many of you are aware of telepresence systems (video conferencing on steroids..:), fewer probably are aware of the fact that the current systems are not interoperable with each other. After talking with Mary and Cullen, we have a rough draft of a charter to work on this problem in DISPATCH,  for discussion on the mailing list. 
  
We don’t yet know whether a working group would need to be spun out or not – that will be clarified as we go. 
  
We think there is enough background info in the work description to start a discussion, and plan to send out use cases for discussion shortly. 
  
Who is the we? Several makers and providers of telepresence systems have gotten together to define the main problem in interoperability – the handling of multi-streams. 
  
Thanks for your thoughts- 
Allyn 

Multi-streams for Telepresence Description of Work 

Background 
One branch of video conferencing has evolved that is focused on immersive “being there” experience.  Referred to in various ways such as virtual conferencing, telepresence or media spaces, early systems were mainly research projects or business systems with limited deployments.  In recent years telepresence systems have seen considerable market success.  Following the model of early systems, the first wave of commercial systems have been typically located in specially designed single-purpose rooms with multiple relatively large displays permitting life size image reproduction, multiple cameras, encoders, decoders and microphones.  These systems have several important characteristics that are different from more traditional video conferencing systems.   
  
The first difference concerns controlling the visual viewpoint in order to improve participant nonverbal communication. These systems preserve essential group meeting characteristics such as eye contact, group gestures, seating order and spatial audio by carefully orchestrating the miking and camera angles at each of the sites . This is distinct from the more traditional approach where the geometric relationship between media streams is not used to preserve inter-stream communication aspects such as eye contact and group dynamics.   
  
A second difference is manipulation of the environment to improve immersion.  With telepresence systems, cinematographic aspects of the local environment reproduction are carefully planned including color, table shape, seating and lighting so that when combined with large high quality displays, a strong sense of a "trompe l'oeil" or "being there" immersive experience is created.  Typical video conference systems do not include these considerations. 
  
As telepresence video systems have become successful in the market, manufacturers have started exploring delivery of the nonverbal communication and immersive values of telepresence via smaller, less expensive and more flexible video conferencing systems for a variety of venues, such as individual offices, homes and kiosks. These are also  telepresence systems, since the audio and video quality is high enough to allow clear image reproduction for nonverbal communication, they are able to send and receive multiple media streams, and large coordinated multi image displays are available for immersive installations.   As the industry develops, the line between telepresence and video conferencing may become blurred as nonverbal communication and immersive installations become broadly available. 
  
Problem 
Although telepresence systems are based on open standards such as RTP, SIP, H.264, H.323 suite, they cannot easily interoperate with each other without operator assistance and expensive additional equipment that translates from one vendor to another. It would be like having to make sure all parties are on the same equipment (and network) when making a telephone call.  A major factor in the inability of Telepresence systems to work with each other is that there is no standard description of the multiple streams that comprise the media flows. 
  
For example, in a multiple screen conference, the video and audio streams sent from remote participants must be understood by receivers so that they can be presented in a coherent and life-like manner. This includes the ability to present the remote participants at their true size for their apparent distance, while maintaining correct eye contact, gesticular cues, and simultaneously providing a spatial audio sound stage consistent with the video presentation.  The receiving device that decides how to display the incoming information needs to understand a number of variables such as the spatial position of the speaker, the field of view of the cameras, the camera zoom, which media stream is related to each of the displays, etc. 
  
Charter 
The Telepresence Multi-Streams work item in DISPATCH is chartered to define and specify the content of media multi-stream messages and the way these will be transported. 
This work is aimed at providing a standard for the exchange of media semantics information that will foster interoperable end stations and conference bridges. This work item will specify the variables that describe the semantics of the media streams and the recommended behavior to achieve interoperability.   
  
This requires considering current widely deployed use cases, such as single and multiple screens, multi-point, Scalable Video Coding (SVC), as well as cases that are expected to be implemented using the protocol framework produced by this work item.  The methodology for describing the variables must allow extensibility of the variables, since telepresence is still a young technology and may have use cases that are not currently considered. 
  
The work item will identify use cases, define requirements, and define a method for describing and transporting information about multiple media streams, including a specification of variables required to support the use cases. This work item will consider the reuse of existing IETF protocols and produce an architecture/protocol framework document describing the protocols required to be implemented to support this functionality.  The document will identify any enhancements required to existing protocols as well as describing new protocol(s) for interoperable multi-streams negotiation that may be required. 
  
Scope 
The scope includes both systems that provide a fully immersive experience, and systems that interwork with them and therefore need to understand the same multiple stream semantics. 
  
The focus of this work is on audio and video multiple streams, however other media types may be considered.  Definition of new application protocols is not within the scope of this work 
  
[Is there anything else we need to explicitly say is out of scope?] 
  
The group will produce 
- Requirements and use cases 
  
- Architectural Framework describing the protocols required to be implemented to support this functionality and identifying existing protocol enhancements and new protocol functionality required 
  
- Specification of a new protocol to support telepresence multi-streams [if required] 
  
Goals and Milestones 

Nov 2010 

Use Cases and Requirements to IESG as Informational RFC 

March 2011 

Architecture to IESG as Informational RFC 

March 2011 

Revise Charter [IF new protocol is not required] 

Nov 2011 

Submit protocol draft to IESG as Proposed Standard RFC 

 _______________________________________________
dispatch mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dispatch

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is solely property of the sender's organization. This mail communication is confidential. Recipients named above are obligated to maintain secrecy and are not permitted to disclose the contents of this communication to others.
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the originator of the message. Any views expressed in this message are those of the individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam system.

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 2 Jun 2010 12:59:52 -0600

On May 19, 2010, at 5:19 AM, stephen botzko wrote:

> Allyn and I co-chair the IMTC multi-streaming group.  The IMTC would be delighted if the IETF takes on this work item.

And at some time in the past, the RAI ADs told IMTC that they would be delighted to see the IMTC bring this type of work to the IETF.

---

From: [email-masked]
Date: Thu, 3 Jun 2010 09:06:35 +0800

Hi Allyn,

Sure. We can do the spliting evaluation later as you said.

Cheers,

Gao

2010-06-03 00:15

收件人
<[email-masked]>
抄送
"IETF DISPATCH list" <[email-masked]>
主题
RE: [dispatch] Multi-streams for telepresence description of work

Hi Gao,
 
I had a chance to talk with Mary about your idea to split up the work, and 
she confirmed that it is good practice in general to do so, but we thought 
it is as yet too early to split up the multi-stream work.  As we progress 
with use cases, problem statement and requirements, we will be able to see 
better if the topic should be split up.
 
Best regards,
Allyn

Hi, 

1. Feeling for the topic 
Technically, I am quit glad to see such discussion about telepresence, 
which is trailblazer for VR(Virtual Reality) production. 

2. About the charter WG 
One revolutionary application, such telepresence, would cover basic 
protocol level, application control level, and interworking issue. 

If the work is just focused on pure interworking issue, then the topics 
can be divided into smaller one, which might be in other basic protocols' 
WG's scope. But if the application itself is new, and has some new 
characteristic, more work should be proposed. 

If the work is also focused on telepresence application, which is a 
conferencing system, the xcon WG might be proper one for application level 
definition. 

If the charter also want to talk the esential aspect of telepresence(or 
more abstractly, about VR), such as how to using SDP, or relative protocol 
to describe the streams and it's collaboration, it would be quite new 
topic. One new WG might be better. 

I find the charter seems covers all the three thing. So, I suggest you 
classify such different level requirement. Then, we may find which part is 
proper for which WG, and which part should be covered by new WG. 

Thanks, 

Gao 

收件人

主题

Hi Folks, 
  
Many of you are aware of telepresence systems (video conferencing on 
steroids..:), fewer probably are aware of the fact that the current 
systems are not interoperable with each other. After talking with Mary and 
Cullen, we have a rough draft of a charter to work on this problem in 
DISPATCH,  for discussion on the mailing list. 
  
We don’t yet know whether a working group would need to be spun out or 
not – that will be clarified as we go. 
  
We think there is enough background info in the work description to start 
a discussion, and plan to send out use cases for discussion shortly. 
  
Who is the we? Several makers and providers of telepresence systems have 
gotten together to define the main problem in interoperability – the 
handling of multi-streams. 

  
Background 
One branch of video conferencing has evolved that is focused on immersive 
“being there” experience.  Referred to in various ways such as virtual 
conferencing, telepresence or media spaces, early systems were mainly 
research projects or business systems with limited deployments.  In recent 
years telepresence systems have seen considerable market success. 
Following the model of early systems, the first wave of commercial systems 
have been typically located in specially designed single-purpose rooms 
with multiple relatively large displays permitting life size image 
reproduction, multiple cameras, encoders, decoders and microphones.  These 
systems have several important characteristics that are different from 
more traditional video conferencing systems.   
  
The first difference concerns controlling the visual viewpoint in order to 
improve participant nonverbal communication. These systems preserve 
essential group meeting characteristics such as eye contact, group 
gestures, seating order and spatial audio by carefully orchestrating the 
miking and camera angles at each of the sites . This is distinct from the 
more traditional approach where the geometric relationship between media 
streams is not used to preserve inter-stream communication aspects such as 
eye contact and group dynamics.   
  
A second difference is manipulation of the environment to improve 
immersion.  With telepresence systems, cinematographic aspects of the 
local environment reproduction are carefully planned including color, 
table shape, seating and lighting so that when combined with large high 
quality displays, a strong sense of a "trompe l'oeil" or "being there" 
immersive experience is created.  Typical video conference systems do not 
include these considerations. 
  
As telepresence video systems have become successful in the market, 
manufacturers have started exploring delivery of the nonverbal 
communication and immersive values of telepresence via smaller, less 
expensive and more flexible video conferencing systems for a variety of 
venues, such as individual offices, homes and kiosks. These are also 
telepresence systems, since the audio and video quality is high enough to 
allow clear image reproduction for nonverbal communication, they are able 
to send and receive multiple media streams, and large coordinated multi 
image displays are available for immersive installations.   As the 
industry develops, the line between telepresence and video conferencing 
may become blurred as nonverbal communication and immersive installations 
become broadly available. 
  
Problem 
Although telepresence systems are based on open standards such as RTP, 
SIP, H.264, H.323 suite, they cannot easily interoperate with each other 
without operator assistance and expensive additional equipment that 
translates from one vendor to another. It would be like having to make 
sure all parties are on the same equipment (and network) when making a 
telephone call.  A major factor in the inability of Telepresence systems 
to work with each other is that there is no standard description of the 
multiple streams that comprise the media flows. 
  
For example, in a multiple screen conference, the video and audio streams 
sent from remote participants must be understood by receivers so that they 
can be presented in a coherent and life-like manner. This includes the 
ability to present the remote participants at their true size for their 
apparent distance, while maintaining correct eye contact, gesticular cues, 
and simultaneously providing a spatial audio sound stage consistent with 
the video presentation.  The receiving device that decides how to display 
the incoming information needs to understand a number of variables such as 
the spatial position of the speaker, the field of view of the cameras, the 
camera zoom, which media stream is related to each of the displays, etc. 
  
Charter 
The Telepresence Multi-Streams work item in DISPATCH is chartered to 
define and specify the content of media multi-stream messages and the way 
these will be transported. 
This work is aimed at providing a standard for the exchange of media 
semantics information that will foster interoperable end stations and 
conference bridges. This work item will specify the variables that 
describe the semantics of the media streams and the recommended behavior 
to achieve interoperability.   
  
This requires considering current widely deployed use cases, such as 
single and multiple screens, multi-point, Scalable Video Coding (SVC), as 
well as cases that are expected to be implemented using the protocol 
framework produced by this work item.  The methodology for describing the 
variables must allow extensibility of the variables, since telepresence is 
still a young technology and may have use cases that are not currently 
considered. 
  
The work item will identify use cases, define requirements, and define a 
method for describing and transporting information about multiple media 
streams, including a specification of variables required to support the 
use cases. This work item will consider the reuse of existing IETF 
protocols and produce an architecture/protocol framework document 
describing the protocols required to be implemented to support this 
functionality.  The document will identify any enhancements required to 
existing protocols as well as describing new protocol(s) for interoperable 
multi-streams negotiation that may be required. 
  
Scope 
The scope includes both systems that provide a fully immersive experience, 
and systems that interwork with them and therefore need to understand the 
same multiple stream semantics. 
  
The focus of this work is on audio and video multiple streams, however 
other media types may be considered.  Definition of new application 
protocols is not within the scope of this work 

The group will produce 
  
- Architectural Framework describing the protocols required to be 
implemented to support this functionality and identifying existing 
protocol enhancements and new protocol functionality required 
  
- Specification of a new protocol to support telepresence multi-streams 
[if required] 
  
Goals and Milestones 

Nov 2010 

  

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is 
solely property of the sender's organization. This mail communication is 
confidential. Recipients named above are obligated to maintain secrecy and 
are not permitted to disclose the contents of this communication to 
others.
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. 
If you have received this email in error please notify the originator of 
the message. Any views expressed in this message are those of the 
individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam 
system.