Subject: Re: [rtcweb] Camera rotation on mobile phones

From: Sergio Garcia Murillo <[email-masked]>
Date: Mon, 27 Apr 2015 11:31:03 +0200

Hi all,

I have just seen that Chrome 43 has added CVO as per my suggestion in 
the message below

  *

    Issue 4145
    <https://code.google.com/p/webrtc/issues/detail?id=4145>Added
    support for CVO (Coordination of video rotation) header extension in
    webrtc. When both sides support CVO, device rotation will not cause
    a new key-frame.

https://code.google.com/p/webrtc/issues/detail?id=4145

I have always thought that it would be a nice addition to the WebRTC 
requirements,  especially in terms of mobile performance and user 
experience (at least in theory). Would it be possible if anyone from the 
Chrome team could share some feedback about it's usage in the real life?

Best regards
Sergio

On 29/09/2013 0:08, Sergio Garcia Murillo wrote:
> Hi all,
>
> Currently, in order to keep image property displayed on the receiver 
> side if the sender rotates the phone (and the camera), it is needed 
> that the sender rotates the image from WxH to HxW so it is sent in 
> upward position over the wire. Also, if the receiver is a mobile phone 
> and has it rotated, it will have to rotate it again to match the 
> current phone orientation.
>
> How about using Coordination of Video Orientation as in 3GPP TS 26.114 
> (http://www.3gpp.org/ftp/Specs/html-info/26114.htm)?
>
> Coordination of Video Orientation consists in signalling of the 
> current orientation of the image captured on the sender side to the 
> receiver for appropriate rendering and displaying. When CVO is 
> succesfully negotiated it shall be signalled by the MTSI client. The 
> signalling of the CVO uses RTP Header Extensions as specified in IETF 
> RFC 5285 [95]. The one-byte form of the header shall be used. CVO 
> information for a 2 bit granularity of Rotation (corresponding to 
> urn:3gpp:video-orientation) is carried as a byte formatted as follows:
>
> Bit#76543210(LSB)
> Definition0000CFR1R0
>
> With the following definitions:
>
> C = Camera: indicates the direction of the camera used for this video 
> stream. It can be used by the MTSI client in receiver to e.g. display 
> the received video differently depending on the source camera.
>
>     0: Front-facing camera, facing the user. If camera direction is 
> unknown by the sending MTSI client in the terminal then this is the 
> default value used.
>
> 1: Back-facing camera, facing away from the user.
>
> F = Flip: indicates a horizontal (left-right flip) mirror operation on 
> the video as sent on the link.
>
> 0: No flip operation. If the sending MTSI client in terminal does not 
> know if a horizontal mirror operation is necessary, then this is the 
> default value used.
>
> 1: Horizontal flip operation
>
> R1, R0 = Rotation: indicates the rotation of the video as transmitted 
> on the link. The receiver should rotate the video to compensate that 
> rotation. E.g. a 90° Counter Clockwise rotation should be compensated 
> by the receiver with a 90° Clockwise rotation prior to displaying.
>
> Table 7.2: Rotation signalling for 2 bit granularity
>
> R1
>
> 	
>
> R0
>
> 	
>
> Rotation of the video as sent on the link
>
> 	
>
> Rotation on the receiver before display
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0° rotation
>
> 	
>
> None
>
> 0
>
> 	
>
> 1
>
> 	
>
> 90° Counter Clockwise (CCW) rotation or 270° Clockwise (CW) rotation
>
> 	
>
> 90° CW rotation
>
> 1
>
> 	
>
> 0
>
> 	
>
> 180° CCW rotation or 180° CW rotation
>
> 	
>
> 180° CW rotation
>
> 1
>
> 	
>
> 1
>
> 	
>
> 270° CCW rotation or 90° CW rotation
>
> 	
>
> 90° CCW rotation
>
> CVO information for a higher granularity of Rotation (corresponding to 
> urn:3GPP:video-orientation:6) is carried as a byteformatted as follows:**
>
> Bit#76543210(LSB)
> DefinitionR5R4R3R2CFR1R0
>
> where Cand Fare as defined above and the bits R5,R4,R3,R2,R1,R0 
> represent the Rotation, which indicates the rotation of the video as 
> transmitted on the link. Table 7.3 describes the rotation to be 
> applied by the receiver based on the rotation bits.
>
> Table 7.3: Rotation signalling for 6 bit granularity
>
> R1
>
> 	
>
> R0
>
> 	
>
> R5
>
> 	
>
> R4
>
> 	
>
> R3
>
> 	
>
> R2
>
> 	
>
> Rotation of the video as sent on the link
>
> 	
>
> Rotation on the receiver before display
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0° rotation
>
> 	
>
> None
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 1
>
> 	
>
> (360/64)° Counter Clockwise (CCW) rotation
>
> 	
>
> (360/64)° CW rotation
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 0
>
> 	
>
> 1
>
> 	
>
> 0
>
> 	
>
> (2*360/64)° CCW rotation
>
> 	
>
> (2*360/64)° CW rotation
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 	
>
> *.*
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 0
>
> 	
>
> (62*360/64)° CCW rotation
>
> 	
>
> (2*360/64)° CCW rotation
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> 1
>
> 	
>
> (63*360/64)° CCW rotation
>
> 	
>
> (360/64)° CCW rotation
>
> Best regards
> Sergio
>

---

From: Justin Uberti <[email-masked]>
Date: Wed, 29 Apr 2015 12:31:25 -0700

CVO is part of the rtcweb-video requirements.

On Mon, Apr 27, 2015 at 2:31 AM, Sergio Garcia Murillo <
[email-masked]> wrote:

>  Hi all,
>
>
> I have just seen that Chrome 43 has added CVO as per my suggestion in the
> message below
>
>
>    -
>
>    Issue 4145 <https://code.google.com/p/webrtc/issues/detail?id=4145>
>    Added support for CVO (Coordination of video rotation) header extension in
>    webrtc. When both sides support CVO, device rotation will not cause a new
>    key-frame.
>
> Currently, in order to keep image property displayed on the  receiver side
> if the sender rotates the phone (and the camera), it is needed that the
> sender rotates the image from WxH to HxW so it is sent in upward position
> over the wire. Also, if the receiver is a mobile phone and has it rotated,
> it will have to rotate it again to match the current phone orientation.
>
> How about using Coordination of Video Orientation as in 3GPP TS 26.114 (
> http://www.3gpp.org/ftp/Specs/html-info/26114.htm)?
>
> Coordination of Video Orientation consists in signalling of the current
> orientation of the image captured on the sender side to the receiver for
> appropriate rendering and displaying. When CVO is succesfully negotiated it
> shall be signalled by the MTSI client. The signalling of the CVO uses RTP
> Header Extensions as specified in IETF RFC 5285 [95]. The one-byte form of
> the header shall be used. CVO information for a 2 bit granularity of
> Rotation (corresponding to urn:3gpp:video-orientation) is carried as a byte
> formatted as follows:
>
> Bit#                   7                 6                 5
> 4               3                 2                 1                 0
> (LSB)
> Definition      0           0           0           0           C
> F           R1          R0
>
> stream. It can be used by the MTSI client in receiver to e.g. display the
> received video differently depending on the source camera.
>
> unknown by the sending MTSI client in the terminal then this is the default
> value used.
>
> F = Flip: indicates a horizontal (left-right flip) mirror operation on the
> video as sent on the link.
>
> R1, R0 = Rotation: indicates the rotation of the video as transmitted on
> the link. The receiver should rotate the video to compensate that rotation.
> E.g. a 90° Counter Clockwise rotation should be compensated by the receiver
> with a 90° Clockwise rotation prior to displaying.
>
> urn:3GPP:video-orientation:6) is carried as a byte  formatted as follows:
>
> Bit#                   7                 6                 5
> 4               3                 2                 1                 0
> (LSB)
> Definition      R5          R4          R3          R2          C
> F           R1          R0
>
> where C and F are as defined above and the bits R5,R4,R3,R2,R1,R0
> transmitted on the link. Table 7.3 describes the rotation to be applied by
> the receiver based on the rotation bits.
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>
>

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Wed, 29 Apr 2015 23:08:00 +0200

Thanks Justin!

Was not aware of it as I didn't saw any feedback on the mailing list. 
Sometimes is difficult to be up to date with latest changes :)

Best regards
Sergio
On 29/04/2015 21:31, Justin Uberti wrote:
> On Mon, Apr 27, 2015 at 2:31 AM, Sergio Garcia Murillo 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi all,
>
>
>     I have just seen that Chrome 43 has added CVO as per my suggestion
>     in the message below
>
>      *
>
>         Issue 4145
>         support for CVO (Coordination of video rotation) header
>         extension in webrtc. When both sides support CVO, device
>         rotation will not cause a new key-frame.
>
>     I have always thought that it would be a nice addition to the
>     WebRTC requirements,  especially in terms of mobile performance
>     and user experience (at least in theory). Would it be possible if
>     anyone from the Chrome team could share some feedback about it's
>     usage in the real life?
>
>     Best regards
>     Sergio
>
>
>>     Currently, in order to keep image property displayed on the 
>>     receiver side if the sender rotates the phone (and the camera),
>>     it is needed that the sender rotates the image from WxH to HxW so
>>     it is sent in upward position over the wire. Also, if the
>>     receiver is a mobile phone and has it rotated, it will have to
>>     rotate it again to match the current phone orientation.
>>
>>     How about using Coordination of Video Orientation as in 3GPP TS
>>     26.114 (http://www.3gpp.org/ftp/Specs/html-info/26114.htm)?
>>
>>     current orientation of the image captured on the sender side to
>>     the receiver for appropriate rendering and displaying. When CVO
>>     is succesfully negotiated it shall be signalled by the MTSI
>>     client. The signalling of the CVO uses RTP Header Extensions as
>>     specified in IETF RFC 5285 [95]. The one-byte form of the header
>>     shall be used. CVO information for a 2 bit granularity of
>>     Rotation (corresponding to urn:3gpp:video-orientation) is carried
>>     as a byte formatted as follows:
>>
>>     Bit#76543210(LSB)
>>     Definition0000CFR1R0
>>
>>     C = Camera: indicates the direction of the camera used for this
>>     video stream. It can be used by the MTSI client in receiver to
>>     e.g. display the received video differently depending on the
>>     source camera.
>>
>>         0: Front-facing camera, facing the user. If camera direction
>>     is unknown by the sending MTSI client in the terminal then this
>>     is the default value used.
>>
>>     F = Flip: indicates a horizontal (left-right flip) mirror
>>     operation on the video as sent on the link.
>>
>>     0: No flip operation. If the sending MTSI client in terminal does
>>     not know if a horizontal mirror operation is necessary, then this
>>     is the default value used.
>>
>>     R1, R0 = Rotation: indicates the rotation of the video as
>>     transmitted on the link. The receiver should rotate the video to
>>     compensate that rotation. E.g. a 90° Counter Clockwise rotation
>>     should be compensated by the receiver with a 90° Clockwise
>>     rotation prior to displaying.
>>
>>     CVO information for a higher granularity of Rotation
>>     (corresponding to urn:3GPP:video-orientation:6) is carried as a
>>     byteformatted as follows:**
>>
>>     Bit#76543210(LSB)
>>     DefinitionR5R4R3R2CFR1R0
>>
>>     represent the Rotation, which indicates the rotation of the video
>>     as transmitted on the link. Table 7.3 describes the rotation to
>>     be applied by the receiver based on the rotation bits.
>>
>     [email-masked] <mailto:[email-masked]>