Subject: [rtcweb] VP8 / H.264 conversion without transcoding

From: Gili <[email-masked]>
Date: Wed, 20 Nov 2013 18:13:55 -0800

Hi,

I'm at the WebRTC World conference. If I understood correctly, Zingaya 
just demoed what they claim is a VP8 to H.264 bridge that converts the 
video without transcoding. My interpretation is that they convert the 
surrounding packet format without re-encoding the actual video. Someone 
who understands codec internals (I don't) might want to validate this 
claim, because it could have a significant impact on the MTI debate.

It sounds far fetched, but imagine the possibilities:

 1. If the video is not being re-encoded, do H.264 royalties apply?
 2. If the conversion is really cheap, is it feasible to connect a H.264
    peer to a VP8 peer and have one of them convert in-line (no MCU)?

Gili

---

From: Bossiel <[email-masked]>
Date: Thu, 21 Nov 2013 08:51:15 +0100

This could be true if they can also walk on water and change it into wine.
To be serious, no it's not possible.
Mamadou

Sent from my iPhone

On Nov 21, 2013, at 3:13, Gili <[email-masked]> wrote:

> Hi,
> 
> I'm at the WebRTC World conference. If I understood correctly, Zingaya just demoed what they claim is a VP8 to H.264 bridge that converts the video without transcoding. My interpretation is that they convert the surrounding packet format without re-encoding the     actual video. Someone who understands codec internals (I don't) might want to validate this claim, because it could have a significant impact on the MTI debate.
> 
> If the video is not being re-encoded, do H.264 royalties apply?
> If the conversion is really cheap, is it feasible to connect a H.264 peer to a VP8 peer and have one of them convert in-line (no MCU)?
> Gili
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Thu, 21 Nov 2013 14:27:27 +0000

This probably refers to an intelligent transcoder versus a brute force transcoder. An intelligent transcoder harvests more data during decoding than just the raw output pixels, and uses this extra data to ease encoding. Data like block partitions, motion vectors, intra modes, quantization parameters, etc.

This is common for common conversions, like MPEG2 to H.264. VP8 and H.264 are much closer formats, so this can significantly improve transcoder performance.

However, this is strictly a performance optimization, with no impact on IPR or licensing issues.

Mo

On Nov 21, 2013, at 2:51 AM, "Bossiel" <[email-masked]<mailto:[email-masked]>> wrote:

Sent from my iPhone

On Nov 21, 2013, at 3:13, Gili <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

  1.  If the video is not being re-encoded, do H.264 royalties apply?
  2.  If the conversion is really cheap, is it feasible to connect a H.264 peer to a VP8 peer and have one of them convert in-line (no MCU)?

Gili

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb
_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 21 Nov 2013 10:46:02 -0800

It's a cool demo and I talked to the team that did it. This is only possible because VP8 and H.264 are very similar codecs. The idea is to not redo the motion vector computations or the DCT but just reuse the values from one format in the other since they are the same. They hope that this can provide reduce the CPU by 1/3 from a full decode then re-encode. It also help reduce loss of quality for video. Other people have talked about this idea over the years and many people think it can give a 2 to 3 speed up so that seems consistent with their goals. 

It still requires the same bandwidth, and has similar other impacts. I can not see any way that it would not require MPEG-LA IPR for 264. 

On Nov 21, 2013, at 6:27 AM, Mo Zanaty (mzanaty) <[email-masked]> wrote:

> On Nov 21, 2013, at 2:51 AM, "Bossiel" <[email-masked]> wrote:
> 
>> 	• If the video is not being re-encoded, do H.264 royalties apply?
>> 	• If the conversion is really cheap, is it feasible to connect a H.264 peer to a VP8 peer and have one of them convert in-line (no MCU)?
>> Gili

---

From: Mohammed Raad <[email-masked]>
Date: Fri, 22 Nov 2013 06:00:57 +1100

Hi,

Typically "smart" transcoding operates at 50 percent complexity or less. We
did a fair bit of this at Dilithium Networks when I was there. There are
also significant memory and delay savings when one operates on the MB level
instead of the picture level. With regards to IPR, there is a significant
number of current patents in this area that are not in any multi
organization patent pool as far as I know. Transcoding in this way also
provides quality advantages over tandem transcoding.

Obviously the more similar the codecs the more one gains. In the case of
VP8 and AVC CBP, there are significant enough differences in essential
elements such as the way quantization is done to make this kind of
transcoding challenging, but it is possible and the numbers mentioned in
the previous post sound about right.

Mohammed
On 21 Nov 2013 20:45, "Cullen Jennings" <[email-masked]> wrote:

>
> It's a cool demo and I talked to the team that did it. This is only
> possible because VP8 and H.264 are very similar codecs. The idea is to not
> redo the motion vector computations or the DCT but just reuse the values
> from one format in the other since they are the same. They hope that this
> can provide reduce the CPU by 1/3 from a full decode then re-encode. It
> also help reduce loss of quality for video. Other people have talked about
> this idea over the years and many people think it can give a 2 to 3 speed
> up so that seems consistent with their goals.
>
> It still requires the same bandwidth, and has similar other impacts. I can
> not see any way that it would not require MPEG-LA IPR for 264.
>
>
> > This probably refers to an intelligent transcoder versus a brute force
> transcoder. An intelligent transcoder harvests more data during decoding
> than just the raw output pixels, and uses this extra data to ease encoding.
> Data like block partitions, motion vectors, intra modes, quantization
> parameters, etc.
> >
> > This is common for common conversions, like MPEG2 to H.264. VP8 and
> H.264 are much closer formats, so this can significantly improve transcoder
> performance.
> >
> surrounding packet format without re-encoding the actual video. Someone who
> understands codec internals (I don't) might want to validate this claim,
> because it could have a significant impact on the MTI debate.
> >>
> >>      • If the conversion is really cheap, is it feasible to connect a
> H.264 peer to a VP8 peer and have one of them convert in-line (no MCU)?
> >> Gili

---

From: Gili <[email-masked]>
Date: Thu, 21 Nov 2013 11:25:48 -0800

Cullen,

Thanks for the background information. I also remember them saying that 
we are only at the beginning of this process and that they expect more 
performance improvements in the upcoming months.

It would be nice to gauge how far we are from being able to do this kind 
of conversion in real-time on a mobile device (I suspect very far, but 
it's worth asking). This has implications for P2P interoperability 
between VP8 and H.264 peers without a middleman.

Gili

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 21 Nov 2013 11:30:15 -0800

On Nov 21, 2013, at 11:25 AM, Gili <[email-masked]> wrote:

> I also remember them saying that we are only at the beginning of this process and that they expect more performance improvements in the upcoming months.

Yes, they were very clear it just a early snap shot of what might be possible (right now it is only one direction for example). Their end goal (not where they are now) is the 1/3 CPU usage.

---

From: Bossiel <[email-masked]>
Date: Thu, 21 Nov 2013 20:49:42 +0100

Entropy decoding (CABAC/CAVLC) must be done to get the macroblock structure(partirions, motion vectors...) and residual (AC/DC coeffs). The only part you're skiping is the inverse transform, scaling and motion compensation. Also, the DPB mgt for reference pictures cannot be skiped.
This said, I don't see how this could save 1/3 CPU.

Sent from my iPhone

On Nov 21, 2013, at 20:00, Mohammed Raad <[email-masked]> wrote:

> Hi,
> 
> Typically "smart" transcoding operates at 50 percent complexity or less. We did a fair bit of this at Dilithium Networks when I was there. There are also significant memory and delay savings when one operates on the MB level instead of the picture level. With regards to IPR, there is a significant number of current patents in this area that are not in any multi organization patent pool as far as I know. Transcoding in this way also provides quality advantages over tandem transcoding.
> 
> Obviously the more similar the codecs the more one gains. In the case of VP8 and AVC CBP, there are significant enough differences in essential elements such as the way quantization is done to make this kind of transcoding challenging, but it is possible and the numbers mentioned in the previous post sound about right.
> Mohammed
>

---

From: Monty Montgomery <[email-masked]>
Date: Thu, 21 Nov 2013 15:00:42 -0500

On Thu, Nov 21, 2013 at 2:49 PM, Bossiel <[email-masked]> wrote:
> Entropy decoding (CABAC/CAVLC) must be done to get the macroblock
> structure(partirions, motion vectors...) and residual (AC/DC coeffs). The
> only part you're skiping is the inverse transform, scaling and motion
> compensation.

You wouldn't even be skipping the inverse transform.  VP8 and h.264
don't use the same DCT, so you'd have to do an inverse to track
transform error propagation, which won't be the same in the two
codecs.  Unless, of course, something else rather clever is going
on...

Monty

---

From: Stefan Slivinski <[email-masked]>
Date: Thu, 21 Nov 2013 14:05:03 -0600

The primary advantage you get from this technique is the ability to skip or dramatically reduce motion estimation on the encoder.  The algorithms are similar at a high level but different enough at a low level to make a light weight transcoder impossible

----- Original Message -----
From: Monty Montgomery [mailto:[email-masked]]
Sent: Thursday, November 21, 2013 02:00 PM
To: Bossiel <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [rtcweb] VP8 / H.264 conversion without transcoding

Monty

---

From: Justin Uberti <[email-masked]>
Date: Thu, 21 Nov 2013 12:25:27 -0800

Even if we could do this conversion in real-time on a mobile device...
since you have both codecs present, why not just send the desired format in
the first place?

On Thu, Nov 21, 2013 at 11:25 AM, Gili <[email-masked]> wrote:

> Cullen,
>
> Thanks for the background information. I also remember them saying that we
> are only at the beginning of this process and that they expect more
> of conversion in real-time on a mobile device (I suspect very far, but it's
> worth asking). This has implications for P2P interoperability between VP8
> and H.264 peers without a middleman.
>
> Gili
>
>
>> It still requires the same bandwidth, and has similar other impacts. I
>> can not see any way that it would not require MPEG-LA IPR for 264.
>>
>>
>>>>         • If the conversion is really cheap, is it feasible to connect
>>>> a H.264 peer to a VP8 peer and have one of them convert in-line (no MCU)?
>>>> Gili

---

From: Mohammed Raad <[email-masked]>
Date: Fri, 22 Nov 2013 07:32:13 +1100

There seems to be a misunderstanding, transcoding really only belongs at
the service provider level. If you have both codecs available there is no
need for it. Also you could simply have one codec with both encoder and
decoder parts and the other with only decoder on your device. That would
mean you dont need transcoding and you dont have both encoders.

Mohammed