Subject: Re: [dispatch] Proposal to add 'haptics' as a new top-level media type - Comments welcome [Was RE: draft-muthusamy-dispatch-haptics-00.txt]

From: David Singer <[email-masked]>
Date: Tue, 10 Nov 2020 14:14:23 -0800

> On 21Oct, 2020, at 18:29 , Dale R. Worley <[email-masked]> wrote:
> 
> Another is that it seems like few of these media formats are
> non-secret.  There are registrations of media formats that are
> patent-encumbered, but the specifics of those formats can be
> documented.  What would really encourage support for a haptics data type
> is one or more open standards.

I think the standards are under way; but the top-level type reflects the existence of a distinct media type and is not, as far as I know, likely to be encumbered.

Though we all know what is needed to present (for example) stereo audio or ’normal video’, haptics doesn’t have such a presentational obvious baseline, so I think general formats may take a little longer than for the more traditional types. I don’t think that should stop the top-level registration, as collecting single-vendor examples will help us see what a general format might look like.

I don’t think we should mix up what’s needed for sub-type registration, with establishment of the top-level type. Yeshwant can and should cite examples in the industry, but it’s not for him to register other people’s formats, of course.

> On 24Oct, 2020, at 9:28 , John C Klensin <[email-masked]> wrote:
> 
> If you are not already in close touch with, and participating
> in, ISO/IEC JTC1 SC29 (the body responsible for ISO/IEC
> 14496-12), make that connection.  If you need help, you might
> approach Stephan Wenger, who is IETF's liaison to that SC (see
> https://www.ietf.org/about/liaisons/ ).  If they are
> establishing the "real" definition for the top-level type and
> they (or their MA) are defining or approving the registrations,
> ending up with a separate registration system in the IETF -- one
> that might not be precisely synchronized -- would be really  bad
> news.  

Indeed, it was conversation there that led me to suggest to Yeshwant that we should pay attention to the media-type registration. I don’t think SC29 can or should define the top-level type to suit only what’s happening at MPEG. The top-level type should be general.

> On 24Oct, 2020, at 9:28 , John C Klensin <[email-masked]> wrote:
> 
> If there is going to be a new
> top-level type, your defining document needs to specify exactly
> how subtypes will be registered and managed.

Thanks, yes: I would assume very similarly to audio and video.

> On 26Oct, 2020, at 7:46 , Ned Freed <[email-masked]> wrote:
> 
> This may be the case, but it's not the only, or most important case you need to
> make when justifying the creation of a top-level type. In particular, you
> need to explain why haptics deserve special recognition rather than living
> under application, as a myriad of other things (word processing formats,
> spreadsheet formats, automative control formats, etc.) do.
> 
> I don't think this is a difficult case to make - it seems to me that haptics
> data represents a fundamentally different sort of data in the same way that we
> differentiate audio, video, etc. But I think section 2 falls short of making
> the case currently.

I’ll try to help Yeshwant improve that section. I think that perceptual types probably deserve a mediatype, as they indicate what kind of expression capability you need (e.g. a screen to present video, speakers for audio, and so on).

> 
> On 27Oct, 2020, at 4:33 , Cullen Jennings <[email-masked]> wrote:
> 
> How do we deal with that this media type is often embedded in a video stream?  Is this really different than game controller inputs and moves in real time settings? 

Indeed, when timed media is multiplexed (a/v files) we have to choose; at the moment, video/mp4 covers video-only and audio/video, and we’ll need to decide what to do if timed haptics is multiplexed in. But note that modern streaming delivery (DASH, HLS) is de-multiplexed, so that would enable correct substream typing.

> On 27Oct, 2020, at 4:43 , Martin J. Dürst <[email-masked]> wrote:
> 
> As an aside, I think delegating the registration of subtypes in a certain top-level type to some other standards organization (as alluded to by John Klensin) is a bad idea because it would create confusion and additional overhead. T

I agree, and I don’t think MPEG is set up for it either; nor is there any pretense that they ‘own’ the industry haptics space. There’s just early work there on handling timed haptics in a harmonious way along with audio and video, and some other early work on looking for a possible general format.

David Singer

[email-masked]

Dave Singer
Multimedia and Software Standards, Apple

[email-masked]

---

From: [email-masked] (Dale R. Worley)
Date: Thu, 19 Nov 2020 23:20:24 -0500

David Singer <singer=[email-masked]> writes:
> I don't think we should mix up what's needed for sub-type
> registration, with establishment of the top-level type. Yeshwant can
> and should cite examples in the industry, but it's not for him to
> register other people's formats, of course.

As a general rule, I agree.  But in practice, the best thing to drive
the creation of a top-level media type is a bunch of subtypes that
people *want* to see registered.  And the best thing to drive that is
minimally-encumbered interoperabilty.

Dale

---

From: Yeshwant Muthusamy <[email-masked]>
Date: Fri, 20 Nov 2020 18:29:10 +0000

Hi Dale,

Not sure if you were aware, but I posted a new version of the haptics I-D late Sunday evening:

https://datatracker.ietf.org/doc/draft-muthusamy-dispatch-haptics/

It was sent out just in time for my presentation in the DISPATCH session at IETF 109. 

This new version (hopefully) addresses all the feedback that I received on version 00, including draft registration of subtypes.

Comments/questions welcome.

Regards,
Yeshwant

Yeshwant Muthusamy, Ph.D. | Senior Director, Standards

[email-masked] | +1 469-583-2171

-----Original Message-----
From: Dale R. Worley <[email-masked]> 
Sent: Thursday, November 19, 2020 10:20 PM
To: David Singer <singer=[email-masked]>
Cc: [email-masked]; Chris Ullrich <[email-masked]>; Yeshwant Muthusamy <[email-masked]>
Subject: Re: [dispatch] Proposal to add 'haptics' as a new top-level media type - Comments welcome [Was RE: draft-muthusamy-dispatch-haptics-00.txt]

> register other people's formats, of course.

As a general rule, I agree.  But in practice, the best thing to drive the creation of a top-level media type is a bunch of subtypes that people *want* to see registered.  And the best thing to drive that is minimally-encumbered interoperability.

Dale