Subject: [websec] Review of draft-ietf-websec-mime-sniff-02.txt

From: Alexey Melnikov <[email-masked]>
Date: Sat, 07 May 2011 17:36:09 +0100

I have a mixed feeling on this document and I am still thinking on 
larger issues with it, so expect a followup message on the subject.
In the meantime, below are more obvious issues:

RFC 2119 is not a Normative Reference, but it should be, as the document 
is using MUSTs.

2.  Metadata

   For octets received via HTTP, the Content-Type HTTP header field, if
   present, indicates the media type.  Let the official-type be the
   media type indicted by the HTTP Content-Type header field, if
   present.  If the Content-Type header field is absent or if its value
   cannot be interpreted as a media type (e.g. because its value doesn't
   contain a U+002F SOLIDUS ('/') character), then there is no official-
   type.

I would prefer if this text is clearer that the last sentence is dealing 
with messages which are invalid according to RFC 2616.

   For octets fetched over some other protocols, e.g.  FTP, there is no

FTP needs an Informative reference.

   type information.

   Note: Comparisons between media types, as defined by MIME
   specifications, are done in an ASCII case-insensitive manner.
   [RFC2046]

This should be a reference. Probably Normative?

3.  Web Pages

   2.  If the octets were fetched via HTTP and there is an HTTP Content-
       Type header field and the value of the last such header field has
       octets that *exactly* match the octets contained in one of the

Aren't you supposed to match media types case insensitively?

       following lines:

   4.  If the official-type is "unknown/unknown", "application/unknown",

I am very curious to know which products are generating "unknown".

Does application/unknown need to be registered in the media type registry?

       or "*/*", jump to the "unknown type" section below.

4.  Text or Binary

   This section defines the *rules for distinguishing if a resource is
   text or binary*.

   1.  The user agent MAY wait for 512 or more octets be to arrive.

Drop "be"?

          Note: Waiting for 512 octets octets to arrive causes the text-
          or-binary algorithm to be deterministic for a given sequence

Does it? I think this section is showing euristics which are likely (but 
are not guaranteed) to produce the correct result.

          of octets.

   4.  If none of the first n octets are binary data octets then let the
       sniffed-type be "text/plain" and abort these steps.

                         +-------------------------+
                         | Binary Data Byte Ranges |
                         +-------------------------+
                         | 0x00 -- 0x08            |
                         | 0x0B                    |
                         | 0x0E -- 0x1A            |
                         | 0x1C -- 0x1F            |
                         +-------------------------+

Does this table try to cover all UTF encodings: UTF-8, UTF-16LE and 
UTF-16BE?

5.  Unknown Type

I think some of the media types listed in this section need registering, 
e.g.
image/bmp, image/webp. But chairs can help out with this.

5.1.  Signature for H.264

I think this algorithm needs to be verified by somebody with experience 
in H.264. I can ask for a RAI Area review.

7.  Video

   This section defines the *rules for sniffing videos specifically*.

   If the first octets match one of the signatures in Section 5 for one
   of the following media types, then let the sniffed-type be the
   corresponding media type and abort these steps:

   o  video/H264

Note that the section 5 currently doesn't define video/H264

   o  video/webm

This media type needs registering as well.

---

From: Adam Barth <[email-masked]>
Date: Sat, 7 May 2011 17:30:48 -0700

Thanks for the review.

On Sat, May 7, 2011 at 9:36 AM, Alexey Melnikov
<[email-masked]> wrote:
> I have a mixed feeling on this document and I am still thinking on larger
> issues with it, so expect a followup message on the subject.

I look forward to your message.

> RFC 2119 is not a Normative Reference, but it should be, as the document is
> using MUSTs.

Fixed.

> with messages which are invalid according to RFC 2616.

Done.

> FTP needs an Informative reference.

Done.

> This should be a reference. Probably Normative?

Done.

> Aren't you supposed to match media types case insensitively?

In general, yes, but this list comes from a particular buggy
implementation that used exactly these octets.  We could expand the
scope of this bandaid, but it's unclear whether that's a good idea.

> I am very curious to know which products are generating "unknown".

I don't have that information.  When collecting the data for this part
of the spec, I wasn't allowed to look at the URLs associated with the
types for privacy reasons.

> Does application/unknown need to be registered in the media type registry?

We certainly could register it if that would be helpful

>  1.  The user agent MAY wait for 512 or more octets be to arrive.

Fixed.

> Does it? I think this section is showing euristics which are likely (but are
> not guaranteed) to produce the correct result.

It's deterministic, but not necessarily "correct" (for some external
notion of correctness).

>      sniffed-type be "text/plain" and abort these steps.

This table is purely empirical.  It's what folks do, for better or worse.

> e.g. image/bmp, image/webp. But chairs can help out with this.

The MIME registry is missing lots of types that are used in the world.
 It would certainly be valuable to align the MIME registry with
reality.

> 5.1.  Signature for H.264
>
> I think this algorithm needs to be verified by somebody with experience in
> H.264. I can ask for a RAI Area review.

Thanks.  I got it from an H.264 expert at Apple and checked it on a
couple example videos.  However, it's the part of the document that's
furtherest from my expertise.

> Note that the section 5 currently doesn't define video/H264

I've already corrected this issue locally.

> This media type needs registering as well.

Thanks!
Adam

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 09 May 2011 10:53:24 +0900

> types for privacy reasons.

Intriguing. I always thought privacy applied to clients, not to (public) 
servers. Or was this data collected from private servers?

Regards,   Martin.

---

From: Adam Barth <[email-masked]>
Date: Sun, 8 May 2011 19:55:10 -0700

The data was collected via opt-in anonymous usuage statistics in a web
browser.  The data likely covers both public and private servers and is
weighted by how often those servers are visited by actual browser users.

Adam