Subject: [apps-discuss] draft-farrell-decade-ni-02 - we think this is done...

From: Stephen Farrell <[email-masked]>
Date: Thu, 05 Apr 2012 21:10:47 +0100

...well, modulo one or two questions still in the document [1]
and getting your review:-)

This document has been discussed in the decade WG as a way to
standardise how to emded hashes into names, which is something
that'll likely be needed there but is clearly more general. As
if to prove that, at IETF-83 we added a binary form that seems
to better fit what the core WG want for doing the same thing in
CoAP. We've worked on it a good bit in the time since and so
we reckon this is pretty much ready now.

Our plan is to try get comments from the decade, core and
appsarea lists and if we're looking good to then ask Barry
Leiba if he'll AD sponsor this document, so it can be done in
time to not slow down CoAP. (Barry's ok with this plan.)

So, please take a read if you're interested in naming things
with hashes and send your comments.

Since this was first discussed on the decade list, I'll suggest
using that [2] as a good place to send comments. Any of the
above-mentioned lists will work though, since various authors
are on each. Probably better to not cross-post to them all
though. (I've sent separate mails to each for that reason.)

Thanks,
Stephen.

[1] http://tools.ietf.org/html/draft-farrell-decade-ni
[2] https://www.ietf.org/mailman/listinfo/decade

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 05 Apr 2012 14:58:05 -0600

> and getting your review:-)

Looks good!

/psa

---

From: Martin Thomson <[email-masked]>
Date: Thu, 5 Apr 2012 14:22:47 -0700

> with hashes and send your comments.

This is my remaining concern.  The draft uses /.well-known/ as the
base "locator".  That severely limits its use.  Access to
/.well-known/ is necessarily constrained.

I might want to identify a resource outside of /.well-known/.  I might
also want to identify a resource that is served elsewhere.  I can't do
either with this scheme.

I'd prefer something that allowed for the embedding of a complete HTTP
URI.  Bonus credit for embedding a any URI.

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 05 Apr 2012 22:25:28 +0100

On 04/05/2012 10:22 PM, Martin Thomson wrote:
> On 5 April 2012 13:10, Stephen Farrell<[email-masked]>  wrote:

Sure. We added that though in section 5. (For URLs anyway.)
Be glad to add more text that makes that clear, and happy
to say its usable with other schemes if you can offer good
words for that.

Cheers,
S.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 5 Apr 2012 14:30:15 -0700

> to say its usable with other schemes if you can offer good

That doesn't quite do it.  If I understand correctly, you want someone
to look at a URI and "know" that this particular path component is
actually a hash.

What I want to do is provide an ni: URI that wraps:

http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.18/jquery-ui.min.js

And have clients (say my browser) load that URI and be able to check
that the content is the right content.  Without knowledge of URI
components that should be opaque.  That is, I want a mechanism for
dealing with mixed content (see plenary) that doesn't require TLS.

This might be done by something like:

ni:sha256:blah:http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.18/jquery-ui.min.js

Though that completely ruins your nice URI form.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 5 Apr 2012 23:56:28 +0200

** Summary:

This may seem nearly trivial, but upon looking closer really is awesome work, something truly only the IETF could create.

** Nits:

What is a "thing"?  (abstract)

The reference from section 3 to section 9.1 appears wrong -- I don't think 9.1 actually defines the registry required here.
If this is referencing 9.3, please make sure there that the "ID" column is left empty (and thus no number consumed) if a number for the binary encoding is not needed.

I don't like the sentence
   Since application
   code often attempts to enforce such encoding, decoders MUST recognize
   the use of URI escape encoding.
and the following sentences too much -- yes, you MUST implement URI syntax, but not just for this reason.
Add examples of percent-decoding (implementers implement from examples!).

How useful is:
   If an application is presented with a HTTP(S) URL with "/.well-
   known/ni/" as the start of its pathname component, then the reverse
   mapping to an ni URI either including or excluding the authority
   might produce an ni URI that is meaningful, but there is no guarantee
   that this will be the case.
Wouldn't it be more appropriate to mandate that /.well-known/ni *MUST* only be used in a way that makes this reverse mapping useful?

s/URL Fragment/URL Segment/ (Fragment is confusing, as it means something specific in URIs.)

Maybe the human-readable form should be prefixed by "ni;" so it is easily recognizable.

http://www.tcd.ie/.well-known/ni/sha256/UyaQV-Ev4rdLoHyJJWCi11OHfrYv9E1aGQAlMO2X_-Q does not resolve (maybe you meant example.com?).

I believe for this to be the replacement of "unsecure links out of HTTPS", the secure media type issue must be solved in the base spec.
(Oh, and is there ever a need to discuss content-coding in this context???)

Please re-spin http://tools.ietf.org/html/draft-hallambaker-decade-ni-params-00 and make sure that Martin's concern is addressed...

** PS

Again, I'm still in awe that a 10-page specification can solve so many problems in one fell swoop.

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 06 Apr 2012 11:21:17 +0100

On 04/05/2012 10:30 PM, Martin Thomson wrote:
> On 5 April 2012 14:25, Stephen Farrell<[email-masked]>  wrote:
> ni:sha256:blah:http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.18/jquery-ui.min.jse
>

Well, it is less nice but following draft-hallambaker-decade-ni-params
we could add support for:

ni:///sha256;abc...?url=http://ajax...

I've probably got the encoding there wrong but would that meet the
need if properly done?

S.

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 06 Apr 2012 11:28:21 +0100

Hi Carsten,

> This may seem nearly trivial, but upon looking closer really is awesome work, something truly only the IETF could create.

Thanks that's nice to hear.

> What is a "thing"?  (abstract)

:-) I like it saying "thing"

> If this is referencing 9.3, please make sure there that the "ID" column is left empty (and thus no number consumed) if a number for the binary encoding is not needed.

Yep, should be 9.3 and I'll add something making the ID colum optional.

> Add examples of percent-decoding (implementers implement from examples!).

Would gratefully accept such.

> Wouldn't it be more appropriate to mandate that /.well-known/ni *MUST* only be used in a way that makes this reverse mapping useful?

Not sure to be honest.

> s/URL Fragment/URL Segment/ (Fragment is confusing, as it means something specific in URIs.)

Sure.

> Maybe the human-readable form should be prefixed by "ni;" so it is easily recognizable.

Sure. Might have to re-do the ABNF to allow for that but do-able.

> http://www.tcd.ie/.well-known/ni/sha256/UyaQV-Ev4rdLoHyJJWCi11OHfrYv9E1aGQAlMO2X_-Q does not resolve (maybe you meant example.com?).

Ack.

> (Oh, and is there ever a need to discuss content-coding in this context???)

Not sure what you mean by "solved" can you elaborate?

> Please re-spin http://tools.ietf.org/html/draft-hallambaker-decade-ni-params-00 and make sure that Martin's concern is addressed...

Will do. Probably over the weekend.

Cheers,
S.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 6 Apr 2012 14:08:34 +0200

> Not sure what you mean by "solved" can you elaborate?

The problem is that the ni: hash only secures the bytes of the payload.
(At least that's my assumption -- it might be worth saying explicitly what is hashed here, BTW.)

The entity I would have been retrieving over a secure HTTPS channel also has metadata.
Many of these are not critical, as they only pertain to the retrieval process (say, the Etag).
However, the media type is critical for correctly interpreting the returned result.
While I'm not aware of the proverbial contract text that means something different when interpreted as SJIS instead of UTF-8, attacks on the basis of swapping the media type are conceivable.  (Also, there is lots of potential for misconfiguration, damaging the reliability.)

I'm naming content-coding as another example that it is not quite clear what is the input to the hash function.
So maybe it is worth having another (normative) document that spells out how exactly ni:// is used in the Browser Web (among others, to solve the "unsecure links" problem)?

Grüße, Carsten

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 06 Apr 2012 16:45:44 +0100

Hiya,

> So maybe it is worth having another (normative) document that spells out how exactly ni:// is used in the Browser Web (among others, to solve the "unsecure links" problem)?

Ah now I see. Yes, I think other documents should define the
hash input bytes, each in their own contexts.

I'd be happy to help out with making a start on the web/ajax
thing, but would need help from someone who knows more about
the input bytes.

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 06 Apr 2012 23:03:35 +0100

So I took a stab at addressing Carsten's and Martin's
good comments. Bit more to do but the significant changes
so far are:

- added a "url=" query string parameter to
   draft-hallambaker-decade-ni-params for Martin's use-case
- added an "nih" scheme to draft-farrell-decade-ni for the
   "human" scheme (nice acronym:-); that seems better than
   using the "ni" scheme with two different but sorta
   similar encodings (base32 and base64Url)

Previews at [1,2] (but might change when co-authors get a
chance to think about those changes).

More comments welcome,
TIA,
S.

[1] http://down.dsg.cs.tcd.ie/misc/draft-farrell-decade-ni.txt
[2] http://down.dsg.cs.tcd.ie/misc/draft-hallambaker-decade-ni-params.txt

On 04/06/2012 04:45 PM, Stephen Farrell wrote:
>>>> I believe for this to be the replacement of "unsecure links out of
>>>> HTTPS", the secure media type issue must be solved in the base spec.
>> While I'm not aware of the proverbial contract text that means
>> something different when interpreted as SJIS instead of UTF-8, attacks
>> on the basis of swapping the media type are conceivable. (Also, there
>> is lots of potential for misconfiguration, damaging the reliability.)
>>
>> I'm naming content-coding as another example that it is not quite
>> clear what is the input to the hash function.
>> So maybe it is worth having another (normative) document that spells
>> out how exactly ni:// is used in the Browser Web (among others, to
>> solve the "unsecure links" problem)?
>
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: "Manger, James H" <[email-masked]>
Date: Tue, 10 Apr 2012 14:08:15 +1000

Stephen,

A couple of "typos" in draft-farrell-decade-ni-02:

The example SubjectPublicKeyInfo is shown in a strange way: each pair of bytes is shown in the opposite order to bytes on the wire (and the offset at the start of the line is in nibbles). The first few bytes of the BER-encoding are 30 820122, while the example starts:
   0000000 8230 2201 0d30 0906 862a 8648 0df7 0101

The BER-encoding is actually:
   30 820122
      30 0D
         06 09 2A86
      ...

Half of the example names are wrong, almost certainly due to this strange format. 

The 120-bit "Binary Name" in figure 6 is wrong. It swaps the order of bytes (after the first). It should be:
  03 53269057 E12FE2B7 4BA07C89 2560A2

The human-readable names (120-bit and 32-bit) are wrong. They have base32-encoded pairs of bytes in the wrong order. They should be:
  sha-256-120;kmtjav7bf7rlos5apseskyfc;????
  sha-256-32;kmtjavy;????

I am not sure what the correct checksums are.

The example checksums (eokv and csdh) look wrong. I assume the 4 base32 chars in the checksum should encode 5+5+5+1 bits respectively of the 16-bit CRC. Hence the last char encodes 1 bit, padded with 4 zeros: either 00000 or 10000, which should give 'a' or 'q' (not 'v' or 'h').

P.S. A CRC16 is calculated over an array of bytes. However the input is defined as a text string. I guess ASCII/UTF-8 encoding is assumed. It might be better to calculate the CRC over the bytes of the truncated hash.

--
James Manger

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 10 Apr 2012 08:48:30 +0100

Thanks James. Looks odd ok, will check it out & get back,
Good catch though,
Cheers,
S

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 11 Apr 2012 03:34:22 +0100

Hi again James,

> Half of the example names are wrong, almost certainly due to this strange format.

Yep. That was me being sloppy ("od -x" did used to do the right thing
on some platform I used sometime in the distant past I think;-)

>    03 53269057 E12FE2B7 4BA07C89 2560A2

Ditto.

>    sha-256-120;kmtjav7bf7rlos5apseskyfc;????

We're deleting the last character, yes. Might need to think more
about I18N for that all right. I'm told dropping the last character
is ok, but have yet to check.

> P.S. A CRC16 is calculated over an array of bytes. However the input is defined as a text string. I guess ASCII/UTF-8 encoding is assumed. It might be better to calculate the CRC over the bytes of…

Fair point. OTOH, that'd mean omitting the "nih:sha-256;" part but
I guess that'd be not much harm.

I suspect that you may be right about the checksum. Will check with
co-authors more involved with CoAP (Ari basically) to see what they
prefer, since its them that want the CRC.

I've posted a -03 that I hope is good now, other than the CRC input.

Thanks again,
Stephen.

---

From: Ari Keranen <[email-masked]>
Date: Fri, 27 Apr 2012 11:34:17 -0500

Hi James,

Thanks for the comments! And sorry for the late reply.

The base32 encoded versions of the CRCs were indeed wrong in the draft. 
Good catch. After getting it right only after a considerable amount of 
hacking, we came into conclusion that it's not worth the effort. If it's 
really that hard to get right, it should be made more simple.

Therefore, we changed the encoding from base32 to base16 (hex) for both 
the hash and the CRC. While adding a few character of length to the 
hashes (but not to CRCs since now the lengths align perfectly), this 
also removed all the complexities with padding that was a bit pain to 
both specify and implement.

Yet, I think it's better to calculate the CRC over the whole string 
since then we protect also the hash-algorithm part. Also, it's a bit 
simpler to implement since you don't have to parse the string and 
convert hash back to binary for the checksum (e.g., if you do the check 
in the UI). We clarified that the input needs to be in UTF-8.

Cheers,
Ari

On 4/11/12 5:34 AM, Stephen Farrell wrote:
>> The example checksums (eokv and csdh) look wrong. I assume the 4
>> base32 chars in the checksum should encode 5+5+5+1 bits respectively
>> of the 16-bit CRC. Hence the last char encodes 1 bit, padded with 4
>> zeros: either 00000 or 10000, which should give 'a' or 'q' (not 'v' or
>> 'h').
>>
>> P.S. A CRC16 is calculated over an array of bytes. However the input
>> is defined as a text string. I guess ASCII/UTF-8 encoding is assumed.
>> It might be better to calculate the CRC over the bytes of the
>> truncated hash.
>