Subject: [Json] "best practices" Vs. Profile for i-json

From: Larry Masinter <[email-masked]>
Date: Fri, 1 Aug 2014 21:55:54 +0000

My conclusion after some discussions at IETF is that i-json needs to separate "best practices" vs "profile".

The SHOULD belongs with best practices: e.g., "numbers in I-JSON SHOULD be representable by IEEE floating point."

If you want a real profile, though, you need clear criteria:
"All numeric values in I-JSON MUST round-trip to IEEE floating point and back to the same string." 
And then if you want an exception, make it application dependent: "the result must be JSON and SHOULD be I-JSON".

But the current doc mixes these together in a way that makes it less useful.

If you don't want both, just do the profile.  

Related topic: consider I-JSON a kind of normalization to an idealized JSON data model. Make the data model explicit.
Insure round-trip between data model and i-json is not information lossy.
Then any operation which defines itself in terms of the data model and not the textual representation would automatically get i-json output, such as Merge/patch and schemas.

Larry
--
http://larry.masinter.net

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 1 Aug 2014 19:01:41 -0400

> And then if you want an exception, make it application dependent: "the result must be JSON and SHOULD be I-JSON".

Not happening unless we move to a binary encoding or a binary exponent
encoding. To round trip the floating point between binary and decimal
exponents is ridiculously non-trivial.

---

From: John Cowan <[email-masked]>
Date: Sat, 2 Aug 2014 02:11:29 -0400

> exponents is ridiculously non-trivial.

I don't say it's trivial, but it *is* a solved problem, if by "round
trip" you mean "guarantee that what one implementation writes, another
implementation reads as the same float."  The current C and C++
standards require round-trippability in this sense.

This is *not* the same as "print a decimal number that is exactly equal
to a given float".  Denormalized floats can have hundreds of digits of
precision.  But if you accept round-tripping as sufficient, then every
32-bit float can be represented with at most 9 decimal digits, and every
64-bit float with at most 17.  In particular, though there is no float
exactly equal to 1.1, there *is* a float which is closer to 1.1 than any
other float, and which may be safely printed as 1.1 and read back as
the original float.

See Quux and JonL's paper "How To Print Floating-Point Numbers Accurately"
at <http://kurtstephens.com/files/p372-steele.pdf> or its successors
at <http://www.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf> and
<http://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf>.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
Mark Twain on Cecil Rhodes: I admire him, I freely admit it,
and when his time comes I shall buy a piece of the rope for a keepsake.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 2 Aug 2014 10:58:23 -0400

> standards require round-trippability in this sense.

Its not a problem that can be solved in an IETF spec by referencing
another spec.

At the very minimum, code is required. But if it is going to be any
use in practice we would need to validate each implementation which is
hard and there will be many non-compliant implementations around and
no way to know which is which.

IF this is a requirement then extending the spec to include a direct
representation of binary exponent floating point is the only way to be
sure it can be relied on.

---

From: Stephen Dolan <[email-masked]>
Date: Mon, 4 Aug 2014 13:29:03 +0100

> "All numeric values in I-JSON MUST round-trip to IEEE floating point and back to the same string."

The exact requirement is more subtle than this.

The number 1.1 cannot be represented exactly as an IEEE754 double. The
closest number which can is:

A = 1.100000000000000088817841970012523233890533447265625

The requirements that most languages make of their double conversion
functions (at least, those that bother to specify such things) are:

(1) parsing a string returns the IEEE754 double which is closer to the
decimal number than any other IEEE754 double (with rules for breaking
ties, and rules for strings representing numbers outside [-DBL_MAX,
DBL_MAX])

(2) printing a double returns some string whose decimal value is
closer to the provided IEEE754 double than to any other IEEE754 double
(with similar exceptions for ties, nan, inf, etc.)

Some languages (e.g. Java) go further and require that printing return
the shortest string compatible with (2). Without this condition, there
are many different strings that validly correspond to A above: "1.1"
(the shortest), "1.10", "1.10000000000000009". In general,
print(parse(str)) need not be equal to str, but
parse(print(parse(str))) must be equal to parse(str).

The point of this is that being a valid string representation of an
IEEE754 number has nothing to do with round-tripping to the same
string, and is not a property of the string itself. Instead, the
condition that specifies "use doubles with json" has to do with when
JSON values are considered equivalent.

Two JSON documents differing only in whitespace outside strings are
(MUST be? SHOULD be?) considered equivalent by any implementation. If
you want to specify "use doubles" you should state something similar,
like:

"An implementation [MUST NOT/SHOULD NOT] distinguish between two
numeric literals whose decimal values round to the same IEEE754
double-precision value (using standard round-to-nearest, ties-to-even
rounding)."

Stephen

---

From: John Cowan <[email-masked]>
Date: Mon, 4 Aug 2014 12:45:48 -0400

> The exact requirement is more subtle than this.

Thank you for explaining so clearly what I have been fumbling so badly.

> double-precision value (using standard round-to-nearest, ties-to-even

+1

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
Business before pleasure, if not too bloomering long before.
        --Nicholas van Rijn

---

From: Nico Williams <[email-masked]>
Date: Mon, 4 Aug 2014 11:55:07 -0500

> double-precision value (using standard round-to-nearest, ties-to-even

Is a MUST really needed?  Suppose you have an encoder that produces
numbers (not irrationals) that require better-than-IEEE754 to represent
exactly.  A parser that can only handle IEEE754 should accept such
numbers, but with loss of precision/range -- exactly what we already
knew to expect from JSON as far as interoperability goes.

My concern is that a MUST NOT -even a SHOULD NOT- is too strong and will
either a) cause implementations with better-than-IEE754 numbers to lose
date at encode time even when the next parser in the chain also has
better-than-IEE754 numbers, or b) this will discourage
better-than-IEE754 implementations.  We might end up stuck with IEEE754
for a long, long time.

It should suffice to have a requirement that parsers support at least
IEEE754 (e.g., not 32-bit integers) and a warning that sending numbers
that cannot be represented with IEEE754 may lead to data loss.

Nico
--

---

From: John Cowan <[email-masked]>
Date: Mon, 4 Aug 2014 13:47:49 -0400

> knew to expect from JSON as far as interoperability goes.

A fair point.  We can use the verbal formula employed by Unicode for
normalization, something like this:

   A sending implementation [MUST NOT/SHOULD NOT] expect a receiving
   implementation to distinguish between two numeric literals whose
   decimal values round to the same IEEE754 double-precision value
   (using standard round-to-nearest, ties-to-even rounding).

How's that?

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
A few times, I did some exuberant stomping about, like a hippo auditioning
for Riverdance, though I stopped when I thought I heard something at
the far side of the room falling over in rhythm with my feet.  --Joseph Zitt

---

From: Nico Williams <[email-masked]>
Date: Mon, 4 Aug 2014 13:01:56 -0500

>    (using standard round-to-nearest, ties-to-even rounding).

Perfect.  Make that a MUST NOT.  Do we need a requirement/recommendation
that parsers implement at least IEEE754?

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Mon, 4 Aug 2014 13:24:19 -0700

How is this different/better than what
http://tools.ietf.org/html/draft-ietf-json-i-json-02#section-2.2 currently
says?  I don’t actually disagree with this formulation, but I think that in
practical terms it’s pretty well isomorphic to the current language.

On Mon, Aug 4, 2014 at 10:47 AM, John Cowan <[email-masked]> wrote:

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: Nico Williams <[email-masked]>
Date: Mon, 4 Aug 2014 15:33:19 -0500

> practical terms it’s pretty well isomorphic to the current language.

I'm happy with the text in draft-ietf-json-i-json-02.

Nico
--

---

From: John Cowan <[email-masked]>
Date: Mon, 4 Aug 2014 17:21:12 -0400

Tim Bray scripsit:

> How is this different/better 

It's fine; convergent evolution.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
There is no real going back.  Though I may come to the Shire, it will
not seem the same; for I shall not be the same.  I am wounded with
knife, sting, and tooth, and a long burden.  Where shall I find rest?
                --Frodo