Subject: [Json] Strengthen interop language on numbers?

From: Tim Bray <[email-masked]>
Date: Thu, 19 Sep 2013 21:24:00 -0700

In section 6, the language asserts that “wide interoperability” can be
achieved by staying within the IEEE754 binary64 bounds.

I think the reason we’re saying that is that we believe that software which
correctly implements IEEE754 is generally available and widely used.  So
maybe we should just say that.  And if we do, we might also point out that,
on that basis, if your numbers are integers and of magnitude < 2**53, you
will achieve complete interoperability when your JSON is processed by such
software.

So maybe rephrase to say:

<new>
This specification allows implementations to set limits on the range of
numbers accepted. While absolute interoperability cannot be guaranteed,
wide interoperability can be achieved by limiting numbers in JSON texts to
those within the precision and magnitude expressible in an IEEE 754:20008
binary64 (double precision) number [IEEE754], because software that
correctly implements that standard is generally available and widely
deployed.  Note that when such software is used, numbers which are integers
and of magnitude less than 2**53 will be interoperable in the sense that
such implementations will agree exactly on the numeric values.

Numeric values that cannot be represented in the grammar above (such as
Infinity and NaN) are not permitted. Attempting to represent numbers that
cannot be exactly encoded as an IEEE 754:2008 binary64 number, such as
1E400, 9007199254740993, or 3.141592653589793238462643383279, may cause
interoperability problems.
</new>

Worth the change?

---

From: John Cowan <[email-masked]>
Date: Fri, 20 Sep 2013 01:26:04 -0400

> interoperability problems.

+1

-- 
Here lies the Christian,                        John Cowan
        judge, and poet Peter,                  http://www.ccil.org/~cowan
Who broke the laws of God                       [email-masked]
        and man and metre.

---

From: "Markus Lanthaler" <[email-masked]>
Date: Fri, 20 Sep 2013 10:49:35 +0200

On Friday, September 20, 2013 6:24 AM, Tim Bray wrote:
> software. So maybe rephrase to say:
> 
> <new>
> numbers accepted. While absolute interoperability cannot be guaranteed, wide
> interoperability can be achieved by limiting numbers in JSON texts to those
> within the precision and magnitude expressible in an IEEE 754:20008 binary64
> (double precision) number [IEEE754], because software that correctly
> implements that standard is generally available and widely deployed.  Note
> that when such software is used, numbers which are integers and of magnitude
> less than 2**53 will be interoperable in the sense that such implementations
> will agree exactly on the numeric values.
> 

+1

--
Markus Lanthaler
@markuslanthaler

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 20 Sep 2013 18:31:01 +0900

+1, except that I would change "precision and magnitude" to "magnitude 
and precision", because it sounds more natural (magnitude is the 
"bigger" of the two things, and so deserves to come first).

Regards,   Martin.

On 2013/09/20 13:24, Tim Bray wrote:
> on that basis, if your numbers are integers and of magnitude<  2**53, you
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json

---

From: Tim Bray <[email-masked]>
Date: Fri, 20 Sep 2013 09:15:50 -0700

Martin, the only way we can settle that issue is in a remote location, at
dawn. Pistols or rapiers?

On Fri, Sep 20, 2013 at 2:31 AM, "Martin J. Dürst"
<[email-masked]>wrote:

> +1, except that I would change "precision and magnitude" to "magnitude and
> precision", because it sounds more natural (magnitude is the "bigger" of
> the two things, and so deserves to come first).
>
> Regards,   Martin.
>
>
>> 1E400, 9007199254740993, or 3.**141592653589793238462643383279**, may
>> cause
>> ______________________________**_________________
>> json mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/json<https://www.ietf.org/mailman/listinfo/json>
>>
>

---

From: Jorge Chamorro <[email-masked]>
Date: Fri, 20 Sep 2013 18:47:39 +0200

On 20/09/2013, at 06:24, Tim Bray wrote:

> numbers which are integers and of magnitude less than 2**53 will be interoperable in the sense that such implementations will agree exactly on the numeric values.

And > -2**53

The interval is (-2**53, 2**53).

But I think it would be better to put it as [(-2**53)+1, (2**53)-1] to emphasize that neither 2**53 nor -2**53 are 'safe', because it's a *very* common mistake to assume that they are.

-- 
( Jorge )();

---

From: Jorge Chamorro <[email-masked]>
Date: Fri, 20 Sep 2013 18:55:19 +0200

On 20/09/2013, at 06:24, Tim Bray wrote:

> expressible in an IEEE 754:20008 binary64 (double precision) number [IEEE754]

There's a typo: 20008 :-)

-- 
( Jorge )();

---

From: "John Levine" <[email-masked]>
Date: 20 Sep 2013 17:01:19 -0000

In article <[email-masked]> you write:
>-=-=-=-=-=-
>-=-=-=-=-=-

This is the IETF.  Cookies.

---

From: John Cowan <[email-masked]>
Date: Fri, 20 Sep 2013 13:46:04 -0400

> dawn. Pistols or rapiers?

I think you mean "Rapiers or pistols?"

-- 
Samuel Johnson on playing the violin:           John Cowan
"Difficult do you call it, Sir?                 [email-masked]
 I wish it were impossible."                    http://www.ccil.org/~cowan

---

From: Jorge Chamorro <[email-masked]>
Date: Fri, 20 Sep 2013 20:02:20 +0200

> But I think it would be better to put it as [(-2**53)+1, (2**53)-1] to emphasize that neither 2**53 nor -2**53 are 'safe', because it's a *very* common mistake to assume that they are.

Oh well, I just realized that you wrote "of magnitude" so mathematically it's right. But I'd use instead "absolute value". And something else to emphasize that neither 2**53 nor -2**53 are 'safe', only the integers in between them.

-- 
( Jorge )();

---

From: Tony Hansen <[email-masked]>
Date: Fri, 20 Sep 2013 16:17:54 -0400

On 9/20/2013 12:24 AM, Tim Bray wrote:
> I think the reason we’re saying that is that we believe that software
> which correctly implements IEEE754 is generally available and widely
> used. So maybe we should just say that. And if we do, we might also
> point out that, on that basis, if your numbers are integers and of
> magnitude < 2**53, you will achieve complete interoperability when
> your JSON is processed by such software.
>
> This specification allows implementations to set limits on the range
> of numbers accepted. While absolute interoperability cannot be
> guaranteed, wide interoperability can be achieved by limiting numbers
> in JSON texts to those within the precision and magnitude expressible
> in an IEEE 754:20008 binary64 (double precision) number [IEEE754],
> because software that correctly implements that standard is generally
> available and widely deployed. Note that when such software is used,
> numbers which are integers and of magnitude less than 2**53 will be
> interoperable in the sense that such implementations will agree
> exactly on the numeric values.
>
> Numeric values that cannot be represented in the grammar above (such
> as Infinity and NaN) are not permitted. Attempting to represent
> numbers that cannot be exactly encoded as an IEEE 754:2008 binary64
> number, such as 1E400, 9007199254740993, or
> 3.141592653589793238462643383279, may cause interoperability problems.
> </new>
>
> Worth the change?

I think this is consistent with what was discussed on the call a few
weeks ago. So +1

s/20008/2008/

Tony Hansen

---

From: "Manger, James H" <[email-masked]>
Date: Sat, 21 Sep 2013 15:18:19 +1000

Jorge,

> But I think it would be better to put it as [(-2**53)+1, (2**53)-1] to
> emphasize that neither 2**53 nor -2**53 are 'safe', because it's a
> *very* common mistake to assume that they are.

Why is that a mistake?
I believe a 64-bit double can exactly represent +/-2**53.
2**53 in 8 bytes is 43 40 00 00 00 00 00 00.

The first integer you cannot represent is 2**53 + 1.

--
James Manger

---

From: Jorge Chamorro <[email-masked]>
Date: Sat, 21 Sep 2013 17:24:24 +0200

> The first integer you cannot represent is 2**53 + 1.

Well yes and no. When you receive a (2**53)-1 you can be sure it's a (2**53)-1 and nothing else, but when you receive a 2**53 it could be both a 2**53 or a (2**53)+1, but you can't tell which one it was.

N bits can represent numbers between 0 and (2**N)-1, but not 2**N because it's one bit too long.

There are 53 bits in the mantissa of an iee754 double, and 2**53 is 54 bits long. What you get when you put a 2**53 in a 754:double is a 2**53 with its last bit dropped, a bit pattern that represents both 2**53 and (2**53)+1.

So it's equal to itself + 1.

If your JSON text happens to contain a 9007199254740993, the receiving end will see it as a 9007199254740992.
If your JSON text happens to contain a 9007199254740992, the receiving can't tell for sure whether it was a 9007199254740992 or a 9007199254740993.

This also happens with (2**53)+2 and (2**53)+3, and with (2**53)+4 and (2**53)+5, etc., all the way up, first in groups of 2, then in groups of 4, etc., but 2**53 is at the frontier between the good and the evil and is often thought to be good, although it's evil.

It's not a safe integer and I think this deserves to be said in the RFC.

Cheers,
-- 
( Jorge )();

---

From: Tim Bray <[email-masked]>
Date: Sat, 21 Sep 2013 09:17:58 -0700

Yeah, best suggestion I’ve heard yet is: “numbers which are integers and in
the range [(-2**53)+1, (2**53)-1] are interoperable in the sense that
implementations will agree exactly on the numeric values”

On Sat, Sep 21, 2013 at 8:24 AM, Jorge Chamorro <[email-masked]>wrote:

> Well yes and no. When you receive a (2**53)-1 you can be sure it's a
> (2**53)-1 and nothing else, but when you receive a 2**53 it could be both a
> 2**53 or a (2**53)+1, but you can't tell which one it was.
>
> There are 53 bits in the mantissa of an iee754 double, and 2**53 is 54
> bits long. What you get when you put a 2**53 in a 754:double is a 2**53
> with its last bit dropped, a bit pattern that represents both 2**53 and
> (2**53)+1.
>
> If your JSON text happens to contain a 9007199254740992, the receiving
> can't tell for sure whether it was a 9007199254740992 or a 9007199254740993.
>
> This also happens with (2**53)+2 and (2**53)+3, and with (2**53)+4 and
> (2**53)+5, etc., all the way up, first in groups of 2, then in groups of 4,
> etc., but 2**53 is at the frontier between the good and the evil and is
> often thought to be good, although it's evil.
>

---

From: Larry Masinter <[email-masked]>
Date: Sat, 21 Sep 2013 11:42:06 -0700

I wonder if it would be helpful to invert the “interoperable” discussion by talking about the interoperability of  “conforming implementations” rather than “interoperable numbers”. That is:

Define “conforming JSON implementation” to be a JSON implementation consisting of

 *   a parser which reads JSON and produces an internal data structure, and
 *   a serializer which takes an internal data structure and creates JSON text
such that serialize(parse(X)) is equivalent to X

 *   using a (defined) equivalence relationship
 *   as long as X uses numbers in the following set (numbers which are integers in range Y or floating point numbers satisfying property Z)

That is, DEFINE what constitutes an interoperable number, don’t try to discover it.
Implementations which have only 12-bit integers are just not conforming implementations since they don’t round-trip everything.

Larry

From: [email-masked] [mailto:[email-masked]] On Behalf Of Tim Bray
Sent: Saturday, September 21, 2013 9:18 AM
To: Jorge Chamorro
Cc: Manger, James H; [email-masked]
Subject: Re: [Json] Strengthen interop language on numbers?

Yeah, best suggestion I’ve heard yet is: “numbers which are integers and in the range [(-2**53)+1, (2**53)-1] are interoperable in the sense that implementations will agree exactly on the numeric values”

On Sat, Sep 21, 2013 at 8:24 AM, Jorge Chamorro <[email-masked]<mailto:[email-masked]>> wrote:

Cheers,
--
( Jorge )();
_______________________________________________
json mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/json

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 23 Sep 2013 08:53:14 -0700

<hat on>

On Sep 21, 2013, at 11:42 AM, Larry Masinter <[email-masked]> wrote:

> 	• a parser which reads JSON and produces an internal data structure, and
> 	• a serializer which takes an internal data structure and creates JSON text
> 	• using a (defined) equivalence relationship
> 	• as long as X uses numbers in the following set (numbers which are integers in range Y or floating point numbers satisfying property Z)
>  

The strong consensus of the WG has been to not add new conformance requirements to the -bis. While your proposal is more definitive than the current wording, it is similar to other conformance proposals that have gone against the WG consensus.

--Paul Hoffman