Subject: [Json] About JSON equality

From: Francis Galiegue <[email-masked]>
Date: Tue, 19 Feb 2013 23:55:24 +0100

The subject has been brought, even though somewhat indirectly, in
another thread on this list, so here I will give a digest of what JSON
Schema came up with.

The definition of JSON value equality, as defined by the core
specification, is as such (link:
http://tools.ietf.org/html/draft-zyp-json-schema-04#section-3.6):

----
Two JSON values are said to be equal if and only if:

      both are nulls; or

      both are booleans, and have the same value; or

      both are strings, and have the same value; or

      both are numbers, and have the same mathematical value; or

      both are arrays, and:

         have the same number of items; and

         items at the same index are equal according to this definition;
         or

      both are objects, and:

         have the same set of property names; and

         values for a same property name are equal according to this
         definition.
----

This definition is only lacking for strings. On some other draft which
I cannot remember off the top of my head, two JSON String values are
to be considered equal if, position wise, their Unicode code points
are the same (and THAT INCLUDES \u0000).

Now, what is the value of defining equality of two JSON values in the
general case (see also my mail about why having to restrict JSON
representations to container instances)?

Of course, there is also the problem that for numeric instances, some
programming languages/parsers cannot guarantee that, since they cannot
reliably represent said instances in a reliable manner, but then this
is why the validation spec explicitly mentions this:
http://tools.ietf.org/html/draft-fge-json-schema-validation-00#section-3.2

About strings and the famous "zero character", this same specification
also says: http://tools.ietf.org/html/draft-fge-json-schema-validation-00#section-3.1

Yes, nothing forbids "\u0000" in a JSON string.

All this to say, should an equality definition find its place in a
potentially revised JSON RFC? Personally, I'm not so sure...

--
Francis Galiegue, [email-masked]
Try out your JSON Schemas: http://json-schema-validator.herokuapp.com

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 19 Feb 2013 23:10:14 +0000

Are you proposing that this be added to 4627bis, or that it be in a
separate document?

Adding it to the existing spec expands that document's scope somewhat,
which we may want to avoid for a variety of reasons.

On 2/19/13 3:55 PM, "Francis Galiegue" <[email-masked]> wrote:

>also says: 
>http://tools.ietf.org/html/draft-fge-json-schema-validation-00#section-3.1
>
>_______________________________________________
>json mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/json
>

-- 
Joe Hildebrand

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 00:19:53 +0100

> Are you proposing that this be added to 4627bis, or that it be in a

That is what I am asking. I believe the more JSON is used, the more
people will want to "compare" JSON values, whatever they may be (and a
hint again: please make RFC 4627bis allow to transfer _any_ JSON
value, not just arrays and objects).

There are two problems here: strings and numbers.

For strings, Unicode code points and their representation in whatever
encoding should be considered equal, and parsers should be able to
handle whatever they are fed with.

For numbers, there is the much more annoying limitation that all
humanly representable numbers just cannot be represented in
programming languages, and that JSON (at least to my eyes) should not
feel compelled to represent NaN (since it is, essentially, language
dependent) nor infinity (since it is an abstract concept and not even
a number).

But both of these problems can be at least formulated as being present
-- the JSON Schema validation spec does. The real question becomes,
will future uses of JSON rely on value equality?

---

From: Nico Williams <[email-masked]>
Date: Tue, 19 Feb 2013 17:37:36 -0600

> are the same (and THAT INCLUDES \u0000).

String equality is definitely a case where normalization does come in.

In practice most Unicode strings are in NFC (normalization form
composed) by accident: it's what input methods typically produce.  But
it doesn't have to be that way.  For some scripts it almost certainly
isn't always that way (I'm thinking of Hangul in particular).  And
then there's the dreaded HFS+, which, for some unknown [to me] reason
that I cannot fathom, normalizes to NFD on create(!).

So, suppose you were creating a JSON representation of file names in a
filesystem.  If that filesystem were HFS+ then you're guaranteed to
have NFD.  If it's any other filesystem (except, perhaps, other
filesystems on OS X, like at least one port of ZFS to it) then chances
are very high that the file names will be in NFC.  This is not really
a contrived example -- I really, really wish it were, but I think it
more than likely that we'll have such uses of JSON.

So, if you want to be able to compare JSON strings for equality, you
do need to specify whether you care about normalization.
Normalization-insensitive string comparison is *not* that expensive
for mostly ASCII strings: the trivial optimization is to have a fast
path that compares characters in logical two-byte windows that move
one byte at a time, and a slow path that gets triggered only whenever
one of those bytes is non-ASCII.  But it's still a PITA.

Nico
--

---

From: "Paul C. Bryan" <[email-masked]>
Date: Tue, 19 Feb 2013 15:38:33 -0800

> are the same (and THAT INCLUDES \u0000).

See
http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-10#section-4.6

Paul

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 19 Feb 2013 16:23:28 -0800

On Feb 19, 2013, at 3:19 PM, Francis Galiegue <[email-masked]> wrote:

> That is what I am asking.

OK, then. I fully disagree.

> I believe the more JSON is used, the more

JSON has been used for many, many years without any strong need for such a comparison semantic in the base spec.

It is fine for there to be a separate standard that is usable by applications that want to compare JSON documents; the rest of the world (and that seems like the large majority) can ignore the comparison document and just use the base spec to understand how to create and consume JSON objects and JSON files.

--Paul Hoffman

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 01:33:49 +0100

On Wed, Feb 20, 2013 at 1:23 AM, Paul Hoffman <[email-masked]> wrote:
[...]
> It is fine for there to be a separate standard that is usable by applications that want to compare JSON documents; the rest of the world (and that seems like the large majority) can ignore the comp…

Point taken. Equality defined in the JSON RFC itself is not the way to
go, and if such an equality is defined, it could be defined in another
document. Now, is there a need for such a document?