Subject: Re: [Json] JSON Schema Language: int53

From: "Manger, James" <[email-masked]>
Date: Wed, 21 Aug 2019 04:54:54 +0000

Surely int53 is fine for code generation. When you see int53 in a schema, generate a field of type long (in Java).
That alone will not enforce a 2^53 limit, but so what.

When a schema has:
  "numberOfSiblings": "int8"
  "dayOfWeek" : "int8"
It doesn't mean that 120 is a sensible number of brothers & sisters, or -55 can be a day of the week. It just means that the valid values for these fields can be represented with the range offered by int8. Apps still need to do their own validation beyond that.

If you omit int53, then your code generation will never create a long despite this being a widely available & useful type.

--
James Manger

-----Original Message-----
From: json <[email-masked]> On Behalf Of Ulysse Carion
Sent: Wednesday, 21 August 2019 5:06 AM
To: Richard Gibson <[email-masked]>
Cc: Carsten Bormann <[email-masked]>; Pete Cordell <[email-masked]>; JSON WG <[email-masked]>
Subject: Re: [Json] JSON Schema Language: extensibility and unspecified properties

[External Email] This email was sent from outside the organisation – be cautious, particularly with links and attachments.

> IMO, the schema should be about representing what the protocol needs
> rather than how it is represented on a target machine.

This is a reasonable stance. But I don't think it's what I'm trying to
solve for. My focus has continually been on code generation from
schemas; the spec's expressive power is intentionally limited
specifically to enable code generation. Optimizing for code generation
is why I *am* concerning myself with how things are represented on
target machines.

There are lots of good options for schemas languages that are focused
on being as expressive as protocols are complex -- you've written one
of these good options, Pete -- but I've been concerned with making a
schema language as inexpressive as mainstream type systems are simple.

I'm not debating whether there are ways of representing int53 in many
languages. My objection is that I would be making up a type that
doesn't exist in existing programming languages natively or
idiomatically. If someone wants to do int53 bounds checks, they can
always use the float64 type, and then do bounds checking after schema
validation. Or they can write an extension.

---

From: Ulysse Carion <[email-masked]>
Date: Thu, 22 Aug 2019 14:54:00 -0700

> If you omit int53, then your code generation will never create a long despite this being a widely available & useful type.

The issue, in my view, is that int64/uint64 is the widely-available
and useful type. Not int53. To support int53 would require that code
generators implement this psuedo-type at runtime.

Consider that gRPC, which has a canonical representation in JSON and
which also has int64/uint64, unconditionally renders these types as
strings. I feel that this approach is more reasonable, as it removes
the footgun of a datatype that most users will assume is equivalent to
the familiar int64, but is not.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 23 Aug 2019 07:02:46 +0200

> generators implement this psuedo-type at runtime.

I’m not quite sure what “implementing a pseudo-type” means, but is that hard?

> the familiar int64, but is not.

Now you are in the prescriptive area — you no longer describe JSON formats, but you impose your own rules on what these data formats can be.  Not so good. 

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 23 Aug 2019 10:59:13 +0200

On Aug 22, 2019, at 23:54, Ulysse Carion <[email-masked]> wrote:
> int53

Oh, it’s int54, because the separate sign bit gives you one more bit.

  (uint53 / nint53) => int54

(I’m ignoring the fact here that nint53 and thus int54 has one more position at the low end than I-JSON provides unambiguously.
But -2**53 == MININT54 can be represented in Javascript-limited JSON, with the only slightly annoying aspect being that you don’t really know whether it represents -2**53 or -2**53-1.)

Grüße, Carsten

---

From: Ulysse Carion <[email-masked]>
Date: Sat, 24 Aug 2019 11:34:45 -0700

> I’m not quite sure what “implementing a pseudo-type” means, but is that hard?

Not hard. It means creating some sort of type/struct/class that has
ToLong/FromLong as methods, and at runtime checks that the values are
in range.

The problem with this approach (I've tried it, briefly) is that for
real-world code that means you end up needing to jump through these
ToLong/FromLong hoops in order to get work done. If added, this would
be the only such inconvenient hoop to jump through. The fact that
there's this hoop to jump through also reduces the strength of James's
point -- that without int53, you would never codegen "long" from
schemas. In fact, even with int53, you still don't generate "long",
you generate a type that's similar but inconveniently different from
it.

Or, as James suggests, you can generate "long" and not worry about
doing bounds checks. But then you lose the property that if you
generate some datatype X from a schema S, that all instances of X,
when serialized as JSON, satisfy S.

> Now you are in the prescriptive area — you no longer describe JSON formats, but you impose your own rules on what these data formats can be.  Not so good.

I agree. I mentioned this only to clarify what I felt was a better
approach for reconciling our desire for int64_t with the realities of
I-JSON. There is no good way to have something that produces int64_t
in the spec, in my view (gRPC inherited int64 from Protobuf, and
that's why they were forced to be prescriptive). Better to be
unopinionated, and have people implement int53 themselves on top of {
"type": "float64" }. I'd prefer to miss a feature than to include a
wart.

I think that a lot of this conversation might be clarified if I just
publish an I-D with what I have so far. Expect another thread soon.

P.S.: I recognize your point about the type perhaps being better named
"int54", Carsten. For the sake of continuity of conversation, I here
used the "misnomer". Happy to change what we're calling it, as long as
it doesn't confuse anyone.