Subject: [Json] Update Standard to support ECMA-262 BigInt

From: <[email-masked]>
Date: Mon, 26 Apr 2021 14:38:56 +0100

Hello,

I'm writing to you as a programmer for a company that uses massive datasets
and starting to utilize 64bit numbers as our system currently heavily relies
on JSON, RFC 8259 I was wondering if there were any plans to add support for
literal notation of BigInt E.G ` "property":9007199254740991n`

Kind Regards

Martin Barker

Project Systems Manager.

Data Protection Officer for Beaconsoft Limited.

<https://aws.amazon.com/certification/certified-solutions-architect-associat
e/> 

 <https://www.jsclasses.org/award/innovation/> 

6 Time Nominee
4 Time Winner

Level 4

<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.thisis
beacon.com%2F&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823244e3e%7C84df9e7fe
9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637297986116422304&sdata=Xgq1Z7ECM6LEpTYNg
aL0B8aPuOwX3pG3xH2ocWU%2FRJU%3D&reserved=0> www.thisisbeacon.com /
<https://www.linkedin.com/in/martin-barker-7745731a/>
https://www.linkedin.com/in/martin-barker-7745731a/

t: +44 (0) 3302 234 588 / d : +44 (0)113 328 2367 / m: +44 (0) 7736160152

<https://s3.eu-west-2.amazonaws.com/beacon.internal/email-signature-images/E
mail-signature-20201203.png> 

<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbcn.to%2F
gy2&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823244e3e%7C84df9e7fe9f640afb43
5aaaaaaaaaaaa%7C1%7C0%7C637297986116432298&sdata=5xncC2Q8nff6PSAPiuUh6puzoVK
PmBcl8ZtvM3XRC0c%3D&reserved=0> 
Breaking news.... Beacon named as Tech Nation Rising Stars 3.0 Cyber Award
2021 Winners. #WeAreTechNation

Beaconsoft Limited. Registered in England 10209657. VAT number 258 0216
17.Registered Office: 71-75 Shelton Street, Covent Garden, London, UK, WC2H
9JQ.This email is confidential and is intended for the addressee only.
Beaconsoft Limited does not accept responsibility for reliance on this email
by anyone other than the intended addressee. Although we have taken steps to
ensure this email and any attachments are virus-free, we cannot guarantee
this to be the case as it has been sent over a public network. Read our
<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.thisi
sbeacon.com%2Fprivacy-policy%2F&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823
244e3e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637297986116442294&sdata
=BwOQTHmIzMB%2BYkc17ZVv1yoJQ%2FZrK%2BAJlUNK8%2F1VEI8%3D&reserved=0> privacy
policy.

---

From: Pete Cordell <[email-masked]>
Date: Thu, 29 Apr 2021 12:22:41 +0100

As no one else has answered this, my understanding, more as an observer 
than as a key working group member, is that you would have to put such 
large numbers in a string and RFC 8259 doesn't care what you put in 
strings.  Things like dates, durations, IP addresses, geolocations and 
large numbers in strings are beyond its scope.

At some point some work towards a separate RFC on higher level semantics 
for JSON string contents might take place but I'm not aware of any such 
work on the table yet.

HTH,

Pete.
-- 
---------------------------------------------------------------------
Pete Cordell
Codalogic Ltd
C++ tools for C++ programmers, http://codalogic.com
Read & write XML in C++, http://www.xml2cpp.com
---------------------------------------------------------------------
On 26/04/2021 14:38, martin.barker=[email-masked] wrote:
> Hello,
> 
> I’m writing to you as a programmer for a company that uses massive 
> datasets and starting to utilize 64bit numbers as our system currently 
> heavily relies on JSON, RFC 8259 I was wondering if there were any plans 
> to add support for literal notation of BigInt E.G ` 
> "property":9007199254740991n`
> 
> Kind Regards
> 
> **
> 
> *Martin Barker*
> 
> Project Systems Manager.
> 
> <https://aws.amazon.com/certification/certified-solutions-architect-associate/>
> 
> 	
> 
> Innovation award <https://www.jsclasses.org/award/innovation/>
> 
> 	
> 
> 	
> 
> 6 Time Nominee
> 4 Time Winner
> 
> 	
> 
> Level4
> 
> www.thisisbeacon.com 
> <https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.thisisbeacon.com%2F&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823244e3e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637297986116422304&sdata=Xgq1Z7ECM6LEpTYNgaL0B8aPuOwX3pG3xH2ocWU%2FRJU%3D&reserved=0> / 
> <https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbcn.to%2Fgy2&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823244e3e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637297986116432298&sdata=5xncC2Q8nff6PSAPiuUh6puzoVKPmBcl8ZtvM3XRC0c%3D&reserved=0>Breaking 
> news.... Beacon named as Tech Nation Rising Stars 3.0 Cyber Award 2021 
> Winners. #WeAreTechNation
> 
> 17.Registered Office: 71-75 Shelton Street, Covent Garden, London, UK, 
> WC2H 9JQ.This email is confidential and is intended for the addressee 
> only. Beaconsoft Limited does not accept responsibility for reliance on 
> this email by anyone other than the intended addressee. Although we have 
> taken steps to ensure this email and any attachments are virus-free, we 
> cannot guarantee this to be the case as it has been sent over a public 
> network. Read our privacy policy 
> <https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.thisisbeacon.com%2Fprivacy-policy%2F&data=02%7C01%7C%7C7707e043fb5341f5a4f008d823244e3e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637297986116442294&sdata=BwOQTHmIzMB%2BYkc17ZVv1yoJQ%2FZrK%2BAJlUNK8%2F1VEI8%3D&reserved=0>.
> 
> 
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: David Kemp <[email-masked]>
Date: Thu, 29 Apr 2021 17:32:59 -0400

Also as an observer, RFC8259 doesn't care what you put in numbers either,
as long as they are digits with no leading zeroes and an optional
fractional part.

"This specification allows implementations to set limits on the range and
precision of numbers accepted."

If an implementation supports bignums (or even int64s) then nothing in the
JSON
spec prohibits it from doing so.  If the implementation supports only IEEE
754 doubles,
then it's the implementation that needs to be updated, not the standard.
If you're
relying on implementations outside of your control, then putting numbers in
strings
is a workaround to prevent them from getting molested along the way.

That said, applications that handle massive datasets, or at least massive
amounts
of serialization, are probably better off using a serialization designed
for that purpose
such as Protobuf or CBOR.

Dave

---

From: Tim Bray <[email-masked]>
Date: Thu, 29 Apr 2021 16:37:51 -0700

Yeah, speaking as the editor of a bunch of those RFCs, I would very much
like it if JSON could be extended to have comments, timestamp literals, and
typed numbers.  But it's too late. JSON is finished, is generated and
consumed by a very large amount of deployed software which is simply not
going to be updated; changing the syntax at this point would be
catastrophic.   For crypto apps, you pretty well have to put the big
numbers in strings.

Sorry we can't help.

On Thu, Apr 29, 2021 at 2:33 PM David Kemp <[email-masked]> wrote:

> relying on implementations outside of your control, then putting numbers
> in strings

---

From: Pete Cordell <[email-masked]>
Date: Fri, 30 Apr 2021 12:34:04 +0100

Maybe its time to work on something that is truly human friendly and not 
a hand-me-down from some other domain?

Like my little side project: Line Oriented Object Notation - LOON.

It's either stripped down JSON or stripped down YAML depending on your 
point of view.

Essentially line-oriented with "Key : Value" format on each line, but 
objects start as "Key {" and arrays starts as "Key [" (so strings can be 
unquoted and start with "{" or "["). Values are just strings with higher 
layers associating/enforcing semantics such as integer, float, date etc.

And looks like:

# Some fake details about me
com.codalogic.aboutme {
     Name: Pete Cordell
     Height: 178
     DoB: 1969-04-18
     Children [
         {
         Name: Sarah
         Height: 170
         }
         {
         Name: Jenny
         Height: 144
         }
     ]
     Grades [
         A
         B
         C
     ]
     PlaceOfBirth: " string with leading spaces! "
     History <<END
         Born a long time again
         in a galaxy far, far away.
     <<END
}

Less quotation marks than JSON, less complexity than YAML.

ABNF:

loon = preamble [ object-body / object / array ]
        [ eol ]
preamble = *( ( ows / comment ) eol )
comment = ows "#" *not-eol
object-body = object-line *( eol object-line )
object-line = comment / directive / object-member

directive = ows "!" *not-eol

object-member = ows full-name ows value
full-name = [ realm "." ] name
realm = name *( "." name )
name = ALPHA *( ALPHA / DIGIT / "-" / "_" )
value = object / array / multiline-string /
         primitive-spec / null1

object = "{" eol [ object-body eol ] ows "}"

array = "[" eol [ array-body eol ] ows "]"
array-body = array-line *( eol array-line )
array-line = array-member ; Comments not allowed
array-member = ows value

primitive-spec = ":" ows primitive-value
primitive-value = null2 / true / false / number /
                   inline-string

null1 = "" ; Empty member value field indicates null
null2 = "\0"
true = true-kw
false = false-kw

; From RFC8259
number = [ minus ] int [ frac ] [ exp ]
decimal-point = %x2E       ; .
digit1-9 = %x31-39         ; 1-9
e = %x65 / %x45            ; e E
exp = e [ minus / plus ] 1*DIGIT
frac = decimal-point 1*DIGIT
int = zero / ( digit1-9 *DIGIT )
minus = %x2D               ; -
plus = %x2B                ; +
zero = %x30                ; 0

inline-string = naked-string / quoted-string
naked-string = *char
quoted-string = quotation-mark *char quotation-mark
multiline-string = "<<" name eol
                    *( *not-eol eol )
                    *not-eol "<<" name
char = unescaped / escaped
unescaped = HTAB / %x20-5B / %x5D-FF
         ; not controls except TAB nor "\"
         ; N.B: quotation mark is NOT escaped
escaped = escape (
             escape / ; \  i.e.: \\ -> \
             ; N.B. quotation-mark is NOT escaped
             %x62 / ; b  i.e.: \b -> backspace
             %x66 / ; f  i.e.: \f -> form feed
             %x6E / ; n  i.e.: \n -> line feed
             %x72 / ; r  i.e.: \r -> carriage return
             %x74 / ; t  i.e.: \t -> tab
             %x75 (4HEXDIG / "{" 1*6HEXDIG "}")
                  ; \uXXXX or \u{XXXXXX} -> U+XXXX
escape = %x5C              ; \
quotation-mark = %x22      ; "

eol = ows ( CR [ LF ] / LF )
not-eol = HTAB / %x20-FF
ows = *WSP ; Optional white space

;; Keywords
true-kw = %x74.72.75.65 ; "true"
false-kw = %x66.61.6C.73.65 ; "false"

;; Referenced RFC 5234 Core Rules
ALPHA    = %x41-5A / %x61-7A   ; A-Z / a-z
CR       = %x0D         ; carriage return
DIGIT    = %x30-39      ; 0-9
HEXDIG   = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
HTAB     = %x09         ; horizontal tab
LF       = %x0A         ; linefeed
SP       = %x20         ; space
WSP      = SP / HTAB    ; white space

Slightly more at https://github.com/codalogic/LOON if anyone is tempted.

If we started now we could be finished in 10 years time :-)

Pete.
-- 
---------------------------------------------------------------------
Pete Cordell
Codalogic Ltd
C++ tools for C++ programmers, http://codalogic.com
Read & write XML in C++, http://www.xml2cpp.com
---------------------------------------------------------------------
On 30/04/2021 00:37, Tim Bray wrote:
> like it if JSON could be extended to have comments, timestamp literals, 
> and typed numbers.  But it's too late. JSON is finished, is generated 
> and consumed by a very large amount of deployed software which is simply 
> not going to be updated; changing the syntax at this point would be 
> On Thu, Apr 29, 2021 at 2:33 PM David Kemp <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     If an implementation supports bignums (or even int64s) then nothing
>     in the JSON
>     spec prohibits it from doing so.  If the implementation supports
>     only IEEE 754 doubles,
>     then it's the implementation that needs to be updated, not the
>     standard.  If you're
>     relying on implementations outside of your control, then putting
>     numbers in strings
>     That said, applications that handle massive datasets, or at least
>     massive amounts
>     of serialization, are probably better off using a serialization
>     designed for that purpose
>     On Thu, Apr 29, 2021 at 7:22 AM Pete Cordell <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/json>
> 
>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 30 Apr 2021 15:48:19 +0200

On 2021-04-30, at 13:34, Pete Cordell <[email-masked]> wrote:
> LOON

Aptly named :-)

The reason JSON worked was that “it was just JavaScript Object Notation”, which was able to quell all the discussions.
We won’t have that luxury for anything new.

It is also unclear we need another text-based format; YAML has pretty much filled that space (like it or not).

For interchange (where it is more important to have a well-developed specification), with an extensible data model that is based on that of JSON, we have CBOR.

Grüße, Carsten

---

From: Rob Sayre <[email-masked]>
Date: Fri, 30 Apr 2021 12:58:38 -0700

On Thu, Apr 29, 2021 at 4:38 PM Tim Bray <[email-masked]> wrote:

> typed numbers.

Well, if JSON had comments, it would have all sorts of extensions by now,
which is why it doesn't have them. Comments that are actually comments
would be nice, but programmers can't be trusted.

> But it's too late. JSON is finished, is generated and consumed by a very
> large amount of deployed software which is simply not going to be updated;
> changing the syntax at this point would be catastrophic.
>

Yes.

> For crypto apps, you pretty well have to put the big numbers in strings.
>

JSON numbers interchanged between languages that have arbitrary-precision
numbers work fine. But, JS BigInt (original post) operations aren't
constant time, so you shouldn't use them for crypto anyway [1].

Put the big numbers in strings if you think JavaScript might be in the
picture. For many other languages, you don't need to. It should also be
possible to detect number precision in a JSON encoder/decoder if really
desired.

thanks,
Rob

[1]
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt#cryptography
(see also the suggestion for JSON stringifying below)

---

From: Tim Bray <[email-masked]>
Date: Fri, 30 Apr 2021 13:07:13 -0700

> constant time, so you shouldn't use them for crypto anyway [1].

Um, some popular JSON libraries in languages that can handle big numbers
fail to do the right thing.  You're really asking for trouble if you put
anything that can't fit in a IEEE double.  *sadface*

---

From: Rob Sayre <[email-masked]>
Date: Fri, 30 Apr 2021 13:20:41 -0700

> anything that can't fit in a IEEE double.  *sadface*

Definitely agree that quality-of-implementation varies, but lots of popular
stuff gets this right:

$ python

Python 3.8.5 (default, Jan 27 2021, 15:41:15)

[GCC 9.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> num_128 = 170141183460469231731687303715884105727

>>> import json

>>> json.dumps(num_128)

'170141183460469231731687303715884105727'

>>> json.loads(json.dumps(num_128))

170141183460469231731687303715884105727

>>> json.loads(json.dumps(num_128)) == num_128

True

I also like Rust's serde-json approach here. The number precision is
enforced by the decoder. Here's a u16 example:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=804012796c9a88a7a2252cb6ccab25eb

In other words, I agree that a string is the best idea if you're not sure
where the JSON is headed, but it works really well often enough. In fact,
letting it break might be desirable, especially if you can enforce
precision like serde-json does.

thanks,
Rob