# LispKit JSON

Library `(lispkit json)` defines an API for representing, querying, and manipulating generic JSON values. The framework includes:

- A representation for mutable and immutable JSON values as defined by RFC 8259
- Functionality for creating and manipulating JSON values including support for reading and writing JSON data
- An implementation of JSON Pointer as defined by RFC 6901 for locating values within a JSON document
- An implementation of JSON Path as defined by RFC 9535 for querying JSON data
- An implementation of JSON Patch as defined by RFC 6902 for mutating JSON data
- An implementation of JSON Merge Patch as defined by RFC 7396 for merging JSON data with JSON patches

## JSON values

Library `(lispkit json)` provides an abstract data type `json` encapsulating potentially very large JSON values. A JSON value has one of the following six types:

- null: the “empty” value
- boolean: representing `#t` and `#f`
- number: either an integer (fixnum) or a floating-point number (flonum)
- string: a sequence of unicode characters
- array: a fixed length sequence of JSON values
- object: a collection of name/value pairs

Library `(lispkit json)` implements a rich API for creating, accessing, and transforming JSON values. There are both mutable and immutable JSON values.

json-type-tag [constant]

Symbol representing the `json` type. The `type-for` procedure of library `(lispkit type)` returns this symbol for all JSON values.

(json? obj) [procedure]
(json? obj strict?)

Returns `#t` if obj is a mutable or immutable JSON value; `#f` otherwise. If argument strict? is provided and set to true, then `#t` is returned only if obj is an immutable JSON value. For checking if a given object is a mutable JSON value, procedure `mutable-json?` can be used.

(json-null? obj) [procedure]

Returns `#t` if obj corresponds to the JSON null value; `#f` otherwise.

(json-boolean? obj) [procedure]

Returns `#t` if obj is a boolean JSON value; `#f` otherwise.

(json-number? obj) [procedure]

Returns `#t` if obj is a numeric JSON value, i.e. an integer or floating-point number. Otherwise, `#f` is returned.

(json-string? obj) [procedure]

Returns `#t` if obj is a JSON string value; `#f` otherwise.

(json-array? obj) [procedure]

Returns `#t` if obj is a JSON array; `#f` otherwise.

(json-object? obj) [procedure]

Returns `#t` if obj is a JSON object; `#f` otherwise.

(json) [procedure]
(json x)
(json x …)

Procedure `json` maps Scheme data structures to immutable JSON values. If no argument is provided, then the JSON null value is returned. If one argument x is provided, then x is mapped to a JSON value following the rules stated below. If more than one arguments x … is provided, then a JSON array is returned whose elements have been created by mapping the corresponding argument.

The following mapping rules are being used for regular Scheme values x:

- The symbol `null` is mapped to the JSON null value
- `#f` and `#t` are mapped to the corresponding JSON boolean values
- Fixnum values are mapped to corresponding JSON integer values
- Flonum values are mapped to corresponding JSON floating-point values
- Symbols (other than `null`) are mapped to JSON string values
- Vectors and growable vectors are mapped to JSON arrays
- Association lists are mapped to JSON objects; association lists need to have the form `(("key1" . value1)("key2" . value2) ...)` or `((key1 . value1)(key2 . value2) ...)`.
- Instances of record types are mapped to JSON objects where each record field and value correspond to a key and mapped JSON value
- Hashtables are mapped to a corresponding JSON object if all keys can be mapped to JSON object keys (symbols and strings) and values can be mapped according to these rules
- JSON values map to itself
- Mutable JSON values map to a corresponding immutable JSON value
- For all other Scheme values, an error is signaled

The inverse mapping is implemented by procedure `json->value`.

(json-object (name value) …) [syntax]

Creates a JSON object with the given name/value pairs. name should be a symbol and value is an expression that evaluates to a value that can be converted to JSON. This is a syntax form that expands to a call to `json` with an association list.

    (json-object (name "John") (age 30) (city "New York"))
    ⇒  #json-object with members: name, age, city

(make-json-array len) [procedure]
(make-json-array len default)

Returns a new immutable JSON array of length len. If JSON value default is provided, then it is used as the default element value of the new array. Otherwise, all elements of the new JSON array are set to the JSON null value.

(json=? json …) [procedure]

Returns `#t` if all JSON values json … are structurally equivalent; otherwise `#f` is returned.

(json-refinement? a b) [procedure]

Returns `#t` if JSON value a is a refinement of JSON value b; otherwise, `#f` is returned. a is a refinement of b if

1. Both a and b are JSON values of the same type,
2. If a and b are arrays, they have the same length n and ai is a refinement of bi holds for every i ∈ [0; n[,
3. If a and b are objects, for every member m of b with value bm, there is a member m of a with value am such that am is a refinement of bm,
4. For all other types, a and b are the same.

This relationship intuitively models that whenever it is possible to read a value at a given location from b, it is also possible to read a value at the same location from a and the value that is read for a is a refinement of the value read from b. The following example showcases this relationship:

    (define a
      (string->json "{ \"a\": [1, { \"b\": 2 }], \"c\": { \"d\": [{}] }}"))
    (define b
      (string->json "{ \"a\": [1, { \"b\": 2, \"e\": 4 }], \"c\": { \"d\": [{\"f\": 5}] }}"))
    (json-refinement? a b)  ⇒  #f
    (json-refinement? b a)  ⇒  #t

(string->json str) [procedure]

Returns a JSON value for the data structure represented in string str.

(bytevector->json bvec) [procedure]
(bytevector->json bvec start)
(bytevector->json bvec start end)

Decodes the given bytevector bvec between start and end and returns a JSON value for the encoded value. If is an error if bvec between start and end does not represent a JSON value encoded in a UTF8-encoded string. If end is not provided, it is assumed to be the length of bvec. If start is not provided, it is assumed to be 0.

(cbor->json bvec) [procedure]
(cbor->json bvec start)
(cbor->json bvec start end)

Decodes a CBOR (Concise Binary Object Representation) encoded value from bytevector bvec between start and end and returns the corresponding JSON value. CBOR is a binary data format defined in RFC 8949. If end is not provided, it is assumed to be the length of bvec. If start is not provided, it is assumed to be 0.

(load-json path) [procedure]

Loads a text file at path, parses its content as JSON and returns it as a JSON value.

(json-members json) [procedure]

If json represents a JSON object, then `json-members` returns a list of all members of this object. Each member is represented as a symbol. For all other JSON values, `json-members` returns an empty list.

(json-member? obj member) [procedure]

Returns `#t` if the JSON value obj is an object and has a member with the given member name (a string or symbol); `#f` otherwise.

(json-member json member) [procedure]
(json-member json member default)
(json-member json members)
(json-member json members default)

Returns the value of the given member (a string or symbol) in JSON object json. If member is a list of member names, follows the path through nested JSON objects. If the member is not found, returns default, or `#f` if default is not provided.

    (define x (string->json "{ \"a\": { \"b\": 42 } }"))
    (json-member x 'a)      ⇒  ((b . 42))
    (json-member x '(a b))  ⇒  42
    (json-member x "c" 99)  ⇒  99

(json-children json) [procedure]

Returns all the direct children of json as a list. For null, boolean, numeric and string values, an empty list is returned. For JSON arrays, the array itself is returned. For JSON objects, the values of the object (without their corresponding keys) are returned in an undefined order.

(json-children-count json) [procedure]

For JSON objects, `json-children-count` returns the number of members. For JSON arrays, `json-children-count` returns the length of the array. For all other JSON types, zero is returned.

(json-ref json ref …) [procedure]

Applies JSON references ref … sequentially to json and returns the result of the last application. `(json-ref x r1 ... rn)` is equivalent to `(json-ref ... (json-ref (json-ref x r1) r2) ... rn)`.

(json-replace json ref1 v1 ref2 v2…) [procedure]

Within json, replaces the value at JSON reference ref1 with v1 followed by replacing the value at JSON reference ref2 with v2 etc. This procedure does not mutate json returning a new JSON value with the changes applied.

(json-replace-all json updates) [procedure]

Applies update list updates to json. An update list is a list of pairs consisting of a JSON reference and a JSON value. The values replace the content at the given reference. They are applied in order. This procedure does not mutate json returning a new JSON value with the changes applied.

(json->value json) [procedure]

Converts a JSON value into corresponding Scheme values. This procedure implements the inverse mapping of procedure `json`, i.e. null values are mapped to the symbol `null`, boolean values are mapped to `#t` and `#f`, numbers are mapped to fixnum and flonum values, strings are mapped to regular Scheme strings, arrays are mapped to immutable vectors, and objects are mapped to association lists with symbols representing the member names.

(json->string json) [procedure]
(json->string json pretty?)
(json->string json pretty? sort?)
(json->string json pretty? sort? slash?)

Returns a string representation of json. The output is pretty-printed if pretty? is provided and set to true. If sort? is provided and set to true, the members of an object are printed in sorted order allowing for a deterministic output. If slash? is provided and set to true, slashes get escaped in strings, allowing outputted JSON to be safely embedded within HTML/XML.

(json->bytevector json) [procedure]
(json->bytevector json pretty?)
(json->bytevector json pretty? sort?)
(json->bytevector json pretty? sort? slash?)

Returns a bytevector of a UTF8-encoded string representation of json. The output options pretty?, sort?, and slash? correspond to the options of procedure `json->string`.

(json->cbor json) [procedure]

Returns a bytevector containing the CBOR (Concise Binary Object Representation) encoding of json. CBOR is a binary data format defined in RFC 8949 that provides a compact representation of JSON-like data structures.

(json-for-each-element f arr) [procedure]

Applies procedure f to every element of JSON array arr. f is a procedure accepting one JSON value as argument.

(json-for-each-member f obj) [procedure]

Applies procedure f to every member of JSON object obj. f is a procedure accepting two arguments: a symbol representing the member name and a JSON value representing the value of the object member.

## Mutable JSON values

mutable-json-type-tag [constant]

Symbol representing the `json` type. The `type-for` procedure of library `(lispkit type)` returns this symbol for all JSON values.

(mutable-json? obj) [procedure]

Returns `#t` if obj is a mutable JSON value; `#f` otherwise.

(mutable-json expr) [procedure]
(mutable-json expr force?)

Procedure `mutable-json` maps Scheme data structures expr to mutable JSON values using the same mapping rules as `json`. This procedure can also be used to turn an immutable JSON value into a mutable JSON value if expr is an immutable JSON value already. The value only gets copied if there are other references to expr. If expr is already a mutable JSON value, then that value is returned by `mutable-json` unless argument `force?` is provided and set to true. In this case, even if expr is a mutable JSON value already, a copy is created and returned.

(json-set! json ref value) [procedure]

Within json, sets the value at JSON reference ref to value, mutating json in place. If json is not a mutable JSON value, then an error is signaled.

(json-append! json ref x …) [procedure]

This procedure appends the JSON values x … to the JSON array at the JSON reference ref of mutable JSON value json. If ref does not refer to an array, no change is made to json and no error is signaled.

(json-insert! json ref index x …) [procedure]

This procedure inserts the JSON values x … in the JSON array at the JSON reference ref of mutable JSON value json. If ref does not refer to an array, no change is made to json and no error is signaled.

(json-remove! json ref …) [procedure]

Removes the values at the JSON references ref … from mutable JSON value json. If a JSON reference does not refer to location where a value can be removed, no change is made and no error is signaled.

## JSON references

### Supported formalisms

Library `(lispkit json)` supports multiple abstractions for referring to values within a JSON document. These abstractions are called JSON references. The most established formalism for referring to a location within a JSON document is JSON Pointer. Here is the complete list of supported JSON references:

- Strings containing a valid JSON Pointer reference as defined by RFC 6901; e.g. `"/store/book/0/title"` is a valid JSON reference.
- Strings containing a valid JSON Location reference. JSON location syntax is based on how values are uniquely identified in JSON Path.
- A fixnum value i refers to the i-th element of a JSON array if i >= 0. If i < 0, then this refers to the n+i-th element assuming n is the length of the array.
- A symbol s refers to member s of a JSON object.
- The empty list `()` refers to the root of a JSON document.
- A list of symbols and integers refers to a sequence of member and array index selections; e.g. `(store book 0 title)`.

### JSON locations

It is recommended to use JSON locations were possible since their semantics is independent of the JSON value they are applied to (unlike JSON Pointer references which have an ambiguous interpretation for their numeric segments).

A JSON location is a path to an element in a JSON structure. Each element of the path is called a segment. The JSON location syntax supports two different forms to express such sequences of segments. Each sequence starts with `$` indicating the “root” of a JSON document. The most common form for expressing the segment sequence is using the dot notation:

    $.store.book[0].title

While accessing an array index is always done using bracket notation, it is possible to also express the access of members of an object using bracket notation as well:

    $['store']['book'][0]['title']

Is is also possible to mix the dot and bracket notation. Dots are only used before property names and never together with brackets:

    $['store'].book[-1].title

The previous example also shows the usage of negative indices, which are interpreted as offsets from the end of arrays with -1 referring to the last element.

### API

(json-location? str) [procedure]

Returns `#t` if string str contains a valid JSON location specification; `#f` otherwise.

(json-pointer? str) [procedure]

Returns `#t` if string str contains a valid JSON pointer specification; `#f` otherwise.

(json-reference? obj) [procedure]

Returns `#t` if obj is a valid JSON reference; i.e. it is either:

- A string containing a valid JSON Pointer reference,
- A string containing a valid JSON Location reference,
- A fixnum value i referring to the i-th element of a JSON array if i >= 0, or to the n+i-th element if i < 0 and n being the length of the array,
- A symbol s referring to member s of a JSON object,
- A list of symbols and integers referring to a sequence of member and array index selections.

(json-self-reference? obj) [procedure]

Returns `#t` if obj is a valid JSON reference that refers to the root of a JSON document; `#f` otherwise.

(json-location ref) [procedure]

Returns a string with a JSON location representation of the JSON reference ref.

(json-pointer ref) [procedure]

Returns a string with a JSON pointer representation of the JSON reference ref.

    (json-pointer "$.store.book[0].title")
    ⇒  "/store/book/0/title"

(json-reference-segments ref) [procedure]

Returns a list of symbols and integers representing the sequence of segments for the given JSON reference ref.

    (json-reference-segments "$.store.book[0].title")
    ⇒  ("store" "book" 0 "title")
    (json-reference-segments "/store/book/0/title")
    ⇒  ("store" "book" 0 "title")
    (json-reference-segments '(a -1 c 2))
    ⇒  ("a" -1 "c" 2)

## JSON Path

The full JSON Path standard as defined by RFC 9535 is supported by library `(lispkit json)`. JSON Path queries are simply represented as strings. They can be applied to JSON values with procedures `json-query`, `json-query-results`, and `json-query-locations`. To illustrate the usage of JSON Path queries, the following JSON value is being defined:

    (define jval (string->json (string-append
      "{ \"store\": {\n"
      "   \"book\": [\n"
      "     { \"category\": \"reference\",\n"
      "       \"author\": \"Nigel Rees\","
      "       \"title\": \"Sayings of the Century\",\n"
      "       \"price\": 8.95 },\n"
      "     { \"category\": \"fiction\",\n"
      "       \"author\": \"Evelyn Waugh\",\n"
      "       \"title\": \"Sword of Honour\",\n"
      "       \"price\": 12.99 },\n"
      "     { \"category\": \"fiction\",\n"
      "       \"author\": \"Herman Melville\",\n"
      "       \"title\": \"Moby Dick\",\n"
      "       \"isbn\": \"0-553-21311-3\",\n"
      "       \"price\": 8.99 },\n"
      "     { \"category\": \"fiction\",\n"
      "       \"author\": \"J. R. R. Tolkien\",\n"
      "       \"title\": \"The Lord of the Rings\",\n"
      "       \"isbn\": \"0-395-19395-8\",\n"
      "       \"price\": 22.99 }\n"
      "   ],\n"
      "     \"bicycle\": {\n"
      "     \"color\": \"red\",\n"
      "     \"price\": 399\n"
      "   }\n"
      " }\n"
      "}")))

Now a JSON Path query `$.store.book[?@.price < 10].title` can be applied to `jval` via procedure `json-query` to extract pairs of matching JSON values and the corresponding JSON references:

    (json-query jval "$.store.book[?@.price < 10].title")
    ⇒  ((#<json "Moby Dick"> "store" "book" 2 "title")
        (#<json "Sayings of the Century"> "store" "book" 0 "title"))

(json-path? str) [procedure]
(json-path? str strict?)

Returns `#t` if string str constitutes a valid JSON query string; `#f` otherwise. If argument `strict?` is provided and set to false, the syntax and semantics of the JSON Path string is slighly relaxed. For instance, non-singular queries are supported in query filters.

(json-path-singular? str) [procedure]
(json-path-singular? str strict?)

Returns `#t` if string str constitutes a valid singular JSON query string; `#f` otherwise. A JSON query is singular if it refers to exactly one location in any JSON document. If argument `strict?` is provided and set to false, the syntax and semantics of the JSON Path string is slighly relaxed. For instance, non-singular queries are supported in query filters.

(json-query json query) [procedure]

Applies JSON Path query to json returning the values matching the query together with the corresponding locations of the matching values. `json-query` returns a list of pairs of JSON values and JSON locations.

(json-query-results json query) [procedure]

Applies JSON Path query to json returning the values matching the query in a list.

(json-query-locations json query) [procedure]

Applies JSON Path query to json returning the locations of matching values in a list.

## JSON Patch

JSON Patch defines a JSON document structure for expressing a sequence of operations to apply to a JSON document. Each operation mutates parts of the JSON document. The supported operations specified by RFC 6902 are represented by the following 6 shapes of lists:

- (add ref json): Add json to the JSON value the JSON pointer ref is referring to
- (remove ref): Remove the JSON value at the location the JSON pointer ref is referring to
- (replace ref json): Replace the value at the location the JSON pointer ref is referring to with `json`
- (move from to): Move the value at the location at which the JSON pointer `to` is referring to with the value at `from`. This is equivalent to first removing the value at `from` and then adding it to `path`.
- (copy from to): Copy the value at the location at which the JSON pointer `to` is referring to with the value at `from`. This is equivalent to first looking up the value at `from` and then adding it to `path`.
- (test ref json): Compares value at `ref` with `json` and fails if the two are different.

From lists of operations it is possible to construct JSON patch objects, which are mutable containers for sequences of operations. They are created with procedure `json-patch`:

    (define jp (json-patch
      `((test "/a/b/c" ,(json "foo"))
        (remove "/a/b/c")
        (add "/a/b/c" ,(json "foo" "bar"))
        (replace "/a/b/c" ,(json 42))
        (move "/a/b/d" "/a/b/c")
        (copy "/a/b/e" "/a/b/d"))))

A more conventional approach would be to define the JSON patch operations in JSON directly and using again procedure `json-patch`:

    (define jp2 (json-patch (string->json (string-append
      "["
      " { \"op\": \"test\", \"path\": \"/a/b/c\", \"value\": \"foo\" },"
      " { \"op\": \"remove\", \"path\": \"/a/b/c\" },"
      " { \"op\": \"add\", \"path\": \"/a/b/c\", \"value\": [ \"foo\", \"bar\" ] },"
      " { \"op\": \"replace\", \"path\": \"/a/b/c\", \"value\": 42 },"
      " { \"op\": \"move\", \"from\": \"/a/b/c\", \"path\": \"/a/b/d\" },"
      " { \"op\": \"copy\", \"from\": \"/a/b/d\", \"path\": \"/a/b/e\" }"
      "]"
    ))))
    (json-patch=? jp jp2)  ⇒  #t

JSON patch objects can be applied to mutable JSON values with procedure `json-apply!`.

json-patch-type-tag [constant]

Symbol representing the `json-patch` type. The `type-for` procedure of library `(lispkit type)` returns this symbol for all JSON patch objects.

(json-patch? obj) [procedure]

Returns `#t` if obj is a JSON patch object; `#f` otherwise.

(json-patch) [procedure]
(json-patch expr)

Returns a new JSON patch object. JSON patch objects are mutable containers for lists of operations. If no argument is provided to `json-patch`, then an empty JSON patch object is returned which can be extended via procedure `json-patch-append!`. If expr is provided, it is either an already existing JSON patch object and a copy is returned, or a JSON value representing the JSON patch operation list according to RFC 6902, or it is a list of operations. Each operation has one of the following 6 shapes:

- (add ref json): Add json to the JSON value at ref
- (remove ref): Remove the JSON value at ref
- (replace ref json): Replace the value at ref with `json`
- (move from to): Move the value at `from` to `to`
- (copy from to): Copy the value at `from` to `to`
- (test ref json): Compares value at `ref` with `json` and fails if the two are different.

JSON pointer is used for specifying references to values within a larger JSON document.

(json-patch-clear! patch) [procedure]

Removes all operations from the JSON patch object patch.

(json-patch-append! patch oper …) [procedure]

Appends operations oper to the JSON patch object patch. Each operation has one of the following 6 shapes: (add ref json), (remove ref), (replace ref json), (move from to), (copy from to), and (test ref json).

(json-patch=? patch …) [procedure]

Compares the JSON patch objects patch … and return `#t` if all patch objects are equivalent; otherwise return `#f`.

(json-patch->list patch) [procedure]

Returns the list of operations for JSON patch object patch.

(json-patch->json patch) [procedure]

Returns a JSON value representing the operations of the JSON patch object patch as defined by RFC 7396.

(json-apply! json patch) [procedure]

Applies a JSON patch object patch to mutable JSON value json using application rules as defined by RFC 7396.

## Merging JSON values

(json-merge json other) [procedure]

Merges the JSON value json with the JSON value other such that the result res of the merge is the “smallest” JSON value that is a refinement of both json and other; i.e. both `(json-refinement? res json)` and `(json-refinement? res other)` hold. If such a merged value does not exist, then `json-merge` will return `#f`.

This approach is also called a symmetrical merge. Intuitively, it combines two JSON values by adding all non-existing values to the merged value and merging overlapping values or failing whenever a symmetrical merge is not possible.

(json-override json other) [procedure]

Merges the JSON value json with the JSON value other using merge semantics which let other override values of json whenever merging as implemented by `json-merge` would fail otherwise. As opposed to procedure `json-merge`, combining arrays does not require the arrays to be of the same length. The resulting array has always the length of the longest of the two arrays and individual elements are combined using `json-override` whenever two elements are available.

(json-merge-patch json patch) [procedure]

Merges the JSON value json with the JSON value patch using the rules defined by RFC 7396. The merged JSON value is returned.

The patch value describes changes to be made to json using a syntax that closely mimics the document being modified. Recipients of a merge patch value determine the exact set of changes being requested by comparing the content of the provided patch against the current value json. If the provided patch value contains members that do not appear within json, those members are added. If json does contain the member, the value is replaced. Null values in patch are given special meaning to indicate the removal of existing values in the target.