Friday, 25 November 2011

Linking in JSON

To be a full-fledged format on the Web, you need to support links -- something sorely missing in JSON, which many have noticed lately.

In fact, too many; everybody seems to be piling on with their own take on how a link should look in JSON. Rather than adding to the pile (just yet), I thought I'd look around a bit first.

What am I looking for? Primarily, a way to serialise typed links (as defined by RFC5988, "Web Linking") into JSON, just like they can be into Atom, HTTP headers and (depending on the HTML5 WG's mood today), HTML.

5988 isn't perfect by any stretch (primarily because it was an after-the-fact compromise), but it does sketch out a path for typed links to become a first-class, format-independent part of the Web -- as they well should be, since URIs are the most important leg holding up the Web.

My immediate use case is being able to generically pull links out of JSON documents so that I can "walk" an HTTP API, as alluded to previously.

I'm also going in with a bit of caution, because we have at least one proof that getting a generic linking convention to catch on is hard; see XLink.

Now, to the contenders.

JSON-LD: JSON for Linking Data

JSON-LD is a JSON format with a Linked Data (nee: Semantic Web) twist.

{
  "@context": "http://purl.org/jsonld/Person",
  "@subject": "http://dbpedia.org/resource/John_Lennon",
  "name": "John Lennon",
  "birthday": "10-09",
  "member": "http://dbpedia.org/resource/The_Beatles"
}

Obviously, if you want to fit RDF into JSON, this is what you'd be looking at. Which is great, but most of the developers in the world aren't (yet) interested in this (no matter how hard the proponents push for it!). It also fails to provide a mapping from 5988; where do I put the link relation type?

I've seen a fair bit of advocacy for JSON-LD, especially on Twitter, but in almost every instance, I've seen the non-believers push back.

JSON Reference

If JSON-LD is too complex / high-level, its opposite would be JSON Reference, a new-ish Internet-Draft by Chris Zyp and Paul Bryan (who are also working on JSON Schema, JSON Pointer and JSON PATCH, currently being discussed in the APPSAWG).

It's effectively a one-page spec, where a link looks like:

{ "$ref": "http://example.com/example.json#/foo/bar" }

This is effectively static serialisation of a type they've defined in JSON Schema, a sort of "meta-schema for links." I'd previously pushed back on that, because it effectively requires schema support / understanding to get the links out of the document -- a real non-starter in many scenarios.

So, I like the concreteness. However, it still lacks any way to talk about relation types, or pop on other metadata; while this could be grafted on separately, the whole point is to have one way to do it.

HAL - Hypertext Application Language

Another attempt is HAL, by Mike Kelly. The JSON portion of his serialisation looks like this:

{
  "_links": {
    "self": { "href": "/orders" },
    "next": { "href": "/orders?page=2" },
    "search": { "href": "/orders?id={order_id}" }
  }
}

Here, links are an object whose members are link relations (yay!).

I'm a bit concerned, however, that this object might be a bit awkward to drop into some formats; it relies on _links to identify the structure, so in some places, you'd need a two-level deep object to convey just a simple link.

Also, there doesn't appear to be any way to link to more than one URI of a given relation type.

What I'd Really Like

I don't have a specific thing in mind yet, and it's entirely possible that any of these proposals could be adapted to my needs (or others, I'm sure that they're out there).

I do have some requirements for consideration, though, along with a few sketches of ideas.

Discoverable

It should be really easy to find links in a document; as discussed above, requiring use of a schema is a non-starter.

This means that there needs to be some sort of marker the link data structure to trigger the link semantics (e.g., how JSON Reference uses "$ref"), and ideally some way to indicate on the document itself that it's using that convention (to avoid collisions, and help processors find these documents).

JSON Reference does this with a media type parameter;

Content-Type: application/json; profile=http://json-schema.org/json-ref

While at a glance that seems reasonable, I have two concerns; first, that JSON itself doesn't define a profile parameter on the media type (this needs to be done properly), and more seriously, that you can't declare conformance to multiple such conventions using this mechanism.

For example, if I want to say that my JSON conforms to this convention on links, and another convention about (say) namespaces, I'm out of luck.

I was in Boston recently (for the OpenStack design summit), and during a lull went up to the W3C offices to have lunch. Having this very much on the mind, I asked TimBL for his take, and we sketched out a sort of JSON metadata container, something like:

{
  "_meta": {
    "json-linking": null,
    "json-namespaces": ["foo", "bar", "baz"]
  },
  // … rest of doc here
}

The exact format and content isn't important here; the idea is having a controlled way (i.e., the keys would probably be in a registry somewhere, and/or URIs) of adding annotations to a JSON document about its contents.

This is not an envelope; it's just a document metadata container, much like HEAD in HTML. You could put links up there too, I suppose, if it'd help. The important part is that you'd know to look for one of *those* because the media type of the document (or one of its parameters, if we go that way) indicates it's in use.

What do people think? Is this JSONic enough (whatever that means)?

Self-Contained

As mentioned in the discussion of HAL above, the link convention needs to be easy to insert in a format, and not be too convoluted. This probably means an object that's "marked" with a particular, reserved key, much as JSON Reference does it. A list of links then becomes an array of objects, which seems pretty natural.

Mappable to RFC5988

Again, as discussed, a mapping to RFC5988 is important to me -- both so that links can be serialised in various formats, with reasonable fidelity, and so that we can pivot from talking about "RESTful" APIs in terms of URIs to talking about them in terms of formats and link relations, as Roy advocates:

A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types).

Extensible

The object defined needs to be explicitly extensible by the format it's in, so that link-specific metadata can be added. See the discussion of namespaces in JSON.

Anchorable

A complement to linking in JSON is linking to JSON. While JSON Pointer looks really promising in this regard, and is getting a fair amount of buzz, but I wonder if another mechanism, analogous to xml:id, is necessary.

The use case here is when you want to link to a specific object in a document that may be changing over time; a JSON pointer can be brittle in the face of such change, while a document-unique identifier is much more stable.

This makes a lot of sense in XML, which is primarily a document format. I'm not sure about whether the justification is as strong in JSON, which is primarily a data representation format, but it's worth talking about.

Just One, Please

Again, there's not much value in having fifteen ways to serialise a link in JSON; it will end up a pretty ugly mess.