An IMS Proposal – Eliminate all use of JSON-LD

I sent the following message to IMS because I am really unhappy with IMS use of the JSON-LD in our JSON-based specifications. Apologies in advance to the fans of RDF. We all hoped that JSON-LD would give us the best of both worlds – but it seems like it is the worst of all worlds. I don’t expect to win this argument – because the people making the decisions are not the people writing the code and feeling the unneeded pain caused by JSON-LD.

Hi all,

I would like to formally propose that we no longer use JSON-LD in any IMS specification going forward. I would like to also propose that we formally standardize prefixes for all specifications we have issued that use JSON-LD so implementations can legitimately parse our data models using JSON reliably.

Furthermore we would alter certifications for JSON-LD specs to generate and accept JSON instead of JSON-LD.

My reasoning is that we are far outside the norm of the modern-day REST web services world – and while there are fans of JSON-LD – they are the same folks that loved RDF and just found a new place to push their ideas.

Our standards are one domain of interest and our use of JSON-LD actually tends to create silos of data models. If we compare the JSON-LD for LTI 2.0 and the JSON-LD for ContentItem – they are completely distinct namespaces and things like the “contact” structure – which *should be the same* are actually completely different – and our dysfunctional use of JSON-LD *discourages* the sharing of data model elements between different specifications.

And if you take a look at CASA using JSON Schema – it is even worse. Simple things like contact information again are given completely different data models.

And as I am starting to write code that crosses these spec boundaries boundaries, I am finding that it is far less important to have globally unique identifiers for the notion of a contact data structure – but instead a way to have a contact data structure that we can share and reuse across many specifications.

I think that the right approach is to go straight to a namespaced OO approach to model our underlying data objects and then when we build a new spec and want to pull in the org.imsglobal.shared.Contact object – we just pull in the object and then the JSON serialization is obvious.

As we move away from document-styled specs to API-styled specs – it would seem like we just should move towards defining our interoperable data formats in a way that makes the development of APIs very simple and straightforward instead of wasting so much effort to achieve some dream of future RDF nirvana.

I now have samples of how I model these JSON documents across services – and I can tell you that (a) we are woefully inconsistent across our specs and JSON-LD is partially *causing* the problem and (b) anything that has to do with properly parsing JSON-LD is really poor given the lack of real toolset support and (c) it is frustrating the increasing way that the certification suites are making slightly harder by randomly throwing in JSON-LD just to break those who just want to parse JSON – the solution is to reverse engineer the certification patterns and build lame JSON parsers instead of really using JSON-LD tool chains.

It is high time to walk away from JSON-LD going forward.

Looking forward to your comments.

/Chuck

11 Comments

  1. Kin Lane says:

    My I suggest maybe both efforts existing side by side. With content negotiation, there is no reason users who needed JSON-LD could get what they need, and others could get the JSON, XML, RSS, HTML, or any other representation they need. While JSON-LD may not be preferred in all situations, there are some use cases where linked data may bring value.

  2. Alfred Essa says:

    What is the implication for Caliper?

  3. Kin – that is an interesting idea – but it makes things a lot more complicated if we when we certify applications for interoperability we insist on testing their ability to produce all the variants. Instead of making things easier, we make it five times harder. Every significant programming language / web environment has well supported built-in libraries for JSON – and if there is support for JSON-LD – it is some dude’s github repo which is partial implementation of JSON-LD from a hackfest weekend two years ago that has not had a patch or fix in two years. If you told Amazon or Twitter that they needed to allow you to use any combination of five different formats to talk to their API – they would tell you to take a leap. JSON-LD is a great format for die-hard RDF refugees to play in a modern space. APIs need to me solid and predictable. They need to be the rock upon which distributed applications are built. There is no place for paying homage to RDF when there is no need for it.

  4. Al – Just use JSON. However Caliper wants to use it.

  5. Fred M Beshears says:

    Charles,

    Thanks for sharing this.

    I don’t have a comment on your suggestion that IMS walk away from JSON-LD.

    However, your comments on API style specs caught my attention.

    “As we move away from document-styled specs to API-styled specs – it would seem like we just should move towards defining our interoperable data formats in a way that makes the development of APIs very simple and straightforward instead of wasting so much effort to achieve some dream of future RDF nirvana.”

    Back when I was attending IMS meetings, there was a lot of talk about the IMS Abstract Framework, and the Open Service Interface Definitions (OSIDs) developed by Open Knowledge Initiative (OKI). However, I haven’t heard that much about OSIDs since I retired in 2007. (It may just be that OKI and OSIDs have been going strong since 2007, but I’m just out of touch.)

    Does this move to API-styled specs that you mention mean that some updated version the OSID idea is making a comeback in IMS land?

    Cheers,
    Fred

  6. I sure hope OSIDs are not making a comeback. OSIDs were the most poorly-architected APIs I have ever seen. The right way to achieve interoperability is not to do it through APIs – but instead to define a wire format and protocols using JSON (for example) and then wrap that “wire interaction” in APIs that are purpose built for each language to make developer’s lives as easy as possible. Without a standard for wire format, there is no interoperability. That was one of the many flaws in OSIDs – they started as APIs first in one language and then tried to migrate the same API to other languages with no concern for a “wire format” or the issues present in distributed systems. There are many more flaws of the OSIDs – too numerous to list here in a comment.

  7. Fred M Beshears says:

    Thanks Charles,

    I didn’t think OKI’s OSID idea, as defined back in 2007, had gained much (any) acceptance.

    Thanks for clarifying how the API-style specs you’re talking about would be a significant improvement over OKI’s approach, and what those improvements are.

    Cheers,
    Fred

  8. Charles Hedrick says:

    XML and JSON-LD ignore what we’re learned about protocols through the experience of TCP-IP. Trying to completely specify options makes it difficult to add extensions and to move to new versions. Also, as you note, it adds difficulties to sharing code. I can’t imagine any actual implementor who wants this stuff. Who do I have to write letters to?

  9. Rob Abel says:

    Hi Chuck – Looks like you are asking some good questions here and providing some reasonable alternatives. I sent this over to the smart people to discuss with you. BTW – the next time you send IMS something it might work better if you actually send it to me. I just happened to come across this in twitter by pure chance . . . . or, since you were on the phone with a bunch of us yesterday you might have mentioned it ;-) Best, Rob

  10. Scott says:

    Hi Chuck,

    Thanks for voicing this.

    Any opinion on following specs aligned with https://openapis.org

    -Scott

  11. Anthony Whyte says:

    First, we need to roll back the hyperbole (the reason why should be obvious):

    “. . .while there are fans of JSON-LD – they are the same folks that loved RDF and just found a new place to push their ideas.”

    A demonstrably false claim about the passions and interests behind the creation of JSON-LD. Read: http://manu.sporny.org/2014/json-ld-origins-2/

    Now for the problem:

    “Our standards are one domain of interest and our use of JSON-LD actually tends to create silos of data models. If we compare the JSON-LD for LTI 2.0 and the JSON-LD for ContentItem – they are completely distinct namespaces and things like the “contact” structure – which *should be the same* are acautlly completely different – and our dysfunctional use of JSON-LD *discourages* the sharing of data model elements between different specifications.”

    In other words, it’s not so much about JSON-LD as our (inexpert) use of it. But, then again, it is all about JSON-LD:

    “I am finding that it is far less important to have globally unique identifiers for the notion of a contact data strucutre – but instead a way to have a contact data structure that we can share and reuse across many specifications.”

    And the solution:

    “I think that the right approach is to go straight to a namespaced OO approach to model our underlying data objects and then when we build a new spec and want to pull in the org.imsglobal.shared.Contact object – we just pull in the object and then the JSON serialziation is obvious.”

    The goodness in the above is the idea of a common set of IMS namespaced entities, the badness is the lack of any consideration given to providing a mechanism for ensuring that the terms, concepts and relationships bound up in those entities can be readily expressed and understood when the data is exchanged or shared. JSON-LD provides such a mechanism.

    Data and semantic interoperability is a central concern of Caliper; I think it a mistake to cast JSON-LD aside because its use has exposed weaknesses in the way we organize ourselves to write and evolve specs.

Leave a Reply

*