Atom and AtomPub were failed

Joe Gregorio’s AtomPub is a failure note is probably not as provocative as its title may make it sound. Joe knows more than the average lunatic out there about AtomPub considering he was co-editor of RFC 5023 and his work at Google so when he says AtomPub may have failed, one is forced to pay attention.

Joe argues that even though AtomPub hasn’t failed as technology, it hasn’t swept the inter-tubes world away as hoped probably because the rationale for creating such a protocol years back doesn’t really pan out any longer.

I share Joe’s slight disappointment at not seeing AtomPub more spread on the web but I don’t entirely agree with the reasons he puts forward:

So why hasn’t AtomPub stormed the world to become the one true protocol? Well, there are three answers:

  • Browsers
  • Browsers
  • Browsers

The world is a different place then it was when Atom and AtomPub started back in 2002, browsers are much more powerful, Javascript compatibility is increasing among them, there are more libraries to smooth over the differences, and connectivity is on the rise.

I’m kind of puzzled, browsers did exist back in 2002, didn’t they? Sure they are more standardized, Javascript is not a shameful programing language like it used to be back in the DHTML days but nonetheless, browsers haven’t fundamentally changed since then. Moreover, through all the years required to release AtomPub nothing prevented the WG in charge of it to reach out for browser vendors. I don’t remember seeing much of them during the long discussions that took place. I find therefore rather harsh to complain at browsers for the fact AtomPub isn’t spread enough. (Note to the Hybi list, don’t make the same potential mistake).

Joe also suggests that XML was overcome by the couple: HTML+JSON. That is very true and Joe acknowledges that he didn’t think this would happen but when he says:

All of the advances in browsers and connectivity have conspired to keep AtomPub from reaching the widespread adoption that I had envisioned when work started on the protocol, but that doesn’t mean it’s a failure.

Conspired? Come on Joe. XMPP certainly suffers from the priority browser vendors have made in the past years but XMPP isn’t built atop HTTP when AtomPub is. I’m sorry Joe but, if anything, the fact Javascript has now so many fantastic libraries make it even simpler to interface with any protocol based on HTTP. Browsers have everything one needs to build an application conversing through AtomPub.

I disagree with Joe’s reasoning and personally I think there is a much bigger issue here to explain the protocol’s difficulties to be a premium choice for public web services.

Let’s review some social network platforms and the way they expose their data.

Wikipedia

Atom: Only used to view global recent changes and utter failure at actually using the format for what it’s good at. In a nutshell, everything is serialized to HTML and packed into the atom:summary element.

API: MediaWiki offers an API to expose and manipulate data which isn’t based on AtomPub but follows quite obviously the same design considering the document’s nature of a wiki. Note that the API supports several output formats but none of which is Atom.

Facebook

Atom: As far as I can tell, Facebook doesn’t offer any support for Atom.

API: Facebook has its own API which doesn’t really map well to AtomPub even though it’s quite often simple CRUD operations.

Twitter

Atom: Twitter exposes friend’s timeline as Atom feeds and does it well. Their API doesn’t support it however.

API: Twitter’s API is also not based on AtomPub.

Last.fm

Atom: No support for Atom.

API: Last.fm’s API doesn’t use AtomPub, well considering they don’t even use Atom itself, it kind of make sense.

Orkut

Atom: Available but not officially supported as per the documentation.

API: Orkut’s API is based on OpenSocial which if I’m not mistaken isn’t based on AtomPub.

MySpace

Atom: Not from  the public website as far as I could tell.

API: MySpace’s API is based on OpenSocial too, thus not based on AtomPub.

Flickr

Atom: Offers Atom feeds for a wide range of their data from their website.

API: Flickr has its own API not based on AtomPub and doesn’t output to Atom.

Google’s various API

Atom: Extensive support for it.

API: Extensively based on AtomPub even though no public support for the AtomPub service document which kind of limits service discovery.

I’m not trying to point fingers at those services, most of them came to life way before AtomPub became seriously defined (and maybe Tim Bray was right to warn the WG it was taken too long) so it’s not really a surprise it’s not used. In most cases though exposed APIs could be mapped to the protocol as well as the Atom format but obviously that would require major and fundamental architectural refactorings which make little economical sense.

It’s worth noting that some services have hard times even exposing data without degrading their values. Atom is a well-defined protocol that offers a simple view on web semantics. Used well, it offers a fantastic ground for mashup of organized and structured data. This is added value I’m wondering if mentioned social network platforms are willing to propose. In some respects Yahoo Pipes might not have to exist if Atom and AtomPub were properly used. Atom is so much more than pure serialization format. Atom can be serialized to JSON and still be Atom. Atom offers a real path to build powerful and meaningful social graph of distributed data.

Ultimately, data is money and I assume it’s a technical issue as much as an economical one. It’s easy to help building the piping for interoperability but it seems harder to actually open up the tap of data.

That being said, as Joe says:

There is still plenty of uses for AtomPub and it has quietly appeared in many places.

Atom and AtomPub aren’t dead but are probably less visible than one might have hoped, used in more traditional places and I suggest you follow Peter Keane to get a feel for it.

Notifixious – A notification platform

I’ve recently registered to Notifixious home page, a notification platform that integrates XMPP natively. Though the application is not looking as polished as one might hope it already provides interesting features.

Overall I think any work towards better notification system is the right idea. We need better ways to notify people or get notified about relevant piece of information. From a technology’s perspective transport protocols (well at the application’s level) already exist to perform the task of carrying the information, they are namely HTTP and XMPP. However there is still quite a large gap to fill about what to carry exactly. Notifixious doesn’t solve that gap per se but offers at least a good base for playing around with some floating ideas. For instance I’ve introduced today the Notifixious’s crowd to LLUP which has been initiated and carried on for a few years now to answer that specific problem. Perhaps it’ll make it way through.

Update: The guys at Notifixious just posted a message about how PubSub is heavily used by their service.

jlib preview – PyQt4 library for XMPP

I’ve been recently working on a library called jlib that providing PyQt4 objects and widgets that can be integrated to a PyQt4 application. In other words jlib is not a new Jabber client but a toolbox to enjoy the benefit of XMPP. The XMPP work is performed by headstock, jlib only glues headstock to PyQt4 through the use of signals/slots.
jlib is not ready yet but here is a preview of a few widgets I’ve already started working on. Ultimately my main interests is in creating a decent toolbox of widgets centered towards XMPP PubSub.

XMPP, AtomPub, headstock, amplee and the BBC

Matthew Wood, from the BBC Radio Labs, posted a few days ago an exciting note regarding fun he was having with XMPP and services such as last.fm to inform user, via XMPP messages, of BBC radios broadcasting music they might like based on thier last.fm profile. I thought this was fantastic but I was even more excited when I read another note where he explained that he was using PubSub as a mean to carry and distribute metadata about BBC shows using Atom entries as the metadata format.
This evening I spent three hours expanding on the simple chat example coming with headstock to talk with the PubSub service. Then I integrated amplee as a way to offer an AtomPub interface at the same time. This means that when the demo starts both a XMPP client, connecting to Matt’s service, and an AtomPub server, using amplee and served by CherryPy, are started. The XMPP client asks the server about PubSub nodes. For each node representing BBC channels I create an atompub collection within its own workspace. Simultaneously I subscribe to those nodes. I then ask the XMPP server for items belonging to those nodes and for each item, representing metadata about a show for instance, I create an atom entry that I store within the AtomPub store.
This means one can then simply subscribe with a feed reader to a given collection and/or a XMPP PubSub node. All of this happening on the fly starting from an empty AtomPub service document.
Eventually I will add support so that when an Atom entry is POSTed to a AtomPub collection, the according PubSub stanza is pushed towards the service (I doubt Matt’s service accepts it though) allowing for microblogging support.
The source code of the example can be found here. If you want to understand how it works you might want to read this quick word I wrote about Kamaelia first which is at the core of headstock.

AtomPub and XMPP

There has been quite a lot of discussions around the use of XMPP in a web context and how to blend HTTP and XMPP protocols for a better social network experience.

I’ve been working on implementing XMPP for a while now on my spare time and even though it’s been a much slower effort I would have liked, I’ve been able to have a fantastic fun with it. Most recently I’ve started integrating XMPP PubSub along with AtomPub using my AtomPub implementation, amplee.
The idea is to use amplee as an AtomPub library from within XMPP PubSub handlers. For instance I map AtomPub collections to PubSub nodes . When an atom entry is published to a node and that the client receives an acknowledgment of that publication, the handler uses amplee to store the entry in the AtomPub collection as well, respecting of course the RFC specification. Similarly if an item is deleted, the handler uses amplee to remove the entry from the collection. This works both ways, we can also run an AtomPub web service which issues the right XMPP stanzas according to the operation carried. So a POST on a collection would publish the atom entry to the XMPP server. The web service would obviously expose the Atom feed of the collection.
This is just a stub, but the idea here is to associate both protocols so that they cooperate to expand the audience of your network. It’d be easy to consider allowing people commenting to a blog entry using their XMPP client. You blog entry would for instance advert the XMPP service and node name where to publish items. The user could then subscribe to than node and publish items.
That’s one reason why I had written amplee as an AtomPub library that could work outside of the HTTP protocol. RFC 5023 defines the protocol using HTTP but the root idea behind it works well in the context of XMPP and maybe AtomPub is the protocol that will rule them all.

XMPP, a ground for social technologies

I’ve been a long fan of the XMPP protocol and I’ve started implementing it using the fantastic Kamaelia. With all the GSoC discussion around it appeared that lots of people were more and more interested in seeing XMPP becoming the natural partner of HTTP in the maze that the Internet has quickly become. Consequently Peter Saint-André created today the Social mailing-list to all people interested in discussing how XMPP could be used in what is that social web of yours.

I’m totally biased but I think there is more to XMPP than IM, the protocol and its suite of extensions provide great power applicable to RIA, whether they reside inside our outside the browser. For instance, I do believe that rather than using Comet one ought to use XMPP to push notifications to the client. In such a case one might consider the client as a node in a cloud of lazily interconnected nodes and then start thinking of the browser as more than a HTML rendering engine and without the resort to abuse HTTP for things it was never meant to support.

I wish browser vendors could start implementing XMPP within the browser as that would provide a fantastic incentive for more applications based on the power of XMPP.