atompub

Noah Gift described a cool project he had started recently named Pylesystem. The idea was to monitor the filesystem for events regarding the creation and deletion of files, more specifically media files like music audio files. Noah explained he wanted to provide a simple metadata storage so that he could query it for getting information about his media library. He decided to use SQLAlchemy to abstract the storage as well as the query mechanism as much as possible whilst providing an easy and high level query interface.

I thought it'd be cool to give it a go using the AtomPub data model to store metadata. I therefore hacked a small example within a few hours this weekend using amplee.

I was quickly able to associate a collection per artist and album and a member per track. I used hachoir-metadata to get a few basic metadata about tracks as well as a couple of simple queries to the Musicbrainz web service to get more advanced information like tags, related URLs, etc.

Finally I used the basic indexing mechanism provided in amplee to perform simple query on the AtomPub store and then generate atom feeds on the fly based on the query result.

Overall it was really fun and I'm happy that it shows how to use AtomPub and amplee in a context that is not directly related to the web. The source code is of course available here.

Note that if you wish to run the code you'll need the latest trunk of amplee. Otherwise you'll need the current stable release of amara, hachoir-metadata and pyinotify.

Matthew Wood, from the BBC Radio Labs, posted a few days ago an exciting note regarding fun he was having with XMPP and services such as last.fm to inform user, via XMPP messages, of BBC radios broadcasting music they might like based on thier last.fm profile. I thought this was fantastic but I was even more excited when I read another note where he explained that he was using PubSub as a mean to carry and distribute metadata about BBC shows using Atom entries as the metadata format.

This evening I spent three hours expanding on the simple chat example coming with headstock to talk with the PubSub service. Then I integrated amplee as a way to offer an AtomPub interface at the same time. This means that when the demo starts both a XMPP client, connecting to Matt's service, and an AtomPub server, using amplee and served by CherryPy, are started. The XMPP client asks the server about PubSub nodes. For each node representing BBC channels I create an atompub collection within its own workspace. Simultaneously I subscribe to those nodes. I then ask the XMPP server for items belonging to those nodes and for each item, representing metadata about a show for instance, I create an atom entry that I store within the AtomPub store.

This means one can then simply subscribe with a feed reader to a given collection and/or a XMPP PubSub node. All of this happening on the fly starting from an empty AtomPub service document.

Eventually I will add support so that when an Atom entry is POSTed to a AtomPub collection, the according PubSub stanza is pushed towards the service (I doubt Matt's service accepts it though) allowing for microblogging support.

The source code of the example can be found here. If you want to understand how it works you might want to read this quick word I wrote about Kamaelia first which is at the core of headstock.