Using Jython as a CLI frontend to HBase

HBase, the well known non-relational distributed database, comes with a console program to perform various operations on a HBase cluster. I’ve personally found this tool to be a bit limited and I’ve toyed around the idea of writing my own. Since HBase only comes with a Java driver for direct access and the various RPC interfaces such as Thrift don’t offer the full set of functions over HBase, I decided to go for Jython and to directly use the Java API. This article will show a mock-up of such a tool.

The idea is to provide a simple Python API over the HBase one and couple it with a Python interpreter. This means, it offers the possibility to perform any Python (well Jython) operations whilst operating on HBase itself with an easier API than the Java one.

Note also that the tool uses the WSPBus already described in an earlier article to control the process itself. You will therefore need CherryPy’s latest revision.

To test the tool, you can simply grab the latest copy of HBase and run:

Then you need to configure your classpath so that it includes all the HBase dependencies. To determine them:

Copy the full list of jars and export CLASSPATH with it. (This is from the HBase wiki on Jython and HBase).

Next you have to add an extra jar to the classpath so that Jython supports readline:

Make sure you’ll install libreadline-java as well.

Now, that your environment is setup, save the code above under a script named stave.py and run it as follow:

You can import any Python module available to your Jython environment as well of course.

I will probably extend this tool over time but in the meantime I hope you’ll find it a useful canvas to operate HBase.

XMPP and IronPython 2 using headstock, bridge and Kamaelia

I am glad to announce that IronPython 2 is now capable of running my XMPP
Python library: headstock.

.NET has already an excellent XMPP SDK called agsXMPP that is a native
.NET/C# framework. However I’m a Python developers at heart and I had
started quite a while ago writing my own XMPP library in Python using the
most excellent Kamaelia framework (designed for concurrency).

For a while IronPython had severe shortcomings that prevented it running
simple Kamaelia applications. Today I was able to run a simplechat demo
using a vanilla IP2 on Windows with only one single modification to the
logging module (thanks Seo). To be honest I didn’t expect it to go through
:)

The chat demo is simple enough but means more complex examples using XMPP
PubSub will work as well (they are all based on the same framework).

Now this isn’t production ready or anything. For instance the TLS support
is broken (hopefully something easy enough to fix) so you won’t be able to
connect to Google Talk for now.

Moreover I’m not sure the code is that fast considering how I had to
simulate an incremental XML parser atop System.Xml (this allows for a
XML stream to be parsed without requiring the full document or even
fragment to be read first).

This is a great news for me because it means I’ll be able to move ahead
with more work using IronPython 2.

Geolocalization and microblogging

I’ve been working recently with Adrian Hornsby who’s been interested in using the microblogging example I had setup to demonstrate headstock, amplee and some ideas about microblogging in general. Today Adrian asked me how to add some gelocalization information to a message flowing through the system. It took me just a couple of hours to implement it so that now you can push a message like: GEO text [lat,long] through your IM client. This will tell the demo to add a georss:point element to the generate atom entry which will eventually lead to a Google map to be displayed in the web page mapping the atom entry.

headstock: XMPP library based on Kamaelia

I’m pleased to announce the release of the first version of headstock, my XMPP implementation using the Kamaelia library. My main motivation behind working on that project rather than using one of the existing libraries was that I wanted to find a challenging project to work with Kamaelia. XMPP seemed challenging enough.
This first release is flagged as beta. Not production ready at all but offers enough so that you can play with it and actually use it for integrating XMPP into your applications. Documentation and tests are badly missing and they will eventually come but for now you’ll have to go through the code. I know this is not attractive but I though it was worth to be released nonetheless. Enjoy.