ws4py – WebSocket client and server library for Python

Recently I released ws4py, a package that provides client and server WebSocket support for Python 2.6 and 2.7.

Let’s first have a quick overview of what ws4py offers for now:

  • WebSocket specification draft-10 of the current specification.
  • A threaded client. This gives a simple client that doesn’t require an external dependency.
  • A Tornado client. This client is based on Tornado 2.0 which is quite a popular way of running asynchronous networking code these days. Tornado provides its own server implementation so I didn’t include mine in ws4py.
  • A CherryPy extension so that you can integrate WebSocket from within your CherryPy 3.2.1 server.
  • A gevent server based on the popular gevent library. This is courtesy of Jeff Lindsay.
  • Based on Jeff’s work, a pure WSGI middleware as well (available in the current master branch only until the next release).
  • ws4py runs on Android devices thanks to the SL4A package

Hopefully more client and servers will be added along the way as well as Python 3.x support. The former should be rather simple to add due to the way I designed ws4py.

The main idea is to make a distinction between the bytes provider and the bytes processing. The former is essentially reading and writing bytes from the connected socket. The latter is the function of making something out of the received bytes based on the WebSocket specification. In most implementations I have seen so far, both are rather heavily intertwined making it difficult to use a different bytes provider.

ws4py tries a different path by relying on a great feature of Python: the possibility to send data back to a generator. For instance, the frame parsing yields the quantity of bytes each time it needs more and the caller feeds back the generator those bytes once they are received. In fact, the caller of a frame parser is a stream object which acts the same way. The caller of that stream object is in fact the bytes provider (a client or a server). The stream is in charge of aggregating frames into a WebSocket message. Thanks to that design, both the frame and stream objects are totally unaware of the bytes provider and can be easily adapted in various contexts (gevent, tornado, CherryPy, etc.).

On my TODO list for ws4py:

  • Upgrade to a more recent version of the specification
  • Python 3.x implementation
  • Better documentation, read, write documentation.
  • Better performances on very large WebSocket messages

32 thoughts on “ws4py – WebSocket client and server library for Python

  1. Pingback: Sylvain Hellegouarch: ws4py – WebSocket client... | Python | Syngu

  2. Mohamed

    On what browser is the echo_cherrypy_server example supposed to work?

    I tried it on 14.0.835.157 beta-m, on Windows 7, 64bit, and it does not work…
    After entering some text and pressing the submit button, nothing happens.
    No errors in the console on chrome, or of cherrypy.

    Thanks.

    1. Sylvain Hellegouarch Post author

      I’m now wondering if it’s not because draft-10 describes the deflate-stream extension which has been dropped from draft-11 and that ws4py doesn’t implement. In such case, it’d be more accurate to say that ws4py implements draft-11 only.

  3. Timothy M. Shead

    Many thanks for ws4py … I can confirm that it works great with cherrypy 3.2.0 and Chrome 14.0.835.159 beta on OSX. It also works great with Firefox 7.0 on OSX, with a small modification … Firefox sends

    Connection:keep-alive, Upgrade

    in the initial handshake, and the current logic at ws4py/server/cherrypyserver.py:125 will only accept

    Connection: Upgrade

    I don’t know enough about the relevant standards to say who should change, but it’s an easy tweak to ws4py if you’re interested.

    Finally, a dumb question: how should I initialize a handler derived from WebSocketHandler? I want to pass arguments to its constructor, but since it’s instantiated by WebSocketTool I don’t see how to get to the new instance to hook it into the rest of my application.

    Cheers!
    Tim

    1. Sylvain Hellegouarch Post author

      Hey Tim,

      1. I will look at the connection header issue. But don’t hesitate to open a bug ticket on github.
      2. Indeed it seems you cannot pass an instance itself. I may have to change that so that both can be done. In the mean time, off the top of my head, I’d say you can perform operations on the created instance from your traditional CherryPy page handler.

      Look at line 369 in https://github.com/Lawouach/WebSocket-for-Python/blob/master/ws4py/server/cherrypyserver.py#L369

      You see you have access to the handler instance and therefore could do whatever you want with it. Note that the handler is also available from the WebSocketHandlerPlugin.

      1. Timothy M. Shead

        Sylvain:

        Thanks for the quick response, I filed the ticket. In the meantime, I figured-out that I can access the handler using cherrypy.request.ws_handler, and it’s working well – I’ve got a simple application that delivers images to the client running on my localhost.

        In the long run, I need to deploy my little app behind an Apache reverse proxy that provides SSL and authentication. Any issues you’re aware of? I have some vague idea that secure sockets are required to go through a proxy, are they supported via cherrypy + ws4py?

        Many thanks,
        Tim

        1. Sylvain Hellegouarch Post author

          Hey Tim,

          Thanks for the ticket.

          As for deployment, I haven’t tried but I’m not certain how reverse proxying and WebSocket will fare together. CherryPy does support SSL but I’ll admit I haven’t yet tried secure WebSocket connection.

          1. Timothy M. Shead

            Sylvain:

            Well, the good news is that, for a CherryPy server configured to use SSL, secure WebSocket connections seem to work.

            This is all running on the same host, next step is to try and make it work through my Apache proxy.

            Cheers,
            Tim

        2. Jan-Philip Gehrcke

          Hey Timothy,

          maybe stunnel can be a help for you:
          http://www.stunnel.org/

          Basically, you can make stunnel accept https connections via e.g.
          accept = 0.0.0.0:443
          and forward them decrypted via e.g.
          connect = localhost:1234
          to your application. By doing so, the SSL encryption is transparent to your Python application which also applies to open websockets if I understand things correctly. Of course this requires a careful security considerations behind your load balancing/stunnel (at least that your Python app must not reachable from the outside).

          stunnel’s use with respect to load balancing and websockets is discussed for example here:

          http://www.mail-archive.com/haproxy@formilux.org/msg02799.html

          http://www.mail-archive.com/haproxy@formilux.org/msg02679.html

          Hope that somehow helps..

          Jan-Philip

          Btw: Thanks to Sylvain for this library. Currently, I am using gevent-websocket, but we all should stick to some standard and so I think I will be using ws4py, too :-)

  4. Mark Place

    Hello Sylvain,
    Thanks so much for your great work on this library. I’m pretty new to Python but have learned a lot from studying your code.

    I’ve having trouble getting the chat example to work using CherryPy 3.2.2 and Chrome 18.0.1025.11 beta-m on WindowsXP. My clients connect just fine and are able to send messages. Publish() is called from ChatWebSocketHandler, and eventually calls output.append(). However, the message is never received on the browser side, even though TCPView shows that the connections are established.

    Any idea of what could be going wrong?

    Thanks again,
    Mark

    1. Sylvain Hellegouarch Post author

      That’s weird indeed. I’ve been refactoring ws4py quite extensively internally since yesterday so hopefully things will be more stable once I commit (in theory soonish this weekend).

  5. Mark Place

    I may have narrowed it down a bit.

    isinstance(data, bytearray) is returning false. isinstance(data.data, bytearray) returns true, however.

    Not sure what this means. BTW I’m running Python 2.6.

  6. David Brooks

    Just getting my head around Web Sockets and ws4py. I had a gevent-based test server working using a WSGI function to send data based on URL, but with your recent code changes it appears that everything has to happen inside some sub-class of WebSocket, with no apparent way of getting the environment. Am I missing something? What about having **kwds on the __call__ of WebSocketUpgradeMiddleware and passing them onto websocket_class ??

    Thanks for the work.
    Dave

  7. Marty

    Hi Sylvain,

    Great work.

    What would greatly help me would be a simple example of a host-based process periodically sending data to a browser once a websocket is established. I’m looking at your chat example, which is great, but I’m more interested in point-to-point communication to different clients, and the threading required not to block.

    Thanks!!
    Marty

  8. Marty

    I downloaded the latest zip file of websockets. In cherrypyserver.py start_handler has some embedded tabs that makes for incorrect formatting, but in such a way that the python interpreter doesn’t complain. As well, at the end of the upgrade routine, on line 197, is the following line:
    addr = (request.remote.ip, request.remote.port)
    It has an embedded tab, and addr is unreferenced for the rest of the method. This is the same text with the incorrect formatting in start_handler, making me think some erroneous cut/paste operations got included.

    Thanks,
    Marty

  9. chaobin

    Hi – I am currently making a higher level of libraries on top of ws4py and gevent, the goal of this little project is to make the realtime web apps development easier. Before I start to make even more decisions into this project, I want a little help from you on the geventserver. I know that geventserver is made out of gevent.pywsgi, somehow, I am still not sure how the server closes a ‘thread’, thus everything including the websocket and other stuff get cleaned up when a client closes purposely or unpurposely an established connection.

    Many thanks on this question.

    1. Sylvain Hellegouarch Post author

      Hi,

      The WSGI server is only there to parse the initial Upgrade request. Once the handshake and the upgrade have been completed, ws4py spawns a new greenlet which runs the WebSocket.run method. That greenlet runs as long as the WebSocket stream is opened (and therefore the underlying socket connection). Whenever the run() method terminates, the greenlet terminates as well.

      1. chaobin

        Hi Sylvain – Thanks very much for what’s explained. Here is the further thinking on this, put it simply, how does server detect if a connection is disconnected? Almost certain, in a realtime application, the established connections will somehow want to talk to each other, broadcasting events or the like, this is where we need a pool or something to manage a collection of previously established connections. The ws4py.websocket provided several handy entry-point to do this, for example, we could register one connection into the pool in websocket.opened, and pop out one in websocket.closed, but, what if the disconnection isn’t a natural one? Instead of being initiated from either server or client, the disconnection is caused by a poweroff. The question is, how do we effectively and reliably detect a disconnected connection?

        Regards,

        1. Sylvain Hellegouarch Post author

          Well in that case, the socket will timeout after a while (usually determined by the underlying OS since ws4py is blocking on the socket) and provoke an exception which would trigger the closed handler.

          With that said, you’re not the only one wishing for something more usable than a timeout (which can be quite long). For now, a solution would be to add some component that simply monitors each socket (like an event loop). Perhaps based on the select family…

          For now, I haven’t found the proper way of doing this without external dependencies and in a portable fashion. I wish to keep ws4py as light as possible with a lib-spirit rather than a framework approach.

          1. chaobin

            Hi Sylvain – Thanks. Well the good thing is that after all there would be a timeout being propagated providing a chance to clean the zombies, though it might be quite long. (I guess it is also possible to globally set a reasonably long timeout? though this decision is by no means to be made inside a library such ws4py)

            The monitor-loop you mentioned, is actually what I had thought about. And it does make sense to me that ws4py isn’t the place where this loop fits into.

            Thanks very much!

  10. sander

    Hi Sylvian,

    Using ws4py in combination with cherrypy I have successfully implemented websockets into my serial port reading web app. Thanks for this !

    But now I have come to a point where my knowledge is running short and I don’t know if it is at all possible. So I hope you could give me some insight…

    I have multiple sources of data. (a serial port, a weather station, a scheduler).
    All of them are connected to the same socket using python clients and publishing messages to all subscribed web pages. But now I would like to filter the outgoing messages depending which page I am on: A ‘weather’ page should only receive weather info. The ‘serial’ page only serial info. But the ‘overview’ page should receive all info.)
    Can I use ws4py to filter outgoing messages depending on some kind of keyword (‘all’, ‘weather’ etc.) that is communicated between the client page and server?
    Or should I create multiple websocket connections which only communicate specific data. (and create all websocket connections when going to the ‘overview’ page.). I have been reading about message brokers and I feel it could be part of the solution, but I am totally new on this issue…

    I guess it is more a conceptual problem and possibly not entirely related to ws4py, but I still hope you can give me your opinion. Thanks !

    Sander.
    The Netherlands.

    1. Sylvain Hellegouarch Post author

      Hi Sander,

      I think you want a pubsub broker and although ws4py could provide it, I made the decision not to because others do it better and I wanted ws4py to be lightweight. You could look at Autobahn which provides such mechanism. Or go wild with a different technology such as ZeroMQ. Both might require major rework on your side but are rock solid and powerful.

      You could also interface a light queue backend with ws4py and provide some sort of in-house dispatching but that depends on your need.

      Hope that helps.

      1. sander

        Hi Sylvain,

        Thanks ! I was kind of fearing this kind of answer. I think I’ll just stick with your lean and mean ws4py ;-)
        I think I am going for one of the two options:
        1. As there won’t be a lot of traffic anyway I can do client side filtering of which messages to show.
        2. Connect to multiple sockets in one page. Or is that a conceptual no-go ?

        Sander.

        1. Sylvain Hellegouarch Post author

          I would rather avoid having several connextions, if not conceptually an issue, I fear resources will dry up quickly.

          Note that the WebSocket protocol is perfectly capable of multiplexing through a single connection: think protocols and extensions.

          Though currently ws4py doesn’t do much with both of those, it is one aspect that I have in mind improving based on use cases.

  11. Lilian

    Hi, I want to use webSocket to publish the message from the server to client.
    How to do?
    In the Cherrpy echo server example, I only the review_message method.
    but how to send the message from the service to notificate all connect client?

Comments are closed.