switching blogs

I have been on livejournal for a while now and I feel it is time to part ways. Where do I go though? I would like to move it back to I have been looking at what or where to switch and I have not found what I want. So I will look into adapting something or creating something that I can blog with. The following is a list of tentative requirements :

1. The html to be static files.
2. The admin and posting be done via XMPP and Pubsub
3. Not require a relational database. I would use something like couchdb if I have to.
4. Be able to post and manage via an XMPP client and the web.

That is it for now. I will document my progress as I continue to explore. Also, I will give more reasoning for my requirements. :)


XMPP or Jabber's 10th Anniversary is coming up. I like the idea of an open day, but anything would be cool. Just a simple gathering in a MUC room would be good, but physical parties would be better! I wish there were more users/advocates of XMPP in Charleston. There is more discussion on this, in the forum, at the following url:

Also, ejabberd's 6th birthday is here and badlop made a cool video, check it out.

It is nice to see great projects grow! I am also very excited about the future of XMPP!

Palaver's present and future

Today, I made a new release of Palaver. It is an XMPP MUC component, created in python and twisted. The newest version is 0.6. This version has many bug fixes and speed improvements.

New features are memcache support for pgsql backend, and a memcache backend. The memcache backend is a more robust version of the in memory storage. All data is stored in memcached. This allows for palaver restarts without losing all of your data. It is not not as persistent as the other storage mechanisms but if you want a quick and fast MUC then it works, and is a bit better than the in memory dict storage.

I have also been adding MUC support to Wokkel and would like to use that protocol implementation for palaver.

The next release will be dependent on wokkel. So, palaver will then become just the storage mechanism code. Eventually, the wokkel code may make it to twisted words. I think this staging method will allow for quick use and testing. So, by the time it reaches twisted words it will have been used and tested well, beyond the required unit tests.

I am excited about the future of XMPP and Twisted words. The flexibility of twisted is gonna allow for a lot of cool components and servers!

Twisted Wokkel Bot

A bot is not a new thing, it has been done in IRC for a long time. In IRC, a bot is a program that listens to messages in a channel and responds to them.

Bots are also being used in XMPP and particularly Multi-User Chat. You can see a weather bot in action at the following url :

Using XMPP you can extend a bot to respond to messages that are not direct chats. XMPP is xml so you can send and respond to xml messages. This makes XMPP better than IRC.

At Chesspark bots are used extensively. There is one in particular used for a King of the Hill game. It combines MUC and PubSub XMPP extensions to achieve a fun game, where you knock your opponent off the hill by winning chess games.

There are many ways to create XMPP bots, but I have currently been working on MUC support for Wokkel. From Wokkel's web page : "Wokkel is collection of enhancements on top of the Twisted networking framework, written in Python. It mostly provides a testing ground for enhancements to the Jabber/XMPP protocol implementation as found in Twisted Words, that are meant to eventually move there. " Wokkel is nice for implementing and staging things before they can go into twisted.

Using the MUC client branch, I will show you how you can create a Jabber MUC bot. We have to use the branch because it is still under development and not in trunk yet. Hopefully it will be soon. Note that everything is not fully implemented but there is enough to make a bot.

This bot will be simple and just keep track of users coming in and out of a room. You will be able to see user activity using a last command.

First we check out the developement branch. Then we install it.

svn co wokkel

python install

After this we will need to create the python class. Will will put it in the file and call it MUCLastBot.

""" A simple last bot wokkel example """
from twisted.internet import defer
import datetime
from twisted.words.protocols.jabber import jid
from wokkel import muc

class MUCLastBot(muc.MUCClient):
   """ """

All this does it import the python modules we will be using and extend the muc.MUCClient class.

Next, we need to initialize and add 'last' support and methods to the bot.

    def __init__(self, server, room, nick):
        self.server   = server     = room
        self.nick     = nick
        self.room_jid = jid.internJID('@'+self.server+'/'+self.nick)
        self.last     = {}
        self.activity = None

    def _getLast(self, nick):
        return self.last.get(nick.lower())
    def _setLast(self, user):
        user.last =
        self.last[user.nick.lower()] = user
        self.activity = user

This initializes the class with attributes we need and sets up methods to support user activity.

After this, we need to have the bot join a room. This is done with the 'initialized' method in MUCClient. This method is called after the XMPP client authenticates and its observers are initialized.

    def initialized(self):
        """The bot has connected to the xmpp server, now try to join the room.
        self.join(self.server,, self.nick).addCallback(self.initRoom)
    def initRoom(self, room):
        """Configure the room if we just created it.

        if int(room.status) == muc.STATUS_CODE_CREATED:
            config_form = yield self.getConfigureForm(self.room_jid.userhost())
            # set config default
            config_result = yield self.configure(self.room_jid.userhost())

This code will make the bot join the room after start up and then will configure room if the status shows that the room was just created. It uses the 'join' method to join the room and when the bot joins the room 'initRoom' will be called.

You will notice the decorator 'inlineCallbacks', you can ignore this. This decorator allows the method to call deferred methods and use yield to get the call back result.

You can go to to learn more. For our purposes we will just ignore it. :)

Next, we need to handle when someone joins and parts the room.

    def userJoinedRoom(self, room, user):
        """If a user joined a room, make sure they are in the last dict

    def userLeftRoom(self, room, user):

All we do is mark their last activity.

We then need to handle commands and logging messages sent by users.

    def receivedGroupChat(self, room, user, body):
        # check if this message addresses the bot
        cmd       = None
        user_nick = None
            cmd, user_nick = body.split(" ")
        except ValueError:
            # value error means it was a one word body
            cmd = body
        cmd = cmd.replace("!", "")
        method = getattr(self, 'cmd_'+cmd, None)
        if method:
            method(room, user_nick)
        # log last message
        user.last_message = body

This method takes a groupchat message and processes it to see if there is a bot command. It also logs last message activity for the user.

So, the command we want to use is 'last', we need that method inorder to complete the bot.

   def cmd_last(self, room, user_nick):
        if user_nick is None:
            # show last person to do something
            self._sendLast(room, self.activity)
            u = self._getLast(user_nick)
            if u:
                self._sendLast(room, u)
                self.groupChat(self.room_jid, 'Sorry %s, That person is unknown to me.' % (user.nick,))

    def _sendLast(self, room, user):
        """ Grab last information from user and room and send it to the room.
        last_message   = getattr(user,'last_message', '')
        last_stamp     = getattr(user,'last', '')
        if room.inRoster(user):
            message = """%s is in this room and said '%s' at %s.""" % (user.nick, last_message, last_stamp)
            message = """%s left this room at %s and last said '%s'.""" % (user.nick, last_stamp, last_message)

        self.groupChat(self.room_jid, message)

The last command checks the activity for the user given, or if no user is given it will then show the last active user. It will also tell you it does not know the user if it has never seen them in the room.

The '_sendLast' method sends that information back to the room as a groupchat.

The bot is now complete, we will need a conf file to start it up. To do this I will use a .tac file for twisted.

from twisted.application import service
from twisted.words.protocols.jabber import jid
from wokkel.client import XMPPClient

from client import MUCLastBot

application = service.Application("lastbot")

xmppclient = XMPPClient(jid.internJID(""), "test")
xmppclient.logTraffic = True
mucbot = MUCLastBot('','last', 'LastBot')

Simply use twistd to run the bot.

twisted -y muc.tac

That is it, have fun and look forward to more XMPP support in wokkel and twisted!

You can download muc.tac and at the following url:

Speeqe about it.

One thing I do everyday is group chat via XMPP. There are lots of ways to do this, via many xmpp clients. If I am on my laptop or computer then having a client works out well. I can chat to my hearts content.

Often, a problem that comes about, is if I am away, and I need to communicate via a chat room, what do I do? I could use a remote computer and a command line XMPP client or I can use Speeqe.

Speeqe is a web based chat creator, you can create and theme your own chat rooms! There are many ways to use Speeqe for various different things, but the solution to my problem, I find very useful, is that you can connect to other chat rooms on the federated XMPP network.

To do this you can use the url trick, for example:

Jabber Development Room

Charleston Linux Users Group

You can just start chatting anonymously or login via a XMPP account.

This trick, and the other things you can do with Speeqe, are cool, but best of all it does not end there. Speeqe is open source!

You can use and add features to Speeqe and the possibilities become endless. :)

The neat thing is it already uses open source software. It uses Strophe, Punjab, Palaver, Apache, and Django.

It can be set up to use any XMPP server and any Multi-User chat implementation. It is very flexible.

More examples of setting up and using Speeqe are sure to come!

Look forward to the official release of Speeqe!

Punjab, the BOSH of Choice.

It has been a long while since the last Punjab release. But I am happy to announce that Punjab is now at version 0.13. This release has many changes.

First, there was a massive code clean up and removal. XMLRPC, SOAP, and HTTP Polling are no longer supported and have been removed. Also, versions of Twisted older than 2.5 are no longer supported. Similar, versions of Twisted Words older than 0.5 are not supported.

Second, speed increases have been made to the handling of xml and requests. There have been code improvements and protocol support fixes. There are protocol and unit tests too.

Also, Version 1.0 is close and should be out very soon.

Finally, in order to answer 'Which BOSH server do you need?'. Look for examples and demos of strophe and other software and technology using Punjab! These examples will be on here and the Punjab home page.

Baseball and Chess

In sports, or any competition, we always want to know who is the best. Also, we want to know who has the best chance to win the game. Some of us always want our favourite player or team to win. But knowing their chances may help with the expectations that may bring heartache or joy or a wager we may have with a friend.
There are many ways, statisticians and mathematicians have come up with, to determine outcomes and strengths of players or teams. One way, in Chess, is finding a players rating by a rating system. At Chesspark, we use a variation of the Glicko rating system. I have always wondered if this rating system would work well with team sports. So, being that I love baseball, I did some research (google search) on baseball and chess ratings. What follows, is a my first experiments in using glicko atings to determine game outcomes and finding the strength of a team.

First, I could not find any information on baseball and Glicko. I did find some on ELO and baseball. There is a site that does ELO ratings on soccer (football) teams as well. ELO and Glicko are rating systems used in Chess or other games involving two players. Team sports do not necessarily use these methods to determine the outcome of the next game. Baseball has a well known method called the Pythagorean expectation. It uses what the teams have done in previous games to determine the outcome. It uses runs scored and runs allowed. You can use this equation in other sports too. I would like to compare ratings versus the Pythagorean expectation and also investigate combining the two.I gathered data from Wrote a python script to parse the data, and compute the ratings. I will release code in upcoming blog posts. Using the script and data we can calculate the final ratings of teams at the end of the season. The results are interesting.

Glicko Rating results for the Major League Baseball 2007 Season:

====================== teams by rating ==========================
team                           rating  rd         total wins - losses 

New York Yankees               1580.0  28.465171  162   94   - 68 
Boston Red Sox                 1571.0  29.260524  162   96   - 66 
Cleveland Indians              1564.0  28.649712  162   96   - 66 
Los Angeles Angels of Anaheim  1562.0  28.888047  162   94   - 68 
Arizona Diamondbacks           1537.0  28.440955  162   90   - 72 
Colorado Rockies               1534.0  28.165939  163   90   - 73 
Detroit Tigers                 1533.0  28.574597  162   88   - 74 
Seattle Mariners               1532.0  28.210795  162   88   - 74 
Toronto Blue Jays              1529.0  28.210438  162   83   - 79 
Philadelphia Phillies          1529.0  28.447520  162   89   - 73 
San Diego Padres               1523.0  28.317698  163   89   - 74 
New York Mets                  1510.0  28.604631  162   88   - 74 
Los Angeles Dodgers            1505.0  28.615645  162   82   - 80 
Minnesota Twins                1495.0  28.312780  162   79   - 83 
Atlanta Braves                 1494.0  28.618743  162   84   - 78 
Chicago Cubs                   1494.0  28.402241  162   85   - 77 
Oakland Athletics              1488.0  28.395109  162   76   - 86 
Texas Rangers                  1487.0  28.586974  162   75   - 87 
Milwaukee Brewers              1474.0  28.445591  162   83   - 79 
St. Louis Cardinals            1464.0  28.453097  162   78   - 84 
Washington Nationals           1462.0  28.858994  162   73   - 89 
San Francisco Giants           1462.0  28.315478  162   71   - 91 
Chicago White Sox              1461.0  28.461948  162   72   - 90 
Kansas City Royals             1461.0  28.856424  162   69   - 93 
Baltimore Orioles              1460.0  28.501870  162   69   - 93 
Tampa Bay Rays                 1450.0  28.742325  162   66   - 96 
Houston Astros                 1448.0  28.453125  162   73   - 89 
Cincinnati Reds                1442.0  28.818955  162   72   - 90 
Florida Marlins                1438.0  28.412856  162   71   - 91 
Pittsburgh Pirates             1424.0  28.681253  162   68   - 94 

NOTE: This is a great example on how close in skill professional baseball teams are.

Now, lets take a game from the 2007 season and determine the outcome via glicko and the same percentage via the Pythagorean expectation. I will use Atlanta Braves games since they are my favourite team. Lets take the 42nd game Atlanta played. This game was versus the Boston Red Sox who was a high rated team all year.

NOTE: This determination is very simplistic and missing some steps, a more in-depth example may follow in other blog posts.

Boston had a record of 29 - 12 (Winning percentage 70.7%)
Atlanta had a record of 25 - 17 (WP 59.52%)

2007-05-19 ATL 1529.0 vs. BOS 1675.0

2007-05-19 ATL 52.562% vs. BOS 72.554%

By rating, Boston should win, they are the strongest. The chance for Atlanta to win is about 47.7%. The outcome of the game was Boston 13 and Atlanta 3. Boston crushed them. Before this, Atlanta split a series with Washington, a very weak team.

By winning percentage, we see that Boston's real percentage is lower than what their percentage should be and we see that Atlanta's real percentage is higher than the determined one. Using that we see that Boston is favoured to win unless luck comes into play.

Both methods show Boston being the favourite, but 10 runs seems too much. :)

Determining the actual chance that Boston will win is a bit more complicated than the above example, with both methods. It is easy to see that using each way shows that Boston is stronger and should win the game, and they did. You can also use other statistical methods to determine how many runs both teams could score and use that to help with determining the outcome. I want to find if I could use ratings to help or replace some of these.I believe ratings is a great indicator of how strong the team is right now (or at the moment) and is a simpler way when looking at strength of schedule and other factors in finding out who will win a series or finish first in the standings.

In posts to follow, I hope to start predicting the outcome of current games, and developing a system to make this easier for me and others. I plan on using XMPP, Pubsub and BOSH. I also want to investigate finding a pitcher or batter's rating using glicko. After that, who knows.

The baseball information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at "".