SQL Workshop

I've recently finished a SQL tutorial for a relational database workshop I'll be teaching. The tutorial covers basic SQL queries with SQLite and MySql, as well as basic database creation with SQLite. The tutorial is very basic, and the examples are geared toward biologists who need to use a database for performing research.

Facebook's Tornado

Today I got my first chance to play with Facebook's Python based web server and framework Tornado. The framework is single threaded and asynchronous, similar to how the Twisted framework operates, and Facebook uses the technology to provide data to the 'Friend Feed'.

Python thread creation and maintenance have relatively high overhead, so asynchronous solutions can provide better performance and scalability than more traditional threaded server implementations. This is especially true for servers with high numbers of concurrent connections, and for long-polling and streaming connections that spend most of their time waiting around for a message to be published to a client. Tornado's benchmarks look impressive against threaded Python web servers, but I wonder why they didn't include Twisted, their closest competitor, in the benchmarking.

My first impression is that Tornado is much easier to use than Twisted, but also much less powerful. Twisted is a monolithic package that can be used with many different networking protocols, but Tornado is focused only on http. Tornado provides built in support for templating, authentication, and signed cookies. One really cool feature Tornado provides is semi-automatic XSRF protection, which should save time for developers who would otherwise need to manually implement countermeasures.

The documentation for Tornado is very slim. For example, I couldn't find any information about how to interact directly with Tornado's main event loop (IOLoop). Fortunately, Tornado's code base is small and readable, so reading the source will quickly get you up to speed.

While implementing a Tornado channel for the AmFast project, I ran into a problem that I have also encountered with Twisted. Both Tornado and Twisted use callbacks to complete a request. As an example, if a long-poll client is waiting for a message, you call RequestHandler.finish() to send a response back to the client when a message is published.

The above method works well for a single server instance, but what about when you have multiple servers behind a proxy? A server process may publish a message for a client that is connected to an entirely different server process. The publishing process has no way to notify the subscribing process that a client has received a message.

This problem can be solved by polling a database table or a file that is accessible to all server processes, but that takes system resources, especially if you have many clients and your polling interval is short. One of my future goals is to figure out a more elegant way for the publishing server process to notify the subscribing server process when a client receives a message.

AmFast adds support for Google App Engine and Django

I just released version 0.4.0 beta of AmFast, a Flash remoting package for Python. The new release includes support for Google App Engine and the Django framework.

Browse the code here.


Check out the live Google App Engine demo.

To get the code:

svn checkout https://amfast.googlecode.com/svn/tags/0.4.0b

YAPT (Yet Another Python Tutorial)

The tutorial I created for the Python workshop is posted here. The tutorial is geared towards bioinformatics users, and includes molecular biology related exercises.

I created the tutorial with the Sphinx documentation package, which turned out to be the perfect tool for the job.

Learn Python Workshop

I'm going to be teaching an intro to Python workshop next month. It will consist of 3 3 hour days and will cover the basics of Python. Dates are June 15th, 17th, 19th. The workshop is free for all U of A students, staff, and faculty. I have never lead a workshop before, so it should be a lot of fun.

Register for the Python workshop

Real-time Flex messaging with AmFast

This example is very similar to the previous, but we're going to use AmFast's HTTP streaming channel to implement real-time flex messaging. Get the complete example code here.

First let's setup the Producer and Consumer in Actionscript.

// Configure the ChannelSet that
// messages will be sent and received over.
import mx.messaging.ChannelSet;

// To use HTTP streaming, the
// StreamingAMFChannel class must be used.
import mx.messaging.channels.StreamingAMFChannel;
var channelSet:ChannelSet = new ChannelSet();
var channel:StreamingAMFChannel = new StreamingAMFChannel("channel-name", "server-url");
channelSet.addChannel(channel);

// Setup a Consumer to receive messages.
import mx.messaging.Consumer;
var consumer:Consumer = new Consumer();

// The consumer's destination is the 'topic'
// name that the consumer will be subscribed to.
// The consumer will receive all messages
// published to the topic.
consumer.destination = "topic";

// Use the ChannelSet that was already created.
consumer.channelSet = channelSet;

// This event listener will be called whenever
// the consumer receives a message from the server.
consumer.addEventListener(MessageEvent.MESSAGE, newMsgHandler);

// The consumer won't start receiving messages
// until it is subscribed.
consumer.subscribe();

// Setup a Producer to publish messages.
import mx.messaging.Producer;
var producer:Producer = new Producer();
producer.destination = "topic";
producer.channelSet = channelSet;

// Create an Async message and send it
// to all other clients subscribed to the topic.
import mx.messaging.messages.AsyncMessage;
var msg:AsyncMessage = new AsynMessage();

// Set the message's body attribute to the
// object that is going to be published.
//
// In this case the object contains
// the X and Y coordinates of a Sprite.
msg.body = {'x': 10, 'y': 10};
producer.send(msg);

Next we'll setup the server.

# Create a ChannelSet to serve messages.
from amfast.remoting.channel import ChannelSet
from amfast.remoting.wsgi_channel import StreamingWsgiChannel
channel_set = ChannelSet()

# Each individual ChannelSet can use
# one or more Channels. When messages
# are published through the ChannelSet,
# they will be published to all subscribed clients,
# regardless of which Channel the clients
# are connected through.

# Create a HTTP streaming channel.
#
# When a client connects to a streaming channel,
# the HTTP connection remains open until the client
# disconnects. When messages are published,
# they are immediately dispatched to clients connected
# to HTTP streaming channels.
stream_channel = StreamingWsgiChannel('stream-channel')

# WsgiChannels objects are wsgi applications
# that can be served with any Wsgi server.
# CherryPy is being used in this example.
cherrypy.tree.graft(stream_channel, '/amf')

# Start the server.
# App() is your root controller class
# for non-AMF functions.
cherrypy.quickstart(App(), '/')

# That's it, our server is up and running.
# It's that simple.

To test the example, download the full code from the repo. To run the example you'll need to install AmFast and CherryPy or Twisted 1st.

# To server the example with cherrypy
python cp_server.py

# To server the example with Twisted
twistd -noy twisted_server.tac

Open two browser windows and browse to 'http://localhost:8000'. Click the 'Subscribe' button in both windows. In the 1st window click on the 'Master' button. In the 1st window, click on the yellow circle, and drag the circle around the screen. As the circle is dragged around the screen, the position of the circle will be updated in the 2nd window. Pretty cool stuff.

Create a chat client and server with Flex and Python

In this example we'll create a simple Flex chat client that uses Producer/Consumer messaging to send and receive messages through a Python server. The server will use AmFast's HTTP polling and long-polling channels. Get the complete example code here.

First let's setup the Producer and Consumer in Actionscript.

// Configure the ChannelSet that
// messages will be sent and received over.
import mx.messaging.ChannelSet;
import mx.messaging.channels.AMFChannel;
var channelSet:ChannelSet = new ChannelSet();
var channel:AMFChannel = new AMFChannel("channel-name", "server-url");
channelSet.addChannel(channel);

// Setup a Consumer to receive messages.
import mx.messaging.Consumer;
var consumer:Consumer = new Consumer();

// The consumer's destination is the 'topic'
// name that the consumer will be subscribed to.
// The consumer will receive all messages
// published to the topic.
consumer.destination = "topic";

// Use the ChannelSet that was already created.
consumer.channelSet = channelSet;

// This event listener will be called whenever
// the consumer receives a message from the server.
consumer.addEventListener(MessageEvent.MESSAGE, newMsgHandler);

// The consumer won't start receiving messages
// until it is subscribed.
consumer.subscribe();

// Setup a Producer to publish messages.
import mx.messaging.Producer;
var producer:Producer = new Producer();
producer.destination = "topic";
producer.channelSet = channelSet;

// Create an Async message and send it
// to all other clients subscribed to the topic.
import mx.messaging.messages.AsyncMessage;
var msg:AsyncMessage = new AsynMessage();

// Set the message's body attribute to the
// object that is going to be published.
//
// In this case the object is a String,
// but the message body can be any
// type of object.
msg.body = "This is being published!";
producer.send(msg);

Next we'll setup the server.

# Create a ChannelSet to serve messages.
from amfast.remoting.channel import ChannelSet
from amfast.remoting.wsgi_channel import WsgiChannel
channel_set = ChannelSet()

# Each individual ChannelSet can use
# one or more Channels. When messages
# are published through the ChannelSet,
# they will be published to all subscribed clients,
# regardless of which Channel the clients
# are connected through.

# Create s standard polling channel.
polling_channel = WsgiChannel('poll-channel')

# A long-polling channel is created similar
# to a normal channel, but the wait_interval
# argument must be set to -1.
long_channel = WsgiChannel('long-channel', wait_interval=-1)

# WsgiChannels objects are wsgi applications
# that can be served with any Wsgi server.
# CherryPy is being used in this example.
cherrypy.tree.graft(polling_channel, '/amf')
cherrypy.tree.graft(long_channel, '/longPoll')

# Start the server.
# App() is your root controller class
# for non-AMF functions.
cherrypy.quickstart(App(), '/')

# That's it, our server is up and running.
# It's that simple.

To test the example, download the full code from the repo. To run the example you'll need to install AmFast and CherryPy or Twisted 1st.

# To server the example with cherrypy
python cp_server.py

# To server the example with Twisted
twistd -noy twisted_server.tac

Open two browser windows and browse to 'http://localhost:8000'. In the 1st window set the messaging url to 'http://localhost:8000/amf' to use the polling channel. In the 2nd window set the messaging url to 'http://localhost:8000/longPoll' to use the long-polling channel. Click the 'Subscribe' button in both windows. Enter a message in the message entry box, and press the 'Publish' button to publish the message. The published message will appear in both clients.