AmFast 0.5.3 Released

AmFast version 0.5.3 has been released. This release contains several important bug fixes. The code can be downloaded from PyPi or checked out from SVN.

ExtJs Component Config

Cross post: ExtJs Component Config

Testing With Browser Mob

The Project

I recently got the chance to work on a project using BrowserMob for automated testing.  BrowserMob allows you to run Selenium test scripts "in the cloud". In my case I was not testing functionality, but instead  testing performance of a Flex app. I needed to test latency and throughput of messages being dispatched through the Flex messaging system via http://code.google.com/p/amfast/. This proved very difficult to test locally, but was a snap with BrowserMob.

The Testing

I created a simple Flex client to send and receive Flex messages in a way that replicated a production environment. I also created a custom server component to replicate the production environment and to help log message data to be analyzed later. After getting the client and server running locally, I signed up for a BrowserMob account and launched several browsers with their web interface.

The whole process was simpler than it should have been, and I was very impressed with how well it worked. I highly recommend trying out but BrowserMob for performance and load testing applications, and I'm hoping to get a chance to try out running more full featured automated functional tests in the future.

Paired Programming

Rally encourages its engineering staff to post to the company blog, so I decided to write a little about my initial experience with paired programming. TLDR: paired programming is better than I expected, increases code quality, and probably productivity.

tirtle: a Spring Web MVC project running on GAE

I decided to put together a simple web app on my way to learning Java and Spring. Tirtle allows users to track daily numbers such as how many calories they eat in a day. The project uses the Spring Web MVC framework and is running on Google App Engine.

The most frustrating part of the project was just getting everything configured and getting a working server up and running. I feel there is a lack of documentation aimed at beginners, although part of my problems may have been related to jumping right in with Spring + GAE. I chose Spring Framework version 3.0 (the latest version), but there seemed to be more documentation and blog tutorials available for version 2.5. I found an official Web MVC tutorial on the Spring Source site, but it only covered version 2.5.

I had several problems figuring out the base configuration settings, and getting all the correct .jar files in my classpath. webmvc.jar was especially mysterious, because it is not included in the Spring Framework distribution. I ended up finding it via a Google search, but I have yet to find the official download from Spring Source.

Once I got a basic server running, the available documentation for how to actually use Spring seemed pretty good. The Web MVC framework works similar to every other MVC framework you have used. Classes are annotated to turn them into controllers, and controller methods are annotated to turn them into request handlers. You can use either JSPs (Java Server Pages: Java embedded in HTML similar to PHP) or a templating system for your view layer.

The Web MVC stuff is all built on top of Spring's dependency injection framework, so the simple MVC annotations you use are actually shortcuts to lots of complicated XML configuration. Spring's standard DI tools can be used to inject other non-mvc dependencies into your controllers. I used the DI features to configure a simple authentication object (didn't have time to figure out how to get Spring Security working) and also an ORM interface to Google's DataStore (objectify-appengine with the help of objectify-appengine-spring).

Unfortunately, the simple web app I developed didn't require much logic, so it didn't help too much in terms of learning how to code Java, but it was an excellent exercise in getting up and running with Spring. The code for the project is on github. Other new-to-Spring users may find Spring easier to learn by starting with a working app like this one, and learning by modification. I hope to have time to add additional features to the code as I learn how they are accomplished in the Java/Spring ecosystem (unit tests, Spring Security, Javascript framework integration, REST api, templating instead of JSPs).

Python Vs Java

After using Python for the past several years, I'm going to be taking on a Java project. I am in the process of learning Java, and I thought I would write up a comparison of the features in each language.

Features Java has that Python doesn't:

• Static typing
• Strict access control (package, public, protected, private)
• Traditional threading implementation wrapped in a decent API
• Bytecode backwards compatibility 

Features Python has that Java doesn't:

• Dynamic objects
• No explicit compile step
• Properties (transparent getters/setters)
• List comprehensions
• Operator overloading
• Generators (create iterators with the 'yield' statement)
• Optional keyword arguments, *args, and **kwargs
• pypi (the cheeseshop) and pip

Static Typing

Static typing has 2 main advantages. Type errors can be caught at compile time, and the compiler can make more optimizations. Statically typed code takes more time to write (creating explicit interface definitions), but you don't need to write manual type checking functions (duck typing) like you often need to in a dynamic language. Static typing also allows you to avoid runtime type errors that you would probably need a unit test to catch with a dynamic language. Javascript and Python both have decent JIT compilers available, so the speed difference between dynamic languages and static languages will continue to narrow.

Strong vs Weak Typing

While static typing may be helpful for some projects, I believe the biggest factor in type usability is not static vs dynamic but strong vs weak. Weakly typed languages allow you to cast objects from one type to another. If you've ever used Perl, PHP, of Javascript you've probably run into some hard-to-debug problems that were caused by implicit casting or automatic type coercion. Languages like these usually have confusing operators like '==='. C doesn't do any implicit casting, but it will let you manually cast in unsafe ways. Java also allows casting, but is safer than C, as it will throw a runtime exception if you try to cast to an incompatible type. On the other hand, Python is strongly typed. There is no such thing as a 'cast' in Python, and there are very few situations where automatic type conversion takes place (arithmetic with operands of different number types automatically convert all operands to the widest type used in the expression). Python's combination of strongly typed objects and dynamic objects with duck typed interfaces is a winner.

Access Controls

Python has no access control modifiers, and instead uses a convention of naming private attributes with a leading under score: '_private_method'. Client code is not 'supposed' to use attributes named with a leading underscore, but there is nothing technical stopping it. I must admit there have been several times when I wish Python had some equivalent construct. Java's 'final' modifier is particularly useful. Considering that many of Python's standard data types are immutable (string, unicode, tuple, frozenset), it's surprising that Python does not offer an easy way to define immutable objects. In Java it's as easy as adding 'private final'.

Threading

Unlike Python and the GIL, Java can utilize multiple cores when executing threads, and it's concurrency interface is wrapped in a nice API. However, I'm not sure how much I'll get to use the threading features. Networking code is increasingly being moved away from threaded implementations to asynchronous/non-blocking solutions such as NodeJS and Twisted, and computation is being distributed on a cluster or 'in the cloud', instead of being run on a single machine.

Other Features

Java lacks many useful features present in Python, and I'm sure I will miss many of them. I hope the list of useful Java constructs grows as I learn more about the language and start working on a production code base.

Javascript File Browser for Server-Side Files

Problem - Galaxy


Galaxy is a web-based bioinformatics toolkit that allows users to create customized data analysis pipelines. It is becoming an extremely common tool, especially in the sequencing field. The project has one major flaw: it is difficult to get large data files into the system. Uploading 2G files (generated from sequencing runs) through a browser is not a workable solution.


Solution - iRods


iRods is a virtual file system commonly used to transfer and share files in the scientific community. It abstracts the storage details and provides tools for access control, sharing, metadata tracking, file type conversion, and high performance multi-threaded file transfer.

Integration


Myself and co-workers Fred Bevins and Susan Miller were tasked with integrating iRods and Galaxy during the iRods code sprint hosted by iPlant in April. My share of the work was a client-side Javascript file browser. The file browser allows users to browse and select server side files. The version I built talks to the iPlant Foundational API, which exposes iRods directories and files. Other back-ends could easily be added to allow the file browser to talk with a standard file system, or any other file repository. Fred and Susan worked on integrating the file browser into Galaxy, which allows Galaxy users to browse and select files in an iRods repository for use in an analysis. The javascript code is available now, and the Galaxy integration code should be available sometime soon.