Making QA easier with Selenium

Earlier this week I wrote a summary of a few ways you can automate and test your Javascript, including the use of the Selenium platform by OpenQA. I say platform because there are several very different ways you can use the Selenium family of programs. In this post I’ll expand on the ones I’ve used.

Developer and Regression Testing

Lets face it, testing *that* screen for the 14th time in a row can get pretty tedious, especially if you have to start from a fresh session and go through half a dozen screens on the way there.

That’s where Selenium IDE comes in. SIDE is a Firefox extention that saves you time by allowing you to record a test once in your browser, and save it. You can then replay that test whenever you like, sitting back while Firefox speeds through your test for you. You can also build assertions into your test so that specific conditions will make it fail.

If you build up a library of tests you can run them all as a batch, while you get on with something else.

Continuous Integration Testing

SIDE is great for when you’ve got a developer or tester sitting in front of a browser, but what about automated tests? In a Continuous Integration environment, checking new code into source control triggers a automated build to happen, usually with associated unit tests. That’s all great for classes and modules that suit unit testing, but what about higher-level testing?

That’s where Selenium RC comes in. RC stands for Remote Control, which makes sense when you understand what it does. Selemium RC is a collection of APIs for most popular programming languages that allows your code to control an actual instance of a browser (it supports Safari, IE and Firefox) and run tests against it.

You can set up a build script that runs through your library of regression tests and reports the results back to you. This means that once you’ve recorded a few different runs through one of your website processes, you can then have those tests run automatically whenever someone checks code into source control.

Automating Javascript Testing

If you’re putting together a software application you probably want to think about setting up build environment, and applications that use a lot of Javascript are no different. A little time invested will probably save headaches down the line. Being able to perform single-click builds of your app is really useful, as explained by Joel Spolsky.

Back in 2005 I started putting together an app (Newsdesk) for Moreover that included what turned about to be about 6000 lines of Javascript. The build environment I came up with wasn’t perfect, and could never remove the need for actual human testing, but it saved an awful lot more work than it took to build. In this article I’ll outline a few ways you can achieve something similar.

Setting up a build

If you’re reading this you should know about encapsulation, the process of restricting outside knowledge of the inner workings of a component. In large systems this is a fundamental “Good Idea”. In compiled systems it’s simple to encapsulate classes and then have them compiled into more monolithic chunks to make them easier to distribute (DLLs, JARs, EXEs, whatever). Usually your IDE/compiler will take care of this for you.

In Javascript a self-disciplined coder can do the same thing. Rather than having one or two large files, you can split your code up into encapsulated components and have them concatenated together into your final files at build-time. This build script can also serve as a platform for code hygene, unit testing, and regression testing.

The core build script

I’d recommend using ANT, (or NANT if you’re using Dot Net). ANT is a flexible, modular, and powerful build program that you can configure to do just about whatever you want. It feels a little strange to write procedural instructions in XML, but once you get over that it’s simple. Using a basic ANT script you can get the latest code from source control, compile it, move it around, rename it, run tests, send an email with the build result to your team, FTP it to your server, or pretty much anything else you can think of.

My build script did roughly this to the Javascript:

• Iterate the build number and add it to the code’s config file
• Running unit tests
• Concatenating code and language files into packages
• Running code compression on packages (removing comments, whitespace, etc)
• Generating Javascript code reference

…but you could also include

• Regression testing
• Code duplication analysis
• GZip pre-compression
• Deploying code to specified server

Build script configuration

If you’re writing a build script you probably ought to think about running it in a couple of modes, because you usually have more than one use case for your build.

Firstly you’re going to want a relatively quick dev build that does quick testing and spits out your compiled code to a local development environment. This might also run some analysis depending on your requirements.

Secondly you might want an occasional analysis build that runs code hygene analysis, runs more in-depth tests, that sort of thing.

Lastly, you might want a deployment build, that retrieves only a specific version (latest, codebase version X, or codebase as of date Z) and outputs a zip or tarball of the compiled codebase ready to be archived and/or copied to servers.

Testing your Javascript

This might seem a bit hard to automate until you discover Rhino. Mozilla’s Rhino engine (in Java) and SpiderMonkey (in C) are invaluable for running Javascript either under a shell or embedded in other applications. Putting Rhino’s JAR file into your classpath allows your build to execute Javascript.

If you want to run JSLint over your source, you can do that. If you want to write your own code analysis tool in JS you can do that too.

This is great for unit testing, but with a UI unit testing can only get you so far. If you want to be a bit more thorough you need regression testing. This can be achieved using Selenium. Selenium is a Javascript-based regression testing engine that allows you to trigger DOM events and test assertions against the resulting DOM. Although this engine is browser-based, you can also call it from code, or ANT, using secondary tools like Selenium RC (Remote Control). There’s also a Firefox plugin that allows you (or your QA team) to build up a library of regression tests that can be run automatically on each build.

More…

I’ve tried not to get into too much detail this time. When I get round to it I might write up areas in more detail. Let me know if you’re interested in something particular.