Parallelizing Test Execution: Speed Run!

Browser tests (sometimes) take a long time to finish. If your automated tests do things such as registering for a new user, adding a dress to a shopping cart, and then renting that dress, while checking along the way that each step is good, it might take upwards of 2-3 minutes. Combine that with the fact that each test file has multiple test cases, and a file might take upwards of 15 or more minutes to finish execution. Now, with multiple test files, your entire test suite might take upwards of an hour to finish execution. This isn't conducive to agile programming. The solution: parallelize your test execution. With Ruby and RSpec, this is handled easily with the addition of two gems: parallel_tests and parallel_split_test. You can parallelize test execution of the test files and then the test cases within the files themselves.

With parallel_split_test v0.3.0, you can use regular rspec test options. This means that you can pass custom formatters and write to specific output files. This comes in handy when you want to create JUnit formatted xml files for Jenkins or other continuous integration systems.

Nat Ritmeyer has written a custom RSpec JUnit Formatter that I've been using:

After installing the parallel_split_test gem, execute the test using this command: parallel_split_test my_spec.rb -r ./junit.rb -f JUnit -o results.xml

NOTE: The one caveat is that parallel_split_test will generate an xml file that contains multiple root nodes. I've attached a function below that will normalize all .xml files in the specified folder.

[gist id=5398225 bump=1]

Using parallel_tests and parallel_split_test, we've managed to cut our execution time from 30+ minutes down to 3 minutes. Theoretically, your tests should only take as long as your slowest individual test case. Your only limitation now is how many nodes are available to your Selenium Grid. You are using Selenium Grid, right?

If you have any questions or comments about parallelization in test execution, you can leave a response in the form below or find me on Twitter at @brokenthumbs. Enjoy!