About   Slides   Home  

Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
RSS Feed
Recent Thoughts
Recent Comments

Running Unit Tests in Parallel

Quicker feedback is more important than ever when it comes to developer testing. More and more developers are switching to run the unit tests from their IDEs with a click of a button. I find it surprising that in spite of such heavy emphasis on quicker feedback, there is no real guidelines to writing independent tests which can run in parallel.

Following are the standard guidelines for good unit tests:

  • Run fast (they have short setups, run times, and tear downs)
  • Run in isolation (reordering possible)
  • Self-contained (does the required set-up and tear down) and independent of other tests
  • Leaves the system in the same state
  • Name tests properly, express intent
  • Should be fully automated and non-interactive
  • Use data that makes them easy to read and to understand
  • Use real data (copies of production data) when they need to

A test is not a unit test if:

  • It talks to a database.
  • It communicates across a network.
  • It touches the file system.
  • You have to do special things to your environment (such as editing configuration files) to run it.

Unit Test Smells:

  • Testing private methods
  • Extensive setup/teardown
  • Brittle tests
  • Slow tests
  • Writing logs and comments in the tests (guess your tests are really complicated)

These a pretty good guidelines for unit testing. It talks about tests being self-contained, independent and order independent. But no one seems to talk about running tests in parallel. When developers write tests shouldn‘t developers write tests that can be run in parallel?

Frameworks like TestNG provides facility to run unit tests in parallel, but I have not noticed teams using this feature. This should be one of the guidelines for writing good unit tests.

Potential issues that could arise due to running the tests in parallel are:

  1. If your tests are sharing/dependent state like Database, running tests in parallel can result in unpredictable behavior.
    My response: First of all, your unit tests should not depend on external state, but even if they do, there are ways to avoid unpredictable behavior. For Ex: for tests that alter database state, you can run your tests inside a transaction, so that other tests don‘t see the state changes in between test runs.
  2. Debugging tests could be a little difficult, if they are run in parallel.
    My response: Writing tests that can run in parallel would mean your tests which depend on external state will be a little more complicated and have to be carefully written. Tests which don‘t depend on external state, should behave the same way. Hopefully once the tests are written they are not difficult to maintain. Also the testing frameworks should provide a way to reduce the thread size to one when debugging tests.

I can see the following really big advantages of running the tests in parallel:

  1. Faster feedback : greater development velocity
  2. Reduction in execution time huge savings from a cost and time perspective
  3. Simulating real world : exposure to potential synchronization issues. In real world, multiple customers will use the application in parallel and hence execution in parallel is a real world requirement.

    Licensed under
Creative Commons License