XNSIO
  About   Slides   Home  

 
Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
     
`
 
RSS Feed
Recent Thoughts
Tags
Recent Comments

Avatars of Test Driven Development (TDD)

Tuesday, March 19th, 2013

It’s easy to speak of test-driven development as if it were a single method, but there are several ways to approach it. In my experience, different approaches lead to quite different solutions.

In this hands-on workshop, with the help of some concrete examples, I’ll demonstrate the different styles and more importantly what goes into the moment of decision when a test is written? And why TDDers make certain choices. The objective of the session is not to decide which approach is best, rather to highlight various different approaches/styles of practicing test-driven development.

By the end of this session, you will understand how TTDers break down a problem before trying to solve it. Also you’ll be exposed to various strategies or techniques used by TDDers to help them write the first few tests.

Outside In – Behaviour Driven Development (BDD)

Tuesday, March 19th, 2013

Recently at the Agile India 2013 Conference I ran an introductory workshop on Behavior Driven Development. This workshop offered a comprehensive, hands-on introduction to behavior driven development via an interactive-demo.

Over the past decade, eXtreme Programming practices like Test-Driven Development (TDD) and Behaviour Driven Development (BDD) have fundamentally changed software development processes and inherently how engineers work. Practitioners claim that it has helped them significantly improve their collaboration with business, development speed, design & code quality and responsiveness to changing requirements. Software professionals across the board, from Internet startups to medical device companies to space research organizations, today have embraced these practices.

This workshop explores the foundations of TDD & BDD with the help of various patterns, strategies, tools and techniques.

Inverting the Testing Pyramid

Tuesday, March 19th, 2013

As more and more companies are moving to the Cloud, they want their latest, greatest software features to be available to their users as quickly as they are built. However there are several issues blocking them from moving ahead.

One key issue is the massive amount of time it takes for someone to certify that the new feature is indeed working as expected and also to assure that the rest of the features will continuing to work. In spite of this long waiting cycle, we still cannot assure that our software will not have any issues. In fact, many times our assumptions about the user’s needs or behavior might itself be wrong. But this long testing cycle only helps us validate that our assumptions works as assumed.

How can we break out of this rut & get thin slices of our features in front of our users to validate our assumptions early?

Most software organizations today suffer from what I call, the “Inverted Testing Pyramid” problem. They spend maximum time and effort manually checking software. Some invest in automation, but mostly building slow, complex, fragile end-to-end GUI test. Very little effort is spent on building a solid foundation of unit & acceptance tests.

This over-investment in end-to-end tests is a slippery slope. Once you start on this path, you end up investing even more time & effort on testing which gives you diminishing returns.

In this session Naresh Jain will explain the key misconceptions that has lead to the inverted testing pyramid approach being massively adopted, main drawbacks of this approach and how to turn your organization around to get the right testing pyramid.

Its all because you did not write a Unit Test…

Tuesday, November 1st, 2011

Recently we realized that our server logs were showing ‘null’ for all HTTP request parameters values.

SEVERE: Attempting to get user with null userName
parameters=[version:null][sessionId:null][userAgent:null][requestedUrl:null][queryString:null]
[action:null][session:null][userIPAddress:null][year:null][path:null][user:null]

On digging around a bit, we found the following buggy code in our custom Map class, which was used to hold the request parameters:

@Override
public String toString() {
    StringBuilder parameters = new StringBuilder();
    for (Map.Entry<String, Object> entry : entrySet())
        parameters.append("[")
                .append(entry.getKey())
                .append(":")
                .append(get(entry.getValue()))
                .append("]");
    return parameters.toString();
}

When we found this, many team members’ reaction was:

If the author had written unit tests, this bug would have been caught immediately.

Others responded saying:

But we usually don’t write tests for toString(), Getters and Setters. We pragmatically choose when to invest in unit tests.

As all of this was taking place, I was wondering, why in the first place, the author even wrote this code? As you can see from the following snippet, Maps already know how to print themselves.

@Test
public void mapKnowsHowToPrintItself() {
    Map hashMap = new HashMap();
    hashMap.put("Key1", "Value1");
    hashMap.put("Key2", "Value2");
    System.out.println(hashMap);
}
Output: {Key2=Value2, Key1=Value1}

Its easy to fall into the trap of first writing useless code and then defending it by writing more useless tests for it.

I’m a lazy developer and I always strive real hard to write as little code as possible. IMHO real power and simplicity comes from less code, not more.

Unit Testing Dilemma: Should I Invest or Not?

Tuesday, November 1st, 2011

Every single line of code must be unit tested!

This sound advice rather seems quite extreme to me. IMHO a skilled programmer pragmatically decides when to invest in unit testing.

After practicing (automated) unit testing for over a decade, I’m a strong believer and proponent of automated unit testing. My take on why developers should care about Unit Testing and TDD.

However over the years I’ve realized that automated unit tests do have four, very important, costs associated with them:

  • Cost of writing the unit tests in the first place
  • Cost of running the unit tests regularly to get feedback
  • Cost of maintaining and updating the unit tests as and when required
  • Cost of understanding other’s unit tests
One also starts to recognize some other subtle costs associated with unit testing:
  • Illusion of safety: While unit tests gives you a great safety net, at times, it can also create an illusion of safety leading to developers too heavily relying on just unit tests (possibly doing more harm than good.)
  • Opportunity cost: If I did not invest in this test, what else could I have done in that time? Flip side of this argument is the opportunity cost of repetitive manually testing or even worse not testing at all.
  • Getting in the way: While unit tests help you drive your design, at times, they do get in the way of refactoring. Many times, I’ve refrained from refactoring the code because I get intimidated by the sheer effort of refactor/rewrite a large number of my tests as well. (I’ve learned many patterns to reduce this pain over the years, but the pain still exists.)
  • Obscures a simpler design: Many times, I find myself so engrossed in my tests and the design it leads to, that I become ignorant to a better, more simpler design. Also sometimes half-way through, even if I realize that there might be an alternative design, because I’ve already invested in a solution (plus all its tests), its harder to throw away the code. In retrospect this always seems like a bad choice.
If we consider all these factors, would you agree with me that:
Automated unit testing is extremely important, but each developer has to make a conscious, pragmatic decision when to invest in unit testing.
Its easy to say always write unit tests, but it takes years of first-hand experience to judge where to draw the line.

Importance of Unit Testing and Test Driven Development (TDD)

Tuesday, November 1st, 2011

Why should developers care of automated unit tests?

  • Keeps you out of the (time hungry) debugger!
  • Reduces bugs in new features and in existing features
  • Reduces the cost of change
  • Improves design
  • Encourages refactoring
  • Builds a safety net to defend against other programmers
  • Is fun
  • Forces you to slow down and think
  • Speeds up development by eliminating waste
  • Reduces fear

And how TDD takes it to the next level?

  • Improves productivity by
    • minimizing time spent debugging
    • reduces the need for manual (monkey) checking by developers and tester
    • helping developers to maintain focus
    • reduce wastage: hand overs
  • Improves communication
  • Creating living, up-to-date specification
  • Communicate design decisions
  • Learning: listen to your code
  • Baby steps: slow down and think
  • Improves confidence
  • Testable code by design + safety net
  • Loosely-coupled design
  • Refactoring

Inverting the Testing Pyramid

Tuesday, February 1st, 2011

As more and more companies are moving to the Cloud, they want their latest, greatest software features to be available to their users as quickly as they are built. However there are several issues blocking them from moving ahead.

One key issue is the massive amount of time it takes for someone to certify that the new feature is indeed working as expected and also to assure that the rest of the features will continuing to work. In spite of this long waiting cycle, we still cannot assure that our software will not have any issues. In fact, many times our assumptions about the user’s needs or behavior might itself be wrong. But this long testing cycle only helps us validate that our assumptions works as assumed.

How can we break out of this rut & get thin slices of our features in front of our users to validate our assumptions early?

Most software organizations today suffer from what I call, the “Inverted Testing Pyramid” problem. They spend maximum time and effort manually checking software. Some invest in automation, but mostly building slow, complex, fragile end-to-end GUI test. Very little effort is spent on building a solid foundation of unit & acceptance tests.

This over-investment in end-to-end tests is a slippery slope. Once you start on this path, you end up investing even more time & effort on testing which gives you diminishing returns.

They end up with majority (80-90%) of their tests being end-to-end GUI tests. Some effort is spent on writing so-called “Integration test” (typically 5-15%.) Resulting in a shocking 1-5% of their tests being unit/micro tests.

Why is this a problem?

  • The base of the pyramid is constructed from end-to-end GUI test, which are famous for their fragility and complexity. A small pixel change in the location of a UI component can result in test failure. GUI tests are also very time-sensitive, sometimes resulting in random failure (false-negative.)
  • To make matters worst, most teams struggle automating their end-to-end tests early on, which results in huge amount of time spent in manual regression testing. Its quite common to find test teams struggling to catch up with development. This lag causes many other hard-development problems.
  • Number of end-to-end tests required to get a good coverage is much higher and more complex than the number of unit tests + selected end-to-end tests required. (BEWARE: Don’t be Seduced by Code Coverage Numbers)
  • Maintain a large number of end-to-end tests is quite a nightmare for teams. Following are some core issues with end-to-end tests:
    • It requires deep domain knowledge and high technical skills to write quality end-to-end tests.
    • They take a lot of time to execute.
    • They are relatively resource intensive.
    • Testing negative paths in end-to-end tests is very difficult (or impossible) compared to lower level tests.
    • When an end-to-end test fails, we don’t get pin-pointed feedback about what went wrong.
    • They are more tightly coupled with the environment and have external dependencies, hence fragile. Slight changes to the environment can cause the tests to fail. (false-negative.)
    • From a refactoring point of view, they don’t give the same comfort feeling to developers as unit tests can give.

Again don’t get me wrong. I’m not suggesting end-to-end integration tests are a scam. I certainly think they have a place and time.

Imagine, an automobile company building an automobile without testing/checking the bolts, nuts all the way up to the engine, transmission, breaks, etc. And then just assembling the whole thing somehow and asking you to drive it. Would you test drive that automobile? But you will see many software companies using this approach to building software.

What I propose and help many organizations achieve is the right balance of end-to-end tests, acceptance tests and unit tests. I call this “Inverting the Testing Pyramid.” [Inspired by Jonathan Wilson’s book called Inverting The Pyramid: The History Of Football Tactics].

Inverting the Testing Pyramid

In a later blog post I can quickly highlight various tactics used to invert the pyramid.

Update: I recently came across Alister Scott’s blog on Introducing the software testing ice-cream cone (anti-pattern). Strongly suggest you read it.

Single Assert Per Unit Test: Myth

Sunday, November 14th, 2010

You are telling me, if I were unit testing a List class and I wrote tests like the following with 2 assert statements in them, then I’m violating xUnit testing “best practices”.

@Test
public void addingElementIncreasesSizeOfTheList() {
	assertEquals(0, list.size()); 
       //or assertListSizeIs(0); - define your custom assert statement
	list.add("Element");
	assertEquals(1, list.size()); // assertListSizeIs(1);
}

If so, how and why?

I understand that if I wrote a test like the following its not very communicative:

@Test
public void removeElementBasedOnItsIndexAndReduceSizeByOne() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
	assertEquals(0, list.size());
}

in this case, it might be better to write 2 test instead:

@Test
public void removeElementBasedOnItsIndex() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
}
 
@Test
public void reducesSizeByOneOnRemovingAnElement() {
	list.add("Element");
	assertEquals(1, list.size());
	list.remove(0);
	assertEquals(0, list.size());
}

Notice our goal was better communication and one of the tests we ended up (incidentally) with had just one assert statement in it (which is better than the previous test which was asserting 2 different things).

Our goal is not one assert statement per test. Our goal is better communication/documentation. Using the one-assert-per-test heuristic helps to achieve better communication. But splitting tests into more tests or deleting an important assert statement from a test just to avoid multiple assert statements in a test is mindless dogma.

Conclusion: One-Assert-Per-Test really means test one aspect/behavior in each test (and communicate that clearly in your test name). It does not literally mean, one test should have only one assert statement in it. Test one aspect/behavior in one test and related aspect/behavior can be tested in another test. Sometimes we need more than one assert statement to assert one aspect/behavior and its helps to add those assert statements in the test for better communication.

So don’t be afraid, go ahead and add that second or third assert statement in your test. Be careful not to fill your test with many asserts, it becomes difficult to understand and debug.

UPDATE:
Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

@Robert, I certainly like your test. Much better.

Here is another naive example, looking forward to brickbats 😉

Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

@Test
public void isMapFriendly() throws Exception {
	RomanDigit one = new RomanDigit('I', 1);
	RomanDigit anotherOne = new RomanDigit('I', 1);
	Map romanDigits = new HashMap();
	romanDigits.put(one, "One");
	romanDigits.put(anotherOne, "AnotherOne");
	assertEquals(1, romanDigits.size());
	assertEquals("AnotherOne", romanDigits.get(one));
	assertEquals("AnotherOne", romanDigits.get(anotherOne));
}

Another example: When subtracting RomanDigits, only powers of ten can repeat.

@Test
public void onlyPowerOfTenCanRepeate() throws Exception {
	assertTrue("I is a power of Ten and can repeat", romanDigit('I').canRepeat());
	assertFalse("V is not a power of Ten and should not repeat", romanDigit('V').canRepeat());
}

Is there a better way to write these tests so that I only need one assert statement per test?

    Licensed under
Creative Commons License