Amongst the gazillion myths about Agile, one of them is “Agile means no design, its a cowboy coders’ feast”. Give me a break, you think its cool to pretend that code just flows out of your fingers without any design/thinking? Why is thinking about design before coding production code looked down upon? I completely understand the pitfalls of big upfront design and I’m not proposing that. Whether you use CRC cards, white board, acceptance tests, mental models or whatever have you, you are designing. You might not think about it as traditional design. But think about it, you are actually designing. And there is nothing wrong with it. You are not a poor developer if you do that. In fact as I know, some of the most respected programmers in the industry do some form of design before they code production code.

In fact I claim that when you are breaking down your user stories you are designing your software. I actually have an example of how breaking down your stories differently can lead to a very different design. In this blog post, I explain why.

One of the things that I have found very useful when building software is to just prototype 2 or 3 different approaches, to explore the problem domain, before actually TDDing the whole solution. Recently I added version control support to FitNesse.  The approach I used was I tried first logical solution that came to my mind. It worked, but as I started using it, I need more stuff and the design I had would not really fit well the new requirement I had. I could have incrementally refactored the design to suite the new requirements, but since I treated the code as prototype, I threw it away and started from scratch. It seemed a lot less effort to throw it away and rebuild it from ground zero. Again this time, I just treated the code as prototype. I was really exploring a new idea and I wanted to test the design by writing some quick and dirty code. Usually these sessions might last a couple of days. These are clearly not months worth of effort.

It turned out that my second design worked for single user, but would not work for multiple users. It had a fundamental design problem that I could not have thought before hand. (May be some other smart people could have found it before we started, but I could not. May be having a pair would have helped, but this was open source and I did not have a pair).  Again this was a quick and dirty way to get a high level design feedback.

This style really complements the whole TDD philosophy. Its not that you do a quick and dirty prototype and you don’t need to do TDD any more. Prototypes really help you explore a broader aspect of your system design than TDD can do. TDD helps get feedback about your design and also helps build a safety net, but there are multiple design options and TDD helps you achieve one of those designs. How do you know if there is some other class or library that can actually work, sometimes TDD can help you with that. But if you are completely unaware of its existence, you would need an exploratory session to figure out your options. Once you know what options you have, TDD can really help you solidify it. Hope you get my point.

Sometimes, I write a high-level automated test and build various prototype. The automated test really cuts the manual check at the end.

Prototype is similar to the spiking practice that XP suggests. Somehow spiking has really fallen of the face of the earth. But XP really recommends you spike out various approaches to solve a problem, pick the best and then TDD the solution.

Prototyping is not a new practice its been around for ages. In fact its been around not just in software but also in other disciplines (craft related) like painting. A lot of times, painters create clay models of their subject to understand the depth or other dimensions about the subject. This really helps them give the real feel to their painting.

Now that your are convinced about the importance of prototyping, the next question is should you always prototype? Of course not. If you are not sure about the design, if you want to explore the problem domain, or if you think there is high technical risk with what you are building, sure. Else its just ritual (waste of time).

So when does prototyping stop? At some point you might feel confident enough that you understand the problem domain enough and you can switch to TDDing your production code. But if the requirements changes and something fundamental on which your design decision was based changes, you might be able to just incrementally refactor your design to suit the new requirement. Sometimes, you hit the point where trying a prototype and throwing away your existing design might be the best option. You are the best judge.

Just got back from the 3rd SDTConf. I’m very happy with the conference overall. We had over 40 participants at the conference. (A little less than I expected, but they were all really passionate people). That’s right, Quality OVER Quantity.

We discussed some really interesting topics. Check out the conference schedule for more details. I also got to pair with Micah Martin on the Avatars of TDD. It was fun. We hacked some Ruby using RSpec.

Big thanks to Jim Newkirk for helping me organize this. Also big things to all our sponsors, Microsoft, Agile Alliance, Rally, Version One, SQE and Open Information Foundation who made this conference possible.

Toolkit: A set of related and reusable classes designed to provide useful, general-purpose functionality. Ex: Collections, IO Library, etc

Framework: A set of cooperating classes that make up a reusable design for specific types of software. Ex: JUnit, Spring, etc

Accordingly to GoF Design Patterns book:

• Creating a class by specifying a class explicitly : If need be use creational patterns instead
• Dependence on specific methods : If need be use Chain of Responsibility or Command pattern instead
• Dependence on Hardware and/or software platforms : Consider using Abstract Factory or Bridge pattern instead
• Dependence on Object Representation or Implementation : Must program to interface and not to implementation. Abstract Factory, Bridge, Memento, Proxy etc patterns can help.
• Dependence on specific Algorithms : Consider using Strategy or Visitor pattern
• Extending functionality by subclassing : Prefer composition over inheritance
• Inability to alter classes conveniently : esp. with third party libraries. Adapter, Decorator and Visitor can help.

What’s surprising is that decades ago people knew about these problems, but still pretty much every system has these problems. What are they doing that these anti-patterns are so sustainable? 

• Inheritance (White-box reuse)
• Composition (Black-box reuse)
• Parametrized Types or Generics

While inheritance makes is easy for extending/reusing behavior, sure has some serious flaws which makes it the last choice for reuse.

• Class inheritance is defined statically at compile time. This means you cannot change the implementations inherited from the parent at run-time.
• Inheritance also breaks encapsulation since the subclass is exposed to the details of the parent’s class.
• There is tight coupling between super class’ implementation and subclass’ implementation. Any changes in the parent, forces the child class to also change.

Some of these drawbacks can be avoided by using Interface based inheritance instead of Class based inheritance.

Usually we want to favor object composition over class inheritance. Object composition overcomes all these problems by talking to composed class via their public interface. Favoring composition also helps in keeping each class encapsulated and focused on one task thus encouraging Single Responsibility principle.

Parameterized types or Generics is also another interesting way to reuse behavior/code. For example, to declare a list of String objects, you supply the String type as a parameter. The language implementation will create a customized version of the List class template for each type of element.

• Clients remained decoupled and unaware of specific class they are using, as far as the underlying class adheres to the defined interface. 
• The client does not need to know with which concrete class its interacting.
• Depending on the context, different implementation classes can be polymorphically provided without having to change client code. Makes it easy for the system to evolve.
• During development even though the concrete class is not available, the client class can be developed. Helps parallelize work.

• From the problem statement, single out all the nouns and verbs – end up with a lot of useless objects which lack clear responsibility. Also can argue for ever whether a method belongs on this object or another (Ex: Should Doctor class have a giveShot method or should Patient class have a takeShot method)
• Focus on collaborations and responsibility – can get overwhelming when you look at a large system. Without tests, they can be very confusing to a 3rd person. Without design metaphors and design patterns one cannot easily understand these systems.
• Try to model the real world – Strict modeling of the real world leads to systems as complicated as real world. Also they seem to be rigid, because they only reflect today’s realities, but not necessarily tomorrows.
• Test Driven Design with Spiking – If the developer does not understand basic design principles and OO concepts, this can lead to big ball of mud. Spiking helps in flushing out the overall approach, while Unit and Acceptance Tests really help in validating the design and creating safety net around the code for easy refactoring. 
• Prototyping followed by Design elaboration phase – one or more quick prototypes helps understand and learn about the system. Based on this learning, developers can choose any of the above design approaches. Big trouble when the prototype is not thrown away and is converted into production code.
• Cow boy coding : Design what? Just type it in notepad and pray it will all work. 

Clearly there is no one best way to do things. My present favorite is Prototyping followed by TDD with a slant on collaborations and responsibility.

Short: They help us solve recurring design problems. Note that : design patterns don’t solve the problem themselves, they help us solve the problem.

Detailed answers:

• Communication, Learning and Enhanced Insight: Over the last decade design patterns have become part of every developer’s vocabulary. This really helps in communication. One can easy tell another developer on the team, “I’ve used Command pattern here” and the other developer understands not just the design, but can also easily figure out the rationale behind it. Design Patterns really help in learning, esp. when you are new on a project. Also this helps in providing developers with better insight about parts of the application or 3rd party frameworks they use.
• Decomposing System into Objects : The hard part about OO Design is finding the appropriate objects and decomposing a system. One has to think about encapsulation, granularity, dependencies, flexibility, performance, evolution, reusability and so on. They all influence decomposition, often in conflicting ways. Design Patterns really helps identify less obvious abstractions. These objects are seldom found during analysis or even the early design, they’re discovered later in the course of making a design more flexible and reusable.
• Determining Object Granularity: One thing I struggle a lot with is finding the right level of abstraction and granularity. Design patterns helps in coming up with objects with different levels of granularity that makes sense. 
• Specifying Object Interface : Identifying the right interface and the relationship between various interface is not a one-shot activity. Usually it takes several iterations to identify the right composition of interfaces. Forget interfaces, most of the times, coming up with a method signature can also be quite tricky. Design Patterns really helps in this area.
• Specifying Object Implementation : How should we implement an Object? Given an interface there could be multiple concrete classes of that type, each one can have very different implementations. Design Patterns provide guidance like Program to an interface (type) not an implementation (concrete class) which can result in really good OO code.
• Ensuring right reuse mechanism : When to use Inheritance, when to use Composition, when to use Parameterized Types? Is delegation the right design decision in this context? There are various questions that comes to a programmer’s mind when they are trying to design highly reusable and maintainable code. Knowledge of design patterns can really come handy when making such decisions.
• Relating run-time and compile time structures : An object oriented program’s run-time structure often bares little resembles to this code structure. Sometimes looking at the code does not give us the insights into run-time structure. Knowledge of design patterns can make some of the hidden structure obvious.
• Designing for change : We all know that lack of continuous refactoring and design that doesn’t take change into account risks major redesign in the future. Over the years we’ve also learnt that big upfront designs can’t standup against the constant change/additon of software requirements. We’ve leant grouping elements with similar change life cycle together yields in far more flexible and extendable design. If we think some behavior or element of behavior is most likely to change, we try to abstract that behavior in one place. While we understand these concepts are important design patterns really make it possible to design such systems. Each design pattern lets some aspect of the system structure vary independently of other aspects, thereby making a system more robust to a particular kind of change.

During the recent Agile Coach Camp, during one of the discussions, Chet Hendrickson talked about using Pollution as a metaphor to better explain Technical Debt. Following is my interpretation of what he proposed.

Martin Fowler explains Technical Debt as the following:

Technical Debt is a wonderful metaphor developed by Ward Cunningham to help us think about the problem where the architecture of a large software system is designed and developed too hastily. In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future. The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments.

Another way of looking at this situation is that if your code base has a lot of code smells or bad design decisions that make it difficult to add new features at the same rate as you are used to, then you know there is something getting in your way and bringing down your efficiency. In this state you realize that you have borrowed way too much time from the system by cutting corners. You have not paid back the system by investing regularly in cleaning up the code base.

Even thought this metaphor is really powerful in conveying the messaging that if you continue to hack code, after a while the code and its design will get in your way and stop you from making any real progress due to the heavy interest rates you’ll end up paying.

As Martin points out

The tricky thing about technical debt, of course, is that unlike money it’s impossible to measure effectively. The interest payments hurt a team’s productivity, but since we Cannot Measure Productivity, we can’t really see the true effect of our technical debt.

Also what I have noticed on lots of teams is rarely a developer intensionally introduces technical debt. Today’s quick fix turns out to be a bad hack and that leads to adding to the technical debt. Also like financial debt, technical debt builds up slowly without the team members realizing it. Since you cannot directly measure Technical Debt, the only time you realize you have technical debt is when its too late.

If you really think about it, Technical Debt is a lot like Pollution. Like pollutants in the environment, quick fixes and hacks, degrades the system in which they live. They constantly bring down developer’s ability to build new features (harm people living in the affected area). Pollutants gradually affect people and as time passes by, makes it exponentially harder and costly to fix it.

What is interesting about the pollution metaphor is that few years back who realized CO2 leads to pollution and global warming. But today its all over the place. People driving 2 stroke vehicles never realized that they were polluting the environment. Similarly, few years back who realized Switch Statements are considered harmful? Now with so much emphasis on Code Smells, we can think about these things and avoid some of them.

Also like Pollution, one person introduces a hack into the system and all it’s neighbors (team members and customers) suffer from it. Note that you don’t suffer as soon as you introduce it, its only after delta T that you start feeling the pain.

The biggest problem I find in dealing with Technical Debt is that one man’s elegant solution might look like another man’s hack. Terms like “clean”, “simple” are very relative. To make matters worse, today’s elegant solution might look like a hack tomorrow to the same person. We constantly keep learning better and simpler ways to do things. Tools and languages are constantly improving and getting powerful. What bothers me the most is that people talk about Simplicity as if it were one thing. Unfortunately, simplicity is not as Black and White as I would have liked it.