Building on top of my previous blog entry: Version Control Branching (extensively) Considered Harmful

I always discourage teams from preemptively branching a release candidate and then splitting their team to harden the release while rest of the team continues working on next release features.

My reasoning:

• Increases the work-in-progress and creates a lot of planning, management, version-control, testing, etc. overheads.
• In the grand scheme of things, we are focusing on resource utilization, but the throughput of the overall system is actually reducing.
• During development, teams get very focused on churning out features. Subconsciously they know there will be a hardening/optimization phase at the end, so they tend to cut corners for short-term speed gains. This attitude had a snowball effect. Overall encourages a “not-my-problem” attitude towards quality, performance and overall usability.
• The team (developers, testers and managers) responsible for hardening the release have to work extremely hard, under high pressure causing them to burn-out (and possibly introducing more problems into the system.) They have to suffer for the mistakes others have done. Does not seem like a fair system.
• Because the team is under high pressure to deliver the release, even though they know something really needs to be redesigned/refactored, they just patch it up. Constantly doing this, really creates a big ball of complex mud that only a few people understand.
• Creates a “Knowledge/Skill divide” between the developers and testers of the team. Generally the best (most trusted and knowledgable) members are pick up to work on the release hardening and performance optimization. They learn many interesting things while doing this. This newly acquired knowledge does not effectively get communicate back to other team members (mostly developers). Others continue doing what they used to do (potentially wrong things which the hardening team has to fix later.)
• As releases pass by, there are fewer and fewer people who understand the overall system and only they are able to effectively harden the project. This is a huge project risk.
• Over a period of time, every new release needs more hardening time due to the points highlighted above. This approach does not seem like a good strategy of getting out of the problem.

If something hurts, do it all the time to reduce the pain and get better at it.

Hence we should build release hardening as much as possible into the routine everyday work. If you still need hardening at the end, then instead of splitting the teams, we should let the whole swamp on making the release.

Also usually I notice that if only a subset of the team can effectively do the hardening, then its a good indication that the team is over-staffed and that might be one of the reasons for many problems in the first place. It might be worth considering down-sizing your team to see if some of those problems can be addressed.

Why do some authors call their tutorials as Cook books?

Its a collection of guidelines (recipes) to use the software. Very similar to cookbooks or recipe books.

Cooking has been used a metaphor/analogy for software development for many decades now. Some people have even compared Developers to Chefs, [poor] Analysts to Waiters and so on.

I find a very close resemblance between the way I cook food and the way I build software.

• Both an very much an iterative and incremental process. Big bang approaches don’t work.
• Very heavy focus on feedback and testing (tasting, smelling, feeling the texture, etc) early on and continuously throughout. We don’t cook the whole meal and then check it. The whole cooking process if very feedback driven.
• Like Software, each meal has many edible food items (features) in it. Each food item has basic ingredients (that fills your stomach)[skeleton; must-have-part of the feature], ingredients that give the taste, color & meaning to the food and ingredients that decorates the food [esthetics]. We prioritize and thin slice to get early feedback and to take baby steps.
• Like in software, fresh ingredients [new feature ideas] are more healthy and sustainable.
• Cooking is an art. You get better at it by practicing it. There are no crash courses that can make you a master cook.
• Cooking has some fundamental underlying principles that can be applied to different styles of cooking and to different cuisines. Similarly in software we have different schools of thoughts and different frameworks/technologies where those fundamental principles can be applied.
• We have lots of recipe books for cooking. 2 different cooks can take the same recipe and come up with quite different food (taste, odor, color, texture, appeal, etc). A good cook (someone with quality experience) knows how to take a recipe and make wonderful food out of it. Other get caught up in the recipe. They miss the whole point of cooking and enjoying food.
• Efficiency can vary drastically between a good cook and a bad cook. A good cook can deliver tasty food up to 10 times faster than a lousy cook.
• Cooking needs passion and risk taking attitude. A passionate cook, willing to try something new, can get very creative with cooking. Can deliver great results with limited resources. Someone lacking the passion will not deliver any edible food, even if they are give all the resources in the world.
• Cooking has a creative, experimental side to it. Mixing different styles of cooking can leading to wonderful results.
• Cooking is a constant learning and exploratory process. This is what adds all the fun to cooking. Not cooking the same old stuff by reading the manual.
• In cooking, there are guidelines no rules. One with discipline and one who has internalized the guidelines can cook far better than the one stuck with the rules and processes.
• “Many cooks spoil the broth”. You can’t violate Mythical Man Month.

Also if we broaden the analogy to Restaurant business, we can see some other interesting aspects.

I’m thrilled to be back in Bangalore. One thing I really missed in US was the craziness of driving on Bangalore roads. Everything was so systematic, there was no thrill, no excitement, no fun, I would be bored to death. The other day I started to think what did I really like about driving on Indian roads? There was something about driving here, that strikes a chord with the way I work [you would have guess by know, .i.e. develop software]. I concluded that, driving on Indian roads have a lot to do with the way I build/develop software .i.e. using Agile methods. Here are my initial thoughts why I think so…

• Chaos from outside : If you are not driving on Indian roads, it looks like the epicenter of chaos and noise. It will take a while for you to figure out why the traffic on the roads are oriented in 360 degrees. Everyone seems to be honking at each other. If you talk to someone who has watched an Agile team work for a week, they can share similar thoughts about chaos and noise. Companies where I have tried to introduce Agile, chaos and noise seems to be one of their top concerns. Once you are part of the system, life is wonderful. Trust me, you will never look back again.
• Embrace change : If you drive though a street and come back to it a week later, there is a high probability that the direction in which the traffic flows would be reversed. Changing the directions of one-ways is very common here. Driving 2 wheels with helmets on is another rule that has changed at least 10 times in the last few years. There is a constant evolution. The good thing is folks embrace change and continue with it. Strikes a chord with what happened yesterday during your planning meeting?
• Heavy emphasis on open communication and continuous feedback : If you watch people in India drive, you might wonder why they honk continuously and sometimes flash their headlights? Well, that’s our way of communication and feedback. If you watch carefully, the intensity of honking and the frequency of flashing lights varies depending on what one driver is trying to communicate to the other. A gentle honk, means, I’m about to overtake you, watch out. Continuous honking means, move out of my way, you are wasting my time. A gentle flash of light means watch out, I’m behind out. All of us know the importance of open communication and continuous feedback on software projects.
• Self organized : If you look at the traffic rules in India, we are not so rigid about the rules. You will find people overtaking from left and right. Sometimes you’ll find group of vehicles jumping the lights when the opposite side is empty. We don’t even have cops standing everywhere with Speed guns. A lot of busy junctions, don’t have cops. People nicely self organize and move on. I believe in self organized teams and I think that is the most effective and scalable way to build software.
• Adaptive planning : Its quite common to find one of the roads in your route blocked due to some maintenance work or accident. It very rare you’ll find people waiting for it to clear up. Immediately people find another route and move on. We plan to deliver and not deliver to a plan. Its very rare to find that everything on a software project going according to the plan. Things change, we always have surprises and we learn to adapt and move one. Also note that let it be heavy rains, floods or heavy winds, none of these seem to bring the traffic to a standstill. People adapt and move on.
• Stressful and Intense : No doubt driving is a lot of fun, but driving on Indian roads is stressful and can be intense. This is the same feedback I get from people who work on Agile projects. Pair programming session can be quite stressful and intense. Trust me, you’ll feel much more exhausted in 4 hours of pair programming session compared to 10 hours of working on your own. Both of these activities needs a lot of attention and can drain you out soon.
• Mitigate risk : From outside, it might feel like driving on Indian roads is very risky. But if you look at the number of deaths caused due to accidents in India, its far less than that caused in US. Please note that number of deaths includes humans and animals. For those who live in suburbs in US, can tell you how many deers they find dead everyday. The trick is with people traveling so close to each other, there are very little chances to be caught by surprise. With constant feedback and continuous collaboration on Agile projects we try to mitigate a lot of project risks. Pair programming and collective ownership is another great way of mitigating the risk of one person being the bottlenecks.
• Maximize the throughput : If you notice the number of people that travel through these roads each day, its amazing. Look at the roads, every inch of the road is occupied and sometimes we don’t even spare the footpaths. We like to maximize the usage of the available resources (capacity utilization). But at the same time we focus on making sure we reach home/office fast. (throughput). Every software project tries to maximize its throughput and Agile gives me a great platform to do so.
• Minimize the operating cost : I’m not too sure, but my gut feel says the amount of money and effort invested in maintaining and building roads in India is far less than what they do in other countries like US. Also consider the ratio of number of cops to the number of people traveling through these roads; it is very very small. We believe in reducing any form of overhead 😉 This is the harsh reality of software projects. But Agile helps me strike a good balance.
• Agility [the verb] : With the short distance between the vehicles, one needs to have really good reflexes to survive on Indian roads. Its amazing to see everyone jamming their brakes when something unexpected happens. They recover and move on. Being Agile [verb] is the key to success on a software project.

Having said all this, its always easy to find things that are done quite different in these 2 scenarios. Taking any analogy to an extreme causes more harm than good.