Many friends responded to my previous post on How Much Should You Think about the Long-Term? saying:

Even if the future is uncertain and we know it will change, we should always plan for the long-term. Without a plan, we cease to move forward.

I’m not necessarily in favor or against this philosophy. However I’m concerned about the following points:

• Effort: The effort required to put together an initial direction is very different from the effort required to put a (proper) plan together. Is that extra effort really worth it esp. when we know things will change?
• Attachment: Sometimes we get attached with our plans. Even when we see things are not quite inline with our plan, we think, its because we’ve not given enough time or we’ve not done justice to it.
• Conditioned: Sometimes I notice that when we have a plan in place, knowingly or unknowingly we stop watching out for certain things. Mentally we are at ease and we build a shield around us. We get in the groove of our plan and miss some wonderful opportunities along the way.

The amount of planning required seems to be directly proportional to the size of your team.

If your team consists of a couple of people, you can go fairly lightweight. And that’s my long-term plan to deal with uncertainty.

Every day I hear horror stories of how developers are harassed by managers and customers for not having predictable/stable velocity. Developers are penalized when their estimates don’t match their actuals.

If I understand correctly, the reason we moved to story points was to avoid this public humiliation of developers by their managers and customers.

Its probably helped some teams but vast majority of teams today are no better off than before, except that now they have this one extract level of indirection because of story points and then velocity.

We can certainly blame the developers and managers for not understanding story points in the first place. But will that really solve the problem teams are faced with today?

Please consider reading my blog on Story Points are Relative Complexity Estimation techniques. It will help you understand what story points are.

Assuming you know what story point estimates are. Let’s consider that we have some user stories with different story points which help us understand relative complexity estimate.

Then we pick up the most important stories (with different relative complexities) and try to do those stories in our next iteration/sprint.

Let’s say we end up finishing 6 user stories at the end of this iteration/sprint. We add up all the story points for each user story which was completed and we say that’s our velocity.

Next iteration/sprint, we say we can roughly pick up same amount of total story points based on our velocity. And we plan our iterations/sprints this way. We find an oscillating velocity each iteration/sprint, which in theory should normalize over a period of time.

But do you see a fundamental error in this approach?

First we said, 2-story points does not mean 2 times bigger than 1-story point. Let’s say to implement a 1-point story it might take 6 hrs, while to implement a 2-point story it takes 9 hrs. Hence we assigned random numbers (Fibonacci series) to story points in the first place. But then we go and add them all up.

If you still don’t get it, let me explain with an example.

In the nth iteration/sprint, we implemented 6 stories:

• Two 1-point story
• Two 3-point stories
• One 5-point story
• One 8-point story

So our total velocity is ( 2*1 + 2*3 + 5 + 8 ) = 21 points. In 2 weeks we got 21 points done, hence our velocity is 21.

Next iteration/sprit, we’ll take:

* Twenty One 1-point stories

Take a wild guess what would happen?

Yeah I know, hence we don’t take just one iteration/sprint’s velocity, we take an average across many iterations/sprints.

But its a real big stretch to take something which was inherently not meant to be mathematical or statistical in nature and calculate velocity based on it.

If velocity anyway averages out over a period of time, then why not just count the number of stories and use them as your velocity instead of doing story-points?

Over a period of time stories will roughly be broken down to similar size stories and even if they don’t, they will average out.

Isn’t that much simpler (with about the same amount of error) than doing all the story point business?

I used this approach for few years and did certainly benefit from it. No doubt its better than effort estimation upfront. But is this the best we can do?

I know many teams who don’t do effort estimation or relative complexity estimation and moved to a flow model instead of trying to fit thing into the box.

Consider reading my blog on Estimations Considered Harmful.

Very very powerful thought when starting something!

Myth: If you do iterations or sprints, you are Agile!

Few companies like Industrial Logic, Yahoo,etc and Individuals like David Anderson, Arlo Belshee, Fred George and many more have paved the way for iteration less agile. The Poppendiecks and the Lean community have been a big influence. So its perfectly fine if you think iterations or sprint are high ceremony.

Back in 2003-2004 when I was working on an offshore maintenance project, we got rid of iterations and it helped us great deal. Since then I’ve been on multiple projects where we’ve done Iterations/Sprints and I’ve always felt that we were wasting a lot of time. On some projects I’ve been able to influence the team to get rid of iterations and start focusing on our throughput. And those teams have not looked back since.

But then the question is:

Why does XP and Scrum have the whole iteration/sprint business?

As defined by original Scrum, Sprint is a fixed time box (1 month) at the end of which you deliver working software to the customers. What this really means is you deploy working software that the customer can use. So this is as good as a release. In some cases it might be internal release instead of a public release. Irrespective, I believe that Sprint == Release (some form) as originally intended.

While XP has the concept of iterations and releases. Which means at the end of the iteration you don’t necessarily release software to your customers. But you have a demo at to the customer at end of the iteration. Sometimes a demo is also a working session.

Never the less, as I understand the advantage of having a fixed time box (iteration/sprint) is to take x amount of time when the team is not really disturbed and in return the team commits to deliver x features (belonging to one or more goal). To be able to commit, the team needs to do some form of analysis and estimation to be comfortable with their commitment.

If you talk to Jeff Sutherland about why he decided to have Sprints as fixed time boxes, he will tell you that at that point the developers were constantly being interrupted by their customers/managers, leading to a context switch. This was yielding very poor productivity and lack of focus. The time box addressed this issue by keeping the customers and managers “out of the room” for the time box duration and also helped the developers to set a clear focus on what needs to be achieved.

Makes sense?

So if you don’t really have these 2 fundamental issues (or if you can educate people about them), why would you want to go thru all this ceremony of planning and estimating the time boxes? Esp. when you are not even using these Sprints as actual releases.

The time boxed way of doing things, forces you towards push scheduling. While this model is successful, it has some drawbacks and hence more and more people (esp. the lean software community) is moving towards pull scheduling. Where there are no artificial time boxes. As and when the features are ready (fully tested), they are deployed. We don’t batch a bunch of features (user stories) together as we do in Sprints. Instead we focus on the smooth flow of the features from inception to deployment. Basically trying to avoid waiting time and thus reducing inventory. Also simply applying queuing theory, you’ll understand that smaller the batch size, better the throughput.

In my case the batch size is couple of thin-sliced features. This really helps us keep the focus and we are not unnecessarily spending time context switching. Its important to note that the time it takes to build these features vary hence we don’t try to put an artificial sprint boundary around them. Whenever they are ready (ideally a couple of days), they are deployed. Zero waiting time. In some cases deployment could be internal and in some cases it could be public releases.

I strongly recommend :

• Why Iterations are wasteful
• Kanban System
• Lean Software Development

How much upfront planning, analysis and/or design do you do on your projects?

For me the answer depends. The more complicated/complex and critical the project is less the amount of upfront thinking (planning, analysis and design), I tend to do. In my opinion, its not really a choice I have.

I can spend months drawing pretty pictures and arguing about it. But only when I start prototyping (exploring) I start to understand the problem at hand. Each prototype is used to jointly explore the idea with all the stakeholders. And after I’ve done a few small (1-2 week) prototypes, we (stakeholders and the team) can actually layout things involved on the project.

At this point we can break down the overall project into smaller chucks and deep dive into them one by one (breadth first) or little by little at a time (depth first). In a way this is my planning, analysis and design process. Of course they don’t stop here, its just the beginning.