Over the last year, I’ve been helping (part-time) Freeset build their ecommerce website. David Hussman introduced me to folks from Freeset.

Following is a list of random topics (most of them are Agile/XP practices) about this project:

• Project Inception: We started off with a couple of meetings with folks from Freeset to understand their needs. David quickly created an initial vision document with User Personas and their use cases (about 2 page long on Google Docs). Naomi and John from Freeset, quickly created some screen mock-ups in Photoshop to show user interaction. I don’t think we spent more than a week on all of this. This helped us get started.
• Technology Choice: When we started we had to decide what platform are we going to use to build the site. We had to choose between customer site using Rails v/s using CMS. I think David was leaning towards RoR. I talked to folks at Directi (Sandeep, Jinesh, Latesh, etc) and we thought instead of building a custom website from scratch, we should use a CMS. After a bit of research, we settled on CMS Made Simple, for the following reasons
  • We needed different templates for different pages on the site.
  • PHP: Easiest to set up a PHP site with MySQL on any Shared Host Service Provider
• Planning: We started off with an hour long, bi-weekly planning meetings (conf calls on Skype) on every Saturday morning (India time). We had a massively distributed team. John was in New Zealand. David and Deborah (from BestBuy) were in US. Kerry was in UK for a short while. Naomi, Kelsea and other were in Kolkatta and I was based out of Mumbai. Because of the time zone difference and because we’re all working on this part time, the whole bi-weekly planning meeting felt awkward and heavy weight. So after about 3 such meetings we abandoned it. We created a spreadsheet on Google Docs, added all the items that had high priority and started signing up for tasks. Whenever anyone updated an item on the sheet, everyone would be notified about the change.
• User Stories: We started off with User Persona and Stories, but soon we just fell back to simple tasks on a shared spreadsheet. We had quite a few user related tasks, but just one liner in the spread sheet was more than sufficient. We used this spreadsheet as a sudo-backlog. (by no means we had the rigor to try and build a proper backlog).
• Short Releases: We (were) only working on production environment. Every change made by a developer was immediately live. Only recently we created a development environment (replica of production), on which we do all our development. (I asked John from Freeset, if this change helped him, he had mixed feelings. Recently he did a large website restructuring (added some new section and moved some pages around), and he found the development environment useful for that. But for other things, when he wants to make some small changes, he finds it an over kill to make changes to dev and then sync it up with production. There are also things like news, which makes sense to do on the production server. Now he has to do in both places). So I’m thinking may be, we move back to just production environment and then create a prod on demand if we are plan to make big changes.
• Testing: Original we had plans of at least recording or scripting some Selenium tests to make sure the site is behaving the way we expected it to. This kind of took a back seat and never really became an issue. Recently we had a slight set back when we moved a whole bunch of pages around and their link from other parts of the site were broken. Other than that, so far, its just been fine.
• Evolutionary Design: Always believed in and continue to believe in “Do the Simplest, Dumbest, thing that could Possibly work“. Since we started, the project had taken interesting turns, we used quite a lot of different JavaScript libraries, hacked a bit of PHP code here and there. All of this is evolving and is working fine.
• Usability: We still have lots of usability and optimization issues on our site. Since we don’t have an expert with us and we can’t afford one, we are doing the best we can with what we have on hand. We are hoping we’ll find a volunteer some day soon to help us on this front.
• Versioning: We explored various options for versioning, but as of today we don’t have any repository under which we version our site (content and code). This is a drawback of using an online CMS. Having said that so far (been over a year), we did not really find the need for versioning. As of now we have 4 people working on this site and it just seems to work fine. Reminds me of YAGNI. (May be in future when we have more collaborators, we might need this).
• Continuous Integration: With out Versioning and Testing, CI is out of question.
• Automated Deployment: Until recently we only had one server (production) so there was no need for deployment. Since now we have a dev and a prod environment, Devdas and I quickly hacked a simple shell scrip (with mysqldump & rsync) that does automated deployment. It can’t get simpler than this.
• Hosting: We talked about hosting the site on its own slice v/s using an existing shared host account. We could always move the site to another location when our existing, cheap hosting option will not suit our needs. So as of today, I’m hosting the site under one of my shared host account.
• Rich Media Content: We questioned serving & hosting rich media content like videos from our site or using YouTube to host them. We went with YouTube for the following reasons
  • We wanted to redirect any possible traffic to other sites which are more tuned to catering high bandwidth content
  • We wanted to use YouTube’s existing customer base to attract traffic to our site
  • Since we knew we’ll be moving to another hosting service, we did not want to keep all those videos on the server which then will have to be moved to the new server
• Customer Feedback: So far we have received great feedback from users of this site. We’ve also seen a huge growth in traffic to our site. Currently hovering around 1500 hits per day. Other than getting feedback from users. We also look at Google Analytics to see how users are responding to changes we’ve made and so on.
• We don’t really have/need a System Metaphor and we are not paying as much attention to refactoring. We have some light conventions but we don’t really have any coding standards. Nor do we have the luxury to pair program.
• Distributed/Virtual Team: Since all of us are distributed and traveling, we don’t really have the concept of site. Forget on-site customer or product owner.
• Since all of this is voluntary work, Sustainable pace takes a very different meaning. Sometimes what we do is not sustainable, but that’s the need of the hour. However all of us really like and want to work on this project. We have a sense of ownership. (collective ownership)
• We’ve never really sat down and done a retrospective. May be once in a while we ask a couple of questions regarding how something were going.

Overall, I’ve been extremely happy with the choices we’ve made. I’m not suggesting every project should be run this way. I’m trying to highlight an example of what being agile really means.

While everyone agrees with the value of eating your dog food, some people claim that this principle cannot be applied to all software industries.

Let’s take the Medical Health Care Industry. Who should build software for Doctors and Nurses to be used in the hospital? Its very unlikely that Doctors will start building software in the side. How do apply this principle here?

What we have today is a bunch of people trying to build software for the hospitals (most of them have not clue on how a hospital operates, those who know a little become Subject Matter Experts and take charge). Similarly there are lot of other industries.

You ask their users how they like the software and you would know. Its not that the development team did not do a good job of building the features right or the business did not do a good job of articulating what they want well. Its just that this model is setup for failure.

• The Agile community realized that, they need to bring the users in and collaborate with them much more.
  • The Scrum community identifies one person or a group, call them Product Owner. They are part of the planning meeting, daily scrum and even the retrospectives & demos. Some (0.1%) of teams are able to get actual users during their demo. Are they confused about the PO being their User?
  • The XP community demands an onsite customer who can guide the team not just during planning, but also during execution. Again the same confusion exists. But the situation is slightly better.
  • Having said that, I really appreciate XP for pushing the knob on automated testing. Automated Testing (esp. Developer testing) is a great way to eat your own dog food. Remember how useful your API tests have proved to be. Tests are clients to your code and they consume your code by acting as client.
• The Design (UX) Community are lot more User focused and tend to spend more time with the actual Users, but that’s very sporadic
• The Lean community have realized that they need to have the development team sit with the business in their work area. They have realized that there are a lot of important lessons to learn from the context of the work place.

Personally I think we need to go way beyond this. If you look at some organizations (esp. Web 2.0 companies and Open Source Projects) they are their own Users. We can certainly learn something from them.

How can we do this? Here are some ideas:

• At least to start with, have the team members take a formal education in the domain they are building the software. Do some case studies and then, spend quality time with the Users (actual Users). Not just interviewing them, but actually working with them (at least shadowing them or being their apprentice).
• Educate the Users more about Software development process and have them work with the team for at least a week or two to under it.
• May be hire people who have actually worked in the field. (You want to make sure their knowledge is up-to-date and they actually know the business really well). Also very important to maintain a good ratio. 1 member for a 10 people team is scary.
• Build tools that can help the actual end users build/configure their software. As developers we build tools which we use on our own projects. Same tools (which were driven by eating their own dog food) can now be used by others to build their software. For years, creating a web presence for a company was a specialist’s job. Today with Google Apps and others, anyone can set up a website, add a bunch of forms, set up email accounts and all that Jazz. The line between a specialist’ role and a business user is blurring. Coz we have the tools to help. Esp. tools built by people for their personal use.
• Again all of this can get you one step closer. But nothing like eating your own dog food.

It appears to me that the Agile Community is falling behind the innovation curve. At conferences, user groups, mailing list, etc, we see the same old same old stuff (may be I’m missing something). So where is the real innovation happening? What space should I be watching?

These were the questions I posed to the group @ the SDTConf 2009. Later, during our discussion at the conference we tried answering them. After a wonderful discussion we come up with some suggestions:

• Web 2.0
  • Highly Scalability, Performance and Operations space
  • No SQL
  • Continuous Deployment and Monitoring space – Owen’s Slides are a good starting point
  • Watch out for conferences like O’Reilly’s Velocity
• Alternative Language (non-mainstream languages) space. Lot of interesting experiments going on in
  • Dynamic language space
  • Functional language space
  • Hybrid language space
• Domain Specific Language space
• Could Computing, Parallel Computing (Grid Computing), Virtualization space
• Code Harvesting Space – Check out Test Driven Code Search and Code Genie as a starting point
• Complex Adaptive Systems and its implication on our social interactions space. Dave Snowden’s work is a good starting point
• eLearning and visual assessments (feedback) of a programming session. Check out Visualizing Proficiency
• Polyglot Programming space
• With Google Apps, people are able to build 100s of Apps each month and get instant feedback on their ideas
• Social Networking and Second Life space
• Conference: Lot of interesting experiments are been conducted in the conference space. Conferences have evolved to something very different from before.
• Distributed Development and Remote Pairing space

If you would like to contribute to this list, please add your point on the SDTConf Wiki.

Lot of organizations in India are considering Agile.

• Agile Adoption in Services Companies is mostly driven by customers and competition.
• For product companies its all about
  • Improving productivity,
  • Reducing time to market and
  • Better quality.

While these companies are adopting agile, they have lots of concerns and questions. One recurring theme I’ve seen, is all these companies are really really afraid of failure. IMHO, this fear leads to a very dysfunctional agile adoption. Teams pick what is easy to do and what fits into their existing model, in the name of reducing risk, call it fail-safe 😉 . With this approach individuals and companies fail to see the real benefit of Agile.

This issue is compounded by the fact that a lot of companies are selling substandard certification, tools and consulting.

Also most Indian companies work in a distributed model. Distributed development is another challenge teams are facing and they don’t really see how to fit Agile in their distributed context.

All these points put together makes it very difficult for team to succeed.

These are my biggest concerns today with Agile adoption in India. Does this ring a bell? Can you share your thoughts?

I would be really happy if you want to share your agile adoption story with the agile software community of India. If you can address some/all of the following questions it would be great:

• Why did your company decide to consider Agile?
• How did you build a business case for going Agile? Can you share some artifacts from this original business case?
• Who all was driving this change?
• What measures were taken to set realistic expectations with both Management and Development teams?
• Did you start with a Pilot project? If yes,
  • What were the criteria for picking a pilot project?
  • What was required to get the pilot project up and running?
  • Was the pilot project a success? What were your success criteria?
  • Post the pilot project what did your organization do?
  • How did you roll out Agile to rest of your organization based on your learning from the Pilot project?
• From when you started to now, can you give us important milestones and some artifacts with regards to the same?
• Was there a key turning point in this journey? If yes, what was it?
• Looking back, what mistakes you think could/should have been avoided? And what mistakes you think were worth committing?
• Where is your journey with Agile heading? What are your future plans?
• What does a typical day in life of each team member (developer, tester, manager, etc) look? Any pictures/artifacts you can share?
• What impact did Agile have on your organization structure?
• What mechanisms did you use to guide your teams if they were going down the right direction?
• After having gone through it, do you think it was worth it?
• Any thing else you would like to share?

Till a year ago, I used to think of command & control as micro-management. Last year I heard Mary Poppendieck give a different perspective about it in her keynote(pdf). That’s when I realized that Command & Control means quite opposite of micro-management.

We all know Command and Control comes from US Army. During wars, the Commanders would direct (command) the troops on ground about the next move (goal/mission). And then the staff (troops) on ground would figure out the best strategy to achieve the mission. This is very different from micro-management.

Once the Commanders gives the commands, there is no way s/he can control the situation on ground. Tactical decisions are made by the staff on the war field and strategic decisions are made by Commanders and others outside the war fields, based on their knowledge about situation on the ground.

(no plan survives contact with the real enemy).

The heart of mission command:
Commanders should issue only the most essential orders. These provide general instructions outlining the principal objective and specific missions. Tactical details are left to subordinates.

“The advantage which a commander thinks he can attain through continued personal intervention is largely illusory.”

http://www.dtic.mil/dticasd/sbir/sbir043/a30a.pdf

@agilemanager
@WardCunningham
@marick
@JoshuaKerievsky
@jbrains
@bpettichord
@nashjain
@venkat_s
@jeffpatton
@testobsessed
@jamesshore
@coreyhaines
@cory_foy
@mfeathers
@unclebobmartin
@rachelcdavies
@sf105
@jeantabaka
@martinfowler
@KentBeck
@RonJeffries
@GeePawHill
@bigballofmud
@olabini
@deborahh
@redsquirrel
@DianaOfPortland
@sai_venkat
@sandeepshetty
@patrickwelsh
@elefevre
@lukehohmann
@simpsonjulian
@dsaff
@tastapod
@gerardmes
@chzy
@dchelimsky
@athought
@rebeccawb
@mpoppendieck
@pramodsadalage
@lisacrispin
@rebeccaparsons
@odersky
@ericries
@PragmaticAndy
@TotherAlistair
@paulk_asert
@kensipe
@jimweirich
@tedneward
@scottdavis99
@neal4d
@pragdave
@stuarthalloway

This is not a comprehensive list. I’m sure I’m missing some more folks. These folks came to my mind immediately or they have been tweeting recently.

I’ve seen some teams getting so caught up in answering the 3 questions of the stand-up meetings, that they even forget the objective behind the standup meeting.

Following are NOT the goals of a standup meeting:

• Get in-sync or to know who is doing what & where they stand. 
  • That information can be achieved through the story-wall (virtual story-wall if you are using a Agile PM tool). If you are a team of 3-4 people and sitting next to each other, you’ll anyway have this information (you better).  
• To identify impediments. 
  • Why wait till the standup meeting to identify something is blocking you. If you can’t solve something yourself, bring it up to others notice immediately. Don’t sit on it.
• Status report. 
  • This is the worst goal you can have for standup meetings. We are moving away from micro-management. 
• Communication to the chickens. 
  • Again they can look at the project status on the story-wall. Or they can stop by in the team room and talk to someone informally. We don’t need to have a meeting for them.
• And so on…

IMHO the real goals of a standup meeting is

• To come up with a Game Plan till the next standup meeting. 
  • Its an opportunity to inspect and adapt 
  • Helps in doing micro-re-prioritization 
  • Helps in keeping everyone on the same page as far as the goals/vision is concerned

If answering the 3 questions helps towards this goal, then go ahead. But don’t think of them as an end in themselves.

For a given feature, we can come up with multiple thin-slices which can be incrementally (sequentially) built.

Thin Slice is the simplest possible functional, usable, end to end slice of functionality.

Thin Slicing a feature is not a new concept. Generally development teams consider the simple happy path scenario as the first thin slice. The alternative flows are considered as subsequent slices. Today when I think of a thin slice, its slightly more sophisticated than just considering happy path and alternative paths.

Let me explain with an example: Lets consider we are building a web app and we need a feature which allows the user to upload their profile picture.

We can come up with the following thin slices:

• User can upload any photo (predefined set of image format) – won’t look good coz the image size can vary and hence where ever its displayed won’t align well. Might not have AJAX support (depends on what is simple and quick to do). All the bells and whistles are pushed for later.
• Build an image scaling facility so that we can reduce the image resolution and hence its size
• Provide an image cropping facility so that users can crop their profile images
• Instead of uploading an image from my disk, provide a facility to pull it from the web. The user provides the URL
• and so on….

Each slice is functional (end to end, we are not just doing the UI or backend bit). This is great for internal feedback. Might not be good enough for a public release. Esp. with just the first slice. In some sense we are incrementally add more power to the feature with each thin slice. From another perspective it feels like we are iterating over the feature and refining the feature as we go along. In either case, We build the feature by adding one by one the thin slice, till the point we feel we can release this feature. Post release we can continue to add more slices.

It is also possible to come up with thin slices which are perfectly releasable. Lets take the same example and see how we could come up with thin slices which are releasable.

• User can upload a photo (predefined set of image format + clear message showing the expected size and resolution). Any image which does not qualify this criteria is rejected.
• Knock off the resolution constraint and build image scaling facility so that we can reduce the image resolution and hence its size
• Knock off the image size constraint and provide an image cropping facility so that users can crop their images to fit the right size
• And so on…

Check out another example of Thin Slicing from the Protest Project.

Please note that the concept of thin slice is also applicable at a much broader scope than just at a feature level. For Ex: On a given project we use the same concept to plan our small & frequent releases. At a portfolio level we apply the same principle to break projects into smaller loosely coupled projects and then prioritize them.

I guess you agree with me that software development is a continuous evolutionary process. While developing a software product one should be in a flow mode rather than an episodic mode. As Sandeep puts it, its activity v/s events. This goes back to the philosophy of eXtreme programming.

If something is good, do it all the time OR as they say, push the nob to 10.

• We realized that integrating early and often is good, hence we started doing Continuous Integration.
• We realized that we cannot plan once and then just follow the plan, we need to continuous keep planning and prioritizing work.
• We realized that code reviews are helpful, hence we started Pair Programming.
• We realized that testing early and often is good, hence we started Test Driven Development.
• We realized that one cannot think of all scenarios and design software upfront. Since software keeps evolving and software degrades (bit-rot) over time, we need to refactor code all the time (mercilessly).
• And so on.

This is what I mean by flow mode (activity) rather than an episodic mode (event).

In Agile, we’ve tried to apply this principle in various places like

• Product Visioning
• Managing Product Backlogs (Requirements)
• Release Planning
• Iteration Planning
• Retrospective
• User feedback
• Daily Stand-up meetings
• And so on…

But what I feel is a lot of it is still very episodic. For example,

• If a developer hits a road block, they sit on it till the next day’s stand-up meeting. Because stand-up meeting is where we discuss roadblocks.
• If we find something is hurting us, we wait till the retrospective to discuss about it.
• If we discover some acceptance criteria cannot be met or is flawed during the iteration, we wait till the demo to communicate that with the customer/product owner.
• If we discover a new killer feature during an iteration, we wait for the planning meeting to discuss that and prioritize it on the backlog.
• We wait till the end of the release to do Performance testing, Hallway Usability testing and other important tests.
• And so on….

The Lean community is trying to address a lot of these issues. As long as we think of software development activities as episodes and not as continuous flow of activities it would be very difficult to really implement kaizen and evolve.

This is supposed to be an introductory presentation on Agile.

In this presentation I give some examples of heavy weight methods and their implications on your project. Then I give a quick overview of Agile methods, the rationale behind it, its origin, its values and principles. I move on to describe that what I see happening today in the industry is really waterfall in the name of Agile. I give some reasons why this is happening and then I give some pointers to move away from this flawed thinking.

Bottom line, Agile is not a Silver Bullet and don’t fall pray to marketing gimmicks. Question dogmatic claims. Adapt Agile to your needs and take baby steps.