The Value Theory: What people value drives their actions.

Value Driven planning is where you plan to do the most valuable things with your resources, and of those valuable things you try to do the most valuable things first if possible.

Target Driven planning is where you have a target or set of targets you try to reach, and you try to do what gets you to your targets.

Target Driven thinking reflects the way we tend to ask people for what we want. For example, “May I have 2 kilos of potatoes please”, “Get me twelve bricks.”, and “I need two tonnes of sand.” What we rarely do is say “This is how my values work or this is what I value, so please do something that will make me happy.”

Similarly when it comes to Agile adoption (or any process for that matter), we see two camps:

• Target Driven Camp: Where people use number of practices, adherence to a specific prescriptive process and checklist based verification of the same, certification, maturity levels, and so on to guide and plan their adoption.
• Value Driven Camp: Organizations highlight what they value, they use their values to guide them with their adoption process. (Please note that their values itself can evolve.) Ex: We value quick turn-around time, so our customers get what they ask for quickly, is this process change inline with our value?

This blog was triggered after reading: Goals Gone Wild: The Systematic Side Effects of Over-Prescribing Goal Setting (a must read).

Thanks to Matthew Leitch for his wonderful summary of Value Driven versus Target Driven planning.

Recently I read Esther Derby’s blog post on Five Reasons to Hire a Coach for Agile Teams. While I agree with all her points, following are the questions or concerns running through my mind.

• What are the risks involved when hiring a coach?
  • What is success ratio? How many teams you know (not heard of in some cooked up report by a consulting company or a tool vendor) who are successful adopting Agile with a coach’s help? And how many teams do you know who have failed trying to adopt agile without using a coach? Let me clarify, when I say adopt Agile, its not about a bunch of practices. Its about continuously evolving the process to make it lean and more efficient and more enjoyable.
  • Can any coach do or do we need a special type of coach? Point I’m getting at is, your chances to succeed is based on the quality of the coach and her experience.
  • In my opinion your chances of failure is much more if you hire an average coach from outside who does not understand the business, organization and team context & culture.
  • The big problem we face today is we don’t have a good way to know if someone is a good coach or not. What are the chances you’ll end up hiring the right coach? Its not sufficient if someone is a certified professional. In some cases, its not even sufficient if someone has written a couple of books.
  • Also in my experience, ability to connect and influence the team members is very important. A good team member is in a much better position to achieve this than any average coach from outside.
• How sustainable is the model of hiring a coach?
  • What happens when the coach leaves? Is the team evolving their process or still using what the coach put together?
  • Hiring a coach from outside might speed up things. But I’m not sure if people will be able to understand the rationale behind doing certain things. In my experience failure is a great learning tool. Taking more time to achieve something (and in the process failing a few times) is not bad, it make something more sustainable and scalable.
• Can you teach/coach someone to be agile? Agile has been around at least for the last 10 years, what are the chances one or more coaches can change that? (I’m not saying its not possible, I’m trying to highlight that its not simple. It involves organizational transformation and change in individual’s mindset).

My advice is before looking for a training or coaching, do sufficient homework?

No amount of process can fill-in for lack of trust. IMHO one needs to focus on building trust and creating an environment with low viscosity, process will automatically fall in place.

In the classical article named “Design Principles and Design Patterns“, Bob Martin (Uncle Bob) defines Viscosity and talks about 2 types of Viscosity from a software design point of view:

Viscosity comes in two forms: viscosity of the design, and viscosity of the environment. When faced with a change, engineers usually find more than one way to make the change. Some of the ways preserve the design, others do not (i.e. they are hacks.) When the design preserving methods are harder to employ than the hacks, then the viscosity of the design is high. It is easy to do the wrong thing, but hard to do the right thing.

Viscosity of environment comes about when the development environment is slow and inefficient. For example, if compile times are very long, engineers will be tempted to make changes that don’t force large recompiles, even though those changes are not optiimal from a design point of view. If the source code control system requires hours to check in just a few files, then engineers will be tempted to make changes that require as few check-ins as possible, regardless of whether the design is preserved.

This is a great explanation of how Viscosity applies in development process. I think the same thought process can be applied to the way an organization operates.

Some organizations have really high viscosity and trying to do the right thing, which is inline with the organizational value system, becomes harder and harder (sometimes impossible). Doing the wrong thing and getting away with it, seems so much easier. Its not just easy its also always quick. It turns out that there are no real motivations for employees to do the right thing. And then the broken window syndrome sets in.

Such organizations soon become endangered species.

On the Agile Alliance LinkedIn group, Eugene Yamnitsky posted the following question:

One of the key Agile transformation success factors is Executive Commitment. Every book and article I’ve read mention this, but there are no good explanation on what it really means.

Great questions!

In my experience, Executive Commitment is very deep and crucial. It’s certainly not as easy as flipping a switch and expecting magic to happen.

What  does this really means?

• First and foremost they need to understand that, for any change program or transformation, its the people and the environment they operate in which makes the difference. Not the new process itself. What Agile tries to do is change people’s behavior and the environment they work in.
• They need to take time to actually understand what is Agile and what it is not. Essentially so that they have the right expectations set. So that they understand the impact of what they are getting into. This should involve talking to practitioners about their experience. Reading at least few experience reports and so on.
• Involving their teams to understand their issues and aspirations. The worst thing the executives can do is decide “Agile is THE way to go” in a closed room meeting and force it down people’s throat.
• Once there is a general awareness about Agile in the organization, ask any one team to volunteer to be the pilot project. Work closely with that team to set realistic plans.
• Once the pilot project is kicked off and folks on that team & supporting teams are sold on the concept, they need to step back & stand behind their teams (support them) and trust them to do their best.
  Send a clear message across the organization that this is important and we really care about this stuff. (There is a big difference between pushing in down people’s throat and doing it collaboratively with their buy-in)
• The Executives need to be open to failures and also need to communicate that clearly to teams saying its OK to fail. We are trying something new, which is a huge paradigm shift and there is certainly risk involved in it. So its OK to fail.
• Also sometimes I find that since the expectation setting is not right, teams are always trying to change their way of working in a hurry, under a high pressure. Its very unlikely that I would do my best under such situations. Give your teams the space and time they need. Like anything new, there will always be a learning curve.
• It usually very helpful to have someone in the organization play an evangelist role. Someone with real first hand experience with Agile. The executives need to invest in the right things.
• The other very important thing that the executives can influence is the organization culture. Encouraging honest and open communication. Not by just doing lip service, but by actually leading by example.
• Like with any change program, everyone (esp. the executives) are always interested to know as soon as possible if the new process is working on not. Is it helping on not? Sometimes people try to put in tracking/measuring mechanisms which are inherently broken. As we all know, you’ll get what you measure. So the metrics you use will influence to a large extend how people will behave. This one single things can screw up the whole adoption process.

I can go on, but I guess this gives you the gist and seriousness of the kind of commitment required.

To conclude, I think it all revolves around

• communication,
• expectation setting,
• collaboration,
• leading by example,
• showing that they care and providing the required support (time, money, effort)
• being open to take the risk of potential failures.

Recently I was going through a presentation on PeopleWare. Tom DeMarco and Timothy Lister published this fantastic book in 1987 and is till date very valid. If you’ve not read this book I would strongly recommend it.

The book talks about Methodology with a capital M and it’s impact. According to the author, effects of Methodology:

• a morass of paperwork
• a paucity of methods
• an absence of responsibility
• a general loss of motivation

This is so valid even in today’s world where lots of companies claim they are doing Agile (with a Capital A).

There are some other very interesting quotes in the book which is worth highlighting:

the major problems of our work are not so much technological as sociological in nature. [p. 4]

because we go about this work in teams and projects and other tightly knit working groups, we are mostly in the human communication business. our successes stem from good human interactions… and our failures stem from poor human interactions [p. 5]

people are not the same! therefore they are not interchangeable

the manager’s function is not to make people work but to make it possible for people to work [p. 34]

people under pressure don’t work better; they just work faster [p. 18]

quality is free, but only to those who are willing to pay heavily for it [p. 23]

organizational busy work tends to expand to ?ll the working day [p. 29]

most organisations don’t set out consciously to kill teams. they just act that way [p. 4]

I think that if you have an organization of empowered self-organized smart employees then, you should consider having:

• Open Salaries: Any employee can know any other employee’s salary. Disclosing salaries is not a crime
• Salary increments/upraisals are not based on individual performance. We want to build an organization where everyone is equal and everyone is putting in their best. If not, then they are in the wrong place. It would be even better if everyone is contributing (net value) more or less the same. (Common to find this in 4-5 people start-ups)
• Team members sit down and mutually decide their salaries (the numbers can be further adjusted based on cross-team discussions)
• Every month the company discloses its Profit and Loss (P&L) statement for the month. Even better if you can automate this and we can see the trend.
• Based on the P&L, employees can get a % increase or decrease in their monthly pay. (can be easily automated)
  • Say the company made 20% profit (2,000K USD) this month.
  • Last month the company had made 15% profit (1,500K USD).
  • So the delta is 5% (500K USD).
  • Now the company decides that some portion of this 5% should either be distributed amongst the employees or used to buy an asset for the company that will directly benefit all the employees (say Mac Book Pro for all employees).
  • Say the company decides to distribute 20% (100k USD) of the 5% profit delta (500K USD) amongst all it’s employees.
  • We distribute this amount (100K USD) to all the employees.
    • One way to do this is such that every one gets the same portion/increment. You can divide the total distributable profit amount (100K USD) by the total number of employees (say 100) which would be 1K USD amongst all its employees. So every one get 100 USD.
    • Another way to do this is instead of everyone getting the same increment, employees get the increment based on some % of their salary. So for example 100k USD needs to be distributed amongst 100 employees whose total salary comes up to 50K USD. Then everyone gets 2% (100K/50K) of their current salary increment. So if someone’s salary was 5k USD per month, then their new salary will be 5.1k USD.
• Some organization might feel doing it every month is too much overhead. May be they want to batch it up and do it once every 6 months. This is fine, except that doing it every month is a great way for all the employees to be hooked into the companies performance.
• Salary corrections because of market changes or other reasons are done on a regular basis but are separated from the salary upraisals.

The standard response I get to this proposal in companies from the management is that

“Employees are not matured enough to handle this in the right spirit”.

My response is

“You can’t learn how to swim by standing outside water and watching others swim, you have to jump in and try”.

What is the purpose of Continuous Integration (CI)?

To avoid last minute integration surprises. CI tries to break the integration process into small, frequent steps to avoid big bang integration as it leads to integration nightmare.

If people are afraid to check-in frequently, your Continuous Integration process is not working.

CI process goes hand in hand with Collective Code Ownership and Single-Team attitude.

CI is the manifestation of “Stop the Line” culture from Lean Manufacturing.

What are the advantages of Continuous Integration?

• Helps to improve the quality of the software and reduce the risk by giving quicker feedback.
  • Experience shows that a huge number of bugs are introduced during the last-minute code integration under panic conditions.
• Brings the team together. Helps to build collaborative teams.
• Gives a level of confidence to checkin code more frequently that was once not there.
• Helps to maintain the latest version of the code base in always shippable state. (for testing, demo, or release purposes)
• Encourages lose coupling and evolutionary design.
• Increase visibility and acts as an information radiator for the team.
• By integrating frequently, it helps us avoid huge integration effort in the end.
• Helps you visualize various trends about your source code. Can be a great starting point to improve your development process.

Is Continuous Integration the same as Continuous build?

No, continuous build only checks if the code compiles and links correctly. Continuous Integration goes beyond just compiling.

• It executes a battery of unit and functional tests to verify that the latest version of the source code is still functional.
• It runs a collection of source code analysis tools to give you feedback about the Quality of the source code.
• It executes you packing script to make sure, the application can be packaged and installed.

Of course, both CI and CB should:

• track changes,
• archive and visualize build results and
• intelligently publish/notify the results to the team.

How do you differentiate between Frequent Versus Continuous Integration?

Continuous means:

• As soon as there is something new to build, its built automatically. You want to fail-fast and get this feedback as rapidly as possible.
• When it stops becoming an event (ceremony) and becomes a behavior (habit).

Merge a little at a time to avoid the big cost at full integration at the end of a project. The bottom line is fail-fast & quicker feedback.

Can Continuous Integration be manual?

Manual Continuous Integration is the practice of frequently integrating with other team members’ code manually on developer’s machine or an independent machine.

Because people are not good at being consistent and cannot do repetitive tasks (its a machine’s job), IMHO, this process should be automated so that you are compiling, testing, inspecting and responding to feedback.

What are the Pre-Requisites for Continuous Integration?

This is a grey area. Here a quick list is:

• Common source code repository
• Source Control Management tool
• Automated Build scripts
• Automated tests
• Feedback mechanism
• Commit code frequently
• Change of developer mentality, .i.e. desire to get rapid feedback and increase visibility.

What are the various steps in the Continuous Integration build?

• pulling the source from the SCM
• generating source (if you are using code generation)
• compiling source
• executing unit tests
• run static code analysis tools – project size, coding convention violation checker, dependency analysis, cyclomatic complexity, etc.
• generate version control usage trends
• generate documentation
• setup the environment (pre build)
• set up third party dependency. Example: run database migration scripts
• packaging
• deployment
• run various regression tests: smoke, integration, functional and performance test
• run dynamic code analysis tools – code coverage, dead-code analyzer,
• create and test installer
• restore the environment (post build)
• publishing build artifact
• report/publish status of the build
• update historical record of the build
• build metrics – timings
• gather auditing information (i.e. why, who)
• labeling the repository
• trigger dependent builds

Who are the stakeholders of the Continuous Integration build?

• Developers
• Testers [QA]
• Analysts/Subject Matter Experts
• Managers
• System Operations
• Architects
• DBAs
• UX Team
• Agile/CI Coach

What is the scope of QA?

They help the team with automating the functional tests. They pick up the product from the nightly build and do other types of testing.
For Ex: Exploratory testing, Mutation testing, Some System tests which are hard to automate.

What are the different types of builds that make Continuous Integration and what are they based on?

We break down the CI build into different builds depending on their scope & time of feedback cycle and the target audience.

1. Local Developer build :
1.a. Job: Retains the environment. Only compiles and tests locally changed code (incremental).
1.b. Feedback: less than 5 mins.
1.c. Stakeholders: Developer pair who runs the build
1.d. Frequency: Before checking in code
1.e. Where: On developer workstation/laptop

2. Smoke build :
2.a. Job: Compiles , Unit test , Automated acceptance and Smoke tests on a clean environment[including database].
2.b. Feedback: less than 10 to 15 mins. (If it takes longer, then you could make the build incremental, not start on a clean environment)
2.c. Stakeholders: All the developers within a single team.
2.d. Frequency: With every checkin
2.e. Where: On a team’s dedicated continuous integration server. [Multiple modules can share the server, if they have parallel builds]

3. Functional build :
3.a. Job: Compiles , Unit test , Automated acceptance and All Functional\Regression tests on a clean environment. Stubs/Mocks out other modules or systems.
3.b. Feedback: less than 1 hour.
3.c. Stakeholders: Developers , QA , Analysts in a given team
3.d. Frequency: Every 2 to 3 hours
3.e. Where: On a team’s dedicated continuous integration server.

4. Cross module build :
4.a. Job: If your project has multiple teams, each working on a separate module, this build integrates those modules and runs the functional build across all those modules.
4.b. Feedback: in less than 4 hr.
4.c. Stakeholders: Developers , QA , Architects , Manager , Analyst across the module team
4.d. Frequency: 2 to 3 times a day
4.e. Where: On a continuous integration server owned by all the modules. [Different from above]

5. Product build :
5.a. Job: Integrates all the code that is required to create a single product. Nothing is mocked or stubbed. [Except things that are not yet built]. Creates all the artifacts and publishes a deployable product.
5.b. Feedback: less than 10 hrs.
5.c. Stakeholders: Every one including the Project Management.
5.d. Frequency: Every night.
5.e. Where: On a continuous integration server owned by all the modules. [Same as above]

General Rule of Thumb: No silver bullet. Adapt your own process/practice.