About   Slides   Home  

Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
RSS Feed
Recent Thoughts
Recent Comments

Archive for the ‘Learning’ Category

What is required for Deliberate or Deep Practice?

Wednesday, December 18th, 2013

There is an old saying that “Practice makes perfect“. But not all practice can make you perfect. There is a specific type of practice usually referred to as deliberate practice or deep practice which can really help you master the skill at hand.

Geoff Colvin, the author of the groundbreaking bestseller Talent Is Overrated explains that deliberate practice can be described by these five characteristics:

  1. It’s designed specifically to improve performance
  2. It can be repeated a lot
  3. Feedback on actions is continuously available
  4. It’s highly demanding mentally
  5. It isn’t much fun

Hence a short, focused and regular practice session is lot more effective than one long, generic, random practice session.

Its also important to note that deliberate/deep practice requires a certain context or mindset to really facilitate the learning. Following are important characteristics:

  • You are slightly off-balance, trying to get back
  • You are constantly getting tons of clear, instant feedback
  • You are at the edge of your ability, but motivated to stretch yourself a little bit more
  • You are staring at you role models, .i.e. you’ve a clear idea of your goals or the kind of person you want to become.

The advantage of deliberate practice is cumulative, hence starting early has a big advantage.

Generally, I used to be against rote learning, but now I’m rethinking through my conclusions.

If this topic interests you, I would strongly recommend the following books:

Some good videos to start on this topic:

Problem Solving Techniques For Evolutionary Design

Thursday, October 10th, 2013

At the Agile Goa conference, I ran a small workshop to help participants understand the core techniques one should master to effectively practice evolutionary design while solving real-world problems.

Key take aways:

  1. Eliminate Noise – Distill down the crux of the problem
  2. Divide and Conquer – Focus on one scenario at a time and incrementally build your solution
  3. Add constraints to future simplify the problem
  4. Simple Design – Find the simplest possible solution to solve the current scenario at hand
  5. Refactor: Pause, look for a much simpler alternative
  6. Be ready to throw away your solution and start again

When the Future is Uncertain, How Important is A Long-Term Plan?

Tuesday, March 27th, 2012

Many friends responded to my previous post on How Much Should You Think about the Long-Term? saying:

Even if the future is uncertain and we know it will change, we should always plan for the long-term. Without a plan, we cease to move forward.

I’m not necessarily in favor or against this philosophy. However I’m concerned about the following points:

  • Effort: The effort required to put together an initial direction is very different from the effort required to put a (proper) plan together. Is that extra effort really worth it esp. when we know things will change?
  • Attachment: Sometimes we get attached with our plans. Even when we see things are not quite inline with our plan, we think, its because we’ve not given enough time or we’ve not done justice to it.
  • Conditioned: Sometimes I notice that when we have a plan in place, knowingly or unknowingly we stop watching out for certain things. Mentally we are at ease and we build a shield around us. We get in the groove of our plan and miss some wonderful opportunities along the way.

The amount of planning required seems to be directly proportional to the size of your team.

If your team consists of a couple of people, you can go fairly lightweight. And that’s my long-term plan to deal with uncertainty.

How Much Should You Think about the Long-Term?

Tuesday, March 27th, 2012

Often people tell you that “You should think about the long-term.”

Sometimes people tell you, “Forget long-term, its too vague, but you should at least think beyond the near-term.”


Unfortunately, part of my brain (prefrontal cortex), which can see and analyze the future, has failed to develop compared to the other smart beings.

At times, I try to fool myself saying I can anticipate the future, but usually when I get there (future) its quite different. I realize that the way I think about the future is fundamentally flawed. I take the present and fill it with random guesses about something that might happen. But I always miss things that I’m not aware of or not exposed to.

In today’s world, when there are a lot of new ideas/stuff going around us, I’m amazed how others can project themselves into the future and plan their long-terms?

Imagine a tech-company planning their long-term plan, 5-years ago, when there were no iPads/tablets. They all must have guessed a tablet revolution and accounted that in their long-term plans. Or even if they did not, it would have been easy for them to embrace it right?

You could argue that the tablet revolution is a one-off phenomenon or an outlier. Generally things around here are very predictable and we can plan our long-term without an issue. Global economics, stability of government, rules and regulations, emergence of new technologies, new markets, movement of people, changes in their aspirations, environmental issues, none of these impact us in any way.

Good for you! Unfortunately I don’t live in a world like that (or at least don’t fool myself thinking about the world that way.)

By now, you must be aware that we live in a complex adaptive world and we humans ourselves are complex adaptive system. In complex adaptive system, the causality is retrospectively coherent. .i.e. hindsight does not lead to foresight.

10 years ago, when I first read about YAGNI and DTSTTCPW, I thought that was profound. It was against the common wisdom of how people designed software. Software development has come a long way since then. But is XP the best way to build software? Don’t know. Question is, people who used these principles did they build great systems? Answer is: Yes, quite a few of them.

In essence, I think one should certainly think about the future, make reasonable (quick) guesses and move on. Key is to always keep an open mind and listen to subtle changes around us.

One cannot solely rely on their “intuition” about long term. Arguing on things you’ve all only guessed seems like a huge waste of time and effort. Remember there is a point of diminishing returns and more time you spend thinking long-term, less your chances are of getting it right.

I’m a strong believer of “Action Precedes Clarity!”

Update: In response to many comments: When the Future is Uncertain, How Important is A Long-Term Plan?

Unit Testing Dilemma: Should I Invest or Not?

Tuesday, November 1st, 2011

Every single line of code must be unit tested!

This sound advice rather seems quite extreme to me. IMHO a skilled programmer pragmatically decides when to invest in unit testing.

After practicing (automated) unit testing for over a decade, I’m a strong believer and proponent of automated unit testing. My take on why developers should care about Unit Testing and TDD.

However over the years I’ve realized that automated unit tests do have four, very important, costs associated with them:

  • Cost of writing the unit tests in the first place
  • Cost of running the unit tests regularly to get feedback
  • Cost of maintaining and updating the unit tests as and when required
  • Cost of understanding other’s unit tests
One also starts to recognize some other subtle costs associated with unit testing:
  • Illusion of safety: While unit tests gives you a great safety net, at times, it can also create an illusion of safety leading to developers too heavily relying on just unit tests (possibly doing more harm than good.)
  • Opportunity cost: If I did not invest in this test, what else could I have done in that time? Flip side of this argument is the opportunity cost of repetitive manually testing or even worse not testing at all.
  • Getting in the way: While unit tests help you drive your design, at times, they do get in the way of refactoring. Many times, I’ve refrained from refactoring the code because I get intimidated by the sheer effort of refactor/rewrite a large number of my tests as well. (I’ve learned many patterns to reduce this pain over the years, but the pain still exists.)
  • Obscures a simpler design: Many times, I find myself so engrossed in my tests and the design it leads to, that I become ignorant to a better, more simpler design. Also sometimes half-way through, even if I realize that there might be an alternative design, because I’ve already invested in a solution (plus all its tests), its harder to throw away the code. In retrospect this always seems like a bad choice.
If we consider all these factors, would you agree with me that:
Automated unit testing is extremely important, but each developer has to make a conscious, pragmatic decision when to invest in unit testing.
Its easy to say always write unit tests, but it takes years of first-hand experience to judge where to draw the line.

Like with Exercising, with Learning also, we Procrastinate. But why?

Sunday, May 15th, 2011

I always knew and believed that I need to exercise and stay in shape to be effective at work. But for few years other burning issues always forced me away from exercising. I kept procrastinating!

Gradually, inch-by-inch, year after year, I found myself becoming a potato. It got to a point where it felt like there was no going back.

Unfortunately many professionals (including myself) do the exact same thing when it comes to Learning and staying up-to-date (keeping our skills in shape.)

We all know the importance of learning, yet we always find reasons not to invest right now.

But why?

If you Google for “avoid procrastination”, you’ll find tons of techniques to help you avoid procrastination. But I don’t know a single person who does not suffer from this.

Luckily for me, from the last 3 years, I’ve really been exercising every day (almost everyday.) On a given day, if I don’t, I feel really crappy.

So far I’ve not seen a similar behavior when it comes to Learning.

So what made me change my attitude towards exercising but not learning?

  • Most important thing, with exercising, both the outcome and output is easy to measure/judge.
    • There are some very easy measurements that can quickly show you the difference between being in-shape or not.
    • Did I workout for 30 mins everyday and is my weight stable? (Output)
    • Do I feel energized and fresh the whole day? (Outcome) (I can easily judge this.)
    • But with learning, how do I measure the outcome? Its a lot more vague and much more long term.
  • Few things changed, I was in a company of people (at work, back home, amongst friends), who were all very conscious about staying in shape. Being surrounded by right people, who felt much better after staying in shape, reinforced my thinking and encouraged me to take the first step. Unfortunately its hard to find similar environment when it comes to learning.
  • I was able to set some simple targets (run for 30 mins every day) and measure if I met the targets or not. If I did not meet those targets, the results were quite obviously immediately. Unfortunately when it comes to Learning I’m not able to find a simple model like this, esp. results being immediately obvious.
  • Personally for me, exercising, which involves mostly physical work, is relatively easy even when I’m tired or feeling lazy. But its hard to learn under such conditions because learning requires high cognitive processing.
  • There are many options for exercising, but I was able to quickly decide on running everyday as its the simplest thing I could do anywhere and still get a good workout. But when in comes to learning, we have so many options and its intimidating to even get started.
  • Exercising usually is fun. Its enjoyable. There is an immediate positive feedback cycle. With learning that’s not always the case. Many times its frustrating.

When it comes to learning, I’ve tried to set various targets like:

  • Everyday for 30 mins, either read a book or blogs or twitter or practice solving some design problems.
    • Problem is, even after doing this, I’m not sure if I’m really benefiting or not. May be in the long run, but not now.
  • Write a blog post every other day. Hoping that, to write a meaningful blog post, I would go read something at least to clarify my thinking.
    • Problem is, I continue to write crap without reading much and I don’t write that regularly either.

What have you tried to keep up with your learning?

P.S: I learn a lot at my work, but its primarily driven by the need at hand. Which is good. But the broader learning and staying up-to-date with things is equally important and that’s what I’m talking about.

Preemptively Branching a Release Candidate and Splitting Teams Considered Harmful

Monday, April 18th, 2011

Building on top of my previous blog entry: Version Control Branching (extensively) Considered Harmful

I always discourage teams from preemptively branching a release candidate and then splitting their team to harden the release while rest of the team continues working on next release features.

My reasoning:

  • Increases the work-in-progress and creates a lot of planning, management, version-control, testing, etc. overheads.
  • In the grand scheme of things, we are focusing on resource utilization, but the throughput of the overall system is actually reducing.
  • During development, teams get very focused on churning out features. Subconsciously they know there will be a hardening/optimization phase at the end, so they tend to cut corners for short-term speed gains. This attitude had a snowball effect. Overall encourages a “not-my-problem” attitude towards quality, performance and overall usability.
  • The team (developers, testers and managers) responsible for hardening the release have to work extremely hard, under high pressure causing them to burn-out (and possibly introducing more problems into the system.) They have to suffer for the mistakes others have done. Does not seem like a fair system.
  • Because the team is under high pressure to deliver the release, even though they know something really needs to be redesigned/refactored, they just patch it up. Constantly doing this, really creates a big ball of complex mud that only a few people understand.
  • Creates a “Knowledge/Skill divide” between the developers and testers of the team. Generally the best (most trusted and knowledgable) members are pick up to work on the release hardening and performance optimization. They learn many interesting things while doing this. This newly acquired knowledge does not effectively get communicate back to other team members (mostly developers). Others continue doing what they used to do (potentially wrong things which the hardening team has to fix later.)
  • As releases pass by, there are fewer and fewer people who understand the overall system and only they are able to effectively harden the project. This is a huge project risk.
  • Over a period of time, every new release needs more hardening time due to the points highlighted above. This approach does not seem like a good strategy of getting out of the problem.

If something hurts, do it all the time to reduce the pain and get better at it.

Hence we should build release hardening as much as possible into the routine everyday work. If you still need hardening at the end, then instead of splitting the teams, we should let the whole swamp on making the release.

Also usually I notice that if only a subset of the team can effectively do the hardening, then its a good indication that the team is over-staffed and that might be one of the reasons for many problems in the first place. It might be worth considering down-sizing your team to see if some of those problems can be addressed.

A Catalog of Papers about Large, Complex, Distributed Systems

Saturday, February 19th, 2011

My friend Srihari has been working on a great project called Systems We Make. His goal is to provide a consolidated catalog of various papers/articles about building large, complex, distributed system.

He personally searches, reads through and filters each the papers for you. This is awesome!

Please support his initiative.

The Future of ELearning is Social Learning by Jane Hart

Monday, August 30th, 2010

Most eLearning simply automate the formal class-room training environment. Only difference is students can be distributed now.

Although a lot of money is spent on creating engaging, interactive, multimedia courseware, one vital ingredient is missing: Other People.

Many companies adopt a blended approach to bring in the other people aspect. But even face?to?face events are often delivered in a fairly traditional way.

Meanwhile outside of the formal learning environment in online colleges (online courses, classroom, workshops, etc) individuals are learning in other ways – often without realizing it ? whether it be by reading, listening or watching or simply by talking to one another.

Only recently informal learning been recognized as important part of learning and that it needs to be supported within an organization.

This presentation explains how:

8 Reasons to Focus on Informal and Social Learning by Charles Jennings

Monday, August 30th, 2010
  1. There are imperatives for continual learning
  2. Learning is a process, not a series of events
  3. Most learning occurs outside classrooms
  4. The vast majority of learning is social
  5. A lot of formal learning is ineffective
  6. People learn better when they are in charge
  7. There’s inherent inertia in formal approaches
  8. Informal and social learning are cost-effective
    Licensed under
Creative Commons License