Planning fallacy - An engineer's point of viewWritten on November 26th, 2019 by @10000TB
“Psychological biases can create cognitive delusion” - A Mckinsey study
When I was on the loo today at work, some printed material installed on the side stall brings up an interesting theory called “planning fallacy”. What it says, in a nutshell, is that people tend to underestimate how much time is needed for a future task. I find the material very interesting and useful, I then went on to read more on the theory, and here is what I learnt from reading and some of my past experiences that echo to the theory.
The planning fallacy was first proposed by two psychologists (check the wikipedia from references list for their background) back in 1979.
“It is a phenomenon in which predictions about how much time will be needed to completea future task display an optimism bias and underestimate the time needed”
A few key points about the theory
1. Past experience sometimes does not help future predictions
“This phenomenon sometimes occurs regardless of the individual’s knowledge that past tasks of a similar nature have taken longer to complete than generally planned”
I find this true. Back then when I was working at a start-up, I have often found myself and others close around tend to “allocate” smaller time frame for many tasks, especially those similar to tasks we have worked/solved in the past.
Take a step back. Speaking of start-up, the vibe is “move fast” (read: heavier workload, overwork, long hours). No one would recklessly act like an asshole, explicitly asking you to finish some task in obviously unreasonable time frame. But the fact that good people in a good environment, and usually taken care of with very good compensation and benefits tend “ask” themselves to voluntarily over work. In addition, a good amount of good people have the imposter syndrome.
As always, don’t forget that engineers always overestimate themselves - at least it is true for myself.
As a result, we “allocate” shorter time frame, subconsciously assuming firstly we are experienced enough to sovle the task (without actual evaluations & investigations - start-up is busy, not all good things are rigorously followed, use your imagination), secondly we will overwork to catch up the deadline, and lastly “I am good - I should be able to do that”.
NOTE that these above are simply some example mentalities I have experienced myself or learnt from others. I don’t doubt there are more out there from other people demonstrating the the same point.
2. The optimism bias only applies to one’s own predictions
“The bias only affects predictions about one’s own tasks; when outside observers predict task completion times, they show a pessimistic bias, overestimating the time needed”
I find that overly self-confidence and many times benign self-esteem are the reason, period. “Psychological biases can create cognitive delusion”
As a result of planning fallacy, especially in software industry, lots of projects end up overuns, not only in time, but also in cost.
I am not a psychologist, nor have I read or learnt anything purposefully on the matter. I speak genuinely from my past experience, and I find that explains and demonstrates the theory points. It probably needs to be debated whether those experiences can serve as proof, though.