WHAT EVERY POST ON SOFTWARE ESTIMATION GETS WRONG
3 MIN READ
Every post on software estimation fails before it starts by asking the wrong question. These posts ask:
What technique can I use to increase the accuracy of my software estimates?
Technique? No. Asking this question highlights the unhelpful underlying belief:
There must be some kind of secret trick I can use instead of thinking.
No shame in this underlying belief as a dev - we must always be on the lookout for edges to be gained in our process.
Better question to ask:
How can I better understand the process of building software and convey this to stakeholders?
There is no “technique” or “one simple trick” to improve software estimation accuracy. It is a natural byproduct of awareness and self confidence, you would do better to work on those than relying on calculation and trickery.
Awareness
Most devs I meet, when estimating tickets, are delusionally self-confident and optimistic. Here are some of their assumptions:
- Infrastructure will work seamlessly
- If it doesn’t, a dedicated team (i.e. not I) will fix it quickly
- No services will fail
- My code will compile correctly the first time
- My code will pass CI the first time
- I won’t get pulled into fixing other unrelated bugs while working on this feature
- My personal life will be completely placid while I work on this feature and not take away any of my focus
- I can write code productively and to a high quality standard for more than 2-4 hours a day for multiple consecutive days (devs with less than a couple years of experience are guilty of this one)
Any dev who isn’t completely green knows that these assumptions are bullshit, we plan and God laughs.
Software estimation depends on awareness of all of these factors and many more — to not willfully blind oneself to these probable deterrents.
You are still searching for a technique? Consider a real situation I often go through. My baby hasn’t pooped in a few days and is grumpy, she’s fussy at night and wakes me up more than usual. As a result I’m more tired than usual at work. How significantly will this impact my work on The Feature? Furthermore, when will my baby finally poop and be less fussy? Only awareness of how these situations play out can answer these questions when they come up in the future. No technique can account for this.
Be aware and don’t delude yourself with unwarranted optimism.
Self Confidence
Assume you’ve gotten pretty good at the “awareness” bit. You listen to your higher, wiser self say “I think this ticket will take 4 days to complete.” What comes out of your mouth when talking to the product manager is “This will take me one day to complete.” They look impressed. You have made them happy. Your mind is awash in premature dopamine.
Despite your best efforts, the ticket takes 4 days to complete. The product manager isn’t thrilled but it’s not blocking a release so it’s not the end of the world. You give another sped-up estimate that contradicts your inner voice again next week.
Why did you not just say “this ticket will take 4 days to complete”? You don’t want to make people feel bad. This is a good and natural impulse within reason, but it’s not one that’s conducive to your job. Part of your job as a dev is to, with your accumulated wisdom and experience, give as accurate an estimate as you are capable of giving. N.B. for more junior-ish devs you will not be capable of giving a very accurate estimate and this should be compensated for by your manager.
When you give the estimate that the person wants to hear, instead of what is realistic, you are lying. You lie because your self-confidence is low. I cannot, in an essay, triage the source(s) of your low self-confidence - perhaps you feel like you will lose your job if you disappoint people (paradoxically it’s more disappointing to have a dev that consistently underestimates timelines).
What I can say definitively is that if you are able to accurately estimate tickets privately but give “what you want to hear” estimates publicly, finding the root of your self-confidence and improving it is the missing link.
End
I doubt this will be very well received because it runs contrary to “conventional wisdom”. All I can offer in defense is that these are the qualities I’ve seen in good estimators throughout my career and what I’ve improved in myself to become a better estimator.