a few good men

Interesting post over on 43Folders on planning and task estimation. I've certainly experienced theses problems in the past but hadn't heard them described as the planning fallacy or optimism bias. The suggested ~44% buffer to add is quite depressing but probably close to true based on my experience. That means in a typical 5 day work week, you'll be lucky to do useful work for even 3 of them.

The flip side of planning is probably something we've all experienced too. You do your best to come up with a realistic schedule, then you are told that that is unacceptable and to reduce it to a wildly unreasonable, unrealistic estimate. Everyone knows it will be unachievable, but that's what you work towards. No wonder so many projects slip - they finish right around when they would have finished with realistic planning. Now that isn't to excuse projects on a terminal death march or those that have gone wildly out of control. But so often, I've been on projects where the end date was set before the planning was done.

In fact, I think in those cases, a realistic schedule might mean the project would never start, so everyone puts on their rose-coloured glasses and agrees to an insane schedule, that meets the deadline. Then the project starts to slip, often before anyone can even do any work it will be late. Over my career I've collected metrics on the various projects I've worked on, formally and informally. I've found quite a bit of resistance to using this real world project data for subsequent planning sessions. Mostly that's been the case when a certain schedule was required, with set number of people and the nasty reality of past experience was meaning the numbers wouldn't add up correctly. Those projects finished when we originally thought they would too.

So how do we move beyond this? Is it really that we are so bad at estimating how long things will take? Or that we can't accept the numbers that we come up with because of the time to market pressures and realities of the market place? Do the planners want to know the truth right at the start, or deal with the successive revelation of the nasty reality over time? Execution by one thousand slips or just planning business as usual?

Comments

[David Robinson]

"I've been on projects where the end date was set before the planning was done."

There's nothing really wrong with this "right-to-left" scheduling. The purpose of planning here is to work out the subsequent scope, cost & quality tradeoffs needed to finish on this date (see the Project Management Triangle - http://en.wikipedia.org/wiki/Project_management ). People seem to forget this though, and think that because the end date is fixed, there's no point in planning at all. They then all rush off to join their death match.

"Is it really that we are so bad at estimating how long things will take?"
Sometimes. Steve McConnell's book on the topic goes into this in great detail. The problem seems to break down like this:

1. We really aren't that good at estimating
2. We try and present point numbers rather than an ever decreasing range of numbers
3. We negotiate the estimates, rather than the scope, cost or quality
4. We do this with senior managers, who tend to get their jobs because they know how to negotiate better than most
5. We don't explain to them where our estimates come from, so can't really justify them. Instead, we just let ourselves get brow-beaten into trying the impossible (and please no one mention stretch goals until you really know about their limitations)

Of course, this is all compounded by the fact that some project managers are simply insane and don't deserve the salary they're drawing. :-)

"Everyone knows it will be unachievable, but that's what you work towards"

It depends on how unachievable. If stretch goals are used properly (and other factors are in place, such as being in a good team, having a nice workplace, etc), then teams try to rise to the challenge (and sometimes even make it). All to often though, the goals really are unachievable, and that's just a demotivation for people. In that case, my experience has been that only the project manager works towards this (and once, not even him(!)). Everyone else just seems to works with the project's natural schedule (rather than some arbitrary Gant chart).

One final point, but if an engineer accepts a negotiated estimate and works towards it, are they not as much to blame for project failure than the project manager? I read the following quote recently, but forgot to write down the source:

"It’s a project manager’s job to make decisions, and an engineers job to make sure they are informed ones”

[Gordon]

Hi Davie,

thanks for commenting. I'd agree that scheduling with a pre-defined end date can be perfectly valid, but only if you are willing to change things like number of people or features or quality. When those aren't negotiable either in the up front planning, it is going to be painful.

I also agree about providing useful information to the project manager. I haven't always experienced project management that have been willing to listen, or perhaps in a position to listen, due to the pressure they themselves are getting from their management.