Wednesday, April 21, 2010

I’m estimating in Apples and entering Oranges on my timesheet

applesandorangesI’ve just had a realisation that for the past month, I’ve been making a fundamental mistake with regards to managing the schedule on a software development project I’m working on.

Someone that’s tried all the various techniques that I have and listened to or read countless case studies on the matter, should never have made such a fundamental mistake.

We have undertaken a massive rewrite of our software recently and have been roaming fairly aimlessly along, building some really nice stuff, but with only a vague picture of what we want to do, and what will wait until later. I’ve felt bad about the lack of direction for awhile but have never had enough pressure on me (or put enough on myself) to make some commitments about where it’s going. I’ve always been able to show plenty of progress – lots of cool looking screens and better features than the old system – but progress to where…I’m driving my car along a very long highway and have no clue as to where the road goes.

Lately, in the nicest possible way, the owner of the business has decided he’d like to know when this project is likely to be ready to launch. I cringed a little, as I wanted to be all Agile, and not set some “pie in the sky” date way out 12 months of so from now, only to completely miss it and….you know how it goes. However it is clearly just as big a mistake to continue to wander aimlessly along with no goals and no solid commitment on my part. I had to face facts and make a few tricky calls.

So I applied my best Agile techniques and started talking about small iterations, monitoring progress, and even evidence based scheduling.

So for the past month, I’ve put together a work log (or backlog), done my estimates of the work to complete, recorded my time and compared results, recording how much time I get to dedicate to the project and how much time I get dragged off onto other stuff.

After a very interrupted month, the results came in. There weren’t too many unexpected figures in there except that my estimates were about 25% too high.

After some discussion with the boss, he agreed to do whatever he can to stop interruptions to my day task switching etc. He means it and has put his money where his mouth is in terms of contracting others to take over some of the work. no complaints here.

Then we started talking about what a fair percentage of my time is, to be dedicated to this project. My gut feeling was that if I could log 70% of my time to the project, that would be great. He couldn’t understand why, with all he was doing, that I would so matter-of-factly" say, achieving productivity of over 80% is a pipedream.

“I want you to be spending at least 90% of your time on this. I’m paying you to work here….” and so it goes.

So today, I've focussed and the first two hours were really productive “zone time”. I started at 8.30am, made coffee, replied to some emails, checked a couple of settings on our network as the new guys contracted to manage that (last week – one of the obstacles removed) were making some changes overnight, and I was into it.

I surface at 11.00am and thought, well there’s 2 hours. So I logged that time. Now that’s 120 / 150 minutes – 80%. Geeez! How is 90% ever going to be possible if I done this much and still only hit 80%. So I went to make a coffee, thereby dropping that figure a little more!

Then it occurred to me, when I estimate these tasks, incredibly roughly with a huge amount of guesswork and gut feel, I do so in a “that’s about a day, but that one is less than half a day, that one will take me a week” and so on.

This form of guesstimation intuitively takes into account all the normal daily interruptions of phone calls, email, coffee making. It’s not fair, or even accurate to estimate in terms of duration (“it will take me at least a week to do all that”) but then record your time in terms of dedicated zone time. I have to record time the same way as I estimate it.

It sound so bloody obvious when you say it out loud, but the fact was, even after the number of different approaches to this type of scheduling and estimating I’ve seen and used, I still made such a stupid mistake.

Anyway, it is now clear that as I have not worked on any other projects or tasks today, Ive hit 100%. How much did i get done? Well, if you look at the tasks I finished, almost exactly what I’d expect to get done in a day. Good estimating Stu! Now I am recording that time fairly my estimates at the end of the month should look a lot more correct too. Not only that, but the boss will be happy seeing my get close to the target he set of 90%…dream on…

Joel Spolsky had a rant along these lines on a recent StackOverflow podcast too – talking about absorbing the minor interruptions into the hours you record against the job – those interruptions are a normal part of the day and should be included in the evidence you take on board, for the next estimate you make.