Government departments are constantly criticised for their lavish spending on outside consultants. Why, the newspapers ask, couldn’t they rely on the inside talent that taxpayers are already forking out for. This paper, mentioned by Eliezer Yudkowsky may give some clue. It turns out that people are not only incredibly bad at predicting how long it will take them to complete a task, they also get worse at predicting it as they know more about it.
A person who knows the ins and outs of a project will estimate the completion time of the project by taking in to account all of the things that they know need to be done for that particular project. In doing so they tend to disregard all of the unforseen stuff-ups that could happen along the way. An outsider will tend to estimate completion time by looking at how long similar projects have atken in the past. It turns out that the latter approach is not only far more accurate, but also estimates completion times far, far longer than ever predicted by insiders.
Could it be that, in some cases, ministries are actually saving us money by getting an outside perspective on a project? Quite conceivably, the money spent hiring the consultant improves project planning to the point that less money will be wasted on false expectations of the project’s success. We should also disregard the protestations of the ministry insiders that the hiring of the consultant was a waste of money because the consultants know very little about the project. It is precisely because they are not involved in the micro-management of the project that they can provide a realistic estimate of the project’s expected success and completion time. Of course, advocating more government spending on consultants is hardly an election winning strategy.