Lies, damn lies, and statistics
Last week David Grimmond wrote (here and here):
However, despite the great informational power of statistics, bear in mind that sample based statistics are still always measured with error.
How often do we hear news items that note something like: ‘according to the latest political poll, the support for the Haveigotadealforyou Party has increased from 9% to 9.5%” etc, but then just before closing the item they state that the survey has a 2% margin of error.
If you are awake to this point you suddenly realise that you have just been totally misled.
With an error margin of 2 percentage points, you cannot make any inference about anything within a 2 percentage point margin.
After discussing this point he states:
One can react to this article in (at least) two ways: one could become a bit more relaxed about the significance of changes reported in statistics or one could seek improvements to the accuracy of statistical collection.
In what areas do you think the first option is appropriate, and in what ways would it be worthwhile to increase spending to improve accuracy?
Yes. And three and four – transparency around the scope, objectives and limitations of (both official and non-official) statistics could be more widely disseminated (there is a huge variance in expected quality between say a Tier 1 statistic and a press initiated political poll), and/or recognition that the capabilities of those using (and/or communicating) statistics to identify and relay those limitations is also a vitally important factor 🙂
Indeed – in fairness to Stats, as an economist I am able to find a lot of good information about shortcomings and scope. The link between the data and having these known for the other people that want to use it is a hard thing though 🙂