Eldritch
01-30-2011, 01:39 PM
Ben Goldacre's Bad Science (http://en.wikipedia.org/wiki/Bad_Science_(book)) is a recommended read for anyone who is concerned about the amounts of quackery and pseudoscience in current public discourse. (He is also a curmudgeon who never forgets a grudge, as the article below demostrates :D)
He publishes a series of columns (http://www.badscience.net/2011/01/how-to-read-a-paper/) under the same title in The Guardian (I know, I know), and I think the latest one is worth sharing:
If science has any authority, it derives from transparency: you can check the claims against the working. Sometimes you hit a brick wall. Sometimes you might consider a shortcut. Let’s look at 3 types of checking.
First up, in the Sun, a child has been born at 7:43 (http://www.thesun.co.uk/sol/homepage/news/3366983/Third-baby-born-at-exactly-743-for-same-parents.html), just like their two siblings (though one was in the evening). The Sun says the odds on this are 300 million to one. This is an easy thing to check, because the information is all there in the paper.
The Sun are wrong. There are 60 minutes in each hour, 12 hours on a clock, that’s 720 minutes. The first child can be born in any minute – we’re not interested in the chance of 3 children being born at 7:43 more than any other minute, just the chances of 3 being born in the same minute.
So once the first child is born, there’s a one in 720 chance of the next child matching on birth time, and if that happens, then a one in 720 chance of the next one matching too. 720 x 720 makes the overall odds of 3 matching birth minutes 518,400 to one, not 300 million.
Since there are 167,000 third or more-th children born in England and Wales each year, you’ll see this coincidence once every three years, more frequently if you include the rest of the kingdom, and even more frequently if the midwife squints at the clock and says: “Oh, was the last one born at 7:43? Well…”
Our next case takes more elaborate checking, since it involves an experiment and its interpretation. Scientists at Lancaster University, say the Daily Mail (http://www.dailymail.co.uk/sciencetech/article-1350118/Child-Defence-New-mobile-phone-app-identifies-predatory-adults-posing-children.html) and the BBC (http://www.bbc.co.uk/news/uk-england-lancashire-12249172), have devised an amazing piece of paedophile identification software. It reads your messages and decides if the person you’re chatting to on the internet is another young person, or an adult who is pretending to be young.
This is a tricky problem to solve on a handheld device, or indeed anywhere. There is a press release (http://domino.lancs.ac.uk/info/lunews.nsf/I/D1C01D88E111646D802578220032C530) on the Lancaster University website explaining that this device has been studied and found to work. I asked for details. The methods and results of this study are secret. No paper has been submitted for publication.
So actually there’s no complicated interpretation problem here: nobody can know what these scientists measured, how they measured it, what the numbers were like, how closely the experiment mirrored a real world situation, or anything at all. When the Raelian cult said they’d cloned a baby, but we weren’t allowed to see it, nobody took them seriously. Until someone’s willing to tell me what they measured and how they measured it, they might as well be Raelians.
Is this flippant? We live in a big world, filled with amazing scientific work to read. It can also be overwhelming, and you need someone to walk you through the forest. This brings us on to our last form of checking, the trickiest of all: how do you know if someone has fairly represented the findings of an entire field, or simply cherry picked the results that suit them, to build a story?
Zoe Harcombe (http://blog.wcrf-uk.org/2010/11/is-zoe-harcombe’s-advice-based-on-solid-scientific-evidence/) sells diet books.
This week in the Daily Mail she was explaining that fruit and veg are actually no good for you (http://www.dailymail.co.uk/femail/food/article-1349960/5-day-fruit-vegetables-myth-claims-nutrition-expert.html). There’s a fascinating conversation to be had about the evidence base on the relationship between diet and health: would you start with Zoe’s work?
We all rely on heuristics, or shortcuts. Trusting an authority is one. Zoe boasts in the Mail that she is “studying for a PhD in nutrition” but she admitted to me, tediously, inevitably, that she’s not registered for a PhD anywhere (although she is thinking about doing one in the future).
Does it matter? We read a precis of research as a shortcut, but once you lose trust, to double check whether someone has fairly represented an entire field, you’d have to read that field’s entire canon, and after many years of work, whatever your other conclusions were, the strongest would be that any timesaving benefit from reading a precis has plainly been annihilated. Given that this is the case, I know it’s harsh, and you may disagree, but in a busy world, I’m not sure I see the point of a Zoe Harcombe.
Ben Goldacre, The Guardian, Saturday 29 January 2011
He publishes a series of columns (http://www.badscience.net/2011/01/how-to-read-a-paper/) under the same title in The Guardian (I know, I know), and I think the latest one is worth sharing:
If science has any authority, it derives from transparency: you can check the claims against the working. Sometimes you hit a brick wall. Sometimes you might consider a shortcut. Let’s look at 3 types of checking.
First up, in the Sun, a child has been born at 7:43 (http://www.thesun.co.uk/sol/homepage/news/3366983/Third-baby-born-at-exactly-743-for-same-parents.html), just like their two siblings (though one was in the evening). The Sun says the odds on this are 300 million to one. This is an easy thing to check, because the information is all there in the paper.
The Sun are wrong. There are 60 minutes in each hour, 12 hours on a clock, that’s 720 minutes. The first child can be born in any minute – we’re not interested in the chance of 3 children being born at 7:43 more than any other minute, just the chances of 3 being born in the same minute.
So once the first child is born, there’s a one in 720 chance of the next child matching on birth time, and if that happens, then a one in 720 chance of the next one matching too. 720 x 720 makes the overall odds of 3 matching birth minutes 518,400 to one, not 300 million.
Since there are 167,000 third or more-th children born in England and Wales each year, you’ll see this coincidence once every three years, more frequently if you include the rest of the kingdom, and even more frequently if the midwife squints at the clock and says: “Oh, was the last one born at 7:43? Well…”
Our next case takes more elaborate checking, since it involves an experiment and its interpretation. Scientists at Lancaster University, say the Daily Mail (http://www.dailymail.co.uk/sciencetech/article-1350118/Child-Defence-New-mobile-phone-app-identifies-predatory-adults-posing-children.html) and the BBC (http://www.bbc.co.uk/news/uk-england-lancashire-12249172), have devised an amazing piece of paedophile identification software. It reads your messages and decides if the person you’re chatting to on the internet is another young person, or an adult who is pretending to be young.
This is a tricky problem to solve on a handheld device, or indeed anywhere. There is a press release (http://domino.lancs.ac.uk/info/lunews.nsf/I/D1C01D88E111646D802578220032C530) on the Lancaster University website explaining that this device has been studied and found to work. I asked for details. The methods and results of this study are secret. No paper has been submitted for publication.
So actually there’s no complicated interpretation problem here: nobody can know what these scientists measured, how they measured it, what the numbers were like, how closely the experiment mirrored a real world situation, or anything at all. When the Raelian cult said they’d cloned a baby, but we weren’t allowed to see it, nobody took them seriously. Until someone’s willing to tell me what they measured and how they measured it, they might as well be Raelians.
Is this flippant? We live in a big world, filled with amazing scientific work to read. It can also be overwhelming, and you need someone to walk you through the forest. This brings us on to our last form of checking, the trickiest of all: how do you know if someone has fairly represented the findings of an entire field, or simply cherry picked the results that suit them, to build a story?
Zoe Harcombe (http://blog.wcrf-uk.org/2010/11/is-zoe-harcombe’s-advice-based-on-solid-scientific-evidence/) sells diet books.
This week in the Daily Mail she was explaining that fruit and veg are actually no good for you (http://www.dailymail.co.uk/femail/food/article-1349960/5-day-fruit-vegetables-myth-claims-nutrition-expert.html). There’s a fascinating conversation to be had about the evidence base on the relationship between diet and health: would you start with Zoe’s work?
We all rely on heuristics, or shortcuts. Trusting an authority is one. Zoe boasts in the Mail that she is “studying for a PhD in nutrition” but she admitted to me, tediously, inevitably, that she’s not registered for a PhD anywhere (although she is thinking about doing one in the future).
Does it matter? We read a precis of research as a shortcut, but once you lose trust, to double check whether someone has fairly represented an entire field, you’d have to read that field’s entire canon, and after many years of work, whatever your other conclusions were, the strongest would be that any timesaving benefit from reading a precis has plainly been annihilated. Given that this is the case, I know it’s harsh, and you may disagree, but in a busy world, I’m not sure I see the point of a Zoe Harcombe.
Ben Goldacre, The Guardian, Saturday 29 January 2011