Thursday, 28 May 2015

Junk science from farm to fork

This has been doing the rounds on Twitter and deservedly so. It's an excellent demonstration of how quack science is conducted, published and reported.

John Bohannon got sixteen volunteers, a statistician and some chocolate with the hope of finding an association with something - anything - for an epidemiological study.

One group followed a low-carbohydrate diet. Another followed the same low-carb diet plus a daily 1.5 oz. bar of dark chocolate. And the rest, a control group, were instructed to make no changes to their current diet. They weighed themselves each morning for 21 days, and the study finished with a final round of questionnaires and blood tests.

It turned out that the chocolate-eating group lost ten per cent more weight than the other two groups. A newsworthy finding? It shouldn't have been.

Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.

... With our 18 measurements, we had a 60% chance of getting some“significant” result with p < 0.05. (The measurements weren’t independent, so it could be even higher.) The game was stacked in our favor.

He then submitted it for - cough - peer review...

Our paper was accepted for publication by multiple journals within 24 hours. Needless to say, we faced no peer review at all. The eager suitor we ultimately chose was the the International Archives of Medicine. It used to be run by the giant publisher BioMedCentral, but recently changed hands. The new publisher’s CEO, Carlos Vasquez, emailed Johannes to let him know that we had produced an “outstanding manuscript,” and that for just 600 Euros it “could be accepted directly in our premier journal.”

And awaited the ladies and gentlemen of press...

We landed big fish before we even knew they were biting. Bild rushed their story out—”Those who eat chocolate stay slim!”—without contacting me at all. Soon we were in the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German and Indian site of the Huffington Post, and even television news in Texas and an Australian morning talk show.

 That's how it easy it is folks. Do read the whole article.

Illegal highs

The government has decided to ban all substances that could have a psychoactive effect, including those yet to be invented/discovered, because some of them might be bad for you. Tobacco, alcohol and coffee will be exempt (for now, at least). The intention is to use this sledge-hammer to crack the nut of 'legal highs', but the legal implications are profound.

I've written about it at Spectator Blogs. Do have a read.

Ian Dunt has written an excellent article about the same issue.

Wednesday, 27 May 2015

Under-reporting alcohol

From the Telegraph...

Just one alcoholic drink a day damages hearts of elderly women, Harvard study warns
Only one alcoholic drink a day damages the hearts of elderly women, a new study has warned.

Despite previous research that suggested a drink a day might protect against some cardiovascular disease, the findings suggested otherwise, at least in the elderly.

Women who have just one alcoholic drink a day are defined as light to moderate drinkers. Yet new research found women with an average age of 76 who drink moderately had small reductions in heart function because of an enlargement of the wall of the heart's main pumping chamber - the left ventricular mass.

I'll take a small reduction in heart function at the age of 76 if it means I have less chance of getting heart disease. And make no mistake, heart disease is significantly less prevalent amongst drinkers than it is amongst teetotallers.

The temperance lobby hates this fact, which is why there is a concerted and growing effort to persuade people that there is no safe level of drinking. I don't know whether this particular study was designed to serve that end but the merchants of doubt will be pleased to see it in the media.

Also in the media recently was a story about drinkers under-reporting what they drink. This comes up from time to time (eg. here in 2013) and could give the unwary reader the impression that Britons drink much more than was previously thought. This would be true if we estimated how much people drink by asking them, but we don't. We work out national consumption by looking at how much is sold. This does not give a perfect picture because some alcohol is never drunk and some alcohol is bought illegally (and therefore doesn't show up on HMRC tax receipts), but it is fairly accurate and it is certainly more accurate than asking drinkers.

Under-reporting was in the news last week because some researchers decided to ask people how much they drink on holiday and on special occasions. Surprise, surprise, they drank more than in the average working week.

This is much as might be expected. Under-reporting in endemic in the alcohol field and researchers are rightly sceptical about self-reported evidence. But the important point is that virtually all alcohol epidemiology is based on self-reported evidence. If people under-report their consumption by 40-60 per cent, then observational studies which show a risk from some disease or other at (for the sake of argument) 50 units a week actually show risk kicking in at 75 units a week.

Alcohol researchers occasionally mention this. Here is Sadie Boniface in 2013, for example...

Much of what is known about the relationship between drinking and harm is based on self-reported data, where consumption was under-reported. This means that the relationship between drinking and harm may have been over-estimated, and that drinking is effectively ‘safer’ than the Government’s drinking guidelines suggest. This raises the question of whether the guidelines should be raised to reflect the actual relationship between drinking and harm, creating perverse incentives for alcohol policy (not just in the UK but worldwide).

So it's not that researchers are unaware of the problem. They just don't seem to be as interested in it as they could be considering the implications (tellingly, Boniface says that "there wasn’t space to discuss [this issue] in detail in the paper [that she had just published]"). Certainly, there is no urgency to raise the guidelines. Quite the reverse.

Last week, the subject got an airing in The Parliament Magazine. Referring to research conducted in the US, it concluded...

In summary, scientific data clearly show a relation between heavy drinking and certain types of cancer. For even light-to-moderate drinking, many studies have shown a slight increase in the risk of certain cancers, especially breast cancer.

However, when subjects likely to be underreporting their intake are removed from that pool, new data indicate that light to moderate drinking is not associated with an increased risk for breast cancer or for other types of cancer.

The question of alcohol and cancer boils down to dose: heavy consumption increases risk but light to moderate consumption is unlikely to increase risk.

This is the exact opposite of what the 'no safe level' faction of the neo-temperance lobby want you to believe so you can expect the message to be confined to niche publications and blogs while claims about the putative health risks of drinking tiny amounts of alcohol - which are not, in all probability, so tiny - proliferate.

Monday, 25 May 2015

Is 'neo-liberalism' making us ill?

I was on BBC Scotland on Saturday morning discussing the hypothesis that 'neo-liberalism' makes us ill with a Marxist academic. The subject came up because another Marxist academic has written a book that blames obesity and stress on the free market. The unspoken message is that we would all be healthier in a socialist society. We'd be thinner, I suppose. I'm not so sure about the rest of the theory.

If you're interested, you can listen to me from 49 minutes here. The author of the book explains her thesis a few minutes earlier.

In the meantime, c'mon the Boro!


Middlesbrough 0 - Norwich 2

Sunday, 24 May 2015

Brace yourself

The Westminster government's decision to introduce plain packaging has left that the tobacco control industry looking for new ways to collect government grants. Having portrayed plain packaging as something akin to a vaccine for lung cancer (no, really), I expect them to now play down the benefits of plain packaging for two reasons. Firstly because it will soon become very clear to the people of Britain that plain packaging doesn't have any benefits, and secondly because if they are perceived to have solved a problem, they will be out of work.

Fortunately for the tax leeches of 'public health', there is always one more batshit crazy policy ripe for expensive pseudo-research.

From the Sunday Post...

Cigarettes could soon be produced in unpleasant colours, with health warnings emblazoned across the stick, under new proposals to make smoking unglamorous.

Experts believe the white papers which traditionally encase tobacco have connotations of purity and cleanliness.

They have conducted research which suggests producing cigarettes in unappealing browns and greens, to represent yellowing teeth or even phlegm, will make them look distasteful, particularly to style-conscious young women.

This idea has been taken wholesale from a nutter in New Zealand who proposed the same thing last May (see Dick Puddlecote here and here)...

Anti-smoking group wants to change white paper to unattractive green, brown and orange shades

Public health researchers say the Government's next step after introducing plain packaging for tobacco should be to make cigarettes ugly by changing them to a dark green or brown colour which made young people think of "slime, vomit or pooh".

A tobacco control lobby group told a parliamentary committee that cigarettes themselves were the "new canvas" for anti-smoking initiatives.

The Sunday Post article is based on the ramblings of Crawford Moodie, one of Gerard Hastings' henchman from the Institute of Social Marketing. He makes the rather implausible claim that focus groups are "incredibly positive" about the idea. Try asking your friends about it this weekend. I suspect that most of them will say it is ridiculous, if they believe you at all (have these muppets noticed realised that a cigarette with brown paper is a cigar?)

More plausibly, the Post says that Moodie has "previously written around 40 expert reports on tobacco packaging". Forty! This gives you an idea of how much money that is swilling around for those who are prepared to stoop low enough. It also explains why you have almost certainly not heard the last of this.

Friday, 22 May 2015

Revisiting the Danish fat tax fiasco

A minister I've never heard of told the chattering classes at the Hay Festival that he supports a sugar tax. I was on the World at One (Radio 4) earlier today explaining the lessons that should be learnt from the Danish fat tax fiasco. I've also written a post for The Spectator so do please have a read.

Thursday, 21 May 2015

Bad science by press release

Ben Goldacre wrote a book called Bad Science a few years ago which explained why obvious quackery was quackery. The targets were deserving, though a bit soft for my tastes (homeopaths, nutritionists etc.), and it sold a lot of copies so it must have said something people needed to hear. He also used to write a column in The Guardian which often worth reading.

On his good days, Goldacre would patiently explain the difference between trustworthy evidence and dubious evidence. Published studies are better than unpublished studies, peer review is better than no peer review, randomised control trials are better than observational studies, literature reviews are better than individual studies, and so on. In short, the pyramid of evidence looks like this...

Goldacre has built his reputation on being a dispassionate, apolitical observer of human folly; a man driven by a thirst for evidence, not ideology. Given the choice between a Cochrane review of randomised control trials (top of the pyramid) and an unpublished conference abstract (third from bottom), it's pretty obvious which one he would prefer, right?

Wrong. In December, the first - and so far only - Cochrane review of e-cigarettes was published. It found 'evidence from two trials that ECs [e-cigarettes] help smokers to stop smoking long-term compared with placebo ECs'.

This week, a conference abstract for another literature review - one that included non-RCTs - was released to the press. It concluded that 'Evidence that electronic cigarettes are effective for smoking cessation long-term is lacking'.

To my knowledge, Goldacre has never mentioned the Cochrane review, nor has he mentioned the systematic review that was published in PLoS One last month which concluded that 'Use of e-cigarettes is associated with smoking cessation'. Nor, indeed, has he mentioned the study in Addiction which found e-cigarettes to be more than twice as effective than nicotine-replacement therapy, or the study published three months ago which found that intensive users of e-cigarettes were six times more likely to quit than non-users. But when presented with an unpublished conference abstract, he wasted no time in spreading the news with this bilious Twitter communiqué...

What's got into him? He has previously referred to e-cigarettes as the 'tobacco industry's latest scheme', seemingly unaware that until 2012 there was no tobacco industry involvement in the sector whatsoever and even today the combined might of global Big Tobacco owns just seven of the thousands of brands on the market. On the subject of vaping, Goldacre has shown himself to be no better than the two-bit hacks who write "I reckon" articles about e-cigarettes based on whatever stray thought enters their head. Gateway! Formaldehyde! Children!

But this is not a post about Goldacre's confirmation bias. There is another mistake he makes which was also made by the Daily Mail, namely that the study shows that 'e-cigs don't help smokers quit'. Even the press release doesn't claim that. It says only that, based on its findings, the evidence is 'lacking'.

Absence of evidence is not evidence of absence, particularly when you look at the results in the abstract. The review found the odds of e-cig users (versus placebo users) staying abstinent after one month were 1.71 (1.08-2.72). After three months they were 1.95 (0.74-5.13) and after six months they were 1.32 (0.59-2.93). The first of these is statistically significant, the others are not, hence the lack of firm evidence for cessation at six months.

The study also mentions that the 'only study to evaluate continuous abstinence' found that e-cigarette users' odds of abstinence were 1.77 (0.54-5.77) after six months compared to the placebo group. This, again, is not statistically significant, but nor is it close enough to 1.0 for a positive effect to be ruled out. The same is true of the other findings, all of which are well above 1.0. It would be quite wrong to say that these statistically non-significant odds ratios proved that e-cigarettes work better than placebos, but it is equally wrong to say that they show that 'e-cigs don't help people quit'. All they show is that larger trials will be needed unless e-cigarettes more than double the chance of quitting. The Cochrane review identified one randomised controlled trial which did indeed find that e-cigarettes more than doubled the chances of quitting. We don't know which studies the conference review included (or why) because, as I say, it hasn't been published.

I wouldn't claim for a moment that the Cochrane review is the last word on e-cigarettes and nor do its authors. Cochrane reviews only look at RCTs and only two RCTs have so far been conducted, making the review weak by Cochrane's high standards. Nevertheless, it does have the merit of being peer-reviewed and published, which is normally the minimum Goldacre requires before taking research seriously. That is, unless he wants to wind up a section of society - most of whom no longer smoke thanks to e-cigarettes - that he deems 'vile', in which case any old press release will do.