Part 5 (1/2)

Bad Science Ben Goldacre 244570K 2022-07-22

He told me, 'I said, 'Sue me.' I'm still waiting.' His offer of 1,000 still stands.

But there is one vital issue we have not yet covered. Because despite the way she seems to respond to criticism or questioning of her ideas, her illegal p.e.n.i.s pills, the unusually complicated story of her qualifications, despite her theatrical abusiveness, and the public humiliation pantomime of her shows, in which the emotionally vulnerable and obese cry on television, despite her apparently misunderstanding some of the most basic aspects of GCSE biology, despite doling out 'scientific' advice in a white coat, despite the dubious quality of the work she presents as somehow being of 'academic' standard, despite the unpleasantness of the food she endorses, there are still many who will claim: 'You can say what you like about McKeith, but she has improved the nation's diet.'

This is not to be shrugged off lightly. Let me be very clear, for I will say it once again: anyone who tells you to eat more fresh fruit and vegetables is all right by me. If that was the end of it, I'd be McKeith's biggest fan, because I'm all in favour of 'evidence-based interventions to improve the nation's health', as they used to say to us in medical school.

Let's look at the evidence. Diet has been studied very extensively, and there are some things that we know with a fair degree of certainty: there is reasonably convincing evidence that having a diet rich in fresh fruit and vegetables, with natural sources of dietary fibre, avoiding obesity, moderating one's intake of alcohol, cutting out cigarettes and taking physical exercise are protective against things such as cancer and heart disease.

Nutritionists don't stop there, because they can't: they have to manufacture complication, to justify the existence of their profession. These new nutritionists have a major commercial problem with the evidence. There's nothing very professional or proprietary about 'Eat your greens,' so they have had to push things further. But unfortunately for them, the technical, confusing, overcomplicated, tinkering interventions that they promote-the enzymes, the exotic berries-are very frequently not supported by convincing evidence.

That's not for lack of looking. This is not a case of the medical hegemony neglecting to address the holistic needs of the people. In many cases the research has been done, and has shown that the more specific claims of nutritionists are actually wrong. The fairy tale of antioxidants is a perfect example. Sensible dietary practices, which we all know about, still stand. But the unjustified, unnecessary overcomplication of this basic dietary advice is, to my mind, one of the greatest crimes of the nutritionist movement. As I have said, I don't think it's excessive to talk about consumers paralysed with confusion in supermarkets.

But it's just as likely that they will be paralysed with fear. They may have a bad reputation for paternalism, but it's hard to picture any doctor in the past century using McKeith's consultation methods as a serious tactic for inducing lifestyle change in his patients. With McKeith we see fire and brimstone hailing down until her subjects cry on national television: a chocolate gravestone with your name on it in the garden; a shouty dressing-down in public for the obese. As a posture it is as seductive as it is telegenic, it has a sense of generating movement; but if you drag yourself away from the theatricality of souped-up recipe and lifestyle shows on telly, the evidence suggests that scare campaigns may not get people changing their behaviour in the long term.

What can you do? There's the rub. The most important take-home message with diet and health is that anyone who ever expresses anything with certainty is basically wrong, because the evidence for cause and effect in this area is almost always weak and circ.u.mstantial, and changing an individual person's diet may not even be where the action is.

What is the best evidence on the benefits of changing an individual person's diet? There have been randomised controlled trials, for example-where you take a large group of people, change their diet, and compare their health outcomes with another group-but these have generally shown very disappointing results.

The Multiple Risk Factor Intervention Trial was one of the largest medical research projects ever undertaken in the history of mankind, involving over 12,866 men at risk of cardiovascular events, who went through the trial over seven years. These people were subjected to a phenomenal palaver: questionnaires, twenty-four-hour dietary recall interviews, three-day food records, regular visits, and more. On top of this, there were hugely energetic interventions which were supposed to change the lives of individuals, but which by necessity required that whole families' eating patterns were transformed: so there were weekly group information sessions for partic.i.p.ants-and their wives-individual work, counselling, an intensive education programme, and more. The results, to everyone's disappointment, showed no benefit over the control group (who were not told to change their diet). The Women's Health Initiative was another huge randomised controlled trial into dietary change, and it gave similarly gave negative results. They all tend to.

Why should this be? The reasons are fascinating, and a window into the complexities of changing health behaviour. I can only discuss a few here, but if you are genuinely interested in preventive medicine-and you can cope with uncertainty and the absence of quick-fix gimmicks-then may I recommend you pursue a career in it, because you won't get on telly, but you will be both dealing in sense and doing good.

The most important thing to notice is that these trials require people to turn their entire lives upside down, and for about a decade. That's a big ask: it's hard enough to get people signed up for partic.i.p.ating in a seven-week trial, let alone one that lasts seven years, and this has two interesting effects. Firstly, your partic.i.p.ants probably won't change their diets as much as you want them to; but far from being a failing, this is actually an excellent ill.u.s.tration of what happens in the real world: individual people do not, in reality, change their diets at the drop of a hat, alone, as individuals, for the long term. A dietary change probably requires a change in lifestyle, shopping habits, maybe even what's in the shops, how you use your time, it might even require that you buy some cooking equipment, how your family relates to each other, change your work style, and so on.

Secondly, the people in your 'control group' will change their diets too: remember, they've agreed voluntarily to take part in a hugely intrusive seven-year-long project that could require ma.s.sive lifestyle changes, so they may have a greater interest in health than the rest of your population. More than that, they're also being weighed, measured, and quizzed about their diet, all at regular intervals. Diet and health are suddenly much more at the forefront of their minds. They will change too.

This is not to rubbish the role of diet in health-I bend over backwards to find some good in these studies-but it does reflect one of the most important issues, which is that you might not start with goji berries, or vitamin pills, or magic enzyme powders, and in fact you might not even start with an individual changing their diet. Piecemeal individual life changes-which go against the grain of your own life and your environment-are hard to make, and even harder to maintain. It's important to see the individual-and the dramatic claims of all lifestyle nutritionists, for that matter-in a wider social context.

Reasonable benefits have been shown in intervention studies-like the North Karelia Project in Finland-where the public health gang have moved themselves in lock, stock and barrel to set about changing everything about an entire community's behaviour, liaising with businesses to change the food in shops, modifying whole lifestyles, employing community educators and advocates, improving healthcare provision, and more, producing some benefits, if you accept that the methodology used justifies a causal inference. (It's tricky to engineer a control group for this kind of study, so you have to make pragmatic decisions about study design, but read it online and decide for yourself: I'd call it a 'large and promising case study'.) There are fairly good grounds to believe that many of these lifestyle issues are, in fact, better addressed at the societal level. One of the most significant 'lifestyle' causes of death and disease, after all, is social cla.s.s. To take a concrete example, I rent a flat in London's Kentish Town on my modest junior medic's salary (don't believe what you read in the papers about doctors' wages). This is largely a white, working-cla.s.s area, and the adult male life expectancy is about seventy years. Two miles away, in Hampstead, where the millionaire entrepreneur Dr Gillian McKeith PhD owns a large property, surrounded by other wealthy middlecla.s.s people, male life expectancy is almost eighty years. I know this because I have the Annual Public Health Report for Camden open on my kitchen table right now.

The reason for this phenomenal disparity in life expectancy-the difference between a lengthy and rich retirement, and a very truncated one indeed-is not that the people in Hampstead are careful to eat goji berries and a handful of Brazil nuts every day, thus ensuring they're not deficient in selenium, as per nutritionists' advice. That's a fantasy, and in some respects one of the most destructive features of the whole nutritionist project, graphically exemplified by McKeith: it's a distraction from the real causes of ill health, but also-do stop me if I'm pus.h.i.+ng this too far-in some respects, a manifesto of right-wing individualism. You are what you eat, and people die young because they deserve it. They They choose death, through ignorance and laziness, but choose death, through ignorance and laziness, but you you choose life, fresh fish, olive oil, and that's why you're healthy. You're going to see eighty. You deserve it. Not like choose life, fresh fish, olive oil, and that's why you're healthy. You're going to see eighty. You deserve it. Not like them them.

Back in the real world, genuine public health interventions to address the social and lifestyle causes of disease are far less lucrative, and far less of a spectacle, than anything a Gillian McKeith-or, more importantly, a television commissioning editor-would ever dream of dipping into. What prime-time TV series looks at food deserts created by giant supermarket chains, the very companies with which these stellar media nutritionists so often have their lucrative commercial contracts? Who puts the issue of social inequality driving health inequality onto our screens? Where's the human interest in prohibiting the promotion of bad foods, facilitating access to healthier foods by means of taxation, or maintaining a clear labelling system?

Where is the spectacle in 'enabling environments' that naturally promote exercise, or urban planning that prioritises cyclists, pedestrians and public transport over the car? Or in reducing the ever-increasing inequality between senior executive and shop-floor pay? When did you ever hear about elegant ideas like 'walking school buses', or were stories about their benefits crowded out by the latest urgent front-page food fad news?

I don't expect Dr Gillian McKeith, or anyone in the media, to address a single one of these issues, and neither do you: because if we are honest with ourselves we understand that these programmes are only partly about food, and much more about salacious and prurient voyeurism, tears, viewing figures and vaudeville.

Dr McKeith puts a cabbie straight Here is my favourite Dr McKeith story, and it comes from her own book, Living Food for Health Living Food for Health. She is in a cab, and the driver, Harry, has spotted her. He tries to spark up a friendly conversation by suggesting that fish contains more omega oils than flax. Dr McKeith disputes this: 'Flax seeds contain far greater levels of the healthy oils (omega-3 and omega-6) in a properly balanced and a.s.similable form.' When Harry disagrees, she replies: 'What do you mean, you disagree? Have you spent years conducting clinical research, working with patients, lecturing, teaching, studying the omega oils in flax, obtaining worldwide data, compiling one of the largest private health libraries on the planet, and writing extensively on the topic? Are you a scientist, a biochemist, a botanist, or have you spent a lifetime studying food and biochemistry as I have done? Where is your scientific authority?' Harry responds that his wife is a doctor, a gynaecologist. 'Is she a food specialist or nutritional biochemist as well?' demands Dr McKeith. 'Um, ah, well, no, but she is a doctor.'

I am not a food specialist, nor am I a nutritional biochemist. In fact, as you know, I claim no special expertise whatsoever: I hope I can read and critically appraise medical academic literature-something common to all recent medical graduates-and I apply this pedestrian skill to the millionaire businesspeople who drive our culture's understanding of science.

Flax seeds contain large amounts of fibre (along with oestro-genic compounds), so they're not very 'a.s.similable', as Dr McKeith a.s.serts, unless you crush them, in which case they taste foul. They're sold as a laxative in doses of 15g, and you will need a lot of the stuff, partly because there's also a problem with the form of omega oils in them: flax contains a short-chain plant form, and these must be converted in your body to the long-chain animal forms which may be beneficial (called DHA and EPA). When you account for the poor conversion in the body, then flax seeds and fish contain rouglily the same amounts of omega oil.

We must also remember that we live not in a laboratory, but in the real world. It's very easy to eat 100g of mackerel-if this were a completely different kind of book I'd be giving you my kedgeree recipe right now-whereas I would suggest that it's slightly trickier to get a tablespoon of flax seed into you. Parsley, similarly, is a rich source of vitamin C, but you're not going to eat an orange-sized lump of the stuff. As for Dr McKeith's further claim that flax is 'properly balanced', I don't know if she means spiritually or biologically, but fish is much higher in omega-3, which most people would say is better.

More importantly, why is everyone talking about omega-3? On to the next chapter.

8 'Pill Solves Complex Social Problem'

Medicalisation-or 'Will fish-oil pills make my child a genius?'

In 2007 the British Medical Journal British Medical Journal published a large, well-conducted, randomised controlled trial, performed at lots of different locations, run by publicly funded scientists, and with a strikingly positive result: it showed that one treatment could significantly improve children's antisocial behaviour. The treatment was entirely safe, and the study was even accompanied by a very compelling cost-effectiveness a.n.a.lysis. published a large, well-conducted, randomised controlled trial, performed at lots of different locations, run by publicly funded scientists, and with a strikingly positive result: it showed that one treatment could significantly improve children's antisocial behaviour. The treatment was entirely safe, and the study was even accompanied by a very compelling cost-effectiveness a.n.a.lysis.

Did this story get reported as front-page news in the Daily Mail Daily Mail, the natural home of miracle cures (and sinister hidden scares)? Was it followed up on the health pages, with an accompanying photo feature, describing one child's miraculous recovery, and an interview with an attractive relieved mum with whom we could all identify?

No. The story was unanimously ignored by the British news media, despite their preoccupation with both antisocial behaviour and miracle cures, for one simple reason: the research was not about a pill. It was about a cheap, practical parenting programme.

At the same time, for over five years now, newspapers and television stations have tried to persuade us, with 'science', that fish-oil pills have been proven to improve children's school performance, IQ, behaviour, attention, and more. In fact, nothing could be further from the truth. We are about to learn some very interesting lessons about the media, about how not to conduct a trial, and about our collective desire for medicalised, sciencey-sounding explanations for everyday problems. Do fish-oil pills work? Do they make your child cleverer and better-behaved? The simple answer is, at the moment, n.o.body could possibly know. Despite everything you have been told, a trial has never been done in mainstream children.

The newspapers would have you believe otherwise. I first became aware of 'the Durham trials' when I saw on the news that a trial of fish-oil capsules was being planned in 5,000 children. It's an astonis.h.i.+ng testament to the news values of the British media that this piece of research remains, I am quite prepared to suggest, probably the single most well-reported clinical trial of the past few years. It was on Channel 4 and ITV, and in every national newspaper, sometimes repeatedly. Impressive results were confidently predicted.

This rang alarm bells for two reasons. Firstly, I knew the results of the previous trials offish-oil capsules in children-I'll describe them in due course-and they weren't very exciting. But more than that, as a basic rule, I would say this: whenever somebody tells you that their trial is going to be positive before they've even started it, then you know you're onto an interesting story.

Here is what they were planning to do in their 'trial': recruit 5,000 children in their GCSE year, give them all six fish-oil capsules a day, then compare how they did in their exams with how the council had estimated they should do without the capsules. There was no 'control' group to compare against (like the Aqua Detox bath with no feet in it, or the ear candle on the picnic table, or a group of children taking placebo capsules without fish oil in them). Nothing.

By now you might not need me to tell you that this is a preposterous and above all wasteful way to go about doing a study on a pill that is supposed to improve school performance, with 1 million worth of generously donated capsules and 5,000 children at your disposal. But humour an old man, and let me flesh out your hunches, because if we cover the theoretical issues properly first, then the 'researchers' at Durham become even more amusingly absurd.

Why you have a placebo group If you divide a group of kids in half, and give a placebo capsule to one group, and the real capsule to the other group, you can then compare how well each group does, and see whether it was the ingredients in the pill that made the difference to their performance, or just the fact of taking a pill, and being in a study. Why is this important? Because you have to remember that whatever you do to children, in a trial of a pill to improve their performance, their performance will improve.

Firstly, children's skills improve over time anyway: they grow up, time pa.s.ses, and they get better at stuff. You might think you're clever, sitting there with no nappy on, reading this book, but things haven't always been that way, as your mother could tell you.

Secondly, the children-and their parents-know that they are being given these tablets to improve their performance, so they will be subject to a placebo effect. I have already harped on about this at phenomenal length, because I think the real scientific story of the connections between body and mind are infinitely more interesting than anything concocted by the miracle-cure community, but here it is enough to remind you that the placebo effect is very powerful: consciously or unconsciously, the children will txpect themselves to improve, and so will their parents and their teachers. Children are exquisitely sensitive to our expectations of them, and anyone who doubts that fact should have their parenting permit revoked.

Thirdly, children will do better just from being in a special group that is being studied, observed and closely attended to, since it seems that the simple fact of being in a trial being in a trial improves your performance, or recovery from illness. This phenomenon is called the 'Hawthorne effect', not after a person, but after the factory where it was first observed. In 1923 Thomas Edison (he of the lightbulb) was chairing the 'Committee on the Relation of Quality and Quant.i.ty of Illumination to Efficiency in the Industries'. Various reports from several companies had suggested that better lighting might increase productivity, so a researcher called Deming went with his team to test the theory at Western Electric's Hawthorne plant at Cicero, Illinois. improves your performance, or recovery from illness. This phenomenon is called the 'Hawthorne effect', not after a person, but after the factory where it was first observed. In 1923 Thomas Edison (he of the lightbulb) was chairing the 'Committee on the Relation of Quality and Quant.i.ty of Illumination to Efficiency in the Industries'. Various reports from several companies had suggested that better lighting might increase productivity, so a researcher called Deming went with his team to test the theory at Western Electric's Hawthorne plant at Cicero, Illinois.

I will give you the simplified 'myth' version of the findings, as a rare compromise between pedantry and simplicity. When the researchers increased light levels, they found that performance improved. But when they reduced the light levels, performance improved then, too. In fact, they found that no matter what they did, productivity increased anyway. This finding was very important: when you tell workers they are part of a special study to see what might improve productivity, and then you do something...they improve their productivity. This is a kind of placebo effect, because the placebo is not about the mechanics of a sugar pill, it is about the cultural meaning of an intervention, which includes, amongst other things, your expectations, and the expectations of the people tending to you and measuring you.

Beyond all this technical stuff, we also have to place the GCSE results-the outcome which was being measured in this 'trial'-in their proper context. Durham has a very bad exam record, so its council will be battling hard in every way it possibly can to improve school performance, with all manner of different initiatives and special efforts, and extra funding, running simultaneously with the fish-oil 'trials'.

We should also remember that bizarre English ritual whereby GCSE results get better every year, yet anyone who suggests that the exams are getting easier is criticised for undermining the achievement of the successful candidates. In fact, taking the long view, this easing is obvious: there are forty-year-old O-level papers which are harder than the current A-level syllabus; and there are present-day university finals papers in maths that are easier than old A-level pipers.

To recap: GCSE results will get better anyway; Durham will be desperately trying to improve its GCSE results through other methods anyway; and any kids taking pills will improve their GCSE results anyway, because of the placebo effect and the Hawthorne effect.

This could all be avoided by splitting the group in half and giving a placebo to one group, separating out what is a specific effect of the fish-oil pills, and what is a general effect of all the other stuff we've described above. That would give you very useful information.

Is it ever acceptable to do the kind of trial that was being carried out in Durham? Yes. You can do something called an 'open trial', without a placebo group, and this is an accepted kind of research. In fact, there is an important lesson about science here: you can do a less rigorous experiment, for practical reasons, as long as you are clear about what you are doing when you present your study, so other people can make their own minds up about how they interpret your findings.

But there is an important caveat. If you do this kind of 'compromise' study, without a placebo group, 'open label', but in the hope of getting the most accurate' possible picture of the treatment's benefits, then you do it as carefully as you can, while being fully aware that your results might be distorted by expectation, by the placebo effect, by the Hawthorne effect, and so on. You might sign up your kids calmly and cautiously, saying in a casual, offhand fas.h.i.+on that you're doing a small informal study on some tablets, you don't say what you expect to find, you hand them out without fanfare, and you calmly measure the results at the end.

What they did in Durham was the polar opposite. There were camera crews, soundmen and lighting men flooding the cla.s.srooms. Children were interviewed for radio, for television, for the newspapers; so were their parents; so were their teachers: so were Madeleine Portwood, the educational psychologist who was performing the trial, and Dave Ford, Head of Education, talking-bizarrely-about how they confidently expected positive results. They did literally everything which, in my view, would guarantee them a false positive result, and ruin any chance of their study giving meaningful and useful new information. How often does this happen? In the world of nutritionism, sadly, it seems to be standard research protocol.

We should also remember that these fish-oil 'trials' were measuring some highly volatile outcomes. Performance at school in a test, and 'behaviour' (a word with a large semantic footprint, if ever I saw one) are huge, variable, amorphous things. More than most outcomes, they will change from moment to moment, with different circ.u.mstances, frames of mind, and expectations. Behaviour is not like a blood haemoglobin level, or even height, and nor is intelligence.

Durham Council and Equazen were so successful in their publicity drive, whether through an uncontainable enthusiasm for a positive result or simple foolishness (I really don't know which) that they effectively sabotaged their 'trial'. Before the first fish-oil capsule was swallowed by a single child, the Eye Q branded supplement and trial had received glowing publicity in the local papers, the Guardian Guardian, the Observer Observer, the Daily Mail, The Times Daily Mail, The Times, Channel 4, the BBC, ITV, the Daily Express Daily Express, the Daily Mirror Daily Mirror, the Sun Sun, GMTV, Woman's Own Woman's Own, and many more. n.o.body could claim that the children weren't well primed.*

- In fact, it's hard to overstate quite how large the fish-oil circus became, over the many years it ran. Professor Sir Robert Winston himself, moustachioed presenter of innumerable 'science' programmes for the BBC, personally endorsed a competing omega-3 product, in an advertising campaign which was ultimately terminated by the ASA since it breached their codes on truthfulness and substantiation. - In fact, it's hard to overstate quite how large the fish-oil circus became, over the many years it ran. Professor Sir Robert Winston himself, moustachioed presenter of innumerable 'science' programmes for the BBC, personally endorsed a competing omega-3 product, in an advertising campaign which was ultimately terminated by the ASA since it breached their codes on truthfulness and substantiation.