Sense about Science Annual Lecture – Dr Fiona Godlee

Firstly I want to apologise for being offline for so long – we moved house and it took a while to get the internet hooked up again. Then there was all the catching-up with work to do……anyway here I am again finally.

So the Sense about Science Lecture. SaS were kind enough to send me an invite to this when I asked. As an ex-Editor myself I am very interested in discussions about peer-review and its various failings.

Fiona Godlee is the current Editor of the British Medical Journal (BMJ); she has been in the job since 2005. In that time she has clearly had to grapple with numerous cases of ethical misconduct, in fact, going on what she told us, it would be fair to say that the medical and pharmaceutical literature is rife with it.

She discussed the problem of reporting bias – not unethical in itself but still something that skews perceptions and misleads the reader. Basically, people don’t like to report negative results. Drugs that don’t work very well will not make any money, and an experiment that shows nothing or worse, shows a problem, will jeopardise funding or even the job of that particular researcher in many cases. This is particularly true in medical research where the funding is usually coming from large pharmaceutical companies, but there is an element of it in academic research too, even where funding comes purely from the government. All university researchers have limited time and resources, and these are being squeezed at every corner. Papers that can be published in high-impact factor journals will advance a researcher’s career and help secure further funding. But papers that report negative results are rarely accepted for publication in high-impact or indeed reputable journals, as the editors know they will not be cited and would bring down their impact factor. This propagates reporting bias and also means that the experiments that did not work are doomed to be repeated again and again in different labs, as no-one knows that someone else has already tried it. This is a waste of the precious resources that are being so tightly squeezed already.

But Godlee’s lecture concentrated mainly on two perceived threats to science  – the external threat of those who refuse to accept the evidence-based approach to science, and the internal threat of undisclosed conflict of interest, which leads to data suppression, ghost- and guest-writing. In my background of materials, physics and chemistry, these are not such a big issue but neverthless it was interesting to consider.

Where researchers receive a large amount of their funding from industry, as is the case in medical research, these sorts of conflicts can always arise. This is because the researchers have a vested interest in the outcome. Of course you could argue that researchers always have a vested interest in the outcome of their research, as breakthroughs will advance reputation and facilitate additional funding (this could be another whole blog postingin itself). However the peer-review system generally does a reasonable job of picking up most of those who attempt to play the system. But when the vested interest is a large financial one, the pressures become greater. Government-funded researchers will generally only lose that one grant if a project does not work out. The decision about who to award the grant to next time is made again using peer-review and is regulated by committees and procedures which, however much the researchers rail against them, are there for a good reason. If all your money, on the other hand, comes from one or two companies who are focussed on being able to sell their particular drugs to the public in a few years’ time, there is a much greater pressure on you to create and publish positive results only. If the project yields a negative, your chances of receiving future funding from that source are very much reduced, being decided upon by a couple of top executives at that company who will tend to scratch the backs of those who scratch theirs.

Godlee had a few proposals to make to deal with this problem.

1) She wants article-level impact metrics rather than journal-level ones to become the norm. I think this is very sensible and could be applied easily, quickly, and would help a lot. Of course, it would not solve the problem on its own but it would give  a better guideline as to which are the good papers. I don’t think that it would prevent journals from publishing dross though, as some of the dross can still be highly cited.

2) She wants to stop big pharma from directly evaluating its own products (the current committee to approve drugs in the UK is made up of industry representatives) . Again this is sensible in theory though in practice I wonder if you would find that most of those with the necessary expertise to make the decisions are involved in some way with the pharmaceutical industry and there is no-one suitable who is not.

3) She wants to see central industry funding for independent advice and independent drugs trials. If you can get them to agree to to it, this is probably a good idea. Not sure why any company would sign up to this though, unless it is made law. This system is apparently in use in other countries, so I guess they must have laws about it. It would be interesting to see how these came about.

4) She wants to see publication of entire data sets to create better transparency. I’ve touched on my opinions about this before. This really only creates transparency if you are in a position to be able to use the data. It might solve some of the issues but it would create a load more when those who do not properly understand the data get hold of it and start to draw conclusions from it.

5) She would also like to see more investigative journalism. This is always a good thing, but how to promote it? Journalists are busy people, with ever shorter deadlines and ever more stories to write. To produce a good investigation they need to be given time and resources to do this, which means that they will be less productive in terms of numbers. Editors will not be prepared in most cases to countenance this unless it is strongly called for by the readers and subscribers. It is up to us!

The final point Godlee made was simple: The public must question everything. This is not a new idea – the motto of the Royal Society translates as “take nobody’s word for it”. But it is sadly a point that, in today’s society of quick headlines and instant gratification, probably does need making.

Explore posts in the same categories: Reviews

Tags: , , ,

You can comment below, or link to this permanent URL from your own site.

2 Comments on “Sense about Science Annual Lecture – Dr Fiona Godlee”

  1. Sean Haffey Says:

    It was refreshing to read someone else’s take on the same lecture I attended. You’d picked up some points I hadn’t.

    Thank you for the pointer to your blog.

  2. […] interest in the outcome of the research (I’ve touched on this in my review of Fiona Godlee’s Sense about Science talk). A way round it might be to make the testers independent of the developers. This is a system that […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: