Better method reporting could help stop commissioning of trials that have no hope of success
Better method reporting in animal experiments could save hundreds of thousands of pounds and stop clinical trials being commissioned that have no hope of a successful result, according to researchers at The University of Manchester.
The team led by Dr Sheena Cruickshank of the Faculty of Life Sciences and Professor Andy Brass from the School of Computer Science analysed 58 papers on research into inflammatory bowel disease published between 2000 and 2014. They found a wide variety in how methods were reported and that vital information about experiments were missing, meaning they couldn’t be accurately reproduced in animal or human models.
In several instances the gender of the animal used wasn’t recorded which can have a bearing on the result as female mice have a stronger immune response to males. How the animals were housed will also impact on the results in experiments about the gut.
'What our research has uncovered is that this lack of data makes it difficult to validate the experiment and the result,' said Dr Cruickshank. 'Crucially this is having an impact of the reproducibility of experiments, both in the animal model and when transferred to human trials.'
The problem first became clear to the researchers when they were looking at building a knowledge base to help pull together research on inflammatory bowel disease (colitis). Working with computer scientists it became clear to the biologists that the data couldn’t be understood by colleagues from different disciplines.
'So much research is now being carried out across disciplines so it’s vital that experiments can be understood by as many scientists as possible,' Professor Brass explains. 'Relying on the reader to make assumptions based on their own experience will only lead to errors, but this is what many papers are asking people to do when they come to replicate the research.
To address the issue the team developed a critical checklist of what information should be included. It covers nine areas ranging from information about the animals, their housing condition, genetics, how colitis is induced, experiment design and monitoring. The checklist is included in their paper due to be published in the journal Inflammatory Bowel Diseases.1
'Our checklist sounds like fairly basic information that should be in all papers. But over the past few years journals have asked for more and more abbreviated methods so information has stopped being included,' says Dr Cruikshank. 'Instead papers are focused on the results and discussion and sometimes you have to go back to a paper from the sixties to find the last time a particular method was accurately recorded.'
While the researchers have been critical of the method reporting they emphasise that their criticism doesn’t mean the research isn’t valid or accurate. The experiments may well be sound, but the lack of detail in how they’re reported makes that judgement more difficult to make and the reproducibility much harder.
Moving forward the Manchester team is recommending the adoption of their checklist as a requirement for publication to improve the quality, comparability and standardisation of studies into inflammatory bowel disease. They believe it will make the interpretation and translation of data to human disease more reliable and ultimately contribute to making clinical trials more successful.
1. The paper 'Quality of Methods Reporting in Animal Models of Colitis' will be published in the journal Inflammatory Bowel Diseases. DOI 10.1097/MIB.0000000000000369