For more than a half a century, the traditional frequentist methodology — wherein predictions are based on a fixed target of estimation — has been entrenched in clinical development and regulatory statistics. Yet, all too often, drugs fail late, even in confirmatory clinical trials, at enormous cost to companies and ethical concerns for the patients, suggesting some shortcomings in these traditional methods.1
As pressure to reduce costs and improve regulatory decision making early in the process intensifies, companies have sought more efficient ways to analyse data and assess the safety and efficacy of drugs.
It turns out that one of the most effective tools to synthesise clinical trial data is far older than even the clinical trial process itself: Bayesian statistics. Interest within the pharmaceutical industry in applying Bayesian methods at various stages of research, development, manufacturing and health economics has been growing for the past 20 years because, based on observed data, it applies the logic of probability to statistical problems.2
Comparing statistical methods
Mathematical methods have long been used to assist with decision making in clinical research, with researchers often depending on the p-value, or observed significance level, to test whether something is statistically significant or not.
The point is to determine the significance of the results from a study in relation to the null hypothesis, which states there is no significant difference between two variables. If the data sample size is big enough, then the test statistic is roughly normal and can give you the p-value. However, if there isn’t a large sample of data, it becomes impossible to produce a reliable inference.
To make sense of the data, statisticians have to separate out the random noise and produce a more accurate picture of the underlying signal. Bayesian methods help to achieve this by borrowing strength from similar — but not identical — bits of information.
In Bayesian statistics, previous and related information is relevant. Past information – whether from previous trials or from scientific literature or real-world data – is considered as part of an ongoing stream of data “in which inferences are being updated each time new data become available.”3 In this way it allows researchers to achieve direct probability statements about unknown information, rather than settling for approximations.
Why Bayesian makes sense in pharma
What, though, does all this mean in clinical trials and drug development? As everyone in the industry knows all too well, the drug approval process is costly, complex and time-consuming.
During clinical trials, companies need to know whether their drug is safe and effective, but they also need to understand its likelihood of success.
This is when Bayesian methodology comes to the fore. It addresses the probability inference: what’s the probability that this new drug is safe and effective, and how likely is it that our current drug development programme will be successful?
In most development programmes, companies already have some information about a molecule or therapy from previous studies, either conducted by that company or by others. Rather than start from scratch, Bayesian statistics allows researchers to leverage this pre-existing information — including from scientific literature — to help determine the probability of success.
When a trial is conducted using Bayesian principles, initial estimates of probabilities are attributed to unknown quantities (the likelihood of a serious event, the likelihood the product will be effective for a given set of patients, etc.) using existing information (say, previous clinical trials) or expert opinion. These probabilities together constitute the prior distribution for the quantities of interest.
As long as those conducting the study construct the prior in an unbiased way (say, incorporating all existing knowledge, not merely that which is favourable to the company’s position), leveraging this information to support a study can dramatically improve study efficiency. It is also economically and ethically preferable to limit the number of in-human studies conducted whenever possible.
Regulators can sometimes be somewhat more rigorous when it comes to Bayesian analyses because they are less familiar with it than the traditional p-value approach. However, this tends to encourage careful, less automatic analyses that are typically very robust and more formally consider the impact of multiple different models and assumptions.
That is not to suggest that Bayesian methodologies are a replacement for p-values, which answer a fundamentally different question than Bayesian probabilities. “The p-value quantifies the discrepancy between the data and a null hypothesis of interest, usually the assumption of no difference or no effect. A Bayesian approach allows the calibration of p-values by transforming them to direct measures of the evidence against the null hypothesis or so-called Bayes factors.”4
For example, in a genomics experiment, researchers will put some of the drug or molecule into cells to assess the expression of different genes. The question in this case will be, is the inhibition or excitation of the genes likely linked to the treatment?
Here, researchers may legitimately be interested in the p-value; they want to know if the data they see are inconsistent with the hypothesis of no differential inhibition or excitation across genes. Here, p-values provide fairly straightforward yes-no answers because the very question is about the observed data.
However, p-values have a number of problems that limit their effectiveness … even when used correctly.
This was emphasised a few years ago by the American Statistical Association (ASA), which released an official “Statement on Statistical Significance and P-Values” and later held two conferences devoted to an investigation of their problems and potential remedies – many of them Bayesian.5
The ASA statement highlights various misconceptions about p-values, including the facts that they are not the probability that the null hypothesis is true or the probability that the data were obtained “by chance alone” (two very common though falsely held beliefs).
p-Values do not measure the size or importance of an observed effect and can only provide evidence against a hypothesis of no difference, not evidence for it. As such, p-values are not useful in proving the equivalence of two treatments.
When conducting a clinical trial or animal study to evaluate the efficacy of a treatment, the question is not about the data itself, but rather about the treatment: is the treatment effective? Is it safe? How likely is it that a trend emerging in the data will continue in future?
This is when Bayesian methodology has even greater usefulness. To predict a future situation, Bayesian statistics enable researchers to determine the probability of something occurring by first quantifying current uncertainty, and then propagating that into the future to get predictive probabilities. The question then is about the benefit for future patients in future trials or in the real world; that is, not merely the patients included in the past trials.
This current and future uncertainty is common in chemistry, manufacturing and controls (CMC) applications whereby companies need to be able to quantify what they know now about product characteristics or manufacturing processes and combine that with additional uncertainty about what will happen in the future.
It allows researchers to address the real questions of interest: is a process comparable with a previous one? What is the probability that a development approach is on target given the observed data? Bayesian statistics combines all that complicated and high-dimensional data and, using 21st century computing power and experts in mathematical probability theory, develops modelling to predict a likely future outcome.
Into the future with Bayes
The pressing needs of both companies and patients to improve the decision-making framework has led many biopharmaceutical companies to seek out statistical experts who specialise in Bayesian methodology. Many recognise the potential that Bayesian statistics present to address complex problems that arise across the product lifecycle — from the probability of success with clinical trials to managing CMC and supply chains to determine the best course of action with the product portfolio.
References
- www.tandfonline.com/doi/full/10.1080/00031305.2019.1566091.
- www.routledge.com/Bayesian-Methods-in-Pharmaceutical-Research/Lesaffre-Baio-Boulanger/p/book/9781138748484.
- www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials.
- www.annualreviews.org/doi/10.1146/annurev-statistics-031017-100307.
- www.amstat.org/asa/files/pdfs/p-valuestatement.pdf.