The purpose of this vignette is to introduce how to simulate and
analyze an adaptive Bayesian clinical trial for binomial outcomes. The
simulation section compromises the design of the trial itself which
provides type I error rate and power at each interim look. We use the
beta-binomial conjugate prior for the estimation of posterior
probabilities. Available historical data can be used as an informative
prior; we use the bayesDP package as the engine for
incorporating the historical data. By default, the model uses a
non-informative prior of Beta(a = 1, b = 1)
with or without the incorporation of historical data. Instead of using
traditional R function, we use pipes to input our parameters.
Let y and N denote the number of events and the sample size, respectively. Let a0 and b0 denote the rate parameters of a Beta distribution. Then, the posterior distribution of the event rate under vague (flat) prior distributions is
θ̃ ∣ y, N ∼ ℬeta(y + a0, N − y + b0).
When historical data is present, y0 and N0 denote the number of events and sample size of the historical data. The weight of the historical data included in the study design and analysis is denoted by α̂. For more details on the computation of α̂, please refer to the vignette of binomial counts available here. The posterior distribution of the event rate with historical data incorporated under vague (flat) prior distributions is
θ̃ ∣ y, N, y0, N0 ∼ ℬeta(y + y0α̂ + a0, N − y + α̂(N0 − y0) + b0),
Since there is no closed-form solution for the difference in beta distributed random variables, we use Monte Carlo simulations to estimate the posterior of the treatment difference.
The estimation of the difference in proportions is θ̃T − θ̃C, where θT is the posterior of the event rates in the treatment group and θC is the posterior of the event rates in the control group.
The following section lays out each of the functions and inputs for carrying out simulations and analyses of Bayesian adaptive trials.
Unlike traditional R functions, the bayesCT package
depends on pipe inputs with different wrapper functions. The key wrapper
functions are:
binomial_outcome: specifying event rates.study_details: specifying sample size, study length,
interim looks, and loss to follow-up.enrollment_rate: specifying enrollment.hypothesis: specifying the hypothesis for both
simulation and analysis.randomize: specifying the randomization scheme.impute: specifying imputation of outcomes at the
interim looks.beta_prior: specifying the non-informative prior.historical_binomial: specifying historical data, if
available. This function should only be used when historical data is
incorporated in the analysis or simulation. For more details on the
method and computation, please see https://CRAN.R-project.org/package=bayesDP.data_binomial: inputting data with a binomial
outcome.analysis: analyzing a trial at an interim
analysis.simulate: simulating trials. Note:
should only be used as the terminal end of the pipe.In the following section, we will discuss the design of adaptive
trials using bayesCT for binomial outcomes. We illustrate
an example for one-arm trial and two-arm trials using the wrapper
functions described above.
In the example below, we will illustrate how to compute power, type 1 error, and other characteristics for an objective performance criterion (OPC) (or performance goal; PG) trial with an observed proportion of events and hypothesis described as follows,
H0 : θtreatment ≥ 0.08 HA : θtreatment < 0.08.
The most important wrapper functions are study_details
and binomial_outcome (especially since there are no default
values).
Binomial events are simulated using an event rate of 0.08. The total sample size is 900 with a study length of 50 days. A 10% loss to follow-up as assumed. Based on this information, the adaptive trials are simulated 2 times to obtain the following output (Note:: for the purpose of generating the vignette quickly, we reduce the number of simulations to 5, you should use a much larger value, e.g., 10,000). The aforementioned inputs were chosen for illustration purposes only.
value <- binomial_outcome(p_treatment = 0.08) %>%
study_details(total_sample_size = 900,
study_period = 50,
interim_look = NULL,
prop_loss_to_followup = 0.10)
# Simulate 2 trials
output <- value %>%
simulate(no_of_sim = 2)
# Structure of the simulation output
str(output)
#> List of 8
#> $ input :List of 4
#> ..$ p_treatment : num 0.08
#> ..$ N_total : num 900
#> ..$ EndofStudy : num 50
#> ..$ prop_loss_to_followup: num 0.1
#> $ power :'data.frame': 1 obs. of 2 variables:
#> ..$ interim_looks: num 900
#> ..$ power : num 0
#> $ type1_error : num 0.5
#> $ est_final : num [1:2] 0.0813 0.0863
#> $ post_prob_accept_alternative: num [1:2] 0.542 0.727
#> $ N_enrolled : num [1:2] 900 900
#> $ stop_expect_success : num [1:2] 0 0
#> $ stop_futility : num [1:2] 0 0To allow for early stopping for success or futility, we can add interim looks to the design. We’ll check for success or futility at the enrollment of the 600-th, 700-th and 800-th subject. Upon adding this interim look requirement, the trial is simulated 2 times to obtain the output.
# Adding interim looks
value <- value %>%
study_details(total_sample_size = 900,
study_period = 50,
interim_look = c(600, 700, 800),
prop_loss_to_followup = 0.10)
# Simulate 2 trials
output <- value %>%
simulate(no_of_sim = 2)
str(output)
#> List of 8
#> $ input :List of 5
#> ..$ p_treatment : num 0.08
#> ..$ N_total : num 900
#> ..$ EndofStudy : num 50
#> ..$ prop_loss_to_followup: num 0.1
#> ..$ interim_look : num [1:3] 600 700 800
#> $ power :'data.frame': 4 obs. of 2 variables:
#> ..$ interim_looks: num [1:4] 600 700 800 900
#> ..$ power : num [1:4] 0 0 0 0
#> $ type1_error : num 0.5
#> $ est_final : num [1:2] 0.0845 0.073
#> $ post_prob_accept_alternative: num [1:2] 0.642 0.242
#> $ N_enrolled : num [1:2] 700 700
#> $ stop_expect_success : num [1:2] 0 0
#> $ stop_futility : num [1:2] 1 1Patient enrollment is assumed to follow a Poisson process. The default enrollment rate is 0.3 patients per day. In this simulation we’ll introduce a step-wise Poisson process with rate λ as follows:
$$ \lambda = \left\{ \begin{array}{ll} 0.3 & \text{time} \in [0, 25) \\ 1 & \text{time} \in [25, \infty) \\ \end{array} \right. $$
This enrollment scheme is illustrated below.
value <- value %>%
enrollment_rate(lambda = c(0.3, 1),
time = 25)
output <- value %>%
simulate(no_of_sim = 2)
str(output)
#> List of 8
#> $ input :List of 7
#> ..$ p_treatment : num 0.08
#> ..$ N_total : num 900
#> ..$ EndofStudy : num 50
#> ..$ prop_loss_to_followup: num 0.1
#> ..$ interim_look : num [1:3] 600 700 800
#> ..$ lambda : num [1:2] 0.3 1
#> ..$ lambda_time : num 25
#> $ power :'data.frame': 4 obs. of 2 variables:
#> ..$ interim_looks: num [1:4] 600 700 800 900
#> ..$ power : num [1:4] 0 0 0 0
#> $ type1_error : num 0
#> $ est_final : num [1:2] 0.0847 0.0758
#> $ post_prob_accept_alternative: num [1:2] 0.64 0.342
#> $ N_enrolled : num [1:2] 600 600
#> $ stop_expect_success : num [1:2] 0 0
#> $ stop_futility : num [1:2] 1 1The hypothesis is an important wrapper function which controls the probability of futility, probability of accepting the alternative hypothesis, probability of early success, the alternative hypothesis, and the treatment difference margin.
Since, in an OPC trial, the proportion of events in the treatment
group are simulated using the input provided, delta
controls the maximum threshold allowed for the trial to succeed/fail.
The default value of delta is 0. Here, we’ll use
delta = -0.03 (i.e. θ̂treatment − 0.08 > −0.03).
We’ll further set the futility probability to 0.05, the expected
success probability for early stopping to 0.90, and the final
probability of accepting the alternative to 0.95. The alternative is
"less" due to the hypothesis function specified above.
value <- value %>%
hypothesis(delta = -0.03,
futility_prob = 0.05,
prob_accept_ha = 0.95,
expected_success_prob = 0.90,
alternative = "less")
output <- value %>%
simulate(no_of_sim = 2)Next, we’ll illustrate imputations for imputing outcomes for subjects loss to follow up. We’ll carry out 5 imputations and draw 1000 values from the posterior of each imputation.
value <- value %>%
impute(no_of_impute = 5,
number_mcmc = 1000)
output <- value %>%
simulate(no_of_sim = 10)The default non-informative beta prior used in the simulation is ℬeta(1, 1). In our OPC trial simulation, we’ll change the default to ℬeta(5, 5). This will increase the weight of the non-informative prior in the simulation. This non-informative beta prior is implemented using beta_prior wrapper function.
Historical data is not required to compute the simulation. However,
if historical data is available, it can be incorporated into the
analysis using the discount prior approach as implemented in the
bayesDP R package.
In our OPC trial, we’ll illustrate historical data incorporation. We’ll assume that the historical data had 5 events in 55 subjects. We’ll incorporate this historical data using the identity discount function.
For more details on the historical data incorporation method and computation, please see https://CRAN.R-project.org/package=bayesDP.
value <- value %>%
historical_binomial(y0_treatment = 5,
N0_treatment = 55,
discount_function = "identity",
y0_control = NULL,
N0_control = NULL,
alpha_max = 1,
fix_alpha = FALSE,
weibull_scale = 0.135,
weibull_shape = 3,
method = "fixed")
output <- value %>%
simulate(no_of_sim = 2)The above flow was for illustrative purposes. Instead of inputting parameters step by step, the trial parameters can be filled in all at once as illustrated below. The pipe function connects all inputs together and the trial is simulated 2 times to obtain results.
value <- binomial_outcome(p_treatment = 0.08) %>%
enrollment_rate(lambda = c(0.3, 1),
time = 25) %>%
study_details(total_sample_size = 900,
study_period = 50,
interim_look = c(600, 700, 800),
prop_loss_to_followup = 0.10) %>%
hypothesis(delta = -0.03,
futility_prob = 0.05,
prob_accept_ha = 0.95,
expected_success_prob = 0.90,
alternative = "less") %>%
impute(no_of_impute = 25,
number_mcmc = 1000) %>%
beta_prior(a0 = 5,
b0 = 5) %>%
historical_binomial(y0_treatment = 5,
N0_treatment = 55,
discount_function = "identity",
y0_control = NULL,
N0_control = NULL,
alpha_max = 1,
fix_alpha = FALSE,
weibull_scale = 0.135,
weibull_shape = 3,
method = "fixed") %>%
simulate(no_of_sim = 2)
str(value)
#> List of 8
#> $ input :List of 23
#> ..$ p_treatment : num 0.08
#> ..$ lambda : num [1:2] 0.3 1
#> ..$ lambda_time : num 25
#> ..$ N_total : num 900
#> ..$ EndofStudy : num 50
#> ..$ interim_look : num [1:3] 600 700 800
#> ..$ prop_loss_to_followup: num 0.1
#> ..$ h0 : num -0.03
#> ..$ futility_prob : num 0.05
#> ..$ prob_ha : num 0.95
#> ..$ expected_success_prob: num 0.9
#> ..$ alternative : chr "less"
#> ..$ N_impute : num 25
#> ..$ number_mcmc : num 1000
#> ..$ prior : num [1:2] 5 5
#> ..$ y0_treatment : num 5
#> ..$ N0_treatment : num 55
#> ..$ discount_function : chr "identity"
#> ..$ alpha_max : num 1
#> ..$ fix_alpha : logi FALSE
#> ..$ weibull_scale : num 0.135
#> ..$ weibull_shape : num 3
#> ..$ method : chr "fixed"
#> $ power :'data.frame': 4 obs. of 2 variables:
#> ..$ interim_looks: num [1:4] 600 700 800 900
#> ..$ power : num [1:4] 0 0.5 1 1
#> $ type1_error : num 0
#> $ est_final : num [1:2] 0.0861 0.0824
#> $ post_prob_accept_alternative: num [1:2] 0.988 0.991
#> $ N_enrolled : num [1:2] 800 700
#> $ stop_expect_success : num [1:2] 1 1
#> $ stop_futility : num [1:2] 0 0In this section, we will illustrate how to perform the design of a two-arm trial without incorporating historical data. The example will compute the type 1 error, power, and other outputs for a superiority trial. The study hypothesis is
H0 : θtreatment − θcontrol ≤ 0 HA : θtreatment − θcontrol > 0.
The binomial events are simulated using an event rate of 0.15 for the treatment group and 0.12 for the control group. The total sample size is 400, with a study length of 30 days. A 15% loss to follow up is assumed. Further, we will illustrate block randomization. The following code simulates a trial 2 times using the piping procedure.
value <- binomial_outcome(p_treatment = 0.15,
p_control = 0.12) %>%
study_details(total_sample_size = 400,
study_period = 30,
interim_look = 350,
prop_loss_to_followup = 0.15) %>%
hypothesis(delta = 0,
futility_prob = 0.10,
prob_accept_ha = 0.975,
expected_success_prob = 1,
alternative = "greater") %>%
randomize(block_size = 9,
randomization_ratio = c(4, 5)) %>%
impute(no_of_impute = 5,
number_mcmc = 5000) %>%
beta_prior(a0 = 0,
b0 = 0) %>%
simulate(no_of_sim = 2)In this section, we will demonstrate how to run an adaptive Bayesian
trial using bayesCT. A sample dataset is provided in the
package. The dataset binomialdata contains the results of
300 subjects from a two-arm trial with binomial outcome. The
complete column indicates whether the outcome was observed,
i.e. loss to follow-up.
data(binomialdata)
head(binomialdata)
#> id treatment outcome complete
#> 1 Patient1 1 0 0
#> 2 Patient2 0 0 1
#> 3 Patient3 1 0 1
#> 4 Patient4 0 0 1
#> 5 Patient5 0 1 1
#> 6 Patient6 0 0 1The minimum input needed to run an adaptive Bayesian trial is the
data itself. The data_binomial input allows the input of
the data. The treatment group (0 for control, 1 for treatment) and
outcome input are essential for the analysis. However, if the complete
input is not provided, the function assumes the outcome data is
complete. A default analysis is carried out below.
input <- data_binomial(treatment = binomialdata$treatment,
outcome = binomialdata$outcome,
complete = binomialdata$complete)
out <- input %>%
analysis(type = "binomial")
#> Are you at the final sample size? There is no opportunity to stop enrollment.
#>
#> Consider setting N_max_treatment and/or N_max_controlWe’ll now illustrate using piping to carry out the complete analysis. First, we’ll assume the following hypothesis:
H0 : θtreatment − θcontrol ≤ 0.02 HA : θtreatment − θcontrol > 0.02
The delta and alternative used to analyze the trial is 0.02 and “greater” respectively. The probability of accepting the alternative is 0.95, the probability of stopping for futility is 0.05, and the probability of stopping for success is 0.90. We will carry out imputations on subjects loss to follow up. Additionally, we will incorporate historical data on the treatment arm.
out <- data_binomial(treatment = binomialdata$treatment,
outcome = binomialdata$outcome,
complete = binomialdata$complete) %>%
hypothesis(delta = 0.02,
futility_prob = 0.05,
prob_accept_ha = 0.95,
expected_success_prob = 0.90,
alternative = "greater") %>%
impute(no_of_impute = 50,
number_mcmc = 10000) %>%
beta_prior(a0 = 3,
b0 = 3) %>%
historical_binomial(y0_treatment = 12,
N0_treatment = 100,
y0_control = NULL,
N0_control = NULL,
discount_function = "weibull",
alpha_max = 1,
fix_alpha = FALSE,
weibull_scale = 0.135,
weibull_shape = 3,
method = "fixed") %>%
analysis(type = "binomial")
#> Are you at the final sample size? There is no opportunity to stop enrollment.
#>
#> Consider setting N_max_treatment and/or N_max_control
str(out)
#> List of 13
#> $ prob_of_accepting_alternative: num 0.95
#> $ margin : num 0.02
#> $ alternative : chr "greater"
#> $ N_treatment : num 125
#> $ N_control : int 137
#> $ N_complete : num 262
#> $ N_enrolled : int 300
#> $ N_max_treatment : int 140
#> $ N_max_control : int 160
#> $ post_prob_accept_alternative : num 0.982
#> $ est_final : num 0.12
#> $ stop_futility : num 0
#> $ stop_expected_success : num 1