class: inverse, center, title-slide, middle # Designing early phase clinical trials with the {ppseq} package <br> ### Emily C. Zabor <p align="center"><img src="images/CC_hires_r.png" width=30%></p> <br> <br> #### R/Medicine #### August 27, 2021 --- class: inverse, center, middle # Motivation --- # Traditional oncology trial design <p align="center"><img src="images/phase1-traditional.jpg" width=60%></p> .large[ - Focus on **cytotoxic** treatments - Primary aim of identifying the **maximum tolerated dose** (MTD) - Use of the 3+3 design or model-based design ] <br> .small[Image source: Theoret MR, Pai-Scherf LH, Chuk MK, Prowell TM, Balasubramaniam S, Kim T, Kim G, Kluetz PG, Keegan P, Pazdur R. Expansion Cohorts in First-in-Human Solid Tumor Oncology Trials. Clin Cancer Res. 2015 Oct 15;21(20):4545-51.] --- # Seamless oncology trial design <p align="center"><img src="images/phase1-seamless.jpg" width=60%></p> .large[ - Focus shifting to **non-cytotoxic** treatments - MTD may not exist or be relevant - **Dose-expansion** cohorts becoming increasingly common ] <br> .small[Image source: Theoret MR, Pai-Scherf LH, Chuk MK, Prowell TM, Balasubramaniam S, Kim T, Kim G, Kluetz PG, Keegan P, Pazdur R. Expansion Cohorts in First-in-Human Solid Tumor Oncology Trials. Clin Cancer Res. 2015 Oct 15;21(20):4545-51.] --- # Dose-expansion in KEYNOTE-001 <p align="center"><img src="images/keynote001-cropped.jpg" width=70%></p> .small[Image source: Kang SP, Gergich K, Lubiniecki GM, de Alwis DP, Chen C, Tice MAB, Rubin EH. Pembrolizumab KEYNOTE-001: an adaptive study leading to accelerated approval for two indications and a companion diagnostic. Ann Oncol. 2017 Jun 1;28(6):1388-1398.] --- # {ppseq} solution: predictive probability monitoring <p align="center"><img src="images/sequential-design.jpg" width=90%></p> .large[ - Done **sequentially** to allow for early stopping - Bayesian approach allows for **flexibility** in number and timing of looks - Can maintain traditional levels of **type I error** and **power** ] <br> .small[Icon source: https://www.vecteezy.com/free-vector/human] --- class: inverse, center, middle # Predictive probability --- # Trial design and data <p align="center"><img src="images/response.jpg" width=50%></p> .large[ - `\(n\)` patients at interim analysis, maximum of `\(N\)` total enrolled patients - `\(H_0: p \leq p_0\)` versus `\(H_1: p \geq p_1\)` ] --- # Posterior probability .large[ Prior: `\(\pi(p) \sim Beta(a_0, b_0)\)` Data: `\(X \sim binomial(n, p)\)` Posterior: `\(p|x \sim Beta(a_0 + x, b_0 + n - x)\)` - Posterior = probability of success based on data **accrued so far** - Success if `\(Pr(p > p_0 | X)>\theta\)` for **posterior threshold `\(\theta\)`** ] --- # Posterior predictive probability .large[ Probability that the treatment will be declared efficacious at the **end of the trial**: .center[ `\(PPP = \sum_{x^* = 0}^{n^*} Pr(X^* = x^* | x) \times I(Pr(p > p_0 | X, X^* = x^*) > \theta)\)` ] Stop the trial early for **futility** if: .center[ `\(PPP < \theta^*\)` for predictive threshold `\(\theta^*\)` ] ] <br> <p align="center"><img src="images/pp-scale.jpg" width=60%></p> --- class: inverse, center, middle # Package overview and example --- # Installing {ppseq} .large[ **Development** version from GitHub: `remotes::install_github("zabore/ppseq")` **.green[Coming soon!]** **Production** version from CRAN: `install.packages("ppseq")` ] --- # Atezolizumab for mUC <p align="center"><img src="images/atezo-headlines.png" width=90%></p> .small[ https://www.cancernetwork.com/view/atezolizumab-indication-in-us-withdrawn-for-previously-treated-metastatic-urothelial-cancer https://www.onclive.com/view/roche-withdraws-atezolizumab-prior-platinum-treated-metastatic-bladder-cancer-indication-in-the-united-states https://ascopost.com/news/march-2021/atezolizumab-s-indication-in-previously-treated-metastatic-bladder-cancer-is-withdrawn/ ] --- # Original dose-expansion design .large[ **Primary aim**: evaluate safety, pharmacodynamics, pharmacokinetics * Expansion cohort in mUC not originally planned * Added through **protocol amendment** * Enrolled **95 patients** * Single **futility look**: stop if 0 responses in first 14 patients - At most 4.4% chance of no responses if true rate is 20% or higher ] <br> <br> .small[ Powles T, Eder JP, Fine GD, Braiteh FS, Loriot Y, Cruz C, Bellmunt J, Burris HA, Petrylak DP, Teng SL, Shen X, Boyd Z, Hegde PS, Chen DS, Vogelzang NJ. MPDL3280A (anti-PD-L1) treatment leads to clinical activity in metastatic bladder cancer. Nature. 2014 Nov 27;515(7528):558-62 ] --- # Re-design of dose-expansion study .large[ * Assume **null** rate of 0.1 and **alternative** rate of 0.2 * Maximum sample size **N = 95** * Check for futility after **every 5 patients** * **Posterior thresholds**: 0, 0.7, 0.74, 0.78, 0.82, 0.86, 0.9, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 0.999, 0.9999, 0.99999, and 1 * **Predictive thresholds**: 0.05, 0.1, 0.15, and 0.2 ] --- # Determine type I error and power <p align="center"><img src="images/power-type1.jpg" width=60%></p> .large[ - Need to determine **type I error** and **power** of decision thresholds - Use `calibrate_thresholds()` to evaluate a grid of thresholds - Jointly calibrates **posterior** and **predictive** thresholds ] --- # calibrate_thresholds() ```r library(ppseq) ``` ```r one_sample_cal_tbl <- calibrate_thresholds( p_null = 0.1, p_alt = 0.2, n = seq(5, 95, 5), N = 95, pp_threshold = c(0, 0.7, 0.74, 0.78, 0.82, 0.86, 0.9, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 0.999, 0.9999, 0.99999, 1), ppp_threshold = seq(0.05, 0.2, 0.05), direction = "greater", delta = NULL, prior = c(0.5, 0.5), S = 5000, nsim = 1000 ) ``` --- # print() calibrate_thresholds() results ```r print(one_sample_cal_tbl, type1_range = c(0.05, 0.1), minimum_power = 0.7) ``` .scroll-output[
pp_threshold
ppp_threshold
mean_n1_null
prop_pos_null
prop_stopped_null
mean_n1_alt
prop_pos_alt
prop_stopped_alt
0.82
0.10
43.080
0.096
0.851
83.110
0.828
0.161
0.82
0.15
39.085
0.080
0.905
81.570
0.805
0.190
0.82
0.20
35.255
0.075
0.913
79.535
0.783
0.213
0.86
0.10
42.590
0.093
0.857
83.240
0.829
0.160
0.86
0.15
39.010
0.081
0.904
81.515
0.804
0.191
0.86
0.20
35.020
0.074
0.914
79.580
0.784
0.212
0.90
0.05
50.975
0.072
0.879
90.160
0.883
0.096
0.90
0.10
38.625
0.061
0.905
81.515
0.795
0.191
0.90
0.15
34.885
0.057
0.931
79.670
0.770
0.224
0.92
0.05
51.010
0.072
0.878
90.080
0.881
0.098
0.92
0.10
38.495
0.060
0.908
81.555
0.796
0.190
0.92
0.15
34.780
0.056
0.930
79.945
0.772
0.220
0.93
0.05
50.105
0.050
0.880
89.880
0.839
0.100
0.94
0.05
46.520
0.050
0.920
88.815
0.826
0.132
0.95
0.05
46.275
0.050
0.920
88.585
0.823
0.135
] --- # optimize_design() ```r optimize_design( one_sample_cal_tbl, type1_range = c(0.05, 0.1), minimum_power = 0.7 ) ``` **Optimal accuracy design:**
pp_threshold
ppp_threshold
Type I error
Power
Average N under the null
Average N under the alternative
0.9
0.05
0.072
0.883
50.975
90.16
**Optimal efficiency design:**
pp_threshold
ppp_threshold
Type I error
Power
Average N under the null
Average N under the alternative
0.92
0.1
0.06
0.796
38.495
81.555
--- # plot() ```r plot(one_sample_cal_tbl, type1_range = c(0.01, 0.2), minimum_power = 0.7, plotly = TRUE) ``` .pull-left[ **Accuracy:** .center[
] ] .pull-right[ **Efficiency:** .center[
] ] --- # Selected optimal design .large[ .pull-left[ **Optimal efficiency design** - 39 patients under the null on average - Type I error of 0.06 - Power of 79.6% ] .pull-left[ **Original design** - 76 patients under the null on average - Type I error of 0.005 - Power of 52.8% ] ] --- # calc_decision_rules() ```r one_sample_decision_tbl <- calc_decision_rules( n = seq(5, 95, 5), N = 95, theta = 0.92, ppp = 0.1, p0 = 0.1, direction = "greater", delta = NULL, prior = c(0.5, 0.5), S = 5000 ) ``` --- # calc_decision_rules() ```r one_sample_decision_tbl ``` .scroll-output[
n
r
ppp
5
NA
NA
10
0
0.0634
15
0
0.0220
20
1
0.0844
25
1
0.0326
30
2
0.0702
35
2
0.0288
40
3
0.0536
45
4
0.0708
50
4
0.0344
55
5
0.0478
60
6
0.0622
65
7
0.0888
70
8
0.0950
75
8
0.0330
80
9
0.0326
85
10
0.0346
90
11
0.0162
95
13
0.0000
] --- # plot() ```r plot(one_sample_decision_tbl) ```
--- # Additional features .large[ * All demonstrated functions also available for **two-sample case** - i.e. Experimental vs standard-of-care treatment arms * `calibrate_thresholds()` is parallelized using the {future} and {furrr} packages - Set up a call to `future::plan()` appropriate for your system - Example: `future::plan(future::multicore(workers = 76))` ] --- # Thank you! <br> .xlarge[ **{ppseq} authors:** Emily C. Zabor, Brian P. Hobbs, and Michael J. Kane **{ppseq} website:** [http://www.emilyzabor.com/ppseq/](http://www.emilyzabor.com/ppseq/) **Slides:** [www.emilyzabor.com/rmedicine2021/](www.emilyzabor.com/rmedicine2021/) **Contact:** <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg> @zabormetrics, <svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> @zabore, <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg"> <path d="M464 64H48C21.49 64 0 85.49 0 112v288c0 26.51 21.49 48 48 48h416c26.51 0 48-21.49 48-48V112c0-26.51-21.49-48-48-48zm0 48v40.805c-22.422 18.259-58.168 46.651-134.587 106.49-16.841 13.247-50.201 45.072-73.413 44.701-23.208.375-56.579-31.459-73.413-44.701C106.18 199.465 70.425 171.067 48 152.805V112h416zM48 400V214.398c22.914 18.251 55.409 43.862 104.938 82.646 21.857 17.205 60.134 55.186 103.062 54.955 42.717.231 80.509-37.199 103.053-54.947 49.528-38.783 82.032-64.401 104.947-82.653V400H48z"></path></svg> zabore2@ccf.org ]