class: center, middle, inverse, title-slide # <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M.15,184.42C-2.17,244.21,23,298.06,64,334.88V512H224V316.51L3.67,156.25A182.28,182.28,0,0,0,.15,184.42ZM509.22,275c-21-47.12-48.5-151.75-73.12-186.75A208.11,208.11,0,0,0,266.11,0H200C117,0,42.48,50.57,13.25,123.65L239.21,288H511.76A31.35,31.35,0,0,0,509.22,275ZM320,224a32,32,0,1,1,32-32A32.07,32.07,0,0,1,320,224Zm16,144H496l16-48H256V512H401.88a64,64,0,0,0,60.71-43.76L464,464H336a16,16,0,0,1,0-32H474.67l10.67-32H336a16,16,0,0,1,0-32Z"></path></svg> COVID-19 vaccine trials <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M201.5 174.8l55.7 55.8c3.1 3.1 3.1 8.2 0 11.3l-11.3 11.3c-3.1 3.1-8.2 3.1-11.3 0l-55.7-55.8-45.3 45.3 55.8 55.8c3.1 3.1 3.1 8.2 0 11.3l-11.3 11.3c-3.1 3.1-8.2 3.1-11.3 0L111 265.2l-26.4 26.4c-17.3 17.3-25.6 41.1-23 65.4l7.1 63.6L2.3 487c-3.1 3.1-3.1 8.2 0 11.3l11.3 11.3c3.1 3.1 8.2 3.1 11.3 0l66.3-66.3 63.6 7.1c23.9 2.6 47.9-5.4 65.4-23l181.9-181.9-135.7-135.7-64.9 65zm308.2-93.3L430.5 2.3c-3.1-3.1-8.2-3.1-11.3 0l-11.3 11.3c-3.1 3.1-3.1 8.2 0 11.3l28.3 28.3-45.3 45.3-56.6-56.6-17-17c-3.1-3.1-8.2-3.1-11.3 0l-33.9 33.9c-3.1 3.1-3.1 8.2 0 11.3l17 17L424.8 223l17 17c3.1 3.1 8.2 3.1 11.3 0l33.9-34c3.1-3.1 3.1-8.2 0-11.3l-73.5-73.5 45.3-45.3 28.3 28.3c3.1 3.1 8.2 3.1 11.3 0l11.3-11.3c3.1-3.2 3.1-8.2 0-11.4z"></path></svg> ## Evaluating <em>causal</em> vaccine efficacy ### Nima Hejazi<br> <span style="font-size: 75%;"> University of California, Berkeley<br> Division of Biostatistics, School of Public Health </span> ### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg> <a href="https://twitter.com/nshejazi" style="color: white;"><span class="citation">@nshejazi</span></a> <br> <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> <a href="https://nimahejazi.org" style="color: white;">nimahejazi.org</a> <br> <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M224,192a16,16,0,1,0,16,16A16,16,0,0,0,224,192ZM466.5,83.68l-192-80A57.4,57.4,0,0,0,256.05,0a57.4,57.4,0,0,0-18.46,3.67l-192,80A47.93,47.93,0,0,0,16,128C16,326.5,130.5,463.72,237.5,508.32a48.09,48.09,0,0,0,36.91,0C360.09,472.61,496,349.3,496,128A48,48,0,0,0,466.5,83.68ZM384,256H371.88c-28.51,0-42.79,34.47-22.63,54.63l8.58,8.57a16,16,0,1,1-22.63,22.63l-8.57-8.58C306.47,313.09,272,327.37,272,355.88V368a16,16,0,0,1-32,0V355.88c0-28.51-34.47-42.79-54.63-22.63l-8.57,8.58a16,16,0,0,1-22.63-22.63l8.58-8.57c20.16-20.16,5.88-54.63-22.63-54.63H128a16,16,0,0,1,0-32h12.12c28.51,0,42.79-34.47,22.63-54.63l-8.58-8.57a16,16,0,0,1,22.63-22.63l8.57,8.58c20.16,20.16,54.63,5.88,54.63-22.63V112a16,16,0,0,1,32,0v12.12c0,28.51,34.47,42.79,54.63,22.63l8.57-8.58a16,16,0,0,1,22.63,22.63l-8.58,8.57C329.09,189.53,343.37,224,371.88,224H384a16,16,0,0,1,0,32Zm-96,0a16,16,0,1,0,16,16A16,16,0,0,0,288,256Z"></path></svg> SER 2021 <svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M377 105L279.1 7c-4.5-4.5-10.6-7-17-7H256v128h128v-6.1c0-6.3-2.5-12.4-7-16.9zm-153 31V0H24C10.7 0 0 10.7 0 24v464c0 13.3 10.7 24 24 24h336c13.3 0 24-10.7 24-24V160H248c-13.2 0-24-10.8-24-24zm64 160v48c0 4.4-3.6 8-8 8h-56v56c0 4.4-3.6 8-8 8h-48c-4.4 0-8-3.6-8-8v-56h-56c-4.4 0-8-3.6-8-8v-48c0-4.4 3.6-8 8-8h56v-56c0-4.4 3.6-8 8-8h48c4.4 0 8 3.6 8 8v56h56c4.4 0 8 3.6 8 8z"></path></svg> <br> --- class: inverse, center, middle .huge[Vaccine Correlates <svg viewBox="0 0 480 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M477.7 186.1L309.5 18.3c-3.1-3.1-8.2-3.1-11.3 0l-34 33.9c-3.1 3.1-3.1 8.2 0 11.3l11.2 11.1L33 316.5c-38.8 38.7-45.1 102-9.4 143.5 20.6 24 49.5 36 78.4 35.9 26.4 0 52.8-10 72.9-30.1l246.3-245.7 11.2 11.1c3.1 3.1 8.2 3.1 11.3 0l34-33.9c3.1-3 3.1-8.1 0-11.2zM318 256H161l148-147.7 78.5 78.3L318 256z"></path></svg>] <style type="text/css"> .remark-slide-content { font-size: 22px } </style> --- ## Correlates of risk and protection Two, interrelated goals of vaccine correlates analyses are to * identify/validate possible __surrogate endpoints__\*; * understand __protective mechanisms__ of vaccines. If an __immune correlate__ is established to __reliably predict vaccine efficacy__, then subsequent efficacy trials may use it as the __primary endpoint__. -- __Accelerates approval__ of * existing vaccines in __different populations__ (e.g., in children); * __new vaccines__ within the same class. .footnote[[*] [Prentice (1989)](https://doi.org/10.1002/sim.4780080407) ] --- ## Correlates of risk and protection .large[__Two levels of correlates analysis__\*:] __Correlates of risk (CoR):__ * Correlation of immune response in vaccine recipients with outcome * Prediction of risk * Evaluated via _associative_ parameters -- __Correlates of protection (CoP)__: * Evaluate immune response's ability to predict vaccine efficacy (VE) * Evaluated via _causal_ parameters .footnote[[*] [Plotkin and Gilbert (2012)](https://doi.org/10.1093/cid/cis238), [Qin et al. (2007)](https://doi.org/10.1086/522428) ] --- ## Correlates of protection __Immune response in vaccine recipients__ * How does VE vary across subgroups defined by immune response? * e.g., [Juraska et al. (2020)](https://doi.org/10.1093/biostatistics/kxy074) -- __Path-specific mediators of vaccine efficacy__ * What percentage of VE is attributable to immune response? * [Cowling et al. (2019)](https://doi.org/10.1093/cid/ciy759) * [Benkeser et al. (2021+)](https://arxiv.org/abs/2103.02643) -- __Stochastic interventional vaccine efficacy__ * How would shifting the immune response distribution impact VE? * [Hejazi et al. (2020)](https://doi.org/10.1111/biom.13375), Hejazi et al. (2021+) --- ## Measuring correlates * Running assays on >30k samples is .red[expensive], __statistically unnecessary__. * Use a __case-cohort design\*__ to _measure immune responses_ in * a stratified random subcohort (\~1600 individuals) * all SARS-CoV-2 infection endpoints -- <img src="data:image/png;base64,#/home/nsh/git/conf_ser2021_txshift_covpn/img/casecohort.png" width="70%" style="display: block; margin: auto;" /> .footnote[[*] [Prentice (1986)](https://www.jstor.org/stable/2336266) ] --- ## Measuring correlates * Case-cohort designs are a special case of _two-phase sampling\*_: * Phase 1: measure baseline, randomization arm, endpoint on everyone. * Phase 2: given baseline, randomization arm, endpoint, select immune response subcohort members with (possibly known) probability. -- * Complete (.red[unobservable]) data unit `\(X = (L, A, S, Y) \sim P_0^X \in \mathcal{M}^X\)`: * `\(L\)` (covariates): sex, age, baseline risk/exposure score, * `\(A\)` (treatment): randomized placebo versus vaccine assignment, * `\(S\)` (exposure): candidate marker immune response profiles at Day 57, * `\(Y\)` (outcome): symptomatic SARS-CoV-2 infection (COVID) * Observed data unit `\(O = (C, C X) = (L, C, C S, Y)\)`, where `\(C \in \{0,1\}\)` indicates selection into the second phase sample. .footnote[[*] [Breslow et al. (2003)](https://doi.org/10.1214/aos/1059655907), [Breslow et al. (2009)](https://doi.org/10.1007/s12561-009-9001-6) ] ??? * In general, we work with _known_ two-phase sampling probabilities in this case-cohort setup, so no need for extra EIF component for efficiency. - SARS-CoV-2 infection is measured starting 7+ days after Day 57 to avoid any possibility of infection having occurred prior to second vaccine dose. --- ## Statistical challenges __Estimation in two-phase designs:__ * Individuals who contract COVID may __differ from other participants__. * Two-phase design .blue[_over-samples_] these individuals. * Augmented inverse weighting methods __account for differences__. -- __CoVPN statisticians are committed to open science:__ * A version-controlled statistical analysis plan (SAP) is [available](https://doi.org/10.6084/m9.figshare.13198595) for review. * An [open source GitHub repository](https://github.com/CoVPN/correlates_reporting) implements methods from the SAP. - Tools: `RMarkdown`, `renv`, `bookdown`, Make, GitHub, Travis CI, AWS. * Proof-of-concept for the validation of modern causal inference methodology in a regulatory context. ??? __Low case numbers due to highly effective vaccines (e.g., 95% VE):__ * Power for CoP analyses driven by __vaccine breakthroughs__. * CoR `\(>\)` 25 breakthroughs; CoP `\(>\)` 50 breakthroughs. --- class: inverse, center, middle .huge[Stochastic Interventional Vaccine Efficacy <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M201.5 174.8l55.7 55.8c3.1 3.1 3.1 8.2 0 11.3l-11.3 11.3c-3.1 3.1-8.2 3.1-11.3 0l-55.7-55.8-45.3 45.3 55.8 55.8c3.1 3.1 3.1 8.2 0 11.3l-11.3 11.3c-3.1 3.1-8.2 3.1-11.3 0L111 265.2l-26.4 26.4c-17.3 17.3-25.6 41.1-23 65.4l7.1 63.6L2.3 487c-3.1 3.1-3.1 8.2 0 11.3l11.3 11.3c3.1 3.1 8.2 3.1 11.3 0l66.3-66.3 63.6 7.1c23.9 2.6 47.9-5.4 65.4-23l181.9-181.9-135.7-135.7-64.9 65zm308.2-93.3L430.5 2.3c-3.1-3.1-8.2-3.1-11.3 0l-11.3 11.3c-3.1 3.1-3.1 8.2 0 11.3l28.3 28.3-45.3 45.3-56.6-56.6-17-17c-3.1-3.1-8.2-3.1-11.3 0l-33.9 33.9c-3.1 3.1-3.1 8.2 0 11.3l17 17L424.8 223l17 17c3.1 3.1 8.2 3.1 11.3 0l33.9-34c3.1-3.1 3.1-8.2 0-11.3l-73.5-73.5 45.3-45.3 28.3 28.3c3.1 3.1 8.2 3.1 11.3 0l11.3-11.3c3.1-3.2 3.1-8.2 0-11.4z"></path></svg>] --- ## Exposure-shifting interventions  --- ## Stochastic interventions __Target Causal Parameter:__ * _Stochastic interventions\*_ as modified exposure policies: `$$d(s,l) = s + \delta,$$` where `\(\delta := 0\)` if `\(s + \delta\)` is .red[not] plausible for a given individual. * Our estimand is `\(\psi_{0, d} := \mathbb{E}_{P_0^d}\{Y_{d(S,L)}\}\)`, mean of counterfactual `\(Y_{d(S, L)}\)`. -- __Efficient Estimation:__ * _Doubly robust_ one-step and targeted minimum loss (TML) estimators rooted in semiparametric efficiency theory. * Allow state-of-the-art ML for flexible nuisance parameter estimation. .footnote[[*] [Díaz & van der Laan (2012)]( https://doi.org/10.1111/j.1541-0420.2011.01685.x), [Díaz & van der Laan (2018)](https://doi.org/10.1007/978-3-319-65304-4_14), [Díaz & Hejazi (2020)](https://doi.org/10.1111/rssb.12362) ] ??? * _Stochastic interventions_ modify the value `\(S\)` would naturally assume by drawing from a modified exposure distribution. * Consider post-intervention value `\(S^{\star} \sim G^{\star}(\cdot \mid L)\)`; static interventions are a special case (degenerate distribution). * Such an intervention generates a counterfactual RV `\(Y_{G^{\star}} := f_Y(S^{\star}, L, U_Y)\)`, with distribution `\(P_0^{\delta}\)`. * Estimate `\(\psi_{0,\delta} := \mathbb{E}_{P_0^{\delta}} \{Y_{G^{\star}}\}\)`, the counterfactual mean under post-intervention distribution `\(G^{\star}\)`. --- ## Quantifying stochastic VE * Causal parameter based on vaccine efficacy (VE) estimands: $$ \text{SVE}(\delta) = 1 - \frac{\mathbb{E}[\mathbb{P}(Y = 1 \mid A = 1, S = s + \delta, L = l) \mid A = 1, L]}{\mathbb{P}(Y(0)=1)} $$ * `\(\mathbb{P}(Y(0)=1)\)`: counterfactual infection risk in the placebo arm. Under randomization, `\(\mathbb{P}(Y(0)=1) = \mathbb{P}(Y=1 \mid A=0)\)`. * Summarizes VE through stochastic shift interventions indexed by `\(\delta\)`. -- * _Inverse probability weighted augmentation\*_ to correct for sampling bias induced by two-phase, case-cohort design. * Open source `R` package\* at https://github.com/nhejazi/txshift * Analysis code at https://github.com/CoVPN/correlates_reporting .footnote[[*] [Rose & van der Laan (2011)](https://doi.org/10.2202/1557-4679.1217), [Hejazi et al. (2020)](https://doi.org/10.1111/biom.13375), [Hejazi & Benkeser (2020)](https://doi.org/10.21105/joss.02447) ] --- ## SVE: pseudo-neutralizing antibody <img src="data:image/png;base64,#/home/nsh/git/conf_ser2021_txshift_covpn/img/mcop_sve_Day57pseudoneutid80.png" width="77%" style="display: block; margin: auto;" /> --- ## SVE: spike protein binding antibody <img src="data:image/png;base64,#/home/nsh/git/conf_ser2021_txshift_covpn/img/mcop_sve_Day57bindSpike.png" width="77%" style="display: block; margin: auto;" /> --- ## Thank you! __Amazing statisticians:__ .pull-left[.tiny[ __Leadership__ * Dean Follmann (NIAID) * Yonghong Gao (BARDA) * .red[_Peter Gilbert (FHCRC, UW)_] __NIAID__ * Martha Nason * Mike Fay * All of NIAID Biostatistics __CoVPN__ * .red[_David Benkeser (Emory)_] * Marco Carone (UW) * Iván Díaz (Weill Cornell) * Alex Luedtke (UW) * ...many others! ]] .pull-right[.tiny[ __Fred Hutch__ * Youyi Fong * Holly Janes * Michal Juraska * Yunda Huang * Ying Huang * Ollivier Hyrien * ...many others! __OWS Company Statisticians__ * Way too many to name! <img src="data:image/png;base64,#./img/covpn.png" width="110"/> ]]