The test-negative design for the estimation of COVID-19 vaccine effectiveness: design evaluation and development of statistical methods in the evolving context

Fast research designs have been proposed for estimating how well different vaccines protect against disease, severe disease, hospitalization, and death from COVID-19. In fact, ongoing study is needed to evaluate different levels of vaccination (2 doses, boosters, different lags between doses, etc) in terms of how well they protect against illness, which may vary depending on the current circulation of virus variants. These fast designs typically involve identifying people who have been tested for COVID-19, often at a test-site or in a hospital. An established design is called the "test-negative design" which specifically involves identifying people who have symptoms associated with the disease in question and who then get tested. Scientists can estimate vaccine effectiveness by comparing people who test positive to people who test negative. If the negatives have higher rates of vaccination, this will indicate effectiveness of the vaccine. But, depending on how the design and statistical methods are applied, there may be bias in the estimation of effectiveness. Our research team has recently noted that, because of challenges of how test data are collected in our healthcare systems across Canada, the classical version of the test-negative design cannot always be applied. For example, some designs have used all test data, rather than only data from those who have certain symptoms. We are interested in evaluating how much bias can be caused by this difference in design, and identifying scenarios in which this can create misleading results. Secondly, we are interested in developing statistical methods that can address the limitations of the regression approach that is essentially the only one currently being used. We will identify limitations of current methods and propose new (or adapted) methods that can address these limitations. Our goal is to produce more reliable statistical methods so that we can improve our monitoring of the benefits of vaccination.