BY-COVID - WP5 - Baseline Use Case: COVID-19 vaccine effectiveness assessment

Survival analysis

Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted.


Survival (time-to-event)

The probability of not getting infected by SARS-CoV-2 beyond a certain time after onset of follow-up (survival function, estimated using the Kaplan-Meier estimator) is reported for different periods.

Strata Time Number at risk Cumulative sum of number of events Cumulative sum of number censored Survival Std. error Cumulative hazard Std. error cumulative hazard
Not fully vaccinated 0 7757 0 0 1.0000 0.00000 0.0000 0.00000
Not fully vaccinated 100 2632 66 5080 0.9849 0.00197 0.0152 0.00200
Not fully vaccinated 200 884 199 6692 0.9089 0.00683 0.0954 0.00751
Not fully vaccinated 300 597 312 6848 0.7669 0.01359 0.2651 0.01769
Not fully vaccinated 400 475 333 6961 0.7398 0.01433 0.3009 0.01934
Not fully vaccinated 500 93 335 7332 0.7312 0.01540 0.3126 0.02103
Fully vaccinated 0 7757 0 0 1.0000 0.00000 0.0000 0.00000
Fully vaccinated 100 2639 65 5073 0.9844 0.00202 0.0158 0.00205
Fully vaccinated 200 863 242 6671 0.8821 0.00782 0.1254 0.00886
Fully vaccinated 300 611 329 6817 0.7745 0.01282 0.2552 0.01653
Fully vaccinated 400 495 344 6926 0.7553 0.01343 0.2803 0.01776
Fully vaccinated 500 100 344 7317 0.7553 0.01343 0.2803 0.01776


Median survival time

The median survival time is the time corresponding to a probability of not obtaining a SARS-CoV-2 infection probability of 0.5. (if NA, the probability of not obtaining a SARS-CoV-2 infection did not drop below 50%)

Characteristic Median survival (95% CI)
fully_vaccinated_bl
    FALSE — (—, —)
    TRUE — (—, —)
Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate). The Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate and accounts for clustering within individuals (as one individual can be re-sampled as control).

A hazard ratio (HR) is computed for the covariate ‘fully_vaccinated_bl’. A hazard can be interpreted as the instantaneous rate of SARS-CoV-2 infections in individuals that are at risk for obtaining an infection (Cox proportional hazards regression assumes stable proportional hazards over time). A HR < 1 indicates reduced hazard of SARS-CoV-2 infection when having completed a primary vaccination schedule whereas a HR > 1 indicates an increased hazard of SARS-CoV-2 infection.

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.029 0.077 0.1 0.772 1.029 (0.828, 1.23)

The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 0.14113991 1 0.7071504
Wald test 0.08000000 1 0.7723077
Score (logrank) test 0.14113660 1 0.7071537
Robust score test 0.08457912 1 0.7711852

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Arm Measure Estimate SE CI.lower CI.upper
fully_vaccinated_bl==FALSE RMST 326.426 2.019 322.470 330.382
fully_vaccinated_bl==TRUE RMST 325.207 2.040 321.208 329.205
fully_vaccinated_bl==FALSE RMTL 38.574 2.019 34.618 42.530
fully_vaccinated_bl==TRUE RMTL 39.793 2.040 35.795 43.792
Measure Estimate CI.lower CI.upper p_value
RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -1.219 -6.844 4.405 0.671
RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.996 0.979 1.014 0.671
RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.032 0.894 1.191 0.671
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the vaccination schedule.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the vaccination schedule.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the administered vaccination schedule. A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘vaccination_schedule_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.152 0.219 0.227 0.503 1.164 (0.646, 1.682)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-BP -0.128 0.238 0.235 0.585 0.88 (0.474, 1.285)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ -0.301 0.412 0.414 0.467 0.74 (0.139, 1.341)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)MD-MD -0.142 0.296 0.289 0.622 0.867 (0.377, 1.358)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 0.7325156 4 0.9472594
Wald test 0.6700000 4 0.9549868
Score (logrank) test 0.7318399 4 0.9473451
Robust score test 0.6771403 4 0.9541178

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Vaccination_schedule Arm Measure Estimate SE CI.lower CI.upper
BP-BP fully_vaccinated_bl==FALSE RMST 326.655 2.428 321.897 331.414
BP-BP fully_vaccinated_bl==TRUE RMST 326.977 2.401 322.271 331.683
BP-BP fully_vaccinated_bl==FALSE RMTL 38.345 2.428 33.586 43.103
BP-BP fully_vaccinated_bl==TRUE RMTL 38.023 2.401 33.317 42.729
MD-MD fully_vaccinated_bl==FALSE RMST 324.622 5.470 313.901 335.343
MD-MD fully_vaccinated_bl==TRUE RMST 321.345 5.629 310.314 332.377
MD-MD fully_vaccinated_bl==FALSE RMTL 40.378 5.470 29.657 51.099
MD-MD fully_vaccinated_bl==TRUE RMTL 43.655 5.629 32.623 54.686
AZ-AZ fully_vaccinated_bl==FALSE RMST 326.690 5.802 315.319 338.062
AZ-AZ fully_vaccinated_bl==TRUE RMST 317.238 6.492 304.514 329.961
AZ-AZ fully_vaccinated_bl==FALSE RMTL 38.310 5.802 26.938 49.681
AZ-AZ fully_vaccinated_bl==TRUE RMTL 47.762 6.492 35.039 60.486
JJ fully_vaccinated_bl==FALSE RMST 326.314 9.174 308.333 344.294
JJ fully_vaccinated_bl==TRUE RMST 328.129 9.580 309.353 346.905
JJ fully_vaccinated_bl==FALSE RMTL 38.686 9.174 20.706 56.667
JJ fully_vaccinated_bl==TRUE RMTL 36.871 9.580 18.095 55.647
Vaccination_schedule Measure Estimate CI.lower CI.upper p_value
BP-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 0.322 -6.371 7.014 0.925
BP-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.001 0.981 1.022 0.925
BP-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.992 0.832 1.182 0.925
MD-MD RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.277 -18.660 12.107 0.676
MD-MD RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.990 0.944 1.038 0.676
MD-MD RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.081 0.749 1.560 0.677
AZ-AZ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -9.453 -26.518 7.612 0.278
AZ-AZ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.971 0.921 1.024 0.279
AZ-AZ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.247 0.837 1.858 0.278
JJ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 1.815 -24.181 27.812 0.891
JJ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.006 0.929 1.089 0.891
JJ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.953 0.478 1.899 0.891
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the NUTS3 residence area.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the NUTS3 residence area.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the area of residence (NUTS3 level). A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘residence_area_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.167 0.199 0.235 0.479 1.181 (0.636, 1.726)
fully_vaccinated_blTRUE:strata(residence_area_cd)732 -0.048 0.283 0.343 0.888 0.953 (0.312, 1.594)
fully_vaccinated_blTRUE:strata(residence_area_cd)733 -0.185 0.219 0.265 0.483 0.831 (0.4, 1.262)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 1.0947216 3 0.7783485
Wald test 0.7500000 3 0.8611512
Score (logrank) test 1.0937299 3 0.7785879
Robust score test 0.7764369 3 0.8550943

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Residence_area Arm Measure Estimate SE CI.lower CI.upper
733 fully_vaccinated_bl==FALSE RMST 326.757 2.357 322.138 331.377
733 fully_vaccinated_bl==TRUE RMST 326.919 2.371 322.273 331.565
733 fully_vaccinated_bl==FALSE RMTL 38.243 2.357 33.623 42.862
733 fully_vaccinated_bl==TRUE RMTL 38.081 2.371 33.435 42.727
731 fully_vaccinated_bl==FALSE RMST 327.062 5.280 316.714 337.410
731 fully_vaccinated_bl==TRUE RMST 320.789 5.565 309.882 331.696
731 fully_vaccinated_bl==FALSE RMTL 37.938 5.280 27.590 48.286
731 fully_vaccinated_bl==TRUE RMTL 44.211 5.565 33.304 55.118
732 fully_vaccinated_bl==FALSE RMST 323.474 5.838 312.033 334.915
732 fully_vaccinated_bl==TRUE RMST 321.090 5.662 309.993 332.187
732 fully_vaccinated_bl==FALSE RMTL 41.526 5.838 30.085 52.967
732 fully_vaccinated_bl==TRUE RMTL 43.910 5.662 32.813 55.007
Residence_area Measure Estimate CI.lower CI.upper p_value
733 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 0.161 -6.390 6.713 0.961
733 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.000 0.981 1.021 0.961
733 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.996 0.839 1.182 0.961
731 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -6.273 -21.307 8.762 0.414
731 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.981 0.936 1.027 0.414
731 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.165 0.807 1.683 0.415
732 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -2.384 -18.323 13.555 0.769
732 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.993 0.945 1.043 0.769
732 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.057 0.728 1.537 0.770