Building head-to-head intervention comparisons using data sourced from separate registries
22 Dec 2016

A common limitation encountered by researchers wishing to analyse the relative effectiveness or safety of a drug, device or other medical intervention using real world observational data is the lack of appropriate comparators in relevant patient populations. Whilst many registry-based datasets are very good at accumulating quality data over long time horizons for a particular type of intervention, they are often less complete with regards to competing interventions. Furthermore, direct comparisons are often compromised by significant confounding secondary to severe imbalance of prognostic correlates of study end-points between the study arms, limiting the researcher’s capacity to attribute any observed advantage in effectiveness or safety to the treatment of interest, over selection effects.

An increasingly popular method for both combining data across multiple registry sources and correcting confounder imbalance between treatment arms is the use of propensity score matching (PSM). In short, the propensity score is a single metric calculated at the level of the individual study subject which captures the theoretical probability that a study subject would have received the intervention of interest based purely on a set of baseline characteristics. These baseline characteristics are typically those exact same patient and disease factors known or suspected to also predict key study end-points such as disease progression or survival. The propensity score can then be used to match patients from the intervention arm to comparable patients in the comparator arm. Differences in assessment or follow-up schedules between the two registries being combined can further be adjusted for by including assessment frequency and follow-up duration as explanatory variables in the derivation of the propensity score.

In the following example, data from the international MSBase multiple sclerosis (MS) outcomes registry was combined with a pharmaceutical company safety and efficacy registry, the Tysabri Observational Programme, to permit a direct propensity-matched comparison of effectiveness and treatment persistence outcomes in MS patients switching to Tysabri (Natalizumab) following relapse on platform therapy, relative to patients switching to an alternate platform therapy product.

Table 1 illustrates the magnitude of the confounder imbalance problem often encountered in the analysis of real-world, observational data. As can be appreciated from the significant p-values, there are considerable differences in these key baseline factors between the two registry cohorts. Thus any head-to-head comparison of the relative effectiveness of Tysabri would be severely compromised by this imbalance.

Table 1: Distribution of baseline characteristics prior to matching

Table 2 summarises the situation after the application of PSM across the two registry sources. What is immediately apparent from the now non-significant p-values is that the sample is now balanced for all relevant baseline confounders. Therefore, any direct comparison suggesting a treatment advantage favouring Tysabri is independent of at least those confounders included in the derivation of the propensity score used to matched the two registries.

Table 2: Distribution of baseline characteristics following PSM

The following links provide full examples of how PSM has been used to generate direct intervention comparisons using data from separate registries:

Spelman, T., Kalincik, T., Zhang, A., Pellegrini, F., Wiendl, H., Kappos, L., ... & Butzkueven, H. (2015). Comparative efficacy of switching to natalizumab in active multiple sclerosis. Annals of clinical and translational neurology2(4), 373-387.
Spelman, T., Kalincik, T., Jokubitis, V., et al. (2015). Comparative efficacy of first-line natalizumab versus IFNb or glatiramer acetate in relapsing MS. Neurology Clinical Practice. 21;258-259