Synergus

In this blog post we will provide a perspective of the current methodology in the EUnetHTA core model and the Guideline “Applicability of evidence” by EUnetHTA. This will be in relation to the impact of variability in health care delivery and its importance. The aim of European collaboration through EUnetHTA is to establish a common evaluation, where different parts of the evaluation can be transferred to the context of the other member states. We would suppose that we may be missing an important issue to accomplish this objective. 

In the summary of the guideline it states: 

In order to make a simplistic summary of what we consider to be the missing dimension, we would summarize what we consider as the current perspective and what we think should be an alternative approach.

Current perspective of general effectiveness

When we evaluate the efficacy, it is from a general perspective with limited impact on the local setting. From this perspective, it is more of a universal “truth”. In these guidelines and in the general discussion, the focus about effectiveness would then emphasis on aspects of differences in population (general) and about comparator (country-specific). If the comparator is applicable and the RCT is right, the ‘general effectiveness’ is translated to a health economic model. This would refer to local cost data and epidemiology to demonstrate what we suggest as “General cost-effectiveness”.

Country specific effectiveness

When the question is changed from a general question about efficacy to a specific question of effectiveness in a particular country, it becomes much more complex than what we can observe is being considered. Hence the title of this blog-series, the illusion of simplicity.

Based on our analysis of the impact of health care delivery on the outcomes and the variability, we would suggest that in many instances there is no such thing as general effectiveness. If our aim is to understand the country-specific clinical and cost-effectiveness, the process should rather be as follows:

Are we missing a key issue in the evaluation of applicability of evidence in a specific country and if so, why?

From our ongoing analysis of disease areas that have significant impact on the outcome of health care delivery, we have found many examples. In the coming months, we are going to post summaries of this research in further blogs.

Given the magnitude of impact on health care delivery on the outcomes, it is rather surprising that this is not reflected in the methodological frameworks.

Considering our earlier analysis, it is not unreasonable that we frequently have scenarios where an RCT has shown significant improvement versus the comparator, however the difference is still rather minimal compared with best practices in the real-world setting.

If we aim to use cost-effectiveness (not efficacy) as an important part of decision making, we would make an incorrect decision unless we care about this difference between the RCT and real-world setting.

If this is such an important question, why has it not been in focus?

From our reflection, we believe these are some of the underlying reasons:

• In traditional clinical trials, the focus is to isolate the noise in order to get a clean signal of the efficacy. In the real world, the priority should rather be to understand the impact of these noises.
• The effect of a drug / intervention is seen in isolation, where in many instances it is only a component contributing to the overall outcome.
• It is obviously important to understand the efficacy of the individual component. However, when evaluating the value, it should be done in the context of all other factors influencing the results. What if it was possible to achieve the same result of improving adherence of an old drug with a ‘simple digital solution’ at low cost, instead of introducing a new expensive drug?
• Due to the lack of high-quality real-world data sources in most countries, there is limited research on the impact of health care delivery.

In summary, we would ask industry and HTA bodies:

• Do we approach the effectiveness question as a generic question rather than specific to a country / setting?
• How well do we understand the effectiveness of current treatments in the countries, in order to understand how this is similar or different to the comparative arm of the clinical study?
• What is our understanding of the variability in health care delivery and how does this impact the outcomes?
• If this is now a potentially important question, what should the methodological approach to this be?

EUnetHTA core model and transferability

We have reviewed the EUnetHTA core model in order to understand the different domains and how they relate to a generic vs. country-specific effectiveness perspective. Here are some of the key aspects of this review:

• A0007 - What is the target population in this assessment? / A0023 - How many people belong to the target population? This addresses the well-recognized question of the challenge to understand how the outcomes will change in the real world. This refers to cases when treatment is likely to be utilized by a group of patients who are less homogenous than those included in the clinical trial.

• A0018 - What are the other typical or common alternatives to the current technology? This addresses the direct comparator of new interventions but does not consider all other factors within health care delivery that influence the outcome.
• A0024 - How is the disease or health condition currently diagnosed according to published guidelines and in practice? What is the impact of diagnostic accuracy, however nothing about the process is related to diagnosis and intervention.
• A0025 - How is the disease or health condition currently managed according to published guidelines and in practice? In this section, the focus is on the guidelines, recommendations and published utilization reviews. It also focuses on the absence of clinical expert surveys.
• A0012 – What kind of variations in use exist across countries/regions/settings? Focus is on high-level statistics, on a national level about the utility of interventions.

The document captures many of the important aspects in relation to Population, Intervention, comparators and outcomes with a framework of how to evaluate these. In the section about Organisational aspects, there are many factors related to health care delivery, but the focus is rather on issues about implementation than the impact of outcomes.

• For updates on our findings, please join the LinkedIn group (Evaluating Transferability of Real World Evidence) where we look forward to your feedback as we share further insights.
• Feel free to comment on the LinkedIn post

References:

1. EUnetHTA core model
2. Applicability of evidence for the context of a relative effectiveness assessment