Why clinical trials fail to address context?

A health service, is a packaged treatment that defines and influences the context around a drug or therapy, it defines a patient life journey, his interaction with the healthcare system and the drug being taken. Examples of healthcare services are medical interventions and chronic disease management programs. 

With RCTs, drugs have been validated separately from the context of its use. However, drugs are not taken by people in labs and formal studies, but by patients living their lives. As such there is a need for more drugs to be validated within the context of their use. Note that a single drug may have many usage contexts depending on the condition, the healthcare system and patient group, resulting in multiple potential health services per drug. 

Unlike single drug treatments, health services can be (and usually are) adjusted to the specific situation and lifestyle of the patient. Even if the intention is to execute the health service identically for all patients, there are external factors that introduce differences in the results of the study: the clinician’s expertise and preferences, the patient lifestyle and engagement in the healthcare program, the caregivers supporting the patient and even geographical and cultural influences. 

Moreover, health services are characterized by fuzzy and long term outcome measures, which makes such studies very complex and expensive to run as they require studies to follow a large population of patients over a long time. 

As discussed in my earlier post Randomized Controlled Trials (RCTs) provide great scientific results, but have a number of practical limitations due to the strict criteria it imposes on study design and data gathering. These limitations are particularly relevant when considering packaged health services instead of drugs. 

Below are the RCT criteria listed as well as a short discussion on the practical limitations of applying  these criteria in the health services context:

Randomisation: It is much more difficult to randomly assign test subjects to 2 different health services than 2 drugs, as both patients and clinicians most likely will have a preferred personal choice. Health services are almost long term (e.g. chronic disease management) and sometimes permanent (e.g. medical interventions). Here any null hypothesis (the null hypothesis is usually: no significant difference in clinical results) will be rejected on a priori grounds. In many contexts there are also ethical reasons to prevent randomization, because of the impact and risk for even healthy test subjects. It is most unlikely that any ethics committee would sanction the random allocation of test subjects to cardiac transplantation. 

Blinding: Blinding is particularly difficult in health service products. It is much easier for patients to be oblivious to which medication was given to them than to not know which type of intervention has been performed on them. Double blinding is a larger challenge yet. Therefore one must assume that both patient and clinician will know which intervention they are allocated to. The lack of blinding may be minimized by choosing a “hard” outcome indicator, such as mortality or morbidity, which cannot be influenced by personal bias. On the other hand, if the outcome indicator is largely subjective (e.g. a change in symptoms or quality of life), lack of blinding will most likely bias the results.

Placebo: is a faked treatment performed in the control population to assess the effect of the treatment under study. Many ethicists reject sham-interventions, other maintain that such trials are ethically acceptable but should conform to certain restrictions that puts very strict requirements on the trial. Such restrictions include that the research question cannot be answered by any other form of trial or study and that the risk of such procedure can be kept to a minimum. It is unlikely that patients will willingly accept the risks and put up with the inconvenience that sham interventions may create. 

Equipoise: One of the major factors limiting clinical trials is the lack of community equipoise. Clinical equipoise is an ethical dilemma introduced by Freedman, which can be paraphrased as: ”genuine uncertainty within the expert medical community on the optimal approach for a certain medical condition”. Equipoise allows clinical investigators to continue a trial until they have enough statistical evidence to convince other experts of the validity of their results, without a loss of ethical integrity on the part of the investigators. Although commonly not based on scientific studies, there is often a consensus of medical practitioners on the effectiveness of therapies. 

Timing: The issue of timing of trials is difficult. Most clinicians agree that any healthcare service would change significantly within first few years. In any healthcare service there exist a learning curve and modifications to the new treatment baseline are made frequently. By including these early patients, one would almost certainly bias the results against the new procedure. On the other hand, it may be difficult and quite unnecessary to initiate a trial when the procedure is widely accepted by both the patient and the medical community. 

Multi-centre: There may be inherent selection biases in referral patterns to the institution (’centre effect’) and study participation; as well as limitations posed by sample size and completeness of data.

As must now be obvious to you there are a number of challenges one must overcome or accept that the validity of results is affected. In my next blog post I will talk about how machine learning based on real-world data can fill this gap.