Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that supports research on pragmatic trials. It gathers and distributes clean trial data, ratings, and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological analyses to compare treatment effect estimates across trials of various levels of pragmatism.
Background
Pragmatic trials are becoming more widely recognized as providing real-world evidence for clinical decision making. However, the use of the term "pragmatic" is not consistent and its definition and assessment requires further clarification. Pragmatic trials are intended to guide clinical practices and policy choices, rather than prove a physiological or clinical hypothesis. A pragmatic trial should try to be as similar to the real-world clinical environment as is possible, including the participation of participants, setting up and design of the intervention, its delivery and execution of the intervention, determination and analysis of outcomes and primary analysis. This is a key difference from explanatory trials (as described by Schwartz and Lellouch1) which are intended to provide a more thorough confirmation of an idea.
Truely pragmatic trials should not be blind participants or clinicians. This can result in a bias in the estimates of the effects of treatment. Practical trials should also aim to recruit patients from a wide range of health care settings so that their results are generalizable to the real world.
Additionally studies that are pragmatic should focus on outcomes that are vital to patients, such as quality of life or functional recovery. This is especially important for trials involving surgical procedures that are invasive or have potentially serious adverse events. The CRASH trial29, for instance focused on the functional outcome to compare a two-page report with an electronic system for monitoring of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 used urinary tract infections caused by catheters as the primary outcome.
In addition to these aspects the pragmatic trial should also reduce the trial's procedures and requirements for data collection to reduce costs. Additionally these trials should strive to make their results as relevant to real-world clinical practice as is possible. This can be accomplished by ensuring their primary analysis is based on the intention to treat approach (as described within CONSORT extensions).
Despite these requirements however, a large number of RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This can lead to false claims of pragmatism, and the usage of the term should be standardized. The development of a PRECIS-2 tool that provides an objective, standardized evaluation of pragmatic aspects is a good start.
Methods
In a pragmatic research study, the goal is to inform policy or clinical decisions by showing how an intervention can be integrated into routine care in real-world contexts. This is distinct from explanation trials, which test hypotheses about the causal-effect relationship in idealized conditions. In this way, pragmatic trials may have less internal validity than explanatory studies and are more susceptible to biases in their design, analysis, and conduct. Despite these limitations, pragmatic trials may contribute valuable information to decision-making in the context of healthcare.
The PRECIS-2 tool measures the degree of pragmatism in an RCT by assessing it across 9 domains that range from 1 (very explanatory) to 5 (very pragmatic). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains scored high scores, but the primary outcome and the method for missing data were below the pragmatic limit. This indicates that a trial can be designed with well-thought-out practical features, yet not compromising its quality.
It is difficult to determine the degree of pragmatism in a particular study because pragmatism is not a have a binary characteristic. Some aspects of a study may be more pragmatic than others. A trial's pragmatism can be affected by changes to the protocol or logistics during the trial. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. Most were also single-center. They are not in line with the standard practice and are only referred to as pragmatic if their sponsors accept that such trials aren't blinded.
A common feature of pragmatic studies is that researchers attempt to make their findings more meaningful by studying subgroups of the trial sample. This can lead to unbalanced analyses that have lower statistical power. This increases the possibility of missing or misdetecting differences in the primary outcomes. In the case of the pragmatic trials included in this meta-analysis this was a significant problem since the secondary outcomes weren't adjusted for variations in baseline covariates.
Furthermore the pragmatic trials may have challenges with respect to the collection and interpretation of safety data. This is because adverse events are typically reported by participants themselves and are susceptible to reporting delays, inaccuracies, or coding variations. It is therefore important to improve the quality of outcomes assessment in these trials, ideally by using national registry databases instead of relying on participants to report adverse events on a trial's own database.
Results
Although the definition of pragmatism does not require that all clinical trials be 100% pragmatist there are benefits when incorporating pragmatic components into trials. These include:
By incorporating routine patients, the trial results can be more quickly translated into clinical practice. However, pragmatic trials may have their disadvantages. For instance, the appropriate type of heterogeneity can help a study to generalize its results to different settings and patients. However the wrong kind of heterogeneity may reduce the assay's sensitivity and therefore lessen the ability of a study to detect small treatment effects.
A variety of studies have attempted to categorize pragmatic trials using various definitions and scoring methods. Schwartz and Lellouch1 created a framework to distinguish between explanatory studies that confirm the physiological hypothesis or clinical hypothesis, and pragmatic studies that inform the choice for appropriate therapies in real world clinical practice. Their framework included nine domains, each scored on a scale ranging from 1-5, with 1 indicating more explanatory and 5 indicating more practical. The domains included recruitment, setting, intervention delivery with flexibility, follow-up and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. 슬롯사이트 et al10 created an adaptation to this assessment called the Pragmascope which was more user-friendly to use in systematic reviews. Our Web Site found that pragmatic systematic reviews had higher average scores in the majority of domains, but lower scores in the primary analysis domain.
This distinction in the primary analysis domain could be explained by the fact that most pragmatic trials analyze their data in an intention to treat way, whereas some explanatory trials do not. The overall score was lower for systematic reviews that were pragmatic when the domains on the organization, flexibility of delivery and follow-up were combined.
It is important to note that the term "pragmatic trial" does not necessarily mean a low quality trial, and there is a growing number of clinical trials (as defined by MEDLINE search, but this is not specific or sensitive) that employ the term 'pragmatic' in their title or abstract. These terms may indicate that there is a greater appreciation of pragmatism in titles and abstracts, but it isn't clear whether this is reflected in content.
Conclusions
As appreciation for the value of real-world evidence grows widespread, pragmatic trials have gained momentum in research. They are randomized clinical trials that evaluate real-world alternatives to care instead of experimental treatments under development, they involve populations of patients which are more closely resembling those treated in routine medical care, they utilize comparators which exist in routine practice (e.g., existing medications) and depend on the self-reporting of participants about outcomes. This approach can overcome the limitations of observational research like the biases that are associated with the use of volunteers and the limited availability and the coding differences in national registry.
Other advantages of pragmatic trials are the ability to utilize existing data sources, and a higher likelihood of detecting meaningful changes than traditional trials. However, pragmatic trials may still have limitations that undermine their validity and generalizability. For example the rates of participation in some trials may be lower than anticipated due to the healthy-volunteer effect as well as financial incentives or competition for participants from other research studies (e.g., industry trials). A lot of pragmatic trials are limited by the need to recruit participants in a timely manner. In addition certain pragmatic trials don't have controls to ensure that the observed differences aren't due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified RCTs published from 2022 to 2022 that self-described themselves as pragmatic. The PRECIS-2 tool was used to evaluate the degree of pragmatism. It includes domains such as eligibility criteria and flexibility in recruitment as well as adherence to interventions and follow-up. They found 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or above) in at least one of these domains.
Studies that have high pragmatism scores tend to have more criteria for eligibility than traditional RCTs. They also have patients from a variety of hospitals. The authors claim that these characteristics can help make pragmatic trials more effective and applicable to everyday clinical practice, however they do not guarantee that a trial conducted in a pragmatic manner is free of bias. Moreover, the pragmatism of the trial is not a predetermined characteristic; a pragmatic trial that does not have all the characteristics of an explanatory trial can produce valuable and reliable results.