Leveraging advanced analytics and AI to expand clinical trial recruitment beyond conventional diagnoses

Mounting evidence suggests that many conditions previously considered distinct can, in fact, share significant phenotypic and genetic overlap. In a recent genome-wide association study (GWAS), for example, a substantive association was found between endometriosis and a wide range of immunological diseases, an association not previously understood. In this study, women with endometriosis were found to have a significantly increased risk (30–80%) of developing autoimmune diseases such as rheumatoid arthritis (RA), multiple sclerosis, and celiac disease. The impressive finding is that these phenotypic conditions may share underlying genetic risk factors and immunological pathways not previously understood. Conventional recruitment paradigms would not typically identify these patients for relevant studies in RA or other autoimmune disorders, despite their heightened risk and potential eligibility.

While an early proof of concept, the findings in this study demonstrate the need for a broader, more integrated approach to clinical trial recruitment that may consider expanded inclusion criteria that enables undiagnosed patients or patients with diagnoses of conditions that are associated with the study target to be screened for participation.

This would be a significant change from current recruitment models that typically prioritize either patient self-identification through pre-screening questionnaires, referral to a study by their diagnosing provider, or the identification of eligible candidates via analysis of structured data within electronic medical records (EMRs). These well-established approaches are inherently limited because they overlook individuals who have not yet been formally diagnosed with the target condition but may share overlapping comorbidities or present with early or atypical symptoms.

Advanced analytics and artificial intelligence (AI) technologies now offer clinical operations leaders an opportunity to substantially broaden their recruitment funnel, transcending the rigid boundaries of traditional diagnostic criteria. We’ve identified several key capabilities that organizations can assemble to enhance this approach to advanced analytics that will widen their recruitment funnel, especially, we expect, for rare conditions and/or those that are difficult or complex to diagnose.

Assemble and integrate large-scale, diverse patient data

AI models require access to extensive and heterogeneous datasets, including both structured (diagnoses, codes, laboratory measurements) and unstructured (clinical notes, imaging reports) data. Robust integration of such information is the foundation upon which advanced AI-driven recruitment solutions are built.

Of course, assembling a vast dataset of representative data may be easier said than done, but the typical large-scale sources of data can support this objective. Electronic health records (EHR), claims data, registries and biobanks, imaging archives, patient-reported wearables data, genomic or other -omics datasets, and externally available or public datasets can all deliver the required insights. Because it can be complex and expensive to acquire such a trove of data, it may be more effective to start in a single, smaller therapeutic area, like autoimmune or psychiatric conditions and leverage the published literature to understand where potential mechanistic links to other conditions may occur.

Pattern recognition and hypothesis generation

Machine learning algorithms can discern patterns across patient populations, identifying phenotypic or genetic markers that correlate with increased risk for certain conditions, even in the absence of a definitive diagnosis. For instance, a patient with a documented history of endometriosis and emerging musculoskeletal complaints may be at higher risk for developing RA. Advanced analytics can flag such patterns for further clinical assessment.

Natural language processing (NLP) is particularly effective in extracting relevant clinical phenotypes from unstructured EHR data, such as physician notes or imaging reports. This technology enables identification of symptoms and risk factors that may not have been linked, such as chronic pelvic pain or unexplained inflammatory markers that precede or parallel emerging diagnoses in RA and endometriosis.

Probabilistic matching and candidate scoring

Advanced probabilistic models can assign a “likelihood score” to patients, based on the constellation of their clinical, phenotypic, and genetic data. In practice, an individual without a formal RA diagnosis but with endometriosis and associated symptoms could be scored as a high-probability candidate, flagged for targeted outreach and pre-screening for upcoming RA trials. This opens avenues for earlier diagnosis, more timely intervention, and inclusion of under-recognized patient populations.

Workflow integration at the site and investigator level

For these analytic advances to impact trial recruitment, the outputs of the analyses must be incorporated into investigational site workflows. Tailored reporting applications can list probable candidates, provide context for their inclusion, and support compliance with privacy and consent requirements when re-contacting individuals whose risks have been algorithmically identified.

Outcomes: enhanced recruitment and accelerated research

Implementing these strategies can yield substantive benefits:

• Early identification and intervention: Patients may be evaluated and diagnosed earlier in their disease trajectory, benefiting from timely access to investigational therapies.
• Broader and faster enrollment: The recruitment pool expands to include individuals at risk, as well as those who fit traditional diagnostic models, accelerating overall trial timelines. One can expect that these patients with complex symptomology, especially without a conclusive diagnosis, will be motivated to pursue the sometimes-lengthy screening process to join a trial for an experimental therapy.
• Increased data fidelity and relevance: By capturing a more representative, diversified patient population that includes early stage, atypical, or comorbid presentations of complex conditions, study findings gain external validity and scientific relevance.

Emerging relationships among diverse disease states underscore the necessity of reimagining recruitment strategies for clinical trials. Advanced analytics and AI provide the tools to transcend historical diagnostic boundaries, identifying and engaging potential participants who might otherwise remain unrecognized. For clinical operations leaders, adoption of these technologies offers a path to more inclusive trials, earlier intervention for patients, and more rapid advancement of therapeutic innovation.

This article was first published in Clinical Leader.