Accelerating Systematic Literature Reviews with AI-Driven Data Extraction
About R-S-S
R-S-S (Regulatory Scientific and Health Solutions) is a functional service provider to the biopharmaceutical industry, supported by regulatory and technical experts who combine their knowledge of drug licensing with Health Technology Assessment (HTA).
Part of HTA (and regulatory) requirements includes evidence synthesis which requires systematic literature reviews (SLRs). R-S-S routinely generate SLRs for multiple clients.
R-S-S have collaborated with giles® in bringing together a new AI-based tool that can facilitate accelerated SLRs.
The Challenge
- The creation of an SLR can be a long process, taking many months and in some cases years to finalise prior to submission.
- In order to properly understand the latest research or any new drug/device, a comprehensive literature review is essential. This can take a long time to complete.
- The costs involved in SLRs (depending on the complexity of projects) can reach beyond £400K in some cases.
- SLRs are typically updated at regular intervals to ensure the latest evidence is incorporated in the body of evidence submitted to regulatory authorities.
The Solution
giles® empowers researchers to search PubMed Central’s extensive database of over 10 million peer-reviewed publications or to upload their own PDFs for in-depth exploration and data extraction.
By enabling researchers to swiftly extract a list of important outcomes for their SLR, giles® accelerates the data extraction phase.
Additionally, researchers can upload documents and use giles® to quickly examine document content, identifying those that meet specific inclusion and exclusion criteria, making the review process faster and more targeted.
Methodology
We tested and evaluated the performance of giles to extract data from PubMed publications using Atopic Dermatitis (AD) as the disease indication. We created a data extraction sheet which consisted of typical parameters (e.g., title, year, treatments, time-points, outcomes).
We focused on key outcomes collected in randomized controlled trials where results have been published, such as EASI (Eczema Area and Severity Index) and IGA (Investigator’s Global Assessment). A manual extraction of these outcomes using the standard SLR process (PRISMA) was conducted and then compared with giles® data extraction. For each outcome, the key parameters were extracted, which spanned across all available treatment arms and time-points from 13 published articles. A total of 1166 data points were extracted.
The evaluation process involved the giles® user initially uploading all published articles used in the manual extraction into the giles® tool. A refined prompt was used, conducting 20 iterations before reaching a stable format that could be applied across all articles and treatment arms. On average, 6.23 questions were posed per paper to retrieve the necessary data points.
Results
giles® demonstrated an average accuracy of approximately 94% in the data extraction of key outcomes (EASI) compared to the manual SLR (gold standard). The AI-driven extraction process accurately identified and interpreted tables, extracting relevant data points and aligning closely with the manual results.
This high level of precision highlights giles®’s potential as a valuable aid for researchers dealing with large quantities of information, offering consistent and reliable extraction performance without sacrificing quality. Thus, using AI-tools shows promise in reducing manual workload in clinical research while maintaining rigorous standards.
*The Average Correlation coefficient between Giles AI extracted and Manually extracted EASI Outcomes was 0.98 (range of 0.93 – 0.99), suggesting exceptionally high agreement.