MetaDTA – Meta-Analysis of Diagnostic Test Accuracy studies
Authors: Suzanne Freeman, Clareece Nevill (nee Kerby), Amit Patel, Nicola Cooper, Terry Quinn, Alex Sutton
Working with Shiny more than 1 year
Abstract: Medical diagnostic tests are used to inform treatment decisions. Meta-analysis (M-A) is used to synthesise the results of multiple diagnostic test accuracy (DTA) studies to establish their accuracy. Statistical methods for such analyses use relatively complex models which can be a barrier for non-statisticians. MetaDTA is an online application which allows a range of stakeholders to interactively conduct, explore and visually examine results of DTA M-A and facilitate sensitivity analyses.
Full Description:
Why was MetaDTA developed?
MetaDTA was developed by the UK National Institute for Health Research (NIHR) Complex Review Support Unit (University of Glasgow - Materials and Guidance - Apps) whose remit is to improve the value of systematic reviews which the NIHR funded. It became apparent that lack of accessible user-friendly software was a major stumbling block for health researchers without advanced statistical expertise with respect to carrying out specialist analyses such as diagnostic test accuracy (DTA) meta-analysis (M-A). It was decided that a Shiny app would be the most effective solution to this problem. We have received excellent feedback on MetaDTA and know it is getting 100s of hours use each month by researchers in many areas of the world, including those in low and middle income countries. In addition to its primary function of allowing non-statistical experts to do research using DTA M-A, we get enthusiastic reports from statisticians who use it because of its ease and efficiency and educators who use it as a practical teaching tool in the classroom. The Cochrane Collaboration (www.cochrane.org), whose work is internationally recognised as a benchmark for high-quality information about the effectiveness of healthcare, have expressed interest in including the app within the software chapter of their DTA M-A handbook. An open-access research article on MetaDTA was published in the journal BMC Medical Research Methodology in April 2019 and is available at Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA | BMC Medical Research Methodology | Full Text.
What is diagnostic test accuracy meta-analysis?
Medical diagnostic tests aim to distinguish between people with and without a condition of interest, often a disease. Diagnostic testing is ubiquitous in healthcare informing treatment decisions and determining care pathways. M-A is a statistical method of synthesising the results of multiple DTA studies to establish their accuracy. In an era of evidence based medicine meta-analyses of DTA studies often represent the best available clinical evidence for the recommended diagnostic tests routinely used in practice. MetaDTA provides a unique tool allowing researchers and patients alike to scrutinise the clinical evidence behind the healthcare decision making process.
To assess accuracy, a diagnostic test is compared to the “gold standard” test which is assumed to provide the true diagnosis of individuals and two parameters, sensitivity and specificity, are calculated. Sensitivity is the ability of the diagnostic test to correctly identify patients with the disease amongst patients who have the disease. Specificity is the ability of the diagnostic test to correctly identify the healthy individuals amongst patients who do not have the disease. A DTA M-A synthesises both sensitivity and specificity taking into account the correlation between them. Recommended statistical approaches are either the bivariate or hierarchical summary receiver operating characteristic (HSROC) models. Without covariates, these two approaches are mathematically equivalent. MetaDTA uses the bivariate model to estimate sensitivity and specificity across all studies and uses the bivariate model parameters to calculate the HSROC model parameters which are then used to plot the SROC curve.
What does MetaDTA do?
MetaDTA conducts M-A of DTA studies using a bivariate model, and displays the individual study and M-A results using a summary receiver operating characteristic (SROC) curve. As well as carrying out analyses, data and results are presented graphically. MetaDTA provides multiple displays for exploring data interactively. Exploration of the robustness of results is encouraged by the ability to explore the impact of excluding specific studies in the evidence base directly within the interface and the option to display those results alongside the results from all studies on the dedicated ‘ Sensitivity Analysis’ page.
There is often variation between DTA studies within an M-A and identifying why studies disagree is an important part of the M-A process. To facilitate the understanding of variation between studies MetaDTA allows visualisation of study-level covariates alongside the SROC curve, including markers of study quality from the QUADAS-2 tool (QUADAS-2 | Bristol Medical School: Population Health Sciences | University of Bristol). The graphical display adapts to the simultaneous presentation of covariates and study quality and includes novel plots of multiple dimensions of study quality from QUADAS-2. High quality graphical output can be downloaded from MetaDTA for inclusion in written reports.
MetaDTA leverages features from 22 existing R packages as well as bespoke code to deliver an accessible interactive analysis solution. For more information on the range of features available within MetaDTA and how to use MetaDTA please see the User Guide available to download from the MetaDTA home page (https://crsu.shinyapps.io/dta_ma/).
Who is MetaDTA for?
Due to the rich feature-set and user-friendliness of the software MetaDTA should appeal to a wide audience including those without specialist statistical knowledge or access to statistical software. MetaDTA contains four datasets from a real-life systematic review in dementia for users to explore and discover the range of options available. MetaDTA can be accessed at https://crsu.shinyapps.io/dta_ma/.
Category: Healthcare
Keywords: Meta-analysis, diagnostic test accuracy, data visualization, evidence based medicine
Shiny app: https://crsu.shinyapps.io/dta_ma_1_43/
Repo: https://github.com/SCFreeman/MetaDTA-v1.45
RStudio Cloud: Posit Cloud
Thumbnail:
Full image:
