Development and clinical validation of real-time artificial intelligence diagnostic companion for fetal ultrasound examination

Objective: Prenatal diagnosis of a rare disease on ultrasound relies on a physician's ability to remember an intractable amount of knowledge. We developed a real-time decision support system (DSS) that suggests, at each step of the examination, the next phenotypic feature to assess, optimizing the diagnostic pathway to the smallest number of possible diagnoses. The objective of this study was to evaluate the performance of this real-time DSS using clinical data. Methods: This validation study was conducted on a database of 549 perinatal phenotypes collected from two referral centers (one in France and one in the UK). Inclusion criteria were: at least one anomaly was visible on fetal ultrasound after 11 weeks' gestation; the anomaly was confirmed postnatally; an associated rare disease was confirmed or ruled out based on postnatal/postmortem investigation, including physical examination, genetic testing and imaging; and, when confirmed, the syndrome was known by the DSS software. The cases were assessed retrospectively by the software, using either the full phenotype as a single input, or a stepwise input of phenotypic features, as prompted by the software, mimicking its use in a real-life clinical setting. Adjudication of discordant cases, in which there was disagreement between the DSS output and the postnatally confirmed (‘ascertained’) diagnosis, was performed by a panel of external experts. The proportion of ascertained diagnoses within the software's top-10 differential diagnoses output was evaluated, as well as the sensitivity and specificity of the software to select correctly as its best guess a syndromic or isolated condition. Results: The dataset covered 110/408 (27%) diagnoses within the software's database, yielding a cumulative prevalence of 83%. For syndromic cases, the ascertained diagnosis was within the top-10 list in 93% and 83% of cases using the full-phenotype and stepwise input, respectively, after adjudication. The full-phenotype and stepwise approaches were associated, respectively, with a specificity of 94% and 96% and a sensitivity of 99% and 84%. The stepwise approach required an average of 13 queries to reach the final set of diagnoses. Conclusions: The DSS showed high performance when applied to real-world data. This validation study suggests that such software can improve perinatal care, efficiently providing complex and otherwise overlooked knowledge to care-providers involved in ultrasound-based prenatal diagnosis.