Gewählte Publikation:
SHR
Neuro
Krebs
Kardio
Lipid
Stoffw
Microb
Homeyer, A; Geißler, C; Schwen, LO; Zakrzewski, F; Evans, T; Strohmenger, K; Westphal, M; Bülow, RD; Kargl, M; Karjauv, A; Munné-Bertran, I; Retzlaff, CO; Romero-López, A; Sołtysiński, T; Plass, M; Carvalho, R; Steinbach, P; Lan, YC; Bouteldja, N; Haber, D; Rojas-Carulla, M; Vafaei, Sadr, A; Kraft, M; Krüger, D; Fick, R; Lang, T; Boor, P; Müller, H; Hufnagl, P; Zerbe, N.
Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology.
Mod Pathol. 2022; 35(12):1759-1769
Doi: 10.1038/s41379-022-01147-y
[OPEN ACCESS]
Web of Science
PubMed
FullText
FullText_MUG
- Co-Autor*innen der Med Uni Graz
-
Kargl Michaela
-
Müller Heimo
-
Plass Markus
- Altmetrics:
- Dimensions Citations:
- Plum Analytics:
- Scite (citation analytics):
- Abstract:
- Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.
- Find related publications in this database (using NLM MeSH Indexing)
-
Humans - administration & dosage
-
Artificial Intelligence - administration & dosage
-
Forecasting - administration & dosage
-
Datasets as Topic - administration & dosage
-
Pathology - administration & dosage