Medizinische Universität Graz - Research portal
Selected Publication:
Efferl, P.
Assessment of a smartphone-based neural network application for the risk assessment of skin lesions under real-world conditions. A systematic retrospective comparison of detection accuracy by 3 dermatologists with digital risk assessment
Humanmedizin; [ Diplomarbeit ] Medizinische Universität Graz; 2023. pp.
[OPEN ACCESS]
FullText
- Authors Med Uni Graz:
- Advisor:
- Hofmann-Wellenhof Rainer
- Kränke Teresa Maria
- Altmetrics:
- Abstract:
- Background: The diagnostic performance of CNNs for diagnosing different types of skin cancer has been developing promisingly in recent times. The use of a smartphone application that can give lay users a basic assessment with the help of an integrated AI may guide them to take faster therapy when necessary. Objective: The performance of a certified smartphone-based neural network application on macroscopic images of skin lesions taken by laypersons in real-world conditions has not yet been reported and is subject to this study. The main objective of the study is to evaluate the risk-assessment accuracy of the smartphone application in comparison to a consensus opinion of a medical Expert Panel. Methods: We analyzed the detection accuracy of the CE-marked algorithm of the SkinScreener© smartphone application with the detection by a consensus opinion/reference standard of an Expert Panel of three dermatologists at the Medical University of Graz. The primary outcome measures were sensitivity, specificity, and accuracy for the trichotomous risk assessment (low-, medium-, and high-risk). Secondary endpoints included interindividual differences in the dermatologists’ diagnostic performance of analyzing the respective skin lesions. Results: The CE-marked smartphone algorithm's performance in risk assessment was 76.9% (CI: {71.7% - 81.5%}) for sensitivity and 80.9% (CI: {78.5% - 83.2%}) for specificity. The overall accuracy was 77.2%. As a secondary endpoint, interindividual differences in dermatologists' diagnostic performance were found to be significant, with 526 out of a total of 1428 cases not showing complete agreement. It was shown that the performance was worse than in a previously conducted study under clinical conditions. Conclusions: Validation of smartphone-based applications such as the SkinScreener© application in a non-clinical setting can be crucial to obtaining sufficient performance data for such applications. A suitable reference standard needs to be found as the gold standard when histopathological verification is not accessible in non-clinical settings. The reference standard used in the form of a consensus opinion showed difficulties in getting a clear consensus opinion for various reasons and thus raises the question of how such a validation using an Expert Panel as a reference standard can be improved or supplemented with other modalities in future studies.