Kristen Fernandez

Artificial intelligence models match or exceed dermatologists in melanoma image classification. Less is known about their robustness against real-world variations, and clinicians may incorrectly assume that a model with an acceptable area under the receiver operating characteristic curve or related performance metric is ready for clinical use. Here, we systematically assessed the performance of dermatologist-level convolutional neural networks (CNNs) on real-world non-curated images by applying computational "stress tests". Our goal was to create a proxy environment in which to comprehensively test the generalizability of off-the-shelf CNNs developed without training or evaluation protocols specific to individual clinics. We found inconsistent predictions on images captured repeatedly in the same setting or subjected to simple transformations (e.g., rotation). Such transformations resulted in false positive or negative predictions for 6.5-22% of skin lesions across test datasets. Our findings indicate that models meeting conventionally reported metrics need further validation with computational stress tests to assess clinic readiness.

Calciphylaxis is a rare condition characterized by medial calcification of small- and medium-sized vessels that subsequently leads to ischemic necrosis. Calciphylaxis most often occurs in patients with end-stage renal disease and secondary hyperparathyroidism. We present a unique case of calciphylaxis in which the patient did not have end-stage renal disease. Instead, primary hyperparathyroidism and/or alcoholic cirrhosis were the more likely causes of her calciphylaxis. In addition, our case demonstrated not only calciphylaxis but also fragmentation and calcification of elastic fibers within the dermis, changes that are most often seen in pseudoxanthoma elasticum. This is the first reported case of calciphylaxis, to our knowledge, with histopathologic changes of pseudoxanthoma elasticum in a patient who is nonuremic.

Importance: Skin cancer is the most common cancer in the US; accurate detection can minimize morbidity and mortality. Objective: To assess the accuracy of skin cancer diagnosis by lesion type, physician specialty and experience, and physical examination method. Data Sources: PubMed, Embase, and Web of Science. Study Selection: Cross-sectional and case-control studies, randomized clinical trials, and nonrandomized controlled trials that used dermatologists or primary care physicians (PCPs) to examine keratinocytic and/or melanocytic skin lesions were included. Data Extraction and Synthesis: Search terms, study objectives, and protocol methods were defined before study initiation. Data extraction was performed by a reviewer, with verification by a second reviewer. A mixed-effects model was used in the data analysis. Data analyses were performed from May 2022 to December 2023. Main Outcomes and Measures: Meta-analysis of diagnostic accuracy comprised sensitivity and specificity by physician type (primary care physician or dermatologist; experienced or inexperienced) and examination method (in-person clinical examination and/or clinical images vs dermoscopy and/or dermoscopic images). Results: In all, 100 studies were included in the analysis. With experienced dermatologists using clinical examination and clinical images, the sensitivity and specificity for diagnosing keratinocytic carcinomas were 79.0% and 89.1%, respectively; using dermoscopy and dermoscopic images, sensitivity and specificity were 83.7% and 87.4%, and for PCPs, 81.4% and 80.1%. Experienced dermatologists had 2.5-fold higher odds of accurate diagnosis of keratinocytic carcinomas using in-person dermoscopy and dermoscopic images compared with in-person clinical examination and images. When examining for melanoma using clinical examination and images, sensitivity and specificity were 76.9% and 89.1% for experienced dermatologists, 78.3% and 66.2% for inexperienced dermatologists, and 37.5% and 84.6% for PCPs, respectively; whereas when using dermoscopy and dermoscopic images, sensitivity and specificity were 85.7% and 81.3%, 78.0% and 69.5%, and 49.5% and 91.3%, respectively. Experienced dermatologists had 5.7-fold higher odds of accurate diagnosis of melanoma using dermoscopy compared with clinical examination. Compared with PCPs, experienced dermatologists had 13.3-fold higher odds of accurate diagnosis of melanoma using dermoscopic images. Conclusions and Relevance: The findings of this systematic review and meta-analysis indicate that there are significant differences in diagnostic accuracy for skin cancer when comparing physician specialty and experience, and examination methods. These summary metrics of clinician diagnostic accuracy could be useful benchmarks for clinical trials, practitioner training, and the performance of emerging technologies.