IMPORTANCE: Reflectance confocal microscopy (RCM) studies have been performed to identify criteria for diagnosis of skin neoplasms. However, RCM-based diagnosis is operator dependent. Hence, reproducibility of RCM criteria needs to be tested. OBJECTIVE: To test interobserver reproducibility of recognition of previously published RCM descriptors and accuracy of RCM-based skin cancer diagnosis. DESIGN, SETTING, AND PARTICIPANTS: Observational retrospective web-based study of a set of RCM images collected at a tertiary academic medical center. Nine dermatologists (6 of whom had ≥3 years of RCM experience) from 6 countries evaluated an RCM study set from 100 biopsy-proven lesions, including 55 melanocytic nevi, 20 melanomas, 15 basal cell carcinomas, 7 solar lentigines or seborrheic keratoses, and 3 actinic keratoses. Between June 15, 2010, and October 21, 2010, participanting dermatologists, blinded to histopathological diagnosis, evaluated 3 RCM mosaic images per lesion for the presence of predefined RCM descriptors. MAIN OUTCOMES AND MEASURES: The main outcome was identification of RCM descriptors with fair to good interrater agreement (κ statistic, ≥0.3) and independent correlation with malignant vs benign diagnosis on discriminant analysis. Additional measures included sensitivity and specificity for diagnosis of malignant vs benign for each evaluator, for majority diagnosis (rendered by ≥5 of 9 evaluators), and for experienced vs recent RCM users. RESULTS: Eight RCM descriptors showed fair to good reproducibility and were independently associated with a specific diagnosis. Of these, the presence of pagetoid cells, atypical cells at the dermal-epidermal junction, and irregular epidermal architecture were associated with melanoma. Aspecific junctional pattern, basaloid cords, and ulceration were associated with basal cell carcinomas. Ringed junctional pattern and dermal nests were associated with nevi. The mean sensitivity for the group of evaluators was 88.9% (range, 82.9%-100%), and the mean specificity was 79.3% (range, 69.2%-90.8%). Majority diagnosis showed sensitivity of 100% and specificity of 80.0%. Sensitivity was higher for experienced vs recent RCM users (91.0% vs. 84.8%), but specificity was similar (80.0% vs. 77.9%). CONCLUSIONS AND RELEVANCE: The study highlights key RCM diagnostic criteria for melanoma and basal cell carcinoma that are reproducibly recognized among RCM users. Diagnostic accuracy increases with experience. The higher accuracy of majority diagnosis suggests that there is intrinsically more diagnostic information in RCM images than is currently used by individual evaluators.