Background
Melanoma is the most lethal of all the skin cancers and has significantly increased in incidence over the past several decades. When melanoma is diagnosed while still confined to the outer layers of the skin, simple excision is generally curative with 5-year survival of approximately 98%. Unfortunately, despite melanoma being amenable to early diagnosis through simple visual inspection, many patients continue to be diagnosed with more advanced disease. As a result, over 57,000 people worldwide (6,850 Americans) are estimated to have died of melanoma in 2020, despite dramatic recent progress in the treatment of advanced metastatic melanoma.
Melanoma is typically detected by simple visual inspection and is most often brought to medical attention because of a concerning appearance. Although visible to the naked eye, early-stage melanomas may be difficult to distinguish from benign skin lesions, even by clinicians. This has led to many missed melanomas despite a growing epidemic of skin biopsies. The number of unnecessary biopsies varies by clinical setting, the expertise of the examiner, and the application of technology. For example, in children in whom melanoma rates are low and changing moles are common, there are over 500,000 biopsies a year to diagnose approximately 400 melanomas in the U.S. each year.
The need to improve the efficiency, effectiveness, and accuracy of melanoma diagnosis is clear. The personal and financial costs of failing to diagnose melanoma early are considerable. On the other hand, inexpert screening for melanoma can lead to numerous unnecessary biopsies. Another concern with increased screening for melanoma is the diagnosis and treatment of indolent lesions that cannot currently be distinguished from potentially lethal cancers. This latter scenario is referred to as ‘overdiagnosis’ and raises concerns about democratizing access to automated mobile diagnostic systems in populations at low risk of dying from melanoma.
Digital imaging and AI for melanoma diagnosis
The general availability of mobile smartphone devices has also brought digital imaging for melanoma diagnosis into the realm of telemedicine. While already on the rise prior to the COVID pandemic, teledermatology volume has recently increased exponentially. The opportunity for direct to consumer tele-consultation for concerning lesions holds great promise for improving access to care, but the often-poor quality and inconsistency of patient-acquired images pose a considerable challenge.
While consumer apps to educate and aid in melanoma detection are an exciting and promising new development, the first-generation apps have been deployed with limited, if any, clinical testing. The availability of inaccurate self-diagnosis apps may lead to delay of consultating with a clinician. Indeed, multiple studies of the accuracy of these apps endorse this concern. As the apps have been trained on data from narrow demographic segments of the population, there is also a concern that their accuracy will be far lower in the broader population.
There has been little oversight of the clinical use of digital images in dermatology. Despite being a visual specialty, there are no requirements of dermatologists to use images to document clinical encounters, let alone to ensure the quality and consistency of such documentation. The cameras and imaging systems used in clinical practice are deemed inherently safe, and no minimal technical requirements have been established. There has, however, been considerable attention given to automated diagnosis. As of February 9th, 2015, the FDA (Food and Drug Administration) assumed responsibility for regulation of consumer diagnostic apps and on February 23, 2015 the FTC (Federal Trade Commission) fined app makers for false advertising. In 2019, the FDA released a discussion paper proposing a regulatory framework for modifications to AI/ML-based SaMD (Software as a Medical Device). On May 25, 2017, the EU released Regulation 2017/745 on medical devices (MDR) and on February 19, 2020, the European Commission published a White Paper to promote a European ecosystem of excellence and trust in AI, a data strategy communication, and a report on the safety and liability aspects of AI.
About ISIC
Goals of ISIC
The primary clinical goal of ISIC is to support efforts to reduce melanoma-related deaths and unnecessary biopsies by improving the accuracy and efficiency of melanoma early detection. To this end, ISIC is developing proposed digital imaging standards and engages the dermatology and computer science communities to improve diagnostic accuracy with the aid of AI. While the initial focus of ISIC is on melanoma, the goals being pursued by ISIC are critical to advancing the broader landscape of skin imaging and artificial intelligence in dermatology; including non-melanoma skin cancer and inflammatory dermatoses.
The proliferation of dermatologic imaging has evolved without benefit of dermatology specific DICOM (Digital Image Communication in Medicine) standards that exist for other medical specialties. This is due in part due to limited dedicated dermatologic imaging industry, as most dermatologic photography can be accomplished with off-the-shelf equipment. The more recent mobile technology era has further complicated the ability to set technical standards that can keep up with the rapid change in the technologies. In both the medical and public realms, the optimal use of images for education, diagnosis, tele-diagnosis, patient triage, mole monitoring, documentation, and coordination of care will all benefit from the development of standards to help to ensure Quality, Privacy, and Interoperability.
Quality: While there may be subtle differences in the quality needed for different applications (e.g., diagnosis v. documentation), images failing to meet minimal quality (e.g., spatial resolution, color accuracy, clarity of focus) will undermine their clinical utility.
Privacy: As in all other medical applications, standards are needed to ensure the privacy of the personal health information associated with dermatology images. The nudity associated with overview dermatology images adds an additional privacy concern.
Interoperability: To optimize the usefulness of dermatologic images, standards are needed to ensure that images and their associated metadata can be effectively shared between systems, patients, and providers.ISIC is working with stakeholders to address each of the following domains:
Technology Standards: The aim of the technology standards group is to promote the adoption of the Digital Imaging and Communication in Medicine (DICOM) standard for dermatology imaging. DICOM is the standard for medical imaging and is used extensively in other medical image producing specialites such as radiology, cardiology, and oncology. Development of medical imaging standards is done under the auspices of the DICOM Standards Committee. DICOM is used to standardize metadata, image file formats, interoperability between imaging devices regardless of manufacturer, interoperability with digital health infrastructure that exists in many health care organisations (e.g., EMRs, PACS). The adoption of standards is key to improving both the utility and quality of imaging in dermatology. Further, the adoption of standards is likely to accelerate the adoption of AI in clinical dermatology.
Technique Standards: Harmonizing approaches to patient and lesion identification, documentation of lesion attributes, and image acquisition (e.g., poses, orientation, lighting, magnification, use of size and color calibration markers, etc.) will help ensure image quality, data integrity, and image comparability.
Terminology & Metadata Standards: Considerable variability exists in the application of specific terms to describe anatomic sites, diagnoses, and clinical, dermoscopic, and histologic morphology. A lexicon of standardized terminology will facilitate teaching, content-based image retrieval, and consistent ground truth for training and implementing AI algorithms.
Privacy: The aim of the privacy working group is to develop privacy frameworks that help ensure maintenance of confidentiality and appropriate consent for use and sharing of images and their associated metadata in dermatology imaging. Dermatology imaging has unique privacy issues that are not addressed by existing health privacy frameworks. Namely, total body photography leaves the subject visually identifiable due the presence of their face, distinctive skin lesions or tattoos. These images are also sensitive images in that they contain nudity and show the patient’s body habitus. Privacy frameworks that provide guidance to clinicians and researchers on the appropriate use dermatology imaging with respect to patient confidentiality are needed. These frameworks must comply with legal, regulatory, and ethical standards. Furthermore, they must meet societal and consumer expectations on how dermatology imaging is used.ISIC is creating resources for the dermatology and computer science communities, including a large and expanding open source public access Archive of skin images. This Archive serves as a public resource of images for teaching, research, and for the development and testing of diagnostic artificial intelligence algorithms. ISIC is engaging the stakeholder communities through meetings, publications, conferences, and the hosting of artificial intelligence Grand Challenges.
ISIC Archive: ISIC has developed an open source platform for the contribution of images of skin lesions under Creative Commons licenses. The images are associated with ground-truth diagnoses and other clinical metadata and available for use in the public domain. Images can be queried using faceted search and downloaded individually or in batches. The initial focus of the Archive has been on dermoscopic images of individual skin lesions, as these images are inherently standardized by the use of a specialized acquisition devices and devoid of many of the privacy challenges associated with clinical images. To date, the images have been provided by specialized melanoma centers from around the world. The Archive is designed to accept contributions from new sources under the ISIC Terms of Use agreement and welcomes new contributors. There are ongoing efforts to supplement the dermoscopic images in the Archive with close-up clinical images and a broader representation of skin types.
Grand Challenges: Beginning in 2016, ISIC has sponsored annual challenges for the computer science community in association with leading computer vision conferences. Over the years, these challenges have grown in scale, complexity, and participation. The earlier challenges were focused primarily on diagnostic accuracy for distinguishing melanoma from other benign and malignant skin lesions. By 2018, the diagnostic performance of the leading algorithms was consistently outperforming clinicians in reader studies. Additional challenges in 2019 and 2020 were designed to address the out-of-distribution problem and assess the impact of clinical context, respectively. The 2020 challenge had over 3,300 participating teams from around the world. In addition to the annual Grand Challenges, ISIC hosts live challenges that allow researchers and students to benchmark the performance of their algorithms using ISIC images on an ongoing basis.
Publications: ISIC Working Groups publish the results of their efforts in clinical and scientific journals and have hosted several special issues in prominent imaging journals. To date, there have been over 1,000 publications in the scientific literature that derive directly from ISIC or rely on ISIC data.
Meetings, Conferences, and Workshops: The Working Groups meet individually on a regular basis and the entire collaboration holds regular meetings twice a year. ISIC regularly hosts workshops on skin imaging at the leading vision science meetings including ISBI, MICCAI, and CVPR and major clinical meetings including the American Academy of Dermatology Annual Meeting. In conjunction with ISIC, Memorial Sloan Kettering Cancer Center holds an annual conference titled Transforming Dermatology in the Digital Era.