51291 Utilizing Style-Transfer and Text-to-image Generation to Debias Melanoma Image Datasets

Timely and precise diagnosis is important in melanoma cases due to a significant proportion of skin cancer fatalities in the United States. Machine learning models that use clinical, genetic, and imaging data present a promising avenue for achieving accurate melanoma classification. However, the potential for bias, specifically with classification of melanoma on skin of color (SOC), poses challenges to their reliability. A potential solution to limited diverse imaging data and the absence of Fitzpatrick skin typing is deep learning style-transfer techniques alongside a text-to-image generative artificial intelligence (AI) framework. Imaging data of 2357 melanoma lesions of light skin background was collected from the International Skin Imaging Collaboration (ISIC). SOC skin imaging was web-scraped. A style-transfer model consisting of two convolutional neural networks (CNNs) was built. The model combined lesion characteristics with diverse SOC backgrounds and resulted in a 44% testing accuracy in classification models detecting melanoma from the generated images. Further model refinement is necessary. Despite the potential, further refinements in image realism and validation are warranted. By augmenting style transfer with text-to-image generation algorithms such as OpenAI's DALL-E 2, precise textual descriptions of lesion morphology or location could enhance visualizations. It is crucial, however, to acknowledge that this technology, while a step forward in addressing bias and diversifying training data, complements rather than substitutes medical expertise and emphasizes the holistic nature of effective healthcare delivery.