38. The Use of Multimodal AI for Facial Analysis in Plastic Surgery

Abstract

BACKGROUND: The face is the most aesthetically sensitive area of the body, making detailed analysis crucial for facial plastic surgery. Facial analysis evaluates facial symmetry and proportions, comparing them to aesthetic standards such as the ‘Golden Ratio.’ With the rise in popularity of facial surgery, there’s a growing need for objective and efficient methods, as manual techniques can be subjective, error-prone, and time-consuming. Advances in Multimodal Artificial Intelligence and Large Language Models (LLMs) offer new possibilities for accurate facial analysis. The purpose of our study is to evaluate the effectiveness of multimodal LLMs for facial analysis in plastic surgery. METHODS: We tested four multimodal LLMs—ChatGPT-4o, ChatGPT-4, Gemini 1.5 Pro, and Claude 3.5 Sonnet—for their capability to analyze facial skin quality, volume, symmetry, and adherence to the golden ratio, using two evaluation forms. We utilized 15 facial images generated by a Generative Adversarial Network (GAN) representing diverse ages and ethnicities. The general analysis form evaluated skin characteristics such as texture, type, thickness, wrinkling, photoaging, and overall symmetry. The facial ratios form assessed structural proportions, including division into equal fifths, adherence to the rule of thirds, and compatibility with the golden ratio. The LLM assessments were compared with evaluations made by a plastic surgeon and manual measurements of the facial ratios in the images. RESULTS: The LLMs displayed variable performance in facial analysis, generally performing better in skin/volume analysis than in facial ratio assessments. Mean scores for skin analysis were: ChatGPT-4o (0.651 ± 0.478), Gemini 1.5 Pro (0.641 ± 0.481), ChatGPT-4 (0.615 ± 0.488), and Claude 3.5 Sonnet (0.544 ± 0.499). In facial ratio assessments, the scores were lower, with Gemini 1.5 Pro achieving the highest mean score (0.351). Inter-rater reliability ranged from poor to substantial based on Cohen’s kappa values (-0.410 to 1.000), indicating inconsistent agreement levels among models’ performances across various faces. CONCLUSION: This study highlights the potential and limitations of LLMs in facial analysis for plastic surgery. While LLMs show promise in assessing qualitative features such as skin characteristics, symmetry, and volume, they currently lack precision in manual measurements of facial ratios. The variability in their performance underscores the need for further refinement. Although not yet capable of replacing traditional methods, LLMs could serve as valuable tools in specific areas of facial analysis. Future research should focus on improving their numerical accuracy and reliability through enhanced training methods and integration with other AI technologies, paving the way for more objective and efficient facial analyses in clinical settings.

Ähnliche Arbeiten