SAFE-PE, A Systematic Assessment Framework for Evaluating Prompt Engineering in Generative AI

Abstract

Prompt engineering is emerging as an essential tradition to use generative AI in such domains as software, learning, health, and creativity. However, the field is yet to have a clear framework on assessing prompt quality, reliability and reproducibility. The comparisons and best practices are complicated by the fact that current efforts are most likely to be based on trial-and-error or task-specific benchmark. We offer A Systematic Assessment Framework for Evaluating Prompt Engineering (SAFE-PE), which implies standard measures and principles, including multi-dimensional evaluation numbers. It puts alongside use of quantitative data (accuracy, diversity, robustness) with qualitative data (interpretability, fairness, ethics) in order to give a holistic picture of prompt performance. We show that the framework is effective through case studies of the large language models Meta AI (LLaMA) in summarization, question and answer, and code generation. SAFE-PE provides a systematic assessment procedure that progresses the timely engineering as a scientific field and enables practitioners to obtain a useful means of using generative AI in a responsible manner.

Ähnliche Arbeiten