Towards Responsible AI: Evaluating Large Language Models Across Trustworthy Dimensions

Abstract

Since the boom of Large Language Models (LLMs), there has been an increase in the integration of these models into real-world applications to boost the efficiency of the work. This raises an urgent question about their trustworthiness across properties, such as truthfulness, safety, robustness, privacy, fairness and ethics. While prior work has been done analyzing and benchmarking individual properties, a comprehensive and standardized trust evaluation remains limited, especially in open-source models. This study systematically evaluates trustworthiness in six state-of-the-art open-source large language models, ranging from 1 billion to 10.7 billion parameters. Utilizing the TrustLLM framework, this study benchmarks models in different properties in over 20 different tasks using unified metrics designed for both generative and classified tasks. The results have uncovered notable disparities among models by revealing that improvements in scale do not uniformly translate to better trustworthiness. It also pointed out that no model was uniformly better in all of the properties of trustworthiness. Thus, this study further reinforces the need for a holistic evaluation of LLMs’ trustworthines.

Ähnliche Arbeiten