Empirical Analysis on User Profile in Personalized LLMs

Abstract

Utilizing user profiles to personalize Large Language Models (LLMs) has been shown to enhance performance on a wide range of tasks. However, the precise role of user profiles and their effect mechanism on LLMs is unclear. This study first confirms that the effectiveness of user profiles stems primarily from their personalization information, with input-relevant information contributing meaningfully only when built upon personalization. Furthermore, we investigate how user profiles affect the personalization of LLMs. Within the user profile, we reveal that it is the historical personalized response produced or approved by users that plays a pivotal role in personalizing LLMs. This discovery unlocks the potential of LLMs to incorporate more user profiles within the constraints of limited input length. As for the position of user profiles, we observe that user profiles integrated into different positions of the input context do not contribute equally to personalization. Instead, user profiles closer to the beginning have more impact on the personalization of LLMs. Our findings reveal the role of user profiles for the personalization of LLMs, and showcase how incorporating user profiles impacts performance to leverage user profiles effectively.

Ähnliche Arbeiten