The field of natural language processing is witnessing significant advancements in the development and application of Large Language Models (LLMs) for psychological and social applications. Researchers are exploring the potential of LLMs to annotate emotion appraisals, predict human attitudes, and evaluate psychological traits. The use of LLMs in these areas has shown promising results, with some models performing close to or even better than human annotators. However, challenges remain in ensuring the reliability and validity of LLMs in these applications, particularly in preserving sentiment and semantic integrity. Noteworthy papers in this area include: Assessing the Reliability and Validity of GPT-4 in Annotating Emotion Appraisal Ratings, which demonstrates the effectiveness of GPT-4 in annotating appraisal ratings and predicting emotion labels. Leveraging Implicit Sentiments: Enhancing Reliability and Validity in Psychological Trait Evaluation of LLMs, which introduces a novel evaluation instrument called Core Sentiment Inventory (CSI) for assessing LLMs' psychological traits.
Advances in Large Language Models for Psychological and Social Applications
Sources
Assessing the Reliability and Validity of GPT-4 in Annotating Emotion Appraisal Ratings
Emotional Multifaceted Feedback on AI Tool Use in EFL Learning Initiation: Chain-Mediated Effects of Motivation and Metacognitive Strategies in an Optimized TAM Model
Leveraging Implicit Sentiments: Enhancing Reliability and Validity in Psychological Trait Evaluation of LLMs
From Annotation to Adaptation: Metrics, Synthetic Data, and Aspect Extraction for Aspect-Based Sentiment Analysis with Large Language Models
Can Large Language Models Predict Associations Among Human Attitudes?
An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses