Advancements in Large Language Models for Human-Computer Interaction

The field of human-computer interaction is witnessing significant developments with the integration of large language models (LLMs). Recent research has focused on leveraging LLMs to enhance personalized interactions, improve task-oriented decision-making, and facilitate natural language understanding. A notable direction is the use of LLMs as design materials for interactive applications, such as museum installations and robot trainers, to create more engaging and dynamic user experiences. Additionally, researchers are exploring the potential of LLMs in addressing challenges like object hallucinations in vision-language models and developing cognitive architectures for co-constructive task learning. Noteworthy papers include Agent-Centric Personalized Multiple Clustering with Multi-Modal LLMs, which proposes a framework for personalized clustering using MLLMs, and DASH: Detection and Assessment of Systematic Hallucinations of VLMs, which introduces a pipeline for identifying systematic hallucinations in vision-language models. These advancements demonstrate the potential of LLMs to revolutionize human-computer interaction and enable more effective, personalized, and adaptive interfaces.

Sources

Agent-Centric Personalized Multiple Clustering with Multi-Modal LLMs

Using a Large Language Model as Design Material for an Interactive Museum Installation

A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks

DASH: Detection and Assessment of Systematic Hallucinations of VLMs

Re-Aligning Language to Visual Objects with an Agentic Workflow

Towards a cognitive architecture to enable natural language interaction in co-constructive task learning

Plan-and-Act using Large Language Models for Interactive Agreement

Building Knowledge from Interactions: An LLM-Based Architecture for Adaptive Tutoring and Social Reasoning

Reasoning LLMs for User-Aware Multimodal Conversational Agents

A Memory-Augmented LLM-Driven Method for Autonomous Merging of 3D Printing Work Orders

Built with on top of