Mitigating Biases and Enhancing Calibration in Vision-Language and Large Language Models

The recent advancements in the research area of vision-language models and large language models (LLMs) have primarily focused on addressing biases, improving calibration, and enhancing the models' ability to handle ambiguity and uncertainty. A significant trend is the development of methods to mitigate biases in these models, with approaches ranging from task arithmetic to targeted stereotype mitigation frameworks. These efforts aim to reduce stereotypical outputs and improve the fairness and inclusivity of AI systems. Additionally, there is a growing emphasis on the calibration of these models, particularly in multimodal settings, to ensure their outputs are reliable and trustworthy. Researchers are also exploring ways to enhance the models' ability to handle referential ambiguity and uncertainty, which is crucial for effective communication and decision-making. Notably, some studies have highlighted the risks of overconfidence and the need for robust strategies to deal with uncertainty without resorting to undesirable stereotypes. Overall, the field is moving towards more responsible and reliable AI systems, with a strong focus on addressing ethical concerns and improving model performance.

Sources

Vision-Language Models Represent Darker-Skinned Black Individuals as More Homogeneous than Lighter-Skinned Black Individuals

Generics are puzzling. Can language models find the missing piece?

Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flaws

Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approach

SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibration

Trigger$^3$: Refining Query Correction via Adaptive Model Selector

A Survey of Calibration Process for Black-Box LLMs

RACQUET: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs

Race Discrimination in Internet Advertising: Evidence From a Field Experiment

Memorization Over Reasoning? Exposing and Mitigating Verbatim Memorization in Large Language Models' Character Understanding Evaluation

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models

Built with on top of