Enhancing LLM Performance Through Data Augmentation and Multi-Task Learning

The recent developments in the research area primarily focus on enhancing the performance and adaptability of large language models (LLMs) through innovative data augmentation and fine-tuning strategies. A significant trend is the use of multi-hop data expansion and attribute-guided synthesis to generate high-quality, diverse training data for specific tasks, which has shown to improve model accuracy across various benchmarks. Additionally, there is a growing emphasis on optimizing multi-task learning frameworks to improve model generalization and task collaboration, with notable advancements in balancing different tasks within a single model. Another key area of progress is the enhancement of data quality for text classification through selective fine-tuning and classification of data into uncovered, difficult, and noisy categories, leading to improved training efficiency and performance. Furthermore, soft prompt fine-tuning strategies are being developed to better adapt LLMs to domain-specific Automatic Speech Recognition (ASR) tasks, resulting in significant reductions in error rates. Lastly, there is a focus on improving the robustness of Natural Language Inference (NLI) models through data augmentation and preprocessing techniques designed to address semantic understanding challenges. Notably, some papers stand out for their innovative approaches: 'AIDE: Task-Specific Fine Tuning with Attribute Guided Multi-Hop Data Expansion' introduces a novel framework for data synthesis that ensures both task relevance and diversity, and 'UnitedSynT5 for Few-Shot NLI' leverages synthetic data augmentation to set new benchmarks in NLI accuracy.

Sources

Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud

AIDE: Task-Specific Fine Tuning with Attribute Guided Multi-Hop Data Expansion

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models

Data Quality Enhancement on the Basis of Diversity with Large Language Models for Text Classification: Uncovered, Difficult, and Noisy

Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning

Improving the Natural Language Inference robustness to hard dataset by data augmentation and preprocessing

Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation

Advancing Single- and Multi-task Text Classification through Large Language Model Fine-tuning

BDA: Bangla Text Data Augmentation Framework

First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI

Built with on top of