Advances in Large Language Models for Information Retrieval and Question Answering

The field of natural language processing is witnessing significant advancements in the development of large language models (LLMs) for information retrieval and question answering tasks. Recent research has focused on improving the efficiency and effectiveness of LLMs in handling long documents, complex queries, and structured data. Notably, single-pass document scanning approaches have shown promise in reducing computational costs while preserving global context. Additionally, the use of LLMs to enrich retrieval indices offline has demonstrated significant improvements in recall and NDCG metrics. Furthermore, research has explored the application of LLMs in causal retrieval, sequential information extraction, and utility-focused annotation, highlighting the potential of these models in advancing the field. Some noteworthy papers in this area include: Single-Pass Document Scanning for Question Answering, which proposes a single-pass approach to question answering that outperforms chunk-based embedding methods and competes with large language models at a fraction of the computational cost. EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline, which introduces a retrieval approach that uses LLMs offline to build semantically-enriched retrieval indices, resulting in significant improvements in recall and NDCG metrics.

Sources

Single-Pass Document Scanning for Question Answering

EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline

Causal Retrieval with Semantic Consideration

Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts

Can LLMs Interpret and Leverage Structured Linguistic Representations? A Case Study with AMRs

Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling

Leveraging LLMs for Utility-Focused Annotation: Reducing Manual Effort for Retrieval and RAG

LLM$\times$MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources

Confidence Regularized Masked Language Modeling using Text Length

From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models

Query Understanding in LLM-based Conversational Information Seeking

Bridging Queries and Tables through Entities in Table Retrieval

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables

Built with on top of