<ul><li>In-Context Learning (ICL) enables Large Language Models (LLMs) to perform tasks without parameter updates by conditioning on provided demonstrations in the prompt.</li><li>ICL has limitations like sensitivity to demonstration order, context length constraints, and computational inefficiency, leading to the proposal of Adaptive Task Vectors (ATV) as a solution.</li><li>ATV is a framework that dynamically generates task vectors conditioned on each input query, enhancing adaptation and generalization capabilities for unseen tasks compared to previous vector-based approaches.</li><li>A theoretical analysis suggests that ATV is expressively equivalent to LoRA under equal rank budgets and more expressive than Prefix-Tuning, providing formal support for its representational advantage.</li></ul>

Adaptive Task Vectors for Large Language Models

Discover more