<ul><li>Researchers introduce MMIE, a large-scale benchmark for evaluating multimodal comprehension and generation in Large Vision-Language Models (LVLMs).</li><li>MMIE consists of 20K curated multimodal queries covering various categories and subfields.</li><li>The benchmark supports interleaved inputs and outputs, evaluating competencies through multiple-choice and open-ended questions.</li><li>An automated evaluation metric with reduced bias and improved accuracy is proposed.</li></ul>

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

Discover more