Evaluating Large Language Models for Multilingual Abstractive Summarization: A Comprehensive Study of English and Bengali
Author1, Author2, Author3, Author4, Author5
Abstract
This study evaluates large language models for multilingual abstractive summarization, focusing on English and Bengali, highlighting the challenges in linguistic quality and factual consistency.
Reality Card
The evaluation reveals that strong English summarization performance does not necessarily transfer to Bengali, emphasizing the need for comprehensive evaluation frameworks for low-resource languages.
GPT-5.6 Luna leads several English lexical metrics but LLaMA 3.3 70B achieves the highest ROUGE and BLEU scores in Bengali.
Different evaluation methods produce varying model rankings, indicating limitations in relying on a single metric for assessment.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.