Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Fix the Structural Bottleneck: Context Compression via Explicit Information Transmission

About

Long-context LLM agents often struggle with growing token, memory, and latency costs, making efficient context compression essential for practical deployment. Existing LLM-as-a-compressor methods remain noticeably inferior to using the full context. We find that this gap partly stems from their inability to preserve contextual information effectively. In this work, we revisit context compression from a structural perspective and identify two key bottlenecks in standard LLM-based compressors: limited coordination among compression tokens during information aggregation, and layerwise dilution that weakens useful signals from intermediate hidden states. To address these limitations, we propose ComprExIT, a new context compression framework based on explicit information transmission. ComprExIT adaptively selects features across frozen LLM layers, then allocates information from anchors to compression slots through a globally coordinated transport plan. Experiments on 12 datasets show that ComprExIT consistently outperforms strong soft-compression baselines, improving average F1 by up to 18.5%, while adding only ~1% trainable parameters and achieving more than 2x faster compression than the fastest baselines. The code will be released upon acceptance.

Jiangnan Ye, Hanqi Yan, Zhenyi Shen, Heng Chang, Ye Mao, Yulan He• 2026

Related benchmarks

TaskDatasetResultRank
Question AnsweringSQuAD
F175.68
162
Question AnsweringHotpotQA
F174.15
132
Question AnsweringSearchQA
EM78
46
Question AnsweringTriviaQA
F178.86
46
Question AnsweringTextbookQA MRQA out-of-domain evaluation
EM56.09
37
Question AnsweringRelExt MRQA out-of-domain evaluation
EM65.31
37
Reading ComprehensionDROP MRQA out-of-domain
EM0.4358
14
Question AnsweringNQ
EM52.23
14
Reading ComprehensionBioASQ MRQA out-of-domain
EM59.64
14
Reading ComprehensionDuoRC MRQA out-of-domain
EM34.31
14
Showing 10 of 11 rows

Other info

Follow for update