Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Towards Corpus-Grounded Agentic LLMs for Multilingual Grammatical Analysis

About

Empirical grammar research has become increasingly data-driven, but the systematic analysis of annotated corpora still requires substantial methodological and technical effort. We explore how agentic large language models (LLMs) can streamline this process by reasoning over annotated corpora and producing interpretable, data-grounded answers to linguistic questions. We introduce an agentic framework for corpus-grounded grammatical analysis that integrates concepts such as natural-language task interpretation, code generation, and data-driven reasoning. As a proof of concept, we apply it to Universal Dependencies (UD) corpora, testing it on multilingual grammatical tasks inspired by the World Atlas of Language Structures (WALS). The evaluation spans 13 word-order features and over 170 languages, assessing system performance across three complementary dimensions - dominant-order accuracy, order-coverage completeness, and distributional fidelity - which reflect how well the system generalizes, identifies, and quantifies word-order variations. The results demonstrate the feasibility of combining LLM reasoning with structured linguistic data, offering a first step toward interpretable, scalable automation of corpus-based grammatical inquiry.

Matej Klemen, Tja\v{s}a Ar\v{c}on, Luka Ter\v{c}on, Marko Robnik-\v{S}ikonja, Kaja Dobrovoljc• 2025

Related benchmarks

TaskDatasetResultRank
Dominant Word Order PredictionUniversal Dependencies Feature 81A
Accuracy93.5
3
Dominant Word Order PredictionUniversal Dependencies Feature 82A
Accuracy98.6
3
Dominant Word Order PredictionUniversal Dependencies Feature 83A
Accuracy95.5
3
Dominant Word Order PredictionUniversal Dependencies Feature 84A
Accuracy82.2
3
Dominant Word Order PredictionUniversal Dependencies Feature 85A
Accuracy98.5
3
Dominant Word Order PredictionUniversal Dependencies Feature 87A
Accuracy97.6
3
Dominant Word Order PredictionUniversal Dependencies Feature 88A
Accuracy100
3
Dominant Word Order PredictionUniversal Dependencies Feature 94A
Accuracy93.6
3
Dominant Word Order PredictionUniversal Dependencies Feature 86A
Accuracy95.5
3
Dominant Word Order PredictionUniversal Dependencies Feature 89A
Accuracy93.1
3
Showing 10 of 13 rows

Other info

Follow for update