Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Models

About

Knowledge graphs (KGs) provide structured, verifiable grounding for large language models (LLMs), but current LLM-based systems commonly use KGs as auxiliary structures for text retrieval, leaving their intrinsic quality underexplored. In this work, we propose Wikontic, a multi-stage pipeline that constructs KGs from open-domain text by extracting candidate triplets with qualifiers, enforcing Wikidata-based type and relation constraints, and normalizing entities to reduce duplication. The resulting KGs are compact, ontology-consistent, and well-connected; on MuSiQue, the correct answer entity appears in 96% of generated triplets. On HotpotQA, our triplets-only setup achieves 76.0 F1, and on MuSiQue 59.8 F1, matching or surpassing several retrieval-augmented generation baselines that still require textual context. In addition, Wikontic attains state-of-the-art information-retention performance on the MINE-1 benchmark (86%), outperforming prior KG construction methods. Wikontic is also efficient at build time: KG construction uses less than 1,000 output tokens, about 3$\times$ fewer than AriGraph and $<$1/20 of GraphRAG. The proposed pipeline enhances the quality of the generated KG and offers a scalable solution for leveraging structured knowledge in LLMs.

Alla Chepurova, Aydar Bulatov, Mikhail Burtsev, Yuri Kuratov• 2025

Related benchmarks

TaskDatasetResultRank
Question AnsweringHotpotQA
EM46.4
173
Question AnsweringMuSiQue (test)
EM46.8
76
Question AnsweringMuSiQue
EM25
38
Question AnsweringHotpotQA (test)
EM64.5
18
Knowledge Graph Information RetentionMINE-1
MINE-1 Score86
17
Question AnsweringSpecsQA (test)
F1 (Factual Correctness)13.5
13
Question AnsweringSpecsQA
FC F113.5
13
Knowledge Graph ConstructionMuSiQue
Total Triples204
10
Knowledge Graph ConstructionHotpotQA
Total Triples Count117
10
Knowledge Graph ExtractionHotpotQA
Avg Edge Multiplicity1.05
8
Showing 10 of 11 rows

Other info

GitHub

Follow for update