Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment

About

Template-based contrastive synthesis is scalable, but its candidates often differ only in a few entity-slots while sequence-level optimization spreads supervision over mostly shared templates. We formalize this as the Resolution Mismatch Problem and propose KARMA, which enumerates schema-constrained paths over domain knowledge graphs and verbalizes them into slot-aligned contrastive candidates. Slot-Parallel Alignment (SPA) then applies a decoupled slot-level objective to route preference supervision to discriminative entity-slots, with slot-aware masked attention serving as an optional packed-evaluation implementation. Across biomedical, computer-science, and chemistry benchmarks, KARMA outperforms base LLM and same-data SFT baselines, and compares favorably with sequence and token-level preference methods.

Jinkyeong Choi, Chaebin Jeong, Donghyeon Park• 2026

Related benchmarks

TaskDatasetResultRank
Medical ReasoningPubMedQA
Accuracy52.2
48
Domain ReasoningMedQA Biomedical
Accuracy54.1
4
Domain ReasoningMMLU Biomedical
Accuracy70.2
4
Domain ReasoningMMLU-Pro Biomedical
Accuracy (MMLU-Pro Biomedical)53.2
4
Domain ReasoningMMLU CS (Computer Science)
MMLU CS Domain Reasoning Accuracy70.1
4
Domain ReasoningMMLU-Pro Computer Science
MMLU-Pro CS Domain Reasoning Accuracy25.6
4
Domain ReasoningMMLU Chemistry
Accuracy (MMLU Chemistry Domain Reasoning)63
4
Domain ReasoningMMLU-Pro Chemistry
Accuracy19.5
4
Domain ReasoningGPQA-Chem
Accuracy20.4
4
Domain ReasoningGPQA Bio
Accuracy (GPQA Bio)66.7
4
Showing 10 of 11 rows

Other info

Follow for update