LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction
About
Large language models (LLMs) encode rich cultural knowledge learned from diverse web-scale data, offering an unprecedented opportunity to model cultural commonsense at scale. Yet this knowledge remains mostly implicit and unstructured, limiting its interpretability and use. We present an iterative, prompt-based framework for constructing a Cultural Commonsense Knowledge Graph (CCKG) that treats LLMs as cultural archives, systematically eliciting culture-specific entities, relations, and practices and composing them into multi-step inferential chains across languages. We evaluate CCKG on five countries with human judgments of cultural relevance, correctness, and path coherence. We find that the cultural knowledge graphs are better realized in English, even when the target culture is non-English (e.g., Chinese, Indonesian, Arabic), indicating uneven cultural encoding in current LLMs. Augmenting smaller LLMs with CCKG improves performance on cultural reasoning and story generation, with the largest gains from English chains. Our results show both the promise and limits of LLMs as cultural technologies and that chain-structured cultural knowledge is a practical substrate for culturally grounded NLP.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Multiple-choice Question Answering | IndoCulture native prompts (test) | Accuracy67.5 | 99 | |
| Multiple-choice Question Answering | ArabCulture 1.0 (test) | Accuracy59.6 | 84 | |
| Story Generation | Native Story Generation China (native) | Cultural Relevance8.6 | 27 | |
| Story Generation | Native Story Generation Indonesia | Cultural Relevance8.5 | 27 | |
| Story Generation | Native Story Generation Egypt | Cultural Relevance6.9 | 27 | |
| Sentence Completion | IndoCulture | Sentence Similarity Score43 | 18 | |
| Sentence Completion | ArabCulture | Sentence Similarity Score42.5 | 18 | |
| Sentence Completion | ArabCulture native prompts (test) | Avg Sentence Similarity44.7 | 18 | |
| Sentence Completion | IndoCulture native prompts (test) | Avg Sentence Similarity42 | 18 | |
| Multiple-choice Question Answering | ArabCulture native prompts (test) | Accuracy47.9 | 10 |