Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

A Domain-Adapted Pipeline for Structured Information Extraction from Police Incident Announcements on Social Media

About

Structured information extraction from police incident announcements is crucial for timely and accurate data processing, yet presents considerable challenges due to the variability and informal nature of textual sources such as social media posts. To address these challenges, we developed a domain-adapted extraction pipeline that leverages targeted prompt engineering with parameter-efficient fine-tuning of the Qwen2.5-7B model using Low-Rank Adaptation (LoRA). This approach enables the model to handle noisy, heterogeneous text while reliably extracting 15 key fields, including location, event characteristics, and impact assessment, from a high-quality, manually annotated dataset of 4,933 instances derived from 27,822 police briefing posts on Chinese Weibo (2019-2020). Experimental results demonstrated that LoRA-based fine-tuning significantly improved performance over both the base and instruction-tuned models, achieving an accuracy exceeding 98.36% for mortality detection and Exact Match Rates of 95.31% for fatality counts and 95.54% for province-level location extraction. The proposed pipeline thus provides a validated and efficient solution for multi-task structured information extraction in specialized domains, offering a practical framework for transforming unstructured text into reliable structured data in social science research.

Mengfan Shen, Kangqi Song, Xindi Wang, Wei Jia, Tao Wang, Ziqiang Han• 2025

Related benchmarks

TaskDatasetResultRank
Boolean ClassificationPolice incident dataset
Accuracy (Death)97.38
7
Criminal methods extractionEMR dataset
Cosine Similarity63.14
7
Numerical Information ExtractionDisaster Report Information Extraction Dataset
Deaths0.9628
7
Police handling extractionEMR dataset
Cosine Similarity65.51
7
Case type classificationEMR dataset
Jaccard Similarity0.8133
7
Geographic Information ExtractionDisaster Report Information Extraction Dataset
Province Accuracy0.9443
7
Boolean classification of Case closureCriminal analysis dataset
Accuracy0.9439
6
Boolean classification of Crime successCriminal analysis dataset
Accuracy0.9578
6
Boolean classification of InjuryCriminal analysis dataset
Accuracy93.64
6
Information ExtractionEMR (test)
Jaccard (Case type)82.55
6
Showing 10 of 15 rows

Other info

Follow for update