ZTab: Domain-based Zero-shot Annotation for Table Columns
About
This study addresses the challenge of automatically detecting semantic column types in relational tables, a key task in many real-world applications. Zero-shot modeling eliminates the need for user-provided labeled training data, making it ideal for scenarios where data collection is costly or restricted due to privacy concerns. However, existing zero-shot models suffer from poor performance when the number of semantic column types is large, limited understanding of tabular structure, and privacy risks arising from dependence on high-performance closed-source LLMs. We introduce ZTab, a domain-based zero-shot framework that addresses both performance and zero-shot requirements. Given a domain configuration consisting of a set of predefined semantic types and sample table schemas, ZTab generates pseudo-tables for the sample schemas and fine-tunes an annotation LLM on them. ZTab is domain-based zero-shot in that it does not depend on user-specific labeled training data; therefore, no retraining is needed for a test table from a similar domain. We describe three cases of domain-based zero-shot. The domain configuration of ZTab provides a trade-off between the extent of zero-shot and annotation performance: a "universal domain" that contains all semantic types approaches "pure" zero-shot, while a "specialized domain" that contains semantic types for a specific application enables better zero-shot performance within that domain. Source code and datasets are available at https://github.com/hoseinzadeehsan/ZTab
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Column Type Annotation | Efthymiou Cross-Domain | F1 Score88.9 | 14 | |
| Column Type Annotation | Limaye Cross-Domain | F1 Score92.4 | 14 | |
| Column Type Annotation | T2D Cross-Domain | F1 Score96.2 | 14 | |
| Semantic Type Annotation | T2D | Micro-F197.7 | 12 | |
| Semantic Type Annotation | WikiTable | Micro F1 Score48.9 | 12 | |
| Semantic Type Annotation | SOTABsch | Micro-F10.792 | 12 | |
| Semantic Type Annotation | SOTAB sch-s | Micro-F178.9 | 12 | |
| Semantic Type Annotation | SOTAB dbp | Micro-F179.3 | 12 | |
| Column Type Annotation | SOTABdbp Cross-Ontology | F1 Score76.2 | 10 | |
| Column Type Annotation | SOTABsch Cross-Ontology | F1 Score74.2 | 10 |