Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

A Unified Framework for Evaluating and Enhancing the Transparency of Explainable AI Methods via Perturbation-Gradient Consensus Attribution

About

Explainable Artificial Intelligence (XAI) methods are increasingly used in safety-critical domains, yet there is no unified framework to jointly evaluate fidelity, interpretability, robustness, fairness, and completeness. We address this gap through two contributions. First, we propose a multi-criteria evaluation framework that formalizes these five criteria using principled metrics: fidelity via prediction-gap analysis; interpretability via a composite concentration-coherence-contrast score; robustness via cosine-similarity perturbation stability; fairness via Jensen-Shannon divergence across demographic groups; and completeness via feature-ablation coverage. These are integrated using an entropy-weighted dynamic scoring scheme that adapts to domain-specific priorities. Second, we introduce Perturbation-Gradient Consensus Attribution (PGCA), which fuses grid-based perturbation importance with Grad-CAM++ through consensus amplification and adaptive contrast enhancement, combining perturbation fidelity with gradient-based spatial precision. We evaluate across five domains (brain tumor MRI, plant disease, security screening, gender, and sunglass detection) using fine-tuned ResNet-50 models. PGCA achieves the best performance in fidelity $(2.22 \pm 1.62)$, interpretability $(3.89 \pm 0.33)$, and fairness $(4.95 \pm 0.03)$, with statistically significant improvements over baselines $(p < 10^{-7})$. Sensitivity analysis shows stable rankings (Kendall's $(\tau \geq 0.88)$). Code and results are publicly available.

Md. Ariful Islam, Md Abrar Jahin, M. F. Mridha, Nilanjan Dey• 2024

Related benchmarks

TaskDatasetResultRank
Explainable AI (XAI) performance evaluationAggregated across five domains (Healthcare, Agriculture, Security, Gender Detection, and Sunglass Detection) (test)
Fidelity2.22
5
Explainable AI Performance EvaluationAgriculture
Composite Score (Entropy-Weighted, Domain-Modulated)4.11
5
Explainable AI Performance EvaluationSecurity
Composite Score (Entropy-Weighted, Domain-Modulated)2.98
5
Explainable AI Performance EvaluationHealthcare Brain Tumor MRI
Composite Score (Entropy-Weighted, Domain-Modulated)4.32
5
Explainable AI Performance EvaluationGender
Composite Score (Entropy-Weighted, Domain-Modulated)4.22
5
Explainable AI Performance EvaluationSunglass
Composite Score (Entropy-Weighted, Domain-Modulated)4.51
5
Showing 6 of 6 rows

Other info

Follow for update