A Unified Framework for Evaluating and Enhancing the Transparency of Explainable AI Methods via Perturbation-Gradient Consensus Attribution
About
Explainable Artificial Intelligence (XAI) methods are increasingly used in safety-critical domains, yet there is no unified framework to jointly evaluate fidelity, interpretability, robustness, fairness, and completeness. We address this gap through two contributions. First, we propose a multi-criteria evaluation framework that formalizes these five criteria using principled metrics: fidelity via prediction-gap analysis; interpretability via a composite concentration-coherence-contrast score; robustness via cosine-similarity perturbation stability; fairness via Jensen-Shannon divergence across demographic groups; and completeness via feature-ablation coverage. These are integrated using an entropy-weighted dynamic scoring scheme that adapts to domain-specific priorities. Second, we introduce Perturbation-Gradient Consensus Attribution (PGCA), which fuses grid-based perturbation importance with Grad-CAM++ through consensus amplification and adaptive contrast enhancement, combining perturbation fidelity with gradient-based spatial precision. We evaluate across five domains (brain tumor MRI, plant disease, security screening, gender, and sunglass detection) using fine-tuned ResNet-50 models. PGCA achieves the best performance in fidelity $(2.22 \pm 1.62)$, interpretability $(3.89 \pm 0.33)$, and fairness $(4.95 \pm 0.03)$, with statistically significant improvements over baselines $(p < 10^{-7})$. Sensitivity analysis shows stable rankings (Kendall's $(\tau \geq 0.88)$). Code and results are publicly available.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Explainable AI (XAI) performance evaluation | Aggregated across five domains (Healthcare, Agriculture, Security, Gender Detection, and Sunglass Detection) (test) | Fidelity2.22 | 5 | |
| Explainable AI Performance Evaluation | Agriculture | Composite Score (Entropy-Weighted, Domain-Modulated)4.11 | 5 | |
| Explainable AI Performance Evaluation | Security | Composite Score (Entropy-Weighted, Domain-Modulated)2.98 | 5 | |
| Explainable AI Performance Evaluation | Healthcare Brain Tumor MRI | Composite Score (Entropy-Weighted, Domain-Modulated)4.32 | 5 | |
| Explainable AI Performance Evaluation | Gender | Composite Score (Entropy-Weighted, Domain-Modulated)4.22 | 5 | |
| Explainable AI Performance Evaluation | Sunglass | Composite Score (Entropy-Weighted, Domain-Modulated)4.51 | 5 |