This compares generated explanations to ground truth data in order to quantify the performance of explainability techniques.

Approaches employed to address this issue involve having multiple annotators and reporting inter-annotator agreement or mean human performance, as well as evaluating the explanations at different granularities to account for disagreements on the precise value of the ground truth.

University of Michigan - Ann Arbor

The major categories of evaluation techniques include: informal examination of explanations, comparison to ground truth, human evaluation, etc.

Explanations are multi-faceted objects that can be evaluated on multiple aspects, such as fidelity, comprehensibility, and others. Therefore, understanding the target of the evaluation is important for interpreting the evaluation results.

Evaluation methods of XAI

Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). *A survey of the state of explainable AI for natural language processing*. arXiv preprint arXiv:2010.00711.

A Survey of the State of Explainable AI for Natural Language Processing

This typically takes the form of high-level discussions of how examples of generated explanations align with human intuition. 

This includes cases where the output of a single explainability approach is examined
in isolation, or when explanations are compared to those of other reference approaches.

Informal examination of explanations

Comparison to ground truth

A more direct way to assess the explanation quality is to ask humans to evaluate
the effectiveness of the generated explanations.

This has the advantage of avoiding the assumption that there is only one good explanation that could serve as ground truth, as well as sidestepping the need to measure similarity of explanations. It is important to have multiple annotators, report
inter-annotator agreement, and correctly deal with subjectivity and variance in the responses.

Learn Before

Related