Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). *A survey of the state of explainable AI for natural language processing*. arXiv preprint arXiv:2010.00711.

University of Michigan - Ann Arbor

Google

Black box techniques, such as deep learning models and the use of language embeddings as features can substantially advance model quality, while their obfuscation of the process by which a model arrives at its results can be problematic. Therefore, In the broader AI community, this growing understanding of the importance of explainability has created an emerging field called Explainable AI (XAI).

Explanations can be categorized along two main aspects:
- The first distinguishes whether the explanation is for an individual prediction (local) or the model’s prediction process as a whole (global).
- The second differentiates between the explanation emerging directly from the prediction process (selfexplaining) versus requiring post-processing (posthoc).

Explainable AI for Natural Language Processing

A Survey of the State of Explainable AI for Natural Language Processing

- Local and global: A local explanation provides information or justification for the model’s prediction on a specific input, while a global explanation provides similar justification by revealing how the model’s prediction process works, independently of any particular input.
- Self-Explaining and Post-Hoc: A self-explaining approach generates the explanation
at the same time as the prediction. In contrast, a post-hoc approach requires that an additional operation is performed after making predictions.

Categorization of explanations

There are five major explainability techniques that differ in the mechanisms they adopt to generate the raw mathematical justifications.

- Feature importance

- Surrogate model

- Example-driven

- Provenance-based

- Declarative induction

A complex NLP approach consisting of different components may employ more than one of these explainability techniques.

Explainability Techniques

The most common set of operations that are used to enable explainability:

- First-derivative saliency

- Layer-wise relevance propagation

- Input perturbations

- Attention

etc.

Operations to Enable Explainability

The major visualization techniques include:

- Saliency

- Raw declarative representations

- Natural language explanation

XAI Visualization Techniques

A model’s quality should be evaluated not only by its accuracy and performance, but also by how well it provides explanations for its predictions. The state of the field in terms of defining and measuring explanation quality includes evaluation and predictive process coverage.

Learn Before

Related