The main idea is to derive explanation by investigating the importance scores of different features used to output the final prediction. Such approaches can be built on features like manual features obtained from feature engineering, lexical features including word/tokens and n-gram, etc.

University of Michigan - Ann Arbor

There are five major explainability techniques that differ in the mechanisms they adopt to generate the raw mathematical justifications.

- Feature importance

- Surrogate model

- Example-driven

- Provenance-based

- Declarative induction

A complex NLP approach consisting of different components may employ more than one of these explainability techniques.

Explainability Techniques

Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). *A survey of the state of explainable AI for natural language processing*. arXiv preprint arXiv:2010.00711.

A Survey of the State of Explainable AI for Natural Language Processing

Feature importance

Model predictions are explained by learning a second, usually more explainable model, as a proxy. Surrogate model-based approaches are model-agnostic and can be used to achieve either local or global explanations. 

Surrogate model

It explains the prediction of an input instance by identifying and presenting other instances, usually from available labeled data, that are semantically similar to the input instance.

Example-driven technique

Explanations are provided by illustrating some or all of the prediction derivation process, which is an intuitive and effective explainability technique when the final prediction is the result of a series of reasoning steps.

Provenance-based

It involves human-readable representations, such as rules, trees, and programs to deliver explainability.

Learn Before

Related