Model-Agnostic Explanations for Decisions Using Minimal Patterns

Kohei Asano, Jinhee Chun, Atsushi Koike, Takeshi Tokuyama

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Recently, numerous high-performance machine learning models have been proposed. Unfortunately, such models often produce black-box decisions derived using opaque reasons and logic. Therefore, it is important to develop a tool that automatically gives the reasons underlying the black-box model’s decision. Ideally, the tool should be model-agnostic: applicable to any machine-learning model without knowing model details. A well-known previous work, LIME, is based on the linear decision. Although LIME provides important features for the decision, the result is still difficult to understand for users because the result might not contain the features required for the decision. We propose a novel model-agnostic explanation method named MP-LIME. The explanation consists of feature sets, each of which can reconstruct the decision correctly. Thereby, users can easily understand each feature set. By comparing our method to LIME, we demonstrate that our method often improves precision drastically. We also provide practical examples in which our method provides reasons for the decisions.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2019
Subtitle of host publicationTheoretical Neural Computation - 28th International Conference on Artificial Neural Networks, 2019, Proceedings
EditorsIgor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková
PublisherSpringer Verlag
Number of pages12
ISBN (Print)9783030304867
Publication statusPublished - 2019
Event28th International Conference on Artificial Neural Networks, ICANN 2019 - Munich, Germany
Duration: 2019 Sept 172019 Sept 19

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11727 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference28th International Conference on Artificial Neural Networks, ICANN 2019


  • Explanation
  • Interpretability
  • Machine learning


Dive into the research topics of 'Model-Agnostic Explanations for Decisions Using Minimal Patterns'. Together they form a unique fingerprint.

Cite this