Learning entropy as a learning-based information concept

Ivo Bukovsky, Witold Kinsner, Noriyasu Homma

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)


Recently, a novel concept of a non-probabilistic novelty detection measure, based on a multi-scale quantification of unusually large learning efforts of machine learning systems, was introduced as learning entropy (LE). The key finding with LE is that the learning effort of learning systems is quantifiable as a novelty measure for each individually observed data point of otherwise complex dynamic systems, while the model accuracy is not a necessary requirement for novelty detection. This brief paper extends the explanation of LE from the point of an informatics approach towards a cognitive (learning-based) information measure emphasizing the distinction from Shannon's concept of probabilistic information. Fundamental derivations of learning entropy and of its practical estimations are recalled and further extended. The potentials, limitations, and, thus, the current challenges of LE are discussed.

Original languageEnglish
Article number166
Issue number2
Publication statusPublished - 2019 Feb 1


  • Information
  • Learning
  • Learning systems
  • Non-probabilistic entropy
  • Novelty detection


Dive into the research topics of 'Learning entropy as a learning-based information concept'. Together they form a unique fingerprint.

Cite this