Learning entropy of adaptive filters via clustering techniques

Ivo Bukovsky, Gejza Dohnal, Pavel Steinbauer, Ondrej Budik, Kei Ichiji, Homma Noriyasu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Learning Entropy (LE) was initially introduced as a measure for sample point novelty by unusually large learning effort of an online learning system. The key concept is that LE is based on pre-Training and further online learning, and the novelty measure is not necessarily correlated to the prediction error. Most recently, the idea of LE was revised as a novel non-probabilistic, i.e., machine-learning-based information measure. This measure is high when a learning system is not familiar with a given data point, so the learning activity to learn novel data points is unusual (regardless of the prediction error), i.e., the learning increments display unusual patterns during adaptation. In this paper, we propose concepts of the learning state and the learning state space so that LE can be approximated via neighbourhood analysis in the learning space. Further, two novel clustering-based techniques for approximation of sample point LE are proposed. The first one is based on the sum of K nearest neighbour distances. The second one is based on multiscale neighbourhood cumulative sum. Also, we preprocess the learning space with dimensionality reduction that is promising for research of LE even with neural networks and potentially with deep neural networks. The performance of novelty detection with the clustering-based sample point LE with dimensionality reduction is compared to the original algorithms of LE, and its potentials are discussed.

Original languageEnglish
Title of host publication2020 Sensor Signal Processing for Defence Conference, SSPD 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728138107
Publication statusPublished - 2020 Sept
Event9th Sensor Signal Processing for Defence Conference, SSPD 2020 - Edinburgh, United Kingdom
Duration: 2020 Sept 152020 Sept 16

Publication series

Name2020 Sensor Signal Processing for Defence Conference, SSPD 2020


Conference9th Sensor Signal Processing for Defence Conference, SSPD 2020
Country/TerritoryUnited Kingdom


  • adaptive filters
  • clustering
  • dynamic detection scheme
  • novelty detection
  • time series
  • unsupervised anomaly detection


Dive into the research topics of 'Learning entropy of adaptive filters via clustering techniques'. Together they form a unique fingerprint.

Cite this