Outcomes

Specific outcomes include:

• Spontaneous goal-oriented sensing ensembles
• Robust abstract intermediate features
• Novel opportunistic classifiers and opportunistic classifier fusion
• Dynamic -short-term- adaptation of the activity recognition chain
• Autonomous -long-term- evolution of the activity recognition system
• Advances in Opportunistic robust Brain-Computer Interfaces

Application-wise, outcomes are activity and context recognition systems providing the following characteristics: (1) working over long periods of time despite changes in sensing infrastructure (sensor failures, degradation); (2) providing the freedom to users to change wearable device placement; (3) that can be deployed without user-specific training.

Anomaly detection in classifier ensemble [PRL]

In a multi-modal sensory recognition systems, sensors may provide anomalous behavior because of noise, failure, displacement, etc. Developing an anomaly detector for each sensor modality may not be feasible or it may need high effort. In specific cases where we use an ensemble of classifiers and each classifier is devoted to a subset of sensors and they are deciding among the same set of classes, we propose an approach to detect anomalous classifiers at the classifier fusion level. The pros and cons of these two levels of anomaly detection is provided in the table below.

We proposed an information theoretic (IT) approach to detect anomalous classifiers. This is done by monitoring the mutual information between classifier decisions in a period of time.

We compared our approach with two other popular modality-level approaches, namely One Class SVM (OCSVM) and Generalized Log-likelihood Ratio (GLR).

In the figure below, we show the recognition performance before and after anomaly detection. In this case, we removed the anomalous classifiers from the ensemble and only fuse the decisions of other untouched classifiers. X-axis demonstrates how many classifiers (sensors) are affected by noise (rotation in a random direction and random degree). Here we have 5 accelerometer, 5 gyro and 5 magnetic sensors. The blue line shows the highest performance we can get by anomaly detection when it works perfectly. This is done in a supervised manner. The performance of IT method is almost equivalent to GLR method, however it is independent of the sensor and noise type.

Anomaly detection and adaptation [BSN13]

Upon detection of anomalous classifier, a retraining procedure starts. Since at run-time we don't have activity labels, we use the fusion decision (which is more reliable than any of the individual classifiers) to train the anomalous classifiers. The process is shown below:

For example, in the following we show a time-line of activities performed by the person.

In this example, we have eight sensors (and one classifier per sensor) and we artificially rotated four of the sensors (by rotating the sensor readings) with 60 degrees in a random direction.

The anomaly detection method was able to detect the rotated sensors [MJ Back,MJ RUA, MJ LUA, Knee]. However for some of them there is a latency.

Once an anomalous classifier is spotted we start retraining them. The figure below shows how the performance of the classifiers are evolved during retraining.

Finally, improvement of the individual classifier performance leads to the improvement of the fusion performance, as shown below. The upper bound is when the ground-truth labels are available at run-time, and the lower bound is when there is no adaptation process going on.