Activity and Context Recognition with Opportunistic Sensor Configurations

FP7-ICT-2007-C, FET open

Goals: The project aims at developing mobile systems to recognize human activity with dynamically varying sensor setups. We refer to such systems as opportunistic, since they take advantage of sensing modalities that just happen to be available, rather than forcing to deploy specific, application dependent sensor systems.

Summary: We envision opportunistic activity recognition systems. They are goal-oriented sensor assemblies that spontaneously arise and self-organize to achieve a common goal, here activity and context recognition.

The objective of OPPORTUNITY is to develop generic principles, algorithms and system architectures to reliably recognize complex activities and contexts despite the absence of static assumptions about sensor availability and characteristics in opportunistic systems.

OPPORTUNITY picks up on the very essential methodological underpinnings of any Ambient Intelligence (AmI) scenario: recognizing (and understanding) context and activity.

Methodologies are missing to design context-aware systems that

  • work over long periods of time despite changes in sensing infrastructure (sensor failures, degradation);
  • provide the freedom to users to change wearable device placement;
  • can be deployed without user-specific training.

This limits the real-world deployment of AmI systems.

We develop opportunistic systems that recognize complex activities/contexts despite the absence of static assumptions about sensor availability and characteristics. They are based on goal-oriented sensor assemblies spontaneously arising and self-organizing to achieve a common activity/context recognition goal. They are embodied and situated, relying on self-supervised learning to achieve autonomous operation. They make best use of the available resources, and keep working despite-or improves thanks to-changes in the sensing environment. Changes include e.g. placement, modality, sensor parameters and can occur at runtime.

Four groups contribute to this goal. We develop:

  1. intermediate features that reduce the impact of sensor parameter variability and isolate the recognition chain from sensor specificities;
  2. classifier and classifier fusion methods suited for opportunistic systems, capable of incorporating new knowledge online, monitoring their own performance, and dynamically selecting most appropriate information sources;
  3. unsupervised dynamic adaptation and autonomous evolution principles to cope with short term changes and long term trends in sensor infrastructure,
  4. goal-oriented cooperative sensor ensembles to opportunistically collect data about the user and his environment in a scalable way.

The methods are demonstrated in complex opportunistic activity recognition scenarios, and on robust opportunistic EEG-based BCI systems.

Partners: ETH Zurich, UoPassau, IDIAP