Stream Machine Learning (StreaML) Open Challenge 2017-2018

Challenge Motivation

After a successful organization of the DEBS Grand Challenge 2017 by the HOBBIT project at the DEBS 2017 Conference, HOBBIT is proud to announce the Stream Machine Learning (StreaML) Open Challenge. The open challenge will ensure continuous participation and systems evaluation.

A monetary prizes will be provided to the best winning system at each cutoff date.

Challenge Overview

The goal of the StreaML Open Challenge competition is to evaluate event-based systems for real-time analytics over high velocity and high volume data streams generated by manufacturing equipment.

The predefined machine learning algorithm should be applied for analysis of the RDF streaming data generated by digital and analogue sensors embedded within multiple molding machines. The data produced by each sensor is clustered and the state transitions between the observed clusters is modeled as a Markov chain. Based on this machine learning-based classification, anomalies are detected by means of sequences of transitions that happen with a probability lower than a given threshold.

Please note, that StreaML Open Challenge reuses the dataset and the task description of DEBS GC 2017 and that in contrast to DEBS Challenge, systems will be continuously evaluated every week and results will be shown on a leaderboard (see detailed description for more details).

Participants can use the sample system and the published benchmark as a reference implementation of anomaly detection algorithm to pass the correctness checks and focus on performance and stability of their systems, which are included into evaluation criteria.

Prizes

The best-winning system will get a prize of 500€

Q&A

Feel free to ask any questions about the challenge under the Issues tab.

Important Dates

Dry-run:

December 2017 to end of January 2018

Training Phase*:

February to end of April 2018

Test Phase:

May 2018

Cutoff and results proclamation:

May 2018

* Submitted systems will be continuously evaluated by the HOBBIT platform with weekly updates of leaderboards.

** Actual dates are for first round of StreaML Open Challenge.

Tasks and Training Data

The StreaML Open Challenge focuses on the task related to the problem of automatic detection of anomalies for manufacturing equipment.

The overall goal of the task is to detect abnormal behavior of a manufacturing machine based on the observation of the stream of measurements provided by each of the machines from observable set, which is dynamic (i.e. machines dynamically join and leave the set of observable machines). The data produced by each sensor is clustered and the state transitions between the observed clusters are modelled as a Markov chain. Based on this classification, anomalies are detected as sequences of transitions that happen with a probability lower than a given threshold.

Organization

Pavel Smirnov – AGT International

Martin Strohbach – AGT International

Support

We would like to explicitly thank Weidmüller (http://www.weidmueller.de/) for the provisioning of the original data set that AGT International used to generate a realistic data set for the purpose of this challenge.

News

01.02.2018: The training phase was started. Now participants are able to test their systems online under workload of the StreaML Open Challenge Benchmark (StreaML v1.1) and register them for the first task of the StreaML Open Challenge. Please find the example of HOBBIT-compatible samples and other helpful instructions here.

01.12.2017: Call for participants for the StreaML Open Challenge announced. The training dataset is available online. The Dry-Run phase will continue till the end of January 2018. Please follow the further announcements.

—————————————————————————————————
StreaML Open Challenge 2017-2018
—————————————————————————————————
After a successful co-organization of the DEBS Grand Challenge with the HOBBIT project (https://project-hobbit.eu/) at the DEBS 2017 Conference, HOBBIT is proud to announce the Stream Machine Learning (StreaML) Open Challenge, which starts in December 2017.

The StreaML Open Challenge will ensure continuous participation and systems evaluation with periodic cutoffs. The first cutoff is in May 2018.
* A monetary prize will be provided to the best-winning system!
* Stay tuned and get ready to participate!

————————————————————————————–
The StreaML Open Challenge at a glance
————————————————————————————–
The StreaML Open Challenge focuses on the task related to the problem of automatic detection of anomalies for manufacturing equipment.
The overall goal of the task is to detect abnormal behavior of a manufacturing machine based on the observation of the stream of measurements provided by each of the machines from observable set, which is dynamic (i.e. machines dynamically join and leave the set of observable machines). The data produced by each sensor is clustered and the state transitions between the observed clusters are modelled as a Markov chain. Based on this classification, anomalies are detected as sequences of transitions that happen with a probability lower than a given threshold.

We herewith invite system developers to participate in the aforementioned scenario. To ensure that the system results are comparable, we will provide the HOBBIT benchmarking platform for the generation of the final results to be included into the system publications. A specification of the hardware on which the benchmarks will be run will be released in due course.

Participants are allowed to use the published benchmark (https://github.com/hobbit-project/sml-benchmark) as a reference implementation of anomaly detection algorithm to pass the correctness checks and focus on performance and stability of their systems, which are included into evaluation criteria.

————————————————————————————–
Prizes
————————————————————————————–
The winner of each round of the challenge will get a prize of 500€.

————————————————————————————–
Important Dates
————————————————————————————–
Dry run: December 2017 – end of January 2018
Training phase: February – end of April 2018
Cutoff and results proclamation: May 2018

————————————————————————————–
Sponsorship
————————————————————————————–
We would like to explicitly thank AGT International (http://www.agtinternational.com) for being a sponsor for the StreaML Open Challenge.