A test-challenge and validation dataset are already available along with evaluation servers. A separate test-dev dataset along with a public leaderboard will be made available soon and can then be used for ongoing evaluation of probabilistic object detection approaches.

We organise a competition and workshop at CVPR 2019 in June 2019, where the best submissions will be presented and $5000 AUD in prize money will be available to the winning entries.

News

January 2019: We are happy to announce that CVPR 2019 is hosting our workshop. Participants of our Robotic Vision object detection challenge will present their approaches and results, and we will announce the competition winners at the workshop.

December 2018: We released our first Robotic Vision object detection challenge, requiring object detection on video data and rewarding accurate estimates of spatial and semantic uncertainty.

Stay in touch and follow us on Twitter for news and announcements: @robVisChallenge.

Coming Soon

Stay tuned for more challenges, focussing on active vision, and active and continuous learning in 2019.

Motivation

Big computer vision challenges and competitions like ILSVRC or COCO had a significant influence on the advancements in object recognition, object detection, semantic segmentation, image captioning, and visual question answering in recent years. These challenges posed motivating problems to the research community and proposed datasets and evaluation metrics that allowed to compare different approaches in a standardized way.

However, visual perception for robotics faces challenges that are not well covered or evaluated by the existing benchmarks.
These challenges comprise calibrated uncertainty estimation, continuous learning for domain adaptation and incorporation of novel classes, active learning, and active vision.

There is currently a lack of meaningful standardised evaluation protocols and benchmarks for these research challenges. This is a significant roadblock for the evolution of robotic vision, and impedes reproducible and comparable research.

We believe that by posing a new robotic vision challenge to the research community, we can motivate computer vision and robotic vision researchers around the world to develop solutions that lead to more capable, more robust, and more widely applicable robotic vision systems.

Organisers, Support, and Acknowledgements

Stay in touch and follow us on Twitter for news and announcements: @robVisChallenge.