Search
Close
  • About Us
    • Governance
    • Our Team
    • Labs
    • Open positions
    • Alumni
  • Research
    • Wearable robotics
      • ReGait++
      • Third Arm
    • Rescue Robotics
      • Flying Robots
      • Legged robots
      • Collaboration, Learning and Tests
    • Educational robotics
      • Cellulo
      • Thymio
    • Publications
    • Open Science
  • Education
    • How to get into robotics
      • How our professors got into robotics
      • How our researchers got started
    • Robotics for kids
      • Activities for Kids
      • R2t2 Mars Mission
    • Graduate Programs in Robotics
    • Resources
  • Tech Transfer
    • Researchers expertise
    • Spin-Offs
    • Swiss Robotics Day
    • Swiss Robotics Ecosystem
    • Resources
  • Equal Opportunities
    • Robotics for Girls
    • Master students
    • Young Researchers
    • Building Equality
    • Women in Robotics
    • Resources
  • Awards & Programs
    • Spin-Fund
    • Master Students Exchange Program
    • PhD/Postdoc exchange program
    • Award for Career Development
    • Swiss Robotics Masters Award
    • Award for Scientific Visibility
    • Women keynote speakers
    • Best Paper Award
    • Ukrainian researchers
  • Outreach
    • Swiss Drone Days
    • Cybathlon
    • Swiss Robotics Days
    • Other Outreach Events
  • Contact
Menu
  • About Us
    • Governance
    • Our Team
    • Labs
    • Open positions
    • Alumni
  • Research
    • Wearable robotics
      • ReGait++
      • Third Arm
    • Rescue Robotics
      • Flying Robots
      • Legged robots
      • Collaboration, Learning and Tests
    • Educational robotics
      • Cellulo
      • Thymio
    • Publications
    • Open Science
  • Education
    • How to get into robotics
      • How our professors got into robotics
      • How our researchers got started
    • Robotics for kids
      • Activities for Kids
      • R2t2 Mars Mission
    • Graduate Programs in Robotics
    • Resources
  • Tech Transfer
    • Researchers expertise
    • Spin-Offs
    • Swiss Robotics Day
    • Swiss Robotics Ecosystem
    • Resources
  • Equal Opportunities
    • Robotics for Girls
    • Master students
    • Young Researchers
    • Building Equality
    • Women in Robotics
    • Resources
  • Awards & Programs
    • Spin-Fund
    • Master Students Exchange Program
    • PhD/Postdoc exchange program
    • Award for Career Development
    • Swiss Robotics Masters Award
    • Award for Scientific Visibility
    • Women keynote speakers
    • Best Paper Award
    • Ukrainian researchers
  • Outreach
    • Swiss Drone Days
    • Cybathlon
    • Swiss Robotics Days
    • Other Outreach Events
  • Contact
Search
Close
  • NCCR Robotics
    • Highlights
    • Governance
    • Our team
    • Labs
    • Past programs
    • Alumni
    • Open positions in robotics
  • Research
    • Wearable Robotics
      • ReGait++
      • Third Arm
    • Rescue Robotics
      • Flying robots
      • Legged robots
      • Collaborations, learning and tests
    • Educational Robotics
      • Cellulo
      • Thymio
    • Publications
    • Open Science
    • Research resources
  • Education
    • How to get into robotics
      • How our professors got into robotics
      • How our researchers got started
    • Robotics for kids
      • Activities for kids
      • R2t2 Mars mission
    • Graduate programs in robotics
    • Resources
  • Tech Transfer
    • Researchers expertise
    • Spin-Offs
    • Swiss Robotics Day
    • Swiss Robotics Ecosystem
    • Resources
  • Equal Opportunities
    • Building equality
    • Women in Robotics
    • Resources
  • Outreach
    • Swiss Robotics Days
    • Swiss Drone Days
    • Cybathlon
    • Photo exhibit
    • Other Outreach Events
Menu
  • NCCR Robotics
    • Highlights
    • Governance
    • Our team
    • Labs
    • Past programs
    • Alumni
    • Open positions in robotics
  • Research
    • Wearable Robotics
      • ReGait++
      • Third Arm
    • Rescue Robotics
      • Flying robots
      • Legged robots
      • Collaborations, learning and tests
    • Educational Robotics
      • Cellulo
      • Thymio
    • Publications
    • Open Science
    • Research resources
  • Education
    • How to get into robotics
      • How our professors got into robotics
      • How our researchers got started
    • Robotics for kids
      • Activities for kids
      • R2t2 Mars mission
    • Graduate programs in robotics
    • Resources
  • Tech Transfer
    • Researchers expertise
    • Spin-Offs
    • Swiss Robotics Day
    • Swiss Robotics Ecosystem
    • Resources
  • Equal Opportunities
    • Building equality
    • Women in Robotics
    • Resources
  • Outreach
    • Swiss Robotics Days
    • Swiss Drone Days
    • Cybathlon
    • Photo exhibit
    • Other Outreach Events

ESS: Learning Event-based Semantic Segmentation from Still Images

Publication date:

  • 2022

Authors:

Zhaoning Sun
Nico Messikommer
Daniel Gehrig
Davide Scaramuzza

Abstract

Retrieving accurate semantic information in challenging high dynamic range (HDR) and high-speed conditions remains an open challenge for image-based algorithms due to severe image degradations. Event cameras promise to address these challenges since they feature a much higher dynamic range and are resilient to motion blur. Nonetheless, semantic segmentation with event cameras is still in its infancy which is chiefly due to the lack of high-quality, labeled datasets. In this work, we introduce ESS (Event-based Semantic Segmentation), which tackles this problem by directly transferring the semantic segmentation task from existing labeled image datasets to unlabeled events via unsupervised domain adaptation (UDA). Compared to existing UDA methods, our approach aligns recurrent, motion-invariant event embeddings with image embeddings. For this reason, our method neither requires video data nor per-pixel alignment between images and events and, crucially, does not need to hallucinate motion from still images. Additionally, we introduce DSEC-Semantic, the first large-scale event-based dataset with fine-grained labels. We show that using image labels alone, ESS outperforms existing UDA approaches, and when combined with event labels, it even outperforms state-of-the-art supervised approaches on both DDD17 and DSEC-Semantic. Finally, ESS is general-purpose, which unlocks the vast amount of existing labeled image datasets and paves the way for new and exciting research directions in new fields previously inaccessible for event cameras.

Reference
Doi: https://doi.org/10.48550/arXiv.2203.10016
Published in: European Conference on Computer Vision (ECCV), 2022,
 Pages 1-20
Read the paper
Version: Published
Access: Open Access
Download
  • Innovation Booster Robotics
  • News
  • Past Events
  • Contact
  • Innovation Booster Robotics
  • News
  • Past Events
  • Contact
  • Journalists and media
  • News bulletin archive
  • Newsletter archive
  • Journalists and media
  • News bulletin archive
  • Newsletter archive
Twitter Linkedin Youtube
NCCR Robotics partner institutions

Leading house

epfl_white

Co-Leading house

eth_white
unizuerich_white
idsia_white
unibe_all_white
empa-all-white
unibasel_white

This website was built by NCCR Robotics and provides useful information on Swiss Robotics. From December 2022, it is no longer updated.

Copyright © 2022 NCCR Robotics. All rights reserved.