Real-time event-based laser tracking

Goal: Track a laser pointer (controlled by a human) with a second, programmable laser pointer based on the input from an event camera in real-time. One additional twist is that the event camera can move! So, the mapping should, ideally, be adaptive. The controllable laser should also be governed by some neuromorphic computation/hardware.

Equipment: Galvanic mirror system with event camera (picture). Laser pointers emitting at 300Hz or 900Hz. 2 x DXplorer cameras (640x480 resolution) to change the perspective. NVIDIA 3070 GPU for compute.

Challenges:

• Learn mapping from event camera to laser position


• Adapt to changing camera positions over time


• Distinguish "efferent" laser pointer from human laser pointer (optional)

For more details and pictures, see wiki: https://wiki.capocaccia.cc/index.php/Robotics-control-and-reinforcement-learning

Go to group wiki Go to wiki users Info

Timetable

Moderator

Members