FrequencyCam: Imaging Periodic Signals in Real-Time

This repository has a ROS/ROS2 node for frequency analysis with event based cameras. FrequencyCam colors pixels in an image with the frequency at which their brightness changes. The end result is a "frequency image" that is very similar to Prophesee's Metavision Vibration Estimation module.

In contrast to the Metavision module, FrequencyCam is open source under a permissive (Apache2 license) and is documented. It uses an IIR digital filter to approximately reconstruct the brightness of each pixel and then measures the time it takes between zero-level crossings. For more details please refer to this paper.

Here are a few videos of FrequencyCam in action:

guitar	quad rotor	heli

Supported platforms

Currently tested on Ubuntu 20.04 under ROS2 Galactic.

Set the following shell variables:

repo=frequency_cam
url=https://github.com/berndpfrommer/${repo}.git

and follow the instructions here

ros2 launch frequency_cam frequency_cam.launch.py

FrequencyCam expects ROS event_camera_msgs messages from the metavision ROS driver here. You should be able to use other cameras (like DVS etc) by converting the messages with a republish nodelet from the event camera tools repository. Alternatively you can use the libcaer driver that publishes suitable messages natively.

Parameters (see launch file):

Input and algorithm related parameters:

• min_frequency: lower bound for detected frequency
• max_frequency: upper bound for detected frequency
• cutoff_period: number of events to use for the filter, see paper. When in doubt set to 5.0 (default).
• num_timeout_cycles: mark pixel inactive if it has not been updated for a time of num_timeout_cycles times pixel period (defaults to 2).
• use_sim_time: set this to true when playing from bag (and play bag with --clock 10000)
• bag_file: only supported under ROS2: play from bag file and write generated frames to disk. If no bag file is given the node subscibes to topics.

Imaging related parameters:

• publishing_frequency: frequency (in Hz) at which frequency image is published (defaults to 20Hz)
• use_log_frequency: use log scale for frequency image coloring
• overlay_events: mark pixels grey for which events have happened but no frequency could be found.
• legend_num_bins: how many color patches to show (defaults to 11)
• legend_num_sig_digits: how many significant digits to use for legend (defaults to 3).
• legend_frequencies: explicitly specify frequencies of color patches to show in the legend.
• legend_width: how many pixels wide the legend should be (defaults to 100).
• scale_image: factor by how much to scale up the image in resolution. Only used when playing from bag under ROS2. Useful for getting nicer fonts for numbers when running from bag for movie generation.

Topics

• ~/image_raw: topic under which frequency image is published.
• ~/events: event topic to subscribe to.

License

This software is issued under the Apache License Version 2.0.