SLEIGHT OF HAND APP

Python + OpenCV + Ableton

work in progress...

This app allows you to play music by moving your hands in front of a web camera.
It uses OpenCV library for hands tracking, transforms hands movement to midi signals (rtmidi2 library) and sends them to Ableton Live.

In the current version User decides on the number of notes he want to add to the screen layout, chooses the notes and preferred octaves from the table using a mouse. Right side of the screen is divided for chosen number of parts and each part has the assigned note displayed on it. By moving his index finger on the left side of the screen user sends Control Change data to the midi_out port (For example, it can be used for controlling sound effects, or volume level in Ableton). Movement of the index finger on the right side of the screen sends selected notes to the midi_out port.

Demos (turn the sound on... obviously)

Silly demo with note layout

Sleight_of_hand_demo3.mp4

Old version demo

Sleight_of_hand_demo1.mp4

Ideas for further development:

real GUI

functions for sending notes with preload presets: like create your own layout of notes, or choose an octave, or choose a scale…

functions for sending cc signal: just signal from hand movement or modulation signal from set (sin, saw, square…) with amplitude controlled by hand

ability to choose a function for each hand from list of functions

working with separate midi_outs

tempo control

getting data through Ableton API

theremin setting?