Play Gesture Synth Turns Hand Tracking into a Browser Instrument

Field notes

The instrument uses a WebAssembly MediaPipe hand landmarker in the browser, translating 21 landmark points per hand into musical controls rather than sending video to a server. Left-hand fingers select scale degrees, while the right hand chooses voicing and controls loudness and filtering; closing it silences the sound. Four sound engines are listed, including Synthwave, Ambient, 8-Bit, and Drum Kit, and the page also describes recording, export, and simultaneous MIDI output.