Solfège Studio — You Sign, It Plays

Field notes

The prototype maps movable-do Curwen signs to sound rather than asking learners to type or play notes. Apple Vision tracks twenty-one joint locations per visible hand, while the classifier uses pose geometry, motion, confidence, and agreement across frames to infer intent. The current macOS build recognizes one or two hands, uses hand height for octave changes, and records recognition decisions with the video so missed notes can be traced to particular frames.