Building Generative AI tools for Music and Dance
A creative lab of dancers, musicians and engineers in Stockholm, exploring AI in movement and dance performance — and building the instruments it needs.
Generative AI for movement, music and dance performance
Reinventing music creation with AI-driven tools prompted by movement.
"Can movement control sound?", "is generative AI ready to create music in real-time?" and "can the hierarchy between music, dance and lighting be challenged with the help of AI?"
We are a group of engineers, dancers and musicians who are exploring the intersection of the latest in generative AI and how it can be used to create new forms of music and dance.
To this end, we're building a new class of instruments and tools that are prompted by movement and AI, and we're exploring how these tools might raise a new genre of cross-disciplinary art-forms. Read the primer on AI in movement and dance performance to see how we think about it.
What we work on
- Movement-controlled instruments — using the body as a controller and a generative AI model as the sound generator.
- Real-time generative music — testing whether today's models can compose and respond fast enough for live performance.
- New performance hierarchies — letting dance drive music and lighting, instead of the other way round.
- Open workshops and hackathons — bringing dancers, musicians and software engineers into the same room in Stockholm.
Who is behind it
Movement, Music & Machines was created by Nils Kakoseos, Mira Buus and Mark Schellhas — a mathematician and musician, a dancer, and a music producer turned technology founder.
Events: workshops & hackathons
Next events
Generative AI tools demo & live performance: our next workshop date is not announced yet. To secure an invite, join the waiting list.
Previous events
AI, movement and dance: common questions
What is Movement, Music & Machines?
Movement, Music & Machines is a creative lab of musicians, dancers and software engineers who build tools and explore methods that push the status quo of generative AI in music and dance. It was created in 2023 by Nils Kakoseos, Mira Buus and Mark Schellhas, and it runs workshops and hackathons in Stockholm.
How is generative AI used in movement and dance performance?
In our work, generative AI is used as an instrument rather than a content generator. Movement is tracked and turned into prompts and control signals, and a machine learning model generates or shapes sound in response. That makes the dancer a musician: the body becomes the controller and the model becomes the sound generator.
Can movement control sound?
Yes. This is the central question of our lab, and the instrument we are building answers it in practice: the body acts as a controller and a generative AI model acts as the sound generator, so a movement produces sound in real time rather than being choreographed to a finished track.
Is generative AI ready to create music in real time?
Partly. Latency, controllability and musical coherence are still the hard problems for real-time generative music, which is exactly why we test the tools in live workshops with dancers and musicians instead of only in the studio.
Why is generative AI less developed in music and dance than in visual art?
Generative AI is alarmingly underdeveloped in music and dance, missing out on vast opportunities. While the visual arts have seen a surge in AI-generated content, music and dance have been left behind — partly because they are time-based, performed and harder to represent as training data. Closing that gap is the reason this project exists.