I was inspired to put my inexpensive (and rarely used) midi controller to use while watching podcasts on YouTube: What could I build if I combine the web APIs for midi and video?^[1]
It was insanely easy, especially thanks to this midi API boilerplate. I was able to get a demo running in just 15 minutes. The first iteration simply allowed me to adjust the playback rate of the video with a dial. At that point I knew that I had to make a sampler.

So I took it a step further: mapping the 16 black and white keys over the duration of the video. So for a 1 minute long video, the first white key triggers 0:00, the middle key triggers 0:30 and the last key triggers 1:00. With just a few more hours of effort I was able to make it more robust and easily programmable.

I had the most fun cranking down the playback rate on the underwater theme from Donkey Kong Country and sampling it over a drum track:

You can check out the repository on GitHub. The code is configured for my Akai MPK Mini but with little effort can be set up to work on your machine. I have other ideas for further development. There are web APIs for all kinds of audio and video processing.

Outside of music, I’ve used it to help get through lulls in long podcasts. I’ve also thought about using it to control a web page but I’m not sure that’s practical. What do you think? Have any ideas for how else a midi controller and web apis could interact?

0. ^ I was also inspired by Live Coding YouTube and by the classic MPC1000.