About

All articles, tagged with “audio”

Disco Snake

Rock Band does a great job of inspiring people to play music, can you develop a game that inspires composition? Lumines and Rez create music while you play, can you make games where music creation is the goal, not a side effect? Pictionary does a great job of using game mechanics to overcome creative block, can you use other game-like constraints to inspire creativity? These were among the questions I asked at GameCamp 2 a few months ago and I was keen to explore them at Music Hack Day London a week ago.

The spectrum of potential game-like musical composition tools is huge, ranging from traditional recognisable music interfaces like keyboards and step sequencers at one end, through things that are designed to be both music interfaces and games like Fractal, Rez and Lumines in the middle to things that are recognisably games at the other. While the middle ground is incredibly interesting, 24 hours at a hack day isn’t really long enough to develop a brand new revolutionary hybrid game/music interface, so instead I decided to repurpose an existing game as a sequencer and picked the simplest one I could think of — Snake.

With the interface chosen, the next thing to do was to think about how to map the interface to music composition. The core mechanic in snake is eating food placed on a grid. Grids have a venerable history in music as step sequencer interfaces with time growing form left to right and pitch or samples selected on the y axis. It seemed natural to map food position to note parameters in a similar way. Using the order in which food is selected to determine the order of notes played frees up the X axis to map to a parameter instead of time and also makes playing the game feel more like a progression through a composition: each piece of food adds to the sequence which is continually looping, the music plays and the composition progresses, there is no turning back or revising. By mapping the X axis to velocity pseudo rests can be added to the sequence by selecting food on the left.

Selecting notes requires some deviation from the normal snake mechanics which normally only make a single piece of food available at any time. This restriction would mean that players wouldn’t compose music, simply reveal it as they ate one piece of food at a time. At the other end of the spectrum turning every square in to food would mean that the next selected note would have to be adjacent to the last note, also overly restrictive. Making a limited number of pieces of food available at any time provides a nice middle ground, allowing the player some freedom in the choice of the next note selected, but not total freedom, a restriction which can lead to serendipitous melodies.

The other major mechanic in snake is colliding with your tail, which ends the game, but becomes harder to avoid as you eat food and get longer. One option would be to use that mechanic to intentionally end the game and the composition, but instead I mapped it to sample selection allowing the player to switch between sounds and start a new sequence to build up multi-timbral polyphonic music. By making the world toroidal players can simply let the snake circle around the world when they have finished composing.

A lot of these design decisions came out while implementing the game using processing/processing.js and HTML 5 audio — a technology stack I’d played around with a bit previously, but wanted to explore further. In the end, for this kind of application I don’t think processing brings enough benefit to outweigh the difficulties it adds to debugging. When running on top of Java errors are often reported as mangled Java call stacks and when running in the browser different errors appear as mangled JavaScript. While I can see the attraction to language designers and implementers of building on top of existing technology it often results in having to implement in one language and debug horrible unrecognisable code in another. Incompatibilities are also horrible. With a couple of hours to go I had the entire game running on Java, but was presented with a blank canvas with no useful Firebug errors when I exported to processing.js and having to perform a binary search by commenting out chunks of code to find the error. Not pleasant.

HTML5 audio is also a somewhat fragile technology. Generating an Audio element for each sample playback event leads to current browsers grinding to a halt while resetting and restarting audio elements often causes glitches and delays. Another problem is that JavaScript timers don’t provide enough accuracy for tight sequence playback timing. In the end I rebranded both bugs as features by switching from very transient drum samples which sounded messy to dubby bass and melancholy bell samples that work quite nicely with glitches and unintentionally loose timing.

At 10PM on Saturday night everything had come together enough for me to lose myself in an hour of ambient bleepy electronica and by the time the presentations started at 3PM on Sunday Disco Snake was done.

I’d like to thank all of the organisers and hackers that made Music Hack Day London a wonderful experience and have been pleasantly surprised at the positive reaction that Disco Snake has generated over the last week. The space between music interfaces and games is a very fertile one that I’ll be exploring further in the future and while it’s not there yet, I hope HTML5 audio fulfils its promise of bringing interactive music applications to everyone on the web in the very near future.

HTML 5 Audio Redux

My recent experiments in to using Procssing.js and HTML5 audio to generate multimedia web applications didn’t get very far. I first tried generating a new HTML 5 audio element for each audio event, which quickly caused the browser to grind to a halt, and my attempts to reuse audio elements by resetting the playback position didn’t seem work, leading me to conclude that HTML 5 audio was only really useful for playing back long audio files, not for building sequencers that play back many short samples. When I spoke to @rem about my findings he was convinced that resetting audio elements should be possible and this weekend’s Music Hackday London has provided the perfect incentive and opportunity to dust off my experiments and start tinkering again. An hour in and sure enough I’ve managed to get audio elements to reset: it seems that the trick is to set currentTime after calling play() on the element, something that seems very counter-intuitive, but seems to work (at least in Firefox 3.6.8 and Safari 5.0.1 on OS X 10.6.4). Now I have reliable sample play back it’s time to start playing around with more interesting interfaces in Processing and there are 26 hours of hacking left: game on!

(cc-sampling+ licensed samples by vitriolix from freesound.org)

Music Again!

Since moving to Brighton 18 months ago I’ve been pretty busy finding my feet, moving house twice, sorting out schools and setting up Linden Lab Brighton, so I haven’t had as much time to make music as I’d have liked. It hasn’t helped that my brother Roo, who collaborated with me on Vanishing Trick has gone from being a medical student to an opthalmologist, so has been pretty busy too.

I’ve still been reading create digital music though, so couldn’t resist buying Luke a copy of Korg DS-10 for his birthday just before Christmas. It’s a great piece of software that reminds me a lot of ReBirth (which is now free!) — the same infectious acid sounds and a fun interface that you can’t resist tweaking. Within hours Luke was coming up with screaming synth noises that would have made the Chemical Brothers proud, so I told him that when he had a track finished we could record it on to the computer, add some more sounds and make a CD.

30 minutes later I had Luke to thank for finally convincing me to unpack my bag of midi and audio cables for the first time since we rolled up in Brighton. While adding bits to Lukes DS-10 and Electroplankton creations, I took the opportunity to finally play around with Ableton Live properly and was completely bowled over. It’s one of those pieces of software that as a software engineer you can’t help admiring. It makes you want to use it and turns the complex process of sequencing and producing music in to joyful fun. I’d been a Cubase user for 10 years, but I’m not going back.

Over Christmas I put together a Live rig that lets me use my ancient Yamaha MFC05 MIDI controller to switch between drum beats, record guitar loops and automatically switch between effects settings without touching the computer. With some tweaking I’m going to get it to do the same for my Nord Modular and switch patches on both the Nord and my POD allowing me to quickly record, compose, arrange and perform music from 5 foot pedals. I’ve also got ITM set up on my iPhone which I’m planning to mount on my guitar giving me a wireless X/Y touchpad that can control any number of Ableton parameters.

I also finally got round to playing around with circuit bending for the first time. Last Christmas Roo got me a reissue Stylophone, which is a fun toy, but whenever I read about circuit bending online I kept thinking it would be fun to try connecting the headphone output of the stylophone to the MP3 input. In the end it only took an hour or so of poking around inside the Stylophone while telling Luke about electronics to find some bend points that turn the mild mannered Stylophone in to a wailing banshee of feedback distortion that’s more Hendrix than Harris. The modded Stylophone is shown below and I’ve added annotations to a picture of the opened up Stylophone on Flickr that shows the points that I connected. If you’d like to hear the evil sounds it makes there are a selection of Creative Commons licensed samples on Freesound, but I advise turning the volume down and using headphones if you give them a listen.

Bent Stylophone

It’s great to be having fun making music again. Thanks Luke and Roo!