A couple of years ago I had great fun putting together the London Geek Community iPhone OSCestra at Open Hack London and I’ve been controlling Ableton Live with iPhone tapped to my guitar as part of 100 robots for a couple of years now so when @andybudd suggested I do a digital music thing for the Brighton Digital Festival I immediately thought that it would be fun to combine the 2 projects by doing a 100 robots performance with audience participation.

The iPhone OSCestra was effectively a distributed collaborative mixing desk with each person controlling the volume and effect parameters on one channel of a playing back Ableton Live set. For the 100 robots performance I wanted to go further and have the audience actually adding parts to the musical performance, so @toastkid and I added extra drum, bass, synth and sample tracks to the 100 robots live set and filled them full of samples that could be triggered by the audience.

While having the samples adjust in tempo to match each song was relatively simple, transposing them to match the key of each song was more complicated. First I built a custom slice to midi preset which mapped the sample transpose to a macro control and used it to slice all of the samples to MIDI tracks, then mapped all of the transpose controls to a single MIDI controller and added a MIDI track which output the appropriate controller value for each song to a MIDI output which was looped back in to Live to transpose the samples.

The next question was how to avoid the performance turning in to a mush if multiple drum tracks or bass parts were playing concurrently. To avoid this we put dummy clips on the normal 100 robots which muted the normal parts when the audience triggered parts were playing. In some cases we let the audience parts add to the music, in others the audience parts would play instead of the normal tracks.

A final question was how to avoid max and I getting lost when the normal parts we play along to were replaced by unfamiliar samples. To deal with this we set the clip quantization on the audience triggered clips to values longer than the clip length. This meant that even if alternate baselines were constantly being launched, we would still hear the normal bassline for a while at the end of each quantization period, so we would know where we were with the track. To tune these settings we did some fuzz testing with semi random MIDI data to see how much madness we could deal with and still manage to play the songs.

With the tests done it was time to perform with 100 robots and 100s of people at the Brighton Dome and Museum.

Many thanks to Steve Liddell for recording the Brighton Museum set, @aral for letting us experiment on his update conference and to everyone who participated and watched. If you’d like to host another performance, please get in touch and if you like the music, please check out the 100 robots blog and consider buying our album from bandcamp.

Lots of exciting 100 robots news! Our debut album, Attack!, has been professionally mastered by Chris at Melograf Mastering who has done an amazing job and made the album sound incredible. The new version is already available at bandcamp and will be available on itunes, amazon and many other download services on Monday. To celebrate the launch we’re playing live at The Hope in Brighton tomorrow night and have set up a new blog where we’ll be giving away a track from the album free every month and I’ll be doing most of my 100 robots related music blogging from now on. Head on over and subscribe to the feed so you don’t miss out. Hope to see you at The Hope tomorrow!

This week I experienced two remarkable combinations of music and the moving image.

Natures 3B from Quayola on Vimeo.

This evening I watched Nature — Mira Calix and Quayola’s audio visual piece which took video footage of flowers blowing in the wind and used motion tracking technology to generate music from the footage. As a concept it was interesting, unfortunately as music it was terrible. The beauty of the footage betrayed the folly of the concept: if a human were to compose music based on the beauty of flowers the way they moved in the breeze might feature, but wouldn’t be the basis of the entirety of the piece. The colour, form and memories triggered by the flowers would surely feature. Turning the flowers to a network of points modulating parameters reduced them to an interesting if psuedo-random system and the resultant synthesised music was predictably cold and pseudo random.

By contrast, a few days ago I had the pleasure to watch Manhatta, a black and white movie about Manhatten made in 1920 by Charles Sheeler and Paul Strand and accompanied by a new soundtrack by the Cinematic Orchestra. Where Nature used machines to generate it’s soundtrack based on an algorithmic interpretation of the movement of flowers, Manhatta uses humans to generate it’s soundtrack based on the emotional impact of the moving image on the musicians. The result is infinitely more moving. The music adds emotion to the moving image, combining feelings of wonder, awe, fragility and insignificance — a uniquely human reaction to the images of the worlds most amazing city that cannot possibly be understood or rendered by an algorithm, no matter how clever.

Art is a human reaction to our world, not something that can be captured in an algorithm.

100 robots first album, “Attack!” is now finished and available to download now from bandcamp. I’m so glad that it is done and very proud of the result. It’s the first album I’ve made since 2005 and the first I’ve produced using Ableton Live, which once again proved to be an amazing piece of software. The ability to quickly cycle through libraries of samples to find the right sound, easy parameter automation and super flexible routing using drum racks were a huge help.

It’s also the first time I’ve tried to put together a big rock/dance production at home which made mastering tricky as we wanted both a big loud rock sound and huge slabs of sub bass. Because the of the huge sub bass, the Fletcher-Munson effect meant that while the RMS loudness of our initial mixes were normal, the a-weighted RMS values were very low, making the mixes sound very quiet when burned to CD.

The mixes couldn’t simply be turned up without clipping or compressing the mixes which would introduce distortion or reduce the dynamic range of the music. Not wanting to get sucked in to the loudness war we ended up in a complicated 3 way trade off between a-weighted RMS, sub bass weight and dynamic range. Audioleak and the Pleasurize Music tools were both really helpful during this process. We ended up with an an album that hit the sweet spot of -14 dBFS A-weighted RMS, has an average dynamic range of DR6 (admittedly less than the recommended DR10) and hopefully still has enough sub bass to work on a big system. A couple of people have commented that it sounds over-compressed, but most people seem to like where we ended up.

I hope you enjoy the album and that you can join us at the launch parties at the Hydrant in Brighton on the 31st of May or at the Maze in Nottingham on the 14th June — they’re going to be great nights!

On Friday I jumped on the train to London to attend Playful 2010, a one day conference put on by mudlark of World of Love fame. Despite billing itself as a day of cross “disciplinary frolicking” and featuring designers, podcasts, discussions of narrative, iphone augmented paper games and Disco Snake the thing that stood out for me was a thread running through the talks that addressed a fundamental of game design: meaningful choices.

Jonathan Smith talked about the dangers of giving people too much freedom in his talk about the Lego Games. Lego is almost a shorthand for freedom: the easy to understand system of knobs and anti-knobs that allows 2 4x2 blocks to be combined in 9 million ways an ultimate sandbox aspired to by games and virtual worlds like Second Life. This open, free system led Travellers Tales to add lots of open, free features to it’s early Lego games that were largely ignored by players who need boundaries and feedback from the game to determine ‘what I want versus what’s expected of me’. Choosing freedom and rebellion is more meaningful when it is clear that I am exercising my freedom and not doing the expected.

Margaret Robertson talked about and in the current sandbox game du jour, Minecraft, which has enough terror and threat in its horror filled night to make the choices made during the day meaningful and to reward mastery of it’s sandbox — a sandbox that compelled Margaret to stay up until early in the morning carving her slides out of earth, building them out of wood and animating them with flowing water and flames burning down the assertion that “games = points”.

It was this misguided assertion that Sebastien Deterding talked about in his look at the ‘gamification’ of the world around us. When all that gamified web sites like foursquare do is allow the accumulation of points and badges there are no meaningful choices, no mastery, no way to rebel against expectations, no play and no fun. Gamification results in loyalty schemes that are no more meaningful than Progress Quest.

The importance of being able to rebel against expectations was echoed by Alexis Kennedy’s talk about delicious misery in Echo Bazaar, a social game that would be another meaningless progression to inevitable success if it weren’t for contrarian missions that allow players to opt-in to getting their characters exiled for scandal or driven insane by demons. These missions inflict real harm on characters, but when properly signposted are the most enjoyed and shared missions: allowing players to be badass. When a game makes success inevitable, misery and failure is play and meaningful escape.

Pat Kane, formerly of Hue and Cry and more recently author of The Play Ethic gave a fascinating talk about wordplay, humour and his journey from disillusionment at the comedy industry, to fascination with humour through the Old Jews Telling Jokes’ stories of Jews laughing in the face of persecution. When misery and failure is inevitable, humour and play is rebellion. An ultimate, meaningful demonstration of freedom and humanity when all hope of victory is gone.

Rock Band does a great job of inspiring people to play music, can you develop a game that inspires composition? Lumines and Rez create music while you play, can you make games where music creation is the goal, not a side effect? Pictionary does a great job of using game mechanics to overcome creative block, can you use other game-like constraints to inspire creativity? These were among the questions I asked at GameCamp 2 a few months ago and I was keen to explore them at Music Hack Day London a week ago.

The spectrum of potential game-like musical composition tools is huge, ranging from traditional recognisable music interfaces like keyboards and step sequencers at one end, through things that are designed to be both music interfaces and games like Fractal, Rez and Lumines in the middle to things that are recognisably games at the other. While the middle ground is incredibly interesting, 24 hours at a hack day isn’t really long enough to develop a brand new revolutionary hybrid game/music interface, so instead I decided to repurpose an existing game as a sequencer and picked the simplest one I could think of — Snake.

With the interface chosen, the next thing to do was to think about how to map the interface to music composition. The core mechanic in snake is eating food placed on a grid. Grids have a venerable history in music as step sequencer interfaces with time growing form left to right and pitch or samples selected on the y axis. It seemed natural to map food position to note parameters in a similar way. Using the order in which food is selected to determine the order of notes played frees up the X axis to map to a parameter instead of time and also makes playing the game feel more like a progression through a composition: each piece of food adds to the sequence which is continually looping, the music plays and the composition progresses, there is no turning back or revising. By mapping the X axis to velocity pseudo rests can be added to the sequence by selecting food on the left.

Selecting notes requires some deviation from the normal snake mechanics which normally only make a single piece of food available at any time. This restriction would mean that players wouldn’t compose music, simply reveal it as they ate one piece of food at a time. At the other end of the spectrum turning every square in to food would mean that the next selected note would have to be adjacent to the last note, also overly restrictive. Making a limited number of pieces of food available at any time provides a nice middle ground, allowing the player some freedom in the choice of the next note selected, but not total freedom, a restriction which can lead to serendipitous melodies.

The other major mechanic in snake is colliding with your tail, which ends the game, but becomes harder to avoid as you eat food and get longer. One option would be to use that mechanic to intentionally end the game and the composition, but instead I mapped it to sample selection allowing the player to switch between sounds and start a new sequence to build up multi-timbral polyphonic music. By making the world toroidal players can simply let the snake circle around the world when they have finished composing.

A lot of these design decisions came out while implementing the game using processing/processing.js and HTML 5 audio — a technology stack I’d played around with a bit previously, but wanted to explore further. In the end, for this kind of application I don’t think processing brings enough benefit to outweigh the difficulties it adds to debugging. When running on top of Java errors are often reported as mangled Java call stacks and when running in the browser different errors appear as mangled JavaScript. While I can see the attraction to language designers and implementers of building on top of existing technology it often results in having to implement in one language and debug horrible unrecognisable code in another. Incompatibilities are also horrible. With a couple of hours to go I had the entire game running on Java, but was presented with a blank canvas with no useful Firebug errors when I exported to processing.js and having to perform a binary search by commenting out chunks of code to find the error. Not pleasant.

HTML5 audio is also a somewhat fragile technology. Generating an Audio element for each sample playback event leads to current browsers grinding to a halt while resetting and restarting audio elements often causes glitches and delays. Another problem is that JavaScript timers don’t provide enough accuracy for tight sequence playback timing. In the end I rebranded both bugs as features by switching from very transient drum samples which sounded messy to dubby bass and melancholy bell samples that work quite nicely with glitches and unintentionally loose timing.

At 10PM on Saturday night everything had come together enough for me to lose myself in an hour of ambient bleepy electronica and by the time the presentations started at 3PM on Sunday Disco Snake was done.

I’d like to thank all of the organisers and hackers that made Music Hack Day London a wonderful experience and have been pleasantly surprised at the positive reaction that Disco Snake has generated over the last week. The space between music interfaces and games is a very fertile one that I’ll be exploring further in the future and while it’s not there yet, I hope HTML5 audio fulfils its promise of bringing interactive music applications to everyone on the web in the very near future.

My recent experiments in to using Procssing.js and HTML5 audio to generate multimedia web applications didn’t get very far. I first tried generating a new HTML 5 audio element for each audio event, which quickly caused the browser to grind to a halt, and my attempts to reuse audio elements by resetting the playback position didn’t seem work, leading me to conclude that HTML 5 audio was only really useful for playing back long audio files, not for building sequencers that play back many short samples. When I spoke to @rem about my findings he was convinced that resetting audio elements should be possible and this weekend’s Music Hackday London has provided the perfect incentive and opportunity to dust off my experiments and start tinkering again. An hour in and sure enough I’ve managed to get audio elements to reset: it seems that the trick is to set currentTime after calling play() on the element, something that seems very counter-intuitive, but seems to work (at least in Firefox 3.6.8 and Safari 5.0.1 on OS X 10.6.4). Now I have reliable sample play back it’s time to start playing around with more interesting interfaces in Processing and there are 26 hours of hacking left: game on!