Almost exactly 6 years ago, the incredible Cory Ondrejka and I met for the first time in real life (having previously blogged together on Terra Nova) at the Austin Game Conference 2004, where we got on like a house on fire. Several months later I joined Linden Lab and (as James and Jim Linden were already taken) Babbage Linden was born. The first task Cory asked me to do was embed the Mono virtual machine in to Second Life as a next generation scripting engine. It was a wonderful project to work on, involving authoring a new LSL compiler back end to generate CIL bytecode, a scavaging garbage collector to allow assembly unloading and a microthread injector to allow 10s of 1000s of scripts to run concurrently on Mono in a single process (work that has been described in detail in talks at ooPSLA, Lang.NET and FOSDEM ). As a long term R&D project it was put on hold a number of times to make way for more important projects like HTTP-Out, Message Liberation and Het-Grid, but eventually we shipped a Second Life simulator that embedded Mono in 2008.
Running LSL on Mono in Second Life was a huge win, allowing scripts to run 100s of times faster in some cases and reducing the average memory footprint of scripts in Second Life by a third, but the big hairy audatious goal for Mono in Second Life was always to enable other languages that targeted the Common Language Infrastructure to run in Second Life. After waiting until the end of last year for Mono 2.6 to implement the bytecode verifier and CoreCLR security sandbox which allowed us to safely run other languages on Mono inside the simulator we started work on adding support for C# in Second Life at the beginning of this year. A team of Linden engineers in Brighton and California did an amazing job overcoming an array of challenges and got to the point where we had Silverlight chess demos, run time configurable script profilers and scripts that used .NET reflection APIs to visualize other C# scripts running in our development simulators earlier this summer.
Alas, tomorrow is my last day at Linden Lab and Babbage Linden will never get to see C# scripts running in the wild in Second Life, but I very much hope that I do. I hope that C# support is eventually added to Second Life and that I don’t have to wait 170 years to turn the handle. As another Babbage said when he failed to build the Difference Engine: “Another age must be the judge”.
Jonathan Smith talked about the dangers of giving people too much freedom in his talk about the Lego Games. Lego is almost a shorthand for freedom: the easy to understand system of knobs and anti-knobs that allows 2 4x2 blocks to be combined in 9 million ways an ultimate sandbox aspired to by games and virtual worlds like Second Life. This open, free system led Travellers Tales to add lots of open, free features to it’s early Lego games that were largely ignored by players who need boundaries and feedback from the game to determine ‘what I want versus what’s expected of me’. Choosing freedom and rebellion is more meaningful when it is clear that I am exercising my freedom and not doing the expected.
Margaret Robertson talked about and in the current sandbox game du jour, Minecraft, which has enough terror and threat in its horror filled night to make the choices made during the day meaningful and to reward mastery of it’s sandbox — a sandbox that compelled Margaret to stay up until early in the morning carving her slides out of earth, building them out of wood and animating them with flowing water and flames burning down the assertion that “games = points”.
It was this misguided assertion that Sebastien Deterding talked about in his look at the ‘gamification’ of the world around us. When all that gamified web sites like foursquare do is allow the accumulation of points and badges there are no meaningful choices, no mastery, no way to rebel against expectations, no play and no fun. Gamification results in loyalty schemes that are no more meaningful than Progress Quest.
The importance of being able to rebel against expectations was echoed by Alexis Kennedy’s talk about delicious misery in Echo Bazaar, a social game that would be another meaningless progression to inevitable success if it weren’t for contrarian missions that allow players to opt-in to getting their characters exiled for scandal or driven insane by demons. These missions inflict real harm on characters, but when properly signposted are the most enjoyed and shared missions: allowing players to be badass. When a game makes success inevitable, misery and failure is play and meaningful escape.
Pat Kane, formerly of Hue and Cry and more recently author of The Play Ethic gave a fascinating talk about wordplay, humour and his journey from disillusionment at the comedy industry, to fascination with humour through the Old Jews Telling Jokes’ stories of Jews laughing in the face of persecution. When misery and failure is inevitable, humour and play is rebellion. An ultimate, meaningful demonstration of freedom and humanity when all hope of victory is gone.
Rock Band does a great job of inspiring people to play music, can you develop a game that inspires composition? Lumines and Rez create music while you play, can you make games where music creation is the goal, not a side effect? Pictionary does a great job of using game mechanics to overcome creative block, can you use other game-like constraints to inspire creativity? These were among the questions I asked at GameCamp 2 a few months ago and I was keen to explore them at Music Hack Day London a week ago.
The spectrum of potential game-like musical composition tools is huge, ranging from traditional recognisable music interfaces like keyboards and step sequencers at one end, through things that are designed to be both music interfaces and games like Fractal, Rez and Lumines in the middle to things that are recognisably games at the other. While the middle ground is incredibly interesting, 24 hours at a hack day isn’t really long enough to develop a brand new revolutionary hybrid game/music interface, so instead I decided to repurpose an existing game as a sequencer and picked the simplest one I could think of — Snake.
With the interface chosen, the next thing to do was to think about how to map the interface to music composition. The core mechanic in snake is eating food placed on a grid. Grids have a venerable history in music as step sequencer interfaces with time growing form left to right and pitch or samples selected on the y axis. It seemed natural to map food position to note parameters in a similar way. Using the order in which food is selected to determine the order of notes played frees up the X axis to map to a parameter instead of time and also makes playing the game feel more like a progression through a composition: each piece of food adds to the sequence which is continually looping, the music plays and the composition progresses, there is no turning back or revising. By mapping the X axis to velocity pseudo rests can be added to the sequence by selecting food on the left.
Selecting notes requires some deviation from the normal snake mechanics which normally only make a single piece of food available at any time. This restriction would mean that players wouldn’t compose music, simply reveal it as they ate one piece of food at a time. At the other end of the spectrum turning every square in to food would mean that the next selected note would have to be adjacent to the last note, also overly restrictive. Making a limited number of pieces of food available at any time provides a nice middle ground, allowing the player some freedom in the choice of the next note selected, but not total freedom, a restriction which can lead to serendipitous melodies.
The other major mechanic in snake is colliding with your tail, which ends the game, but becomes harder to avoid as you eat food and get longer. One option would be to use that mechanic to intentionally end the game and the composition, but instead I mapped it to sample selection allowing the player to switch between sounds and start a new sequence to build up multi-timbral polyphonic music. By making the world toroidal players can simply let the snake circle around the world when they have finished composing.
A lot of these design decisions came out while implementing the game using processing/processing.js and HTML 5 audio — a technology stack I’d played around with a bit previously, but wanted to explore further. In the end, for this kind of application I don’t think processing brings enough benefit to outweigh the difficulties it adds to debugging. When running on top of Java errors are often reported as mangled Java call stacks and when running in the browser different errors appear as mangled JavaScript. While I can see the attraction to language designers and implementers of building on top of existing technology it often results in having to implement in one language and debug horrible unrecognisable code in another. Incompatibilities are also horrible. With a couple of hours to go I had the entire game running on Java, but was presented with a blank canvas with no useful Firebug errors when I exported to processing.js and having to perform a binary search by commenting out chunks of code to find the error. Not pleasant.
HTML5 audio is also a somewhat fragile technology. Generating an Audio element for each sample playback event leads to current browsers grinding to a halt while resetting and restarting audio elements often causes glitches and delays. Another problem is that JavaScript timers don’t provide enough accuracy for tight sequence playback timing. In the end I rebranded both bugs as features by switching from very transient drum samples which sounded messy to dubby bass and melancholy bell samples that work quite nicely with glitches and unintentionally loose timing.
At 10PM on Saturday night everything had come together enough for me to lose myself in an hour of ambient bleepy electronica and by the time the presentations started at 3PM on Sunday Disco Snake was done.
I’d like to thank all of the organisers and hackers that made Music Hack Day London a wonderful experience and have been pleasantly surprised at the positive reaction that Disco Snake has generated over the last week. The space between music interfaces and games is a very fertile one that I’ll be exploring further in the future and while it’s not there yet, I hope HTML5 audio fulfils its promise of bringing interactive music applications to everyone on the web in the very near future.
My recent experiments in to using Procssing.js and HTML5 audio to generate multimedia web applications didn’t get very far. I first tried generating a new HTML 5 audio element for each audio event, which quickly caused the browser to grind to a halt, and my attempts to reuse audio elements by resetting the playback position didn’t seem work, leading me to conclude that HTML 5 audio was only really useful for playing back long audio files, not for building sequencers that play back many short samples. When I spoke to @rem about my findings he was convinced that resetting audio elements should be possible and this weekend’s Music Hackday London has provided the perfect incentive and opportunity to dust off my experiments and start tinkering again. An hour in and sure enough I’ve managed to get audio elements to reset: it seems that the trick is to set currentTime after calling play() on the element, something that seems very counter-intuitive, but seems to work (at least in Firefox 3.6.8 and Safari 5.0.1 on OS X 10.6.4). Now I have reliable sample play back it’s time to start playing around with more interesting interfaces in Processing and there are 26 hours of hacking left: game on!
