The Creation Engine No. 2

A couple of years ago I had great fun putting together the London Geek Community iPhone OSCestra at Open Hack London and I’ve been controlling Ableton Live with iPhone tapped to my guitar as part of 100 robots for a couple of years now so when @andybudd suggested I do a digital music thing for the Brighton Digital Festival I immediately thought that it would be fun to combine the 2 projects by doing a 100 robots performance with audience participation.

The iPhone OSCestra was effectively a distributed collaborative mixing desk with each person controlling the volume and effect parameters on one channel of a playing back Ableton Live set. For the 100 robots performance I wanted to go further and have the audience actually adding parts to the musical performance, so @toastkid and I added extra drum, bass, synth and sample tracks to the 100 robots live set and filled them full of samples that could be triggered by the audience.

While having the samples adjust in tempo to match each song was relatively simple, transposing them to match the key of each song was more complicated. First I built a custom slice to midi preset which mapped the sample transpose to a macro control and used it to slice all of the samples to MIDI tracks, then mapped all of the transpose controls to a single MIDI controller and added a MIDI track which output the appropriate controller value for each song to a MIDI output which was looped back in to Live to transpose the samples.

The next question was how to avoid the performance turning in to a mush if multiple drum tracks or bass parts were playing concurrently. To avoid this we put dummy clips on the normal 100 robots which muted the normal parts when the audience triggered parts were playing. In some cases we let the audience parts add to the music, in others the audience parts would play instead of the normal tracks.

A final question was how to avoid max and I getting lost when the normal parts we play along to were replaced by unfamiliar samples. To deal with this we set the clip quantization on the audience triggered clips to values longer than the clip length. This meant that even if alternate baselines were constantly being launched, we would still hear the normal bassline for a while at the end of each quantization period, so we would know where we were with the track. To tune these settings we did some fuzz testing with semi random MIDI data to see how much madness we could deal with and still manage to play the songs.

With the tests done it was time to perform with 100 robots and 100s of people at the Brighton Dome and Museum.

Many thanks to Steve Liddell for recording the Brighton Museum set, @aral for letting us experiment on his update conference and to everyone who participated and watched. If you’d like to host another performance, please get in touch and if you like the music, please check out the 100 robots blog and consider buying our album from bandcamp.

(Concept by @toastkid ) Movember is over. Thanks to everyone who donated.

Lots of exciting 100 robots news! Our debut album, Attack!, has been professionally mastered by Chris at Melograf Mastering who has done an amazing job and made the album sound incredible. The new version is already available at bandcamp and will be available on itunes, amazon and many other download services on Monday. To celebrate the launch we’re playing live at The Hope in Brighton tomorrow night and have set up a new blog where we’ll be giving away a track from the album free every month and I’ll be doing most of my 100 robots related music blogging from now on. Head on over and subscribe to the feed so you don’t miss out. Hope to see you at The Hope tomorrow!

There was a slide in the early talks that Cory Ondrejka used to give about Second Life about alien abductions in Second Life. One of the most exciting moments in Second Life for the early Lindens was when a resident constructed a UFO and flew around the world abducting other residents and then returning them to the world with a commemorative t-shirt. It was exciting because it was unanticipated. The Lindens had created a virtual world that enabled interaction and someone had taken it and run with it to create a fun and engaging experience.

So, once I’d finished implementing a simple interest management and collision detection system for the Brighton Digital Festival JavaScript Jungle to enable interactions, I thought I would implement an alien abductor as a hat tip to Second Life.

The JavaScript first adds a UFO from You’re The Boss 2 to the supplied div along with an SVG canvas containing a hidden translucent tractor beam path before binding to the see and tick events. The tick handler implements a state machine which either moves the UFO towards a random spot, a target creature that the UFO has seen or drags the target off screen for diabolical experimentation.

The most interesting part of the code on line 155 which replaces the target’s position method with one which returns the target’s position, but doesn’t update. This allows the UFO to move the target while the position updates made by the target’s own code call the new read only position method. Tom Parslow‘s boids look especially mournful flapping around and turning towards the flock while being captured.

While the alien abductions in Second Life and the JavaScript jungle are meant to be fun and mostly harmless, the same mechanisms that enable them can be used for griefing in virtual environments and malware in software at large. The ability for scripted objects in Second Life to self replicate caused dozens of problems with grey goo attacks for every amazing virtual ecosystem and many malicious cage attacks for every playful alien abductor.

The message passing concurrency model adopted by LSL actually made direct attacks on other scripts of the kind used by the JavaScript Jungle UFO very hard, but things are much harder in JavaScript’s browser environment even when separating scripts in iFrames.

Luckily projects like Caja and Belay (which is being worked on by another ex-Linden, Mark Lentczner ) are working on the problem of making multiple scripts work safely in the same browser.

The challenge for sandboxes like Second Life and the JavaScript jungle is to allow interesting and meaningful interactions with emergent properties and unanticipated consequences without allowing malicious scripts to destroy that environment. Building the JavaScript Jungle was a lot of fun and made for another great Brighton Digital Festival project. Many congratulations to @premasagar, @ac94, @purge and everyone else for making it a success. Maybe next time we can try to build a secure JavaScript Jungle that is both secure and expressive.

This week I experienced two remarkable combinations of music and the moving image.

Natures 3B from Quayola on Vimeo.

This evening I watched Nature — Mira Calix and Quayola’s audio visual piece which took video footage of flowers blowing in the wind and used motion tracking technology to generate music from the footage. As a concept it was interesting, unfortunately as music it was terrible. The beauty of the footage betrayed the folly of the concept: if a human were to compose music based on the beauty of flowers the way they moved in the breeze might feature, but wouldn’t be the basis of the entirety of the piece. The colour, form and memories triggered by the flowers would surely feature. Turning the flowers to a network of points modulating parameters reduced them to an interesting if psuedo-random system and the resultant synthesised music was predictably cold and pseudo random.

By contrast, a few days ago I had the pleasure to watch Manhatta, a black and white movie about Manhatten made in 1920 by Charles Sheeler and Paul Strand and accompanied by a new soundtrack by the Cinematic Orchestra. Where Nature used machines to generate it’s soundtrack based on an algorithmic interpretation of the movement of flowers, Manhatta uses humans to generate it’s soundtrack based on the emotional impact of the moving image on the musicians. The result is infinitely more moving. The music adds emotion to the moving image, combining feelings of wonder, awe, fragility and insignificance — a uniquely human reaction to the images of the worlds most amazing city that cannot possibly be understood or rendered by an algorithm, no matter how clever.

Art is a human reaction to our world, not something that can be captured in an algorithm.

A week ago over 5000 people streamed through the foyer of the Brighton Dome to see and build hundreds of amazing things at the first Brighton Mini Maker Faire. Luke and I went along with 2 laptops, a scanner and a pile of pens, paper, glue and scissors to make a video game with what felt like most of those 5000 people.

We arrived at 9:30 in the morning and were still working out how to plug out laptop in to the big plasma screen when the doors opened at 10:00. From then until the doors closed at 17:00 our table was a tornado of cutting, gluing, drawing and colouring as dozens of children and adults dived in to the task of drawing bosses for our shoot ‘em up with wild abandon. For a while my picture scanning, data wrangling and game copying efforts kept up with the stream of submissions and people were delighted to see their creations flying around on the big screen within minutes of their creation. Soon enough though, the stream turned in to a deluge and by midday I had a sizable backlog of pictures to process.

Despite working non-stop all day with only a 20 minute break to grab a milk shake and have a quick look around I ended up with a backlog of dozens of pictures at the end of the day. At that point another problem emerged: the game is designed to slowly get harder at each level, but with so many bosses to add the game would get impossibly hard before half of the bosses were seen. Realizing that I had a lot more work to do before the game would be finished I released an initial version at the end of the faire and collapsed in an exhausted heap at the after party.

All of this is by way of being a long winded explanation as to why “You’re The Boss 2” wasn’t finished a week ago. Last night I finally got around to scanning in all of the remaining images, tweaked the difficulty curve to make it possible to get to the end and released “You’re The Boss 2 Extended” which can now be downloaded here.

Despite being one of the most exhausting days of my life, it was also one of the most enjoyable. It was incredibly rewarding seeing dozens of children and adults alike delighting in creating something fun together and watching Thomas Truax perform with his DIY instruments while talking to a professional gingerbread house maker made for a truly magical end to the day. I’m very proud to have been part of the first ever Brighton (not-so) Mini Maker Faire and look forward to taking part in many more (although I might bring along a friend to help next time!).

I hope you enjoy playing You’re The Boss 2 as much as we enjoyed making it.

The Brighton Digital Festival continued this weekend with BarCamp Brighton 6 which was super interesting and lots of fun as always.

I was a bit worried that my Terra Nova style talk on the philosophy of games, virtual worlds and magic circles would be too esoteric, but the room was packed and the talk generated some great discussion.

A video of the talk is now available at The Internet Archive thanks to @stevepurkiss and the slides are available on SlideShare. Thanks to everyone who came along to my talk and BarCamp and to @jaygooby and @profaniti for organising a wonderful event.

One of the events that kicked off Brighton Digital Festival was dConstruct, the always thought provoking conference run by clearleft.

As usual I found most of the sessions interesting, but not always relevant as there’s a heavy design rather than development focus. The most relevant talk this year was Kevin Slavin’s final talk, Reality is Plenty, which argued that augmented reality is not the next big thing, just as it wasn’t in 2005.

Despite Kevin having a dig at Second Life and having spent a lot of time working on Augmented Reality with Blast Theory while at Nottingham University, I mostly agreed. While there are definitely use cases which benefit from augmented reality (fighter pilot navigation systems and things like Carbon Goggles which are all about making invisible aspects of objects visible) and virtual reality (simulation and virtual meeting spaces) there are plenty of others which are better served by other interfaces. Environments like Second Life are particularly exciting as they allow people to quickly prototype systems to discover which applications work and which don’t.

With both AR and VR it’s tempting to argue that they allow for intuitive interfaces as they model or overlay the real world: people know how to navigate a 3D space so they know how to use a 3D environment and they know how to use AR as they can see. Anyone who has done their time climbing the Second Life learning curve or trying to use AR to find their way around will know this clearly isn’t true. Apparently more abstract interfaces like maps, which talk to the mind rather than the senses are often much easier to use.

There’s a lot of work to be done to make both AR and VR as easy to use as 2D interfaces, let alone as natural as using real world senses. Now that the huge technical problems around networking virtual environments and tracking real world objects with mobile devices are starting to be solved, it is mostly UI work that needs to be done to make these technologies more widely used.

Even if the UX issues are solved there will still be many cases where speaking to the mind is much better than speaking to the senses.

Over the last couple of years I’ve found myself using python decorators to annotate handlers for web requests more and more, both when using Django and with micro-frameworks like mnml and newf.

Where the same functionality is required for all handlers, or the required functionality can be determined from standard request or response headers, using WSGI or Django middleware is fine, but where the required functionality is varies based on the handler its much cleaner to use a parameterised decorator than poluting the environment or response objects just to control the middleware. Functionality can be added to a framework as a suite of decorators and plugged together in an aspect oriented way like lego to easily build up sophisticated behaviours.

Unlike other mechanisms for implementing macros, templating or aspect orientation that introduce a new language, python decorators are pure syntactic sugar that under the hood are simply rewritten as python expressions:

@requires_oauth_scope("email")
def notify_friends(request):
    pass

Is simply shorthand for:

def notify_friends(request):
    pass

notify_friends = requires_oauth_scope("email")(notify_friends)

This simplicity is powerful as it allows decorators to also be used as normal functions, for example to build up higher level decorators that bundle common decorator configurations, but it also means that decorators potentially interact badly with another powerful Python feature: introspection.

In the above example the undecorated notify_friends function has the __name__ “notify_friends”, but the decorated function has the __name__ “requires_oauth_scope”. When decorators are used extensively, this can seriously impact the usefulness of introspection for debugging or generating documentation.

Decorating your decorators with the functools @wraps decorator, which copies the __name__ of the wrapped function over to the wrapping function solves this introspection problem, but introduces another: the decorators now become invisible to introspection. In the example above the __name__ of the decorated function would now be “notify_friends” as in the undecorated case, but we wouldn’t know that the function had been decorated or not.

A potential solution to this new problem is to store the details about the decoration in another attribute that can be inspected at runtime. In addition to copying over the __name__ attribute, functools.wraps also copies over the target __dict__ by default, allowing it to be used to store information about the decoration and be correctly copied over when decorators are chained:

from functools import wraps

def requires_oauth_scope(scope):

    def decorator(target):

        target.__dict__["my_project_requires_oauth_scope"] = scope

        @wraps(target)
        def wrapper(*args, **kwargs):
            # return target(*args, **kwargs) or FORBIDDEN if token does not have required scope

        return wrapper
    return decorator

By constructing decorators in this way we get the benefits of python decorators and more declarative C# style attributes that are visible to introspection.

Back in 2005, while I was working on Second Life in Nottingham, before Linden Lab Brighton existed, I ran a workshop as part of the Screenplay “Boss Frenzy!” day at the Radiator Festival which allowed children to collaboratively create a computer game by drawing or making bosses with collage.

Dozens of people came to the Broadway in Nottingham and got busy with pens, pencils, paper, scissors, glue and magazines to design bosses for our “You’re The Boss!” shmup. We had an amazing time and created a charming and delightful game which I talked about on the original Second Life blog.

I immediately thought of it when we started planning the Brighton Maker Faire a couple of months ago and was delighted when the project was accepted. Unfortunately 6 years of bit rot had taken it’s toll and disaster loomed after discovering that I’d hosted the Game Maker files on the web space provided by an old ISP account and didn’t have them on my patchy backups. Luckily the ever amazing Torley had a copy of the executable and with the help of a decompiler I was able to recover the Game Maker files I needed to run the project again.

So, if you’re near Brighton on the 3rd of September and like the idea of collaboratively making an arcade game with scissors, glue and pens then please come along. If you have a Windows machine then check out the game we made in Nottingham in 2005. I think it’s still charming and delightful 6 years on. You can download it here.

This time round I’d like to make the game completely out of Creative Commons licensed works, so please suggest CC licensed books, comics and pictures that might make good source material in the comments, or bring them along on the day.