On Friday I jumped on the train to London to attend the first scalecampuk, an unconference about scalability, at the Guardian offices.

The sessions were all very interesting and mostly very relevant. I learned new things about XSS and CSRF and Django’s defences against them from Simon Willison, new things about messaging, pubsub, queues and data stores (process 1 message at a time, use message hospitals, send URLs to unavailable data that can be polled for with JavaScript and that just check memcache entries) lots about Varnish ) and it’s use at Wikia from Artur Bergman (Wikia runs off 18 apaches and 8 varnishes with 60GB of RAM and SSDs to serve 25 million pages and 950Mbps at peak, Varnish is generally better than squid ), but you need a modern kernel).

Lots of the talks were about moving storage, caching and queuing out of the application and just writing a small piece of business logic to tie them together. Against this background Alex Evans’ talk about the back end for Media Molecule’s Little Big Planet stood out like a sore thumb. Having not enjoyed using a Java web stack, Alex has just rewritten the whole of the back end as proprietary technology as a single binary in order to know the code from end to end. While it may be true that having custom physics or rendering middleware might distinguish Little Big Planet from other games, I can’t believe that custom technology to serve HTTP requests is going to be a competitive advantage. I hope Alex’s good ideas become Redis contributions rather than a maintenance nightmare and barrier to agility.

The lightning talks were also very good. Simon’s “ScaleFail” talk about the Guardian MP expenses app was hilarious (lesson: do load testing) and Gareth’s talk about Dumbo (a Python Hadoop client) was very useful.

At times it felt like the talks were suited to an ops audience, but “Dev’s should know about this!” was a regular refrain. Don’t worry: I listened and learned a lot. Thanks to everyone who made it a great day.

Was without a doubt the phrase I heard most often yesterday, especially if you include variants like “Not Like Second Life”, “A bit like Second Life” and “Unlike Second Life”. Whatever else it’s achieved, Second Life has definitely become the frame of reference for the small and somewhat myopic crowd that made up the delegates at the sparsely populated Virtual Worlds Conference in London yesterday.

Vastpark is not like Second Life because it works in a web browser. Everyone on the web integration panel seemed to agree that virtual worlds in a browser is the next step, so I was glad to be there to question the TechCrunch consensus. How does having a world in a browser help? What does back and forward mean to a virtual world? What does it mean for presence to have 10 tabs open looking in to different parts of the same virtual world? Why would you want your view further constained by extra web browser widgets? Isn’t 3D in the browser going to be a blood bath for the next few years? Aren’t you really just using the browser as a download path? I suggested that the final question was the real reason that developers are pushing virtual worlds on the web and that the integration that most people want is to be able to use existing web and 2D media while using virtual worlds and use web services as a universal data bus between virtual worlds and other web aware platforms.

MPEG-V is not like Second Life because it’s a standard defined by 35 companies which is much better than the emerging Linden led standard according to Dr. Yesha Sivan in what was the worst talk I’ve heard in a long time. Not only did he make the standardisation process sound like a 3 year political bun-fight by people who didn’t know much about virtual worlds and who might come up with a bad standard, he managed to spell MPEG and Google incorrectly, called Sun’s Darkstar, Blackstar and attributed a Ugotrade quote to Philip Rosedale amongst other clangers. He was roundly rebutted by a large part of the audience including Tara5 Oh who questioned the need for old fashioned standards processes in the web era. Thank goodness for rough consensus and running code.

Most of the virtual worlds talked about in the investment panel were not like Second Life, but were nearly all Club Penguin clones. This copy the big exit attitude was called out by one of the audience as it seemed to be at odds with a lot of the talk about wanting to back the first in a market, but at least one of the panel is still looking for a successful 18+ social world play. The panel ended with a show of hands from people wanting money and people wanting to invest, but the economic climate made the whole affair very muted with lots of the panelists saying that they are slowing down rates of investment as it’s difficult to get existing companies off their books.

As with Virtual Policy 08 and the Virtual Worlds Forum the most valuable parts of the conference were the spaces between sessions. I had another very worthwhile discussion with Adam Frisby of OpenSim about C# script compatibility between OpenSim and Second Life. The straw man design we talked about was to have an idiomatic .NET interface for event handling that can be used by C# scripts and adapted for LSL scripts and a set of static library methods for manipulating the world that would be used directly by LSL scripts and wrapped by user created libraries to provide an idiomatic object oriented interface. Adam was particularly interested in the idea of user created wrapper libraries as it would allow the creation of an OpenSim interface library that could be ported to Second Life and implemented in terms of the ll* static methods. OpenSim could then agree to support the common behaviour of this library in Second Life and OpenSim instead of having to support the gamut of ll* methods some of which don’t map well to OpenSim internals. As well as defining a common set of events and ll* static methods that are supported on both platforms there would need to be a way of extending the interface with new events and library methods. In addition Adam was interested in making the event propogation configurable so that a single script could respond to events on many objects in a scene. This would effectively add a script interest management layer to OpenSim’s scripting interface. Where platforms provide differing interfaces to scripts we would also need to decide how scripts query the available interfaces or how they behave when interfaces are not available.

Overall a worthwhile trip, but not because of the conference. This Friday I’ll be talking at the online head conference about conferencing in Second Life which has the advantage of requiring no travel making marginal conferences like the Virtual Worlds Conference less risky to attend while allowing all of the serendipitous networking opportunities that make real life conferences worthwhile.

A couple of weeks ago the great and the good of web development descended on Brighton for the wonderful clearleft produced dconstruct conference and once again I’m glad I went along.

Steven Johnson kicked off with a talk about how Dr. John Snow’s innovative data visualization of a cholera epidemic and the wisdom of dead crowds helped convince people of the water borne nature of the disease. It was an interesting story, but it mostly ended up being a plug for his book and geoblogging aggregator outside.in.

Next up, Aleks Krotoski talked about how games had spent decades creating incredibly compelling user experiences in silos without much contact with each other the academic HCI community or the web. Meanwhile the web is very interested in creating similarly sticky experiences using virtual rewards to encourage participation. Aleks’ conclusion was that the two communities should talk and I agree.

Daniel Burka talked about similar themes in his talk about the evolution of Digg. The most interesting anecdotes where about how top diggers started off as a good incentive, but became a disincentive when new users saw how unachievable the scores had become and how the recommendation engine is now a good way to encourage some of Digg’s passive audience to get involved.

Matt Jones and Matt Bidaulph talked about their successful Silicon Roundabout startup dopplr. Jones talked about visual design and delighters which sounded a lot like Alek’s virtual rewards in games. SL uber-hacker Bidaulph talked made another gaming analogy, talking about how embedding dopplr in other sites and vice versa achieves a similar seamless experience to streaming maps in games: removing the load screens and jumps that used to bedevil console games and still are the normal experience when using the web. He also talked about the importance of using message queues and asynchronicity in services like dopplr which pull information from across the web.

Joshua Porter‘s talk on Leveraging Cognitive Bias in Social Design was the stand out talk for me. He talked about exploiting people’s tendency to pattern match to generalise isolated positive case studies on web sites and on framing account creation as something to do to avoid losing features rather than something that gains features as a way to play on the tendency to value losses greater than gains. His description of the how the 9x mismatch between customers (who over value the application they already have by 3 times) and developers (who over value the application they have developed by 3 times) creates a huge barrier to application adoption was particularly interesting.

Tantek Celik‘s talk about using hCard and rel=me links to create portable, auto-updating social network profiles and data to reduce the fatigue induced by inviting all of your friends to many social networks was the most practical session of the day. I’m going to try playing around with rel=me links and Google’s social graph API here soon.

Jeremy Keith gave a grandiose talk to end the day which wove together psychohistory from Asimov’s Foundation Series with Critical Mass and The Wisdom of Crowds to talk about how network effects and power law distributions cause some social software to explode in popularity while others wither, but that despite The Tipping Point being sold in business sections as a how-to book, it is fundamentally a retrospective and that predicting or engineering tipping points or network effects is notoriously hard. It was a great talk and the conclusion that social software is more of a lottery than a science is valid, but still: you have to be in it to win it.

Well, not exactly. Having blogged previously on Terra Nova, the original Creation Engine and currently on the Official Second Life Blog, I’m not exactly stumbling blinking in to the blinding light of the blogosphere. Recently a number of things have come up that I’ve wanted to write more than 140 words about, but that wouldn’t fit on the Official Second Life Blog any more, so I’ve finally stopped mooching off other people and set up my own blog.

One reason I hadn’t got around to it sooner is that I’ve been torn between platforms. Although it’s been tempting to throw up a WordPress blog every time I’ve had something to talk about, I really wanted to build a blog in Django that I could tinker and experiment with. Although it’s just a matter of plugging bits together, it still takes a few hours to get a basic Django blog up and running and longer to add all the bells and whistles. I finally managed to break the impasse last week when I came across this list of Django blog engines and after some routing around decided to go with byteflow which has all the bells and whistles but is made of standard Django bits and is eminently tinkerable.

So, that’s what you see here: a default byteflow blog running on Django trunk running in mod_python as a virtual host (alongside the slateit.org and carbongoggles.org Django apps) inside apache2 running on ubuntu dapper on a virtual machine hosted by bytemark. It took long enough to get round to, but once I’d found byteflow it only took an hour to set up. I’ll be kicking the wheels and tinkering over the coming weeks, but if you find anything broken, please let me know.