The early days of RESTful hypermedia API design tends to involve lots of homogeneous collections. In the case of CREST vnd.ccp.eve.Api-v1 pointed to the logged in vnd.ccp.eve.ccp.Capsuleer-v1 which pointed to a vnd.eve.ccp.CharacterCollection-v1 of contacts which pointed to many vnd.ccp.eve.Capsuleer-v1 representations, one for each capsuleer contact.
Adding search makes things more complicated. A typical search resource will accept a query and return a collection of hyperlinks to heterogeneous resources. EVE allows capsuleers to have contacts that might be other capsuleers, agents, corporations or alliances. In the future capsuleers may also be able to have coallitions in their contacts list. In this hypothetical future a contacts management application developed now would end up receiving search results that it wouldn’t understand. After requesting a resource and receiving an unknown representation the client can ignore the resource, but it would be better to filter out unknown resources as part of the query.
Server side filtering is typically added to search via extra query parameters. In our CREST example we might add q=Jayne&type=capsuleer parameters if we just wanted to find the capsuleer Jayne Fillon. An unfortunate consequence of this design is that we end up with application level types used to filer searches as well as Media Types used for HTTP content negotiation.
It would be nice to just use content negotiation, but we want to filter the resources that are referenced by the returned resource, not the returned representation itself. The search request might return a vnd.ccp.eve.Collection-v1 representation, but we want to make sure that the hyperlinks in that collection only point to vnd.ccp.eve.Capsuleer-v1 resources.
As a client we’d like to specify is that we want search to return a vnd.ccp.eve.CollectionOfCapsuleerReferences. If the client wants to include corporations in the search results it should be able to specify a CollectionOfCapsuleerOrCorporationReferences. We’d like a richer media type system.
While this could be implemented in the backend just by switching on the Accept type, the combinatorial explosion of potential search results quickly makes this impractical. A simpler way to experiment with this approach would be to implement a proxy which can query the API for Collection-v1 representations and convert them in to arbitrary CollectionOfFooAndBarReference representations.
The proxy could also be used to inline reprsentations, allowing clients to request a CollectionOfFooAndBars rather than a collection of hyperlinks. If clients only wanted a subset of the full Foo and Bar representations thay might ask for CollectionOfJustNameFromFooAndJustNameFromBars. These additions would address some of the biggest headaches caused by API designers who have to decide which fields to denormalise in to collection resources to avoid clients having to make a huge number of requests in order to provide meaningful choices to users.
This approach potentially provides a lot of the benefits of GraphQL to RESTful clients just through normal HTTP content negotiation protocols and maintains the benefits of maintaining a small number of versions to API developers. If a composite type refers to a type which is deprecated in the underlying API, it can return as deprecated response as normal.
As with programming languages, it seems that if we’re going to support strong media types it’s useful to also support generic media types.
A year ago I gave a talk at EVE Vegas about building RESTful CREST applications. My #1 recommendation was to specify representations in requests, but that’s hard to do when there is little documentation on which representations are available and what they contain.
Fortunately CREST is self describing: send ...read more
Free Tests For Everyone!
Modern software development is sometimes colourfully described as being similar to firing tracer bullets at a target. Rather than spending time doing a lot of research, design and specification up front, the smallest, simplest version of the software is built and the feedback gathered from its use is used to ...read more
Investing In Testing
Last year I was talking to an engineer at Droidcon London who was working on an Android app with 100% test coverage. I immediately asked whether he thought 100% test coverage was worthwhile: many software engineering teams strive to achieve 100% test coverage, but few succeed because it’s an ...read more
There have been a number of blog posts recently about exciting new organisational structures. As Cory points out “Every early stage company thinks it has reinvented management”: a very dangerous belief when betting on a new organisational structure can be much riskier than betting on the wrong product.
It starts ...read more
Beyond Time Dilation?
EVE online is a remarkable game. On Monday over 2000 people spent over 20 hours destroying virtual spaceships worth 200,000 USD in real money in what was the likely the largest battle in a video game ever. That EVE is capaple of supporting such large engagements is an amazing ...read more
In mid-2004 I first started exploring Second Life. Version 1.4 had just been released and Philip Rosedale had said in the press release “My fantasy is to be Uma Thurman in Kill Bill, and now I can. I’d pay $10 for her yellow jumpsuit and sword moves and ...read more
Parse By The Sea
Brighton Digital Festival
The Brighton Digital Festival starts this week and I’m very happy to be helping out with Facebook London‘s contributions: Parse By The Sea, a mobile app Hackathon featuring Parse on the 26th of September, and helping to Connect The Brighton Digital Festival by sponsoring Metranet to provide high ...read more
One Universe, Many Scales
Following In My Father’s Footsteps
From 2 years before I was born, until just before I started working on Second Life at Linden Lab, my Dad worked at an innovative technology company with a large consumer photography business: Kodak. From January next year I’ll be working at an innovative technology company with a large ...read more
Caching Shared, Private Data With Ningx
As with many other social services, a large amount of the data in EVE Online and Dust 514‘s New Eden universe is shared between subsets of users. Some corporation data should only be accessible to the corporation’s members, market prices should only be accessible to capsuleers and infantry ...read more
Adding Vary Header Support To Nginx
Although Nginx supports proxy caching it doesn’t provide support for the HTTP Vary header out of the box. This is a problem if you want to use Nginx to proxy different versions of the same URI which Vary on Content-Language or proxy different representations of a RESTful resource specified ...read more
Load Balancing Stateful Services With Nginx
The EVE online network architecture uses stateful proxy servers which manage sessions for players connected to the cluster via the EVE client. The client sends requests to the proxy which are forwarded on to sol servers maintaining the game state and the sols send notifications to the proxy which are ...read more