Storage Toolkit

Internet Backplane Protocol - Design, Details and Desire

Rupert Goodwins ZDNet.co.uk

Published: 16 Apr 2002 17:44 BST

Although still at the research stage, the IBP Service and Client API have been written, and working software produced, illustrating along the way some of the practicalities of producing such a system.

IBP storage takes place on servers called depots, which accept requests from remote servers and return results called capabilities. There are very few operations, as befits a model that emulates IP's deliberately simple range of functions: allocate, manage, store, load, two flavours of copy, and return status.

Allocate reserves and assigns an array of bytes on the depot, specifying how big the array is, how long it will survive and other attributes. The array is protected from third party interference by cryptographically secure passwords; after allocating, the manage function lets the client move pointers within the array, check what's going on or set some of its capabilities. Store and Load move data into and out of the array, and Status tells the client about the depot, and changes some of its parameters.

The design of IBP breaks down into four major areas, making the most of the advantages of an Internet model while minimising the weaknesses and limitations. As the basic function of any storage scheme is to move data around, special functions copy and mcopy have been created to move data from depot to depot. It is possible to do this through store and load through a client sucking data from one place and putting it in another, but efficiency and trust issues mean that it's far preferable to make a depot a temporary client capable of moving data under its own steam, on request. Copy does a basic data transfer over a TCP stream, while mcopy is more general. It provides a range of functions, from simple data transfer to real-time media streaming, and gives the client considerable control over what to do in the case of errors, how to optimally use the underlying infrastructure, and so on.

As anyone who's been on the Internet for longer than a day will realise, security is one of the biggest headaches, and storage is particularly unforgiving here. Capabilities are returned by the depot to the client in the form of long, cryptographically secure byte strings. These then form the basis of client request authentication: if the connection between the client and the depot is secure and the depot itself remains uncompromised, the presentation by the client of the secure byte string will authorise further actions. This is the only security feature IBP supports -- the security of the data itself is up to higher levels in the client. The depot is entirely agnostic about what's stored within it, and stores it transparently.