ODIN: Secure P2P on the Web

Posted by code_monkey_steve on Sep 24, 2006 Sep 24

The first thing you need to understand for this to make sense is that the magic of OFF is all in the block encoding which turns meaningful bits into ownerless, quasi-random blocks identified by their SHA1 hashes. Once this transformation is performed, there is no way for anyone to know the content of a block on the network, as it has no meaning until it’s combined with other blocks (and, in that way, may actually “belong” to multiple files, simultaneously).

The thing that first screamed “Rails app!” to me is the fact that, once the content has been OFF-encoded into blocks, any transport mechanism can be used to move these blocks around, allowing for a network-agnostic peer-to-peer system. Using HTTP, I could write my own smart, caching web peer (with combined web UI) in Rails, but since it’s using HTTP, I could also use dumb web server file servers as mass block stores (no CGI required!). Eventually, I could add (trivial) support to talk to the existing OFF network (also HTTP, AFAIK). If I watned to get really crazy, I could make the app a 3-way network bridge with BitTorrent (which also uses SHA1-identified data blocks).

This is an extension of earlier musings about extending HTTP (or coming up with some other standard) for enumerating mirrors of a particular web resource, something like a per-directory MIRROR file (in XML). A local web-proxy/daemon would cache the status, speed, and locality of various mirrors, and attempt to provide the best location for a particular resource (Use Case #1: Linux RPM repositories). Throw in resource discovery, and you have a basic peer-to-peer network, all over HTTP. While this project assumes that each peer is serving OFF-encoded blocks, there’s no reason why this assumption couldn’t be refactored-out, leaving a generic HTTP P2P system.

To date I’ve already completed most of the functionality of ODIN: fast block encoding (more on that later), file insertion and extraction, basic file transfer and peer discovery. I’ll post more when I have something to show-off. ;)