Push syndication

[fanf]

Some of my friends think Livejournal should be news though I'm not really convinced. Although using NNTP as the transport protocol has significant efficiency advantages, it doesn't help with the interesting problems of distributing LJ's network effects.

I tend to think that a more modern solution to the efficiency problem would be to create a way for (RSS or Atom etc.) syndication feed updates to be pushed rather than pulled (and polled). Fortunately this is a sufficiently obvious approach that other people are working on it so I can just sit back and let the lazy web do its thing. But what protocol should be used for the notifications?

My answer comes via the Radio Free RFC episodes about XMPP. The Extensible Messaging and Presence Protocol, aka Jabber, is the IETF's open standard for instant messaging. It has an extension known as PubSub which supports generic publication of update notifcations and subscription to a notification feed, and there's a profile of PubSub for handling updates to an Atom feed: http://www.xmpp.org/drafts/draft-saintandre-atompub-notify-02.html

All that remains to be worked out is how to handle the interesting access control features of "friends" lists in a distributed and easy-to-use manner...

Comments

[mstevens.livejournal.com]

Distributing blogs is easy - one just uses movable type.

Distributing the friends page is easy - everyone can use some local RSS aggregator (eg bloglines (http://www.bloglines.com/) if it was Open Source).

I'll handwave the scalability issues a bit, but they're actually pretty decent, because everyone is only reading a few (hundred) other blogs.

The hard part is the identity management - access control on posts and authenticated ids for comments. I've thought about this a little, as far as I'm aware ciphergoth has thought about it a lot more and more professionally. My ideas are roughly this:

We assume a distributed livejournal package similiar to Movable Type that runs on a server and provides both a blog and an aggregation service. It autogenerates a GPG key for the user and provides it at a known location on the blog (possibly referenced by meta tags in the blog HTML).

Everyone queries for RSS feeds, but their request is GPG signed by the requesting aggregator. The server looks up the GPG key for that user from their blog, verifies the request, and if access is allowed, serves GPG encrypted RSS data back to the requestor.

People still setup their friends lists in a similiar manner to LJ, but instead of entering usernames they enter blog URLs.

A similiar protocol for authenticated comments. Note the user is never aware they're using encryption - passphraseless keys are automatically created and stored on the server.

[pjc50.livejournal.com]

The "interesting access control features" are the core of livejournal's advantage, not an add-on. Note also that it's very hard to distribute, as e.g. the information about who can see a post is private to the poster.

Livejournal solves the single-sign-on problem within its domain. It also provides centralised anti-abuse services, another thing which is hard to replicate.

[ewx.livejournal.com]

It's not just NNTP(-like) transport that I want, also the better user interface.

[senji.livejournal.com]

Also there are comparatively minor (but important) details like a consistent commenting interface to worry about.