I’ve put Wingate on temporary hiatus while I work on the next big piece of that project: Bidirectional Federation. Wingate as a standalone community service is coming along nicely, but the real power - the real end game - was to allow communities to share channels between themselves. In effect, allow communities to create their own Virtual Social Networks. Most current federation solutions are more like “RSS On Steroids” - FORBUS hopes to make it possible for subscribers to post data back to publisher instances.

The requirements for this were that:

  • the transport layer be something open-source and easy to provision
  • the library itself be unopinionated in terms of how the data to drive it is stored
  • the transport be something very fast and capable of handling immense bandwidth if needed
  • the mechanism be “fire and forget” so that publishers won’t be blocked by slow subscribers, and subscribers can handle incoming traffic by whatever means they choose without fear of affecting anything upstream
  • the mechanism utilize already proven acl/auth for better security
  • the transport needs to support both pub/sub and list/queue capability
  • the transport needs to support federated-access caching of entities

The best fit for this - at least what looked best to me from the perspective of a Rails developer - was Redis. Hence the name of the library became “FORBUS” which stands for:

Federation Over Redis BUS

Up until the latest version of Redis I was leaning towards something like RabbitMQ, but then Redis added ACL capability. It was now possible to create additional users and grant them access on a keyword level. Now it became possible to create message bus queues for a specific client and only that client would be able to see it. Perfect! This work borrows code from a Redis-based message bus I was working on a couple of years ago called “Redbus”.

Once I get further along I’ll open up the repository, but in general the services provided for federation include:

  • A cache space for “Imprints” of user profile information (name, avatar, handle, profile-link). Instances can use that to drive creation of virtual users on their local instance
  • A cache space to list all available channels, their latest post id, and details about the channels, Subscribers can use this to request federation access and keep track of “latest post” counters.
  • Mutual message-bus LISTs on each end of a federated Channel - the Publisher (Channel owner) sends out new posts on their LIST, the Subscribers sends out replies on their LISTS which the Publisher ingests and turns into Posts which then go out to all Subscribers on their LISTS
  • An RPC capability whereby one side of the Federation can request some specific information and can wait on a keyword to appear in the the other side’s message-bus as a reply.

There’s obviously more to it, there’s an ecosystem of models that provides the framing that an App will use to drive this. Above that up in the App’s code there are no limitations. The information about Redis access and subscribed channels and all the rest can be represented and persisted in any way the coder wants. The framing is mainly to provide a way to describe and organize what’s going on.

The plan is to get FORBUS operational, and then refactor Wingate to use it. This will require some torqueing around of Post, User, and Channel models and how they operate. The end result should be a smaller more flexible monolith which has federation capability.