cloudparts

Stream FrameworkDormant

A Python library, formerly called Feedly, for building news feeds, activity streams and notification feeds by fanning out each activity to followers’ feeds in Redis or Cassandra, with Celery doing the work.

What it is
Activity feeds
Type
Open source
Made by
Thierry Schellenbach
License
BSD-3-Clause
Status
Dormant. No release on PyPI since 1.3.7 in March 2017; a few community fixes were merged since, most recently in December 2025.
Facts checked
24 October 2026
Website Source code

What it is

Stream Framework implements fan-out on write. An activity is a small record, actor, verb, object, optional target, time and extra context, such as a user pinning a photo to a board. When it is added, a manager class stores it in the author’s own feed and then starts a fan-out: it looks up the author’s followers and inserts the activity into each follower’s feed, splitting the work into Celery tasks of 100 feeds each. Followers can be put in HIGH and LOW priority groups that go to separate queues, so active users’ feeds stay fresh during a spike while inactive users’ wait on their own workers. Reading a feed is then one sorted-range read, which is why the project describes itself as “heavy writes and extremely light reads”.

Feeds are sorted containers of activities. Timeline storage, which holds the sorted lists, and activity storage, which holds the activity data, can each be Redis or Cassandra. Besides flat feeds there are aggregated feeds, which group activities together, and notification feeds. It was built at Fashiolista, where it ran the flat feed, the aggregated feed and the notification system.

The project has been mostly idle for years. The last PyPI release is 1.3.7 from March 2017, aimed at Cassandra 2.1; a Python 3.10 fix and support for Amazon Keyspaces were merged into the GitHub repository in December 2025 but not released. Its authors went on to build Stream (getstream.io), a hosted feed API, which the README recommends over running Cassandra, Redis and Celery yourself.

Why teams pick it

  • A small, readable implementation of fan-out on write, with priorities and chunked background tasks
  • Aggregated and notification feeds as well as flat timelines
  • Redis for small or fast setups, Cassandra for large ones, behind the same API
  • BSD-licensed and pure Python, so it is easy to read and adapt

Watch out for

  • Effectively unmaintained: no release since 2017, and the released code predates current Python and Cassandra versions
  • You run Redis or Cassandra plus Celery workers and a broker yourself
  • Pure fan-out on write: an author with millions of followers means millions of feed writes per post
  • The PyPI metadata says GPL while the repository’s licence file is BSD-3-Clause

Specs

Compare with alternatives →
Data model
Activities (actor, verb, object, target, time) in sorted feeds
Storage engine
Redis or Cassandra for both timelines and activities
Written in
Python
Ways to run it
A library; you run Redis or Cassandra, Celery workers and a broker

How it fits with other technologies

A manager that fans each new pin out to the author’s followers (from the README) · python

from stream_framework.feed_managers.base import Manager

class PinManager(Manager):
    feed_classes = dict(normal=PinFeed)
    user_feed_class = UserPinFeed

    def add_pin(self, pin):
        activity = pin.create_activity()
        # adds it to the user's own feed, then starts the fan-out
        self.add_user_activity(pin.user_id, activity)

    def get_user_follower_ids(self, user_id):
        ids = Follow.objects.filter(target=user_id).values_list('user_id', flat=True)
        return {FanoutPriority.HIGH: ids}

manager = PinManager()

See it in a real stack

No architecture uses Stream Framework yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2012–2025

Thierry Schellenbach and his colleagues at Fashiolista built a feed library for its timelines and notifications, and published it on PyPI as Feedly in August 2012. Its design followed what large sites had described: Twitter’s 2013 architecture, Redis-based with a database fallback, was in the docs’ own words “very similar to Fashiolista’s old approach”. Releases came quickly, often several a week, through 2013 and 2014.

At the request of feedly.com, the RSS reader, the project was renamed Stream Framework, first released under the new name in October 2014. Its authors, Schellenbach and Tommaso Barbugli among them, also started Stream, a hosted service for feeds, and steered users towards it. The last releases came in October 2016 and March 2017.

Why it’s built this way

Fan-out on write
Reading a feed is far more frequent than posting. Copying each activity into every follower’s feed when it is created makes reads a cheap range read, at the cost of many writes.
Celery tasks, chunked and prioritised
Fan-out to many followers is slow, so it runs in the background in chunks of 100 feeds. Separate queues for active and inactive users keep the feeds people actually read up to date under load.
Redis or Cassandra storage
Redis keeps everything in memory and is simple to start with; Cassandra stores much larger feeds on disk with fast writes, which suits fan-out.

Timeline

  1. 2012

    First released on PyPI as Feedly, built at Fashiolista

  2. 2013

    Dozens of Feedly releases, from 0.1 to 0.9

  3. 2014

    Renamed Stream Framework at feedly.com’s request; 1.1.0 in October

  4. 2017

    1.3.7, the last PyPI release, in March

  5. 2025

    Python 3.10 and Amazon Keyspaces fixes merged, unreleased

Sources5 · facts checked 24 October 2026

Improve this page on GitHub