Here at Comcast, in addition to working on innovative products, we are constantly working on the software infrastructure that sits "below the waterline" to make our services faster and more reliable.
Today we’re excited to announce that we’ve open sourced Sirius, a distributed system library Comcast uses to speed up our applications’ access to reference data. In our case, our reference dataset is the world of entertainment: when was Despicable Me released; how many episodes were in Season 4 of Seinfeld; when the next episode of The Voice is airing (and on which channel?) or even which shows are available to watch during events like our recent Watchathon.
Many of our applications, like the XFINITY TV website or the XFINITY TV Go, need access to this data with nearly every action—so having very fast access to it is a must-have for a responsive user experience. Existing open source technologies didn’t provide us with the right combination of this fast access with developer convenience, so we set about inventing a new way of managing this reference data. Sirius allows our applications to keep this reference dataset entirely within memory using native data structures, achieving our desired tradeoff.
We’ve been using Sirius successfully in production since August 2012—nearly two years—and are happy to announce that it is available under an open source license from our Github account: http://github.com/Comcast/sirius. As a community-oriented company, and one where we rely every day on open source software, we believe it is important to contribute solid technology like Sirius back into the open source community to enrich that ecosystem for all.