Technology
Redirector, Our Cloud Connection Management Tool is Now Open Source
One of the great challenges associated with cloud computing at scale is how to manage the connection load when millions (and even billions) of devices are maintaining always-on cloud connections. We developed software to address this issue in our cloud-enabled X1 video platform, and today we’re excited to announce that we’ve made a version of the software available open-source.
We built Redirector because we weren’t finding available tools that supported the effective, scalable management of the sort of persistent, long-standing cloud connections that our cloud-native X1 platform requires.
Redirector is a server traffic-shaping tool designed to address two challenges with traditional load balancers: First, long-term connections are often maintained in hardware load balancers that were not able to handle scale in the order of millions of simultaneous connections. Second, even though such load balancers are able to distribute connections evenly amongst large groups of servers, they lack they ability to direct internal test clients to specific versions of servers and to release new software to backend servers in a controlled manner.
In building the Redirector server traffic-shaping tools we focused on several core areas:
Design
Redirector Framework
The Redirector framework is a dynamic context-based gateway tool. The Redirector uses service-discovery in combination with a rules engine to define the particular version of a given application that the client needs to be redirected to and returns an IP address of particular application.
Redirector Gateway
The core component of Redirector system is Redirector Gateway . It receives requests from clients and returns information about a particular host that the client should connect to. Redirector Gateway uses a distributed key/value store (Zookeeper) as source of truth for service discovery and Redirector WS for rule engine facilities. In situations when the source of truth is down, Redirector Gateway uses its local file system copy of Last Known Good service discovery and rule engine data.
Redirector Model
The Redirector Gateway processes rules based on the following model:
The Redirector Gateway considers all these constituents as a single aggregate called Redirector Model. Redirector Gateway uses separate model for each particular application. The Redirector Gateway updates the models on notification sent from Redirector Web Service through Zookeeper. When the Redirector Gateway receives a notification, it pulls the Model and uses it until the next notification. Every time the model is updated, it is also persisted in the local backup. Even if the Redirector Gateway is restarted, the persistent Last Known Good model facilitates the functioning of the Redirector Gateway with reasonable expectations even when the sources of truth may not be available to load the most recent models.
Redirector Service Discovery
The Redirector Gateway is designed to return IP address for any particular version of the application based on the rules using service discovery pattern. There are 2 modes currently supported for service discovery. One is static mode, and the other is dynamic mode.
In Static service discovery mode, during model update the Redirector Gateway takes a snapshot of all service discovery data from zookeeper, keeps it in-memory and then uses it until next model update. In Dynamic service discovery mode, as long as the host is registered in zookeeper it is available to be returned back by the Redirector Gateway. If the host is not registered in zookeeper, it will not be returned back as a potential target host.
Redirector Web Service
The Rules are managed by Redirector Web Service (Redirector WS) through an Admin UX . Redirector WS saves rules in Zookeeper and then Redirector Gateway loads new rules from there.
We strongly encourage everyone to use, improve and contribute back to the effort. You can download the source code at https://comcast.github.io/redirector/
Redirector could not have been possible without the work of many engineers and architects, drawing on feedback from our internal users across the organization.
(As a note, this blog describes the Redirector open-source software and its capabilities and does not describe specific services or approaches deployed by Comcast.)