The X1 Voice Remote has been extraordinarily popular with customers since we first introduced it in 2015. We know the experience is amazing, and that people love to use it, so the digital home engineering team set out to discover how it could leverage technology built for television to deliver innovative connected home experiences.

Last month we announced a bunch of powerful new voice commands for home automation and security. Using these commands, customers can turn up the heat by simply saying “Xfinity Home, I’m cold”;  ask the remote to show the feed from a specific camera in their home; or even change the colors of their smart lights.

The things we love about the voice remote in a connected-home setting are very similar to the factors that make it such an awesome way to watch television. Voice doesn’t just replace button presses. As our products become ever more feature-rich and capable, voice prevents them from becoming too complex by flattening the user interface in ways that are intuitive and profound.

Today, those experiences are seamlessly available to our connected home customers, but getting to that point required a good deal of work by our engineering teams behind the scenes to create new experiences without harming those already in place.

We started creating connected home commands in 2016 with a couple of relatively simple utterances. Customers could say “Xfinity Home, Arm” or “Xfinity Home, Disarm” to arm or disarm their home security systems. Even that step required some new approaches.

First, we added a “guard word” for connected Home commands. By advising customers to say “Xfinity Home” before their command (“disarm”), we could ensure that our voice control platform didn’t go looking for a movie or a song called “disarm”.  Adding a guard word opened a lot of possibilities, but we still didn’t have a way to support more complex functions.

To accelerate that process, we created a dedicated voice team, focused entirely on building voice integrations for connected home commands. The voice team works closely with the Xfinity Home apps team, ensuring that new features and tools are always considered within the context of voice control.

The newly formed voice team built our Voice Action Processing Service (VAPS), which processes commands that come from Comcast’s homegrown Natural Language Processing agent and determines which other services and API’s to call. An example of this would be launching the app for viewing a camera feed.

With that platform in place, we began to expand our connected home voice capabilities. This included adding direct commands for lights and thermostats using Xfinity Home’s ZigBee Radio. When we made that update, customers could say “Xfinity Home, all lights off” or “Xfinity Home, set the bedroom thermostat to 72 degrees.” Later that year, we added the ability to control thermostats and other devices that don't use ZigBee by connecting directly in the cloud. 

From there, we worked closely with our Applied Artificial Intelligence team to develop the capacity for more natural commands. Now saying “Xfinity Home, I’m hot” will trigger the system to drop the thermostat by a couple degrees, while saying “Xfinity Home, make it brighter” will raise the dim level of connected lights.

It’s all a work in progress, and the fun thing about what we’re doing is that we’ll never really be done. As we add more connected home and automation features, our team is now designed to immediately start thinking of how to adapt voice control to deliver those features. We are excited to share what’s next.

(Shiv Dhondiyal, Senior Manager, Software Development & Engineering co-authored this post)