Jun 2, 2015

What happened with your Internet last night, and what we are doing about it

UPDATE: June 4, 2015

The page where customers who were impacted by this issue can request their credit is now live. Click here for more information.

UPDATE: June 12, 2015

For customers in CA, WA, and AZ who were impacted by this service issue, we have extended the availability of the online credit request form to Monday, June 15, 2015 at 12:00 pm EST. Click here for the form.

Last night, Internet service for a number of our customers in the western part of the country was degraded or unavailable for several hours.  We had a team on this immediately, and were able to restore full service to most customers by 9pm PT.

We know that having a fast, reliable connection to the Internet is vital and that interruptions of this sort are unacceptable. We’re sorry that we didn’t live up to that last night. We are giving a credit to customers who were impacted.

What happened

A piece of hardware in our backbone network failed yesterday morning. Hardware issues of this sort are fairly commonplace and redundancy in our network addressed any immediate concern.  

Most backbone networks, including Comcast’s, are designed to heal themselves and route traffic along alternate paths much like a detour would route automobile traffic around a street closed for construction.  

One type of traffic that gets automatically rerouted is Domain Name System (DNS) traffic.  Unfortunately, some of that traffic shifted in an unexpected way and overloaded local DNS server capacity causing many customers to experience service interruptions.

What we are doing about it

Since the incident, we have been working non-stop to fully understand the problem and bring additional DNS capacity online in the affected areas, so we can prevent it from happening again.

We all rely on the Internet and expect it to be there when we need it.  We are directly reaching out to those who reported problems last night to offer our apologies and a credit for lost service.

We are also building a Web site that impacted customers can visit to receive their credit. We will update this post with a link to that site as soon as it is available and will share the link on Twitter through our customer support handle @comcastcares.

Tags : Customer Experience, Customer Service, XFINITY Internet


Follow Us
comments powered by Disqus