Verizon Wireless experienced another outage on its LTE network on Wednesday, causing customers across the country to lose access to their ultra-quick mobile broadband connections and deal with the slower data speeds of Verizon’s 3G network. Verizon restored the network to normal late Wednesday, but this latest glitch shows that Verizon’s one-year-ancient 4G rollout is still experiencing some growing pains. As the first to launch LTE on a large scale, Verizon has become a guinea pig for the entire industry, making it the first operator to encounter and deal with the bugs remaining in the new technology.
Unlike the major outage Verizon experienced in April, Wednesday’s LTE problems didn’t affect all customers and it affected them intermittently. April’s outage affected all high-speed data connections to 4G customers – new HTC Thunderbolt and MiFi owners couldn’t even get a 3G connection and had to resort to the dial-up-modem-like speeds of Verizon’s 2G network.
This time around customers affected couldn’t link up with the LTE network, and while there were reports of some customers losing their 3G access as well, most devices were able to fall back on Verizon’s 3G EV-DO network. EV-DO doesn’t pack the punch of LTE, but is enough to support most of what you would want to do on a smartphone. Verizon issued this statement on Thursday morning:
“Verizon Wireless 4GLTE service returned to normal Wednesday evening after the company’s network operations team resolved a technical issue. Some 4G customers had reported intermittent or unavailable 4G service, or devices operating on 3G, for periods of time starting late Tuesday. Throughout this time, all customers were able to make voice calls, send and receive text messages, and 3G data devices operated normally.”
In April’s outage, a software glitch originating deep within Verizon’s network core started knocking down elements of Verizon’s new LTE service delivery architecture, taking down the servers that identified individual customers as they went through the network. That kicked all of Verizon’s 4G customers off both of Verizon’s data networks. CDMA-only customers weren’t affected because Verizon manages them on its older CDMA service delivery systems. This time, the problem doesn’t appear to have originated in the core, rather in the radio network itself – otherwise 3G data would have been knocked out as well. Or Verizon may have learned its lesson from that last outage and figured out a way to shunt 4G customers over to the 3G core, where their devices would behave like ordinary CDMA phones.
When Verizon built its new LTE network, it didn’t just mount new fatter radios on its towers; it changed the fundamental design of every aspect of its network: moving from hardware to software driven base stations, evolving its network service delivery systems from ancient hierarchical voice-centric chains of gateways to new flat IP architectures, and replacing ancient copper backhaul links with fiber Ethernet to the tower. It was also the first operator to do such a complete overhaul on such a huge scale. For many of the vendors in its network, Verizon’s early 4G launch marked the first time their new LTE equipment was deployed commercially. I’m really surprised more bugs haven’t reared their hideous heads in the year the LTE network has been online.
Still, Verizon’s customers probably won’t cut the carrier too much slack. Verizon, after all, bills itself as having the country’s “most reliable network.” In April the repercussions from the nationwide outage weren’t too terrible, probably because Verizon only had a few hundred thousand LTE customers at that point. Today it has several million. What happens when an outage occurs and it has tens of millions of 4G customers?
View full post on GigaOM
