[IP] Phone System's Weak Link: reliance on commercial power
Storms Cause Greater Outages
In New Fiber-Optic Networks
As BellSouth Races to Recover
By PETER GRANT
Staff Reporter of THE WALL STREET JOURNAL
September 17, 2004; Page B1
CHARLOTTE, N.C. -- Ron Royster stepped out yesterday for only a few
minutes from BellSouth Corp.'s command center here, where technicians
closely monitor the havoc Hurricane Ivan is wreaking on its phone
network. But as soon as Mr. Royster, the center's director, returned,
he could tell conditions had deteriorated.
It was midmorning. Until then, Ivan's impact had been manageable. But
an alarm in the Network Reliability Center indicated a fire may have
erupted at a central phone office in Marion, Ala. Another office
reported that its backup electrical generator had stopped working,
forcing it to go on batteries. Most worrisome, the storm had severed
fiber-optic lines -- the lifeline of modern communications networks --
in Pensacola, Fla., and on the Alabama-Mississippi state line.
"Things went downhill pretty fast," said Mr. Royster, who hasn't had a
day off since Aug. 29 as the Southeast has been pounded by three
back-to-back hurricanes.
It was round one in Ivan vs. BellSouth, which provides local phone
service to tens of millions of people in nine Southeastern states --
all in Ivan's path. And it isn't exactly a fair fight. BellSouth's
disaster-recovery resources already have been stretched thin by the two
hurricanes that hit Florida, Charley and Frances.
Furthermore, because of fiber optics and other new technologies
BellSouth has added to upgrade its networks over the past decade, the
systems are, ironically, more vulnerable to disasters. The key problem:
Many phone networks that used to rely on their own electricity now
depend partly on commercial power. That means that when the utility
company's power lines go down, the phones may go down, too.
Like other regional Bell companies, BellSouth prides itself on its
ability to maintain service during disasters and restore it quickly
when it fails. Phone executives love to tell stories of the numerous
times that people have been able to use their telephones during power
failures.
These stories are about more than bragging rights. In today's fiercely
competitive telecommunications market, dealing with adversity is a
major selling point. Phone companies have been quick to point out that
many of the new phone services offered by some cable companies and
others using Internet technology often go out in power outages. Now,
with their new local fiber networks, phone companies are more
vulnerable on this score.
To improve reliability, BellSouth in 1995 consolidated 42 recovery
facilities into two, one in Charlotte and the other in Nashville. The
recent spate of hurricanes has put those centers through their toughest
test yet.
"It's been pretty intense," Mr. Royster says. "First Charlie, then
Frances and now this."
Phone companies had unparalleled reliability records when their
networks consisted of copper wires that stretched from central offices
to homes and workplaces with nothing in between. The copper lines not
only carry calls but low-voltage electrical power as well. That is why
phones generally have continued to work even when the local power
company's lines went out. In addition, the central offices have their
own backup power supplies that kick in during power outages. As a
result, most service
failures have been due to lines that were downed by fallen trees or car
accidents.
But in the past decade, phone companies added fiber optics and devices
known as "digital loop carriers," small file-cabinet-sized pieces of
equipment, between central offices and homes. The devices have greatly
boosted the capacity of the lines, cutting costs and making new
services including high-speed Internet connections possible.
The digital-loop devices also run on electricity from the local power
company's network, however. While they are equipped with batteries,
that backup lasts only about eight hours -- and less if there's a lot
of Internet traffic over the network. Once the batteries run out, phone
and Internet service goes dead unless a backup generator can be
installed. While some digital-loop carriers have a generator, BellSouth
says it would be economically unfeasible to put a generator at all
65,000 of them.
BellSouth officials say they are exploring ways to make the batteries
last longer. "We constantly look for technological advances that would
allow us to have a better safety margin," says Richard Burns, the vice
president responsible for BellSouth's network-operations support. "But
it's only in these huge, widespread natural disasters that commercial
power failures last long enough, and are widespread enough, to have a
significant impact."
Initially, Hurricane Frances knocked out phone service for 775,000
BellSouth subscribers -- 13% of its Florida customer base. Most of that
was due to power losses, not from downed lines. By contrast, only
350,000 customers lost service at the height of Hurricane Andrew in
1992, before many of the BellSouth networks were upgraded. A large part
of the difference is because Andrew hit only two counties, while
Frances hit all of BellSouth's Florida operations. But Frances's larger
impact also was
due to the system's new susceptibility to power outages, BellSouth
executives say.
Wireless phone service was also affected. While cellphone towers are
built to withstand hurricane-force winds -- and most did -- they use
power to transmit their signals. And in areas where the power went out,
the towers' backup power, a combination of generators and batteries,
also sometimes failed. In Mobile, Ala., 38% of Cingular Wireless's
coverage area lost service, in Biloxi, Miss., 32% lost service, while
in Birmingham, Ala., about 23% lost service, says Calie Shackleford, a
Cingular
spokeswoman. Cingular is owned by BellSouth and SBC Communications Corp.
Power was very much on the minds of the 50 or so technicians who manned
the BellSouth disaster recovery center in Charlotte yesterday. They
watched as the number of central offices on backup generators climbed
to 118 at midafternoon, from 63 at about noon. Even more worrisome were
those that were relying on batteries, meaning their generators had
failed and they had about eight hours of juice left. If one died,
thousands of homes and offices would lose their phone service.
Fortunately, all the central offices that switched to batteries did so
for only a few minutes while generators were refueled. During one tense
moment, Wayne Bell, a technician, stared at a screen that indicated
that one generator didn't start after being refueled. That only lasted
for a moment, though.
"They got it back," he said, clasping his hands in a prayer motion over
his head and looking up in thanks.
But other problems erupted. On other screens technicians monitored the
number of digital-loop carriers that were on batteries and the number
that had failed, meaning their batteries had died. By midafternoon, 397
had failed and 1,193 were running on batteries. Each one conducts
service to as many as 500 households. The ones that fail won't be fixed
until the commercial power goes back on or crews attach backup
generators.
As Ivan moved north, BellSouth officials moved forward quickly with
their recovery plan for areas hit first. A fleet of trucks with 600
emergency generators that had been parked at a staging area south of
Valdosta, Ga., began to move out. By noon, 11 tankers with diesel and
unleaded gas -- to fuel the generators and repair trucks -- were
rolling from Jasper, Fla., to three sites in Alabama. By midafternoon,
114 digital-loop carriers were working on generators.
As of late afternoon yesterday, 350,600 customers had lost their phone
service due to Hurricane Ivan -- most of that due to the loss of local
power.
BellSouth executives say that the recovery process still under way in
Florida didn't have a major impact on the company's ability to move
resources into Ivan's disaster zone. All of the company's 1,100
generators were used to restore phone service knocked out by Frances.
But by the time Ivan hit, hundreds of them had been freed up because
commercial power had been restored. As of yesterday, BellSouth had
reactivated service for all but 50,900 of the customers whose phones
went out during Frances.
Write to Peter Grant at peter.grant@xxxxxxx
<http://online.wsj.com/article/0,,SB109537424767920279,00.html>
-------------------------------------
You are subscribed as roessler@xxxxxxxxxxxxxxxxxx
To manage your subscription, go to
http://v2.listbox.com/member/?listname=ip
Archives at: http://www.interesting-people.org/archives/interesting-people/