Hancock Bank Its headquarters a complete loss in Katrina, the bank built a rugged data center farther from the coast.

IN RETROSPECT, putting a data center on the fifth floor of a glass high-rise office building just a half-mile from the Gulf of Mexico wasn't such a good idea. During Hurricane Katrina, Hancock Bank's Gulfport, Miss., headquarters building was devastated.

Today, the most visible change to the bank's disaster preparedness plan is a new $16 million data center farther inland, but that's just one of many changes Chief Operating Officer Shane Loper says will pay off when the next disaster strikes.

The bank now operates on a "4/24" plan that requires customer-facing systems to be operational within four hours of a disaster, and core business systems within 24 hours. "All of the things we are doing come with a price," Loper says. But because the bank is regional, it needs to ensure that a local disaster in Gulfport won't affect its other branches.

"You want to locate your data center in an area that has the lowest threat profile possible, and that means separating it from the headquarters offices and away from downtown areas when possible," says Stephanie Balaouras, an analyst at Forrester Research Inc.

Hancock's new data center is still in Gulfport, but the hardened, lights-out facility is farther inland on the area's highest point. It can withstand 200 mph winds, can be managed remotely and has dual 820-kilowatt generators with enough fuel to stay up and running 24 hours a day for a month

. The old data center's server infrastructure had been mostly consolidated using VMware virtual machine technology when Katrina struck. "Using more advanced techniques such as server virtualization to enable high availability and disaster recovery are good best practices," says Balaouras. The virtual server files and associated data were backed up and could be quickly set up on hardware in a backup data center in Chicago. But the challenge was getting them to Chicago and loading them from tape.

"[Just] the tape-restoration process required 16 hours," and 36 hours elapsed before all systems were up and running, says Jeff Andrews, vice president and manager of information security.

The new system replicates virtual server files and data over an MPLS network, reducing the boot recovery process to about 45 minutes.

Disaster drills have also changed. "When I was driving up to Chicago [after Katrina], I was scared to death, because I knew there were things we never tested," says Andrews. Today, everything is tested under a full production load.

On the front end, many employees have remote access and use PCs to access hosted virtual desktops.

But technical issues aren't what worry Ron Milliett, director of IT services. "Communication ... was the single biggest problem we had," he says.

The bank did have a few phones that used Sprint Nextel Corp.'s iDen push-to-talk technology, and they did work, so the number of those devices was upped to 250 after the storm. But the bank plans to eventually replace them with phones that use mobile broadband technology. The bank also signed on with the U.S. Department of Homeland Security's National Communications System. NCS programs allow companies providing critical infrastructures to get priority on land-line and wireless phone calls during a disaster and on getting critical lines fixed.

Hancock Bank also keeps a database of employee contact information and has deployed an alert system that can push messages out to phones and e-mail accounts. Alternatively, employees can call a toll-free number to hear updates during a disaster. But the best technologies and procedures won't work if staffers lack clothing, food and shelter, says Loper. "We had leadership, but they were running the business," he says. Meanwhile, many employees were arriving at work with only the clothes on their backs.

So the bank identified an emergency logistical coordinator at every location. "You want the company taking care of the basic needs so that your associates are focused on getting the business back up and running," Loper says. "We are many levels above where we were," Loper says, but he also expects that when the next disaster strikes, the bank will still face problems no one had anticipated. "Once the core things get resolved, there are thousands of fires that have to be stamped out every day," he says. But, Loper adds, he'd much rather be stamping out the little fires than fighting the big ones.