Would You Like to Schedule Your Next Disaster? One Expert Suggests It.

This week on InfoWorld’s blog, The Deep End, author Paul Venezia takes a look at the benefits of scheduling downtime in data centers to better prepare for and recover from disasters.

“Unprecedented Arctic blasts cause all manner of mayhem, especially in areas where cold and snow are generally mythical. And mayhem means hard-learned lessons,” Venezia writes.

“But that's a problem caused by Mother Nature. You can't prevent it, nor can it be even realistically forecasted. Man-made problems, however, can be upsetting on another level altogether.”

Venezia details the issues that arise with man-made problems.

He references the enforced power shutdowns companies sometimes implement. Though they can be difficult and annoying, especially if they last up to four hours, the fact that these shutdowns are scheduled makes them a little easier to handle than unexpected natural disasters. Venezia suggests thinking of it as “disaster recovery on your terms”.

“Having the entire room go dark will set a feast for gremlins, as storage arrays spin down for the first time ever and cooling systems that have been running nonstop for years go silent. Objects in motion tend to stay in motion, indeed. When those systems fire back up, it's a virtual guarantee that something will fail,” writes Venezia.

So how do you make the best of a shutdown situation? Here is Venezia’s advice:

  • Identify everything that can reasonably be powered down
  • Reduce the data center to the leanest it can be without going completely offline
  • Leave storage arrays running
  • Shut down as many physical servers as possible

“Every watt you can remove from the UPS load will give you more time on the clock, and that's the goal,” Venezia explains.

In the event of a true emergency, however, when the UPS has reached its limit, a data center will need a rapid power-down – but it needs to be done the right way.

Venezia says this emergency plan should be scripted as much as possible beforehand so that if something drastic happens, staff can act fast and accurately.

“Most people don't really think about why data centers need to run 24/7/365. Even when they're largely dormant, actually powering down these otherwise critical systems only leads to mayhem,” Venezia writes.

He explains this is one of the key benefits of virtualization. There is the ability to power down servers during low-use periods. However, if it can be avoided, people generally never choose to power servers all the way down.

But there are those dreaded incidents where it cannot be avoided.

“When we can't (avoid it), we take a deep breath and head into the breach, hoping for the best, yet planning for the worst,” writes Venezia.


For more information, visit Paul Venezia’s blog post here: