|
by Rick Rancher and Randy Coleman
What began as a
statement from our CEO
at an investors meeting in
2000 resulted in a project
that ultimately saved our
company substantial
money and helped position
the Chemicals Group IT
operations to be spun off
a few years later.
At that meeting, Sir John Brown stated
that all business units at BP either had
a Business Continuity Plan (BCP), or
they would have one by the end of the
year. While we had hurricane preparedness
plans at some sites, and a rock solid
IT Disruption Recovery Plan, few of us
had a true BCP and we did not know
what it should look like.
As Director of Operations for the
Americas and Asia, we had tested our
IT DR plan regularly and I had been
waiting for the “Big One” to show our
stuff. But now it was time to venture
out to the business sites and have serious
conversations with them to prepare
our BCP. We had 43 sites in over 30
countries, so this was no easy task.
Chemical plants are not usually situated
in the best part of town. “When
your ride from the airport includes
three armed bodyguards in a bullet
proof Suburban, you have the sense this
isn’t going to be your typical Business
Impact Analysis,” said one of the consultants
working with us!
With the difficulty reaching some of
these locations, we wanted to gather information as quickly as possible. The
project was easier because we had just
completed the Y2K initiative, which
generated mountains of information
for us to review. During this process,
we were able to determine the Recovery
Time Objectives (RTO) and the
Recovery Time Capability (RTC). We
also discovered that a majority of what
the business needed to operate did not
reside in our data centers. It was in
spreadsheets on someone’s desktop or in
a pricing system that ran on a server in
the chemical plant that was written by
some teenage guru with no supporting
documentation.
Rent vs. Own
BP was a company that tended to
outsource certain operations. This philosophy
would ultimately play a key
role in our ability to get an ROI
from our BCP. Upon completion of the
business impact analysis process, we
determined we would need 20 servers to
recover and provide business continuity
for our largest chemical plant. Those
same 20 servers could also be used to
recover four to five smaller facilities.
We put together a cost/benefit analysis
of our options.
Keep in mind we were a company
that loved to outsource. What we
discovered was that in many cases
these outsourced projects were acquiring
equipment for proof of concept,
R&D and testing. Once the project was
accepted into production, the equipment
– perfectly good machines – used
in the earlier stage was being sold for
pennies on the dollar.
We decided if we were able to build
our own recovery site and capture the
equipment being sent to redemption,
we could save substantial money and
get an ROI within the first twelve
months. But we needed at least 20 servers
to support the BCP of our largest
chemical plant by the end of the year,
and we couldn’t wait for throwaways
to fill our soon-to-be-built recovery
site. And so we re-wrote the SLAs of
existing projects to house the proof of
concept, R&D and test servers in our
recovery site. This provided the various
projects with unlimited access to these
servers, with the understanding that in
the event of a declaration by one of the
chemical plants, we would take over
the machines since the disaster recovery
process would be a higher priority
than the R&D or testing projects. So
before you know it, we had more than
20 servers available to us and we started
looking for a suitable location to build
our recovery site.
If you build it they will come
By bringing our recovery in-house, we
also wanted to be able to “quick ship”
equipment to our sites if necessary. This
became an important consideration
in our final choice and one that has
paid big dividends over the years. We
ultimately selected a site that already
housed a data center for another company
so the necessary infrastructure was
already in place. In addition, we wanted
it big enough to move equipment
from our alternate site vendor that we
owned. As part of our analysis of alternatives,
we discovered that we could
save over $500,000 per year with this
move alone. Soon after we opened and
equipped our recovery site, the Refinery
Group got wind of the progress we had
made. How did you do it? Can we use
your recovery site? All of a sudden my
budget was starting to look really good!
The Refinery Group provided 12 more
servers for the recovery site to handle
their requirements and they picked up a
third of the operating/build cost.
By thinking outside the box, we saved
substantial money and truly got an ROI
from our BCP. In summary, we moved
from very expensive real estate at our
alternate site vendor to a very cost-effective
site that we owned and controlled.
We were able to equip it for pennies on
the dollar and it put us in a position to
quick ship to any of our North American
facilities within 12 hours.
The hidden value of our BCP
Having a tested, workable BCP with a
recovery site that you own and control is
a tremendous value, but it wasn’t until
a few years later that I fully realized just
how valuable it was. In our business,
incidents happen all the time. Having
a timely response to mitigate the disruption
is simply prudent. However, in
2004 we were told that the company
was planning to spin off the Chemicals
Group and take it public as a separate
entity. We were tasked with identifying
and dissecting all the corporate
systems that our 12,000+ employees
were utilizing.
This would have been a monumental
task if it were not for the work we had
done on the BCP. Everything you need to
recover your business is the same as what
you need to operate it. We already had
the answers. Because we had the BCP, we
were able to accomplish the separation
from corporate in about 6 months.
We never made it to Wall Street.
Prior to going public another chemical
company offered almost $10 billion for
BP Chemicals. But without the information
and strategies we put together
while creating our BCP, we would
probably still be trying to figure out
how to become a separate entity. And I
wouldn’t be retired!
About the Authors
Rick Rancher has 35 years experience in IT.
He was Director of Operations, Americas and
Asia when he retired from BP after 25 years
of service. He now enjoys time with his family
and his advisory role to Monarch Business
Resiliency (MBR). He can be reached at rick.
rancher@hotmail.com. Industry veteran Randy
Coleman has over 25 years experience in disaster
recovery and business continuity planning.
He is presently VP, Sales and Marketing for
MBR in Atlanta, GA and can be reached at
randy.coleman@monarchresiliency.com or
www.monarchresiliency.com
|
