Written by Kathleen Lucey, FBCI   

 

Blackswan

Hunting the Black Swans in Your Continuity Program


This is Vol. II, No. 10 in the DRG ongoing series regarding hunting and mastery of the black swans in your continuity program.

"Black Swans" in your Continuity Program are those events that remain outside the range of normal expectations, and may well produce a significant negative impact when they occur. For reasons of budget, culture, or simple lack of awareness, we just do not see or deal with these potentially devastating exposures in our enterprise continuity capability. This series discusses some of the most common of these "black swans" in business continuity programs, those that are really staring us in the face and screaming for attention.

Already published:

Volume I

Quarry 1: Employee Availability for Response Activities.
Quarry 2: The Level of Individual Employee Commitment to BCM
Quarry 3: Exercising Your Plans
Quarry 4: Exercising Your Plans: Objectives and Annual Programs
Quarry 5: Exercising Your Plans: Business Unit Continuity Plans
Quarry 6: Exercising Your Plans: Technology Recovery Plans
Quarry 7: Exercising Your Plans: Logistics, Communications, and Support Plans
Quarry 8: Lessons Learned
Quarry 9: New Year's Resolutions
Quarry 10: 10 Steps to Building a Black Swan-free Business Continuity Management Program
Quarry 11: New Year's Resolutions
Quarry 12: Developing "Black Swan Sighting" Skills: Warm-up Exercises

Volume II:

Quarry 1: The Centrality of Power: Seeing the Connections
Quarry 2: Power Outages: Isolation Effects
Quarry 3: Power Outages: How Employers Can Get Involved
Quarry 4: Cascading Effects on the Support Fabric
Quarry 5: Deeper Dives to Narrower Terrains: Dive 1
Quarry 6: Deeper Dives in Wider Terrains: Dive 1
Quarry 7: Cascading Black Swan Events
Quarry 8: Cascading Black Swan Events 2: Avian Flu Outbreak
Quarry 9: Black Swans in Our Midst: Debugging Your Response Preparations

Volume II: Quarry 10: Black Swans in Our Midst: Effective BSE (Black Swan Event) Management in Your Recovery Plans: Part I

So you have diligently run annual tests of your DR Plans and your BC Plans. You have even been updating your Business Impact Analyses annually. The auditors have accepted your exercise program results for years, and have checked that box as "passed". Your management is pleased with the audit reports. Should they be?

Let's consider for a moment what you may actually need to do to respond to an interruption event. After all, isn't this the actual objective that should be targeted by your testing program? In order to understand this, let's take a look at a diagram that outlines the type of teams that you will need to manage your recovery effectively during an interruption.

Incident-Management-layout

First, look at the bottom left of the diagram. These two areas -- Business Continuity Teams and Information Technology Recovery Teams -- are what most people think of as the components of their plan. There could be hundreds of individual plans here, one for every instance of each business function, one for every IT application, and many others for IT and Facilities infrastructure components. For example, where are you going to place that all-important Help Desk function? In either Information Technology or Business Continuity – that is not the important issue. What IS important is that a Help Desk Team exists and has a plan, and that its recovery progress be communicated through the assigned Recovery Coordination Team.

Moving up, again on the left side of the diagram, you will also see here a coordination function (oval shape) for each of these plan types. Depending on the complexity of your organization and the number of locationss affected by the event, you may need multiples of these coordination teams. Note that if there are multiple coordination teams, you will need ONE charged specifically with regrouping of all information and its transmission to the Business Continuity Coordination communications function for all of the teams that it represents. Such teams will also provide a single point of communications from the Business Continuity Coordination function (more on that later).
Examples of the type of information that might be communicated upward from working recovery teams to Coordination Teams are:

  • Arrival of which team members at the designated recovery site.
  • Time when the request for delivery of offsite backup media to the designated recovery site was made.
  • Inability to locate (may be injured, deceased, or missing, or simply impossible to reach) staff members responsible for execution of a specific plan. Names of those who have indicated that they can get to the recovery site. Names of those who have key recovery roles but who cannot participate in this recovery effort (physically impossible to get there due to transportation interruption, blocked roads, or because of personal responsibilities for injured or deceased family members, etc.).
  • Time of arrival of backup media from offsite storage as well as any hardcopy / portable media information ("Battle Boxes") at the designated recovery location.
  • List of team members who will require housing, food, or other support services for themselves and/or their families.
  • List of team members who will require transportation pickup at a pre-designated location.
  • Ongoing recovery status: issues, timing, progress, etc.
  • Any personnel status changes.

...and many others...

As your eye moves up the page on the left side, you will see a purple octagon. This is perhaps the most critical function within the entire diagram, because correct and efficient communications during the entire event are essential for the effective management of any recovery process. This is all the more important because it is frequently omitted entirely from the recovery organization. This function is the fulcrum of that effort; recovery time will be decreased and its accuracy supported by using this function effectively. Senior personnel, such as internal auditors whose work will remain interrupted for the duration of the event and others as appropriate and available, should be assigned as staff to this communications function (and rehearsed with Business Continuity staff when exercises are performed) to support the Business Continuity Manager and his/her support staff, who have primary responsibility for this function.

Other recovery functions may develop information for the various audiences that must be kept informed about the situation, including employees, business partners, customers, regulators, and executive management, general press, among others. The flow of information for everything relating to the recovery effort will go through this team. (Note that this team will also be receiving information from all of the teams listed on the right side of the diagram. More on these teams next month.) This information flow includes such items as the following: team deployment issues, support requirements, and issues, problems, solutions, progress reports, etc. The Business Continuity Coordination Team will be receiving information from all areas of the recovery organization. It is their responsibility to collate all of this information, make sense of it, and forward it to the appropriate Interruption Response Team for action or information.

It should be clear to you now that the purpose of regrouping this information within the Business Coordination function is to provide both upward and downward flow of information in a coherent manner, as well as a definitive set of specific support requirements to the Interruption Response Management functions that will be authorizing expenditures to support the recovery. Such authorizations will then flow down through Business Continuity Coordination to the Recovery Coordination Teams. Think of the Business Continuity Coordination function as a kind of intelligent central point for consolidation and coordination of information from many different sources.

This is not to say that inter-team ongoing coordination among related business units or technology areas should not be taking place as planned. In fact, there should be an "interlock" among the various plans. Such an interlock is defined as follows:

  • The plan for Team A should clearly document the type of information and its form that should be communicated to Team B.
  • The plan for Team B should document what information can be expected to be received from Team A, and the order in which multiple communications can be expected to be received.
  • This is what makes for the aforementioned "interlock", which includes both of the above requirements. Included within the Plan for Team B should also be information about contacting Team A (and any other teams from which information is expected) if the information has not been received.

Without a smooth and effective communications process, everyone outside of the immediate recovery teams may receive only sporadic and unofficial communications. Decision makers will not have a clear basis for making the appropriate decisions, and recovery teams may not receive the critical resources that they need to ensure recovery in the most expeditious manner.

During a recovery there is no time to make communications mistakes; it is a time when communications must be clear and complete, and flow effectively to those who need to use that information. Failure to recognize the need for such effective communications, and subsequent failure to implement and test the effectiveness of such procedures is a very potent black swan. It is a black swan because it has not been recognized as a required element and therefore has not been included and rehearsed in exercises in order to debug the process. And yet even a small amount of thought about how an interruption response would actually be conducted leads us directly to a deep understanding of the importance of smooth communications during an interruption.

Information from all inputs (more on the right-side teams next month) therefore comes into Business Continuity Coordination and is organized, clarified, and passed on to the appropriate teams within Interruption Response Management so that support and funding decisions, as well as communications decisions, can be made. Interruption Response Management teams then work through Business Continuity Coordination to disseminate decisions for support (transportation pick-up routes, hotel accommodations, cash disbursement, etc.) to the "boots on the ground" recovery team members. This channel will operate both up and down for the duration of the event, with communications decreasing in frequency with the passage of time. However, regular meetings will continue to be convened to report status until the affected entity returns to full "business as usual" status, either at a repaired location or at new premises configured to meet current requirements.

It is therefore absolutely necessary that this process be rehearsed in order to ensure that it works smoothly and completely, and in both directions. It is advisable to run exercises that test as many aspects of this communication process as possible, in both directions. Team members should be dealing with a fully elaborated event scenario that allows the identification of previously unknown challenges.

Rehearsal of these functions is at least as important (and probably more important) as refining technical recovery procedures. Ensuring that communications are conducted smoothly and effectively is probably the single most important function in the entire spectrum of recovery activities. Really. This black swan will bite very deeply if you do not ensure that your people are trained and your communications procedures are smooth and effective. Blunt the effect of that black swan's bite by verifying and fine-tuning your recovery communications processes.

 

About the Author:
Kathleen Lucey, FBCI, is President of Montague Risk Management, a business continuity consulting firm founded in 1996. She is a member of the BCI Global Membership Council, past member of the Board of the BCI, and the founding President of the BCI USA Chapter. IBM chose her as the first winner of its Business Continuity Practitioner of the Year Award in 1998. She speaks and publishes widely in both North America and Europe. Kathleen may be reached via email at kathleenalucey@gmail.com