It’s no secret that Hurricane Sandy has worked a number on communications infrastructure, but now a time-lapse video created by the University of Maryland (1) provides a clearer picture of when and where internet outages occurred as the storm made landfall. Green means the host is Up, red means the host is Down. After the hurricane track ends, the animation shows some hosts recovering from the storm[youtube video=”pyqE87MFdqw” width=”420″ height=”300″]
What Hurricane Sandy has ‘yet again’ taught end users around the world, is that an emergency plan is always a good idea.
Frontier had impacted client’s. And we are able to point to those who had or allowed us to build diversity and resilience into their design and those who did not. More important, we had clear access to other examples, other providers who, despite planning, failed to deliver resilience and redundancy and despite the expense of a redundant offering, did not deliver.
Here is an explanation.
Let’s start with the obvious, what our design would look like and what would happen when disaster hits.
In our scenario, we tend to push, more and more, converged voice and data across a single pipe. We also, promote the use of a second data connection as a backup or active ‘always on’ link to pick up the load ‘if’ the primary link is impacted (in anyway, including scheduled or emergency maintenance).
This is great, except in the case of New Jersey, where various parts of the city and industrial community were simply under water. In this case, the business or location itself was impacted as well.
In this case, it is not about redundancy planning, it is more about disaster recovery or pandemic planning.
In our scenario, the following would still apply.
- In the event of a primary link failure, the secondary would kick in
- In the event of a secondary failure the site would in fact be impacted, but site communications would still be in-tact because all communication is resolved, or exists in our ‘voice cloud’
- If a call is destined to the impacted location, and the primary and secondary path are UN-available, then the ‘voice cloud’ is smart enough, and intuitive enough to sense this. From there, it will follow a predefined path:
- Play the auto attendant, behave as business as usual, provide the options when you call the location (IE: Thank you for calling XYZ co, if you know the extension press it now, or Dial the following…..)
- If you dial the extension, we can predetermine or quickly program via a trouble ticket, call redirect of an extension to a cell phone, or other location. We could also re-direct the entire phone number (or DID) to another location – or in this example, all calls from New Jersey to ring to the Connecticut office.
- Once service is restored, it could automatically, or via manual intervention, revert back to the original rules.
- If a data centre is impacted, it would simply fail over or remain in our cloud which does not have any single point of dependency on a single server or location of a server.
Don’t all the providers offer this?
What Hurricane Sandy taught us users (again, unfortunately) is that existing communication providers are not the ideal partner to help you objectively prepare for this.
We have demonstrated examples of multi-day failures of not only the obvious ‘last mile connectivity’ (the actual link of fibre or copper into your building). This was because some of the carrier’s core communications networks themselves were impacted by the Hurricane and weather conditions as well. For example. Long Distance providers with a single Point of Presence (POP) or place where they keep their core information was in some cases not redundant. Data Centres that did not have adequate backed up power (UPS’) or arrangements for a continuous supply of diesel fuel. These outages were not limited only to the smaller providers. These were some of the ‘big’ guys. It was surprising to me. The common response was Force Majeure.
Force majeure…. what is that?
Definition of ‘Force Majeure’
A French term literally translated as “greater force”, this clause is included in contracts to remove liability for natural and unavoidable catastrophes that interrupt the expected course of events and restrict participants from fulfilling obligations.
My Interpretation of ‘Force Majeure’
A term used by lazy organizations who have contractually defined ‘an act outside of their control’ to mean practically anything including traffic conditions, lack of qualified people, labour distribution etc.
Frontier Network’s view is that pandemic planning and fail over should not be considered a product or something that you need to consciously purchase.
(1) The authors of this animation are Aaron Schulman, Ramakrishna Padmanabhan, and Neil Spring from the University of Maryland