What happened at OVH?
OVH is the largest hosting provider in Europe and the third-largest in the world. The cloud computing company provides virtual private servers, dedicated servers, and other web services.
On Wednesday, March 10, 2021, a fire destroyed OVH’s SBG2 data center in Strasbourg and four rooms of data center SBG1 and caused a shutdown of the servers in SBG3 and SBG4. It took 100 firefighters and 43 fire trucks to extinguish the fire in the five-story building. No one was injured. However, the fire took down around 3.6 million websites across 464,000 distinct domains. The websites that went offline included online banks, webmail services, news sites, online shops, and several government websites.
OVH founder and chairman Octave Klaba kept updating the world about the incident with a thread on his Twitter account. As the fire blazed, he immediately advised in his first Tweet all the datacenter’s customers to enact their disaster recovery plans.
We have a major incident on SBG2. The fire declared in the building. Firefighters were immediately on the scene but could not control the fire in SBG2. The whole site has been isolated which impacts all services in SGB1-4. We recommend to activate your Disaster Recovery Plan. — Octave Klaba (@olesovhcom) March 10, 2021
We have a major incident on SBG2. The fire declared in the building. Firefighters were immediately on the scene but could not control the fire in SBG2. The whole site has been isolated which impacts all services in SGB1-4. We recommend to activate your Disaster Recovery Plan.
Who were the most affected by the fire?
Many OVHcloud customers lost data permanently, while many other websites are still offline until this date. The ones most seriously affected were those who ran their own dedicated physical machine servers at the OVH data center, instead of virtual servers in a cloud. They retained more control over data and infrastructure and inherently higher security. However, they did not benefit, in this particular incident, from the automatic backups of OVHcloud virtual machines in its cloud, and they needed to be highly prepared to activate their disaster plans.
Unfortunately, according to comments on Twitter, the number of affected customers without DR plans was substantial. In a nutshell, IT teams or IT partners who took DR lightly and didn’t implement tight backups and disaster recovery plans have failed their customers and caused them substantial losses.
Technical experts in the field are flooded with messages of 100 percent reliability. Still, experience and knowledge proved that risks are always there and “what could go wrong, would go wrong.”
On their website, OVH dedicated an entire section for disaster recovery where they stated: “According to technology research and advisory firm Gartner, network downtime can cost an enterprise more than $300,000 per hour. How much data would you lose? How fast will your communications and customer-facing capabilities be restored?”
What lessons to learn from this disaster?
Be prepared
Fabricated attacks, malicious intrusions, data theft, natural disasters, and force-major might occur any day. Be prepared.
Robust DRP
Setting up a robust disaster recovery plan with data backups is not an option but a necessity. They are your online, and in many cases offline, businesses only resource in the event of a natural disaster or cyber-attack. Your plan with detailed instructions on responding to unplanned incidents should include specialized skills, integrated strategy, and advanced technologies, including orchestration for data protection and recovery.
Test and Assess
Test your disaster recovery. Setting up the plan with goal identification, team assignments, emergency procedures, and so forth should be well thought of and communicated. However, like any contingency plan, it should be tested and evaluated regularly.
Trusted IT partner
Choosing a trusted IT partner with specialized skills and tools to set up, manage and maintain your IT infrastructure while focusing on protecting your business against IT failures and covering any risk with backups off-server and disaster recovery tight plans is vital.
How can ITWORKS ME help you set your Disaster Recovery Solutions?
We help you avoid that tweet: “Our platform is down. We will update you when it back up.”
At ITWORKS ME, we believe in designing business continuity solutions based on the highest standards available. We study all aspects of your operation and advise on the procedures that guarantee disaster recovery and the lowest downtime. We tailor your DRPs specifically to your business industry and environment. From virtualized disaster recovery plans, network disasters, cloud disasters to Data center disaster recovery plans, our DRPs can range in scope from basic to enterprise.
Contact ITWORKS ME sales team to know more about Disaster Recovery Solutions we offer.