OVH, one of the worlds largest data center, suffered a huge fire in one of its data centers in Strasbourg, France. Its SBG2 data center suffered total loss, while its other data centers in near proximity to it also suffered severe service disruptions. Though thankfully no one was injured in this fire.
This fire in total impacted about 3.5 million websites in the short term, and in many cases websites and companies had to resort to disaster recovery plans or face a total data-loss.
This fire incident cannot be seen in isolation or be treated as OVH’s own problem. OVH data centers and facilities are state of the art . OVH data centers have various ISO certifications, and also it is very difficult to doubt that they have been following various best practices especially related to fire safety and standards. This incident has to be seen in totality of the entire IT infrastructure industry because very few companies may be at the level where OVH is in terms of experience, technology and deployment of IT infrastructure services like bare metal servers, Cloud Solutions, Private racks etc.
This incident has lessons for every one of us – Data Centers, Web Hosters, Website Owners and the general Public. So now let me put below what should be our learnings from this fire incident.
Learnings for Data Centers
The biggest learning outcome from this incident for the data centers is that they should very critically review their fire safety standards and various other protocols related to it. It is very clear that either fire safety standards have to be improved, or the data centers themselves should go one step ahead in implementation of these standards.
The other major outcome for data centers is loss of face when such an incident happens. Though only a detailed investigation can pinpoint where exactly the fault was, but the reputation of the data center is already at stake. This does not mean that data centers of other companies or entities are better off. Rather clients of any data center, would now question them w.r.t. preparedness and mitigation of such kind of incidencies.
W.r.t. loss of data in this fire, though most customers of data centers understand that there are no guarantees w.r.t. safety and backups of data, but leaving aside legal issues, the clients of data centers hold them responsible at at-least operational and practical level. The loss of data also leaves a bad taste in customers’ mouths which very likely will impact their future relationship with the data-center.
Learnings for Web Hosters
Here we consider a web hoster any of the following – A Cloud Infrastructure Provider, Server Rack Providers/Colocators, Shared Hosting Providers and Reseller Hosting Provider. Basically any entity or an organisation which has a B2B dealing with the data center or with their own providers can be considered as a hoster.
The biggest issue for web hosters is that their clients are very unlikely to understand and appreciate the events leading to loss of data. They just want their website and apps to run without any disruption, and most are not even ready to pay a little extra so that their hoster can invest more into data backups and other such activities.
As web hosters, they have to accept the way their clients are, and any efforts by them to push clients to pay extra for data backups etc. would not lead to any better results. In a price sensitive segment, if the hoster were to charge extra, it may very well turn their clients to their competitors. Also hosters should carefully draft their Terms of Service (TOS) and Service Level Agreements (SLA) so that they are not held responsible for the loss of data.
The biggest lesson for a hoster is to be ready for a Cherobyl moment. God forbid if the data center where all their infrastructure is provisioned, were to catch fire or get in the grip of any natural or unnatural disaster, which leads to total loss of data, what strategy do they then have to come out from it? In worst case scenario it can lead to even closure of business. So therefore preparation is the key.
Learnings for Website Owners.
As a website owner your best and most important strategy is to always have backups of your data in your local PC or laptop. You should always have local backups irrespective whether your hosting company has committed to backups or not. Ultimately it is your website.
As most websites heavily rely on MySQL/MariaDB database engine, it is even more important to get the SQL files backup to your local PC as binary restores of data from backups can fail. If your website runs on WordPress, paid backup service is available by various plugin providers like Jetpack etc.
Here it is important with which kind of hoster you are dealing with. We here at Pack Web Hosting, consider client’s data extremely important, and though legally not required, but as a goodwill gesture try to keep up with backups.
Learnings for the General Public
Though common masses embrace digitization of data with open arms, but its very difficult for them to understand the complexities and technicalities behind it. If the data loss were to happen with a financial institution like a bank or with an insurer, we can very well understand the devastating consequences it can have for the masses.
Its not only financial data, but even the loss of free email service data or any such data, can lead to severe disruption of our day to day lives.
So what can a common man do. My sincere and honest advice is that especially w.r.t. financial data, people should have their passbooks regularly updated or their financial statements duly signed and attested by the bank staff. God forbid if ever things were come to it, but in case it does, you would be better prepared.
W.r.t. taking backups of free email service like gmail etc., this is very easy. Here you have two options
Option A) Configure a POP3 account in your local PC with the option of leave a copy of messages on the server enabled. Then download all your email to your local PC while at the same time your email remains on the server also.
Option B) Configure email forwarding to another free email provider. This works very well for all the future emails received by you. But your old emails will remain only at your old provider.
The fire incident at OVH should not be treated in isolation. It has lessons for each and every one of us. Now we should have a serious look into our data backup strategies so that we are well prepared for any such kind of future incidents and eventualities.