Data loss happens as a result of a variety of events including accidental damage, deletion, or modification of data, hardware malfunction or failure, local disasters such as fire, flood, or a spilled drink on a computer, lost or stolen devices, modification, deletion, or corruption of files by virus or malicious code, or even ransomware attacks. According to one report, 70% of small firms go out of business within a year of a large data loss incident!
It is imperative to adequately back up your organization’s data and secure it. The last thing an organization wants is to lose time based on improper backup processes. Even worse, having to tell their customers their data has been lost.
Below are two real-world examples we have encountered from organizations that did not have adequate backup processes and had the unthinkable happen to their data. Both cases below involve organizations that received an annual IT audit in which their respective auditors communicated that there were deficiencies in their backup processes. Unfortunately, both organizations routinely pushed back on the observations, which resulted in catastrophic results.
The Case of the Safe Deposit Box
Organization A’s data backup process was intense and meticulously logged. First, their backup software wrote the backup data to disk, then copied that backup data to tape, next an application verified the data on the tape, and lastly the tape backups were then transported off-site to be stored in a safe deposit box at a local bank.
A computer operator would rotate tapes off-site and bring back tapes from a preceding backup daily while retaining 14 days of full backups off-site. The tapes were properly labeled and tracked, and as a result, they were easily able to demonstrate the care that was taken in virtually every aspect of their backup process.
their auditors observed that any testing the company performed was from the backups stored on disk, not the off-site tapes. The auditors noted that the company should simulate the “full process” they would go through in an actual worst-case situation which would include retrieving the tapes to restore the data, otherwise, there would be a risk of an unforeseen outcome. The organization rejected this finding saying that the verification process which was completed after the data was written to the tape substantiated a “test” of the backup data which had been committed to tape.
One day, a hardware issue resulted in a significant amount of production data lost including the backup data which had been written to disk. The organization retrieved the preceding night’s tapes from the bank vault and brought them back to the office to begin the restoration process. It was then they discovered that all of the preceding night’s tapes were blank. Bewildered, they then went back to the bank vault and brought back the tapes from two nights prior, only to find that those tapes were also blank.
Eventually, the cause of their blanked tapes was traced to the location of their safe deposit box within the bank vault. Their box was located next to the vault door, which utilized powerful electromagnets in its locking mechanisms. Their tapes were being effectively wiped clean on a nightly basis when the vault was secured, rendering them useless from a recovery perspective. This result could have been identified and rectified before the incident if only the organization had considered the auditor’s findings and properly tested their backup tapes.
The Case of Accessible off-Site Backups
Organization B’s data backup process had fewer “moving parts” than Organization A’s, however, was not without its own shortcomings. First, their backup software wrote the backup data to a local disk, then the backup data was copied to a Windows network location at a remote site the company controlled. Thorough testing of backup data, both local and remote, was routinely performed without issue.
their auditors observed that the remote location where backups were kept was logically accessible at all times from the production network via a simple Windows share. The audit finding was that their remotely stored backup data could be in jeopardy should the network become compromised or infected with malware. The organization rejected this finding claiming they did not have the time or resources to make substantial changes to their process. Since they perceived the risk to be minimal, they decided to simply accept it.
One day, an employee clicked on a malicious link in an email that infected the company’s network with ransomware. The malware quickly spread and encrypted all of the company’s production data. The organization’s technology department moved quickly to retrieve their backup data from their remote site only to find that data having also been encrypted.
The cybercriminal demanded a six-figure ransom to provide the organization with the key to decrypt their data. In the end, the company was forced to pay the ransom. As we discussed in a previous article, paying a ransom should be an absolute last resort and should not be counted upon to be a reliable solution for many reasons. This outcome could have been avoided had the organization not ignored the auditor’s findings, in addition to implementing structured training and phishing simulation tests for their employees.
Simply completing an audit is not sufficient, organizations must take into consideration the auditors’ findings and implement improvements, as necessary. Choosing to accept a particular risk must be a decision that is carefully evaluated, and organizations must ensure they understand the full ramifications of “what can go wrong”.
It is important to ensure that you thoroughly understand how your backup process works, identify where are the most likely points of failure, and comprehensively test the process to ensure that your organization’s backup data is going to be there when you need it.
While both examples we list above were caused by and resulted in two very different things, they share one commonality, neither organization carefully considered the auditor’s findings to implement the necessary improvements. In both cases, implementing appropriate solutions would likely not have required significant expenditures or massive changes to their existing processes. In addition to recommending regular audits, we also suggest having a ransomware response plan and conducting regular training and testing of your employees.
Our team performs all manners of other IT-centric internal audit projects to assist management in maintaining oversight of the IT function. If you are unsure which areas of technology your audits should be focused on, we can also assist management in performing a risk assessment to help define an IT audit plan and schedule.
No matter the type of project, our objective is to ensure that we provide IT audits that address the most critical risks to your organization. Visit our Technology Advisory Services page to learn more about all our services.
Contact Us
Calvetti Ferguson works with middle-market companies, private equity firms, and high-net-worth individuals nationwide. Regardless of the complexity of the compliance, assurance, advisory, or accounting need, our team is ready to help you. Please complete the form below, and we will follow up with you shortly.