Virtually Avoiding Disaster

Order a reprint of this story
Close (X)

ORDER A REPRINT

To reprint an article or any part of an article from Hospitality Upgrade please email geneva@hospitalityupgrade.com. Fee is $250 per reprint. One-time reprint. Fee may be waived under certain circumstances.

SEND EMAIL

June 01, 2010
Virtualization
Lyle Worthington - lyle.worthington@hsbresort.com

View Magazine Version of This Article

When disaster strikes, will your IT department be ready?  Are you prepared for a worst-case-scenario in your data center?

Business Continuity Planning
Developing a business continuity plan is critical for ensuring that, should a disaster occur, your business can return to normal quickly with the least amount of interruption and data loss as possible.  Business continuity planning should cover much more than just your servers, software and PCs, as a recovery from any sort of disaster would most certainly affect the entire organization.  For the purposes of this article, however, I will be referring only to the backup and recovery of your servers and data.

A good business continuity plan (BCP) starts with a risk assessment.  How much do you stand to lose (financially, operationally via productivity losses, long term via drop in customer perception, etc.) every second that a server or application is unavailable?  Where are all of the points of failure in your server room that could cause a significant outage?  Your goal is to design a solution that mitigates these risks while keeping the cost of the solution low enough to justify the expense, based on probability that the disaster will occur. 

Not every server or application will need to be restored immediately or with zero data loss.  You are planning for recovery from a complete disaster, so you should think about this as not just a typical backup and restore plan.  Some applications can stay down, some can run on slower hardware with fewer people accessing them, and some will need to be up immediately on comparable hardware with no data loss.  These are all factors to consider when you do your risk assessment and write your BCP.

At one extreme you could have an exact mirror of your data center in another location (or several locations).  The hardware might be different, but there would be enough of it to bring your entire environment online immediately with a minimally noticeable performance drop.  You might even be load balancing between the data centers. This is the most expensive way to do it, but provides the most redundant system.  At the opposite extreme, you might just back everything up to tape and put the tapes in a lock box off property.  In this case, if your server room explodes, you would have to procure all new equipment, get your backup tapes and slowly rebuild everything– losing any data that had changed since your last backup.  Downtime could be days or even weeks.
 
Where you fit between these two extremes is probably determined mostly by cost, but might also be impacted by the level of expertise in your IT department or a lack of IT understanding by your management group.  Whatever the reason, you may be tolerating more downtime and data loss than necessary.  Server virtualization allows you to extend the financial and operational savings from your production environment into your backup and disaster recovery environment.  This will open the door to a more sophisticated BCP at a much better price point.

Virtual Continuity
Before you can recover from a disaster, you must have backed up data to restore and backup servers to restore to.  Building an extensive backup and disaster recovery plan in a virtual environment is considerably less expensive than in a dedicated server environment for three main reasons.

Reduced Physical Hardware
You need less physical hardware in your production environment; consequently, you will need less in your backup environment. With the ability to restore entire machines onto completely different hardware without having to worry about hardware compatibility, you can also spend less on your backup servers.  You can move old or retired servers to your backup environment instead of buying duplicate servers.  Your only concern is that you have a fairly modern processor that supports virtualization.

Operational Efficiency
Full server restores are considerably faster with virtual servers than physical ones.  By creating backups that are essentially full virtual machines, restoring from backup can be as simple as adding the backed up server to inventory and powering it on. 

One of the more important requirements of your BCP should be verification and testing of your plan on a regular basis.  With virtualization, you are able to test your disaster recovery procedures quickly without disruption to your production environment.  By speeding up this process, you are further reducing your operational overhead. 

On the high end, by purchasing replication technology for your SAN and recovery software for your hypervisor, you can automate the entire disaster recovery process.  Your production environment will be mirrored and monitored by software, bringing you back up instantly in the event of a disaster.  This limits human error during the restore process and frees up your IT team to focus on restoring your production environment.

Licensing Savings
You can avoid having to buy a full server backup license for every virtual server as snapshots and clones are provided by your hypervisor for free.  If you do choose to purchase backup software, it is generally licensed by physical server – which would be running several virtual servers.  With either of these options, you will have the ability to very quickly bring your server online in a backup environment without having to worry about re-installation or driver issues.  Depending on how many virtual servers you have and the density you achieve, this could mean considerable savings.

BCP on a Budget for SMBs
If you can tolerate a little downtime and have no money to spend, you can still build a functioning disaster recovery environment.  You can use a combination of exported snapshots, clones and scripted data copies to back up your servers in a way that provides for a quick recovery.  Since clones and snapshots are essentially full virtual servers you can power them up immediately on your backup server.  The only thing left to restore is the most recent data backup.  This is all done without having to purchase any expensive backup hardware or software.

If you do have a little money to spend, you can build a high end solution at a fraction of the cost of a traditional non-virtual environment.  You will need to use a disk-based solution for your primary backups.  Purchase or build a backup server that has enough disk space to run every server you would like to be up immediately after failure, plus the additional amount needed to keep your standard backups.  You don’t need a lot of horsepower in this server nor do you need really fast hard drives, but you do need a good amount of RAM and a decent RAID card so you can keep your backup data safe.  Set this server up as a file server and add it in as a storage repository to your production virtual environment.  You can now clone machines directly to this backup server and even power up backed up virtual machines from your production servers.  Another advantage of building a backup device like this is that you can run your encryption, compression and archiving processes in your backup environment, freeing up resources on your production servers. 

This one device solves two major problems for you: 1) it gives you a repository for your backups, and 2) it can function as a backup for your primary SAN.  For the virtual servers you’ve decided will need to be back up immediately, leave the most recent full server backup restored, uncompressed, and ready to power on.  Install your hypervisor on at least one backup server, and add this backup file server as a storage repository.  You can now power on your backed up virtual machines for testing or recovery in seconds. 

Just remember that even though this is your backup, you still need to protect your data.  Secure your backup servers, use RAID and keep the systems patched.  Consider a second backup device, such as encrypted external hard drives, that you can take off site.  This is especially important if your backup servers are in the same general area of your production servers.

Business continuity and disaster recovery planning are insurance policies for your organization, and it may be difficult to justify the expense if you’ve never experienced a major disaster before.  Virtualization should help open the doors by reducing your costs and simplifying the backup and recovery process.  Regardless of which solution you choose to implement, I urge you to at least do something.  Remember, graveyards are filled with people who ignored the worst case scenario.

Lyle Worthington is the CIO of Horseshoe Bay Resort. He can be contacted at lyle.worthington@hsbresort.com.
 

©2010 Hospitality Upgrade
This work may not be reprinted, redistributed or repurposed without written consent.
For permission requests, call 678.802.5302 or email info@hospitalityupgrade.com.

 

Some Additional Tips/Tricks

If you have multiple physical servers, strongly consider some form of shared storage, such as a SAN, for your data.  This will allow you to move virtual machines between physical servers without having to copy large sets of data.  Pay attention to the risks of using a SAN, though, as having all your data on one device exposes you considerably should that device fail. 

Use VSS with your Windows servers that support it.  You can then use free utilities like DriveImage XML or Robocopy to copy data that would normally be locked. 

For servers with large sets of data such as file servers or databases, create at least two virtual drives, each as separate files. Install your operating system and applications that do not change often on one drive, then put your more frequently changing applications and databases on the other. This will allow you to reduce the amount of data you must back up with each cycle.  If you are able to put your virtual drives on different disk arrays you can reduce the I/O impact on one while you are backing up the other.  Your less frequently changing drive and virtual server settings can be backed up only when significant changes are made, while your data drive is backed up  aggressively.  This will free up space to keep more extensive backups of the important data, while still providing you the ability to do a full server restore. 

Take advantage of differential or incremental backups of your data to conserve even more space. 

If you have a backup room, use it!  Remember, this is your disaster recovery environment so it is important for it to be far enough away from your production server room that a disaster striking one will not affect the other.



want to read more articles like this?

want to read more articles like this?

Sign up to receive our twice-a-month Watercooler and Siegel Sez Newsletters and never miss another article or news story.