Friday, September 23, 2011

CrashPlan IOS & Android Mobile App Now Available

My favorite online backup provider, Crashplan, has released IOS and Android apps allowing access to your backed up files! Via the app you can download and easily keep up-to-date any files in your backup to local storage on your mobile device. One great use of this is syncing your Music collection from your backup versus some other added service. Check them out in your device's respective market/store.

Stumble Upon Toolbar

Thursday, September 22, 2011

Working with Veeam SureBackup

I began using Veeam Backup and Replication several months ago in the vSphere environment that I manage.  I can honestly say since switching to Veeam, backups have no longer been a pain in my posterior.  With Veeam, my backups just work - quickly and with low impact to the virtual environment.  So now that I have these great backups taking place what is next?

One of the great things about Veeam Backup and Replication is that is comes with a suite of great, unique features - some of the big reasons it ended up being my "backup" solution.  The focus of this article is the SureBackup feature that in essence is a live verification of your backups.  I'm not talking some data hash comparison between the source and backup data.  I'm talking about live booted systems from the backup data including application checks - all automated.

In a nutshell, SureBackup fires up the backed up image(s) of the virtual guests inside of a network bubble(Virtual Lab) on a designated ESX(i) host and then runs some checks against those VM's to ensure they are running properly indicating a successful backup - then optionally tears the whole bubble down when done.  As part of the configuration, you  decide which VM's are brought to life inside the bubble and checked, typically the list is based on dependencies of the systems being verifed.  For example - to check almost any Windows based server that is part of a Windows domain, you will need an Active Directory / DNS server included.

So a bit more about this network bubble.  In order to run the backed up VM's and not need to re-address them on the network, worry about name collisions, etc., Veeam uses the concept of a Virtual Lab.  A Virtual Lab is really composed of small linux proxy / router appliance and some network settings that are automatically deployed to a specific ESX(i) host.  Behind the scenes isolated portgroup(s) are created on the ESX(i) host which are interconnected by the linux proxy appliance.  It does take a basic grasp of networking to understand how to answer the network configuration for the proxy appliance - possibly the biggest hurdle to creating a Virtual Lab and SureBackup jobs.  Check with your friendly network administrator if you have any questions - it really isn't so difficult but if not done properly will cause you and the network headaches!  Once setup, the proxy creates isolated networks that are identical to your production networks so the VM's being run there have no idea anything is different.  The proxy keeps their traffic off of your production network and can provide NAT'd access from the production network to the isolated systems for testing / restore activities - very cool!

What can you do with SureBackup?  I've already mentioned the primary purpose of SureBackup - to validate your backups by actually bringing them online and testing the OS and applications automatically on a schedule. There is another very handy way to leverage the SureBackup feature that is more so implied by the name of the Virtual Lab configuration.  An option in a SureBackup job is to leave things running and not tear the job down automatically.  And what this really allows for is potentially automated rebuilding of a copy of production systems in an isolated environment for testing of patches, configurations, upgrades, etc.  And given the NAT features of the proxy appliance, on a properly configured network end users can easily have access to these test systems.  I'm sure there are other creative uses - please mention them in your comments!

During my time thus far learning and working with SureBackup, I've gathered the following tips:

  1. Be realistic about the IO load you put on your backup target storage.  The SureBackup VM's are running off this storage so align your space and IO requirements with the underlying storage accordingly.  Small numbers of large SATA drives, or a highly de-duplicated disk system may not be well suited to acceptably performing SureBackup jobs.
  2. Be cognizant of the load the SureBackup VM's will put on your ESX(i) host.  If you create large SureBackup jobs with many VM's, they require RAM and CPU like any normal VM.  A handy feature is the ability to, per VM, have Veeam automatically scale back the RAM allocated by some percentage - use it.
  3. Set Timers accordingly.  SureBackup has timer settings used to determine if a job is taking too long or the OS / application is not responding(i.e. a backup did not work!).  Depending on your storage performance, etc these may need adjusting - the defaults should be fairly sufficient.  I found I needed to slightly increase the Application Initialization timeout for VM's with apps that really hit up the disk on start-up (SQL, Exchange).
  4. Access to the backup files.  As it stands today a SureBackup job ran against a backup done with Reverse Increment will lock the files and prevent a subsequent backup from running successfully while the SureBackup Job is active.  For long running lab / testing situations the current suggestion is to copy the backup to another Veeam server, import it, and run SureBackup there against the copy.  If you use Forward Incremental backups check the Veeam forums for any concerns.
  5. Don't adjust the proxy appliance or isolated networks for the Virtual Lab from the vSphere client.  Doing so will create a logical disconnect from what the Veeam Server expects and what it finds.  It will break the Virtual Lab configuration.
As I continue to explore and work with SureBackup I'm sure more ideas will surface on ways to further leverage this great technology.  Please post a comment with your ideas!

Stumble Upon Toolbar

Friday, August 5, 2011

Blah Blah Cloud

If you are in the position of: answering management's questions about the cloud, making the actual decisions about using the cloud, or really anything about the "blah blah cloud" - you owe yourself a listen to a Packet Pushers Show#55. Lots of great questions to ask your (potential) cloud provider and other points to ponder. While often spiking a high level on the technical / nerd meter, this show is a great listen for IT folk, purchasing, and maybe even legal types.

Stumble Upon Toolbar

Friday, July 29, 2011

VMware vSphere 5.0 vRAM License Change

Gabrie van Zanten (@gabvirtualworld) has posted on his blog what I'd expect should be a positively received rumor about VMware updating vRAM entitlements for vSphere 5.0.

I've been a fan of the vRAM based entitlement because it is tied to what is often the most key scaling factor in a virtual environment - RAM. Even though it did not impact my environment, I felt the initial entitlements were much to low for modern hardware, applications, and operating systems. Furthermore as VMware touted increased scalability in 5.0, the initial entitlement calculations indicated one would pay heavily from the pocketbook to take advantage of those improvements.

VMware was listening and has responded more quickly than could have been expected! Given the constant improvements in hardware and decreasing costs, I am a bit curious how much of a tail chasing situation this could end up being.

Go read Gabe's article for the details.

Stumble Upon Toolbar

Thursday, July 28, 2011

Excel 2003 Slow to Open Network Files After File Validation Update Installed

We've mostly completed the cut-over to Windows 7 and Office 2010 where I work. That said there are a few PC's with more 'complicated' configurations we have delayed updating that still run Windows XP and Office 2003. So accordingly we have maintained our WSUS environment to keep Office 2003 updates coming through.

In our last round of monthly PC patching, the update addressing KB2501584 was approved for PCs to install. Shortly afterwards reports started coming in from the few XP/Office 2003 computers in service, that Excel was hanging up when trying to open Excel shortcuts pointing out to files on the network - hmm. The long and short of the story is this MS article explaining how the Office File Validation function slows down opening network files in Excel 2003.

Two of the article's provided solutions I found to be "corporate unfriendly": either copy the file to the local disk first, or upgrade to Office 2007 or 2010. The third option was split into two paths: A) to use a MS Fix IT installer (normal users cannot do this), or B) Create a registry entry to disable OFV in Excel(again most users cannot / are not capable of doing this). Option 3 fix regardless of how it was done created a HKCU registry key - so now we have a per user, per computer situation to remedy - no "PC-wide" fix option.

Thankfully we already push out a few user scope registry entries via a custom Active Directory Group Policy, so we had a familiar delivery mechanism available to us. After some updating to that custom .ADM file and some testing we finally had a corporate quality fix for the issue.

Why didn't we just uninstall the OFV update? Removing security features isn't really a path we prefer to take. Yes - we have effectively disabled it for Excel, but the other Office application are still protected and didn't suffer the same problem that Excel did. One of the many usability versus security trade offs.

There are lots of great resources out there for making your own ADM files for use in Group Policies. If you work with AD / Group Policies it is a great skill to have in you toolbox.

Stumble Upon Toolbar

Tuesday, July 19, 2011

VDI and the Microsoft VDA License

If you are working on or looking into VDI, you should ensure an appropriate level of attention is given to the licensing side of whichever solution you'll use. Along with the myriad of technical details to keep in mind, the licensing side of a VDI project is just as critical.

I am by no means an expert in Microsoft Licensing and you should ALWAYS consult with your preferred vendor to arrive at a warm fuzzy feeling when it comes to license compliance (is there such a thing with regards to licensing?). If you haven't guessed already this article is going to be Microsoft center, albeit you can have linux and other OS based virtual desktops.

To start off, VDI is the acronym for Virtual Desktop Infrastructure. In short this is the running of (usually multiple) virtual desktop operating systems on a physical system and accessing them remotely. There are troves of articles out on the net about the merits, challenges, and use cases for VDI.

The initial thought for licensing these virtual instances of a desktop OS is often simply to go buy a license for each one and be done. That is not the case when it comes to Microsoft Windows. Microsoft realized there is value in running their software this way and subsequently created new licensing for such use cases. This is where VDA comes into play.

Two methods exists, that I'm aware of, for properly licensing virtual instances of Windows desktops: VDA and SA. SA or Software Assurance comes into play for corporate systems that have their Desktop OS SA maintained. These systems can be used to access VDI instances. The other is the VDA license. The VDA license is a bit more flexible (and costly) than SA because it can apply to a non Microsoft client device as well as non-corporate devices (contractor, employee owned PC,etc). You can read lots more about Microsoft and Virtual Desktops on Microsoft's Website.

The big question that I've always had until recently around the VDA license is: is it just an "Access Right" license or does it also include the virtual desktop OS? In the Microsoft world "Access" licenses, or CALs as they are often called, are very common - so quite naturally I wondered if the annual subscription based VDA license was only a "CAL" or more so. So after receiving a not so clear response from my Microsoft vendor, I consulted Google for other's experiences. I ran across this VMware Community thread some time ago and subscribed to it with hopes a more definitive update would be posted. Just today a post came through from MoffattThomas, quoting a Microsoft representative stating(summarized): the VDA includes the OS key for the virtual desktop. So there you go - Virtual OS and access rights!

Stumble Upon Toolbar

Wednesday, June 29, 2011

VMware Hardware Assisted Virtualization with Windows Server 2003 x32

VMware recently published KB 2001372 regarding the use of Hardware-assisted virtualization offered by Intel VT-x and AMD-V with Windows Server 2003 SP2 32 bit. To summarize the article:
  • Pre SP2 it was more efficient to use binary translation (software).
  • Starting with SP2, using Intel VT-x or AMD-V could provide significant performance improvements, depending on workload of course.
  • A VM with "Guest OS" set to "Microsoft Windows Server 2003, ...", will use binary translation by default(the Automatic selection).
I wanted to do a few rudimentary tests to see if it might be worth changing on my Windows Server 2003 SP2 VMs. These are by no stretch incredibly technical tests - my intentions were to see if changing to hardware assisted virtualization helped, and hopefully I might appreciate similar results in more typical workloads. I performed two tests: The first was VM boot-up to Login screen, the seconds was using the Memory Performance Tester by Joey Dieckhans that can be downloaded from Eric Sloof's website. I ran two separate Windows 2003 SP2 VM's through the tests.

Starting with boot up times - I found a consistent 3% to 4% improvement with hardware-assisted virtualization turned on (instruction set and MMU). Most probably don't cycle their Windows servers through enough reboots to find value in this alone, but longer term, it will be interesting to record how normal workloads might benefit.

With the Memory Performance Tester tool there was near zero change in the "MB per Second" figure in either configuration. I also monitored the CPU load in the VM during the memory test and found the same - no difference.

I look forward to trying this setting in a few pilot VMs and comparing their performance to past trends.

Stumble Upon Toolbar

Sunday, May 29, 2011

Veeam Backup and Replication Prep Checklist

I recently implemented Veeam Backup and Replication in my employers vSphere environment. Previous to using Veeam, 'traditional' in-guest agent backups had been used. While agent / OS based backups aren't necessarily bad, they are not the most effective way to get things done in a virtual environment. To get the most benefit from a virtualized environment and realize some other truly beneficial 'side effects' a hypervisor level backup should be used.

The downsides to agent / OS based backup methods in a virtual environment are: they consume CPU cycles on each VM and consequently take away from the host on the whole, they are slow because they have to check every file, they can also cause access times to be reset on every file which may not be ideal, and in event of a full restore, there is likely some requirement to get a base OS up and running (time consuming). The list goes on but in summary guest OS agent based backups are not time effective or host resource efficient.

Now when working with backups at the hypervisor level, there are some prerequisites to have in place for backups to work properly. One of the more important (painful to correct) items is ensuring any data that will get backup up is on hypervisor backed storage. By this I mean the backup tool, because it operates at the hypervisor level of the stack, can not see storage that a guest may have direct access to via physical RDM, or in Guest iSCSI mapped luns. Resolving this issue (if possible) can be a good source of downtime for those who don't have sVMotion enabled licenses (like myself). There have been a handful of articles that show little to no performance trade-offs between vmdk backed storage and RDM or in guest mapped luns.

So during my Veeam Backup and Replication implementation I created this Guest VM checklist the ensure things when a bit more smoothly the first time through.
  • Ensure CBT is set to false on all guest vDisks - Veeam will fix automatically
  • Ensure Data to be backed up is on vmdk or virtual RDM backed storage
  • Ensure vDisks are not set as persistent - snapshots are key to the backup process
  • Put Page / Swap file on seperate vdisk and Exclude the vdisk from backup - optional but will save on backup / replicating of pagefiles
This isn't meant to be a advertisement for Veeam but in short it was the selected solution for the following reasons:
  • Veeam is the leader at innovating backup and disaster recovery features in the virtualization space. It is a product built from the ground up for virtualized environments and the feature additions over each major release are significant.
  • vPower NFS and Instant Recovery offer incredible value - you must try these if you have not.
  • Ability to truly validate backups, by starting up the VM's from the backups in an isolated setting to ensure application functionality.
  • "Lab" functionality that is great for testing patches, upgrades, etc
  • For you Hyper-V folks, support is coming in Veeam v6!
  • There is substantially more value in a Veeam solution than just 'backup'

Stumble Upon Toolbar