The Blog is Back!and I Made a Change!
Learn more featured post
Jasonnash
22
July, 2009

Protecting Your VMware Fusion VMs with Time Machine

After helping someone recover data from a corrupt virtual machine last night I thought this would be informative to many people.  A lot of us Mac users still have to run WIndows for one reason or another.  If you do I highly suggest you look at VMware’s Fusion that allows you to run it as a Virtual Machine under OSX so that it doesn’t require a reboot like Boot Camp.  If you’re reading this you probably already are, but it’s worth mentioning.  A lot of us Mac users, me included again, also use Apple’s Time Machine to backup our systems.  For those that don’t, Time Machine is an integrated backup system built in to OSX.  It lets you automatically back up your system and documents to a local or remote drive.  You can connect an external drive via USB and Firewire to do it or access a network drive.  In my case I have an external 1TB hard drive connected to my Airport Extreme Base Station.  Apple also sells all-in-one devices called time Capsules.  This network drive solution is the most convenient for us as we have four Macs in the house and they all backup to that same disk.  The beauty of Time Machine is that it is completely automated.  Once per hour it backs up any changed files to the shared disk over the network, wired or wireless.  You don’t even notice it happening.  Not only does Time Machine keep the latest copy of your files but you can enter the “Time Machine” and roll back hours, days, weeks, or months to find the exact revision of your files you want.  Finally, if you buy a new Mac or rebuild your existing system you can do a complete system restore by just booting off the OSX  disk and telling it to recover your system from the network drive.  I did this just two weeks ago when upgrading hard drives in my OctoMac Pro.  New drives in, boot off the OSX disc, and in under 30 minutes I was exactly back to a working system on a new RAID0 set.

Time Machine is great but it has some limitations.  One is that if it sees a file get changed it backs up the entire file.  It doesn’t break it down in to any sort of sub-file level pieces.  This becomes extremely important on large files, like those that hold your virtual machine under Fusion.  If you change one byte in a 10GB file it will back that entire file up again.  Time Machine also shouldn’t be used to backup files that are in use, open files as they are called as you may end up with a corrupt copy on the Time Machine drive due to changes occurring to one part of the file while Time Machine is still backing up another part of the file.  This has been the problem with backing up virtual machines (VMs).  Many virtualization applications such as Parallels and Fusion would put your entire VM in a large file making it impossible to easily backup.  With the current versions of Fusion you can get around these limitations.

The first thing you need to do is get your VM like you want it.  Go ahead and install your apps and get it ready to use.  If you have an existing working VM that is fine too.  The idea is to make all the major application installs now, if you can.  That will reduce backup traffic later.  Now that the VM is ready you need to enable a feature in Fusion called AutoProtect.  You should be using this anyway.  It will use more space on your system’s hard drive, but that is configurable.

AutoProtect in Fusion

The AutoProtect feature creates a “snapshot” at a set interval so that you can restore your VM back to its state at that point in time. The idea here is that a snapshot is just a file on your disk.  It contains the changes, or deltas, that have occurred since the last snapshot (or original VM file if no snapshots exist).  So, you get your VM like you want it.  Then you create a snapshot.  If you install a new application those changes would actually be written to the snapshot file, not the original VM data file.  Then you create a second snapshot.  Any change you make now goes to that second snapshot file.  If you decide you don’t like what you’ve done you can “roll back” your VM to the first snapshot, or even back to the original VM data file.  It basically just throws that snapshot file away and boots the system from the one you select.  It’s a great way to protect yourself during application upgrades or OS patches.  Take a snap, make the changes and if something bad happens you just roll back.  Nice!

If you look at the screenshot above you will see that mine is set to take a snapshot every day and I keep up to 15.  Mine is set to keep 5 daily snapshots, then combine those in to a weekly for 5 weekly snapshots, and then combine those in to 5 monthly snapshots.  So this lets me “roll back” my VM to 5 months ago without using an enormous amount of disk space.  If you want to use less disk space just reduce the number of snapshots to keep.  This option is enabled from within the Virtual Machine settings for each VM you have.

So, how does this tie to Time Machine?  Remember early we said Time Machine backs up entire files when they change and you really shouldn’t trust a file that was open when backed up?  This is how we get around that.  So take a look here at a screenshot of my Windows XP VM folder:

Snapshots

You can view your own by going to the Virtual Machines folder (normally under Documents), right-clicking it and then choosing Show Package Contents.  What I want you to notice is that due to snapshots my VM is not a single large file.  It’s a lot of smaller files. The old snapshot files, the ones I created before but are not currently using are not open and not being changed.  The only one being actively changed is the snapshot that was created this morning.  Tomorrow that snapshot will be closed, unchanging, and available for Time Machine to backup.

The only caveat is that should  you restore your VM from Time Machine you need to go in to Virtual Machine -> Snapshots on the toolbar and roll back one day from the latest snapshot.  That way you boot off the one that was closed and not being actively changed.  What you have with this solution is a very robust recovery process.  If you install an app or update that breaks the VM you can use the AutoProtect snapshots to roll back to yesterday or earlier today.  If your system is totally down you have a reliable backup of your VM on the separate Time Machine drive.  If you start using Time Machine you get complete protection on your OSX system as well.  All without having to worry or think about doing a backup.

20 thoughts on “Protecting Your VMware Fusion VMs with Time Machine”

  1. As an additional suggestion, I would advise people make a shared folder out of the VM onto the Mac file system pointing their “My Documents” folder to a folder on the OSX side.

    This will have several benefits:

    1. Security. When Windows craps itself, you can just restore a new VM and lose no work.
    2. Time Machine backups
    3. The ability to swap between multiple VMs (XP, Win7 etc) that utilize the same data.

    1. I’m actually working on another post about doing just that for people that don’t want to TM the entire VM. I don’t…just to save space and I figure I can rebuild a VM in no time if needed thanks to the Fusion automated provisioning.

    2. I’ve been pleased to refer to this thread as I am a new Mac user with VMFusion (3.0.2) alongside OSX.

      Might you please expand on your suggestion of — ‘I would advise people make a shared folder out of the VM onto the Mac file system pointing their “My Documents” folder to a folder on the OSX side.’

      I’m not sure how to do this and what the resulting benefit would be. All help much appreciated.

      with regards………

  2. Thanks for this…very helpful. I’d been wandering here as recently played with Auto-Protect for a friend but hadn’t gone this far with it.

  3. HI, I found this blog post very helpful. I’m completely dependent on my vm and was looking for a convenient way to back it up. I was debating between something like the linksys wrt610n router hooked up to a usb hard drive, or time capsule (the former seems to be a lot cheaper). I want a backup solution that is toatally automated like time machine, but don’t know whether this would be equally as possible with a router, external hard drive and a backup program for windows. What would you recommend? Also, since I never actually use os x, would I notice a major performance boost if i used bootcamp instead? I have 4 gigs of memory and a 2 ghz processor and use a screenreader that is very memory intensive.

    1. Too bad you don’t actually use OSX, but I understand. If you don’t use it then I’d suggest you look at Boot Camp. You’d probably get a decent speed increase from doing that. I won’t say you’ll get something as seamless as Time Machine with a Windows Backup application but I think you can get close. The real key is to store your documents in a standard place and not spread out.

    1. That’s a good point. It’s something I overlooked because I haven’t hit that problem. If you read the post you link to, and I’ve seen it before as well, it says that the metadata change started a while back but right here on my Mac Pro with the latest Fusion it backs up the VM fine. This was a new build of a new VM on Fusion 2.x. Same way with my rebuilt VM on my MacBook Pro…so I haven’t seen Time Machine skip my VMs when doing a backup, but it’s worth checking to make sure so thank you.

  4. This doesn’t work very well in the long run. You may have smaller files, but because the autoprotect eventually deletes the older snapshots, you’ll end up having to back up the entire thing again.

    A better solution is to just create a sparse bundle and put your VMs in them.

  5. hi jason, i’m about to put time capsule’s ability to restore a vmware fusion virtual machine to the test, since i think my hard drive is on its way out. i’ve followed your suggestions about enabling snapshots, and i’ve moved my vmware fusion directory to a shared folder. i went into my time capsule backup, and sure enough, my vmware fusion machine is there. i even went into vmware fusion and had it open the vmware machine from the time capsule, and it booted up just fine (not sure if it was actually booting from the tc, because it didn’t give me any messages about activation or anything). my question: is there anything else i can do to make sure that my virtual machine will restore sucessfuly (i.e. ensure that it is not corrupt?)
    Thanks,
    Tasha

    1. You did pretty much what I’d suggest..if you can open the VM from the Time Capsule volume I’d say you’re good. If you have enough HD space copy that VM to a new directory on your local hard drive and load it from there too…but if not, booting it off the Time Capsule should tell you what you need to know. Just make sure it booted that one and not the one you already have on your drive. Boot both at the same time if you are unsure.

  6. I have a MacBook Pro with VMware Fusion. It is about 18 months old.
    My problem: I can not open Windows. When I click the VMware icon on the dock a black screen comes up with the big “Play” arrow and this error message-
    “File not found: Windows XP Professional-000003.vmds
    This file is required to power the virtual machine. If this file was moved, please provide its location”
    Efforts to resolve: The Apple store genius bar spent 30 to 40 minutes on it,twice. My company IT person (this is the only Apple we have) plus our outside consultant have spent many hours on it, all with out success. Calls to the VMware support were also no help.
    History: The hard drive was replaced by the Apple store. A few days prior to this I added a mobile backup drive and started Time Machine. Upon receiving the laptop back from the drive replacement, I followed the “utilities-migration-” instructions and the files, 150 gig approximately, were restored. All the Apple side files and all the the VMware side programs and data files were back to their original state. It worked for 3 1/2 days. Then, when I had VMware in suspension, while working on the Apple side, I clicked on the VM icon and the error message was there.
    Consequences: I run a business using this computer. I need to get to my XP “my documents” folder asap.

    Thanks for taking time to read this. I am open to all suggestions.

    1. I know this is years later, but I’m dropping a reply here to the future, relevant to the above.

      TimeMachine seems to silently skip files that have certain kinds of ‘Input/Output’ errors on copy attempts. Meanwhile, VMWare (via its occasional random access to those same files) often still works, even if files with such damage are part of the vmwarevm/vmdk set.

      The toxic mixture of these two behaviors is that you may think your Time Machine backup was complete — it finished, no errors! — but in fact one of the files inside the .vmwarevm bundle (essentially a directory that only opens when you right-click ‘show package contents’) is missing from the backup.

      This bit me after I upgraded a hard drive. I had 2 separate Time Machine backups of the old drive, over many days/months. The Lion restore to the new drive succeeded without errors. But launching one of the VMs gave these file-not-found errors related to one of the vmdk files (though not listing the exact name).

      Comparing to my old disk, I found that there was *1* vmdk file on the old missing from both the restore and all the backups. Trying to copy it over manually with cp/rsync/etc resulted in about 230MB (of the 2GB) coming over, then the dreaded “Input/Output” error. Apparently, Time Machine had been hitting this same error *for months*, and just proceeding with everything else without any report of a problem.

      Ultimately I was lucky: by copying over the old file with ‘dd’ (“dd if=path-to-old of=path-to-where-it-should-be-on-new bs=1k conv=noerror,sync” – found via http://www.waldonell.com/me/node/158 which is someone who had a similar problem with Parallels), I wound up with a same-sized, mostly complete version of the missing file, where ‘dd’ just filled in zeros where there were errors instead of crapping out. That was enough to get over the VMWare missing-file error, boot, and get the critical data out. I was probably helped by the fact that about 20GB of the disk contents was non-critical data I can regenerate from other sources… the damage must have landed there rather than in my critical software/data.

      This seems to bite people using VMs especially; perhaps it’s just the largest dataset into which one random file error is likely to be detected as fouling the functionality.

      VMWare Fusion4 is now advertising ‘Time Machine Compatible Snapshots’, and yet regular use of snapshots, by splitting changing state over many more individual files, might in fact make the chance that there’s one damaged file (which is thus missing from TM backups) among the necessary restoral set larger. So I’d guess a lot more people will be bitten by this.

      HTH,

Leave a Reply

Your email address will not be published. Required fields are marked *