BusinessContinuity_0.jpg

NetConnect Blog - Your Resource For IT Tips, Tricks and News

Critical Bug Affecting VMware Hosts Running ESXi 6

Posted by Eric Kupfer on Dec 4, 2015 9:27:03 AM

software-bug.jpgRecently, a bug was discovered that affects changed block tracking on VMware hosts running ESXi 6. This is a critical flaw as changed block tracking (CBT) is the mechanism that many backup products (including Veeam) uses to decide what data has changed and needs to be backed as part of an incremental backup job. Unfortunately, the result of the bug is that there is no guarantee a successful backup job will actually allow you to restore good data.  

The options for a workaround to the problem were not practical for most of our customers and included rolling back the host to a previous version of ESXi or shutting down the virtual machine during the backup process.

VMware has released a patch to resolve the issue. Network Center, Inc. is recommending that all customers running ESXi 6 hosts apply this patch as soon as possible to insure integrity of their backup data.

Unfortunately simply applying the patch does not resolve the issue. Resetting the changed block tracking map is also required to insure consistency. Veeam has created a script that will reset CBT on virtual machines. The script requires that Powershell and VMware CLI are installed on the management station where it is executed.

A couple of quick warnings:

  • Resetting CBT may impact the amount of time a backup job will run as each block of data needs to be evaluated during the next backup cycle. Please take this into consideration if your backup window is constrained.
  • Also, the script creates a VMware snapshot, resets CBT then removes the snapshot. This process may “stun” a virtual machine. Therefore it is recommended that the script is run during off-peak hours.

VMware_logo.jpgAgain, this issue only impacts servers running ESXi version 6. All previous versions of VMware are unaffected. If you have questions, concerns or would like assistance applying the patches please reach out to anyone at Network Center, Inc. and we will be happy to help!

Please review the supporting documentation and find a link to the CBT reset script that Veeam has provided.

VMware KB: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2136854&sliceId=1&docTypeID=DT_KB_1_1&dialogID=840649925&stateId=0%200%20840663565

VMware patch info: http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2137545

Veeam KB article: https://www.veeam.com/kb2075

Veeam CBT reset script: https://www.veeam.com/kb1113

VMware CLI download: https://my.vmware.com/web/vmware/details?downloadGroup=VCLI600&productId=491

 NCI Contact Us Button

Topics: VMware

VMware vSphere 6 Released

Posted by Mike Pagan on Mar 12, 2015 4:46:44 PM

vsphere6announced01Earlier today VMware released a new version of their flagship product, vSphere. The new release includes 650 new features and innovations, most notably VMware Virtual SAN (VSAN) 6, vSphere Virtual Volumes, and multi vCPU Fault Tolerance support. We'll cover the new and upgraded vSphere 6 features in another blog post.

There are a couple of back-end changes in the latest version of vSphere, so if you are thinking of upgrading in short order, please refer to this support article KB2110293 before starting the process.  

An additional word of caution: if you use Veeam Backup & Replication (or any other virtualization-aware backup product) check your product's supported software versions. The official Veeam twitter account posted today that they will have a update for their flagship product soon to introduce vSphere 6.0 support.

twitter

If you have any questions regarding the upgrade process, please contact us. We'd be happy to work with you to answer your questions.

You view view the official press release here.

Contact Us Today!

Topics: vSpehere 6, VMware, VSAN

Subscribe to Email Updates

Recent Posts

Posts by Topic

see all