I just found out that VMware has updated their in-guest time synchronization with the 7.0 Update 1 release. Previously, we had to manually disable some of the advanced time synchronization ‘features’ that didn’t adhere to the front-end GUI option that said to not synchronize the guest time with the host. For most VMs, it is not that big of a deal, but for SQL Servers running in a highly available configuration, this act could break your availability solution.
By default, it shows that it was set to disabled, and under most circumstances, the hypervisor follows these settings. However, several situations exist where the hypervisor will reset the in-guest time, even though the setting is explicitly set to disable this feature.
- A virtual machine is resumed from a suspended state
- A vMotion occurs
- A snapshot is taken
- A snapshot is restored from
- A virtual disk is shrunk
- The VMware Tools service is restarted, which includes a guest reboot
These time sync actions can move a guest’s time backwards as well as forwards. More details about this conflict of settings are found in VMware KB1189. This action is extremely detrimental to the state of SQL Server high availability features, such as Availability Groups and Failover Cluster Instances, which depend on the in-guest time closely aligning with the Active Directory synchronized time. This action must be explicitly disabled to ensure that these maintenance items do not trigger an unexpected failover of the SQL Server HA solution.
To correct this prior to vSphere 7.0 U1, we’d have to add the following entries into the advanced parameters for the virtual machine.
BUT! Starting in vSphere 7.0 Update 1, they added an improved GUI option to allow you to forcefully stop this in-guest synchronization via the front-end. Edit the virtual machine settings, go to Advanced then VMware Tools, and make sure these two checkboxes are unchecked.
Additional details are updated now in VMware KB1189.
For those of you that have not updated yet to at least 7.0 U1, I encourage you to review this setting for your SQL Servers that are in an highly available configuration to see if this manual adjustment applies to you. If you’re current in your patching, review the use cases and adjust your VM settings appropriately!
Thank you VMware for updating this setting!