5 Best Practices for VM Patch Management
Patch management isn’t just about enabling automatic updates – you can’t just set it and forget it.
This approach may work for a while, but it’s only a matter of time before an automatic update introduces a problem with your virtualization infrastructure. Your organization should take a formal, methodical approach to patch management that covers all the bases.
Patch management in virtualized environments often focuses on virtual machines and the software running on them, but host servers and supporting infrastructure also require patching.
Include supporting components
In some organizations, virtualization hosts are joined to a management domain to make the infrastructure easier to manage and secure. This domain is separate from the domain where user accounts and other production resources reside.
If your organization groups its virtualization hosts into a separate domain, you must fix the domain components with the virtualization hosts. This includes domain controllers and DNS servers, but may also include additional components that should be addressed as part of your remediation strategy.
Remember firmware updates
Virtualization host servers rarely require firmware updates, unlike virtual hard disks. Storage vendors tend to release firmware updates more frequently than server vendors. Virtual hard disks typically reside on a Cluster Shared Volume, which is normally hosted on external storage arrays. VHD firmware updates can resolve newly discovered vulnerabilities, fix bugs related to iSCSI or Fiber Channel connectivity, or optimize storage performance.
Not all automated patch management systems can deploy firmware updates. You may need to use a proprietary tool from your hardware vendor to check for, download, and deploy firmware updates.
In production environments, virtualization hosts are almost always clustered. This is mainly for fault tolerance; if a cluster node fails, the virtual machines running on the cluster node can fail over to healthy nodes where the virtual machines run uninterrupted. This same architecture is also useful for patches. Patching a host server almost always requires a reboot, and cluster-aware patching architectures can migrate VMs between hosts as needed during patching.
Even still, patches can be disruptive. Patching a virtualization host is usually not a problem because the tools apply the patches to the C: drive – and the virtual machines reside elsewhere. If the remediation process occurs during a busy time of day, a host that is already stretched may see further performance degradation if it has to absorb virtual machines from a host that is being remediated. Plan to apply patches during off-peak hours to mitigate this issue.
Consider multiple patch management tools
You might need more than one patch management tool. For example, a patch management tool that works for Hyper-V hosts will not necessarily work for vSphere hosts. Similarly, VMs that reside in AWS, Azure, or Google clouds do not require host-level patching because the cloud provider takes care of that.
You may find that there are supporting infrastructure components that require patching. Depending on the nature of these infrastructure components and their location, you may need a separate tool.
Determine a patch management workflow
Develop a formalized patch management workflow. This workflow defines the steps that will be followed as part of the patch management process. The patch management workflow should address the following:
- how you will discover and download new patches;
- what tools you will need to deploy patches;
- how you will test the patches;
- how long the testing process is expected to take;
- how quickly you will deploy critical patches;
- the procedure for restoring a problematic patch; and
- responsible for the patch management process.
Test your patches
Test the patches that you plan to apply to your virtualization hosts. One way to do this is to create a small test/development environment configured to mimic production. This environment helps you run test/dev workloads and patch testing to ensure that patch updates don’t disrupt online infrastructure.