When the virtual infrastructure components of your operation are, shall we say, unwell, there are a few likely culprits. The most likely of these are not patching the host or keeping your virtual machine (VM) integration tools software up to date on the guest OS.
The following tips will help you stay on top of your virtual infrastructure and avoid such problems:
- Set up automated alerts. Use the alert features in your software to save you time and trouble. Hypervisor management software has the ability to create email or SNMP alerts that send critical information from software or hardware to your email, messaging app, or even your monitoring software. There are several basic alerts you can set up as email notifications (e.g., host CPU being over utilized, host memory being over utilized, lost network connection to the host, high availability issues, etc.). Setting up these automated alerts will keep you from having to dig into your infrastructure all the time.
- Monitor the health of the hardware/software. Be proactive, not reactive, with how you monitor your infrastructure. Like any software, you don’t want to neglect it and face repercussions. Paying attention to alerts and rectifying issues right away will help you avoid unexpected downtime.
- Patching the hypervisor. Utilize the tools provided by the hypervisor software to apply patches. For example, VMware patches are generally released on a quarterly basis, but the key is to know the timing and process for whichever virtual software package you have. Checking every few months for new patches is a smart way to keep your system healthy. You can also sign up for alerts via your software that will let you know when there is a major bug fix or security issue. On the Hyper-V side, follow the same patching cycle you would use for your server OS.
- Updating hardware firmware. Most hardware vendors will provide firmware updates to which you can subscribe.
One you have your virtual infrastructure in good shape, you need to also make sure your virtual machines stay healthy. Updating your VM integration tools, maintaining templates, and patching the guest OS are critical. Here are a few, specific virtual machine-related health tips:
- Most hypervisors will let you know when your VM integration tools are out of date and will allow you to update without interacting with the guest OS.
- Have a set of templates or prebuilt, generic virtual machines on hand to help you respond quickly to requests for new systems or tests for HMI functionality. Templates to keep on hand, depending on licensing and base OS, can include: Windows Server 2012 R2, Windows Server 2008 R2, and Windows 7 for HMI. If you have vendor-supplied software, such as Rockwell Automation’s software and support, you can request templates from them.
- Your virtual infrastructure is only as good as your guest OS, so make certain you are patching it for bug fixes and security reasons. Windows releases patches the second Tuesday of each month. You can always check with your systems integrator or software solution to see what patches they approve.
Planning for Disaster
Beyond the heath maintenance tips listed above, you also need to be prepared for the worst-case scenario. Having a plan in place will save you from panicking in the event of corrupted virtual machines, corrupted storage devices, or sudden outages. Knowing exactly what you will do in these situations—and how long it takes to restore VMs—is paramount. Plan for losing one VM, all your VMs, or your entire site. A good action plan should include steps for restoring VMs, who exactly will perform the steps, and training for those individuals. If you run 24/7, multiple people will need to be trained for disaster recovery situations and each staff member should know how to test the backups. Training should include restoring backups to a disconnected network and making sure the health of your backups is validated. Review your disaster recovery plan at least annually.
The more you maintain your virtual infrastructure, the easier it becomes, and the more benefits you will see. When you call your software or hardware vendor with any issues, the first thing they are going to ask is, “Is it up to date? Has it been patched? Are you keeping your firmware up to date?” If you are already on top of these maintenance items, you don’t have to waste your time; you can get straight to the root of the problem, fix it, and be up and running more quickly.
Eric Schwinger is MIT analyst II at Interstates Control Systems Inc., a certified member of the Control System Integrators Association. See Interstates’ profile on the Industrial Automation Exchange.
