Post

Virtual Machines Best Practices

Posted
By Fábio Ferreira
7 min read
Virtual Machines Best Practices

Summary

This post gathers practical guidance for operating Azure Virtual Machines: how billing works, storage choices, networking considerations, security hardening, image and automation best practices, patching strategies, and troubleshooting steps you can apply in production.

A. Quick checklist (operational)

  • Naming & tagging: enforce env, owner, cost-center, backup, business-unit. Tags drive chargeback and policy targeting.
  • Right-size first: run a few days of monitoring (CPU, RAM, disk queue) before applying RIs/Savings Plans.
  • Billing state: prefer Stopped (Deallocated) to avoid compute costs when the VM is idle; shutdown /s inside the guest does not deallocate.
  • Availability: choose single VM + AS/Zone/VMSS based on SLA requirements and recovery objectives.
  • Encryption: enable Encryption at Host and use CMK (Disk Encryption Set) for compliance-sensitive workloads.
  • Trusted Launch: use Gen2 + Trusted Launch for boot integrity and attestation where possible.
  • Monitoring: VM Insights (AMA), Boot Diagnostics, and alerts for CPU, disk queue, and agent heartbeat.
  • Backups: configure Recovery Services Vault with appropriate retention and test restores regularly.
  • Patching: centralize with Azure Update Manager + maintenance configurations and pre/post scripts.

B. Cost & pricing (detailed)

  1. Spot VMs
    • Use for interruptible workloads (CI builds, batch, stateless compute). Expect eviction; prefer Deallocate eviction to preserve disks.
    • Configure Max price if needed; -1 uses PAYG price as the cap.
  2. Commitments: Reserved Instances (RI) and Savings Plans
    • RIs: 1- or 3-year commitment on a size/region (scope: subscription/Shared/MG). Good for stable, predictable workloads. Family flexibility allows size changes within series.
    • Savings Plans (compute): commit $/hour for 1 or 3 years. More flexible across sizes/regions than RIs.
    • Combine with Azure Hybrid Benefit (AHB) to reuse Windows/SQL Server licenses and reduce costs further.
  3. Disk and storage cost
    • Prefer Premium SSD v2 for most production workloads: better perf/$ than legacy Premium and broadly supported.
    • Ultra and Premium SSD v2 are data-disk-only for high IOPS workloads.
    • Standard HDD is being retired, avoid for new critical workloads.
  4. Practical controls
    • Automated start/stop for dev/test via scripts or Schedule-based Automation to avoid paying for idle VMs.
    • Use tagging + budgets + cost alerts to detect unexpectedly large spend (e.g., many expensive VM sizes or orphaned disks).

C. Security hardening (practical steps)

OS hardening

  • Apply a hardened baseline: disable unused services, remove local admin rights, enforce strong password/SSH key policies, enable EDR or equivalent, ensure time sync and NTP config.
  • Keep the Azure VM Agent installed and updated, many platform features depend on it (Run Command, Extensions, Boot Diagnostics).

Network hardening

  • Avoid exposing RDP/SSH to the internet. Use a hardened bastion/jumpbox or Azure Bastion.
  • Place management hosts in a dedicated management subnet with strict NSG rules and logging.
  • Use NSG + ASG grouping to reduce rule explosion and follow least privilege for ports.

Identity & secrets

  • Prefer Managed Identities to access Key Vault / Azure APIs rather than storing credentials on the VM.
  • Use Key Vault for secrets and rotate them on a schedule.

Disk & host encryption

  • Encryption at Host protects data in use on the host side and is MS-recommended for new VMs.
  • Use Customer-Managed Keys (CMK) stored in Key Vault via a Disk Encryption Set when you need key control and audit.
  • Note: Azure Disk Encryption (ADE) (BitLocker/dm-crypt) is being retired, prefer Encryption at Host.

Trusted boot & attestation

  • Use Gen2 images with Trusted Launch (Secure Boot + vTPM + Boot Integrity Monitoring) to detect early-stage compromise.

Network access to PaaS

  • Prefer Private Endpoints (Private Link) for resource-level private access; disable public network access on PaaS resources when possible.

D. Networking & connectivity (detailed)

NICs & IP configuration

  • Each VM has 1+ NICs depending on size. Configure private IPs (dynamic or static) and attach Public IPs only when needed.
  • To add/resize a NIC, the VM must be Stopped (Deallocated).

NSG, ASG, UDR

  • NSGs at subnet + NIC level: prefer coarse subnet-level controls plus tight per-NIC overrides for sensitive workloads.
  • Application Security Groups (ASG) group VMs logically for simpler NSG rules.
  • User-Defined Routes (UDR): force traffic through NVAs/firewalls (e.g., 0.0.0.0/0 -> NVA IP). Enable IP forwarding on the NVA NIC.

VNet Peering & hub topology

  • Peering is non-transitive; design hub-and-spoke or Virtual WAN for large multi-spoke topologies.
  • Peering supports cross-region, cross-subscription and cross-tenant scenarios but requires bilateral configuration.

Private connectivity

  • Service Endpoints: subnet-level, legacy approach to reach PaaS services via MS backbone.
  • Private Endpoints (Private Link): resource-level private IPs, DNS integration via Private DNS Zones, works over ExpressRoute/VPN to on-prem.

Proximity Placement Group (PPG)

  • Use PPG to co-locate VMs for low-latency clusters (HPC, DB replicas). Create PPG before VMs and reference at creation.

E. Disks & storage (detailed)

Disk types & choices

  • Ultra Disk: extreme IOPS and throughput, tunable, data-disk only, for top-tier DBs.
  • Premium SSD v2: high IOPS/throughput, recommended for modern prod workloads.
  • Premium SSD: solid production option with good latency.
  • Standard SSD: cost-effective for dev/test and light workloads.
  • Standard HDD: legacy, being retired, avoid for new apps.

OS vs data vs temp

  • OS Disk: contains the OS and boot files, keep it on a reliable disk SKU and snapshot it pre-change.
  • Data Disks: attach/detach as needed; observe per-VM disk limits and IOPS available per SKU.
  • Temp Disk: ephemeral and non-persistent, use for pagefile, swap, or caches only.

Snapshots & images

  • Use managed snapshots for quick restores and to create managed disks from snapshots.
  • Capture generalized VMs into images (Compute Gallery) for reproducible deployments and versioning.

Performance tuning

  • Place I/O-heavy workloads on disks sized for the desired IOPS/throughput. Consider striping multiple disks (and use Storage Spaces or LVM) only when supported and necessary.

Resizing and conversions

  • You can often resize disks upwards online; decreasing OS disk size typically requires creating a new disk from a resized copy and replacing it (stop the VM to be safe).
  • To change disk SKU (Standard→Premium), use the Disk > Size + performance blade; some conversions require VM stop.

F. Images, deployment & automation

  • Build a golden VM (install baseline software, security agents, configuration) and generalize:
    • Windows: sysprep /generalize /oobe /shutdown and clean C:\Windows\Panther.
    • Linux: waagent -deprovision+user (or distro-specific deprovisioning).
  • Publish to Azure Compute Gallery for versioning, replication, and sharing.

Bootstrapping

  • Linux: prefer cloud-init for on-first-boot provisioning (user-data/customData).
  • Windows/Linux: use Custom Script Extension (CSE) for post-deploy bootstrapping; integrate with VMSS via extensionProfile.

VM Scale Sets (VMSS)

  • Prefer Flexible orchestration for modern use: supports mixed sizes, visible VMs in RG, easier maintenance.
  • Use autoscale with metrics (CPU, memory via custom metrics), schedules, and profiles for predictable scaling.

Infrastructure as Code

  • Use ARM/Bicep/Terraform modules for VM builds including NICs, disks, extensions, and PPG/AS/zone placement to make deployments repeatable and auditable.

G. Patching & maintenance

Azure Update Manager

  • Use Update Manager for assessments and scheduling. Create Maintenance Configurations with maintenance window, classification, and recurrence and assign them to VMs, RGs or MGs.

Hotpatching & reboots

  • Hotpatch (Windows Server Datacenter Azure Edition) reduces reboots for supported patches, prefer when available.

Pre/post actions

  • Attach pre/post scripts to maintenance windows to drain traffic, stop services, or perform backups before a patch, and to re-enable services afterward.

Testing

  • Test patches in a staging ring (a small set of VMs or a VMSS slot) before applying at scale.

H. Observability & troubleshooting

Boot diagnostics & Serial console

  • Enable managed boot diagnostics to capture screenshots and serial logs for failed boots.
  • Use Serial Console to access the VM when network-based access fails, ensure local accounts and credentials are available.

VM Insights & agent

  • Install Azure Monitor Agent (AMA) and enable VM Insights to collect CPU, memory, disk, network, and map data to a Log Analytics workspace.
  • Configure relevant performance counters and custom logs for application-level telemetry.

Alerts & metrics

  • Create alerts for CPU > threshold, Disk Queue Length, free space, and heartbeat (agent not reporting).
  • Use action groups to notify on-call teams, trigger runbooks, or integrate with PagerDuty/Slack.

Redeploy & host problems

  • Use Redeploy (migrate VM to a different host) when host-level issues are suspected. Redeploy + Reapply also reapplies extensions and policies.

I. Operational notes & commands

Moves & immutability

  • You can move VMs across resource groups or subscriptions (same tenant) with VM > Move. Cross-region requires replication (ASR) or recreate from snapshots.
  • Immutable: VM name, VNet and region cannot be changed, plan early.

Common commands

  • Deallocate (stop and deallocate via CLI):
1

az vm deallocate --resource-group MyRG --name MyVM
  • Redeploy a VM to another host:
1

az vm redeploy --resource-group MyRG --name MyVM
  • PowerShell deallocate (skip graceful shutdown):
1

Stop-AzVM -ResourceGroupName MyRG -Name MyVM -Force

Backups

  • Use VM > Backup to create Recovery Services Vault and policies. Regularly test restore procedures to validate backup integrity.

Maintenance & automation

  • Automate common ops: start/stop schedules, snapshot before changes, automated image publishing to Compute Gallery.
Cloud, Azure-104
azure vm security
This post is licensed under CC BY-NC-ND 4.0 by the author.