During my career I’ve done quite some vSphere and VSAN implementations . The hardest part of implementing VSAN on a vSphere cluster is (usually) the network part. Especially in a stretched VSAN cluster configuration, there are some specific network requirements/best practices which should be followed, but because you don’t implement them on a daily basis you forgot some of the settings: you usually keep switching between the “health check” and “configure” tabs in the vSphere client #trialanderror .
It was during a troubleshooting session of a VSAN implementation were it was, that I thought: “There must be a clever way of doing this, this should be automatizible/scriptable!”. I’ve started with a small PSscript, which helped with troubleshooting VSAN, but then I build out the same script (BIG time) a.k.a. 700 lines of code (and counting).
My goal of this script: Enable (stretched) VSAN on an existing vSphere cluster, including the required network configuration as autonomous as possible.
And that is difficult as you need a lot of information, which must be gathered, processed and checked before VSAN can be enabled!
So let’s start with a word of caution: Yes, I’ve used this script to deploy production environments (with succes), but as which every script/program faults can occur!
You have been warned!
I’ve built-in some check-points, so you have full control over the different phases of the script.
Below a list of steps/checkpoints:
Flow of the script
below a overview of the different steps/decisions/checkpoint within the script:
|1||Are the PowerCLI modules loaded? not -> load them|
|2||Not connected to a vCenter > connect to a vCenter|
|3||Stretched VSAN Cluster questionaire:|
– Are we going to deploy a stretched cluster (a crucial question for course of the script).
– when deploying a stretched cluster: Are we going the deploy Witness network traffic separation?
|4||Select the non-VSAN enabled vSphere cluster which will be used for configuring VSAN.|
|5||Stretched cluster ? yes > select Witness host|
|6||vSphere HA enabled ? yes > automatically disable HA|
|7||Are all ESXi hosts versions the same ? no > manually update to the same version.|
|8||Check each Cluster ESXi host configuration:|
– Is host joined to another VSAN cluster -> leave existing VSAN cluster
– inventory VSAN enabled VMKernel interfaces (if any)
|9||Check DvSwitch configuration on cluster ESXi hosts.|
When multiple DvSwitches are available, select one which must be used for VSAN.
Automatically configure Jumbo Frames (if not already configured)
|10||VSAN (layer 2) network inventory part:|
If ESXi hosts are already connected to VSAN enabled network, re-use network.
if not the provision new networks or use existing DVS port groups.
|11||VSAN (layer 3) network inventory part:|
Based on VMKernel information, retrieve network subnet information per VSAN network . If information is not available it will be requested from the script runner (you).
Per VSAN network an (internal) IP pool is generated and will be filled with inventory information from step 8.
|12||VSAN Stretched cluster inventory part:|
Based on the IP pool information from step 11, hosts are automatically placed into a fault domain.
When necessary hosts can be moved between fault domains (not recommended).
|13||VSAN Layer 3 network configuration part:|
ESXi hosts without an existing VSAN connection, based on the information from step 8, 11 and 12, will be provisioned with VSAN-enabled VMkernel interfaces.
If VMK interface is already available on the selected network, VSAN will be enabled on that VMK interface and/or configured with a free IP address from the corresponding IP pool.
|14||MTU check on the VMK interfaces > Jumbo Frames will be configured automatically.|
|15||ESXi host route check,|
For stretched clusters only: if routes to other sites (and/or witness) site are not available on the host, routes are provisioned automatically.
For witness network traffic separation: the mgmt (vmk0) interface will be configure for traffic type witness.
|16||Enable and configure VSAN on vSphere cluster.|
for stretched clusters only:
With witness network traffic separation -> enable VSAN on VMK0 on witness appliance.
Create fault domains based on information from step 12 and configure VSAN stretched cluster.
|17||Enable vSphere HA:|
for stretched clusters only: Adjust HA admission control policy and set advanced configs.
So when starting the script it starts with the initialization phase.
Basic information is gathered and you will be requested to select a non-VSAN enabled vSphere cluster from the list (no screenshot)
After the cluster selection, the connected ESXi hosts are inventoried.
When you deploy a stretched cluster, a Witness appliance must be selected (the witness appliance must be already connected to the vCenter server).
Maybe in a newer version, the witness appliance will be deployed automatically (options are endless).
Cluster and ESXi hosts pre-checks will be executed.
In the rare occasion that a host is still joined to a VSAN cluster, which it shouldn’t as it is now part of a non-VSAN enabled cluster, the script will configure the host to leave the VSAN cluster after a human confirmation (read: you).
The script now will check for VSAN enabled VMK-interfaces and will determine the DVS and corresponding DVS-port groups based on that. If no VSAN enabled VMK-interface can be found, you’re opted to create new port group here or select existing ones.
When multiple DVS’s are found, you have to select a single one which should be used for the configuration.
When a stretched VSAN cluster is selected, you will be asked which port group should be used as a preferred site.
Let’s move on to the IP part.
Based on the VMK-interface information, IP subnets per DVS portgroups are determined and IP Pools are created (in the background).
The IP pool is filled with all available information, so when new (free) ip address are required no duplicates are allowed.
An overview of the gather IP information is shown per DVS portgroup before you continue.
During this part a lot of intelligent IP wizardry is happening, BUT IT WORKS!
For stretched cluster, hosts need to be placed in a site. Based on the gathered information, the hosts are placed in either the preferred or the secondary site.
You have the option to switch hosts to another site (which ISN’T recommended) as the IP subnets are usually site-local and in line with the site preference.
And from now on it’s goes very quickly:
The script has gathered all the required information and start configuring the VSAN Cluster.
When started, the script does not allow you to interfere nor it has checkpoint which must be manually confirmed.
In this screenshot you can see that esxp002 to esxp006 does not have interfaces on the VSAN network, the script wil automatically select an IP address an will configure new VMK-interfaces on the correct portgroup, set the IP, enable jumbo frames.
It check’s the route tables of each host, add routes when needed.
When using Witness traffic seperation, it checks VMK0 for traffic type witness and when not available it will be configured.
After the network configuration it automatically enables VSAN.
When a stretched cluster is selected, it creates fault domains (when they are not already have been configured), it enables the stretched cluster (including configuring the witness appliance)
And as a last step, it enables vSphere HA and configures the admission control required for stretched VSAN clusters.
I owe you a screenshot for the last part, but unfortunately I wasn’t able to catch a good screenshot of it (before I lost my access to my temporary lab-environment). Damnit ..
So with this script you can enable stretched VSAN clusters within minutes (and the health status will be OK!).
This script is still Work-In-Progress, but can be found HERE (https://github.com/Datacenter-Dennisch/CreateVSANcluster—PowerCLI).
I’m honored of sharing this script with you, the #vCommunity.
This script is one of my most advanced Powershell scripts which I’ve build so far, and I’m a little bit proud on myself (I’m tapping myself on the shoulder). Please try to run this script in your (greenfield) environment and watch when the magic happens. It’s awesome, It’s fantastic, it’s great .. I almost have some tears in my eyes of joy … Ok, maybe that’s a little bit too much …
On the serious note: please test this script and give me feedback! At this moment I don’t have a lab at my disposable anymore, which I can use to test my script. So I’m depending on you!
If you do have a stretched VSAN cluster LAB environment which I can use, please let me know and I can advance the script even more!
I didn’t had the time anymore to finish the script as I wanted: It works (very well), but not all features that I had in mind have been implemented.
July 9, 2020
[…] used simple examples in this blog , but this can be make as comprehensive as you want.Check out my VSAN Cluster build script: I used the above methods extensively in that script. It provides a foolproof script, for […]