In this series of articles, I wanted to share my experience with running VMware Cloud Foundation in a nested environment to help those who are interested in getting more “hands-on” experience with the product.
What exactly are we setting up in this nested environment? The equipment discussed below will be enough to deploy the Management Workload Domain which consists of a four-node vSAN Cluster that includes a vCenter Server, (2) Platform Services Controllers, NSX Manager, (3) NSX Controllers, SDDC Manager, and (3) LogInsight Nodes.
Although I have enough equipment in my lab environment to setup VMware Cloud Foundation on physical servers, I prefer to use a nested environment for the following reasons:
- Reduced Power Consumption: My physical lab includes four Dell PowerEdge R630 Servers and a single Dell S4810-ON Switch. When powered-on this equipment uses approximately 588 Watts, whereas a single HPE DL160 Gen9 Server that I use for my nested environment consumes 117 Watts.
- Reduced Noise: A single HPE powered-on registers 55.6db noticeably quieter than all of the Dell equipment powered-on.
- Reduced Heat: Each Dell PowerEdge R630 Server outfitted with a single 495W power supply generates 1908 BTU, and the Dell S4810-ON switch generates 800 BTU for a total of 8432 BTU. Whereas a single HP DL160 Gen9 Server with a single 550W power supply uses 2201 BTUs.
- Reduced Deployment Time: The nested environment allows me to leverage Clones, I can restore a new VMware Cloud Foundation environment with four hosts from a clone in 18 minutes.
I started with the following equipment:
| Description | SKU | QTY | Total |
| HPE ProLiant DL160 Gen9 E5-2609v4 1P B140i 4LFF 550W | 830574-S01 | 1 | 929.94 |
| HPE DL160 Gen9 E5-2609v4 CPU Kit (2nd CPU) | 801288-B21 | 1 | 409.99 |
| Kingston 32GB DDR4-2400MHz Reg ECC Module | KTH-PL424/32G | 4 | 687.96 |
| Samsung 2TB 860 EVO SATA III 2.5″ SSD | MZ-76E2T0B | 1 | 397.99 |
| Total | 2425.88 |
The initial installation of VMware Cloud Foundation requires 52 vCPUs and 116 GB of memory. What I found was that the CPU is the biggest bottleneck, during deployment I monitored performance and witnessed high CPU Ready Times. I attempted to use a single Intel Xeon E5-2609v4 CPU, but the performance was atrocious. Adding a 2nd CPU provided better performance.
To lower cost you could reduce the amount of memory to 96 GB, although you may see memory ballooning during the deployment. It’s difficult to reduce the size of the SATA drive from 2TB, when using thin-provisioning the deployment requires 860GB.
I know that the Intel NUC and Supermicro E300-8d are popular choices for Home Lab environments. It may be difficult to use either of those choices due to their low CPU core count.
With the hardware above my installation fail when Cloudbuilder attempts to deploy the LogInsight nodes which require 8 vCPUs each. At that step you will have an opportunity to re-size the existing virtual machines to provide enough CPU capacity. I’ll discuss this in a future article.
In the next article I’ll cover the steps I took to prepare the ESXi host for the nested environment.