My vCenter Server is now running. I'll configure some vCenter Server and PSC settings, then start adding ESXi hosts.

Configure vCenter Server Administrators and Windows Session Authentication

In order to be able to grant Active Directory users permission to access vCenter Server, we first have to add Active Directory domain as an identity source. From the Control Center machine, open a new web browser window/tab and head to the PSC Web Interface https://psc1.domain.com/psc. Click Appliance Settings, then click Manage. Click the Join button and enter the domain info and the appropriate credentials. Reboot the PSC appliance after clicking OK to complete the domain join process. Wait till the PSC finish rebooting, and all services are up.

Next go to https://vc1.domain.com/. If it is version 6.5, the HTML5 option will be available, however, I don't recommend using it yet, since it still lacks many features. Open the Flash-based vSphere Web Client, and log in using the default SSO user. Go to Hosts and Clusters, click on vc1.domain.com. Choose the Manage tab, then Settings, and under Advanced Settings, click the Edit button. Look for vpxd.certmgmt.certs.minutesBefore and change to 10 then click OK. (KB article).

Go to Home then Administration. Under Single Sign-On, click Configuration. Under Policies, click Edit and change the password's Maximum Lifetime to zero to disable password expiration (never do this in a production environment). Move to Identity Source tab, and click the green plus sign. Select Active Directory (Integrated Windows Authentication) and click Next. The Domain name field should be automatically populated. 

Under Single Sign-On, click Users and Groups, click on Groups, and click Administrators, click the Add Member icon, and add Domain Admins group. Repeat the steps to add Domain Admins to SystemConfiguration.Administrators as well.

Create vSphere Cluster and Add Hosts

It must have been more than 10 minutes since we set vpxd.certmgmt.certs.minutesBefore advanced option. Now is the time to create a new Datacenter, a new Cluster, and add the hosts using their FQDN. The steps are straightforward, nothing tricky here, as long as you don't enable DRS or HA features, since this is a lab environment.

Next up, I'll configure Distributed Switch, and create the appropriate Distributed Port Groups, and other VMware products.

vCenter Server Deployment

Our environment is now ready for deploying vCenter Server Appliance, whether version 6 or 6.5.

Deploy External PSC

I mount the ISO on the Control Center, and install the Client Integration Plugin if it is vCenter Server Appliance version 6 (since version 6.5 doesn't require that anymore). During the deployment, I choose External PSC, sync the time to the Active Directory domain controller, and enable SSH access. Reference

After the PSC is fully deployed, open the VAMI (https://psc1.domain.com:5480/) and set the root password to never expire, and choose the correct timezone. Open Putty, connect to the PSC, and set the default shell to BASH using the following command (KB article): chsh -s /bin/bash root

Configure VMware CA as Subordinate AD CA

Using Putty, create a new folder: mkdir /tmp/vmca

Launch the Certificate Manager (KB article): /usr/lib/vmware-vmca/bin/certificate-manager

Choose "Replace VMCA Root certificate with Custom Signing Certificate and replace all Certificates". For Machine_SSL_CERT.cfg, set the Name to Machine-SSL. For machine.cfg, set the Name to machine. For vsphere-webclient, set the Name to webclient. For certool, set the name to VAMI-Subordinate.

Save the CSR to /tmp/vmca. Keep Putty session connected, we'll use it again in a second.

Now launch WinSCP, and connect to the external PSC using root credentials. Go to /tmp/vmca and copy vmca_issued_csr.csr to the Control Center machine. Open the file in Notepad, choose Select All, and Copy. Open a web browser and go to http://dc1.domain.com/certsrv and click on Request a Certificate, then Advanced Certificate Request. On the Saved Request paste the copied text in the empty box, and for the Certificate Template, choose "vSphere6.0 VMCA", then Submit. Download the certificate using Base-64-encoded format. Save the file as "PSC-Cert.txt".

Go back to http://dc1.domain.com/certsrv and click on Download a CA certificate, certificate chaing, or CRL, choose Base 64, and click on Download CA certificate. Save the file as "DC1-Cert.txt".

Now open a new Notepad window, and copy the content of PSC-Cert.txt and paste in the empty Notepad file. Save the file as ca.crt but don't close it yet. Next copy the content of DC1-Cert.txt and paste at the end of ca.crt, then save again.

Back to WinSCP, copy ca.crt to /tmp/vmca. Now back to Putty, and choose Continue to importing Custom certificate(s) and key(s) for VMCA Root Signing certificate. For the Certificate, type /tmp/vmca/ca.crt, and for the Key type /tmp/vmca/vmca_issued_key.key. Press Y to confirm the certificate replacement, and wait till the process finishes.

Deploy vCenter Server

Mount the ISO on the Control Center machine if it is not, and under External Platform Services Controller, choose vCenter Server. Sync the time to the Active Directory domain controller, and enable SSH access.

After the vCenter Server is fully deployed, open the VAMI (https://vc1.domain.com:5480/) and set the root password to never expire, and choose the correct timezone. Open Putty, connect to the vCenter Server, and set the default shell to BASH using the following command (KB article): chsh -s /bin/bash root

Replace VAMI SSL Certificate

For the VAMI (virtual appliance management interface) that we access on port 5480, the SSL certificate hasn't been replaced with a trusted one. To replace it, perform the following steps for both PSC and vCenter Server (KB article):

Using Putty, connect to PSC. If it is version 6, run the following command (not needed if it is version 6.5) : /usr/lib/applmgmt/support/scripts/postinstallscripts/lighttpd-vecs-integration.sh

Now for both version 6 and version 6.5, open WinSCP, and connect to PSC. Copy the ca.crt that we created earlier to /etc/applmgmt/appliance/ directory. Edit the following file: /opt/vmware/etc/lighttpd/lighttpd.conf to add the following line: ssl.ca-file = "/etc/applmgmt/appliance/ca.crt" and save the changes. Now back to Putty, run the following command: /etc/init.d/vami-lighttp restart. The VAMI should receive a new trusted SSL certificate. Repeat the steps for vCenter Server.

Conclusion

The vCenter Server is now running, and all the certificates are trusted. In the next I will open vSphere Web Client and configure other parts of the environment.

First Nested Virtual Machines

Now that all nested servers are deployed and have their shared storage, it's time to deploy VMs in this nested environment.

Deploy and Initial Configuration

The first virtual machine to create inside that nested environment is the Domain Controller. Open vSphere Client (or the H5 client) to connect to one of the nested ESXi. Create a new Windows Server virtual machine, and place it in the iSCSI storage. After installation is done, I use the Server Manager to configure multiple settings, which I usually do for all Windows Server virtual machines (install VMware Tools, configure TCP/IP settings, set the time zone and sync to an NTP, disable Internet Explorer Enhanced Security Configuration for Administrators, disable Windows Updates as this is an isolated lab environment, enable Remote Desktop, disable the Windows Firewall, and rename the computer).

Active Directory Domain Services and DNS

Next step is to install the AD DS role and reboot the machine. It will automatically install DNS role as well. Open the DNS management console and create new Reverse DNS Zones as necessary. Then create "A" records for all ESXi, vCenter Server Appliance, and any Linux or workgroup-based Windows machines, and don't forget to check the "Create associated pointer (PTR) record".

From Group Policy Management, set "Maximum password age" to "0" in order disable the password expiration of Active Directory User Accounts.

Active Directory Certification Authority

On the same VM, install AD CS role, and choose "Certification Authority" and "Certification Authority Web Enrollment". For the AD CS Configuration part, choose Enterprise CA, and SHA256 for the hash algorithm. Go to http://localhost/certsrv/ and download the Root CA certificate in Base64 format to the domain controller desktop, and rename it something like "Enterprise-Root-CA.crt". Now proceed with creating the two templates as explained in VMware KB2112009.

This has been a short post, but in the next post I'll explain how I deploy vCenter Server using external PSC, and configure the VMware Certification Authority to become a subordinate of the AD CS Certification Authority we just created.

Stay tuned!

Note: I intend to publish more posts that show how to do each step in details.

Lab Introduction and Physical Architecture

Physical Architecture

In order to test new VMware products/features, I use the available hardware to build my very own lab. Usually I use one server with 256GB of RAM, and a FC/iSCSI external storage. The server comes with 4 NICs which are connected to a layer-3 switch. On the switch, I create the necessary VLANs and enable inter-VLAN routing. Afterwards, I configure the ports as Trunks. After installing the OEM-customized version of ESXi on the physical server, I add all 4 NICs in the same virtual standard switch, and enable the "Promiscuous Mode" on the switch settings. The number of Port Groups and their associated VLAN ID, or the number of Uplinks (vmnics) that I assign to each port group depends on the lab I'm creating.

Creating the VMs on the Physical Server

I start by creating nested ESXi servers. It's the same procedures as creating a regular VM, except you'll need to enable an option called "Expose hardware assisted virtualization to the guest OS", noting that this option is not available in legacy vSphere Client. Several people explain how to do this, so if you don't know how to create a nested ESXi VM, refer to their blog posts (hint: use Google to search for "create nested ESXi"). Other tips include reserving all memory, and using VMXNET3 adpaters.

The second type of VM is creating a new Windows Server VM and use it for two purposes:

1-  An iSCSI target for all the nested ESXi servers, thus providing them with a shared storage

2- A "Control Center" that I use to manage the nested environment.

For these purposes, I add a new role: "iSCSI Target Server" (refer to this link for more information). Also, I install Mozilla Firefox, Google Chrome, Notepad++, Putty, WinSCP on the Control Center machine. If this Control Center is able to reach the Internet, I sync its time with a reliable time server.

Nested ESXi Shared Storage

On the nested ESXi servers, I configure a Software iSCSI Adapter. I take note of the IQN and add it to iSCSI Target Server, (and if necessary, the physical external iSCSI storage).

Now that all nested servers are deployed and have their shared storage, it's time to deploy VMs in this nested environment.

Stay tuned for the next post.

Note: I intend to publish more posts that shows how to do each step in details.