Category Archives: VI4

ESX is Going Away – How to Migrate to ESXi

If you didn’t know it yet, VMware announced a while back that future releases of VMware will not include the “traditional” ESX Server. From their site:“VMware vSphere 4.1 and its subsequent update and patch releases are the last releases to include both ESX and ESXi hypervisor architectures. Future major releases of VMware vSphere will include only the ESXi architecture.”

If you are in a “24/7/365” shop then the applications running in your private cloud should currently be in virtual data centers (vDC) that are contained in DRS/HA clusters and the migration can be completed with no downtime to the applications. However, there are still other systems, such as development and test systems or possibly some minor infrastructure services applications that may not benefit from vSphere’s availability features. I know many people have scheduled outages, shutdowns, etc. during the upcoming holidays. It may the best time to migrate to ESXi…

A Little Bit About ESXi 4.1

I have actually been pushing ESXi since version 3.5. In every plan and design engagement where I have been involved, I have always started with ESXi as the version to be used unless there was a compelling reason NOT to use ESXi. I think the only thing right now that requires ESX is HP’s Matrix. I have yet to find anyone that uses that behemoth. There have been many improvements to the features of ESXi since version 4.0. VMware has a nice “Why ESXi” web page to explain these new features. The biggest thing is that ESXi has a smaller footprint and has fewer security updates needed than the “traditional” ESX Server. For instance, the latest round of patches has 11 for ESX and only two for ESXi. The small footprint allows ESXi to be installed on a USB stick, an SDHC or just on less disk space. There is also a nice VMware KB article explaining all of the differences between the current version of ESX and ESXi.

Here are some other things that I like about ESXi:

Treat the ESXi Server like it is an appliance

The ESXi installation should be treated as firmware. It is even called firmware in Update Manager. This means that there is no actual upgrade. The firmware is installed fresh every time. This also brings the idea of having stateless servers at some point in time. More on that later.

Kickstart Scripting

Really, there are three methods for a mass deployment. One is to use Host Profiles, but this requires manual steps and the extra costs associated with Enterprise Plus licenses. A second method is to install a base install of ESXi using the method I outlined in a previous post and then use PowerCLI or the vSphereCLI to customize the server. The new preferred method is good ‘ol Kickstart.

Several Boot Options

Now, you can boot from local disk, SAN Disk, a USB Key or an SDHC Card. One of the arguments some have against booting from USB or SDHC is the dreaded single point of failure (SPOF). My answer has always been that HA will cover this. And, if you think about it, an internal array controller is a SPOF as well. But, now you can boot from SAN if you wish. Just remember to create a space for logs and core dumps if you go the route of USB or SDHC.

Active Directory Integration

ESXi servers can now become members of a Micro$oft Active Directory Domain so that administrators can authenticate to the directory. Setting this up is done through the vSphere Client under the configuration tab. It appears that you can call the administrators group anything you want in AD as long as it is “ESX Admins”. Maybe that will change in future versions.

Hardware Monitoring

You can just do away with SNMP monitoring and use the Common Information Model (CIM) providers. The stock version comes with some basic CIM monitoring, but the major hardware companies provide custom baked versions of ESXi with OEM-specific CIM providers for more granular monitoring. Rather than setting up your monitoring software for SNMP, you just point it to the ESXi server and set it up for CIM or WBEM. If you are not using a monitoring server, you can use the vCenter server to handle alerts and alarms.

Enhanced Tech Support Mode and vCLI Operations

Using Tech Support Mode, formerly known as “Unsupported Mode,”  is now supported. You can log in as an administrative user and all commands run in TSM are logged. The biggest reason many people went into TSM was to kill a stuck VM. There is now a vCLI operation that will allow this.

The Migration Path to ESXi

Here is the “Super Dave Method” for migrating to ESXi. Like I mentioned before, if you have at least ESX Clusters with vMotion enabled, this will be a relatively painless process, with no downtime. Obviously, if you are also upgrading from a previous version, there will be reboots required for VMware Tools and possibly to upgrade the VM Hardware to version 7.


Yeah, you heard me. Take some time to familiarize yourself with the differences between ESX and ESXI by setting up at least one test system so you can hammer it before changing your production systems. The nice thing is that you don’t need hardware to do this. You can set up a VM and run ESXi as a VM.

vMA, vCLI and PowerCLI – Oh My

If you are already familiar with doing things from the ESX console, the vCLI will be very familiar to you. If you are a Winders guy, you should already know PowersHell, so the PowerCLI will be familiar. Also, get to know the vSphere Management Appliance (vMA). The vMA is a RHEL5 based VM Appliance that is pre-configured with the Perl SDK and vCLI. It also includes vi-fastpass command, which allow you to pass authentication through to the hosts and to vCenter. It also includes vilogger, which allows you to create a (very) poor man’s syslog server. I think you are better served using Splunk! or phplogcon. Either will allow you to parse the syslog entries eisier if you need to do any troubleshooting or forensics. If you want a nice guide to the vMA, head over to the vGhetto for some nice tips and tricks. If you are already a Linux shop and RHEL is not your cup of tea, then you can bake your own vMA with the Perl SDK and vCLI, but you will lose the vi-fastpass and vilogger capabilites. vi-fastpass is not a huge loss and you can set up your home baked vMA to include syslogd and phplogcon.

Create a Kickstart Script

The most efficient way of performing an installation on more than one server is to script it. VMware now supports using Kickstart. Test your KickStart script on the test ESXi server, weather it is physical or virtual, you should be able to test most of the functionality. Check out the nifty “DryRun” setting too. I am not going to rehash what has already been done or said regarding Kickstart, so here are some decent links:

Take a look at this VMware Labs Fling if you want to create a nice deployment server that automates the ENTIRE process. This brings up the idea of having stateless ESXi servers. As each one is inserted into the vDC, ESXi is automatically set up for you. All the “important stuff” is stored on shared disk.

If you decide to go the route of using PowerCLI, take a look at this nice post.


Back up EVERYTHING! ‘Nuff said? There is no better feeling when the shit hits the fan than to have a good backup. Make sure you back up the vCenter database and capture the configuration settings of all of the ESX servers. If you have Enterprise Plus, capture a host profile for each server. Take screen shots of the configurations settings in the vSphere Client. Include networking and all of the tabs in the vSwitches. Include storage and all of the tabs of any iSCSI initiators. Don’t forget the IQNs! Check the advanced settings for any tweaks. Document the whole thing.

Pick the First Victim and Move Your VMs

If you are using DRS and HA, disable HA. Then place the first victim host in Maintenance Mode. This should automagically move the VMs to other hosts in the cluster if DRS is working. Or you can manually (*GASP*) vMotion your VMs. If they are on local storage, set up an iSCSI server – OpenFiler is free. You can then use Storage vMotion to move the VMs to the iSCSI storage.

Remove from inventory

Yep! If the server is a part of a cluster, remove it from the cluster. Before removing it from inventory, unassign the license. Then, remove it from inventory. Once it has been disassociated from the vCenter Server, shut it down.

Now is a good time to update the BIOS, device firmware, etc. Blow out the dust. Perform some hardware TLC. If you do not trust yourself, disconnect it from the SAN. If you are using software iSCSI initiators or NFS, no worries because you will need to reconfigure this stuff after the install.

Install ESXi

Now is the time to actually perform your first ESXi installation. Go ahead, we’ll wait.

Post Configuration

Once ESXi is installed, you can add it back to the vCenter Server, add the license and perform any post configuration steps like apply the host profile or run your PowersHell script for post configuration. Once the post configuration is completed, confirm that all of the settings are correct and match what you documented previously. If all is good, add it back to the original cluster. If you want to set up Enhanced vMotion Compatability (EVC) and it’s not currently set up in the original cluster, you can create a new cluster for the ESXi Servers with EVC enabled.


Repeat the process for each server. If you decided to create a new cluster, start moving VMs to the new cluster.

More Tips

If you are installing ESXi to a USB Stick or an SDHC card:

  • Make sure the stick or card is supported by the hardware manufacturer.
  • Set up a syslog server. This is very important.
  • If you don’t set up a syslog server, make sure you have a swap partition on the centralized storage. If you do not do this, the logs are deleted on reboot. Then set up a syslog server.

Make sure that during the test phase you understand how to set up authentication, syslog settings and nay required custom settings. Also make sure you know your way around the DCUI and the Tech Support Mode console.

A Few Gotchas With vSphere 4.1! Updated

Since everyone else in the world is heralding the release of vSphere 4.1, I figured I would post some bad news. The stuff you may want to know BEFORE you jump into upgrading to vSphere 4.1. Before I start, I want to make it clear that vSphere 4.1 is a great product overall. And I have already been leaning to ESXi, so the announcement that this will be the last release with the “traditional” ESX has been expected. I will talk about ESXi and its improvements in a later post. I just want you to be aware of these rather significant Gotchas.

Gotcha #1 – Read Only Role allows members to add VMKernel NICs

From the release notes (You actually READ these, right?):

  • Newly added users with read-only role can add VMkernel NICs to ESX/ESXi hosts
    Newly added users with a read-only role cannot make changes to the ESX/ESXi host setup with the exception of adding VMkernel NICs, which is currently possible.

    Workaround: None. Do not rely on this behavior because read-only users will not be able to add VMkernel NICs in the future.

This is a fairly big security issue. I just LOVE the workaround notes. To be fair, I have found only one installation in my experience that uses the Read-Only Role. In my opinion, if they don’t have access to the physical data center, they don’t need any access to vCenter. But this is just something that should have been corrected before release.

Gotcha #2 – ESX/ESXi installations on HP systems require the HP NMI driver

  • ESX installations on HP systems require the HP NMI driver
    ESX 4.1 instances on HP systems require the HP NMI driver to ensure proper handling of non-maskable interrupts (NMIs). The NMI driver ensures that NMIs are properly detected and logged. Without this driver, NMIs, which signal hardware faults, are ignored on HP systems with ESX.

    CAUTION: Failure to install this driver might result in silent data corruption.

    Workaround: Download and install the NMI driver. The driver is available as an offline bundle from the HP Web site. Also, see KB 1021609.

It seems that every time HP releases a new set of SIM agents for ESX, something breaks. Is this VMware’s way of putting it on HP? Or was this an “OOPS”? If you search for “HP VMware NMI Driver” you come up with nothing. No download. It was no where to be found on Monday, but I did find it today on the HP support site.

Gotcha #3 – VMware View Composer 2.0.x is not supported in a vSphere vCenter Server 4.1 managed environment

The basic issue here is that vCenter 4.1 only works on a 64-bit system. View Composer only works on a 32-bit system. From the KB Article:

“VMware View Composer 2.0.x is not supported in a vSphere vCenter Server 4.1 managed environment as vSphere vCenter Server 4.1 requires a 64 bit operating system and VMware View Composer does not support 64 bit operating systems.
“VMware View 4.0.x customers who use View Composer should not upgrade to vSphere vCenter Server 4.1 at this time. Our upcoming VMware View 4.5 will be supported on VMware vSphere 4.1.”

Don’t these guys talk to each other? Didn’t they learn their lesson with the PCoIP issues? And why can’t you just admit it in the release notes instead of putting a link to the KB article? I completely missed this Monday morning.

Gotcha #4 – vCenter Installer SILENTLY Changes SQL Server Settings to Allow Named Pipes

  • vCenter Server installation or upgrade silently changes Microsoft SQL Server settings to enable named pipes
    When you install vCenter Server 4.1 or upgrade vCenter Server 4.0.x to vCenter Server 4.1 on a host that uses Microsoft SQL Server with a setting of “Using TCP/IP only,” the installer changes that setting to “Using TCP/IP and named pipes” and does not present a notification of the change.Workaround: The change in setting to “Using TCP/IP and named pipes” does not interfere with the correct operation of vCenter Server. However, you can use the following steps to restore the setting to the default of “Using TCP/IP only.”
  1. Select Start > Programs > Microsoft SQL Server 2005 > Configuration Tools > SQL Server Surface Area Configuration.
  2. Select Surface Area Configuration for Services and Connections.
  3. Under the SQL Server instance you are using for vCenter Server, select Remote Connections.
  4. Change the option under Local and Remote Connections and click Apply.

Can you hear the DBAs pissing and moaning?

Gotcha #4a – SQL Database is changed to Bulk Recovery Model (updated 10/27)

This on is funny. I just found out about it on 10/27/2010. When is comes to SQL for the vCenter database, VMware recommends using a simple recovery model. So, with their attention to detail, the upgrade process changes the database to a bulk recovery model. Inn this model, the logs keep growing until a backup purges it. No good.

Transaction log for vCenter Server database grows large after upgrading to vCenter Server 4.1 –


Again vSphere 4.1 brings some great improvements and some welcome changes. As the product matures and more vendors work with the APIs, we will see some nice features that will help you in your journey to the private cloud. The Gotchas listed above may not exist if quality assurance is tightened. I think I would rather hear that a release is delayed because of pending bug fixes. How long will we need to wait to fix these? In any case, if the Read-Only Role or the View Composer gotchas don’t apply, then jump right in and install or upgrade to vSphere 4.1. Just make sure you install the NMI drivers and fix the SQL settings.

Update 2010-07-16

I got a tweet from William Lam last night. It looks like versions are hard-coded in Capacity-IQ making it incompatible with vSphere 4.1. Will also explains two ways to make it work.

vShield Zones – Some Serious Gotchas

OK..I’ll admit it: I am spoiled by the capabilities of vSphere. What other platform lets you schedule system updates that will occur unattended and without outages of the applications being used? I don’t mean the winders patches, they require a monthly reboot. I am talking about the hypervisor updates. VMware Update Manager coordinates all of this for you. Then along comes vShield Zones to break it all.

First, let me explain what I am trying to do. To simplify things, vShield Zones is a firewall for vSphere Virtual Machines. Rather than regurgitate how it works, take a look at Rodney’s excellent post. A customer has decided to use vShield Zones to help with PCI Compliance. The desire is that only certain VMs will be allowed to communicate with certain other VMs using specific network ports, and to audit that traffic. ’nuff said.

vShield Zones seems to be the perfect solution for this. It works almost seamlessly with vCenter and the underlying ESXi hosts. It provides hardened Linux Virtual Appliances (vShield Agents) to provide the firewalling. It provides a fairly nice management interface to create the firewall rules and distribute them to the vShield Agents. Best of all, IT’S FREE! At least for vSphere Advanced versions and above. Keep in mind, that this is still considered a 1.x release and some things need to be worked out.

Now, on to the gotchas.

Gotcha #1 – Networking

When it comes to networking, the vShield Agent is designed to sit between a vSwitch that is externally connected via physical NICs (pNICs) and a vSwitch that is isolated from the outside world. The vShield Agent installation wizard will prompt you to select a vSwitch to protect. This is illustrated below. The red line indicates network traffic flow.

Click the Image to Enlarge

Click the Image to Enlarge

This works like a champ in this configuration, using a vSwitch for management, which is naturally on an isolated network to begin with, using a vSwitch for VMs to connect to the vShield Agent and using a vSwitch to connect everything to the outside world.  This can also be deployed with limited down time. If you are lucky enough to have the Enterprise Plus version, you may want to use a vNetwork Distributed Switch or even a Cisco 1000v. You will need to make some manual configurations to make this work as outlined in the admin guide.

The gotcha is with blade servers or “pizza box” servers that have limited I/O slots. If all of the VM traffic must flow through the same physical NICs and you use a vSwitch, then you need the vShield Agent to protect a port group rather than an entire vSwitch. You will need to create a vSwitch with a protected port group and connect it to the pNICs. Then you you can install the vShield Agent. Once the vShield Agent is installed, you will need to go back to the vSwitch attached to the pNICs and add an unprotected port group. This is illustrated below. The red line is the protected traffic and the blue line is the unprotected traffic.

Click on Image to Enlarge

Click on Image to Enlarge

As you can see, there is an unprotected Port Group (ORIGINAL Network). This needs to be added to the vSwitch AFTER the vShield Agent is installed. If the ORIGINAL Network is already a part of the vSwitch, it will need to be removed BEFORE installing the vShield Agent. In order to avoid an outage, you will need to disable DRS and manually vMotion all VMs off of the ESX/ESXi host before installing the vShield Agent and modifying the port groups.

Gotcha #2 – DRS/HA Settings

The vShield Agents attach to isolated vSwitches with no pNIC connection. As you should already know, using DRS and vMotion on an isolated vSwitch could cause inter-connectivity between VMs to fail. By default, you cannot vMotion a VM that is attached to an isolated vSwitch. You will need to enable this by editing the vpxd.cfg file. You will also need to disable HA and DRS for the vShield Agents so they stay on the hosts where they are  installed. Both are well documented. Obviously, you will need to install a vShield Agent on every ESX/ESXi host in the cluster.

The Gotcha here is that, with HA disabled for the vShield Agent, there is no facility for automatic startup. There is an automatic startup setting in the startup/shutdown section of the configuration settings. First, this is an all-or-nothing setting. Second, according to the Availability Guide:

“NOTE The Virtual Machine Startup and Shutdown (automatic startup) feature is disabled for all virtual machines residing on hosts that are in (or moved into) a VMware HA cluster. VMware recommends that you do not manually re-enable this setting for any of the virtual machines. Doing so could interfere with the actions of cluster features such as VMware HA or Fault Tolerance.”

So, if a host fails, HA will restart all protected VMs on different hosts. If the host comes back on line, you risk having DRS migrate protected VMs back to that host. This will cause those VMs to become disconnected because the vShield Agent will not automatically start. If a host fails, hope that it fails good enough so it won’t restart.

Gotcha #3 – Maintenance Mode

At the beginning of this post, I mentioned how VMware Update Manager has spoiled me. VUM can be scheduled to patch VMs and hosts. When host patching is scheduled, VUM will place one host in Maintenance Mode, which will evacuate all VMs. Then, it will apply whatever patches are scheduled to be applied, reboot and then exit Maintenance Mode. It will repeat this for each host in a cluster. This works great unless there are running VMs that have DRS disabled, like the vShield Agent.

In the test environment, when a host was manually set to enter Maintenance Mode, it would stall at 2% without moving the test VMs. I am not sure the order that VMs are migrated off, but none were migrated in the test environment. This could vary in different installations. Here’s the gotcha: you cannot power the vShield Agent off because the protected VMs would become disconnected. You cannot migrate it to a different host because it would cause a serious conflict and cause protected VMs to become disconnected. The only thing you can do is place the host in Maintenance Mode, then MANUALLY (*GASP*) migrate all of the protected VMs and then power the vShield Agent off. So much for automated patch management. We’re back to the “oughts.”


I said already that vShield Zones is a 1.x product. It’s a great firewall, but it has a few gotchas that you need to consider. The benefits may outweigh the negatives. But vSphere is a 4.0 product.Some of this should be able to be addressed by tweaking vCenter or host settings.

vShield Zones should be smart enough to allow us to select specific port groups to protect rather than an entire vSwitch. I guess whatever scripting is being done in the background will need to be changed for this. Maybe we need a Ghetto vShield?

One of the REALLY smart people at VMware should be able to tell us the “order of migration” when a host is placed in Maintenance Mode. Once that is determined, there is probably a configuration file somewhere that we could tweak to change it.

There should be a way to set up automatic startup and shutdown of individual VMs. The Startup/Shutdown settings sort of deprecated once DRS was introduced. The only time it is useful is with a stand-alone server or in a NON-DRS cluster. I guess the only thing that could be done is to add a script somewhere in rc.d or rc.local to start up these VMs, but how can that be done in a “supported” fashion with ESXi and is it supported in either ESX or ESXi?

I brought these issues up with some VMware engineers and they assure me that they are working on this. Hopefully they will figure it out soon. I hate doing things manually. It seems like it is anti-cloud.

vSphere 4.0 Quick Start Guide Released

The vSphere 4.0 Quick Start Guide: Shortcuts down the path of Virtualization has finally arrived!

I received a pre-release edition of the book at VMworld 2009. This guide has a great selection of shortcuts, tips and best practices for setting up and maintaining vSphere 4. I would be an excellent addition to any VMware administrator’s bookshelf. The book’s size also makes it a great reference for consultants as well. It will easily fit into your backpack.

It was authored by the following geniuses from the community:

Shows these guys some love and pick up a copy to support their efforts.

ESX vs. ESXi Which is Better? Revisited.

For over a year now, I have started off telling customers in Plan and Design engagements that they would be using ESXi unless we uncovered a compelling reason to NOT use it. The “which do I use” argument is still going strong. Our blog post “ESX vs. ESXi which is better?“  was posted in April and is still the most popular. It seems to be a struggle for many people to let go of the service console. VMware is trying to go in the direction of the thinner ESXi hypervisor. They are working to provide alternatives to using the service console.

VMware has provided a comparison of ESX vs. ESXi for version 3.5 for a while. Well, VMware posted a comparison for ESX vs. ESXi for version 4 last night. It’s a great reference.

Changes to the ESX Service Console and ESX vs. ESXi…again

A whitpaper was posted in the VMTN communities Thursday outlining the differences between the ESX 3.x and ESX 4.x service console. It further offers resources for transitioning COS based apps and scripts to ESXi via the vSphere Management Assistant and the vSphere CLI. Also mentioned briefly was the vSphere PowerCLI. If you are a developer or write scripts for VMware environments, also check out the Communities Developer section.

I hear it time and time again…The full ESX console is going away. ESXi is the way to go. I know there are valid arguments for keeping ESX around, but they are few. Failing USB keys may be a valid argument, but I have not heard of this happening. If that is the case, use boot from SAN. You need SAN anyway. As for hung VM processes, there are a few ways to address this in ESXi.

If the techie wonks at VMware are publishing articles about how to transition to ESXi, then resistance is futile…you WILL be assimilated…

Setting up a Splunk Server to Monitor a VMware Environment

In a previous article, I compared syslog servers and decided to use Splunk. Splunk is easy to set up as a generic Syslog server, but it can be a pain in the ass getting the winders machines to send to it. There is a home brewed java based app on the Splunk repository of user submitted solutions, but I have heard complaints about its stability and decided that I was going to set out to find a different way to do it.

During my search, I discovered some decent (free!) agents on sourceforge. One will send event logs to a syslog server (SNARE) and one will send text based files to a syslog server (Epilog). Using the SNARE agents appear to be more stable than using the Java App and does a pretty good job. So I basically came up with a free way to set up a great Syslog server using Ubuntu Server, Splunk, SNARE and Epilog.

I created a “Proven Practice Guide” for VI:OPS and posted it there, but it seems that it is stuck in the approval process. I usually psot the doc on VI:OPS and then link to it in my blog post, and follow up later with a copy on our downloads area. To hurry things along, I also posted it in both places:

VMTN: I/O Performance in vSphere, Block Sizes and Disk Alignment

Yes folks, it rears its ugly head again…Disk Alignment… If you have not read it yet, check out the whitepaper on disk alignment from VMware.

First, chethan from VMware posted a great thread on VMTN about I/O performance in vSphere. The start of the thread talks about I/O, then leads into anice discussion about block size. A couple of weeks ago, Duncan Epping posted a very informative article about block sizes. It convinced me to use 8MB blocks in VMFS designs.

Finally, the thread kicked into a discussion about disk alignment. As you know, the VMFS partitions created using the VI Client will aoutmatically be aligned. This is why I advocate NOT putting VMFS partitioning into a kcikstart script. The whitepaper demonstrates how to create aligned patrtitions on winders and Linux guests as well. The process is highly recommended for any intensive app. But I have always questioned the need to do this for system drives (C:) on guests. To do it requires a multi step process or the use of a tool, like mbrscan and mbralign, And I have wondered if it was worth the effort. Well, Jason Boche gave me a reason why it should be done across the board. And it makes sense: “This is an example of where the value of the savings is greater than the sum of all of its parts.”

Jas also outlined a very nice process for aligning Linux VMs and fixing a common Grub issue. Thanks for the tip Jas!

I should also thank everyone else involved: Chethan, Duncan and Gabe!

vSphere Install and Upgrade Best Practices KB Articles and Links

So, I use NewsGator to aggregate a BAZILLION feeds from several sources, blogs, like this one, actual news feeds and a bunch of VMware feeds. The VMware feeds are from the VI:OPS and VMTN forums. The VMTN forums allow you to create a custom feed by selecting the RSS link at the bottom right of each page or you can get a feed from a specific section of the forum by clicking the link on the bottom left of a list. On of the custom feed options is to get a feed of the new KB articles.

VMware has released quite a lot of new KB articles surrounding vSphere. They just released nice best practice guidelines for installing or upgrading to ESX 4 and vCenter 4. They are short and to the point. There is also a nice article covering best practices for upgrading an ESX 3.x virtual machine to ESX 4.0. One thing I noticed, but never thought about is this :

“Note: If you are using dynamic DNS, some Windows versions require ipconfig/reregister to be run.”

Eric Seibert over at vSphere-Land posted a nice set of “missing links” for everything vSphere. This is a nice, comprehensive set of links to evetrything you need for vSphere upgrades or installs.So, go check that out as well.

Installing VMware ESX 4 in Text Mode

There are many reasons to install VMware ESX in text mode. The main reasons I use text mode are that it seems quicker for me and text mode responds better when using remote console connections, such as iLo, DRAC or console over IP. Previous versions of VMware used a text mode that incorporated Anaconda and was very similar to the text mode for RPM based Linux distributions. The new text mode in ESX 4 is VERY rudimentary when compared to the earlier versions. Hoever, it performs very well and is fairly straight forward to use.

The text mode installer uses simple lists of choices. Usually, 1 is for continue or to answer yes. Some items will have more than one choice. Here is a screenshot:


The console OS truly appears as a VM in this version. You must create a datastore and then a VMDK that represents the COS. A disk of sufficient size will be required for this. My first attempt, using an 8GB disk failed. My second attempt, using a 10GB disk was successful.

You can download a doc outlining text mode installation HERE.

FLASH: ESX 4 Console OS is REALLY a VM this time!

While I was setting up ESX in text mode for my next blog post, I discovered that the installation sequence first creates a VMFS file system and then creates a VMDK file for the console OS. I confirmed it in the VIC. Here is a screen shot:


Click to enlarge image

I also noticed that the logs are now in a separate directory:

2009-04-29_174129Click to enlarge image.

Running VMware ESX 4 RC in a VMware 6.5.2 VM.

I just set up another quick VI4 lab on my laptop for the purposes of capturing screen shots and testing some things out. I was worried because I was not able to start VMs in this lab using ESX 4 Beta 2, but everything is fine again! Here is a screen shot of a Winders 2003 VM running inside an ESX 4 RC VM which is running inside of Workstation 6.5.2 on an Ubuntu machine.


Click on the image for a full-size view.

My VMX settings were from a post on VMTN when I was trying to get ESX 3.0.x to run on a WS 6.0.  Actually, XTraVirt came up with the solution originally.

Well, my VMX has not changed MUCH since then. I only added some parameters for sharing SCSI disks so I don’t need an iSCSI server. I found THAT information on Duncan’s Blog.

# Start DAC Customization

guestOS = “other-64”

monitor_control.restrict_backdoor = “true”
# monitor_control.virtual_exec = “hardware”
monitor.virtual_exec = “hardware”
monitor_control.vt32 = “true”


# For SCSI disk sharing
disk.locking = “FALSE”
diskLib.dataCacheMaxSize = “0”
diskLib.dataCacheMaxReadAheadSize = “0”
diskLib.dataCacheMinReadAheadSize = “0”
diskLib.dataCachePageSize = “4096”
diskLib.maxUnsyncedWrites = “0”

bios.bootDelay = “5000”

ethernet0.present = “TRUE”
ethernet0.connectionType = “custom”
ethernet0.wakeOnPcktRcv = “FALSE”
ethernet0.vnet = “/dev/vmnet3”
ethernet0.virtualdev = “e1000”
ethernet1.present = “TRUE”
ethernet1.connectionType = “custom”
ethernet1.vnet = “/dev/vmnet3”
ethernet1.virtualDev = “e1000”
ethernet1.wakeOnPcktRcv = “FALSE”
ethernet2.present = “TRUE”
ethernet2.connectionType = “custom”
ethernet2.vnet = “/dev/vmnet3”
ethernet2.virtualDev = “e1000”
ethernet2.wakeOnPcktRcv = “FALSE”
ethernet3.present = “TRUE”
ethernet3.connectionType = “custom”
ethernet3.vnet = “/dev/vmnet3”
ethernet3.virtualDev = “e1000”
ethernet3.wakeOnPcktRcv = “FALSE”
ethernet4.present = “TRUE”
ethernet4.connectionType = “custom”
ethernet4.vnet = “/dev/vmnet3”
ethernet4.virtualDev = “e1000”
ethernet4.wakeOnPcktRcv = “FALSE”
ethernet5.present = “TRUE”
ethernet5.connectionType = “custom”
ethernet5.vnet = “/dev/vmnet3”
ethernet5.virtualDev = “e1000”
ethernet5.wakeOnPcktRcv = “FALSE”

ethernet0.addressType = “generated”
ethernet1.addressType = “generated”
ethernet2.addressType = “generated”
ethernet3.addressType = “generated”
ethernet4.addressType = “generated”
ethernet5.addressType = “generated”
# End DAC Customization

My next posts will involve installing ESX 4 in text mode and some very interesting findings during that install….