I think I may have just found the source of some real bottlenecks in my virtual environment.

VMWARE is this incredibly powerful platform, and the ESXi version lets even a SOHO environment run multiple virtual machines, housing a whole environment in a single big box. I absolutely love it, and support several virtual environments using it in a few places. There are, however, some confusing settings that aren't easily understood or documented. When you're dealing with inexpensive gear, the issues are even less explored and discussed. We make our best guess and see what happens.

In this case, I think I've discovered something that's really be slowing me down. If you take a look at the Advanced CPU settings under the Resources tab of your Virtual Machine configuration, there's an option for "Hyperthreaded Core Sharing". You have three choices. None, Any, and Internal. I'd been using "Any" thinking that I'd like the VMWARE server determine where to allocate threads from my VM based on load.



The problem, I think, is that because I'm using this in a very small environment, with just some cheap SATA drives connected right to the system board -- rather than a fancy RAID controller with caching and on board processing -- what can happen is that one thread can get stuck waiting for disk I/O to complete. If the processor is pushing some threads from another virtual machine through that same processor on the "hyperthread" (which is like using the unused register set on a processor during the same cycle) it can actually cause that other thread to hang up with it -- causing the slow down in just one VM to actually impact others.

I think. Maybe.

It's pretty deep technical meat here, and outside my usual area of study -- so maybe one of my guru friends here will comment. In any case, setting this to "NONE" and keeping each VM to it's own virtual processor seems to correct the problem.

I'd love to hear the thoughts of those of you who get deeper with this stuff.