PC for running virtual machines

[lollandster]

I have found that using virtual machines for everything not GPU intensive is the way to go. I am now debating whether upgrading the computer will make that experience better.

My current setup is AMD-2700X with 16GB RAM and Nvidia GTX1060 6GB running Windows 10 LTSC 2019 and VirtualBox.
Guest OS is mostly Windows 10 and Ubuntu and I usually have 1-2 VMs running at the same time.

The experience has been good so far when running Eclipse for programming and debugging MCUs, but it is sometimes slow especially with web-browsing and using VisualStudio Code.

I am thinking that a faster processor will help. I'm thinking AMD-3900X or Intel I9-9900. But maybe a server class processor like Xeon or Threadripper is more suitable? My budget is flexible, but if the gain is small then I'll stay on my current setup.

I'm also wondering if changing VM software can help. I like VirtualBox since it's so easy to install (no need to register or login anywhere). Some of the issues are with low fps so maybe a VM software with better graphics drivers is the way to go?

[martin1454]

Currently the best bang for the buck is the AMD 3900X, and it beats the 9900K in most tests, so I would go for that - It should be fine for what you are doing, but use some money for a good SSD and 32-64gb ram at least

[SilverSolder]

Agree about the virtual machine approach, it's the way the IT world is going (cloud) and it works for small business / lab / home too.

For running tons of VMs,  look for a machine with a lot of cores and a lot of RAM...  ideally, you want to be able to assign one, two, even more cores to each VM without stealing too much resources from other VMs. 

The clock speed is less important for this workload (within reason),  you need more of a "bulldozer" type computer that is very difficult to slow down, as opposed to a "sports car" that can go very fast as long as you don't have more than one passenger with you!

@martin1454, that AMD CPU looks extremely impressive,  that kind of horsepower would have required 2 sockets not that long ago.

[Bicurico]

I bought a second hand IBM X3650M3 server. They cost 100-200 Euro and can be cheaply upgraded CPU and memory wise.
You can then have independent SSD for each virtual machine.
As an OS I use the free version of VMWare ESXi.
Such a server can be fully remote controlled including on/off and BIOS access.
The CPU can be 2x Xeon X5680 giving you 12 cores, 24 virtual ones with Hyperthreading.

The downside is the noise. You dont  want to be in the same room.

Regards
Vitor

[nctnico]

I have found that using virtual machines for everything not GPU intensive is the way to go. I am now debating whether upgrading the computer will make that experience better.

My current setup is AMD-2700X with 16GB RAM and Nvidia GTX1060 6GB running Windows 10 LTSC 2019 and VirtualBox.
Guest OS is mostly Windows 10 and Ubuntu and I usually have 1-2 VMs running at the same time.

The experience has been good so far when running Eclipse for programming and debugging MCUs, but it is sometimes slow especially with web-browsing and using VisualStudio Code.

I am thinking that a faster processor will help. I'm thinking AMD-3900X or Intel I9-9900. But maybe a server class processor like Xeon or Threadripper is more suitable? My budget is flexible, but if the gain is small then I'll stay on my current setup.

I'm also wondering if changing VM software can help. I like VirtualBox since it's so easy to install (no need to register or login anywhere). Some of the issues are with low fps so maybe a VM software with better graphics drivers is the way to go?

Do you have an SSD? If not then upgrade to SSD first.

[Mr. Scram]

Knowing what to upgrade is a matter of looking at what is currently biting you. You'll need to look closely at your current system and whether CPU, RAM or I/O is loaded when slowdowns occur. Just blindly upgrading to something else is unlikely to yield the proper results.

[sokoloff]

I’m running on a Dell R720 with 96 GB ECC,
2xE5-2637 v2 @ 3.5GHz. The noise (other than during boot) isn’t too bad though it lives in the basement anyway. The other issue with older enterprise gear like this is the power draw is quite a bit higher than a more modern desktop turned server. In an area with cheap power, that matters less and it’s surely still more efficient than running a separate PC for every VM.

A server running 24/7/365 will run 8766 hours/year. Every extra 100 W is 877 extra kWh (ignoring HVAC cost or saving). At $0.20/kWh, a 100 W delta is $175 per year difference.

[Mr. Scram]

I’m running on a Dell R720 with 96 GB ECC,
2xE5-2637 v2 @ 3.5GHz. The noise (other than during boot) isn’t too bad though it lives in the basement anyway. The other issue with older enterprise gear like this is the power draw is quite a bit higher than a more modern desktop turned server. In an area with cheap power, that matters less and it’s surely still more efficient than running a separate PC for every VM.

A server running 24/7/365 will run 8766 hours/year. Every extra 100 W is 877 extra kWh (ignoring HVAC cost or saving). At $0.20/kWh, a 100 W delta is $175 per year difference.

Going with older stuff doesn't make a lof of sense if you run the hardware 24 hours a day.

[lollandster]

Currently the best bang for the buck is the AMD 3900X, and it beats the 9900K in most tests, so I would go for that - It should be fine for what you are doing, but use some money for a good SSD and 32-64gb ram at least

I'd like to know if virtualization is equal or if Intel is better at it.
I do need more RAM. The main reason I don't use more than two VMs at the time is lack of RAM.

I bought a second hand IBM X3650M3 server. They cost 100-200 Euro and can be cheaply upgraded CPU and memory wise.
You can then have independent SSD for each virtual machine.
As an OS I use the free version of VMWare ESXi.
Such a server can be fully remote controlled including on/off and BIOS access.
The CPU can be 2x Xeon X5680 giving you 12 cores, 24 virtual ones with Hyperthreading.

The downside is the noise. You dont  want to be in the same room.

Regards
Vitor

I have to look into VMWare ESXi. Maybe I'm doing it all wrong.

Do you have an SSD? If not then upgrade to SSD first.

I got my first SSD 8 years ago and have never looked back. I forget that some still run OS on spinning rust.

I’m running on a Dell R720 with 96 GB ECC,
2xE5-2637 v2 @ 3.5GHz. The noise (other than during boot) isn’t too bad though it lives in the basement anyway. The other issue with older enterprise gear like this is the power draw is quite a bit higher than a more modern desktop turned server. In an area with cheap power, that matters less and it’s surely still more efficient than running a separate PC for every VM.

Knowing what to upgrade is a matter of looking at what is currently biting you. You'll need to look closely at your current system and whether CPU, RAM or I/O is loaded when slowdowns occur. Just blindly upgrading to something else is unlikely to yield the proper results.

I have looked at the Task Manager to get a feel of this, but it can be hard to judge some times. Stuff like AMD-V may have limits I don't know about, maybe VT-x is better? This is why I ask. Looking at the Task Manager it looks like I have a well balanced system. It might come down to poor settings in the VM.

A server running 24/7/365 will run 8766 hours/year. Every extra 100 W is 877 extra kWh (ignoring HVAC cost or saving). At $0.20/kWh, a 100 W delta is $175 per year difference.

Power usage isn't a problem. The power is relatively cheap here and Norway is cold so the waste heat is not wasted at all. 2xE5-2637 v2 is about par with R7-2700X according to passmark.

[sokoloff]

I have to look into VMWare ESXi. Maybe I'm doing it all wrong.

I used to use ESXi, then switched some workloads to FreeBSD jails, and now run on Proxmox (which offers both kvm and container style virtualization in one solution).

I'm happy with proxmox and would recommend at least having a look at it. It's free (and you can disable the nag screen to subscribe to the paid support offering).

[olkipukki]

My current setup is AMD-2700X with 16GB RAM and Nvidia GTX1060 6GB running Windows 10 LTSC 2019 and VirtualBox.
Guest OS is mostly Windows 10 and Ubuntu and I usually have 1-2 VMs running at the same time.

You can always try build-in MS Hyper-V hypervisor that actually type 1 instead of VirtualBox type 2.
GPU support might be much better too.

If you are using VMs just for UIs and need simultaneous access at the same time, imho, no point in server-grade ESXi.

For example,
One of my setup is proxmox on workstation with GPU passthrough to Windows 10 that means a minimum performance penalty to run Windows in full speed.
The bunch of FreeBSDs are running in background on this machine as well. Of course, I can launch Ubuntu or Windows and get access to them as remote machines.
Also, I can run 2nd (,3rd etc.) Windows 10 (or Ubuntu) at the same time and passthrough USB/another GPU, but will need either a new set of keyboard/mouse/monitor for 2nd VM or KVM switch.

Another setup, very similar to yours, except VMware Workstation.
Windows 10 as host and a few VM's allows me easy switch between, deploy a new VM quickly to do some stuff/experiments/testings etc.

[lollandster]

I didn't expect this, but I tried VMware Player and it worked much better. I remember having problems with VMware in the past (several years ago now) so I didn't have much hope when I tried it, but everything feels smoother now. I can also watch videos with smooth motion and audio and view 3D models in kiCad without rendering errors. I guess VMware has vastly superior graphics emulation and that most of my troubles where graphics related. USB pass-thorough worked flawlessly also. I hope my first impression holds.
I wonder if Fusion360 will work in the VM now.

CPU usage is about the same so I'm now convinced that it wasn't main bottleneck and upgrading it might not have changed much.

I still need more RAM, but I hear that the prices will fall soon due to over-production, so I'll wait a few weeks.

You can always try build-in MS Hyper-V hypervisor that actually type 1 instead of VirtualBox type 2.
GPU support might be much better too.

I guess just using what I know was a mistake. I don't like that Hyper-V is Windows only, but if the performance is superior then I can live with it. I will give it a try tomorrow.

[olkipukki]

This is a primary reason why I choose VMWare Workstation/Fusion from day one, reuse same VM's on different OS's with good hardware support without major issues and not rely on something else.

[olkipukki]

I wonder if Fusion360 will work in the VM now.

I didn't try Fusion in VM as far as I remember, but heavy graphical demanding packages such as Altium Designer or SolidEdge works okay for occasional usage. I would not be happy to work everyday in that environment, might be personal. The high-clocked HEDT with enough memory could help to smooth UX experience, I hope.


 

[brucehoult]

If your VMs are mostly idle and you use them in bursts, rather than having long running background jobs using lots of cores, then the 2700X is perfectly fine.

The upgrade you need is MOAR RAM.

16 GB is low for one OS these days, let alone several. Get 32 GB or better 64 GB or 128 GB into that machine if it will fit.
 
VirtualBox is fine.

[Mr. Scram]

If your VMs are mostly idle and you use them in bursts, rather than having long running background jobs using lots of cores, then the 2700X is perfectly fine.

The upgrade you need is MOAR RAM.

16 GB is low for one OS these days, let alone several. Get 32 GB or better 64 GB or 128 GB into that machine if it will fit.
 
VirtualBox is fine.

Only if RAM is actually running out on the client OSs.

[brucehoult]

I’m running on a Dell R720 with 96 GB ECC,
2xE5-2637 v2 @ 3.5GHz. The noise (other than during boot) isn’t too bad though it lives in the basement anyway. The other issue with older enterprise gear like this is the power draw is quite a bit higher than a more modern desktop turned server. In an area with cheap power, that matters less and it’s surely still more efficient than running a separate PC for every VM.

A server running 24/7/365 will run 8766 hours/year. Every extra 100 W is 877 extra kWh (ignoring HVAC cost or saving). At $0.20/kWh, a 100 W delta is $175 per year difference.

I've got a 2990wx 32 core ThreadRipper with 128 GB of DDR4 and a couple of Samsung Pro SSDs. It draws 85W when more or less idle. I probably spent $50 or $100 more than I strictly needed to in order to get a quiet machine and it is virtually silent at idle even in a home office in a spare bedroom situation. The i7-8650U NUC I was using before it makes more noise if I max out even one core than maxing out all 32 on the ThreadRipper.

Getting any kind of older Xeon is a crazy idea compared to getting a modern i7/i9 or Ryzen. Just make sure you can plug in enough DIMMs. And get something with as efficient a power supply and as big (slow) fans as you can.

[SilverSolder]

Older Xeons are fine (ok, not older than Westmere due to the virtualization improvements and 32nm shrink)  -  at idle, they certainly don't burn full power and it is absolutely possible to have a quiet and powerful workstation that uses them.

[brucehoult]

Older Xeons are fine (ok, not older than Westmere due to the virtualization improvements and 32nm shrink)  -  at idle, they certainly don't burn full power and it is absolutely possible to have a quiet and powerful workstation that uses them.

Maybe you're right about Westmere, the previous generation was certainly awful.

I had a work-issued 8 core Mac Pro with dual 2.26 GHz (2.53 turbo) E5520 "Gainestown" (45nm Nehalem). It used more power at idle than the i7-860 2.8 GHz quad core (Lynnfield 45nm Nehalem) Hackintosh I built the same year used flat out -- AND the i7 built Firefox (my job at the time) significantly faster than the 8 core Xeon. I recall something like seven minutes vs nine minutes, while a Core2 Duo MacBook Pro took an hour.

The place I worked as recently as 18 months ago were still using Sandy Bridge Xeon build servers. I managed to sneak an 18 core i9 in there as a personal "PC" (at 20% the cost of a "server"), and immediately gave the rest of the team accounts on it!

[Mr. Scram]

Maybe you're right about Westmere, the previous generation was certainly awful.

I had a work-issued 8 core Mac Pro with dual 2.26 GHz (2.53 turbo) E5520 "Gainestown" (45nm Nehalem). It used more power at idle than the i7-860 2.8 GHz quad core (Lynnfield 45nm Nehalem) Hackintosh I built the same year used flat out -- AND the i7 built Firefox (my job at the time) significantly faster than the 8 core Xeon. I recall something like seven minutes vs nine minutes, while a Core2 Duo MacBook Pro took an hour.

The place I worked as recently as 18 months ago were still using Sandy Bridge Xeon build servers. I managed to sneak an 18 core i9 in there as a personal "PC" (at 20% the cost of a "server"), and immediately gave the rest of the team accounts on it!

Funnily enough the companies ahead of the curve are shortening their hardware cycles. Replacing even desktop hardware every three or even two years turns out to be good for productivity. It pains me to see companies trying to save a few bucks while leaving huge chunks of change on the table. At least the bean counters are happy.

[brucehoult]

Sure, when your cost to employ someone is well over $100k, letting them spend $5k a year on a fast computer doesn't have to improve productivity by much to be worth it.

[Mr. Scram]

Sure, when your cost to employ someone is well over $100k, letting them spend $5k a year on a fast computer doesn't have to improve productivity by much to be worth it.

Keeping people happy is also worthwhile. Spending a fortune on recruitment and training and then not taking care of people is an all too common mistake.

[SilverSolder]

Ford 8N tractors from the 50's and earlier are still popular -  there's someone/something for everyone!

[brucehoult]

Ford 8N tractors from the 50's and earlier are still popular -  there's someone/something for everyone!

Not sure what that's got to do with here, but old tractors never die -- they just retire to the seaside.

[DimitriP]

For running tons of VMs,  look for a machine with a lot of cores and a lot of RAM...  ideally, you want to be able to assign one, two, even more cores to each VM without stealing too much resources from other VMs. 

A long* time ago in computer terms this were true with the CPUs available at the time.
Now, past the middle of 2019, running anything @1.x or 2.x GHz is a waste of time, effort and money.
Especially on a "single user" machine hosting multiple VMs.

Fast CPU with less cores wil run a "single user" faster than a machine with "more cores" @2.x GHz.
Sure, the application matters but when it comes to programming MCUs, designing PCBs, or reading Datasheet PDFs, how important is multithreading ?

GHz , and single thread performance is still king  in my book.