Using VMs for development enviroment

[Harvs]

A few people on here in the past have stated they have a VM for each major project specifically setup with all the tools etc needed.

As I spend more time on the road in different places juggling different clients projects (and some that get dragged up from the past) this idea sounds like it's got some real merit.

For those that do it, what are your experiences/recommendations?
- What VM software do you find best? (I mostly use Windows code/pcb tools with some Linux server development)
- Are there any significant hassles?
- What's performance like on a modern decent PC running things like Altium, VS and Eclipse?
- Does unplugging and plugging in embedded USB hardware cause any issues? (i.e. I assume it can be automatically routed through to the VM every time it's plugged in.)
- Any tips on managing them / archiving.
- Anything else?

[IanB]

I don't use VMs much, but if I do I find VMware works better than Virtual PC.

It's best to have a modern processor with hardware virtualization. Most recent processors support this.

You can never have too much memory. 8 GB minimum, 16 GB better. If your VM starts paging to disk you will soon be re-programming your machine with a hammer.

You will also want a fast disk and disk interface. Not as important as memory, but faster is always better.

If you want to program/debug micros, you will need to check carefully to find what works. VMs often have limited access to external hardware and ports.

[GeoffS]

I've used VMware (Workstation) for a number of years and I've been happy with it.

As IanB said, lots of memory. I currently run an older AMD 1090T (6 core) with 16GB of memory. Host OS is Linux and guests are both linux and Windows.

Graphics response in the latest versions is pretty good (I do a bit of CAD stuff) and direct hardware access to USB ports is supported.

[Harvs]

If you want to program/debug micros, you will need to check carefully to find what works. VMs often have limited access to external hardware and ports.

That's actually my biggest concern.  I've got a i7-3770k with 16GB ram and decent drives, so I imagine performance will be adequate (at least IDEs if not CAD.)

VMware appears to have a free version that I could probably at least try the programmers and see if it works (Pickit3 and STLINK/V2 are the main ones I use.)

[GeoffS]

If the device is USB based, you should have no problems. 
USB ports can be switched from the host OS to guest OS seamlessly.  I do this with cameras, a weatherstation and Arduinos.

[Rigby]

For me the bottleneck has always been disk IO.  Get an SSD or you'll find yourself waiting a lot more than you should.

Spinning disk already has terrible seek time, and with a VM you have to divide that time up between concurrent operating systems.  You feel it fast.  If you wind up swapping in the host OS or the guest, kiss a few minutes goodbye.

[nerdyHippy]

If you want to program/debug micros, you will need to check carefully to find what works. VMs often have limited access to external hardware and ports.

I've only got experience with virtualbox, but I've never had a compatibility problem that upgrading to the latest version didn't fix. Granted my requirements aren't terribly esoteric - i.e. programming whatever micro with whatever USB-based programmer. Can you elaborate on what problems you've had?

[hamster_nz]

The great thing about a VM is that any licensed software gets tied to the VM's MAC address or disk signature, so can be moved around quite easily if your upgrade your PC.

I use Virtual Box and have no complaints, other than you can't dynamically grow disks.

[IanB]

Can you elaborate on what problems you've had?

I haven't tried to use microcontroller dev tools from a virtual machine. But since VMs in general are somewhat insulated from the physical hardware, you do need to check what works with the setup you have in mind. You don't have any guarantees.

[Harvs]

So would getting a second ssd just to hold the vm's disk be the best solution?

They're not that expensive anymore so if it makes a real difference it could be worth it.

[IanB]

So would getting a second ssd just to hold the vm's disk be the best solution?

I wouldn't go that far. A large ordinary disk should be fine for VMs unless you have some requirements out of the ordinary.

[hamster_nz]

The great thing about a VM is that any licensed software gets tied to the VM's MAC address or disk signature, so can be moved around quite easily if your upgrade your PC.

I use Virtual Box and have no complaints, other than you can't dynamically grow disks.

You can create a dynamic disk that will grow as needed but it will NOT shrink down again.

On VMware you can build a VM with a 20GB disk, and if you run low on space you can edit the VM and say "oh, make that disk 80GB". If the VM is running Windows you then hop into Storage Manager, rescan the disk and expand your C: file system by 60GB.  Virtual Box doesn't let you do that... Even in the Virtual Media Manager with a powered off VM you can't change the "Virtual Size"  of a disk.

Unless of cause I am being a stupid user and just can't find the feature.

[plesa]

I'm also using VMware workstation. There is only one exception- hardware keys, it does not work with VMWare.But it is rare nowadays.

[Jeroen3]

I use Virtualbox as a sandbox for new tools (new versions and such) to test of they don't mess up my normal workstation.
Graphics is fine, you can play video in it, but you might get some mouse-lag when using cad tools.
USB is fine, but you have to tell virtualbox to route that specific PID/VID to guest. Otherwise the host locks it and you cannot pass it over anymore.

Yes, it is slow if you have a single hard drive.
And I've only seen the Windows XP mode thing being capable of shrinking a virtual disk.

[nctnico]

For me the bottleneck has always been disk IO.  Get an SSD or you'll find yourself waiting a lot more than you should.

Spinning disk already has terrible seek time, and with a VM you have to divide that time up between concurrent operating systems.  You feel it fast.  If you wind up swapping in the host OS or the guest, kiss a few minutes goodbye.

I use Virtualbox on Linux (32 bit with 12GB). The problem with Windows is that it tries to swap as much as possible to disk to keep as much memory unused. Far from ideal when you need lots of memory for running a VM. Linux tries to keep everything in memory (including a disk cache) so it only starts swapping when there is no other alternative. For example: sometimes I compile a complete Linux system which is several GB of source code and object files. Linux keeps everything in memory so there is no difference in time between a (silent) 5200rpm hard drive and an SSD.

[Harvs]

I ended up picking up an SSD on the way home anyway.  I wanted a separate drive for the jobs I'm currently working on that I could pull and take with me easily if and when needs be.

I popped Win7, Eclipse and Keil on a VMware machine, and well, it pretty much performs as fast as on the host machine.  Really I can't complain so far.

[krish2487]

VirtualBox's USB support is horrible and mostly broken. VMWare's is much better. It's a shame because I would prefer a FOSS product, but end up using VMWare because, well, it actually works.

Centos + KVM.

works just fine.

[synapsis]

I use VMWare for almost all of my development environments and haven't had any problems since Workstation version 6 or so.

I've used Eclipse, IAR, Visual Studio, Arduino's crap IDE, Android development... all without problems. In addition, I use it at work with PLC packages from Siemens, Rockwell, Mitsubishi, Omron, etc... and even with the odd hardware interfaces they require it works fine.

Plus being able to take the whole development environment off my desktop to a laptop for field debugging is priceless in itself.

[codeboy2k]

Plus being able to take the whole development environment off my desktop to a laptop for field debugging is priceless in itself.

I do this.  I use a separate virtual machine for each development project.  I use a 256GB  sata3 on an USB 3 external drive, and it's pretty fast.
This way I can carry the SSD disk from home to office to client without a hitch.

Just remember to backup your project source code to another disk; the tools can be replaced but the source cannot.
Carrying the disk around has an extra risk associated with it that it will be lost or damaged.

[jeremy]

I use VMWare fusion 6 on Mac OS. I run various instances of OSX, windows, linux and solaris pretty much every day. I have an i5 with 16GB of ram with a PCI Express SSD and I have no noticeable slowdown. In fact I'm usually running 2-3 VMs at once.

Nowadays I never get any problems related to virtualisation other than the occasional really odd USB device (and usually it is the vendor's fault, not VMWare). Oh and I can't attach bluetooth low energy devices to any slave machines, not supported yet apparently.

PS: the best bit is that when you upgrade your laptop/desktop, you just need to copy + paste and all of your software is migrated!

[krish2487]

Centos as host.

Sorry, I should have be more specific in the first post.

[neslekkim]

So would getting a second ssd just to hold the vm's disk be the best solution?

I wouldn't go that far. A large ordinary disk should be fine for VMs unless you have some requirements out of the ordinary.

I'm doing that on my office laptop, once you have tried it, you wont go back
(three ssd's, one for os/programs, one for sources, and one for all kind of stuff, amongst databasefiles and virtual machines, couldn't be happier)

[nctnico]

I'm not a fan of using seperate disks. I quit doing that about 20 years ago. The problem is that you always run out of space on one disk and start putting stuff on the other disk(s) and thereby scatter information all over the place. If you are worried about losing data: get a NAS and use rsync to make a backup while you are sleeping.

[Rigby]

I'm not a fan of using seperate disks. I quit doing that about 20 years ago. The problem is that you always run out of space on one disk and start putting stuff on the other disk(s) and thereby scatter information all over the place. If you are worried about losing data: get a NAS and use rsync to make a backup while you are sleeping.

That's a disk sizing and/or data management problem, not a disk quantity problem.

There's nothing fundamentally wrong with multiple disks, especially given that one can mount disks as folders within other disks using symbolic links or junctions.

[neslekkim]

I'm not a fan of using seperate disks. I quit doing that about 20 years ago. The problem is that you always run out of space on one disk and start putting stuff on the other disk(s) and thereby scatter information all over the place. If you are worried about losing data: get a NAS and use rsync to make a backup while you are sleeping.

Have 256GB+256GB+1TB, and data that I need to keep, is on server, only what I use here, I don't want to have an big disk that I cannot reformat if the OS craps on me, with this setup, I can take out the two last disk, reformat the maindisk, or replace it, and get up and running in notime. (kinda)
but yeah, stopped using multiple partitions, like I did for long time ago, but most of the problem back then, was the max size of the bootpartition, that is not an problem now, and regarding how windows works, it works good with multiple disk, instead of single disk, compiletimes are insanely fast compared to one single disk..

[Rigby]

If one understands what IOPS are and how they apply to physical disk, one would never consider a single-disk virtualization solution.

Run a VM from a USB 2.0 thumbdrive to see what an exaggerated version of that feels like.

[nctnico]

That may be true if you run a a couple of dozen heavily loaded server VMs but for desktop use a single disk is just fine as long as the OS does proper disk caching and memory management (IOW: I wouldn't use Windows as an OS to run VMs on).

[Rigby]

Linux grinds the disk plenty if it's doing actual work.

[nctnico]

True but only if it really has to or has nothing better to do. Like I mentioned earlier: even for a huge compilation task it doesn't make a difference to use an SSD or a relatively slow hard drive on Linux.

[Rigby]

Well then it's a good thing no one uses Windows.

[Harvs]

Well then it's a good thing no one uses Windows.

I have up to now, simply the dev tools I need to make a living are Windows only.

While I'm not willing to bet the farm on it just yet, swapping to Linux and running all the dev tools under VMs would be an interesting proposition.

But how do you go with USB devices that only have Windows drivers?  Is it as simple as the VM with Windows on it can communicate with the USB device without the host OS being capable of recognising it?

[jeremy]

But how do you go with USB devices that only have Windows drivers?  Is it as simple as the VM with Windows on it can communicate with the USB device without the host OS being capable of recognising it?

yep

[Harvs]

But how do you go with USB devices that only have Windows drivers?  Is it as simple as the VM with Windows on it can communicate with the USB device without the host OS being capable of recognising it?

yep

Wow...  I always assumed it just wouldn't work.

[jeremy]

Well with VMWare, it just provides a standard generic "device" driver which can be selectively attached to any USB device plugged into the system. When it is attached, the host can't touch it and the VM has all of the USB traffic forwarded to the virtualised USB chipset.

I've had a >99% success rate with it, even with random devices from all over the place. You can even attach a USB HDD with NTFS to a windows VM and pull the data out through there if you don't have the necessary filesystem drivers to mount it with your host. Same goes for EXT2/3/4 on windows/mac. I have in the past attached a USB drive to a linux vm for data recovery purposes.

There is also a VMWare tool which exports a physical system to a VM. I used it for an old win98 PC which had some obsolete PLC software for which the disks had been lost many years ago. Worked perfectly, and now everyone can have a copy of the software on their modern computers.

[Rigby]

VMware Converter.  Yep, it works great.

If you run windows on either the host or the guest, which most do, at least use a separate disk for the VM. Ideally you would use an SSD that held both the host and guest. 

[Rigby]

Wow...  I always assumed it just wouldn't work.

Linux Windows and Mac all support USB "pass-through" which does what you imagine.  Passes it straight through to the VM.

Some virtualization platforms support PCI and PCI-E pass through as well.  Hyper-V does this on windows server with supported cards.  This is how one can use high performance workstation graphics cards in a server, through a virtual machine or machines, over remote desktop and get great frame rates with real CAD apps remotely.

Virtualization tech has come a very long way.

[Harvs]

Please excuse my noob questions, but is there a particular flavor or configuration of Linux that works best as the host in this situation?

[alex.forencich]

VirtualBox's USB support is horrible and mostly broken. VMWare's is much better. It's a shame because I would prefer a FOSS product, but end up using VMWare because, well, it actually works.

What does that mean? Please be a little specific.

On Windows it just doesn't work most of the time. When it does I find it flakey. For some reason it just doesn't attach the USB device to the VM. I am on Windows 7 x64, might be better on x86. I couldn't be bothered to find out, VMWare just works.

That's very interesting.  I have had no problem with passing various USB devices through to a 64 bit Windows guest in Virtualbox under 64 bit Arch Linux.   I have an old USB based scope that I used to use that way all the time, and it even worked on my eee pc.  It was slower than molasses due to the single-core Atom processor, but it worked. 

More recently, I have had no issues with passing a USB network card through so I can use my N5306A PCI express protocol analyzer.  The dang thing is network booted, so it needs a dedicated network interface and the control software installs all manner of BS including a 3rd party DHCP server and FTP server.  Incidentally, I tried to install the control software on an older windows laptop a few days ago, and I couldn't get it to connect correctly, even after completely turning off the firewall.  I ended up setting up another VM.  But with all the baggage it drags along, I think the VM solution is definitely the way to go, even if the host itself is windows.  It also works just fine with a shitty Chinese EPROM programmer. 

Most of my development is done under Linux, so having a working VM is very important when I do need to fall back in Windows.  The one thing that eally gets to me, though - every time I turn it on, it has some damned windows update to install.  Every. Freaking. Time.

[SeanB]

Turn Windows update to manual install, or download and let me decide. that way when you are finished with the VM or are going home you can do the updates and leave them to run without bothering you. Just minimise the VM window while it is doing that, and check after an hour to see if it has the nag screen about rebooting. Reboot the VM and then shut it down.

[alex.forencich]

Turn Windows update to manual install, or download and let me decide. that way when you are finished with the VM or are going home you can do the updates and leave them to run without bothering you. Just minimise the VM window while it is doing that, and check after an hour to see if it has the nag screen about rebooting. Reboot the VM and then shut it down.

I suppose I should probably do that, but it's more just surprising that they release so many updates that basically every time I start up the VM, there is something new to install. 

[westfw]

So are people sharing an OS virtual disk between their various VMs?
With WXP I could get the OS, a toolset, and a set of projects, on a 10G virtual disk, and that wasn't so bad.  But W8 is much larger (30G according to "system requirements") and does seem as tunable for smaller systems, and that starts to get painful.  Sharing a single W8 OS disk and putting the tools and projects on separate disks would be OK, except that most tools tend to heavily pollute the OS disk with DLLs and such.  Sigh.

My experience (Older Mac Pro 8core/10Gmem host, Virtual Box, no SSDs) has been pretty pleasant.  Host OS disk caching seems pretty effective; the first time you start up AS6 (for example) is pretty painful (as it is on real hardware!), but subsequent startups are better, even if you've rebooted the VM in between.

Don't start up enough VMs that your host starts swapping/paging, though!  Ouch!  (2G has been enough RAM for WXP VMs.)

[jeremy]

I run all my stuff on a little macbook retina 13inch. The secret is not really much of a secret. Get an enormous SSD, get it on PCIX, get a ton of RAM. Rarely is anything CPU bound. 512GB PCIX SSD, 16GB RAM for me and it cost about AU$2000 if I recall. Given that I use my computer for all of my work basically every day, this paid for itself practically straight away. I can have a win8 VM with 6GB, linux VM with another 6GB and still leave 4GB for the host or for another little VM (usually a little install that just includes some stupid vendor tool that I have to use).

The mac/pc/linux wars are not a thing for me, I just use all of them 

And yes, fast SSDs are expensive. But so is your time! And the reason I use VMs is to keep the systems separated, it would be against my goals to have a shared disk. I'd rather just pay for the disk space.

[nctnico]

So are people sharing an OS virtual disk between their various VMs?

I'm using a network share to store the data. The network share is on the same machine as the VMs so that is extremely fast. I found sharing a local folder with Virtualbox to work flaky (I didn't test the latest version though).

[hans]

I'm an occasional user of VirtualMachines on VMware player or Virtualbox (most often Vbox).

In general, I don't see the comments made here. Maybe I'm not using them enough to encounter them, or they are specific to the host platform (I use Windows 7 in Test Mode), but USB and shared folders support has all worked mostly OK for me. USB took 2 tries initially, but it took my G25 wheel joystick fine and installed the "custom" driver inside the Windows machine.

I've even ran some OpenGL games inside Vbox, as it's a nicer place to be when you're trying to see how their internals work. That way I don't spoil my host Windows, where I run all games in Steam with it's lovely DRM & Punkbuster-kinda bots.
DirectX seems to be broken or very flaky support, also in VMware player (the sole reason I have it installed). Especially on Linux with ATI graphics. I hate that, I was hoping it may be an alternative to applications that don't run on Wine but benefit from (light) 3D work.