The problem with the cheap USB isolators is they are USB 2 Low or Full (12 Mbps) speed only, no High (480 Mbps) speed, so don't play well with anything that needs to stream a lot of data fast. The power isolation particularly sucks as the DC-DC converter used typically is rather noisy with very poor regulation, and is often a 1W model so limiting the device side load to under 200mA. However if one works for your particular application, great, as they do provide useful additional protection, though I wouldn't trust them if you actually need to float instrumentation at a high voltage with respect to ground, as although a genuine ADuM3160 is rated for 2.5kV isolation, its likely the far east manufacturers have FUBARed their PCB creepage and clearance distances, or used a DC-DC converter that cant pass a HiPot test at anywhere near that voltage.
I recommend plugging the PC and all the other power supplies into the same power strip to ensure any ground loops are minimum length and area (i.e. not right back to your main breaker panel), and in North America to guarantee everything is on the same mains phase to reduce the possible potential between devices due to SMPS Y capacitor leakage currents.
As Nctnico and Gamerpaddy have pointed out, bond the chassis and/or signal grounds of all the kit you are working with, (if you aren't using a USB isolator, including the PC), to create an equipotential zone, and you wont have any issues when you connect/disconnect stuff. Laptops can be difficult to bond to, as ones with accessible ground points like the jack posts of D connectors for legacy ports are likely too old to run the software you need. On modern laptops you may have to settle for bonding to the shell of a dummy USB or HDMI plug, after checking that all the port shells are actually commoned together. If that's too difficult, then at least isolate the mains supply to the whole setup before changing any connections.
If you do bond everything and have a good earth ground to your bench (i.e. a dedicated clean ground for RF work), you may want to ground the whole setup to see if you get any useful noise reduction.