EEVblog Electronics Community Forum
Electronics => Projects, Designs, and Technical Stuff => Topic started by: InitialDriveGTR on April 03, 2013, 05:56:15 am
-
I modded my EVGA GTX670 FTW into 1/2 of a GRID K2 using methods discussed here: https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/ (https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/)
Anyways, Yeah its cool to be able to convert a consumer graphics card into many of their professional counterparts. The thing is while cards like a K5000 have more benefit to someone say who is doing freelance mechanical engineering, Not having to pay a huge amount of money for a card like a K5000 but still have that CAD oriented performance is huge. But all the other cards you can change my 670 into are realistic for an individual. They are all oriented towards larger server class systems. They are designed to scale. A lot. The features that a GRID K2 offer are not cheap, and to me would offer nothing I would ever use that my 670 can't already do. So what I'm trying to do, is see how this card will perform on its own. I have another one on its way, so I can run SLI like in the "true" GRID K2 and see how that setup performs.
This is an idea I've wanted to try for my college dorm/apartment, but never had a method to implement it. Most of my friends I live with have low spec'd macbooks or pc's. My goal is using say 8 virtual machines, each with the ability to use up to directx11, have my desktop not only host a Counter Strike Source dedicated server, but also the computers that are connecting to it as well. Anyone who wants to play on my server can, and from any device that can use RemoteFX through window's RDP service. Essentially my own private cloud gaming system.
So anyone who figures something out to do with the Tesla or GRID cards, post it here.
-
Nice :). I got some questions for you.
So, the card can be shared with multiple VMs? or do you need one per VM?
Also, what VM software are you using?
And finally, do you know if it will work with Xen based Virtualization?
-
Nice :). I got some questions for you.
So, the card can be shared with multiple VMs? or do you need one per VM?
Also, what VM software are you using?
And finally, do you know if it will work with Xen based Virtualization?
Well first off, me researching this stuff was completely kicked off by your work (gnif). I'm not sure if everything I say is correct, but according to my testing and information publicly available, I've made conclusions on various aspects of this fantastic mod, mostly that I feel NVidia could really earn a Gold Star in the technology world allowing any kid with a nice graphics card to do more. If NVidia offered a program where students who verify their status as an engineering student, would be allowed to unlock their gaming graphics card to perform as it's professional counterpart, I think they could get some huge kudos in the technology industry. Let the people like us experiment with this kind of technology in an environment that a college student could afford on a summer job's pay, and the possibilities are endless.
/End_Preaching
Anywho my understanding is that having a GRID K2 card in your system means that a server acting as a host to multiple virtual machines will supply graphics proccesing (Using said GRID K2) to multiple remote connections simultaniously. My guess is that the special hardware features used in GRID K2 cards acts sort of like a graphics processing pool, where when a Remote user needs Graphics processing, they get it. I guess this could be considered a "On Demand" Graphics card. I guess what I'm saying is that my understanding is that using one GRID K2, and multiple remote clients running some moderately graphics resource intensive application will all share the same card.
I'm using what came with my Windows 2012 Server disk; Windows 8 Enterprise is running on one virtual machine in Hyper-V (A part of windows server 2012) which I will eventually replicate to allow multiple sessions.
-
Anywho my understanding is that having a GRID K2 card in your system means that a server acting as a host to multiple virtual machines will supply graphics proccesing (Using said GRID K2) to multiple remote connections simultaniously. My guess is that the special hardware features used in GRID K2 cards acts sort of like a graphics processing pool, where when a Remote user needs Graphics processing, they get it. I guess this could be considered a "On Demand" Graphics card. I guess what I'm saying is that my understanding is that using one GRID K2, and multiple remote clients running some moderately graphics resource intensive application will all share the same card.
If multiple VMs work, my guess is that it is a hyper-visor that runs on the card which allows sharing of its resources. Not sure, but interesting tech none the less.
-
Each VM carves out a specified amount of RAM from the graphics card and and then uses what it can for resources on the GPU. The amount of RAM is set in a view server or in vSphere in the case of VMware (MS/Citrix does essentially the same thing).
The hypervisor controls the sharing process through the XORG service running on the host in VMware's case. I've been told you can also do it on Quadro 4000-6000 cards as well since they also have the native support.
You could farm out your GPU but you would need a hypervisor to do it like Hyper-V or ESXi. Both solutions are kind of expensive. As far as VMware goes, each VM would only be allowed 512Mb of video RAM anyways.
-
What is the difference between the Geforce (origonal) and the professional (modded) counterparts, except for Linux drivers? I already have a tool to overclock the GPU as much as can be thermally maintained, and it is (to the best of my knowledge) already using all of its memory. What is left? I run on Windows.
-
What is the difference between the Geforce (origonal) and the professional (modded) counterparts, except for Linux drivers? I already have a tool to overclock the GPU as much as can be thermally maintained, and it is (to the best of my knowledge) already using all of its memory. What is left? I run on Windows.
It has nothing to do with the speed of the card for day to day usage or gaming, it enables features used in professional environments for CAD/3D modelling and a few other things, which gives a performance boost as well as higher quality output. Also enables > 3 multi display output for windows, and > 2 for linux.
-
Here's a quick question. After modding the 680 to a GRID K2, I try to run GPU-Z in windows and it freezes. At the bottem it recognizes that there's a K2 but it cannot pull up any of the specs.
Has anyone else attempted to run GPU-Z after modding and had the same issue/gotten it to work? It seems to be running ok with no problems but I wonder if GPU-Z isn't aware of GRID cards yet.
-
Here's a quick question. After modding the 680 to a GRID K2, I try to run GPU-Z in windows and it freezes. At the bottem it recognizes that there's a K2 but it cannot pull up any of the specs.
Has anyone else attempted to run GPU-Z after modding and had the same issue/gotten it to work? It seems to be running ok with no problems but I wonder if GPU-Z isn't aware of GRID cards yet.
It works fine, see here:
https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/msg209140/#msg209140 (https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/msg209140/#msg209140)
Sounds like you either have an old GPU-Z, you have damaged your video card or something is screwy in your windows install.
-
Here's a quick question. After modding the 680 to a GRID K2, I try to run GPU-Z in windows and it freezes. At the bottem it recognizes that there's a K2 but it cannot pull up any of the specs.
Has anyone else attempted to run GPU-Z after modding and had the same issue/gotten it to work? It seems to be running ok with no problems but I wonder if GPU-Z isn't aware of GRID cards yet.
It works fine, see here:
https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/msg209140/#msg209140 (https://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/msg209140/#msg209140)
Sounds like you either have an old GPU-Z, you have damaged your video card or something is screwy in your windows install.
Ahhhh ya know what, I didn't even think to run it with my old (unmodded) GTX 580 in there to see what it would do......and it did the same thing :phew: . Now just back to trying to figure out why it is only reserving 256Mb in VMware instead of the 512Mb I set. I wish there was some software by VMware/Nvidia that let us administer the card directly.
-
I actually joined this forum because of gnif and his turning Nvidia cards into their professional counterpart. My interest is virtualization.
Unlike many/most AMD graphics cards, Nvidia only supports VGA passthrough in their professional (Quadro,...) line of "multi-OS" graphics cards. A Nvidia user for many years, I bought a Quadro 2000 card, the cheapest of the Quadro cards that supports VGA passthrough.
My application is photo editing and software like Photoshop and other Adobe titles that can utilize some of the extra features in these cards. Video editing would be another application that can benefit from these Nvidia features. In my case, I run Windows 7 as a virtual machine on a Xen hypervisor with a Linux dom0 (host). Using VGA passthrough I get baremetal performance in the VM. See http://forums.linuxmint.com/viewtopic.php?f=42&t=112013 (http://forums.linuxmint.com/viewtopic.php?f=42&t=112013).
It is a real pity that Nvidia locks down VGA passthrough as well as Linux / Xen / KVM / VMware etc. support. As it is, I can only recommend buying AMD cards to Xen users, or place a link to gnif's thread.
@gnif: The grid and similar Nvidia cards indeed support multiple VMs. These cards use what Nvidia terms the "Virtual GPU (vGPU) Hypervisor", see http://www.nvidia.com/object/cloud-gaming-benefits.html (http://www.nvidia.com/object/cloud-gaming-benefits.html).
Unfortunately here is where my knowledge ends.
-
I actually joined this forum because of gnif and his turning Nvidia cards into their professional counterpart. My interest is virtualization.
Unlike many/most AMD graphics cards, Nvidia only supports VGA passthrough in their professional (Quadro,...) line of "multi-OS" graphics cards. A Nvidia user for many years, I bought a Quadro 2000 card, the cheapest of the Quadro cards that supports VGA passthrough.
My application is photo editing and software like Photoshop and other Adobe titles that can utilize some of the extra features in these cards. Video editing would be another application that can benefit from these Nvidia features. In my case, I run Windows 7 as a virtual machine on a Xen hypervisor with a Linux dom0 (host). Using VGA passthrough I get baremetal performance in the VM. See http://forums.linuxmint.com/viewtopic.php?f=42&t=112013 (http://forums.linuxmint.com/viewtopic.php?f=42&t=112013).
It is a real pity that Nvidia locks down VGA passthrough as well as Linux / Xen / KVM / VMware etc. support. As it is, I can only recommend buying AMD cards to Xen users, or place a link to gnif's thread.
@gnif: The grid and similar Nvidia cards indeed support multiple VMs. These cards use what Nvidia terms the "Virtual GPU (vGPU) Hypervisor", see http://www.nvidia.com/object/cloud-gaming-benefits.html (http://www.nvidia.com/object/cloud-gaming-benefits.html).
Unfortunately here is where my knowledge ends.
Welcome :).
Yes, I have used the PCI pass back for NVidia acceleration under Xen in the past also.
[rant]
I used to recommend NVidia to everyone that came my way for which card to get, simply because their Linux support has always been pretty good even if they keep their drivers closed. This stopped when I discovered that they wont give triple head users support under Linux, and when I realised the cost of a single GPU quadro is almost 2x the cost of a dual GPU GTX690, and they have the identical GPUs on them. They are charging hand over fist for this, and IMO it is just dirty.
[/rant]
Will AMD work like the GRID under Xen? If so I would be very interested.
-
...Will AMD work like the GRID under Xen? If so I would be very interested.
A bit early to say, but AMD recently (March 27) announced its Sky series of Cloud Gaming graphics cards - see http://www.amd.com/us/products/desktop/workstation/cloud/Pages/cloud-gaming.aspx#4 (http://www.amd.com/us/products/desktop/workstation/cloud/Pages/cloud-gaming.aspx#4).
Now, if AMD plays it smart and opens their driver development to the open source public, perhaps along with some card donations to selected developers, they could even surpass Nvidia and take the pole position.
What makes me wonder is that Nvidia seems to develop their vGPU hypervisor as a closed source project, which looks like a major effort. Had they chosen an open source hypervisor such as Xen or KVM as the basis for their vGPU hypervisor, or better even integrated their product into such hypervisors, I guess they could have saved themselves a lot of money and work. This is pure speculation of a non-professional, just based on not having seen any open source stuff for the Nvidia Grid cards on the web.
What worries me is unfair play, like Microsoft having integrated open source code into their closed source code and only when they got caught did they retribute some code.
-
With regard to the K1 and K2 boards, here is something of interest: http://www.phoronix.com/scan.php?page=news_item&px=MTIyMjg (http://www.phoronix.com/scan.php?page=news_item&px=MTIyMjg).
EDIT: Looks like the new Nvidia Linux drivers support VGX graphics card, but not the GPU sharing.
As for Xenserver and Xen (the free version), see http://lists.xen.org/archives/html/xen-users/2012-11/msg00090.html (http://lists.xen.org/archives/html/xen-users/2012-11/msg00090.html). Xenserver will support the VGX hypervisor, but there is no work (yet) done in the free Xen version.
-
Anyone who wants to play on my server can, and from any device that can use RemoteFX through window's RDP service.
From what I read, RemoteFX (unlike VMware Horizon) won't discriminate between consumer and professional video cards.
The card has to support DX11 and must have a WDDM 1.2 driver. That's it.
http://social.technet.microsoft.com/wiki/contents/articles/16652.remotefx-vgpu-setup-and-configuration-guide-for-windows-server-2012.aspx (http://social.technet.microsoft.com/wiki/contents/articles/16652.remotefx-vgpu-setup-and-configuration-guide-for-windows-server-2012.aspx)
-
To summarize, there are two applications where professional NV video cards would make a difference: number crunching and virtual GPU.
Both features are enabled in software - appropriate drivers in the first case and VIB package in the second (for VMware hypervisor).
Number crunching is a bit of a vague definition, it includes CAD/CAM as well as video manipulation/encoding, etc.
Essentially anything that can take advantage of NV architecture.
Here is one real-life scenario. Mainframe-at-home sort of thing...
You have one computer tucked into the closet. 1-2CPUs, 2-3 PCIe slots. Running ESXi with the latest Horizon View 5.2.
You should have no problems running 5 to 10 VMs on it. Everything else in the house is a thin client running one of the VMs of this box.
Your smartphone and tablet can be such thin client. Maybe Google TV and Raspberry PI. Just good networking (wired and Wi-Fi) required.
If you have two GTX690 (modded to GRID K2) in the box, you can have 4 people play games with the equivalent of a GTX 680 each. At least in theory...
-
I cannot for the life of me start xorg server or NVIDIA-SMI on ESXI server using a modded card. |O |O |O |O
I would greatly appreciate anyone who has had success in starting xorg server and/or enabling NVIDIA-SMI (after updating ESXI with the NVIDIA .vib + most current vmware updates) using a modded card, to share the process that you used to enable this configuration/functionality.
Unrelated Note:
Currently, (due to my frustration :palm:) I am running my modded GTX 680 4GB now GRID k2 card on a windows8 machine with hyper-v installed.......hyper-v recognizes the GRID k2 and can share the card with vms. In addition, nview desktops functionality is enabled and simultaneously managing multiple virtual desktops on 4 high res monitors under windows8 .........not what I was looking for by modding the card, but it definitely turns windows into a whole different beast.
:wtf: ESXI
Someone, anyone, SOSOSOSOSOS!
THX
BDX
-
So OP, have you gotten your second 670 FTW modded and got it working? I have been running an SLI 670 FTW setup for a bit now and this sounds like something I might be interested in trying.
-
Looks like ESXI was not recognizing the GRID k2 as my secondary GPU, but the primary. I Changed this by switching the physical slots my pci devices were located at....seeing as my bios does not have primary/secondary VGA assignment I was unaware of this for what seemed like an eternity....never assume!
I hope this helps someone else along the way.
Alas, after 1+ years of searching for a way to accelerate the cloud, its mine :-// :-+ :-+
Thanks to all who have been apart of this.
F*** microsoft for slandering vmware.....and everything else.
VMWARE RULES!!!
Ahhhhhh..........it feels............. goooooooooooooooooood
-
Hello everybody,
I'm new on this forum and I would like to thank everybody (and especially gnif) for the mod of geforce GTX 680 to Quadro K5000.
I'm a gamer. I don't want more performance but just one (quadro) functionnality more.
I play games in 3D with the help of nvidia 3D vision on my monitor (input signal 1080P 3D 120Hz).
I would like to do the same but with passive 3D dual projection. I need therefore the option of a quadro K5000 to activate the possibility of passive stereo with 2 projectors connected to the graphic card. Here is the link to the procedure to do passive stereo with a quadro.
http://nvidia.custhelp.com/app/answers/detail/a_id/3012/~/how-to-configure-passive-or-dual-pipe-stereo-with-quadro-cards-in-windows-7. (http://nvidia.custhelp.com/app/answers/detail/a_id/3012/~/how-to-configure-passive-or-dual-pipe-stereo-with-quadro-cards-in-windows-7.)
Could someone please confirm that the option is available with a modded GTX 680 to quadro K5000?
Thank you,
Soulnight ;)
-
I've been looking at ESXi recently, and I think I'm starting to figure out how it actually does remote rendering for vSGA - and if I'm right, it a bit lame. It runs Xorg locally, and seems to do something similar to what VMware Player might do running a Windows VM on Linux - use OpenGL Xorg acceleration to implement Direct3D functions. It fires up a VM outputting to the local virtual headless Xorg server accelerated by the local GPU. It captures this output, encodes it into a video stream (also GPU accelerated), and sends it down the TCP connection to the client.
I can't make my mind up if that is really clever or really crude and unsophisticated. Of course, I could be entirely wrong, but given the components it uses, that is how it appears to hang together.
-
Praise all around! This is quite amazing stuff. We've been using Nvidia gaming cards for inventor & 3ds max for a while now and are beginning to hit their limits. This coincides with my interest in going freelance.
So here's my question~ typically (as I understand it and please correct me if I'm wrong) much of the AutoDesk software that I use can only recognize one card, even if you crossfire or SLI them or whatever. Through Xen or some other vm-type setup, would it be possible to harness, say, three of these modded 680s or 690s as one uber-workstation card?
I also like the idea of building a graphics capable VM server for the house so to use everything else as a thin-client, so kudos for any progress made in that front.
-
Quick question re: usability of modded nvidia cards. Is there any performance/functionality gain to be had for OpenCL? Reason I ask is I might need some opencv.org stuff soon-ish, and as always everything has to be faster, cheaper, cheaper, faster. In that order.
-
Through Xen or some other vm-type setup, would it be possible to harness, say, three of these modded 680s or 690s as one uber-workstation card?
No.
I also like the idea of building a graphics capable VM server for the house so to use everything else as a thin-client, so kudos for any progress made in that front.
You can do this using ESXi vSGA to share a single GPU among multiple thin clients, but you can also do it using the much more straightforward and higher performing Xen PCI passthrough or ESXi vDGA with one server-side GPU per client, using something like Kainy or Splashtop to stream desktop video output over the network to your preferred device of choice.
You could even approximate what vSGA does by using something simple like VMware Player or VirtualBox, both of which provide guest 3D acceleration by leveraging host's OpenGL capabilities. Fire up multiple VMs, and minimize them. Access them remotely using Kainy. You wouldn't even need to mod the GeForce card into a Quadro to make that work. This approach is similar to what ESXi vSGA does (that's why it needs a Xorg service per GPU running).
There are many ways to implement a solution, depending on what your use-case is and what level or performance you require.
Is there any performance/functionality gain to be had for OpenCL?
No.
-
Is there any performance/functionality gain to be had for OpenCL?
No.
Concise and to the point, thanks. :) This way I don't have to read that long thread just to find out there's no point in modding (for my use case).
-
Is there any performance/functionality gain to be had for OpenCL?
No.
Concise and to the point, thanks. :) This way I don't have to read that long thread just to find out there's no point in modding (for my use case).
Glad I could help. :)
-
To summarize, there are two applications where professional NV video cards would make a difference: number crunching and virtual GPU.
Both features are enabled in software - appropriate drivers in the first case and VIB package in the second (for VMware hypervisor).
Number crunching is a bit of a vague definition, it includes CAD/CAM as well as video manipulation/encoding, etc.
Essentially anything that can take advantage of NV architecture.
Here is one real-life scenario. Mainframe-at-home sort of thing...
You have one computer tucked into the closet. 1-2CPUs, 2-3 PCIe slots. Running ESXi with the latest Horizon View 5.2.
You should have no problems running 5 to 10 VMs on it. Everything else in the house is a thin client running one of the VMs of this box.
Your smartphone and tablet can be such thin client. Maybe Google TV and Raspberry PI. Just good networking (wired and Wi-Fi) required.
If you have two GTX690 (modded to GRID K2) in the box, you can have 4 people play games with the equivalent of a GTX 680 each. At least in theory...
Well, there is a 3rd application for a professional NV card: VGA passthrough under Xen or KVM. Nvidia seems to have disabled this feature in all but a few select Quadro and Grid cards.
VGA passthrough is used to run for example a Windows VM on a Linux/Xen machine with full graphics acceleration using a dedicated graphics card for the VM. This is great for Linux users who need to run Windows occasionally, for example to use commercial Windows applications (e.g. Photoshop) or games that aren't available on Linux, and don't run in Wine. Before VGA passthrough the only real option for Linux and Windows users was dual-boot.
-
The nVidia GRID cards do not have local video output but the "regular" cards modded into K2s do. If you are sharing a modified-to-K2 card between several VMs, what shows when you hook up a monitor to the outputs?
If you are using one ESXi server at home to virtualize, say, 4 desktops, wouldn't it be nice to have one video card with 4 display port outputs displaying one virtual machine per port. Then pass-thrugh a separate usb port to each VM and use display port based KVM-switch to switch between the VMs.