[MOVED] Hacking NVidia Cards into their Professional Counterparts

[gordan]

If the OS is 64bit - why does it matter where the BAR maps?

It doesn't. What DOES matter is if the total IOMEM size is limited to <= 3GB, because I can easily see a situation where you might want a LOT more IOMEM than that. Say you have a motherboard with 7 PCIe x16 slots (e.g. EVGA SR-2). Say you have 7 K6000 GPUs, packing 12GB of VRAM each. Say you want to boost the direct BAR size to be at least 12GB so you can map all of VRAM directly into memory address space. On Nvidia cards the direct BAR size has to be a power of 2, which means 16GB per card. Times 7, that's 112GB of IOMEM range. If the BIOS just craps itself in a situation where sum(IOMEM) >  3GB you have a serious functionality problem, even though technically it shouldn't be a problem - the BARs in question are 64-bit addressable so mapping those 112GB of IOMEM above 4GB shouldn't be a problem at all.

This is not entirely correct.  I have not seen anywhere that VMWare is doing VGX as of yet, instead opting to stick with their vSGA.  There are some similarities but they are definitely not the same thing.  Citrix is the first to release NVidia VGX integration, likely stemmed from their partnership work with NVidia to create this and providing software for NVidias demo servers.  At the very least, when comparing VMWare vSGA with Citrix and VGX, you are leaving out the required VMWare Horizon View package which is definitely not free.

The difference between VGX and vSGA is largely semantic. They both essentially achieve the same thing. Granted, VGX is going to be somewhat more efficient because there could be less binary level translation going on, but in reality I don't think it's going to be that substantial, mainly because the overhead isn't that big in the first place. As an example of the overhead ball park, it'll be similar to the overhead that WINE incurs when translating DirectX calls into host's OpenGL calls, which is typically in single figure % points. The other thing to bear in mind is that the way it works isn't really revolutionary at all - you could home-brew something that does the same thing. You just run a headless Xorg server on physical GPU with a remote desktop connection capable of streaming real-time video. VNC/rdesktop is too slow for gaming grade streaming performance, but there are other protocols commonly used that are up to it, e.g. WiTop. It's not some magical new technology, things like this have been in very common use since the days of dumb X terminals 30 years ago. The only new thing is streaming video protocol optimized for performance with GPU based offloading codec.

[mrkrad]

Good info! I was going to ask - 512K for both bios on K2 and 1 megaBYTE (256k x 4) for K1 - is that a normal size for uefi?

btw - grid K1 for $1800 (several) http://www.amazon.com/gp/offer-listing/B00CA5GCZ0/sr=8-1/qid=1386027459

here's a grid K2 for $1500'ish : http://www.ebay.com/itm/NVIDIA-900-52055-0020-000-GRID-K2-8GB-Graphics-Card-by-nVidia-/321257272457?pt=LH_DefaultDomain_0&hash=item4acc6cf489

which is the actually more than  K2's/Tesla K10/K20 go for.

In case you want to buy one and hack away at it.

I'd trade my K2 for a K1 since I need many low-end vm's over [many] high end vm's. 16gb of GDDR3 would do me better than 8GB of GDDR5 for doing firefox/office eh.

VGX is wanted because you use nvidia drivers and NVIDIA bios, and i think that is the only way i'll ever be able to virtualize OSX [client/server] with full QE/CI graphics.

If you have an idea how to virtualize several server/client OSX vm's - i'm all ears, honestly I'd probably be happy with parallels if it supports ISCSI or FCoE/FC storage (thin) and tons of ram and remote access (VNC) with sound and some sort of GPU sharing tech?

I plan to run it on MAC PRO so it would be 100% legal as esxi 5.5 works fine.

The problem with using PASS-THROUGH VGA mode is the lack of vmotion and snapshot veeam backups. There is no VSGA(shared) driver for OSX so you end up with sVGA2 no QE/CI which is rather useless for a vm.

this is the HP card i'm talking bout : 

-- really good PDF on the situation with graphics virtualization by HP http://h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA4-1701ENW.pdf

so 4 Quadro K3000m PCI ID: 10de:11be <-- notice the id? Can you strap that??

[gordan]

Approximately 200KB/BIOS sounds about right (2 BIOSes on K2, 4 BIOSes on K1).

ION: There is now a wiki set up to make the information discovered in this epic thread more easily searchable and accessible.

It is early days, but more info will be summarized from this thread and put there to avoid the need for newbies to read through all 50-odd pages of this thread to find the one post containing the information that is relevant to what they are trying to do. Of course, the ongoing modding effort discussion continues here.

Those of you interested in this effort primarily for virtualization purposes may find this thread of interest.

[mrkrad]

https://forums.geforce.com/default/topic/609099/nvidia-stop-garbage-about-shield-streaming-support/

gordan - you see the thread here about BETA and shield? I think you may have been right about it being all in the drivers.

http://forum.xda-developers.com/showthread.php?t=2394478

[tema]

Hi all again
I successfully modified SPARKLE SXS4501024D5SNM GeForce GTS 450  1GB to Quadro 2000
board is reference nvidia gts 450 http://www.ixbt.com/video3/images/ref/gts450-scan-back.jpg
upd:
gpu passthrough works fine.

Initial values are:

index	meaning	resistance
1	3 byte value D	none
2	3 byte value C	35k
3	4 byte values 8-f	none
4	4 byte values 0-7	25k

device / resistors table

device name	R1	R2	R3	R4
gts 450	none	35k	none	25k
Quadro 2000	35k	none	5k	none

furmark: http://www.ozone3d.net/benchmarks/furmark_192_score.php?id=120616

Will it work with DDR3 GTS450? I need Quadro ability to show 10-bit color. Thx.
http://www.asus.com/Graphics_Cards/ENGTS450_DC_SLDI1GD3/

[echen1024]

Can you hack the ASUS GTX660 2Gb model? I would be interested in using a quadro.

[C_Coffie]

Can you hack the ASUS GTX660 2Gb model? I would be interested in using a quadro.

I haven't found anyone that has successfully done it yet. I'm curious about this as well. If you are able to turn it into a quadro it would be a K4000.

[mrkrad]

the gtx660 is like the K3000m only 1 bit off.

ASUS GTX 660 Device Id:    10DE 11C0

Seems like these guys are best shot:

11bf   GK104GL [GRID K2]   
11c0   GK106 [GeForce GTX 660]   
11c2   GK106 [GeForce GTX 650 Ti Boost]   
11c3   GK106 [GeForce GTX 650 Ti OEM]   
11c4   GK106 [GeForce GTX 645 OEM]   
11c6   GK106 [GeForce GTX 650 Ti]   
11c7   GK106 [GeForce GTX 750 Ti]   
11c8   GK106 [GeForce GTX 650 OEM]

[gordan]

mrkrad, I guess you missed the post where I explained that soft-modding is ineffective on Kepler class cards (GKxxx GPUs).

My 780Ti arrives, and I can confirm that it is based on the exact same PCB as the Titan. Modding the 3rd nibble works the same way as it does on the 780 and the Titan, so you can make it into  K6000 easily.

Unfortunately, a quick visual comparison of the resistors I suspected of controlling the 4th nibble strap shows that they are in the same place on both PCBs, rather than at least one being in the alternate position (for nibble values >= 0x8). So no info on the 4th nibble strap location yet. Then again, given the 780Ti is cheaper than the Titan anyway and trivial to modify into a K6000, I imagine it's going to become the second easiest / most popular mod after the GTX680 to K10 (just remove one resistor off the back of the PCB).

Happy modding.

[mosmo]

mrkrad, I guess you missed the post where I explained that soft-modding is ineffective on Kepler class cards (GKxxx GPUs).

My 780Ti arrives, and I can confirm that it is based on the exact same PCB as the Titan. Modding the 3rd nibble works the same way as it does on the 780 and the Titan, so you can make it into  K6000 easily.

Unfortunately, a quick visual comparison of the resistors I suspected of controlling the 4th nibble strap shows that they are in the same place on both PCBs, rather than at least one being in the alternate position (for nibble values >= 0x8). So no info on the 4th nibble strap location yet. Then again, given the 780Ti is cheaper than the Titan anyway and trivial to modify into a K6000, I imagine it's going to become the second easiest / most popular mod after the GTX680 to K10 (just remove one resistor off the back of the PCB).

Happy modding.

Which 780ti did you get Gordon? So by modding the 3rd Nibble we can turn it in to a K6000, what does the 4th nibble give us exactly?

[gordan]

Which 780ti did you get Gordon? So by modding the 3rd Nibble we can turn it in to a K6000, what does the 4th nibble give us exactly?

I got the EVGA base model and flashed it with the SSC BIOS. So I got the performance of the most overclocked EVGA card for the price of the cheapest EVGA card. 100% stable, OCCT-ed overnight. GK110 is a demon OC-er, with a bit of tweaking I'm sure it'll go a lot faster, but Kepler BIOS Tweaker doesn't seem to be quite up to the job of power limit modding on it. If anyone knows of a better tool, please, do share.

The 4th nibble would let us mod the 780 and Titan into a K6000 as well. Without the 4th nibble info, only the 780Ti is moddable into a K6000.

[mosmo]

Which 780ti did you get Gordon? So by modding the 3rd Nibble we can turn it in to a K6000, what does the 4th nibble give us exactly?

I got the EVGA base model and flashed it with the SSC BIOS. So I got the performance of the most overclocked EVGA card for the price of the cheapest EVGA card. 100% stable, OCCT-ed overnight. GK110 is a demon OC-er, with a bit of tweaking I'm sure it'll go a lot faster, but Kepler BIOS Tweaker doesn't seem to be quite up to the job of power limit modding on it. If anyone knows of a better tool, please, do share.

The 4th nibble would let us mod the 780 and Titan into a K6000 as well. Without the 4th nibble info, only the 780Ti is moddable into a K6000.

Fantastic, lets hope the rest will follow your bravery and mod some more 780ti's. I am probably going to buy the same one you have.

[gordan]

Fantastic, lets hope the rest will follow your bravery and mod some more 780ti's. I am probably going to buy the same one you have.

It really isn't that big a deal. There aren't even any "warranty void" stickers over the screws on the card (well, on mine at least), so you don't obviously void the warranty by taking the heatsink off. Also, given you are soldering a relatively large 1206 resistor that is easy to both solder and unsolder, the risk of damage is minimal. Conveniently, length of the 1206 is the same as the distance between the pins you need to connect on the EEPROM.

Melt a tiny amount (and I do mean tiny amount) of solder on the ends of the resistor first. Make sure it is just on the ends and hasn't spilled toward the middle, you don't want to short the pin inbetween. Then just place the resistor so it's connecting the pins in question, and press down on each end of it for 10-15 seconds with a soldering iron.

[mrkrad]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

[gordan]

I'm no better, I'm afraid. I have given up on soldering anything smaller than 1206 size manually.

[mosmo]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

[gordan]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

Or go one better and get a hot air reflow iron. A decent one of those costs more than a Titan, though.

One other possible alternative is to get conductive glue.

But none of those preclude the requirement for very, very steady hands; a good night's sleep and no caffeine for 24 hours before attempting makes a noticeable difference.

[mosmo]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

Or go one better and get a hot air reflow iron. A decent one of those costs more than a Titan, though.

One other possible alternative is to get conductive glue.

But none of those preclude the requirement for very, very steady hands; a good night's sleep and no caffeine for 24 hours before attempting makes a noticeable difference.

+1 to conductive glue.

[makblast]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

Or go one better and get a hot air reflow iron. A decent one of those costs more than a Titan, though.

One other possible alternative is to get conductive glue.

But none of those preclude the requirement for very, very steady hands; a good night's sleep and no caffeine for 24 hours before attempting makes a noticeable difference.

Hi
Very interesting, my 780ti is arriving tomorrow :-)
Have you looked at the nvidia-smi output? Especially the "compute mode =" and whether it can be changed e.g with -c "All On" option :-)

[gordan]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

Or go one better and get a hot air reflow iron. A decent one of those costs more than a Titan, though.

One other possible alternative is to get conductive glue.

But none of those preclude the requirement for very, very steady hands; a good night's sleep and no caffeine for 24 hours before attempting makes a noticeable difference.

Hi
Very interesting, my 780ti is arriving tomorrow :-)
Have you looked at the nvidia-smi output? Especially the "compute mode =" and whether it can be changed e.g with -c "All On" option :-)

I run on XP64, and for some reason the XP64 driver bundle doesn't include the nvidia-smi utility.

[mosmo]

Just a little OT, but does anyone know a reliable way of modifying the BIOS on a GTX card so that the fan starts at 0 RPM?
I am building an HTPC that will run 24/7 and to prolong the life of the fan and keep the HTPC quiet under idle, I only want the fan to kick in when temps reach 40-45 degrees.
From researching other forums, it seems that the fan speed profiles are hard coded to the cards and keep the majority from going below 10% fan speeds. I tried kepler bios editor and set the minimum to 0% but my GTX 670 still runs at a minimum 10% fan speed.

[Luke212]

if its too hard to make a Titan a K20X, is it possible to make Titan a K40? K40 is the new 12GB 2880 shader Telsa. If the K40 4 bit Id is closer maybe it will be an easier mod?

[gordan]

If the 4th nibble is the same as a 780, 780Ti or Titan, then yes. Otherwise, no, not until the location of the 4th nibble strap is confirmed.
Currently, that means only 780ti -> K6000 is confirmed. I see no good reason to mod to a Tesla instead of a K6000, since Quadro feature set is a superset of the Tesla feature set.
Before asking questions like this, look through this:
ftp://download.nvidia.com/XFree86/Linux-x86/319.49/README/supportedchips.html
and find whether nibbles 1,2,4 match. If they do and the cards are all GK110 based, then the current method will work. Otherwise it won't.

[baconsteak]

It would be nice if you would offer a service of modding the cards Some of us have unsteady hands. Only good for nvflash strapping ..

the best avice I can give is not to skimp on soldering iron. Get a good one with pointed tips so you can apply the tiniest of solder amounts.

Or go one better and get a hot air reflow iron. A decent one of those costs more than a Titan, though.

One other possible alternative is to get conductive glue.

But none of those preclude the requirement for very, very steady hands; a good night's sleep and no caffeine for 24 hours before attempting makes a noticeable difference.

I could easily solder anyone’s card if they were nearby me.

Those ultra pointed tips are a pain imo. I found that they loose heat when you touch the component and you have to have the heat turned up more. When its just sitting there it starts to oxidize unless you turn it back down. They just aren't a big enough heat sink and the temp control is all out of whack.

I use a hakko 936 with the 900M-T-B tip that came with it. I think they are around the $80 mark now. When I was working I preferred the chisel type D tips but they are easy enough with either. I used to do 0603s at work.

The key really is flux if you don't get it soldered right away before the flux in the solder evaporates. Get some no clean flux in a syringe tube and some fine tweezers.

My arms or hands shake a fair bit. I just hold them hard against the desk while I'm soldering.

[beckend]

Just wanted to let you know, I succeeded modding Palit Jetstream 4GB 680 GTX into the Tesla K10. Used the sharp end of a razorknife and removed the 4th resistor the dirty way.