cancel
Showing results for 
Search instead for 
Did you mean: 

Dedicated PhysX configuration

Korth
Level 14
My X99 rig currently has:
two EVGA GTX980 04G-P4-3988-KR cards (GM204-400-A1 GPU @1291MHz/1393MHz, 4GB 256-bit GDDR5 @7010MHz, 263W)
one NVIDIA Tesla K80 GPU Accelerator card (dual GK210 GPUs @560MHz/865MHz, 24GB 384-bit GDDR5 @5000MHz, 300W)

This rig is primarily intended for (and is actually used for) constant engineering work - I run tons of CAD/CAE softwares, AutoDesk, Altair, ANSYS, JMAG, Keysight, some SPICEs and MATLABs, some 3D sim and visualization stuff. I don't actually own the Tesla card (it costs over $6k and my employer leases it from NVidia) but I don't just let it sit idle while gaming! It's a mighty beast but it doesn't have any display outputs, so when I'm gaming it gets to be a dedicated overkill PhysX card while my x16/x16 GTX980s do all the heavy lifting.

I currently have the GTX980 SLI installed in the (first) PCIE_X16_1 and (third) PCIE_X16_2 slots, Tesla card installed between them in the (second) PCIE_X8_2 slot, my employer's Comay E28 BladeDrive (a PCIe 2.0x8 device) below in the (fourth) PCIE_X8_4 slot. My E5-1680-3 processor is basically a Xeon-binned i7-5960X and supports 40 PCIe 3.0 lanes. I have no M.2 or SATAe or USB3 devices installed which could cause PCIe conflicts on this motherboard. My 1250W Platinum PSU is quite capable of sustaining the entire system (including all three GPU cards) at peak load, no issues. Win7x64SP1 (6.1) with DirectX 11 (6.01.7601.17514), NVIDIA's GeForce Game Ready Driver (WHQL361.43), NVIDIA PhysX System Software (9.15.0428) with a GK2xx GPU emulation hack to run x8 dedicated PhysX on a Tesla.

The PhysX works great, my raw fps hasn't increased but it never (never) dips or wavers when things get crunchy.
The Tesla card actually runs cool all the time, that Enterprise-grade passive cooler never exceeds ~55C under maximal GPU load. The EVGA cards run hot, those fancy EVGA ACX2.0 coolers (plus backplates) struggle to keep temps below 65C when working hard. The top GTX980 is the harder worker but has proximity to the chassis rear exhaust fan, the bottom GTX980 consistently runs about 10C hotter because the slots are crowded. The BladeDrive never breaks a sweat, the heatsink is rather plain (not as exciting for gamers as the one G.Skill put on their rebranded Comay G24 BladeDrives) but impressively cool and efficient. My chassis has six 140mm fans (2 front intake, 1 side intake, 1 rear exhaust, 2 top exhaust), it could accommodate more fans and lots of rad.

Any recommendations about how to better configure or optimize this hardware and PhysX setup? Mostly looking for ways to bring GPU temps down without throttling performance, without installing custom liquid cooling, and without doing anything at all which could void warranty on the Enterprise hardware (which I don't own). Any sort of decent AIO/CLC coolers for GTX980 cards?

(Please don't advise upgrading the SLI to a pair of GTX980Ti or TitanX GPUs, they won't help much for work and I simply can't afford to dump such money into games right now, lol. And AMD cards do not support all the software I run, the proprietary NVIDIA tech is too deeply entrenched.)
"All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others." - Douglas Adams

[/Korth]
10,040 Views
10 REPLIES 10

Nate152
Moderator
Hello Korth, you seem like a pretty smart guy and know your way around a pc pretty good.

I don't own the 980 so I'm not sure exactly what temp they start to throttle but 65c isn't too hot and they shouldn't start throttling until they get around 80c.

If you want them to run cooler than 65c (which in my opinion is perfectly fine) there is an AIO hybrid cooler for the evga gtx 980 but I don't know how well it would fit with the other pcie slots populated.

http://www.amazon.com/EVGA-Hybrid-GeForce-Cooling-400-HY-H980-B1/dp/B00V9BX1GO

Kudos to you for getting PhysX to work on the tesla card but as powerful as the 980's are I doubt using the tesla card as a PhysX card get gets you any performance gain. Without a dedicated physX card in an sli configuration physX should be placed on the 2nd gpu, I'm not sure you can optimize it other than placing PhysX on the cpu which probably won't get you a performance gain and in some situations may even hurt performance.

To me your gpu temps are fine and doesn't need the AIO cooling but if you want to go that route the option is there.

Korth
Level 14
Worst (peak) temps measured on my last 24-hour Prime95 run were

(ambient 34C)
CPU 73C
R5E 58C
1st GTX980 = 81C
Tesla = 56C
2nd GTX980 = 93C (some throttling)
BladeDrive = 47C
RAID = 57C

Normal temps during normal use aren't that high, of course, even when gaming. But temps on my EVGA cards are much worse now than a year ago - they get hotter quicker under lighter loads, their temps are never less than 65C and often exceed 75C when gaming, they now constantly run the (loud) fans at 100% and their temps are steadily climbing each month - it just seems like it might be time for a repaste or a cooling upgrade. The EVGA ACX2.0 cooling solution is utter garbage (hot and loud) when compared to ye olde reference NVTTM design, lol. My ambient environment is warm, and there's nothing I can do about that, which doesn't help.

Getting PhysX to work on a Tesla wasn't too hard. Once you can search past all the internet criticisms and find actual working driver code, lol.
"All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others." - Douglas Adams

[/Korth]

NemesisChild
Level 12
I agree the ACX cooling is terrible, just can't see circulating hot GPU air within my case.

This why all of me GPU's have the reference design/coolers.
I tried 780Ti's in SLI, both with the ACX coolers, it was a like an oven inside my case.
And I tried every imaginable case fan configuration to try to reduce or alleviate heat build up.
Intel i9 10850K@ 5.3GHz
ASUS ROG Strix Z490-E
Corsair H115i Pro XT
G.Skill TridentZ@ 3600MHz CL14 2x16GB
EVGA RTX 3090 Ti FWT3 Ultra
OS: WD Black SN850 1TB NVMe M.2
Storage: WD Blue SN550 2TB NVMe M.2
EVGA SuperNova 1200 P2
ASUS ROG Strix Helios GX601

Korth
Level 14
It looks like I might need to install waterblocks after all. Or upgrade my "gaming" GPUs. 😞

Hoping to avoid liquid cooling in this rig. I've done loops before and I'm not really paranoid about leakage - but I am running over $10k worth of hardware which doesn't belong to me (along with almost $5k that does) so I just can't risk it, lol. My experience is that liquid cooling isn't any cooler or quieter anyhow once the rad gets saturated with heat it can't bleed off, so I suspect watercooling would not do any better in my rig unless I could bring down background temps or string the rads outside to a cooler environment.

I suppose I've got nothing to lose by trying out a PCMA-based TIM but Indigo Xtreme is not available for my whole-GPU application (and it's stupidly overpriced anyhow). I'll probably go with trusty old Indalloy-4 Thermal Interface Solder, we've got spools of it sitting around the hot air stations, haha.
"All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others." - Douglas Adams

[/Korth]

xeromist
Moderator
You might try moving PhysX to the SLI cards. As Nate mentioned you already have a good deal of horsepower and letting the Tesla idle might mean less heat overall. I know it's tempting to want to use awesome hardware since it's sitting there but it might not be your most optimal configuration.

Alternatively, there's the old trick of taking the side panel off of the case and pointing a box fan at it. If you find temps go down with this arrangement you could explore adding intake fans to the side panel to blow on the cards. Or you could try one of those dedicated slot coolers. They're a cheap mistake if they don't help and don't require any modding of the case.
A bus station is where a bus stops. A train station is where a train stops. On my desk, I have a work station…

Korth
Level 14
Well I used to have the pair of EVGA Classified GTX980 cards (04G-P4-3988-KR) in x16/x16 SLI. Each running a 1291MHz/1393MHz GM204 GPU and 4GB of 256-bit GDDR5 at 7010MHz. I paid nearly $900 each for them back in Feb 2015, lol, +100% reference price to gain +15% reference clock. GPU tech back then looked (to me) like it was on a plateau: 28nm TSMC had already been around for 2-3 years, I foolishly predicted it would remain for 2-3 more. I was unhappy when Titan X released three weeks later. Unhappy again when 980Ti released three months later. I'd bought these cards for gaming pwnage (duh!) but also for helping run productivity and GPGPU, I knew they'd go obsolete but not in just one season! The GTX1000 family (alongside a couple slews of AMD offerings) just passed by unnoticed while I moped.

We all know "gamer" GPU cards lack DPFP/FP64 so they generally suck at hard GPGPU, yet I was happy I could sustain a modest 330 GFLOPS - the equivalent of a dozen PlayStation 4s! My boss wasn't at all as happy so he issued me a company (leased) NVIDIA Tesla K80 GPU Accelerator card (699-22080-0200-500). Two 562MHz/875MHz GK210 GPUs, each with 12GB of 384-bit GDDR5 at 2500MHz. This beast could crunch over 2900 GFLOPS (alongside my existing 330 GFLOPS), which was a dramatic productivity increase. Not at all very useful for gaming - doesn't have even have any display outputs, haha - but I wasn't about to let a $5000 piece of hardware sit idle when gaming so (with some help) I eventually managed to cobble up a GK2xx emulation hack to make it serve (unwillingly but obediently) as a x8 PhysX card under DX11. Ugly and cantankerous, and a ridiculously overkill luxury of barely noticeable merit, but still ... it was nice to have.

Well, the lease expired and they took away "my" Tesla, lol. A fine excuse to replace those yucky old GTX980 cards with a shiny new pair of reference Titan Xp cards (900-1G611-2500-000) plus a fancy new reference SLI-HB bridge! Ubergaming fps dominance is mine again! I even want to upgrade to a "faster" monitor, lol. Alas, my FP64 crunch is now around "only" 750 GFLOPS. Expecting (hoping) the boss issues me a new GPU-A card, and hoping that it's gonna be Pascal based so forced PhysX will run that much better and smoother.

Air cooling my system has become a easier now that there's empty space where that monstrous Tesla used to sit. It just happened to be located in a very awkwardly inconvenient place. My temps are all still good but I notice the overall fan noise is a bit quieter.

I've gone wireless - long story, but no wired network until the 10G fiber gets installed - and I found, interestingly, that the ASUS 3T3R dual band WiFi accessory provided with my ASUS X99 mobo isn't as magnificent as promised. The ASUS 3T3R gizmo is really neat (and comes "free" with ASUS mobo) and it lets you position your antennas where desired (usually high up to maximize signal and minimize ugly) - but the antennas themselves are just mediocre and it turns out there's signal latency between them and that long wire that leads to your computer (where they actually get processed). My new TP-Link Archer T9E card uses the same Broadcom chip, the same (albeit rebranded) drivers, and the same spec AC1900 - but it actually performs as promised, consistently achieving and sustaining fastest "theoretical" speeds at each band - because while the antennas all stick out of the back of your computer, and it fills up a PCIe slot (usually that tiny one above or between the GPU cards, lol), it is able to process the signal "instantly" for excellent beam-forming (meaning that it finds and holds the "best" 1300Mbps/600Mbps signal shortcuts to and from your WiFi router, directly through ten walls and three rooms and two floors and every obstacle within). The black PCB and black heatsink and black backplate are all just gravy.

Spent a little money on external audio stuff, lol, perhaps not quite as leet as dedicated audio cards but still sounds awesome to my ears and minimizes clutter inside the box.
"All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others." - Douglas Adams

[/Korth]

xeromist
Moderator
I use optical out with separate components as well. I've never felt like audio quality was lacking. I think the only thing that differentiates some setups is people who have surround headphones. Positional audio can be helpful for FPS games.
A bus station is where a bus stops. A train station is where a train stops. On my desk, I have a work station…

Korth
Level 14
Even stereo positional audio (through headphones) is useful. But I think a decent (5.1 or better) surround-sound setup is where it really shines. I've been unimpressed by the few surround-sound headphones/headsets I've tried (even when they mount multiple discrete speakers within each ear can) - although I admit my old and industrial-damaged hearing ain't quite what it used to be. FWIW - and I expect true audiophiles would disagree - the consensus from every decent audio card I've ever heard and everybody I've met who uses them and every review/article I've read is that people can't really hear any real difference in audio quality once you exceed CD-quality (16-bit 44.1KHz) anyhow, higher specs are more about brag and swag and marketing than meaningful audio quality. Especially since the majority of digital sound sources (ripped from CD, downloaded, or within games) is recorded/encoded in CD-quality.

I might throw in a dedicated audio card eventually. Mostly so I can tinker and hack it into an improvised oscilloscope, lol. The highest-end offerings (from Asus and Creative) seem a little confusing at a glance - and I'd have to spend time researching and comparing them (and all their components) - and even the ROG-integrated audio seems to offer better fidelity than my ears can discern. Quite happy with how much oomphf my little audio box and middly 2.1 speakers can put out. I'd also considered trying out a Fiio Olympus 2-E10K, but my O2+ODAC is such a fine little beast (and can be moved so easily from machine to machine) that further comparison (and spending) seems pointless.
"All opinions are not equal. Some are a very great deal more robust, sophisticated and well supported in logic and argument than others." - Douglas Adams

[/Korth]

was the heat problem solved ?

Am guessing the top GFX card was overheating ?
Am running SLI as well and the top card is always running 10 to even 15C hotter.
My bottom card is running very cool.. cooler than normal..
This just because of specing the heat fro the bottom card is sucked in to the card on top.

if you dont care bout how your computer looks or you dont have side pannel. you can please a 120mm fan on the backplate of the top card. The fan should NOT blow on the back plate but suck.. ( euh you get my point) no need crew down the fan or things like that. if have a fan with some rubber feet it will just stay on its spot anyways. lowest RMP is okay you dont want change whole air flow in your case.
Try more, Try harder, Try and previal