Advanced search

Message boards : Number crunching : Running more than two GPUGRID WU's with dual GTX 590's...

Author Message
Profile WirelessDude
Send message
Joined: 3 Aug 11
Posts: 21
Credit: 189,614,059
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwat
Message 21951 - Posted: 4 Sep 2011 | 7:03:02 UTC

Is anybody having problems running more than two GPUGRID WU's (whether they are the long or short runs and the combinations thereof) with dual GTX 590's? It seems when the third WU kicks in, the client comes to a very slow crawl. Combinations of thw two GPUGRID WU's and other project WU's seem to work fine, as well as up to four non-GPUGRID WU's crunching at the same time (as I have been for months before adding the GPUGRID project...)

The two GPUGRID WU's work fine whether they are running on one GTX 590 or split between the two GTX 590's. But, again, when that third GPUGRID WU kicks in, the client crawls to almost a frozen state. Closing the client returns the OS to normal operation. Restarting the client presents the same results...

Profile Carlesa25
Avatar
Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21953 - Posted: 4 Sep 2011 | 10:48:38 UTC - in response to Message 21951.
Last modified: 4 Sep 2011 | 11:11:59 UTC

Hi: Unfortunately I can not give answer to your problem.

What I can assure you that the performance of the GTX590 is very low, pay less than my GTX295.

I have been comparing their execution times and are even higher with my GTX295 ... incomprehensible.

You may have a problem with the ports PCIe (x16 or x8)

Let us also mention that the hyperthreading is not suitable for these compute-intensive tasks. Greetings.

Profile WirelessDude
Send message
Joined: 3 Aug 11
Posts: 21
Credit: 189,614,059
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwat
Message 21957 - Posted: 4 Sep 2011 | 19:26:22 UTC - in response to Message 21953.
Last modified: 4 Sep 2011 | 19:26:52 UTC

Thanks much for the hyperthreading advice! It seems GPUGRID did not like it...

I am now running GPUGRID full blast on my GPU's without any issues.

A shame about the performance differences between the GTX 590 and GTX 295. It's one of those give and take situations, I guess! ;o)

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 2343
Credit: 16,249,865,968
RAC: 4,089,892
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21958 - Posted: 4 Sep 2011 | 21:51:50 UTC - in response to Message 21953.

What I can assure you that the performance of the GTX590 is very low, pay less than my GTX295.

I have been comparing their execution times and are even higher with my GTX295 ... incomprehensible.

You messed up this comparsion.
Your GTX 295: I44R2-GIANNI_KKFREE5-22-100-RND0060_1 GPU:55843s CPU:8542s
My _ GTX 590: I199R0-GIANNI_KKFREE5-26-100-RND0232_0 GPU:29524s CPU:4959s
Your GTX 295: s0r391-TONI_SH2MS3-30-100-RND2495_1 GPU:37524s CPU:4471s
My _ GTX 590: s0r444-TONI_SH2MS3-30-100-RND3830_0 GPU:18488s CPU:2803s
Your GTX 295: p27-IBUCH_3_wtEGF_110726-4-10-RND1225_1 GPU:67528s CPU:9134s
My _ GTX 590: p2-IBUCH_2_wtEGF_110726-7-10-RND0986_0 GPU:34546s CPU:4621s

Let us also mention that the hyperthreading is not suitable for these compute-intensive tasks.

This is almost true. There is only one FPU (Floating Point Unit) in every HyperThreaded CPU core, and the running applications should not overload those FPUs for optimum performance. You should free some HT cores (the actual number depends on how much FPU the running applications are using), but turning HT completely off could be a waste of CPU resources. Right now, I'm running 6 CPU tasks in the same time with 3 GPU tasks.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 2343
Credit: 16,249,865,968
RAC: 4,089,892
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21959 - Posted: 4 Sep 2011 | 21:56:17 UTC - in response to Message 21951.

Is anybody having problems running more than two GPUGRID WU's (whether they are the long or short runs and the combinations thereof) with dual GTX 590's? It seems when the third WU kicks in, the client comes to a very slow crawl.

I have a GTX 590 and a GTX 580 in this host, and I'm running 3 GPU tasks in parallel without problems for months.

Profile WirelessDude
Send message
Joined: 3 Aug 11
Posts: 21
Credit: 189,614,059
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwat
Message 21960 - Posted: 4 Sep 2011 | 22:12:13 UTC - in response to Message 21959.
Last modified: 4 Sep 2011 | 22:15:40 UTC

It seems turning hyperthreading off is much more forgiving, but the issue still crops up...

After turning off system overclock settings and turning hyperthreading back on, the problem goes away completely! I've been running all other projects with the OC'd settings for months (pre-GPUGRID). I guess I have some more tweaking to do...

By the way, I was running dual X5650's (2.67GHz) Xeons @3.6GHz on an eVGA SR-2 Series 5520 motherboard.

Thanks for all the GPU stats and feedback, folks!

Profile Carlesa25
Avatar
Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21961 - Posted: 5 Sep 2011 | 0:06:37 UTC - in response to Message 21958.


You messed up this comparsion.
Your GTX 295: I44R2-GIANNI_KKFREE5-22-100-RND0060_1 GPU:55843s CPU:8542s
My _ GTX 590: I199R0-GIANNI_KKFREE5-26-100-RND0232_0 GPU:29524s CPU:4959s... etc...


Hello: I guess I made myself worse, compraciĆ³n was my GTX295 tasks and tasks of WirelessDude.

Obviously a GTX590 in normal yields 50% more than a GTX295 and will be my next change.. Greetings.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 2343
Credit: 16,249,865,968
RAC: 4,089,892
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21966 - Posted: 5 Sep 2011 | 17:41:52 UTC - in response to Message 21960.
Last modified: 5 Sep 2011 | 17:44:23 UTC

It seems turning hyperthreading off is much more forgiving, but the issue still crops up...

After turning off system overclock settings and turning hyperthreading back on, the problem goes away completely!

These errors are usually caused by overclocking, overheating, insufficient PSU, interfering applications, aging GPU.

I've been running all other projects with the OC'd settings for months (pre-GPUGRID). I guess I have some more tweaking to do...

GPUGrid is tolerating less GPU overclocking than other GPU projects. Even different kind of GPUGrid workunits are tolerate different level of GPU overclocking (the more GPU usage they run at, the less overclocking they can run with.)

By the way, I was running dual X5650's (2.67GHz) Xeons @3.6GHz on an eVGA SR-2 Series 5520 motherboard.

For a more comprehensive insight into your problems, you should share with us your GPU clock and voltage settings, PSU type, CPU overclocking settings in detail.

Thanks for all the GPU stats and feedback, folks!

You're welcome!

Profile Zarck
Send message
Joined: 16 Aug 08
Posts: 145
Credit: 328,473,995
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21967 - Posted: 5 Sep 2011 | 23:27:08 UTC - in response to Message 21951.
Last modified: 5 Sep 2011 | 23:27:42 UTC

Test Folding with you 590,

https://fah-web.stanford.edu/projects/FAHClient/wiki/BetaRelease

@+
*_*
____________

Profile WirelessDude
Send message
Joined: 3 Aug 11
Posts: 21
Credit: 189,614,059
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwat
Message 21968 - Posted: 6 Sep 2011 | 2:11:55 UTC - in response to Message 21967.
Last modified: 6 Sep 2011 | 2:47:44 UTC

Well... I reverted back a few BIOS revisions after chatting with fellow SR-2 builders. It seems BIOS A46 (June 2010) is the magic release for my use. The OC'ing seems stable at 3.6GHz...

Also, I was clued-in on SLI mode being a culprit as well after I installed a 5th GPU into the system (latest graphics driver allows for up to a 4-way SLI + an extra GPU for PhysX). When the 4 GPU's were having problems, the lone PhysX GPU was happily crunching away. I turned off the SLI mode and now all 5 GPU's are running fine for the time-being. It seemed when the CPU's are relatively busy, even at 85%, the GPU's had a hard time starting the GPUGRID WU's. Now, I have everything running 100% with no problems so far with any of the WU's...

Here's hoping that the issue has been resolved!

Thanks, again, everyone!

Post to thread

Message boards : Number crunching : Running more than two GPUGRID WU's with dual GTX 590's...

//