Advanced search

Message boards : Number crunching : Problem with not getting WU for second card

Author Message
TheFiend
Send message
Joined: 26 Aug 11
Posts: 99
Credit: 2,500,123,465
RAC: 14
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32357 - Posted: 28 Aug 2013 | 9:48:39 UTC

Running dualGTX670's and all of a sudden yesterday it stopped downloading WU's to run on my second card.


28/08/2013 10:29:45 | | Starting BOINC client version 7.0.64 for windows_x86_64
28/08/2013 10:29:45 | | log flags: file_xfer, sched_ops, task
28/08/2013 10:29:45 | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6
28/08/2013 10:29:45 | | Data directory: D:\BoincApplication Data
28/08/2013 10:29:45 | | Running under account Administrator
28/08/2013 10:29:45 | | Processor: 6 AuthenticAMD AMD Phenom(tm) II X6 1090T Processor [Family 16 Model 10 Stepping 0]
28/08/2013 10:29:45 | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 htt pni cx16 popcnt syscall nx lm svm sse4a osvw ibs skinit wdt page1gb rdtscp 3dnowext 3dnow
28/08/2013 10:29:45 | | OS: Microsoft Windows XP: Professional x64 Edition, Service Pack 2, (05.02.3790.00)
28/08/2013 10:29:45 | | Memory: 8.00 GB physical, 9.58 GB virtual
28/08/2013 10:29:45 | | Disk: 465.76 GB total, 464.58 GB free
28/08/2013 10:29:45 | | Local time is UTC +1 hours
28/08/2013 10:29:45 | | CUDA: NVIDIA GPU 0: GeForce GTX 670 (driver version 310.90, CUDA version 5.0, compute capability 3.0, 2048MB, 1992MB available, 2634 GFLOPS peak)
28/08/2013 10:29:45 | | CUDA: NVIDIA GPU 1: GeForce GTX 670 (driver version 310.90, CUDA version 5.0, compute capability 3.0, 2048MB, 1990MB available, 2634 GFLOPS peak)
28/08/2013 10:29:45 | | OpenCL: NVIDIA GPU 0: GeForce GTX 670 (driver version 310.90, device version OpenCL 1.1 CUDA, 2048MB, 1992MB available, 2634 GFLOPS peak)
28/08/2013 10:29:45 | | OpenCL: NVIDIA GPU 1: GeForce GTX 670 (driver version 310.90, device version OpenCL 1.1 CUDA, 2048MB, 1990MB available, 2634 GFLOPS peak)
28/08/2013 10:29:45 | | Config: report completed tasks immediately
28/08/2013 10:29:45 | | Config: use all coprocessors
28/08/2013 10:29:45 | GPUGRID | URL http://www.gpugrid.net/; Computer ID 146390; resource share 500
28/08/2013 10:29:45 | GPUGRID | General prefs: from GPUGRID (last modified 23-Aug-2012 23:52:31)
28/08/2013 10:29:45 | GPUGRID | Host location: none
28/08/2013 10:29:45 | GPUGRID | General prefs: using your defaults
28/08/2013 10:29:45 | | Reading preferences override file
28/08/2013 10:29:45 | | Preferences:
28/08/2013 10:29:45 | | max memory usage when active: 6142.37MB
28/08/2013 10:29:45 | | max memory usage when idle: 7370.84MB
28/08/2013 10:29:45 | | max disk usage: 100.00GB
28/08/2013 10:29:45 | | max CPUs used: 5
28/08/2013 10:29:45 | | max download rate: 499999 bytes/sec
28/08/2013 10:29:45 | | max upload rate: 499999 bytes/sec
28/08/2013 10:29:45 | | (to change preferences, visit a project web site or select Preferences in the Manager)
28/08/2013 10:29:45 | | Not using a proxy
28/08/2013 10:29:46 | GPUGRID | Restarting task I39R1-NATHAN_KIDKIXc22_6-4-50-RND9237_1 using acemdlong version 800 (cuda42) in slot 4
28/08/2013 10:29:46 | GPUGRID | Sending scheduler request: To fetch work.
28/08/2013 10:29:46 | GPUGRID | Requesting new tasks for NVIDIA
28/08/2013 10:29:47 | GPUGRID | Scheduler request completed: got 0 new tasks
28/08/2013 10:29:47 | GPUGRID | NVIDIA GPU: Upgrade to the latest driver to use all of this project's GPU applications
28/08/2013 10:29:47 | GPUGRID | No tasks sent
28/08/2013 10:29:47 | GPUGRID | No tasks are available for Long runs (8-12 hours on fastest card)
28/08/2013 10:31:06 | GPUGRID | update requested by user
28/08/2013 10:31:08 | GPUGRID | Sending scheduler request: Requested by user.
28/08/2013 10:31:08 | GPUGRID | Requesting new tasks for NVIDIA
28/08/2013 10:31:09 | GPUGRID | Scheduler request completed: got 0 new tasks
28/08/2013 10:31:09 | GPUGRID | NVIDIA GPU: Upgrade to the latest driver to use all of this project's GPU applications
28/08/2013 10:31:09 | GPUGRID | No tasks sent
28/08/2013 10:31:09 | GPUGRID | No tasks are available for Long runs (8-12 hours on fastest card)


I've never come across the phrase " NVIDIA GPU: Upgrade to the latest driver to use all of this project's GPU applications" before.

It is seeing both cards when BOINC starts and both cards are enabled in CC_CONFIG

Running 310.90 but have also tried 320.49 and it makes no difference - it still comes back with the phrase - NVIDIA GPU:Upgrade to the......... and won't download a second WU!!!

HELP!!!

TJ
Send message
Joined: 26 Jun 09
Posts: 815
Credit: 1,470,385,294
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32362 - Posted: 28 Aug 2013 | 10:22:29 UTC - in response to Message 32357.
Last modified: 28 Aug 2013 | 10:25:16 UTC

I think this has to do with the queue's almost empty. There are only a few tasks available at the moment. The one's that are using the latest app, ACEMD 8.00 need one of the latest beta driver. The project checks your driver and will send cuda 4.2 or cuda 5.5 to your PC.

You can install the latest beta driver, 326.80. This will give you cuda5.5 tasks, if available.

The have the highest change for work, you could set to accept "all applications", if you haven't done this already.
____________
Greetings from TJ

TheFiend
Send message
Joined: 26 Aug 11
Posts: 99
Credit: 2,500,123,465
RAC: 14
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32364 - Posted: 28 Aug 2013 | 10:50:41 UTC

Have updated to the Beta driver now and that message has now disappeared.

Funny thing is my other rig has not suffered from that same message and is quite happily running 2 WU's and has had no problem getting any....

TJ
Send message
Joined: 26 Jun 09
Posts: 815
Credit: 1,470,385,294
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32366 - Posted: 28 Aug 2013 | 11:24:09 UTC - in response to Message 32364.

Have updated to the Beta driver now and that message has now disappeared.

Funny thing is my other rig has not suffered from that same message and is quite happily running 2 WU's and has had no problem getting any....


I see that your other rigs have only cuda42 WU´s (that is due to the driver (lower than 315)). It is possible that at the moment your other rig requested work, these cuda42 tasks where all gone and only cuda55 available. But you didn´t got those due to the "wrong" driver.
Nice to see that all is working again.
____________
Greetings from TJ

TheFiend
Send message
Joined: 26 Aug 11
Posts: 99
Credit: 2,500,123,465
RAC: 14
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32367 - Posted: 28 Aug 2013 | 11:30:47 UTC

LOL... just checked and my other rig has just started showing update driver error.

Updating that one now as well


TheFiend
Send message
Joined: 26 Aug 11
Posts: 99
Credit: 2,500,123,465
RAC: 14
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32368 - Posted: 28 Aug 2013 | 11:33:30 UTC

The GTX670 rig has finally got a 2nd WU- a CUDA55 one!!!

TJ
Send message
Joined: 26 Jun 09
Posts: 815
Credit: 1,470,385,294
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32369 - Posted: 28 Aug 2013 | 11:42:10 UTC - in response to Message 32368.

That´s what we want :)
____________
Greetings from TJ

Bjarke
Send message
Joined: 1 Mar 09
Posts: 8
Credit: 74,871,366
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32383 - Posted: 28 Aug 2013 | 14:34:10 UTC

I have the same problem with my Quadro 4000 despite having the latest driver (as of today) version 320.78 including CUDA 5.5

28/08/2013 16:22:39 | | CUDA: NVIDIA GPU 0: Quadro 4000 (driver version 320.78, CUDA version 5.50, compute capability 2.0, 2048MB, 1960MB available, 486 GFLOPS peak)
28/08/2013 16:22:39 | | OpenCL: NVIDIA GPU 0: Quadro 4000 (driver version 320.78, device version OpenCL 1.1 CUDA, 2048MB, 1960MB available, 486 GFLOPS peak)


28/08/2013 16:26:48 | GPUGRID | Sending scheduler request: To fetch work.
28/08/2013 16:26:48 | GPUGRID | Requesting new tasks for NVIDIA
28/08/2013 16:26:49 | GPUGRID | Scheduler request completed: got 0 new tasks
28/08/2013 16:26:49 | GPUGRID | NVIDIA GPU: Upgrade to the latest driver to use all of this project's GPU applications
28/08/2013 16:26:49 | GPUGRID | No tasks sent
28/08/2013 16:26:49 | GPUGRID | No tasks are available for Long runs (8-12 hours on fastest card)


At the time of WU request 527 unset WU's are available according to the server status.

My host: http://www.gpugrid.net/show_host_detail.php?hostid=134464

TJ
Send message
Joined: 26 Jun 09
Posts: 815
Credit: 1,470,385,294
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32386 - Posted: 28 Aug 2013 | 15:28:44 UTC - in response to Message 32383.

It could be "bad luck" and that all WU´s are gone. The server page is not real time, it is a "picture" refreshed every hour, half an hour? The admins will know.

Or it has to do with the compute capability. It is 2.0 for your card and that it must be 3.0 or higher for this type of WU. I don´t know exactly, I thought compute capability of 1.3 would be enough. Someone else has to explain this to you (and me).
____________
Greetings from TJ

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32411 - Posted: 28 Aug 2013 | 19:33:42 UTC

It's not the compute capability. It seems like the WUs for CUDA 4.2 are a bit scarce by now and that the CUDA 5.5 app refuses to work with anything (significantly?) lower than 326.80. there may be good reasons for this, I don't know.

Well, Quadro drivers usually take their sweet time until release due to the additional veryfications. If you don't need the Quadro features you might be able to install the regular driver as a temporary work-around, though I don't know if the driver will allow this.

MrS
____________
Scanning for our furry friends since Jan 2002

TJ
Send message
Joined: 26 Jun 09
Posts: 815
Credit: 1,470,385,294
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32420 - Posted: 28 Aug 2013 | 21:04:27 UTC - in response to Message 32411.

Aha thanks. Will you please explain the compute capability then? I know that a card with compute capability of 1.0 is no use for cuda.
____________
Greetings from TJ

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32422 - Posted: 28 Aug 2013 | 21:38:05 UTC - in response to Message 32420.

I'm not keen on recommending a beta driver to everyone - for Titan/GTX780 GPU's yes, but only because everything before didn't work. For non GK110 cards, is there any benefit from the new 5.5 app and what's the risks of using a Beta driver?
____________
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

klepel
Send message
Joined: 23 Dec 09
Posts: 189
Credit: 4,721,425,539
RAC: 1,808,309
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32429 - Posted: 28 Aug 2013 | 23:45:06 UTC
Last modified: 28 Aug 2013 | 23:46:43 UTC

Same question here. What is the benefit of changing driver for Cuda 5.5 capability? I have used the Nvidea Driver: 311.6 with Cuda 4.2 for a long time and never had major hick-ups, not even with Noelias. So why should I change? Is it faster?

Bjarke
Send message
Joined: 1 Mar 09
Posts: 8
Credit: 74,871,366
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32439 - Posted: 29 Aug 2013 | 8:26:38 UTC

My Quadro 4000 is now getting work again.

I noticed that i'm only getting CUDA 4.2 applications though my graphic card supports CUDA 5.5.

It seems like the WUs for CUDA 4.2 are a bit scarce by now and that the CUDA 5.5 app refuses to work with anything (significantly?) lower than 326.80. there may be good reasons for this, I don't know.



So I guess this has something to do with either the driver version being only 320.78 for my quadro GPU or perhaps the compute capability of 2.0.

Profile MJH
Project administrator
Project developer
Project scientist
Send message
Joined: 12 Nov 07
Posts: 696
Credit: 27,266,655
RAC: 0
Level
Val
Scientific publications
watwat
Message 32440 - Posted: 29 Aug 2013 | 8:31:12 UTC - in response to Message 32439.


CUDA 5.5 app refuses to work with anything (significantly?) lower than 326.80.


Correct. This is because 326.80 contains an important fix for 780 and Titan cards. Any ealier version is too unstable to use.

MJH

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32493 - Posted: 29 Aug 2013 | 18:55:18 UTC - in response to Message 32420.

Aha thanks. Will you please explain the compute capability then? I know that a card with compute capability of 1.0 is no use for cuda.

My comment was a direct reply to your suggestion "It is 2.0 for your card and that it must be 3.0 or higher for this type of WU". Not supporting Fermis any more (2.0 and 2.1) would be a huge step for the project. That's not something which would be done without proper throught, reason and announcement. As far as I know even CC 1.1 cards technically still work, it's just that they've become too slow and inefficient to make any sense here.

MrS
____________
Scanning for our furry friends since Jan 2002

Profile MJH
Project administrator
Project developer
Project scientist
Send message
Joined: 12 Nov 07
Posts: 696
Credit: 27,266,655
RAC: 0
Level
Val
Scientific publications
watwat
Message 32510 - Posted: 29 Aug 2013 | 21:27:07 UTC - in response to Message 32493.
Last modified: 29 Aug 2013 | 21:35:05 UTC


As far as I know even CC 1.1 cards technically still work,


Nope, we killed off support for those some months ago with the last app. cc1.3 (Geforce 200 series) is the earliest supported now, and you'll probably not be wanting to run one of those unless someone else is paying the power bill..

We have to optimise the program for each hardware/cc generation, and the more different versions we support the more complex the code becomes. Since pre 1.3 cards are all so slow by contemporary standards, discontinuing support for them makes our job easier with only a minimal impact on GPUGRID's throughput.

MJH

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 32549 - Posted: 30 Aug 2013 | 20:59:11 UTC - in response to Message 32510.

cc1.3 (Geforce 200 series) is the earliest supported now, and you'll probably not be wanting to run one of those unless someone else is paying the power bill..

Agreed. Thanks for the clarification!

MrS
____________
Scanning for our furry friends since Jan 2002

Profile MJH
Project administrator
Project developer
Project scientist
Send message
Joined: 12 Nov 07
Posts: 696
Credit: 27,266,655
RAC: 0
Level
Val
Scientific publications
watwat
Message 32552 - Posted: 30 Aug 2013 | 21:03:44 UTC - in response to Message 32549.

I should say 260 and later. Lower end 200s are 1.1. Thanks to a scheduler misconfiguration, the few machines connected with such cards are presently getting WUs (which immediately fail)

Post to thread

Message boards : Number crunching : Problem with not getting WU for second card

//