Message boards : Number crunching : Stealth CUDA?
| Author | Message |
|---|---|
|
Gerry Rough  Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level  Scientific publications    | |
|
I've noticed that my CUDA device is never seen as running on the tasks tab (always "waiting to run"), yet when I look an hour later, the time to completion is one hour less, and the elapsed time is one hour more, as if to say that the 9800 GT is indeed working, but doesn't show up as running. Go figure! | |
ID: 13015 | Rating: 0 | rate:
 /   Reply
Quote
| |
 Paul D. BuckSend message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level  Scientific publications    | |
|
I cannot remember when and which versions started accounting for the GPU time more correctly ... we would have to go back through the notes to find where they made the changes and which version actually started to get it right. | |
| ID: 13017 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1620 Credit: 8,894,687,378 RAC: 19,752,053 Level  Scientific publications | |
I've noticed that my CUDA device is never seen as running on the tasks tab (always "waiting to run"), yet when I look an hour later, the time to completion is one hour less, and the elapsed time is one hour more, as if to say that the 9800 GT is indeed working, but doesn't show up as running. Go figure! That sounds like you have the "Don't use GPU while the computer is in use" option set, and it's working as intended: If you look at the task, it will always be 'waiting to run' - waiting for you to go away! But while you're not looking, it will be running - you could see that with a remote monitoring tool like BoincView, or even a remote copy of BOINC Manager on another machine. | |
| ID: 13019 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Gerry Rough Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level Scientific publications | |
I've noticed that my CUDA device is never seen as running on the tasks tab (always "waiting to run"), yet when I look an hour later, the time to completion is one hour less, and the elapsed time is one hour more, as if to say that the 9800 GT is indeed working, but doesn't show up as running. Go figure! That's what I thought too when I first encountered it. But here are my preferences from GPUGrid: Suspend work while computer is in use? no Suspend GPU work while computer is in use? no And again in my BAM! preferences: Do work while computer is in use? Yes Use GPU while computer is in use? Yes So that idea didn't work out too well. I think Paul's idea makes more sense: since the cpu time is very small, perhaps it shows up as waiting because the cpu is running under the radar, so to speak. BOINC isn't noticing the work being done because the cpu is being used so little. I wondered at the beginning as well if any or all of my BOINC version, driver version, or compute capability were the cause of the angst. Here is a copy and paste from my first post: CUDA device: GeForce 9800 GT (driver version 19038, compute capability 1.1, 512MB, est. 65GFLOPS) BOINC version: 6.6.36 It does seem though that the WUs are running fine and within normal expected times, so it is not as though something is causing an error or the WU is not apparently, if not visually, running normally. ____________  (Click for detailed stats) | |
| ID: 13020 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1620 Credit: 8,894,687,378 RAC: 19,752,053 Level Scientific publications | |
|
Have you looked in your message log? | |
| ID: 13022 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Gerry Rough Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level Scientific publications | |
Have you looked in your message log? I think you must have eaten your Wheaties this morning! Went to the messages tab, and I did find a few 'resuming' messages and no 'preempting' messages. That seems to explain it. Then went to the advanced tab and checked the 'Use GPU while computer is in use' box. WOW!! This BOINC thingy really works like I tell it to! Go Figure! :-) ____________ (Click for detailed stats) | |
| ID: 13023 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Paul D. Buck Send message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level Scientific publications | |
Have you looked in your message log? Just remember, you have now set local preferences on that machine and the web based preferences are not going to affect that machine's operation. I don't know why it did not propagate the preferences correctly though you are using a version, at least for GPU purposes, that is fairly well out of date. As to the run times they are the cumulative of the actual consumption, but not of wall clock time so you could have taken a week to run up those numbers. Again, these are UI and Web issues that folks like Richard have been nagging about for like forever ... anyway, it is good to know you are going again... | |
| ID: 13024 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Paul D. Buck Send message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level Scientific publications | |
Have you looked in your message log? Just a question, does 6.10.11? I know we asked for those messages to be made available .... were they added in the later versions? Or are they still among the missing? | |
| ID: 13025 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1620 Credit: 8,894,687,378 RAC: 19,752,053 Level Scientific publications | |
Have you looked in your message log? Testing with v6.10.10..... 05/10/2009 19:58:47 GPUGRID [cpu_sched] Preempting p845000-IBUCH_005_pYEEI_2909-4-20-RND5912_1 (removed from memory) 05/10/2009 19:59:29 GPUGRID [cpu_sched] Starting p845000-IBUCH_005_pYEEI_2909-4-20-RND5912_1(resume) 05/10/2009 19:59:29 GPUGRID Restarting task p845000-IBUCH_005_pYEEI_2909-4-20-RND5912_1 using acemd version 671 No, looks like it's still debug only for the preempt. And this particular one doesn't obey 'Run according to preferences' - mine are set to 'Run always', and I still got the behaviour quoted. A couple (batteries, exclusive app) were moved from 'always' to 'prefernces' for v6.10.10, but evidently not this. We ought to get to the bottom of this preferences malarky. Gerry, if you wouldn't mind helping with some research (consider it a favour returned), could you: Have a look inside global_prefs.xml (*) - do you see a tag for <run_gpu_if_user_active>? (about six lines down in mine). If so, what is its value (0 or 1)? Then, the same question about global_prefs_override.xml - is the tag there (third line this time), and can you confirm the value is 1? (It should be, as Paul says you've just created the file) I'm rather expecting, unless the Wheaties have worn off, that the tag line will be missing from the main global_prefs.xml file. If so, please could you try changing the setting on the website, then changing it back again, and finally updating the project? I'm wondering if that would cause a missing setting to be added to the file. In your case, it's a bit complicated by your use of BAM - I don't use that system, so you know best which way to change settings - I'll leave the details to you. Finally, once the <run_gpu_if_user_active> tag is properly in the main preferences file(s) and set to 1, could you go into BOINC Manager advanced|Preferences again, and click 'Clear' - that should delete the _override file and restore normal running under BAM/web control, but without stealth mode. [ * all files referenced live in your BOINC Data folder - see second or third line in messages at startup for location ] | |
| ID: 13026 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Gerry Rough Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level Scientific publications | |
We ought to get to the bottom of this preferences malarky. Gerry, if you wouldn't mind helping with some research (consider it a favour returned), could you: I think I see that: <run_if_user_active>1</run_if_user_active> <run_gpu_if_user_active /> Then, the same question about global_prefs_override.xml - is the tag there (third line this time), and can you confirm the value is 1? (It should be, as Paul says you've just created the file) Here it is: run_if_user_active>1</run_if_user_active> <run_gpu_if_user_active>1</run_gpu_if_user_active> I'm rather expecting, unless the Wheaties have worn off, that the tag line will be missing from the main global_prefs.xml file. If so, please could you try changing the setting on the website, then changing it back again, and finally updating the project? I'm wondering if that would cause a missing setting to be added to the file. In your case, it's a bit complicated by your use of BAM - I don't use that system, so you know best which way to change settings - I'll leave the details to you. Since the Preferences were set to 1 already as you suggested, I went into the advanced|preferences tab and hit the clear button. Updated/synchronized BOINC with the web based BAM! settings (tools tab|synchronize with BAM!: that is how they synchronize in BAM!) and everything seems to be running fine: CUDA WU is running smoothly in regular mode and visible. I think ya done did it! Also, what version should I be using for BOINC on that machine for my CUDA, or should I just wait for the new stable version to come out. 6.6.36 might not be the latest, but it seems to be running stable and without hiccups for now. Any other update suggestions would be welcome. HTH ____________ (Click for detailed stats) | |
| ID: 13033 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Paul D. Buck Send message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level Scientific publications | |
|
*MY* update suggestion to you is that if it is working now ... don't ... | |
| ID: 13043 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1620 Credit: 8,894,687,378 RAC: 19,752,053 Level Scientific publications | |
|
The first set is interesting: <run_if_user_active>1</run_if_user_active> <run_gpu_if_user_active /> That space before the closing / has been causing all sorts of problems with optimised and 'anonymous platform' xml files. Although it appears (and in standard XML would be accepted) to set the flag, I wonder if BOINC failed to parse it correctly and hence created the 'stealth mode'? Where could it have come from - BOINC's own web code, or is that one you set through BAM? My samples yesterday were copied from a machine running BOINC preferences only, with no contact with BAM - and they all showed the explicit 0 1 format. So I suspect BAM. FWIW, the parsing bug should finally have been eliminated with the very latest BOINC v6.10.13, released for alpha testing last night. But I agree with Paul: that on its own is not a sufficient reason for upgrading! | |
| ID: 13044 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Gerry Rough Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level Scientific publications | |
*MY* update suggestion to you is that if it is working now ... don't ... Correct. Just missed the opening character in the copy and paste. That would have really fouled up the works, much like missing the first bracket in a BBCode tag. Yikes! ____________ (Click for detailed stats) | |
| ID: 13046 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Gerry Rough Send message Joined: 28 Sep 09 Posts: 15 Credit: 3,688,434 RAC: 0 Level Scientific publications | |
Good. But actually I was thinking about the other upgrades when I wrote that: the compute capability has been upgraded, at least if I read that correctly elsewhere here at GPUGrid, and the driver version. But since everything looks good, perhaps it is good to leave well enough alone. Still getting used to being conscious of the little things while running GPUGrid. I've been with alphas before, but with CUDA code and an entirely new technology to help out with, I suppose you can never be too careful to get it right. :-/ ____________ (Click for detailed stats) | |
| ID: 13047 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Paul D. Buck Send message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level Scientific publications | |
Still getting used to being conscious of the little things while running GPUGrid. I've been with alphas before, but with CUDA code and an entirely new technology to help out with, I suppose you can never be too careful to get it right. :-/ For what it is worth the GPU addition has all the flavor of the days of BOINC Beta where almost every day was an adventure and we went through version after version trying to get something that would work and be stable. The only good news is that we have several projects that reliably issue work for GPUs, two for ATI, hope for OpenCL in the works and BOINC versions that almost work correctly ... Sadly we also seem to be duplicating the mode where a change will get in and it is nearly impossible to get it back out (Strict FIFO on GPU tasks) even though it is deadly to most users of BOINC ... (FIFO is great for single project users, death if you want a queue and resource allocations to your liking)... | |
| ID: 13056 | Rating: 0 | rate:
/ Reply
Quote
| |
|
skgiven Volunteer moderator Volunteer tester  Send message Joined: 23 Apr 09 Posts: 3968 Credit: 1,995,359,260 RAC: 0 Level  Scientific publications | |
Just remember, you have now set local preferences on that machine and the web based preferences are not going to affect that machine's operation. I don't know why it did not propagate the preferences correctly though you are using a version, at least for GPU purposes, that is fairly well out of date. [/quote] It is a "Default Boinc" Location Thing! If you configure a Work location but you dont add your computer to the Work group, the configurations you set wont propagate to your system. | |
| ID: 13150 | Rating: 0 | rate:
/ Reply
Quote
| |
Message boards : Number crunching : Stealth CUDA?
