Advanced search

Message boards : Number crunching : NOELIA_sh2NOTCL system freezes with CUDA 3.1

Post to thread Subscribe Sort
Author Message
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27126 - Posted: 23 Oct 2012 | 7:02:11 UTC

I have a NOELIA_sh2NOTCL task running, and crashing, for 3days on a GTX470, XPx86, GPUGrid only computer.

The WU failed on 5 other systems:
5936664 136638 19 Oct 2012 | 8:22:53 UTC 19 Oct 2012 | 8:28:30 UTC Error while computing 0.00 0.00 --- Long runs (8-12 hours on fastest card) v6.16 (cuda42)
5954003 109360 19 Oct 2012 | 15:18:19 UTC 19 Oct 2012 | 17:42:53 UTC Error while computing 5.00 0.56 --- Long runs (8-12 hours on fastest card) v6.16 (cuda42)
5955043 136673 19 Oct 2012 | 22:21:39 UTC 19 Oct 2012 | 22:30:50 UTC Error while computing 0.00 0.00 --- Long runs (8-12 hours on fastest card) v6.16 (cuda31)
5955695 79880 20 Oct 2012 | 2:54:10 UTC 20 Oct 2012 | 2:56:11 UTC Error while computing 3.43 0.00 --- Long runs (8-12 hours on fastest card) v6.16 (cuda31)
5956199 136706 20 Oct 2012 | 5:25:14 UTC 20 Oct 2012 | 5:46:35 UTC Error while computing 0.00 0.00 --- Long runs (8-12 hours on fastest card) v6.16 (cuda42)
5956525 135026 20 Oct 2012 | 8:03:21 UTC 25 Oct 2012 | 8:03:21 UTC In progress --- --- --- Long runs (8-12 hours on fastest card) v6.16 (cuda31)

Although the task is now at 97% on my system, it was at 93% yesterday evening. At this stage I just want it to finish so I can start crunching other tasks, and know if there is anything wrong with the system...

I think this is just an example of CUDA3.1 problems. Even if the task didn't restart it would take well over 24h to complete on a GTX470. Even a GTX480 could not complete this WU on CUDA 3.2.
So, can we move to CUDA 4.2 only for the long tasks?

Thanks,
____________
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

ID: 27126 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dskagcommunity
Avatar
Send message
Joined: 28 Apr 11
Posts: 456
Credit: 817,865,789
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27131 - Posted: 23 Oct 2012 | 16:18:39 UTC
Last modified: 23 Oct 2012 | 16:25:40 UTC

deleted half of the message...didnt read all correct ;)

But still dont know what it has to do with cuda 31, it failed on 42 systems too. These are likly fail on 42 when you dont overvolt&underclock on some GPUs like i must to with a stockclocked! 560TI. My Cuda31 card 285GTX runs these units without any problems.

For the freezing problem, i cant say it has to do with the Cuda app 31 or 42 because i had rarely freezes on both Cuda apps over the longer time i compute now for gpugrid.
____________
DSKAG Austria Research Team: http://www.research.dskag.at



ID: 27131 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
mymbtheduke
Send message
Joined: 3 Sep 12
Posts: 40
Credit: 186,780,650
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 27136 - Posted: 23 Oct 2012 | 20:37:24 UTC

I have a NOELIA_sh2NOTCL running that is at 9 hours and 31% complete. It is still running but 30 hours for one task is a bit much.

The GPU is at 84% and is a 560ti at 925Mhz, Phenom X6 at 2.8 and 16 Ghz RAM at 1600Ghz.

ID: 27136 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27140 - Posted: 24 Oct 2012 | 9:41:15 UTC - in response to Message 27136.

I think it's fair to say that 1.43days on a GTX470 is a bit long, and that's without the freezing, which may or may not be related.
____________
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

ID: 27140 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlesa25
Avatar
Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27142 - Posted: 24 Oct 2012 | 11:06:55 UTC
Last modified: 24 Oct 2012 | 11:21:02 UTC

Hello: I am running one of these tasks on my GTX 590 and is at 66.15% after 8.16 hours, 96% load on the GPU, Cuda 4.2, I reckon it will take just over 12h. are really heavy duty. Greetings.

ID: 27142 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
jlhal
Send message
Joined: 1 Mar 10
Posts: 147
Credit: 1,077,535,540
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27143 - Posted: 24 Oct 2012 | 11:50:25 UTC - in response to Message 27142.

I had to cancel this NOELIA WU yesterday ! More than 43 hours elapsed and frozen
on a Gigabyte GTX 690 !

http://www.gpugrid.net/workunit.php?wuid=3757561
____________
Lubuntu 16.04.1 LTS x64

ID: 27143 | Rating: 0 | rate: Rate + / Rate - Report as offensive    Reply Quote
Post to thread

Message boards : Number crunching : NOELIA_sh2NOTCL system freezes with CUDA 3.1

//