Message boards : Number crunching : Bad batch of TONI-AGGd tasks
| Author | Message |
|---|---|
|
Richard Haselgrove Send message Joined: 11 Jul 09 Posts: 1576 Credit: 5,807,811,851 RAC: 9,772,970 Level  Scientific publications       | |
|
Both my active hosts - GTX470 and GTX670 - are showing "Energies have become nan" errors for all recent tasks in this batch (short queue). | |
ID: 27194 | Rating: 0 | rate:
 /   Reply
Quote
| |
|
[AF>Belgique] bill1170 Send message Joined: 4 Jan 09 Posts: 13 Credit: 837,852,199 RAC: 279,940 Level  Scientific publications | |
|
Same for me. The problem started November 1st around 22:00 UTC. I had to stop temporarily GPUgrid as I receive only WUs of this faulting "Toni" batch. | |
| ID: 27195 | Rating: 0 | rate:
/ Reply
Quote
| |
|
flashawk Send message Joined: 18 Jun 12 Posts: 297 Credit: 3,572,627,986 RAC: 0 Level  Scientific publications | |
|
I just had to download 5 before I got a NOELIA that would run. That brings my total to 8 failed TONI wu's, it sucks because GPUGRID doesn't know that a task has failed and one of my video cards sits idle for sometime. | |
| ID: 27199 | Rating: 0 | rate:
/ Reply
Quote
| |
 GDFVolunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 14 Mar 07 Posts: 1957 Credit: 629,356 RAC: 0 Level  Scientific publications  | |
|
Yes, | |
| ID: 27201 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Toni Volunteer moderator Project administrator Project developer Project tester Project scientist Send message Joined: 9 Dec 08 Posts: 1006 Credit: 5,068,599 RAC: 0 Level  Scientific publications | |
|
Sorry guys, I cancelled them. I was fooled because some of them went ok. Those which fail, appear to do so at the start. | |
| ID: 27202 | Rating: 0 | rate:
/ Reply
Quote
| |
|
flashawk Send message Joined: 18 Jun 12 Posts: 297 Credit: 3,572,627,986 RAC: 0 Level Scientific publications | |
|
Thanks Toni for you're quick action, they seem to be running fine now on all my GPU's. | |
| ID: 27237 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Toni Volunteer moderator Project administrator Project developer Project tester Project scientist Send message Joined: 9 Dec 08 Posts: 1006 Credit: 5,068,599 RAC: 0 Level Scientific publications | |
|
Thanks to you. It took a week to debug, but was very instructive in the end. Fixed workunits are called "AGGd2", and should have high GPU usage. | |
| ID: 27251 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Doesn't appear to be working for me. Just noticed my industrial quantity of errors. Two slightly different water cooled 580s running at stock speeds. | |
| ID: 27315 | Rating: 0 | rate:
/ Reply
Quote
| |
microchip Send message Joined: 4 Sep 11 Posts: 110 Credit: 326,102,587 RAC: 0 Level  Scientific publications | |
|
I also get TONI WUs that error out, with either "Energies have become nan" errors or, after a short period of crunching, with "output file absent" errors. This is really starting to annoy me. More so as I also get NOELIA WUs that run till the end only to report "output file absent" | |
| ID: 27401 | Rating: 0 | rate:
/ Reply
Quote
| |
|
Bruce Kennedy Send message Joined: 15 Jan 09 Posts: 3 Credit: 171,242,754 RAC: 0 Level  Scientific publications | |
|
I'm also getting Toni and NOELIA WU's failing after several hous. I'm setting to no new work for a few days to see how this shakes out. | |
| ID: 27413 | Rating: 0 | rate:
/ Reply
Quote
| |
Message boards : Number crunching : Bad batch of TONI-AGGd tasks
