Advanced search

Message boards : Number crunching : Immensive credits difference: ADRIA vs. PABLO tasks

Author Message
Profile Michael H.W. Weber
Send message
Joined: 9 Feb 16
Posts: 31
Credit: 207,888,684
RAC: 292,080
Level
Leu
Scientific publications
watwat
Message 46524 - Posted: 18 Feb 2017 | 8:54:22 UTC

Same GPUs, VERY similar run times, almost 4-fold less credits for ADRIAs tasks. Why is that?

ADRIAs tasks:

15997518 12330750 16 Feb 2017 | 9:13:03 UTC 17 Feb 2017 | 12:03:51 UTC Fertig und Bestätigt 48,100.37 22,760.09 146,375.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)
15997517 12330749 16 Feb 2017 | 9:13:03 UTC 16 Feb 2017 | 23:12:08 UTC Fertig und Bestätigt 48,463.93 23,111.61 175,650.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

PABLOs tasks:
15993065 12326872 14 Feb 2017 | 8:43:45 UTC 14 Feb 2017 | 23:43:29 UTC Fertig und Bestätigt 53,238.56 20,422.74 421,800.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)
15991483 12325510 13 Feb 2017 | 12:15:47 UTC 14 Feb 2017 | 8:36:56 UTC Fertig und Bestätigt 52,702.50 18,334.29 421,800.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

Michael.
____________
President of Rechenkraft.net - Germany's first and largest distributed computing organization.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 1814
Credit: 9,960,094,194
RAC: 6,342,775
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46525 - Posted: 18 Feb 2017 | 11:26:15 UTC - in response to Message 46524.
Last modified: 18 Feb 2017 | 11:27:13 UTC

Your first ADRIA task missed the 24h bonus (+50%), so it has only +25% bonus.

PABLO's tasks have 49.000~54.000 atoms and 15.625.000 steps
ADRIA's tasks have 104.159 atoms and 3.750.000 steps

Credit ratio: ~2.4 times
Runtime ratio: ~1.09 times (~1.38 times on my hosts using SWAN_SYNC)

ADRIA's tasks do more calculations on the CPU, so they do less calculations on the GPU.

However, the disparity in the credit/time ratio should be addressed. The method used for estimating the credits before a batch is issued should be corrected.

Profile Michael H.W. Weber
Send message
Joined: 9 Feb 16
Posts: 31
Credit: 207,888,684
RAC: 292,080
Level
Leu
Scientific publications
watwat
Message 46533 - Posted: 22 Feb 2017 | 8:50:53 UTC
Last modified: 22 Feb 2017 | 8:56:23 UTC

While I do very well understand your point with the 24 hrs deadline, the examples below demonstrate that despite the deadline issue something has to be fixed with the credits system. So, this time the 24 hrs deadline is met in each case:

GTX 770 system - all tasks match 24 hrs deadline but differ unproportionally in credits:

PABLO:
16008824 12340825 20 Feb 2017 | 19:40:19 UTC 21 Feb 2017 | 8:30:18 UTC Fertig und Bestätigt 46,014.89 6,223.11 252,750.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

ADRIA:
15993427 12327182 14 Feb 2017 | 13:10:23 UTC 15 Feb 2017 | 3:39:39 UTC Fertig und Bestätigt 52,101.35 17,701.32 175,650.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

In the above case, the task with less run time receives significantly more credits.

GTX 970 system - all tasks match 24 hrs deadline but differ unproportionally in credits:

PABLO:
15991483 12325510 13 Feb 2017 | 12:15:47 UTC 14 Feb 2017 | 8:36:56 UTC Fertig und Bestätigt 52,702.50 18,334.29 421,800.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

ADRIA:
15997517 12330749 16 Feb 2017 | 9:13:03 UTC 16 Feb 2017 | 23:12:08 UTC Fertig und Bestätigt 48,463.93 23,111.61 175,650.00 Long runs (8-12 hours on fastest card) v8.48 (cuda65)

So, although exactly the same system was used for computation and in each case shown above the 24 hrs deadline was met, the credits still differ significantly at comparable run times (= electricity consumption).

Your argument above regarding # of steps vs. # of atoms is also somehow less conceivable to me (but might require more in depth understanding): In ADRIAs case we have less steps but significantly more atoms while in PABLOS case it is vice versa.
In toto, a given GPU runs comparably long and consumes comparably much electricity. Shouldn't these virtual credits somehow reflect this?
When can we expect a change?

Michael.
____________
President of Rechenkraft.net - Germany's first and largest distributed computing organization.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 1814
Credit: 9,960,094,194
RAC: 6,342,775
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46534 - Posted: 22 Feb 2017 | 16:25:16 UTC - in response to Message 46533.

Your argument above regarding # of steps vs. # of atoms is also somehow less conceivable to me (but might require more in depth understanding): In ADRIAs case we have less steps but significantly more atoms while in PABLOS case it is vice versa.
This is not an argument, simply the facts. Maybe I wasn't clear on this, but I agree with you.

To further understand the behavior of the GPUGrid app, take a look at the "CPU Time" column:

PABLO:
16008824 12340825 20 Feb 2017 | 19:40:19 UTC 21 Feb 2017 | 8:30:18 UTC Fertig und Bestätigt 46,014.89 6,223.11 252,750.00

ADRIA:
15993427 12327182 14 Feb 2017 | 13:10:23 UTC 15 Feb 2017 | 3:39:39 UTC Fertig und Bestätigt 52,101.35 17,701.32 175,650.00

The ADRIA task has 2.84 times CPU time than the PABLO task.

In toto, a given GPU runs comparably long and consumes comparably much electricity.
The actual power consumption of the GPU is different while crunching workunits from different batches, regardless of the GPU usage percentage you can read by 3rd party tools. In this instance the GPU's power consumption is less while crunching an ADRIA_MI_FAAH_TRP445TYR_INH than crunchhing a PABLO_adaptive_goal_KIX_CMYB.

Shouldn't these virtual credits somehow reflect this?
They should, but these credits are assigned "manually" for each batch, based on an estimation. This estimation is inadequate.

When can we expect a change?
I don't know. The good thing is that this affects everyone the same. The bad thing is that this disparity encourage some volunteers to selectively abort the less "profitable" workunits - but as these abortions counts as failures, therefore you can quickly reduce the daily quota of this host to zero by doing so.

To moderate this large difference in the credit over time ratio you can use the SWAN_SYNC environmental setting, which will make every workunit a bit faster, hopefully the slower ones speed up more.

Stefan
Volunteer moderator
Project developer
Project scientist
Send message
Joined: 5 Mar 13
Posts: 258
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 46540 - Posted: 23 Feb 2017 | 11:03:53 UTC

Looking into it. Pablo probably has to adjust his

Profile Michael H.W. Weber
Send message
Joined: 9 Feb 16
Posts: 31
Credit: 207,888,684
RAC: 292,080
Level
Leu
Scientific publications
watwat
Message 46554 - Posted: 24 Feb 2017 | 13:52:33 UTC

Well, I have now a power meter attached to my computer showing me that there is indeed less electricity consumption with ADRIAs task but, again, it is not proportional to the credits difference.

But OK, let's see what happens.

I wished GPUGRID would have more tasks available. The way it currently is handled, unfortunately, it is quite an unreliable project which leaves my machines unnecessarily idling for significant durations. Also, the quota of 2 tasks per machine is, under these idling conditions, a bad idea - at least for the small tasks.

Michael.
____________
President of Rechenkraft.net - Germany's first and largest distributed computing organization.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 1814
Credit: 9,960,094,194
RAC: 6,342,775
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 46555 - Posted: 24 Feb 2017 | 21:34:52 UTC - in response to Message 46554.
Last modified: 24 Feb 2017 | 21:35:18 UTC

Well, I have now a power meter attached to my computer showing me that there is indeed less electricity consumption with ADRIAs task but, again, it is not proportional to the credits difference.
True. It should be corrected. But from the scientific point of view credits are unimportant. To put it in another way: I rather help a worthy project for less credits, than a worthless for more credits. To put it in another perspective: the scientists / students rather spend their time on finding new (worthy) simulations, than on correcting the credit system.

I wished GPUGRID would have more tasks available. The way it currently is handled, unfortunately, it is quite an unreliable project which leaves my machines unnecessarily idling for significant durations.
This comes up every time when there's a shortage, but remember that we volunteered to help this project, not the project volunteered to keep our computers busy. Also, it is advised to have backup project(s) (=resource share set to zero) to avoid idling.

Also, the quota of 2 tasks per machine is, under these idling conditions, a bad idea - at least for the small tasks.
There's an ongoing argument about this, as others have quite the opposite point of view: 1 Work Unit Per GPU

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 193
Credit: 25,979,525
RAC: 19
Level
Val
Scientific publications
wat
Message 46561 - Posted: 25 Feb 2017 | 2:36:26 UTC - in response to Message 46555.

Zoltan,

How does one go about making a back-up project that can stop working when a gpugrid project becomes available? I'd love to do this, but only know how to do one project at a time, so my computer has just been sitting turned off in a corner.
____________
Cruncher/Learner in progress.

Simonas Urbelis
Send message
Joined: 30 Sep 15
Posts: 1
Credit: 54,855,322
RAC: 135,372
Level
Thr
Scientific publications
wat
Message 46562 - Posted: 25 Feb 2017 | 9:54:20 UTC - in response to Message 46561.
Last modified: 25 Feb 2017 | 9:56:16 UTC

A backup project is any project with the resource share set to 0. Just attach another project, and set the share on said project's website to 0. Once that's done, your client will request a single unit at a time while all non-backup projects have no work, and you're out of units to run.

Although do keep in mind that other projects have CPU tasks too, and you'll need to uncheck "use CPU" if you wish to keep running GPU only.

Profile Logan Carr
Send message
Joined: 12 Aug 15
Posts: 193
Credit: 25,979,525
RAC: 19
Level
Val
Scientific publications
wat
Message 46567 - Posted: 28 Feb 2017 | 2:51:16 UTC - in response to Message 46562.

A backup project is any project with the resource share set to 0. Just attach another project, and set the share on said project's website to 0. Once that's done, your client will request a single unit at a time while all non-backup projects have no work, and you're out of units to run.

Although do keep in mind that other projects have CPU tasks too, and you'll need to uncheck "use CPU" if you wish to keep running GPU only.


Thank you
____________
Cruncher/Learner in progress.

Erich56
Send message
Joined: 1 Jan 15
Posts: 346
Credit: 1,457,365,177
RAC: 2,705,716
Level
Met
Scientific publications
watwatwat
Message 47731 - Posted: 2 Aug 2017 | 18:27:19 UTC

Once more, there is a big mis-relation between credits for certain tasks:

e3s5_e1s1p0f218-ADRIA_FOLDA3D75_crystal_ss_contacts_50_a3D_3-0-1-RND1216_0
runtime: 14.317 secs; 137.850 credit points

e3s6_e2s2p0f249-ADRIA_FOLDPG01_crystal_ss_contacts_50_proteinG_2-0-1-RND8994_0
runtime: 26.299 secs; 86.100 credit points

e73s7_e65s33p0f13-PABLO_P05067_0_IDP-0-1-RND9622_1
runtime: 22.978 secs; 252.750 credit points

klepel
Send message
Joined: 23 Dec 09
Posts: 126
Credit: 1,702,473,012
RAC: 1,862,691
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 47752 - Posted: 7 Aug 2017 | 2:36:08 UTC

Just like to ask project-team:
Why these "ADRIA_FOLDA3D50_crystal_ss_contacts_20" do not run as short tasks?
They take less than 2 hours on a GTX1070. This sseems perfect to me to fit under the definition of short runs. Is it just negligence of ADRIA or is there a deeper reason?
I am asking as I think it would be helpful to stick to the definitions, so that participants with lesser cards get there fair share of the work units. Especially, when there are so few short WUs available.

Post to thread

Message boards : Number crunching : Immensive credits difference: ADRIA vs. PABLO tasks