Advanced search

Message boards : Number crunching : Return window too short, queue too deep

Author Message
Scott A. Howard
Send message
Joined: 29 Dec 08
Posts: 9
Credit: 9,407
RAC: 0
Level

Scientific publications
wat
Message 16981 - Posted: 12 May 2010 | 12:18:51 UTC

Hi folks, here's the problem. The project is running on a machine that is free for GPU computing about 12 hours/day. The work units look to take about 52 hours to complete. That's about 5 days time to complete a WU. I am getting WUs rejected because they are returned after the deadline.

You guys need to do two things:

1) Open the window for the return of the results, say by 50%. If you cannot, why not?

2) Allow me to specify the queue depth for pending jobs. You are sending 2 at a time to my machine. Due to the time it takes the first to process, the second one is aging out. I would rather see a delay before my next sequential WU is downloaded rather than having it sit there aging.

I appreciate the opportunity to participate. Maybe you guys could focus a bit on the scheduling aspect for a bit though?

Profile K1atOdessa
Send message
Joined: 25 Feb 08
Posts: 249
Credit: 370,186,977
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 16982 - Posted: 12 May 2010 | 12:34:12 UTC - in response to Message 16981.

1) Open the window for the return of the results, say by 50%. If you cannot, why not?


There are a lot of threads about this. The work builds on the previous WU's, so the developers decided it was better to have a 5 day window and keep the progress moving.


2) Allow me to specify the queue depth for pending jobs. You are sending 2 at a time to my machine. Due to the time it takes the first to process, the second one is aging out. I would rather see a delay before my next sequential WU is downloaded rather than having it sit there aging.


You can set your "Additional work buffer" to something small, like .02 days (about 45 minutes). That will prevent GPUGRID from sending more than 1 WU per GPU until the one running is about 45 minutes from completing. However, this will impact all projects -- if you are offline for 12 hours, there is a chance that other (CPU) work will run dry in the meantime.

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 16984 - Posted: 12 May 2010 | 13:24:07 UTC - in response to Message 16982.


You can set your "Additional work buffer" to something small, like .02 days (about 45 minutes). That will prevent GPUGRID from sending more than 1 WU per GPU until the one running is about 45 minutes from completing. However, this will impact all projects -- if you are offline for 12 hours, there is a chance that other (CPU) work will run dry in the meantime.


This highlights the need to be able to configure CPU tasks separately from GPU tasks!
On my systems I tend to manually cache up enough CPU tasks for several days at a time: Select No New Tasks for GPUGrid, up the cache level to 7 days (or whatever you like), download the queue of CPU tasks, then change the cache back to 0.04 (or similar), and finally enable New Tasks for GPUGrid again.

Would be much easier with a separate GPU Tab on Boinc, for many reasons!
I have already asks for this - it might take others to lend their weight...

Scott A. Howard
Send message
Joined: 29 Dec 08
Posts: 9
Credit: 9,407
RAC: 0
Level

Scientific publications
wat
Message 16987 - Posted: 12 May 2010 | 15:15:03 UTC - in response to Message 16982.

1) Open the window for the return of the results, say by 50%. If you cannot, why not?


There are a lot of threads about this. The work builds on the previous WU's, so the developers decided it was better to have a 5 day window and keep the progress moving.


You guys have all the stats on the GPU and WU throughput. Have you actually projected the WU throughput that would be achieved by opening the window to 7 days? The breadth of the set of parallel queues would widen because more WU could be turned around in that timeframe. I suggest that your throughput would actually increase. What do you guys think those 2 days would do to your scheduling and throughput for the project? Increase or decrease it? Do you have any modeling to show how an increase would negatively impact the throughput?

Maybe you can point me to a FAQ that shows where you demonstrate the negative impact on the project?

Profile K1atOdessa
Send message
Joined: 25 Feb 08
Posts: 249
Credit: 370,186,977
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 17017 - Posted: 14 May 2010 | 0:31:05 UTC - in response to Message 16984.

This highlights the need to be able to configure CPU tasks separately from GPU tasks!


Yup. I'd like that feature to have like 3 days worth of CPU WU's but not a ton of extra GPU WU's just sitting idle for hours while the others crunch. I'll vote for your cause.

Post to thread

Message boards : Number crunching : Return window too short, queue too deep

//