My i7-8700 is left with nothing to do.
http://www.gpugrid.net/results.php?hostid=475515

I will put it on Folding.

My Ryzen 1700 is still busy with plenty of QC tasks… and there are many more in the queue. How can it be that your 8700 doesn't get any? This system is also a Linux based one, is it not?
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Yes, that is the point, there are plenty of tasks available. It seems that they just place a limit on them. I think it is to guard against machines that produce a lot of errors, but mine doesn't. I think the limit should be increased.

Note that my i7-8700 was running QC only, and ran through a lot of them per day. I have a Ryzen 1700 also, but run just four cores on QC (two work units running two cores each). That machine has no problem getting work, and I will let it run. But if they ever want to get their mountain of work done, they will have to let the high-productivity machines get them. The Androids won't do it.

As you probably know, there was some discussion of it earlier, though it does not tell you much.
http://www.gpugrid.net/forum_thread.php?id=4823

And Richard Hasselgrove (as usual) has the best handle on it:
http://www.gpugrid.net/forum_thread.php?id=4825

Jim,

Your last work unit reported at 07:13 with an error. I don't see any others after that. Is it possible that server put your machine in "time out" until you reported a new work unit that validates?

I've not seen a limit yet on the work units for CPU. I'm running a i7 6950X and it been steadily busy since I got it running under Ubuntu. I usually get about 24 at a time which corresponds to my daily limit of 0.5 days + 0.1 extra

Z
____________

Your last work unit reported at 07:13 with an error. I don't see any others after that. Is it possible that server put your machine in "time out" until you reported a new work unit that validates?

CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/pro/linux-64/repodata.json.bz2>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

OK, I will try it again later and see how it goes.

Found my r7 1700 system idling with a daily quota of 4 hit. Why would so many WUs fail?

Found my r7 1700 system idling with a daily quota of 4 hit. Why would so many WUs fail?

Found my r7 1700 system idling with a daily quota of 4 hit. Why would so many WUs fail?

My system gets random CondaHTTPErrors as well. From a layman's perspective this seems to be a bottleneck.

Are volunteers possibly risking being blacklisted by thrashing the Conda Cloud? Is there another way for the project to distribute/cache the necessary packages?

@Zalster: That conda was getting too many download requests from users so it refused to download the packages on your machine at that moment. Should work next time I assume.

Indeed the new short WUs probably contact the conda cloud too often.

Can you give me an estimated runtime of these WUs to know how many of them to pack together?

Can you give me an estimated runtime of these WUs to know how many of them to pack together?

Here is my i7-8700
http://www.gpugrid.net/results.php?hostid=475515

They were often less than 10 minutes each, and I was running three at a time. You could pack 10 of them together insofar as I am concerned (or at least 4).

Very interesting comparison of run times. Running intel myself, Ryzen seems to struggle.

Tasks take approx 10mins now but I'd prefer to crunch tasks <60mins.

Running intel myself, Ryzen seems to struggle.

The following is largely anecdotal, but I've found that 4-core tasks are more efficient than to two 2-core tasks. After 1 hour, 4-cores had accumulated (slightly) more credit, which includes start up time for each task and so on. My CPU does not support Hyper Threading, but it might be worth a separate test, if you're looking for best efficiency.

With more cores memory and disk through put seem especially relevant for QC.

With more cores memory and disk through put seem especially relevant for QC.

Ok thanks for the reports! The problem is that the WU runtime scales quadratically to the number of electrons in the molecule so larger molecules will take longer. But I assume I can go at least 5x the current length for this batch.

I am getting errors on QC tasks "Disk limit exceeded". They are all SELE6.
Tullio

I'm starting to see those too. Just had 4 of them error out on my machine.

+1

I am running SETI@home and Einstein@home on both Linux boxen and also on a Ulephone smartphone with Android 7.1.1, Atlas@home on my Windows 10 PC.Goodbye GPUGRID.
Tullio

The last three QC have all erred for me with "Disk usage limit exceeded" also. It is time to give it a rest until they can get it fixed, hopefully soon.

The last three QC have all erred for me with "Disk usage limit exceeded" also. It is time to give it a rest until they can get it fixed, hopefully soon.

I am getting a lot of the "Disk usage limit exceeded" errors now. Was getting a few several days ago but now it is nearly all that error out. It is unclear whether the error message refers to disk capacity or frequency of disk write/reads are exceeded. It would be nice if the project folks would let us know why the error and if there is anything that we can do to reduce the probability of encountering these errors.

Basically, there is no point in continuing to run these WU's since nearly all are erring and these SELE6 jobs are thrashing all my machines regardless of number of threads allowed. I finally figured out a way to shrink the linux disk cache from eating all my ram leaving only < 1% free but even leaving at least 4% ram free doesn't stop the thrashing. Maybe if I spring for 32 GB on the 8 core machines with currently 16GB each, the thrashing will be reduced but that won't help the disk errors. Might try on one machine to see out of curiosity.

I finally figured out a way to shrink the linux disk cache from eating all my ram leaving only < 1% free but even leaving at least 4% ram free doesn't stop the thrashing. Maybe if I spring for 32 GB on the 8 core machines with currently 16GB each, the thrashing will be reduced but that won't help the disk errors.

Please delete. Each time I edit something, it posts a new message.

Yeah, I just stopped accepting new QC work units until they figure out what the problem is.
____________

172,128 QC ready to send, 48 users. No comment.
Tullio

172,128 QC ready to send, 48 users. No comment.
Tullio

And now this:

Thu 11 Oct 2018 10:10:44 AM CDT | GPUGRID | Aborting task 123_35_37_39_42_da3ae375_n00001-SDOERR_SELE6-0-1-RND5707_4: exceeded disk limit: 59944.94MB > 57220.46MB

Yes, I have also reluctantly set preferences not to accept any more production QC tasks until the disk usage problem is identified and eliminated. I had 6 machines with 16 threads (1/2 of available threads) on the project but all these WU's are just thrashing my machines and producing errors after an hour of so wasted cpu time. I am configured to run QC beta should any fixes be attempted.

Tried two more QC tasks, they all fail the same way. Complete silence from admins.
Tullio

Decided to give it a try again. Rough estimates are 1 valid for every 4 errors.

Almost all validates were after another computer errored out but not due to size limits.
So there are batches of work units out there that don't exceed the limit but failed for other reasons.

However, on my computers, almost all of my errors were size related. So the original error for this thread still exist.


____________

Going back to Aug24, my QC completion record shows 194 errors out of 454 QC WU's processed. This is about a 42.7% failure rate and a small random sampling of the error causes reveals almost all due to 'disk usage limit exceeded.' Would be nice to get an explanation what specifically this error means.

I am running other 4 BOINC projects, both on Linux and Windows 10. Some use also GPUs, some don't but use Virtual Box, so I have a vast experience on all kind of errors. But all give me a feedback by admins or other volunteers with similar experiences. Here only silence.
Tullio

The error has occurred on some other projects where the disk size usage went past a limit set by the app. It wasn't a limit on the PC running the task.

of the few validating, this is the biggest so far

http://www.gpugrid.net/workunit.php?wuid=14584472

4943 seconds credit 1279.49
____________

Ok I will ask Toni if he can increase the WU disk space. But at some point we will just write your whole disk full as it seems...

I am making some new ones now to send out maybe by tomorrow.

I received a notice from the server that QC tasks require 77 GB. But I have more than 700 GB available to BOINC on my two Linux boxen. No other BOINC project requires that much space.
Tullio

I received a notice from the server that QC tasks require 77 GB. But I have more than 700 GB available to BOINC on my two Linux boxen. No other BOINC project requires that much space.
Tullio

The new ones won't really require any disk space because they are smaller molecules. Although I don't know how BOINC does it, like if it reserves the maximum space per WU or it just cuts the WU if it exceeds the max space.

They work for me. I have downloaded 24 work units (a quota limit), and they take only 1 GB disk space in total. I am running three at a time (4 cores each), and they are taking only about 1 GB memory each.

They run for about 40 to 60 minutes on my i7-8700, which is ideal.

No problems on my computers. Seem to be running ok.
____________

It seems also they give a correct progress figure, not the usual 10%.
Tullio

Since I restarted QC, the QMML50 are running great. No memory or disk limits reached or exceeded. So far 32 in progress and 6 of 6 successful completions. They seem to do the 10% + 1.8% X 50 = 100%. Times so far are ranging a little less than 40 min to nearly two hours per WU with the longer completions on slower two thread at 2 GHz machines and the others running 4 threads at 4 GHz.

Thank you all in any case for hanging on through all the troubles of the large WUs.

I mean, the larger problematic ones are obviously of more interest but I believe we should rethink the design of those large molecules and maybe try to break them down into their constituent components

Maybe you could split QC into long and short runs like the GPU jobs? That would make it possible to choose.

I mean, the larger problematic ones are obviously of more interest but I believe we should rethink the design of those large molecules and maybe try to break them down into their constituent components so that they don't take half a hard-drive to compute. It's not trivial but it seems like it's necessary if we want to keep running them on GPUGRID.
Thank you all in any case for hanging on through all the troubles of the large WUs.

Yes I would also be interested in helping with the larger ones. How large of an SSDs are we talking about? 1, 2, 4 Terabytes?

Yes I would also be interested in helping with the larger ones. How large of an SSDs are we talking about? 1, 2, 4 Terabytes?

Well, not really SSD. HDD would be just fine. Right now scratch files are kept wherever BOINC's "slot" directory is (as expected from a well-behaved application).

Any SSD won't cut it. You'd need the expensive stuff with larger faster write buffers. Or use several HDDs (each on another connection, with several BOINC instances to spread the slots around). But really you should look into Optane, but it will cost a fortune.

The argument that QMML has little value is disturbing to me. Either there is scientific justification or not. And please stick with a plan. All these abrupt changes make me question the research goal. I like to feel involved too, but the community shouldn't get to change course.

Further I fear splitting the project will just further complicate things for everyone.

And please always communicate upfront how your tasks will impact the volunteer's systems (with TBW, conda requests and so on). Volunteers should not have to find out themselves.

Any SSD won't cut it. You'd need the expensive stuff with larger faster write buffers.

On Oct. 17, Stephan wrote:

"The new ones won't really require any disk space because they are smaller molecules."

We are up to 74 Linux users. But recent QC tasks run well on my two Linux boxen. The one with a GTX 750 Ti GPU board is also running a GPU task alongside a CPU task. All this on an Opteron 1210 of 2008 vintage. But this SUN workstation is still my main host.
Tullio

The one with a GTX 750 Ti GPU board is also running a GPU task alongside a CPU task.

So I have successfully installed and tested the new WD Red 4T NAS HDD into 2 of my computers. They are up are running again. So however the project decides to proceed with the larger molecules, I hope to be prepared.
____________

the project decides to proceed with the larger molecules, I hope to be prepared.

The final 10% always takes 90% of the time.
Tullio

Apologies.. Looks like I trashed around 23 CPU work units. Internet was down for most of the day and I didn't notice it until just now. Not sure if the internet being down caused the errors or if it was something with the computer but it appears to be running normal. Will losing internet after the work units download but before they start to crunch cause them to error if they can't contact the server when they start??
____________

Long answer: Yes.