Advanced search

Message boards : Multicore CPUs : Experimental QC app for Windows 10

Author Message
Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48980 - Posted: 18 Feb 2018 | 16:37:43 UTC
Last modified: 18 Feb 2018 | 18:00:04 UTC

Dears,

have a test QC app for Windows 10. QC relies on the PSI4 software which does not have a Windows version. Therefore, we'll rely on WSL (Windows Subsystem Linux - a nice lightweight Linux-like environment to run native apps inside Windows), provided by Microsoft in the recent version of Windows 10.

Unfortunately I don't think there is a way to prevent the distribution to non-W10 and non-WSL hosts.

Therefore, to run QC tasks, the first time only you will have to prepare the environment as follows...


  1. Enable WSL. The procedure unfortunately varied over time, the most current being here
  2. Install the Ubuntu distribution via the App store (it is free of course).
  3. Launch the newly-installed Ubuntu command line via its icon.
    It will request a password for a new user being created. Provide one.
  4. Now you have a working Ubuntu environment. As for the native ones, manual installation of the gcc package is required. So, in the Ubuntu bash prompt, execute the following commands (the prompt reads like "(your_chosen_user)@MACHINENAME:path")

    sudo apt update
    sudo apt install gcc



(Of course, if you already have WSL installed, you will just need step 4.)

Technical details: In addition to the normal Boinc project and slot directories, the app will use space in the /tmp/gpugrid_miniconda directory inside your WSL installation (not directly accessible from Windows). Unlike the Linux app, the space is not automatically cleaned up when you uninstall BOINC or reset the project. To clear it, either uninstall Ubuntu, or remove it from the Ubuntu prompt.

Juha
Send message
Joined: 3 Aug 17
Posts: 5
Credit: 3,037,619
RAC: 0
Level
Ala
Scientific publications
wat
Message 48983 - Posted: 18 Feb 2018 | 21:09:58 UTC - in response to Message 48980.

Unfortunately I don't think there is a way to prevent the distribution to non-W10 and non-WSL hosts.


You can use <min_os_version> in plan class spec. The way I read the documentation, in order to use <min_os_version> you'll need to setup different plan classes for Windows and Linux. Since you use wsl.exe that means minimum version is Fall Creators Update, or 10.0.16299.xxx.


For those having more than one user on their hosts, WSL distros are installed per-user. If you want all users to be able to run QC tasks on WSL you need to install Ubuntu for all users.

Juha
Send message
Joined: 3 Aug 17
Posts: 5
Credit: 3,037,619
RAC: 0
Level
Ala
Scientific publications
wat
Message 48985 - Posted: 18 Feb 2018 | 22:33:15 UTC - in response to Message 48980.

Not working too well. It unpacks miniconda and that's it. According to stderr it starts pre_script.py but python doesn't appear in Task Manager (or disappears too fast). wsl.exe keeps running though.

I tried starting it from command prompt and it ran after I changed sysnative to system32.

These don't work right:

<setenv>PSI_SCRATCH=$PWD</setenv> <setenv>PATH=$PROJECT_DIR/miniconda/bin:/usr/bin</setenv>

PATH C:\BOINC data/projects/www.gpugrid.net/miniconda/bin:/usr/bin PSI_SCRATCH C:\BOINC data\slots\6


You can set envvars and do path translation but you need Insider build for that. Share Environment Vars between WSL and Windows

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 48986 - Posted: 19 Feb 2018 | 0:06:02 UTC

Whats the overhead for all this? Doesn't seem like its worth it.

Profile Dingo
Avatar
Send message
Joined: 1 Nov 07
Posts: 14
Credit: 17,375,251
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 48987 - Posted: 19 Feb 2018 | 4:37:46 UTC
Last modified: 19 Feb 2018 | 4:43:19 UTC

I have downloaded a few tasks but the first two aborted:

This is the machine http://www.gpugrid.net/show_host_detail.php?hostid=453402


<core_client_version>7.9.0</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 195 (0xc3)</message>
<stderr_txt>
Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
tar: lib/python3.6/ctypes/macholib/__pycache__/__init__.cpython-36.pyc: Cannot open: File exists
tar: lib/python3.6/tkinter/__pycache__/__main__.cpython-36.pyc: Cannot open: File exists
tar: lib/python3.6/multiprocessing/__pycache__/__init__.cpython-36.pyc: Cannot open: No such file or directory
tar: lib/python3.6/concurrent/futures/__pycache__/__init__.cpython-36.pyc: Cannot open: File exists
tar: lib/python3.6/encodings/utf_32_be.py: Cannot open: File exists
tar: lib/python3.6/importlib/__pycache__/machinery.cpython-36.pyc: Cannot open: File exists
tar: lib/python3.6/ctypes/__pycache__/_endian.cpython-36.pyc: Cannot open: No such file or directory
tar: lib/python3.6/venv/scripts/common/activate: Cannot open: File exists
tar: lib/python3.6/xml/sax/__pycache__/__init__.cpython-36.pyc: Cannot open: No such file or directory
tar: lib/python3.6/html/__pycache__/__init__.cpython-36.pyc: Cannot open: No such file or directory
tar: Exiting with failure status due to previous errors
14:51:18 (10644): c:\windows\sysnative\wsl.exe exited; CPU time 0.015625
14:51:18 (10644): app exit status: 0x1
14:51:18 (10644): called boinc_finish(195)

</stderr_txt>
]]>



If I look in the stderr that is running it contains:

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
15:32:04 (5812): c:\windows\sysnative\wsl.exe exited; CPU time 0.031250
15:32:04 (5812): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)




I am not sure if I am going to let them run as the finish time is estimated over a day and still climbing:
____________

Proud Founder and member of



Have a look at my WebCam

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48989 - Posted: 19 Feb 2018 | 10:43:00 UTC - in response to Message 48987.

Thanks for your inputs. It's not working well enough. I deprecated the app version.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48990 - Posted: 19 Feb 2018 | 10:49:15 UTC - in response to Message 48989.

By the way: WSL does not seem to propagate the kill signal to stop the app. Feel free to kill it. The wu should be as short as the linux ones (order of hours).

STFC9F22
Send message
Joined: 10 Nov 17
Posts: 6
Credit: 78,198,775
RAC: 49
Level
Thr
Scientific publications
wat
Message 48991 - Posted: 19 Feb 2018 | 11:50:44 UTC - in response to Message 48980.

Unfortunately I don't think there is a way to prevent the distribution to non-W10 and non-WSL hosts.

For information –

I am running a GTX 1070 on a Win 10 Home 64 bit PC. My GPUGRID preferences were set to run ACEMD long runs, but allowing work from other applications (which I have now changed to disallow) and my Computing preferences are set to maintain enough work for 0.05days. My PC is not set up to run these test tasks.

Prior to the release of this test, these settings resulted in requesting work about an hour before completion of the running task and receiving a long run or occasionally a short run, but always limited to the download of a single task. Following this release, I have instead received multiple downloads of these tasks (ten for the most recent request) which on completion of the running task then fail immediately and are returned. If my configuration is common there may be large numbers of these tasks sitting idle on machines unable to run them.

Rantanplan
Send message
Joined: 22 Jul 11
Posts: 166
Credit: 129,564,063
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48995 - Posted: 19 Feb 2018 | 15:08:35 UTC

For me , it works "fine" .

But i can´t start an installed desktop-enviroment.

Therefore for nice running i have to make two directorys:

/dev/input/
/dev/input/mice/

whatever it stand for it , i don´t know. After EVERY restart of the Ubuntu WSL Enviroment.

Which directorys will be saved ? "home" for sure ?

Greets

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 48996 - Posted: 19 Feb 2018 | 16:03:38 UTC - in response to Message 48995.
Last modified: 19 Feb 2018 | 16:04:14 UTC

Honestly I did not know you could run desktop environments in WSL. Check if this helps... https://github.com/Microsoft/WSL/issues/637

Note that you should only need the Ubuntu command prompt once, i.e. to install gcc. (And I hope to figure out a way to get around it)

T

Rantanplan
Send message
Joined: 22 Jul 11
Posts: 166
Credit: 129,564,063
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48997 - Posted: 19 Feb 2018 | 16:11:51 UTC - in response to Message 48996.

(And I hope to figure out a way to get around it)


Never used ->

sudo su


Profile Chilean
Avatar
Send message
Joined: 8 Oct 12
Posts: 86
Credit: 151,726,480
RAC: 165,117
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49011 - Posted: 20 Feb 2018 | 2:41:32 UTC

Why not use VirtualBox like LHC?
____________

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49013 - Posted: 20 Feb 2018 | 12:34:23 UTC - in response to Message 49011.
Last modified: 20 Feb 2018 | 13:04:35 UTC

NO NO NO. VBox apps receive so much less support. It's a waste of CPU overhead, memory and bandwidth to run. If the admin wants processing power they will write a native app. Vbox just pushes the burden to donors who will just go to another project due to the issues vbox causes.

I am currently running LHC/RNA on one system but it is far from my most supported project due the vbox requirements.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49015 - Posted: 20 Feb 2018 | 18:33:29 UTC

Well, I got one of the new ones - c00772_00836-DOMINIK_QMML2_m0000001104-0-1-RND8254. Seems to have initialised - contrary to your original tech note, I can see (in Windows) that it has created temporary files in

C:\Users\Richard Haselgrove\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\tmp\gpugrid_miniconda

It's been running for about 20 minutes, and after an initial burst of progress doesn't seem to have moved on from 1.068%. I'll leave it alone for a while and see what happens.

On a related note, now I've got some sort of a Linux build running on the machine, I've successfully installed BOINC from LocutusOfBorg's PPA and test-run the client. With no GUI supplied with WSL, I haven't yet found a way of managing it (boinccmd finds my underlying Windows boinc client) but there's hope...

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49016 - Posted: 20 Feb 2018 | 18:58:58 UTC - in response to Message 49015.

There still seems to be an issue as before:

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
18:34:21 (2888): c:\windows\sysnative\wsl.exe exited; CPU time 0.031250
18:34:21 (2888): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

If i change sysnative to system32 and run the above command from CMD then the task will run. Otherwise it just sits at 1% forever.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 49017 - Posted: 20 Feb 2018 | 19:38:36 UTC - in response to Message 49016.

@bri - do you have w10 32 or 64 bit? Forgot to mention that only 64 bit W10 will work.

@richard - it SHOULD be possible to run boinc under wsl, and manage it with the windows client. It's not what the app is attempting to do...

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49018 - Posted: 20 Feb 2018 | 19:43:16 UTC - in response to Message 49017.

@richard - it SHOULD be possible to run boinc under wsl, and manage it with the windows client. It's not what the app is attempting to do...

Understood, but as a long term Windows user your link to WSL gave me a first opportunity to play with Linux, which might come in useful sometime. It also gave me a chance to do a test install of the v7.9.2 BOINC build which Gianfranco made this morning, and which will go into full testing probably tomorrow morning.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49019 - Posted: 20 Feb 2018 | 19:51:02 UTC - in response to Message 49015.
Last modified: 20 Feb 2018 | 19:52:10 UTC

It's been running for about 20 minutes, and after an initial burst of progress doesn't seem to have moved on from 1.068%. I'll leave it alone for a while and see what happens.

That is normal. It should move on within 30 minutes or so.


On a related note, now I've got some sort of a Linux build running on the machine, I've successfully installed BOINC from LocutusOfBorg's PPA and test-run the client. With no GUI supplied with WSL, I haven't yet found a way of managing it (boinccmd finds my underlying Windows boinc client) but there's hope...

I use BoincTasks running on a Win7 machine to manage BOINC running on my Ubuntu machines over the LAN. Maybe it would work on the Win10 machine to your Linux running on the same machine? I can give the folder and file locations for my case if it would help.

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49020 - Posted: 20 Feb 2018 | 20:22:56 UTC - in response to Message 49019.

I am running W10 X64

Juha
Send message
Joined: 3 Aug 17
Posts: 5
Credit: 3,037,619
RAC: 0
Level
Ala
Scientific publications
wat
Message 49021 - Posted: 20 Feb 2018 | 22:31:11 UTC

A warning to anyone trying to run BOINC client in WSL. Watch memory usage of your system carefully.

Earlier I run BOINC client in WSL to see if creating a WSL wrapper would in theory be a workable idea. I completed one task successfully, no problem. But running BOINC client in WSL makes WSL leak memory about 2 GB per hour.

I could reclaim the leaked memory by quitting all apps in WSL and closing all open WSL windows.

Also note that some AV products have problems with WSL and may also leak memory.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49024 - Posted: 21 Feb 2018 | 7:52:54 UTC - in response to Message 49019.

That is normal. It should move on within 30 minutes or so.

Sadly, it was still stuck at 1.098% after 10 hours, so I've tossed it back into the pot.

I use BoincTasks running on a Win7 machine to manage BOINC running on my Ubuntu machines over the LAN. Maybe it would work on the Win10 machine to your Linux running on the same machine? I can give the folder and file locations for my case if it would help.

I'm fine with the LAN stuff, thanks - I've got 8 machines all summarised on an ancient tool called 'BoincView', a predecessor of BoincTasks. For this experiment, I'll probably need to split the port numbers: I have had two separate Windows clients running on the same machine, with a different Manager for each - I'll need to look back at my notes for doing that.

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49025 - Posted: 21 Feb 2018 | 9:05:47 UTC - in response to Message 49024.

Mine still stuck at 1% from last night. The task I started via the command line got to 94% then just stopped.

Going to put linux on a spare machine to run these tasks for now.

Will keep trying any new tests tho.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49026 - Posted: 21 Feb 2018 | 10:45:20 UTC - in response to Message 49024.

That is normal. It should move on within 30 minutes or so.

Sadly, it was still stuck at 1.098% after 10 hours, so I've tossed it back into the pot.

I use BoincTasks running on a Win7 machine to manage BOINC running on my Ubuntu machines over the LAN. Maybe it would work on the Win10 machine to your Linux running on the same machine? I can give the folder and file locations for my case if it would help.

I'm fine with the LAN stuff, thanks - I've got 8 machines all summarised on an ancient tool called 'BoincView', a predecessor of BoincTasks. For this experiment, I'll probably need to split the port numbers: I have had two separate Windows clients running on the same machine, with a different Manager for each - I'll need to look back at my notes for doing that.


My CMD command for starting a 2nd client with another port.

"c:\Program Files\BOINC\boinc.exe" --allow_multiple_clients --redirectio --detach_console --gui_rpc_port 31418 --dir C:\ProgramData\BOINC2

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49027 - Posted: 21 Feb 2018 | 12:00:50 UTC - in response to Message 49026.

And mine to start a second Manager to attach to that client :-)

start D:\BOINC\boincmgr.exe /m /n 127.0.0.1 /g 31418 /p password

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49030 - Posted: 21 Feb 2018 | 12:47:07 UTC

Don't think I've seen a cmd line to start the MGR. I just don't have any BOINC MGRs open anymore. I did make shortcuts in my Win7 machine to start them though. Good to know.

Profile Dingo
Avatar
Send message
Joined: 1 Nov 07
Posts: 14
Credit: 17,375,251
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49032 - Posted: 21 Feb 2018 | 13:02:20 UTC
Last modified: 21 Feb 2018 | 13:04:35 UTC

So 3 of the new tasks started and ran for varies times from 4655 minutes and using 4 cores each but had a finish time in the days not hours.

I aborted the tasks and the start of the stderr is the same as the first lot of tasks.

Name c00483_00544-DOMINIK_QMML2_m0000001127-0-1-RND6440_0
Workunit 13223873
Created 21 Feb 2018 | 10:32:07 UTC
Sent 21 Feb 2018 | 11:36:04 UTC
Received 21 Feb 2018 | 12:56:21 UTC
Server state Over
Outcome Computation error
Client state Aborted by user
Exit status 203 (0xcb) EXIT_ABORTED_VIA_GUI
Computer ID 453402
Report deadline 26 Feb 2018 | 11:36:04 UTC
Run time 4,655.70
CPU time 0.16
Validate state Invalid
Credit 0.00
Application version Quantum Chemistry, beta test v3.19 (mt)
Stderr output
<core_client_version>7.9.0</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
22:39:02 (9748): c:\windows\sysnative\wsl.exe exited; CPU time 0.140625
22:39:02 (9748): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

</stderr_txt>



Should I have left them run, would they have finished in hours ???? I aborted them because the Task Manager did not show any cPU usage when they were running.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49033 - Posted: 21 Feb 2018 | 13:12:24 UTC - in response to Message 49032.
Last modified: 21 Feb 2018 | 13:27:20 UTC

So 3 of the new tasks started and ran for varies times from 4655 minutes and using 4 cores each but had a finish time in the days not hours.

The estimated times have become all messed up in that last couple of days. They are OK on my old Linux machines that have been running for a while, but on a new Linux machine (i7-4770 running 2 cores per WU) that I attached on 18 February they are showing 3 days 8 hours, even though they really take only about 2 to 4 hours. The estimates should have started to correct by now, but I don't see any movement yet.

I have this set, as I do for all my Linux machines:
<cc_config>
<options>
<rec_half_life_days>1.000000</rec_half_life_days>
</options>
</cc_config>


EDIT:
They start to correct after they run for a while though, probably because I also include this in the app_config.xml:
<app>
<name>QC</name>
<max_concurrent>2</max_concurrent>
<fraction_done_exact/>
</app>

But it takes several tens of minutes before they change much even then.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 49034 - Posted: 21 Feb 2018 | 14:03:00 UTC - in response to Message 49033.

Some cumulative answers...

1. I expect WUs to run for ~3 hours, like the Linux ones.
2. Time estimates are done by BOINC with its obscure adaptive algorithms, please don't take them seriously.
3. If the WU hangs, feel free to kill it. It should be visible in the task manager as "Python"
4. The "sysnative" path is correct (only the wrapper sees it).

Jacob Klein
Send message
Joined: 11 Oct 08
Posts: 1089
Credit: 1,366,642,764
RAC: 732,345
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49035 - Posted: 21 Feb 2018 | 14:19:06 UTC

PROBLEM:

I have my preferences set to NOT receive these tasks, yet the scheduler keeps sending them. As you can see below, I've set to only run selected applications, but NOT the Quantum Chemistry ones. And I've made sure to set the "Accept work from other applications?" to No. Yet I keep receiving "Quantum Chemistry, beta test 3.19 (mt)" tasks.

Can you please fix it?

Run only the selected applications ACEMD short runs (2-3 hours on fastest card): yes
ACEMD long runs (8-12 hours on fastest GPU): yes
ACEMD Beta: yes
Quantum Chemistry (CPU): no
Quantum Chemistry (CPU, beta): no
Python Runtime : yes
If no work for selected applications is available, accept work from other applications? no

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 49036 - Posted: 21 Feb 2018 | 14:29:18 UTC - in response to Message 49035.

Perhaps the "accept beta work" is overriding the individual selections?

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49037 - Posted: 21 Feb 2018 | 14:35:30 UTC - in response to Message 49036.

Should there be input.zip in the slot folder along with output.zip file?

I see it in job_windows_319.xml but not in the folder.

Looks like something is getting stuck after running:

<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>bash miniconda-installer -b -u -p /tmp/gpugrid_miniconda</command_line>
<stdout_filename>miniconda-installer_wrapper_task.out</stdout_filename>
<time_limit>360</time_limit>
<weight>1</weight>
</task>

The next part starts to run but no python task appears in task manager.

Nothing else happens after this is runs:

14:24:48 (10432): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49038 - Posted: 21 Feb 2018 | 14:44:39 UTC - in response to Message 49037.

Should there be a value input instead of "$NTHREADS" in the xml below?



<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>bash miniconda-installer -b -u -p /tmp/gpugrid_miniconda</command_line>
<stdout_filename>miniconda-installer_wrapper_task.out</stdout_filename>
<time_limit>360</time_limit>
<weight>1</weight>
</task>

<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>/tmp/gpugrid_miniconda/bin/python pre_script.py</command_line>
<stdout_filename>pre_script_wrapper_task.out</stdout_filename>
<setenv>NTHREADS=$NTHREADS</setenv>
<setenv>PATH=/tmp/gpugrid_miniconda/bin:/usr/bin</setenv>
<checkpoint_filename>fraction_done.tmp</checkpoint_filename>
<fraction_done_filename>fraction_done.tmp</fraction_done_filename>
<weight>90</weight>
</task>


<unzip_input>
<zipfilename>input.zip</zipfilename>
</unzip_input>

<zip_output>
<zipfilename>output.zip</zipfilename>
<filename>.+</filename>
</zip_output>

</job_desc>

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49039 - Posted: 21 Feb 2018 | 15:17:38 UTC - in response to Message 49033.

They are OK on my old Linux machines that have been running for a while, but on a new Linux machine (i7-4770 running 2 cores per WU) that I attached on 18 February they are showing 3 days 8 hours, even though they really take only about 2 to 4 hours. The estimates should have started to correct by now, but I don't see any movement yet.

There is movement now. The estimates are 3 days 11 hours. We are heading in the wrong direction.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49040 - Posted: 21 Feb 2018 | 15:26:37 UTC - in response to Message 49039.

They are OK on my old Linux machines that have been running for a while, but on a new Linux machine (i7-4770 running 2 cores per WU) that I attached on 18 February they are showing 3 days 8 hours, even though they really take only about 2 to 4 hours. The estimates should have started to correct by now, but I don't see any movement yet.

There is movement now. The estimates are 3 days 11 hours. We are heading in the wrong direction.

If the progress is 1%, and it's taken 1 hour to get there, BOINC will be thinking that the full task will take another 99 hours.

If progress is still 1% after 3 hours, BOINC will be thinking it will take another 297 hours...

There's smoothing in place so you won't see exactly those figures, but they do depend critically on the progress% - and until that starts ticking up, the 'Remaining' estimate will go to hell in a handcart.

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49041 - Posted: 21 Feb 2018 | 15:51:43 UTC - in response to Message 49040.
Last modified: 21 Feb 2018 | 15:52:50 UTC

There's smoothing in place so you won't see exactly those figures, but they do depend critically on the progress% - and until that starts ticking up, the 'Remaining' estimate will go to hell in a handcart.

Here is what I see:

GPUGRID 3.18 Quantum Chemistry (mt) c00212_00275-DOMINIK_QMML2_m0000010593-0-1-RND0323_0 02:45:31 (05:28:27) 100.00 80.220 00:40:48 2/26/2018 3:14:16 AM 2C Running High P. [12] 00:01:55 1200.89 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00097_00124-DOMINIK_QMML2_m0000010563-0-1-RND6016_0 03:10:53 (06:18:48) 100.00 79.179 00:50:11 2/26/2018 3:14:16 AM 2C Running High P. [15] 00:14:28 260.55 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00062_00095-DOMINIK_QMML2_m0000010563-0-1-RND1634_0 00:37:30 (01:14:08) 100.00 11.510 04:48:24 2/26/2018 3:14:16 AM 2C Running High P. [2] 00:03:53 1143.13 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00166_00284-DOMINIK_QMML2_m0000010567-0-1-RND8911_0 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 3:14:16 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00129_00181-DOMINIK_QMML2_m0000010569-0-1-RND2974_0 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 3:14:16 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00189_00254-DOMINIK_QMML2_m0000010569-0-1-RND5515_0 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 3:14:16 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00000_00024-DOMINIK_QMML2_m0000001065-0-1-RND0158_4 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00136_00178-DOMINIK_QMML2_m0000010648-0-1-RND4003_0 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00400_00424-DOMINIK_QMML2_m0000001050-0-1-RND9096_6 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00891_00915-DOMINIK_QMML2_m0000001086-0-1-RND4522_1 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00098_00122-DOMINIK_QMML2_m0000001082-0-1-RND9163_2 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00748_00772-DOMINIK_QMML2_m0000001054-0-1-RND4655_4 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00025_00048-DOMINIK_QMML2_m0000001017-0-1-RND4562_3 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC
GPUGRID 3.18 Quantum Chemistry (mt) c00468_00492-DOMINIK_QMML2_m0000001042-0-1-RND7395_6 - (-) 0.00 0.000 03d,11:39:04 2/26/2018 10:10:26 AM 2C Ready to start 0.00 MB i7-4770-PC


There are three running at a time. They have been in "high priority" mode for a couple of days, since the remaining time estimates are so long. I may detach and try again. Something is wrong somewhere. I have run them on enough machines by now to know this is different somehow.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49045 - Posted: 21 Feb 2018 | 16:01:01 UTC - in response to Message 49041.

I don't see your progress %ages in that table - what are they showing?

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49046 - Posted: 21 Feb 2018 | 16:06:59 UTC - in response to Message 49045.

I don't see your progress %ages in that table - what are they showing?

They are these numbers (right after the 100.00% CPU usage):
80.220
79.179
11.510

I wanted to post a PNG of BoincTasks, but couldn't find an easy way to do it.


Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49049 - Posted: 21 Feb 2018 | 16:29:23 UTC - in response to Message 49046.

You have to upload the image to a hosting site - I'm using imgur at the moment, after the first two threw me out (blocked image display on third-party sites). Then, use [img] tags to display the external image.

The other key data controlling estimated remaining run time for your host is http://www.gpugrid.net/host_app_versions.php?hostid=357332 - that's used in the "obscure adaptive algorithms" that Toni mentioned.

The key figure is the 'Average processing rate' (APR) for the version you're currently running - 3.18 - which should be a recent averaged speed expressed in GigaFlops. 0.38 feels incredibly slow for a decent 8-core i7 - that will be the reason for your 3+ day estimates for unstarted tasks.

BOINC takes account of the declared <rsc_fpops_est> value for the tasks and the time taken - I believe there's some doubt whether it takes accurate account of the CPU time for all cores combined for mt tasks. If you divide <rsc_fpops_est> (units: floating point operations, a dimensionless number) by APR (10^9 flops per second), you should get the expected time in seconds.

captainjack
Send message
Joined: 9 May 13
Posts: 138
Credit: 951,578,780
RAC: 248,214
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49051 - Posted: 21 Feb 2018 | 17:49:52 UTC

Tried my first QC_Beta task this morning. It has been running for 1.5 hours and shows 11% complete. It looks like it is only running on one CPU (thread) even though it is allotted 4 CPU's (threads). Task Manager shows one thread in use and the "top" command in Ubuntu also shows one thread in use. I will let it keep running and see what happens.

Please let me know if you need any additional information.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49052 - Posted: 21 Feb 2018 | 18:29:03 UTC
Last modified: 21 Feb 2018 | 18:39:29 UTC

I'm trying another one, but again stuck at 1.098%. No sign of Python in the Windows task list:



I'll look through the commands in the job.xml file to see if I can work out what's going wrong.

The first task in the job is

<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>bash miniconda-installer -b -u -p /tmp/gpugrid_miniconda</command_line>
<stdout_filename>miniconda-installer_wrapper_task.out</stdout_filename>
<time_limit>360</time_limit>
<weight>1</weight>
</task>


I would expect to see a file "miniconda-installer_wrapper_task.out" somewhere in the rootfs, but Windows can't find it. Bash is running, though not as a sub-task from BOINC. I'll stop and think again for a while.

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49053 - Posted: 21 Feb 2018 | 18:44:33 UTC - in response to Message 49052.


The first task in the job is

<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>bash miniconda-installer -b -u -p /tmp/gpugrid_miniconda</command_line>
<stdout_filename>miniconda-installer_wrapper_task.out</stdout_filename>
<time_limit>360</time_limit>
<weight>1</weight>
</task>


I would expect to see a file "miniconda-installer_wrapper_task.out" somewhere in the rootfs, but Windows can't find it. Bash is running, though not as a sub-task from BOINC. I'll stop and think again for a while.



It's in the \boinc\slots\# folder for that task.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49054 - Posted: 21 Feb 2018 | 19:09:30 UTC - in response to Message 49053.

So it is, and it seems to have completed successfully.

The next task is

<task>
<application>c:\windows\sysnative\wsl.exe</application>
<command_line>/tmp/gpugrid_miniconda/bin/python pre_script.py</command_line>
<stdout_filename>pre_script_wrapper_task.out</stdout_filename>
<setenv>NTHREADS=$NTHREADS</setenv>
<setenv>PATH=/tmp/gpugrid_miniconda/bin:/usr/bin</setenv>
<checkpoint_filename>fraction_done.tmp</checkpoint_filename>
<fraction_done_filename>fraction_done.tmp</fraction_done_filename>
<weight>90</weight>
</task>

File 'pre_script_wrapper_task.out' exists, but is empty.

The task seems to have checkpointed once:

<checkpoint_cpu_time>0.015625</checkpoint_cpu_time>
<checkpoint_elapsed_time>127.046558</checkpoint_elapsed_time>
<fraction_done>0.010989</fraction_done>

but made no progress since then. There's no 'fraction_done.tmp' file.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 732
Credit: 4,285,282
RAC: 0
Level
Ala
Scientific publications
watwatwatwat
Message 49055 - Posted: 21 Feb 2018 | 19:21:38 UTC - in response to Message 49054.
Last modified: 21 Feb 2018 | 19:34:10 UTC

Do you have anything useful in stderr.txt or *.out files? While the app is running, it should create output/* subdirectories in the slot directory, and update the fraction_done regularly.

The Ubuntu shell should allow you to see additional temporary files, which are created in /tmp (e.g. ls -l /tmp) and check running processes (e.g. top).

The 1-not-4 threads may a problem with environment variables to be passed from Windows/Boinc/Wrapper to Linux processes. As Juha pointed out it differs between W10 releases. Which means that run times will be longer. I wish I saw at least one success...

Profile Chilean
Avatar
Send message
Joined: 8 Oct 12
Posts: 86
Credit: 151,726,480
RAC: 165,117
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49056 - Posted: 21 Feb 2018 | 19:38:56 UTC - in response to Message 49013.

NO NO NO. VBox apps receive so much less support. It's a waste of CPU overhead, memory and bandwidth to run. If the admin wants processing power they will write a native app. Vbox just pushes the burden to donors who will just go to another project due to the issues vbox causes.

I am currently running LHC/RNA on one system but it is far from my most supported project due the vbox requirements.


Hmmmm, I mean yeah vLHC is a hassle, but so is WSL.

Unless WSL doesn't have as much overhead as virtualizing a whole machine.

I wonder if LHC could tap into WSL if this were the case?
____________

Bri
Send message
Joined: 17 Nov 16
Posts: 7
Credit: 152,438,020
RAC: 0
Level
Ile
Scientific publications
watwat
Message 49057 - Posted: 21 Feb 2018 | 19:45:58 UTC - in response to Message 49055.

[quote]Do you have anything useful in stderr.txt or *.out files? While the app is running, it should create output/* subdirectories in the slot directory, and update the fraction_done regularly.

This is all I get in the stderr.txt file:

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
19:14:02 (2400): c:\windows\sysnative\wsl.exe exited; CPU time 0.015625
19:14:02 (2400): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

No output folder is created in the slot directory and pre_script_wrapper_task.out is empty.

The only way I can get the python task to appear in task manager is to run the following from the command prompt after changing the current directory to the slot directory of the project..

c:\windows\sysnative\wsl.exe /tmp/gpugrid_miniconda/bin/python pre_script.py

After running this an output folder is created and the task progress will go up to 94% but then stop. I guess at that point the rest of the script is missing to send the completed work due to running from cmd.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49059 - Posted: 21 Feb 2018 | 19:58:20 UTC - in response to Message 49055.
Last modified: 21 Feb 2018 | 20:00:11 UTC

I see the same in stderr.txt

There is an 'output.zip' file in the slot directory, which contains a (plaintext) soft-link to a task output file in the project directory - but the real output file hasn't been created.

I can see the whole gpugrid_miniconda directory in tmp - it contains 18,091 Files, 1,155 Folders occupying 343 MB

Jim1348
Send message
Joined: 28 Jul 12
Posts: 614
Credit: 1,199,451,727
RAC: 134,958
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49061 - Posted: 21 Feb 2018 | 20:37:06 UTC - in response to Message 49049.

The key figure is the 'Average processing rate' (APR) for the version you're currently running - 3.18 - which should be a recent averaged speed expressed in GigaFlops. 0.38 feels incredibly slow for a decent 8-core i7 - that will be the reason for your 3+ day estimates for unstarted tasks.

That is undoubtedly unrealistic. But the measurements for my other Haswell and Ivy Bridge machines aren't much different. For example, one of my i7-3770 machines gives a value of 0.29308704654104, but it is showing the normal estimated times and completes the work units at the normal rate (2 to 4 hours currently).
http://www.gpugrid.net/host_app_versions.php?hostid=448995

It is probably some glitch on my machine, and I will detach when these are completed and try again.
Thanks for the image tip. I could use a good hosting service, and the presentation is rather obscure without it.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49062 - Posted: 21 Feb 2018 | 20:42:28 UTC - in response to Message 49061.

Thanks for the image tip. I could use a good hosting service, and the presentation is rather obscure without it.

This project is using an old version of the BOINC website code, which can't render images with an https url.

Imgur made an https direct link for the task manager view I posted, but I just knocked off the 's' and all was good.

captainjack
Send message
Joined: 9 May 13
Posts: 138
Credit: 951,578,780
RAC: 248,214
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49064 - Posted: 21 Feb 2018 | 23:28:50 UTC

Task finished with this error message:

zip warning: Permission denied
boinc_zip() failed: 18


Link to the task https://www.gpugrid.net/result.php?resultid=17133298

Task ran for 6.94 hours which may be about right considering it was running on one thread only.

Rantanplan
Send message
Joined: 22 Jul 11
Posts: 166
Credit: 129,564,063
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49074 - Posted: 22 Feb 2018 | 11:37:40 UTC

ok, after yesterday , the boinc client won`t load
beta wus anymore.

Normal wus , no problem.

Profile mg13 [HWU]
Avatar
Send message
Joined: 18 Nov 09
Posts: 3
Credit: 93,765
RAC: 0
Level

Scientific publications
wat
Message 49097 - Posted: 24 Feb 2018 | 17:38:49 UTC

Hello to all,
I executed the instructions of the first message to the letter and looking at the boinc manager, I noticed that I downloaded the WU QC with an estimated time of 4 minutes.
In the first minute of processing the percentage of advancement was steady at zero, after which and the match from 30% up to 41.5% in a minute, then it went to 1.098% and from about 9 hours that remains steady there.
He blew me another 2 WUs and they did the same thing.
What I do? I abort the WU?

Thanks and sorry for my english but not knowing I used a translator to write the message.
____________

captainjack
Send message
Joined: 9 May 13
Posts: 138
Credit: 951,578,780
RAC: 248,214
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 49098 - Posted: 25 Feb 2018 | 0:02:06 UTC

I just ran another Windows QC beta task and got the same result as the previous 3 that I ran. It only ran on one thread and abended with the following messages:

<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>

CondaValueError: prefix already exists: /tmp/gpugrid_miniconda/envs/qmml

er: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
10:17:01 (6328): c:\windows\sysnative\wsl.exe exited; CPU time 0.046875
10:17:01 (6328): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)
15:13:28 (6328): c:\windows\sysnative\wsl.exe exited; CPU time 0.046875
zip warning: Permission denied
boinc_zip() failed: 18


At this point, I don't see any value in continuing to run these beta tasks until the known issues are fixed. Please let us know when the issues are addressed and I will turn them back on.

rigglybob
Send message
Joined: 24 Oct 17
Posts: 2
Credit: 9,118,925
RAC: 0
Level
Ser
Scientific publications
wat
Message 49102 - Posted: 26 Feb 2018 | 9:52:34 UTC - in response to Message 49098.

Another here sitting on 1.098%, checkpointed once, no output folder. Looks like it trys multiple times to unzip something during the first minute or so as see a zip prog keep popping up in task manager, so perhaps a permissions issue?

biodoc
Send message
Joined: 26 Aug 08
Posts: 121
Credit: 826,678,261
RAC: 28,108
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49103 - Posted: 26 Feb 2018 | 11:48:11 UTC

It could be a python issue.

Try:

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb
sudo dpkg -i python-support_1.0.15_all.deb

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49104 - Posted: 26 Feb 2018 | 12:54:29 UTC - in response to Message 49103.

For those of us following along with the "running Linux under Windows" experiemnt, installing an unknown software package from an unknown (to us) source, with no word of explanation, feels uncomfortable.

I've followed your url from Librarian as far as python-support package in Ubuntu, which is more comfortable - not least, because the page is security-signed by the same company which was responsible for the Ubuntu download in the Windows store. But it still only has this explanation:

This package contains the 'update-python-modules' script, which takes care of byte-compilation of Python-only modules.

Which leaves me none the wiser. Could you give us a few words of explanation about what this package does, and why you think it might address our current problem with this app? Have you tried it yourself?

I have half-a-dozen tasks lined up for testing: the first has reached 1.098% (before I paused it as a waste of time), and I can use the others as needed. But I need to know what I'm doing, and have some confidence that my test will produce a useful answer, either positive or negative. What am I looking for?

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49105 - Posted: 26 Feb 2018 | 13:05:17 UTC - in response to Message 49103.

It could be a python issue.

Try:

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb
sudo dpkg -i python-support_1.0.15_all.deb


If this is the issue then shouldn't everyone have the error as the Win10 WSL app should have the same distribution installed?

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49106 - Posted: 26 Feb 2018 | 13:12:29 UTC - in response to Message 49105.

It could be a python issue.

Try:

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb
sudo dpkg -i python-support_1.0.15_all.deb

If this is the issue then shouldn't everyone have the error as the Win10 WSL app should have the same distribution installed?

I think everyone trying to run the W10 app does have the same problem. There are plenty of success reports for the native Linux version, but I don't think I've seen anyone report a home run under W10.

biodoc
Send message
Joined: 26 Aug 08
Posts: 121
Credit: 826,678,261
RAC: 28,108
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49107 - Posted: 26 Feb 2018 | 14:53:41 UTC - in response to Message 49105.

It could be a python issue.

Try:

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb
sudo dpkg -i python-support_1.0.15_all.deb


If this is the issue then shouldn't everyone have the error as the Win10 WSL app should have the same distribution installed?


It sounds like everyone is downloading Ubuntu from the microsoft store. That's Ubuntu xenial xerus which has python 3, I believe, so installing python-support may solve the problem of running python version 2.xx based apps.

python-support: "automated rebuilding support for Python modules"

It's worth a try.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 883
Credit: 1,726,038,170
RAC: 1,104,572
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49109 - Posted: 26 Feb 2018 | 16:03:52 UTC - in response to Message 49107.

OK, I gave it a try. Needed to 'sudo' the wget - failed without, no write permission. Seemed to install OK once stored locally - unpack, setting up, processing triggers, return to prompt.

But GPUGrid Beta behaviour is exactly the same. Started new task: runs for ~2 minutes, creates wrapper_checkpoint.txt

1 0.046875 117.000000

Creates empty pre_script_wrapper_task.out

Reverts progress %age to 1.098

Updates stderr.txt

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
15:46:42 (2688): c:\windows\sysnative\wsl.exe exited; CPU time 0.046875
15:46:42 (2688): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

and shows no CPU usage, no visible progress from that point forward. Putting it back to sleep so the CPU can do production work.

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49110 - Posted: 26 Feb 2018 | 16:36:18 UTC

So the procedure from the GPUGrid admin needs to be updated. :( Or there is just something else fundamentally wrong.

The server status page doesn't have the error rate of anything but the non-beta GPU tasks to know if anyone has completed one.

biodoc
Send message
Joined: 26 Aug 08
Posts: 121
Credit: 826,678,261
RAC: 28,108
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49112 - Posted: 26 Feb 2018 | 17:54:01 UTC - in response to Message 49109.

OK, I gave it a try. Needed to 'sudo' the wget - failed without, no write permission. Seemed to install OK once stored locally - unpack, setting up, processing triggers, return to prompt.

But GPUGrid Beta behaviour is exactly the same. Started new task: runs for ~2 minutes, creates wrapper_checkpoint.txt

1 0.046875 117.000000

Creates empty pre_script_wrapper_task.out

Reverts progress %age to 1.098

Updates stderr.txt

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
15:46:42 (2688): c:\windows\sysnative\wsl.exe exited; CPU time 0.046875
15:46:42 (2688): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

and shows no CPU usage, no visible progress from that point forward. Putting it back to sleep so the CPU can do production work.


Thanks for trying. I'd try to mess with it but I'd rather stay on native linux rather than boot back to Windows.

zombie67 [MM]
Avatar
Send message
Joined: 16 Jul 07
Posts: 168
Credit: 272,793,232
RAC: 13
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49113 - Posted: 28 Feb 2018 | 23:13:51 UTC - in response to Message 49107.

It sounds like everyone is downloading Ubuntu from the microsoft store. That's Ubuntu xenial xerus which has python 3, I believe, so installing python-support may solve the problem of running python version 2.xx based apps.

python-support: "automated rebuilding support for Python modules"

It's worth a try.


According to this:

https://askubuntu.com/questions/766169/why-no-more-python-support-in-16-04

According to one answer on this bug report: bugs.launchpad.net/ubuntu/+source/python-support/+bug/1577172 , "python-support was removed from Debian and later from Ubuntu. Please use dh_python2 from dh-python package instead."

____________
Dublin, California
Team: SETI.USA

PappaLitto
Send message
Joined: 21 Mar 16
Posts: 399
Credit: 2,736,118,092
RAC: 1,016,929
Level
Phe
Scientific publications
watwat
Message 49114 - Posted: 1 Mar 2018 | 2:09:34 UTC - in response to Message 49113.

It sounds like everyone is downloading Ubuntu from the microsoft store. That's Ubuntu xenial xerus which has python 3, I believe, so installing python-support may solve the problem of running python version 2.xx based apps.

python-support: "automated rebuilding support for Python modules"

It's worth a try.


According to this:

https://askubuntu.com/questions/766169/why-no-more-python-support-in-16-04

According to one answer on this bug report: bugs.launchpad.net/ubuntu/+source/python-support/+bug/1577172 , "python-support was removed from Debian and later from Ubuntu. Please use dh_python2 from dh-python package instead."

Have we gotten a confirmed success story from downloading python repositories?

mmonnin
Send message
Joined: 2 Jul 16
Posts: 173
Credit: 288,961,689
RAC: 97,122
Level
Asn
Scientific publications
wat
Message 49117 - Posted: 1 Mar 2018 | 12:52:32 UTC

I checked all the posters in this thread and only found 1 user with completed tasks and that was Rantanplan with their "Linux 4.4.0-43-Microsoft" distro. Of course Toni has their computers hidden.

http://www.gpugrid.net/results.php?hostid=467700

rigglybob
Send message
Joined: 24 Oct 17
Posts: 2
Credit: 9,118,925
RAC: 0
Level
Ser
Scientific publications
wat
Message 49119 - Posted: 1 Mar 2018 | 15:46:11 UTC
Last modified: 1 Mar 2018 | 15:54:20 UTC

Tried installing Python support - no difference.
Tried installing Python 2.7 - no difference.

Contents of the slot directory once it's stopped on 1.098% is:

boinc_lockfile
boinc_task_state.xml
init_data.xml
initial_file_list
job.xml
m0000001148-c00041-00061_n00020_QMML-Batch.h5
miniconda-installer
miniconda-installer_wrapper_task.out (which looks ok)
output.zip
pre_script.py
pre_script_wrapper_task,out (which is empty)
psi4.in
run-qmml.py
stderr.txt (just same content as others have posted - see below)
wrapper_26198_windows_315.exe
wrapper_checkpoint.txt

All that's in the stderr is:
Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
15:37:47 (10508): c:\windows\sysnative\wsl.exe exited; CPU time 0.015625
15:37:47 (10508): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

Happy to poke around if someone can tell me where to poke!
Edit: Just noticed too that elapsed time of the task is taking 2 seconds for every second counted.[/b]

biodoc
Send message
Joined: 26 Aug 08
Posts: 121
Credit: 826,678,261
RAC: 28,108
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49131 - Posted: 2 Mar 2018 | 13:08:51 UTC - in response to Message 49117.

I checked all the posters in this thread and only found 1 user with completed tasks and that was Rantanplan with their "Linux 4.4.0-43-Microsoft" distro. Of course Toni has their computers hidden.

http://www.gpugrid.net/results.php?hostid=467700


Is "Linux 4.4.0-43-Microsoft" for real?

Could someone check with:

uname -r

or

cat /proc/version

Carlos Augusto Engel
Send message
Joined: 5 Jun 09
Posts: 37
Credit: 2,017,004,354
RAC: 895,263
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49132 - Posted: 2 Mar 2018 | 14:40:29 UTC - in response to Message 49131.

Yes, that is correct.
Linux 4.4.0-43-Microsoft #1-Microsoft.
____________

Bedrich Hajek
Send message
Joined: 28 Mar 09
Posts: 345
Credit: 4,152,302,659
RAC: 1,690,862
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49140 - Posted: 4 Mar 2018 | 4:03:29 UTC

Milkyway has a multicore app that works in Windows 10.

Name de_nbody_1_31_2018_v168_20k__diff_seedruns_3_1517255210_197000_3
Workunit 1573441110
Created 4 Mar 2018, 3:21:31 UTC
Sent 4 Mar 2018, 3:24:19 UTC
Report deadline 16 Mar 2018, 3:24:19 UTC
Received 4 Mar 2018, 3:42:52 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x0)
Computer ID 714509
Run time 9 min 29 sec
CPU time 24 min 57 sec
Validate state Valid
Credit 18.82
Device peak FLOPS 12.58 GFLOPS
Application version MilkyWay@Home N-Body Simulation v1.68 (mt)
Peak working set size 11.32 MB
Peak swap size 8.41 MB
Peak disk usage 2.07 MB
Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
<search_application> milkyway_nbody 1.68 Windows x86_64 double OpenMP, Crlibm </search_application>
Using OpenMP 3 max threads on a system with 12 processors
<search_likelihood>-7.056300171358846</search_likelihood>
22:42:44 (8520): called boinc_finish(0)

</stderr_txt>
]]>


http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=2274736735


Here is the stupid question. Can something like that be adapted for QC here? If not, why not?


Profile mg13 [HWU]
Avatar
Send message
Joined: 18 Nov 09
Posts: 3
Credit: 93,765
RAC: 0
Level

Scientific publications
wat
Message 49141 - Posted: 4 Mar 2018 | 14:38:34 UTC
Last modified: 4 Mar 2018 | 14:39:50 UTC

Hello everyone, I made this discovery by detaching ubuntu and leaving WSL active that the WU turned for 6 minutes until you get up to almost 90%, after which it did not go further and progress and returned to the usual 1.098%.

This is the content of the stderr.txt file:

15:10:41 (624): wrapper (7.9.26016): starting
15:10:41 (624): wrapper: running c: \ windows \ sysnative \ wsl.exe (bash miniconda-installer -b -u -p / tmp / gpugrid_miniconda)
15:16:44 (624): task c: \ windows \ sysnative \ wsl.exe reached time limit 360
15:16:44 (624): wrapper: running c: \ windows \ sysnative \ wsl.exe (/ tmp / gpugrid_miniconda / bin / python pre_script.py)

I hope I have been helpful.

Profile Chilean
Avatar
Send message
Joined: 8 Oct 12
Posts: 86
Credit: 151,726,480
RAC: 165,117
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwat
Message 49153 - Posted: 10 Mar 2018 | 21:51:49 UTC

I too get the 1.098% "bug".

stderr:

Please run using "bash" or "sh", but not "." or "source"\nminiconda-installer: line 14: return: can only `return' from a function or sourced script
Python 3.6.3 :: Anaconda, Inc.
18:43:38 (12812): c:\windows\sysnative\wsl.exe exited; CPU time 0.046875
18:43:38 (12812): wrapper: running c:\windows\sysnative\wsl.exe (/tmp/gpugrid_miniconda/bin/python pre_script.py)

____________

Profile [B@P] Daniel
Send message
Joined: 17 Sep 16
Posts: 5
Credit: 330,749,014
RAC: 904,917
Level
Asp
Scientific publications
watwat
Message 49399 - Posted: 4 May 2018 | 21:18:38 UTC - in response to Message 48980.

Dears,

have a test QC app for Windows 10. QC relies on the PSI4 software which does not have a Windows version. Therefore, we'll rely on WSL (Windows Subsystem Linux - a nice lightweight Linux-like environment to run native apps inside Windows), provided by Microsoft in the recent version of Windows 10.

Did you try to rebuild PSI4 using MinGW or Cygwin? After doing so it should work on earlier Windows versions too.
____________

Post to thread

Message boards : Multicore CPUs : Experimental QC app for Windows 10