Multiple MLDS.exe apps running

Questions and Answers : Issue Discussion : Multiple MLDS.exe apps running
Message board moderation

To post messages, you must log in.

AuthorMessage
Peter

Send message
Joined: 22 Sep 20
Posts: 5
Credit: 108,820
RAC: 0
Message 1358 - Posted: 12 Sep 2021, 1:52:36 UTC

I have noticed that over a dozen copies (13 total) of the MLDS.exe app are running on my Win 7 machine even though there are currently no WUs in progress. There are WUs being reported as completed as recent as 11 am 9/11/21. Is this expected behavior. If I kill these tasks, will a new WU spawn a new app?
ID: 1358 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
swiftmallard
Avatar

Send message
Joined: 23 Sep 20
Posts: 24
Credit: 15,318,198
RAC: 1,992
Message 1359 - Posted: 12 Sep 2021, 12:48:10 UTC - in response to Message 1358.  

I have noticed that over a dozen copies (13 total) of the MLDS.exe app are running on my Win 7 machine even though there are currently no WUs in progress. There are WUs being reported as completed as recent as 11 am 9/11/21. Is this expected behavior. If I kill these tasks, will a new WU spawn a new app?

I would remove the project, restart Boinc, and then reattach. That typically solves issues like this.
ID: 1359 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 1360 - Posted: 13 Sep 2021, 1:12:43 UTC

Are they using up any CPU time? If they are, and the client says there's no WUs currently *running*, then there's an issue.

When you say there's "no WUs in progress" .. do you mean none currently running, but there are some that are partially complete but waiting for their turn again? or that there's no WUs even partially complete?

There are two reasons why I can think there may be stray executable threads sitting around. One is that the client (especially on windows) sometimes spawns threads which aren't actually use, so they're harmless, not using any compute power, and we just ignore them. The other is if the WU is in progress but the client suspends it for some reason (because it's time to run another project, for example), by default the client will keep the executable loaded in memory but paused.. so it again doesn't use any CPU resources, but does use up memory. This is done so that resuming is much easier. I think there's a BOINC client setting to change that default behavior and force suspended WUs to unload the exe from memory.

IF the client says that no mlc WUs are running or in progress, AND there are mlds processes still running and consuming CPU time, then that's a bug that needs to be fixed.
ID: 1360 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter

Send message
Joined: 22 Sep 20
Posts: 5
Credit: 108,820
RAC: 0
Message 1361 - Posted: 13 Sep 2021, 3:08:50 UTC - in response to Message 1360.  

I removed the MLC project, restarted the client and killed all the mlds.exe task that were running. I then reattached the MLC project. One WU has completed and a second is in progress. Two mlds.exe tasks appear in Task manager; one is showing CPU usage and varying amounts of memory in use., the other is showing 0 CPU % but varying amounts of memory. I guess it is possible these are both threads for the same WU. When reviewing the BOINC client event log, there are error messages corresponding to the completed WU. I have copied the latest set of messages. I will continue to monitor to verify that there are no lingering tasks.

9/12/2021 4:44:22 PM | MLC@Home | Starting task ParityModified-1630699508-15369-1_1
9/12/2021 9:56:41 PM | MLC@Home | Sending scheduler request: To fetch work.
9/12/2021 9:56:41 PM | MLC@Home | Requesting new tasks for CPU
9/12/2021 9:56:47 PM | MLC@Home | Scheduler request completed: got 1 new tasks
9/12/2021 9:56:47 PM | MLC@Home | Project requested delay of 31 seconds
9/12/2021 9:56:49 PM | MLC@Home | Started download of ParityModified-1630380152-25517-6
9/12/2021 9:56:50 PM | MLC@Home | Finished download of ParityModified-1630380152-25517-6
9/12/2021 10:12:38 PM | MLC@Home | [error] Can't rename output file slots/0/mlds.stderr to projects/www.mlcathome.org_mlcathome/ParityModified-1630699508-15369-1_1_r935545738_2: Error 32
9/12/2021 10:12:44 PM | MLC@Home | [error] Can't rename output file slots/0/mlds.stdout to projects/www.mlcathome.org_mlcathome/ParityModified-1630699508-15369-1_1_r935545738_3: Error 32
9/12/2021 10:12:50 PM | MLC@Home | Computation for task ParityModified-1630699508-15369-1_1 finished
9/12/2021 10:12:51 PM | MLC@Home | Starting task ParityModified-1630380152-25517-6_0
9/12/2021 10:12:52 PM | MLC@Home | Started upload of ParityModified-1630699508-15369-1_1_r935545738_0
9/12/2021 10:12:52 PM | MLC@Home | Started upload of ParityModified-1630699508-15369-1_1_r935545738_1
9/12/2021 10:12:55 PM | MLC@Home | Finished upload of ParityModified-1630699508-15369-1_1_r935545738_0
9/12/2021 10:12:55 PM | MLC@Home | Finished upload of ParityModified-1630699508-15369-1_1_r935545738_1
ID: 1361 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter

Send message
Joined: 22 Sep 20
Posts: 5
Credit: 108,820
RAC: 0
Message 1362 - Posted: 19 Sep 2021, 13:30:37 UTC - in response to Message 1361.  

I continue to see evidence of "ghost" occurrences of the MLDS app after computation of WUs has completed. There a six MLDS tasks with no CPU usage reported in Task Manager.
ID: 1362 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
swiftmallard
Avatar

Send message
Joined: 23 Sep 20
Posts: 24
Credit: 15,318,198
RAC: 1,992
Message 1363 - Posted: 19 Sep 2021, 16:46:22 UTC - in response to Message 1362.  

Your tasks page shows you have nothing in progress. If you delete these tasks, they shouldn’t show as aborted on your tasks list. If they do, there is a server issue somewhere and pianoman will want to know about it. If they are simply gone after deletion, wait a couple days before getting more work and see if any reappear. You may wish to note the unique task name before doing anything so you can see if it’s the same ones popping up.
ID: 1363 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter

Send message
Joined: 22 Sep 20
Posts: 5
Credit: 108,820
RAC: 0
Message 1365 - Posted: 24 Sep 2021, 0:06:37 UTC - in response to Message 1363.  

No active tasks and seven inactive instances of MLDS.EXE showing in Task Manager. I would have to conclude that they are not exiting properly at the conclusion of a WU.
ID: 1365 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan
Avatar

Send message
Joined: 5 Jul 20
Posts: 25
Credit: 348,811
RAC: 0
Message 1367 - Posted: 24 Sep 2021, 2:56:19 UTC - in response to Message 1365.  

Just saw same thing on one of my PC's from a task last run about 1 month sgo on this project.

Program was sitting idle using 0 cpu cycles but consumed the original allocated memory amount. It was running from the slots directory where BOINC started it.

I had to kill the running program with Windows Task Manager because even exiting BOINC did not stop it.

The slots folder still had all the files that were copied over to run the (long ago exited task) task. This might of been a task that crashed upon startup or errors out some time later.

To be safe, I deleted all the files in the slots directory before rebooting to apply the needed monthly Windows updates and so BOINC would realize the slots directory was unused upon its startup.

ID: 1367 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter

Send message
Joined: 22 Sep 20
Posts: 5
Credit: 108,820
RAC: 0
Message 1368 - Posted: 26 Sep 2021, 0:26:03 UTC - in response to Message 1367.  

After exiting BOINC and removing the mlds tasks from Task Manager, can I safely delete the slots folders that contain the mlds.exe for any slot that is currently not in use? Are there any other BOINC files that need to be updated?
ID: 1368 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan
Avatar

Send message
Joined: 5 Jul 20
Posts: 25
Credit: 348,811
RAC: 0
Message 1369 - Posted: 26 Sep 2021, 7:05:33 UTC - in response to Message 1368.  

After exiting BOINC and removing the mlds tasks from Task Manager, can I safely delete the slots folders that contain the mlds.exe for any slot that is currently not in use? Are there any other BOINC files that need to be updated?

Just to be safe, after exiting BOINC and before I deleted any unused files in the SLOTS folders I made a backup of the whole BOINCdata FOLDER and all the sub-folders to a empty flash drive using the Windows GUI COPY/PASTE commands.
* That way If I broke something I could restore BOINC to it's previous state after exiting BOINC and deleting the data folder contents then copying back the saved data.
* This will take a little bit of time depending on how large the data folder is.

I don't think there is any BOINC files that need to be updated, I didn't change anything.
Modifying and/or deleting any of the files in the data folder incorrectly can cause BOINC to be inoperable forcing you to delete the data folder and do a clean install of the BOINC program or restoring the data folder from you backup copy.

As I noted in my previous post all I did was DELETE THE CONTENTS OF THE PREVIOUSLY USED SLOTS FOLDER(S) after exiting.
BOINC should (and did in my case) delete any unused SLOTS folders when you restart it.

ID: 1369 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 1371 - Posted: 29 Sep 2021, 18:25:05 UTC

This is new behavior with the new v9.90+ client, right? Not with the old one?

v9.90 moved to using a "wrapper"... where there's BOINC-provided wrapper program that runs a a generic, unmodified binary to process the code. I'd say the majority of BOINC projects run the wrapper for their clients, we were a bit of an odd duck because we modified our client to use the BOINC API directly. However, there were issues with this and pytorch (both wanted to use the SIGALRM posix signal), so we moved off of that for the latest client. It's certainly possible that the wrapper is, sometimes, on windows, not cleaning up all its child threads. But since we didn't write the wrapper (literally we're using the binary from https://boinc.berkeley.edu/trac/wiki/WrapperApp), I think this might be worth opening a bug report on the main BOINC github repo.

:(
ID: 1371 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan
Avatar

Send message
Joined: 5 Jul 20
Posts: 25
Credit: 348,811
RAC: 0
Message 1372 - Posted: 29 Sep 2021, 19:53:44 UTC - in response to Message 1371.  
Last modified: 29 Sep 2021, 19:55:02 UTC

Mine goes back to almost the end of August (26 Aug 2021) which I reported in post 1340 which was a v9.90 task,

I have not run any since then, let me know if you want I can try to run one or two TEST v9.96 on my Windows 8.1 PC and/or on one of my Windows 7 PC's.to see what happens.

ID: 1372 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PDW

Send message
Joined: 1 Jul 20
Posts: 8
Credit: 25,000,139
RAC: 0
Message 1429 - Posted: 29 Nov 2021, 11:50:04 UTC - in response to Message 1371.  

This is new behavior with the new v9.90+ client, right? Not with the old one?

v9.90 moved to using a "wrapper"... where there's BOINC-provided wrapper program that runs a a generic, unmodified binary to process the code. I'd say the majority of BOINC projects run the wrapper for their clients, we were a bit of an odd duck because we modified our client to use the BOINC API directly. However, there were issues with this and pytorch (both wanted to use the SIGALRM posix signal), so we moved off of that for the latest client. It's certainly possible that the wrapper is, sometimes, on windows, not cleaning up all its child threads. But since we didn't write the wrapper (literally we're using the binary from https://boinc.berkeley.edu/trac/wiki/WrapperApp), I think this might be worth opening a bug report on the main BOINC github repo.

:(

Have found hundreds of mdls.exe applications in Task Manager on all Win 7 machines that have been running this app (v9.90) for several days.
The tasks aren't using CPU but each consume between 1k and 250k of memory and disk space.
The machines were running out of memory and/or BOINC disk space trashing this project and other project work.
The app is unfit for continuous use on Windows 7 uness you are rebooting every couple of days or prepared to regularly go and kill these ghost tasks in Task Manager.
ID: 1429 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Issue Discussion : Multiple MLDS.exe apps running

©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)