mldstest

Questions and Answers : Issue Discussion : mldstest
Message board moderation

To post messages, you must log in.

AuthorMessage
Sergey Kovalchuk

Send message
Joined: 1 Jul 20
Posts: 31
Credit: 123,959
RAC: 0
Message 466 - Posted: 11 Sep 2020, 17:17:02 UTC

a mldstest application appeared on the server
it is desirable to activate the choice in the settings
1 "run only selected applications" in preferences
2 mark the application or its version with the "beta test" flag
then the option "run beta test" will appear in preferences
ID: 466 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 469 - Posted: 12 Sep 2020, 18:57:34 UTC - in response to Message 466.  

a mldstest application appeared on the server
it is desirable to activate the choice in the settings
1 "run only selected applications" in preferences
2 mark the application or its version with the "beta test" flag
then the option "run beta test" will appear in preferences


Correct, We've created an mlds test application to roll out test WUs on there. However, the idea is that no real WUs will go on there, just testing new WUs, testing new clients there first, and seeing how new WUs perform on client systems.

I don't want to mark the application there as "beta", because the last time I did that with a new client, all non-beta users stopped receiving WUs until they checked "allow beta apps" in their preferences, so it clearly doesn't work the way I think it does, and I'm loathe to tempt fate again.

I agree there should be the option to choose not to run this app in preferences, and I'm surprised its not there automatically. We're looking into it.
ID: 469 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Sergey Kovalchuk

Send message
Joined: 1 Jul 20
Posts: 31
Credit: 123,959
RAC: 0
Message 471 - Posted: 12 Sep 2020, 20:30:14 UTC - in response to Message 469.  

I agree there should be the option to choose not to run this app in preferences, and I'm surprised its not there automatically. We're looking into it.


https://boinc.berkeley.edu/trac/wiki/AppFiltering

BOINC offers two mechanisms allowing users to control the set of apps for which they get jobs.
The first, lets you designate apps as "beta test", and lets users opt-in to getting jobs for these apps.
The second, "app filtering", lets users select apps individually.

App filtering is disabled by default. To enable it, add to your html/project/project_specific_prefs.inc:
define ('APP_SELECT_PREFS', true);


The app filtering user interface is part of project-specific preferences (i.e., the "Preferences for this project" link on their account page).
If the feature is enabled, users are shown:
• A list of apps and check boxes.
• A checkbox labeled "If no work for selected applications is available, accept work from other applications?"
ID: 471 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 472 - Posted: 12 Sep 2020, 22:05:34 UTC - in response to Message 471.  

I agree there should be the option to choose not to run this app in preferences, and I'm surprised its not there automatically. We're looking into it.


https://boinc.berkeley.edu/trac/wiki/AppFiltering

BOINC offers two mechanisms allowing users to control the set of apps for which they get jobs.
The first, lets you designate apps as "beta test", and lets users opt-in to getting jobs for these apps.
The second, "app filtering", lets users select apps individually.

App filtering is disabled by default. To enable it, add to your html/project/project_specific_prefs.inc:
define ('APP_SELECT_PREFS', true);


The app filtering user interface is part of project-specific preferences (i.e., the "Preferences for this project" link on their account page).
If the feature is enabled, users are shown:
• A list of apps and check boxes.
• A checkbox labeled "If no work for selected applications is available, accept work from other applications?"


Ahh, good, thanks for the pointer. We'll enable app filtering later tonight. It also looks like I can turn on beta testing too and it will impact only the mldstest app, not the main one.
.
We may do both, because eventually we'd like to have multiple (non-beta-testing) apps anyway, and I expect people to be able to filter by apps to their choosing.
ID: 472 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bozz4science

Send message
Joined: 9 Jul 20
Posts: 142
Credit: 11,536,204
RAC: 3
Message 490 - Posted: 17 Sep 2020, 16:17:58 UTC - in response to Message 472.  

Great effort! Love to this implemented and thanks as well to Sergey Kovalchuk for your valuable pointers.
ID: 490 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tex1954

Send message
Joined: 15 Sep 20
Posts: 2
Credit: 683,420
RAC: 0
Message 503 - Posted: 20 Sep 2020, 16:07:26 UTC

Speaking of tests... I recently got a TEST WU on my 6180 server board machine ID: 2943 and it seemed to complete properly on my end.

MLC@Home 9.60 Machine Learning Dataset Generator (test) rand0002-1599798878-26687-0_2 01d,02:23:33 (01d,02:14:00) 9/19/2020 8:01:10 PM 9/19/2020 8:19:10 PM 99.4 Reported: OK Lin-61XX 270.57 MB

However, it shows a validate error on another computer as well as the 2943 computer and we got ZERO points for it.

Considering the fact it took so long, also the fact it was a voluntary TEST WU, seems to me you could issue us a few points for using our systems to troubleshoot and validate your code.

WE pay for the parts and electricity anyways, would be nice to get some token so it doesn't seem as if we are paying for nothing.

Regards,

Tex1954
ID: 503 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 505 - Posted: 20 Sep 2020, 16:24:00 UTC - in response to Message 503.  

Correct, I'm working out the server side validation on the ds3 WUs. Client side appears to be fine, but server side is having some issues. It's hard to test these things outside of the full boinc framework, which is why the mldstest app exists. To limit exposure there are only a handful of WUs released on mldstest versus the main app.

I'm sorry you didn't receive credit for it, I'm trying to have the server re-validate existing WUs (for you and a few others), but I'm still debugging and can't 100% guarantee that you will. We try very hard to make sure this doesn't happen, but we're not perfect.

Future releases of mldstest clients will be marked as beta, so you'll have to explicitly opt in to test WUs that may not be stable.
ID: 505 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 462
Credit: 21,406,548
RAC: 0
Message 506 - Posted: 20 Sep 2020, 17:09:34 UTC

all mldstest apps are now marked as beta, which means you must explicitly enable the "Run test applications?" preference to receive them.

Please consider doing so, as testing really helps. But do so with the knowledge that things may break.
ID: 506 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tex1954

Send message
Joined: 15 Sep 20
Posts: 2
Credit: 683,420
RAC: 0
Message 513 - Posted: 22 Sep 2020, 12:04:45 UTC - in response to Message 505.  

Oh! So that why so few of them...

No worries if it's just a once in a while thing, especially since it seemed to complete well on our side.

I signed up for TEST applications anyways to help.

Well, cool beans! Keep up the good work!

8-)
ID: 513 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Issue Discussion : mldstest

©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)