How does MLC verify results without running multiple tasks per work unit (redundancy)?

Message boards : Cafe : How does MLC verify results without running multiple tasks per work unit (redundancy)?
Message board moderation

Author	Message
W8n4Singularity Send message Joined: 30 Aug 20 Posts: 25 Credit: 47,025,926 RAC: 0	Message 1136 - Posted: 7 Apr 2021, 2:46:38 UTC Last modified: 7 Apr 2021, 2:47:10 UTC Asking just out of curiosity. Some older tasks did have other canonical results such as this one: https://www.mlcathome.org/mlcathome/workunit.php?wuid=1120016 Newer tasks are validated without redundancy: https://www.mlcathome.org/mlcathome/workunit.php?wuid=2602952 Thanks! ID: 1136 · Rating: 0 · rate: /      Reply Quote
An0ma1y Send message Joined: 3 Aug 20 Posts: 8 Credit: 7,650,164 RAC: 0	Message 1138 - Posted: 7 Apr 2021, 19:40:13 UTC hmm, i hadn't noticed this, but this is a very good question, i would like to know this as well. ID: 1138 · Rating: 0 · rate: /      Reply Quote
bozz4science Send message Joined: 9 Jul 20 Posts: 142 Credit: 11,536,204 RAC: 3	Message 1139 - Posted: 8 Apr 2021, 11:53:31 UTC Isn't each network learning task a stochastic process, in the sense that no 2 tasks will ever (very unlikely) produce the same result? ID: 1139 · Rating: 0 · rate: /      Reply Quote
W8n4Singularity Send message Joined: 30 Aug 20 Posts: 25 Credit: 47,025,926 RAC: 0	Message 1141 - Posted: 8 Apr 2021, 20:09:45 UTC - in response to Message 1139.   Last modified: 8 Apr 2021, 20:10:12 UTC Very interesting. If that is the case, does that also mean that MLC would need to process far more results to reach statistical significance versus a project that uses redundancy and can prove that the calculated results are valid? ID: 1141 · Rating: 0 · rate: /      Reply Quote
pianoman [MLC@Home Admin] Project administrator Project developer Project tester Project scientist Send message Joined: 30 Jun 20 Posts: 462 Credit: 21,406,548 RAC: 0	Message 1150 - Posted: 16 Apr 2021, 21:11:08 UTC Each network is compared against a test dataset that is held back from being sent to you users. If the network performs well on that test set (and is a valid network, and is not identical to a previously submitted result) then it is considered valid. ID: 1150 · Rating: 0 · rate: /      Reply Quote
pianoman [MLC@Home Admin] Project administrator Project developer Project tester Project scientist Send message Joined: 30 Jun 20 Posts: 462 Credit: 21,406,548 RAC: 0	Message 1151 - Posted: 16 Apr 2021, 21:14:34 UTC I'm working on adding a few more criteria that I'll keep vague to stop potential cheating, but in general that's why we don't do multiple results per WU, because they're stochastic and won't be identical at all. In fact, this stochastic nature is what we're trying to capture so it helps. ID: 1151 · Rating: 0 · rate: /      Reply Quote

©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)