Credits for new "rand_automata" work units

Questions and Answers : Issue Discussion : Credits for new "rand_automata" work units
Message board moderation

To post messages, you must log in.

AuthorMessage
Conan
Avatar

Send message
Joined: 1 Jul 20
Posts: 5
Credit: 593,744
RAC: 2,719
Message 594 - Posted: 5 Oct 2020, 9:49:05 UTC

For me these new "rand_automata" work units are running about 5.5 times as long as the "ParityModified" work units (around 39,500 seconds vs around 7,500 seconds).

So wondering (as none of mine have validated yet) if the credits are to be increased for the longer run time?

Thanks
Conan
ID: 594 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bozz4science

Send message
Joined: 9 Jul 20
Posts: 141
Credit: 11,533,376
RAC: 61,455
Message 595 - Posted: 5 Oct 2020, 11:29:32 UTC - in response to Message 594.  
Last modified: 5 Oct 2020, 11:58:52 UTC

My Xeon 5660 usually crunches dataset 1/2 WU in ~ 8500s (2:20h) and needed ~45000s (12:30h) for those dataset 3 rand WU on average (based solely on <10 so far but very little variance in runtimes lets me think that they are rather representative already). That is factor 5.3x longer runtimes for an increased 595 credit per WU. So at least for my machine, those prior runs WU yield much higher return per runtime. Both types of WU give me on a per hour basis ~110/h vs. ~48/h. Hope that reference helps. Don't know if those ratios are comparative for other rigs.
ID: 595 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bluestang

Send message
Joined: 5 Oct 20
Posts: 2
Credit: 1,151,344
RAC: 0
Message 604 - Posted: 5 Oct 2020, 22:33:02 UTC - in response to Message 594.  

For me these new "rand_automata" work units are running about 5.5 times as long as the "ParityModified" work units (around 39,500 seconds vs around 7,500 seconds).

So wondering (as none of mine have validated yet) if the credits are to be increased for the longer run time?

Thanks
Conan


I joined this project with 1pc PC to test to contribute to my team and BOINC Mgr says it's in maintenance. Glad I didn't add my other bigger machines yet. So I come here on this forum and see this. So you get less credits than the old WUs, but the WUs take way longer? That's just messed up and pretty stupid IMO. I guess I'll move on to another project that pays properly.
ID: 604 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bozz4science

Send message
Joined: 9 Jul 20
Posts: 141
Credit: 11,533,376
RAC: 61,455
Message 605 - Posted: 5 Oct 2020, 22:49:22 UTC - in response to Message 604.  
Last modified: 5 Oct 2020, 22:50:18 UTC

Note that you do get 2.3 times more credit for those big WUs. They just return less on a per hour runtime basis. And that’s just for my rig. Don’t know if that’s representative. Also don’t really mind as it is also amongst the higher returning projects I have seen so far. GPU Support is coming your way soon. Ultimately your choice...
ID: 605 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 454
Credit: 14,284,704
RAC: 3,190
Message 608 - Posted: 5 Oct 2020, 23:39:57 UTC

Currently, Dataset 1+2 WUs generate 260 credits, Dataset 3 WUs generate 595 credits.

Our initial pass at calculating how much credit a WU should get is based on how long a single WU takes to run on an otherwise idle benchmark machine (one of your developer workstations). However, there are so many variables that go into this, everything from cache size to memory speed to AVX/SSE extension availability to OS to, these days, the quality of the CPU cooler; that no number will seem "fair" to everyone. So we do the best we can.

On our benchmark system (a modern but hardly top-of-the-line system with AVX2), dataset 3 WUs take about 2.1 times longer than datasets 1+2. Since dataset 1+2 were 260.0 credit/WU, and our contributors seemed reasonably happy with that value, we scaled up the dataset 3 WUs about the same. Now, knowing that dataset 1+2 were also relatively easier networks to train, we padded the number a little higher to try and account for it.

However, once WUs have been out in the wild, it's perfectly reasonable to re-examine the average runtimes of WUs on everyone's system and maybe make some tweaks to the credit as we go. I intended to look at the runtimes after about a month, and make adjustments to credit if warranted. It wouldn't surprise me if runtimes for dataset 3 are more than ~2.1x longer on the same system than 1+2 WUs, especially on older systems with less cache, and slower memory.
ID: 608 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bluestang

Send message
Joined: 5 Oct 20
Posts: 2
Credit: 1,151,344
RAC: 0
Message 609 - Posted: 6 Oct 2020, 0:57:25 UTC

Thanks for the response and explanation.

Someone mentioned a GPU app coming? How soon?
ID: 609 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Conan
Avatar

Send message
Joined: 1 Jul 20
Posts: 5
Credit: 593,744
RAC: 2,719
Message 610 - Posted: 6 Oct 2020, 1:43:50 UTC

Thanks pianoman, no worries.
Some of my work units have now validated and they do get 595 credits, so more than the shorter WUs.

Thanks
Conan
ID: 610 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 454
Credit: 14,284,704
RAC: 3,190
Message 613 - Posted: 6 Oct 2020, 5:30:20 UTC - in response to Message 609.  

Thanks for the response and explanation.

Someone mentioned a GPU app coming? How soon?


We're hoping to get GPU support in `mldstest` in a week or two, but there are a lot of variables, not the least of which is I don't have a CUDA test machine local, only ROCm, so don't hold me to it. More info in this forum thread: https://www.mlcathome.org/mlcathome/forum_thread.php?id=89 .
ID: 613 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pianoman [MLC@Home Admin]
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Jun 20
Posts: 454
Credit: 14,284,704
RAC: 3,190
Message 628 - Posted: 9 Oct 2020, 4:46:16 UTC

After going through the DB and looking at average run times, we're upping the credit for all new rand_automata (dataset 3) WUs to 959.4 credits. Out benchmark machine apparently chews through dataset 3 WUs pretty fast compared to an average machine. There are still old rand_automata WUs out there that have the old value, but any new ones (and more are queuing now) will have the new value.
ID: 628 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bozz4science

Send message
Joined: 9 Jul 20
Posts: 141
Credit: 11,533,376
RAC: 61,455
Message 629 - Posted: 9 Oct 2020, 8:36:18 UTC - in response to Message 628.  
Last modified: 9 Oct 2020, 8:36:45 UTC

Great! My RAC should quickly jump back then to what it previously was before the introduction of the rand WU. And I am glad I wasn’t wrong about my guess about the average runtime of those WU after all. Have a great weekend!
ID: 629 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Issue Discussion : Credits for new "rand_automata" work units

©2021 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)