It's been a while since we've posted an update, but that doesn't mean the project has been idle! If you've been following on our Discord server you'll know we've continued to make progress, and thanks to our volunteers, today is a day of celebration!
Here's a summary of the current project status:
- DS2 Computation is complete! As of 1 Apr 2022, we finally crossed 10,000 trained networks threshold for ParityModified, completing our computation for DS2. This has taken a long time, and the complete dataset should help researchers understand how neural networks encode data.
- All DS1/DS2 tarballs are available for download from https://www.mlcathome.org/mlds.html. This is your work, and now its free for you or anyone else to study and build upon!
- DS3 tarballs still pending. Computation for DS3 completed last year, but we have not uploaded to full datasets to the website for download yet. We've been focused on analysis, and the sheer size of the dataset can cause headaches making bundling a time-consuming task. We'll post here when they're available.
- DS4 WUs are out! DS4 WUs are out for our CPU client, and progress has started there. DS4 is much more complicated to manage on the backend because it has multiple training sets that have different requirements, but we're pushing new WUs out as fast as we can.
- We're pausing GPU WUs: It saddens us, but we have not been successful updating our GPU clients to support DS4 WUs. And as we shift our focus to analyzing the results we do have, we have less and less time to focus on client development beyond the CPU client. When the current GPU queue runs dry, we won't be sending out more GPU work until we have time to re-prioritize porting a GPU client again. Maintaining a GPU client has taken much more time and effort than anticipated, and unless we can get outside help it will remain a low priority for the time being. We truly appreciate our GPU volunteers, but at the moment we don't have any work to send, and encourage you to turn your hardware to support other worthwhile projects that can support your hardware!
- We're exploring porting the CPU client to Rust. In addition, our reliance on PyTorch has become more of a hindrance to portability than an asset. While the neural network ecosystem in rust is not nearly as robust, the ability for rust to compile a static binary targeting a large number of architectures and operating systems is very appealing to portability. As such, we're looking to port our MLC CPU client to pure rust, with an option to support GPUs from the same code base in the future. If you know Rust and are interested, please contact the MLC Admins.
Please note that there are still DS2 WUs in the work queue, we ask that you please continue to crunch them, as it's always better to have more samples as spares. However, we don't plan to queue up any more DS1/2/3 WUs, and all new WUs added will be DS4 or later. This applies to the GPU queue as well.
We're really excited for DS4 WUs going forward, and it should help show our theory that similar networks cluster in parameter space in both feed forward and CNN-based networks as well as the RNNs used in DS1/2/3. Beyond DS4, we have some ideas but have nothing concrete at the moment. We'll keep you updated as we move forward.
Thanks again to all our volunteers for supporting the project and helping science.
-- The MLC@Home Admins(s)
Discord invite: https://discord.gg/BdE4PGpX2y