We realize this will cause us to lose some volunteers, but that's why we're trying to be upfront about this now so that everyone can decide if and how to allocate their BOINC contributions accordingly. We hope that you'll consider leaving MLC in your projects list and help us crunch WUs when we have them, but understand if choose not to.

* DS4 WUs are out! DS4 WUs are out for our CPU client, and progress has started there. DS4 is much more complicated to manage on the backend because it has multiple training sets that have different requirements, but we're pushing new WUs out as fast as we can.

Tried running the Windows test app again of the two runs I tried today:
The first one completed successfully and has validated.
The second one failed just like the one from a few weeks ago with.

The issue with the failed app is that the native BOINC API uses SIGALRM (without documenting that it does) , and so does PyTorch (barely documenting that it does). The app ran fine when run standalone, but once it was launched by the boinc client it crashed once pytorch overwrote the SIGALRM handler and boinc tried to call it. The fix is to re-write the app using the boinc wrapper, which means undoing a lot of the work I put in to use the native API in the first place (which annoys me), and means that WUs created for the old app won't be compatible with those from the new app. And its just turned out to take a while longer than planned.

We've been talking about DS4 for months, and the code is ready for larger testing. Unfortunately, we rolled out a test client a few weeks ago that failed miserably, because of an incompatibility between PyTorch and the native BOINC API. there's a way around this, but it requires more development, and a change to how WUs are specified, and we've been working on it ever since.

Is it not a (too) big gap between 1M and 500M ?

Summary
The paper is written is now being internally edited/reviewed. It'll take a round or two of that before we post to arXiv and look for a conference to submit. There was some puzzling results with one of the DS2 machines which may require us to re-compute some of those WUs with updated training data, but those were fast to compute and it won't take long for our volunteers to redo them if necessary.

That means we've started looking at client updates and DS4! We started this weekend on client v9.90, for which we are targeting the following features:

• Upgrade to PyTorch v1.8
  
• DS4 support! .. Our first CNN networks.
  
• ROCm upgrade to 4.0/4.0.1 (polaris support)
  
• Re-examine GPU performance/utilization
  
• Static linking for at least the CPU client (maybe GPU too).. dropping appimage requirement. This may not be possible, but would be a nice benefit of moving to PyTorch v1.8
  

his means we'll start mixing in other types of WUs than just DS2 WUs into the GPU queue again, balancing between the remaining DS1/DS2 and DS3 WUs. When DS4 is ready, we'll rebalance again.

* We spent some time last week gaining access to both Windows and Linux machines with CUDA, so hoepfully GPU clients should be released to testing this week. We're having some issues packaging the ROCm version but will try to iron those out into testing as well.

At any rate, the Quarantine ones ran only 16 times faster on a GTX 1060 as on a single core of an i7-4771. If you ran them on 16 cores of a Ryzen 3000 series, you could do the same amount of work on less power on a CPU. I was slightly disappointed and a bit surprised.

There are things happening, its just not publically visible at the moment.