This Week in [email protected]
Notes for Jan 5 2021
A weekly summary of news and notes for [email protected]

Summary
Happy 2021!

This past week marked the 6 month anniversary of this project, so we wanted to look back at how far we've come, assess what's gone well, find areas we need to improve, and talk about the future.

Overall, the community response has been amazing and we feel like we've built a really solid foundation for understanding and evaluating neural networks. We'll list some of our thoughts below, and we encourage you, the community to comment below with your thoughts as well.

The Good

• The actual work completed: You, the community, has trained over 260,000 neural networks for this project, modelling 110 different automata/machine types, with more in the works. When DS3 is done, we'll have over 1,000,000 trained networks. This is truly a unique contribution to science, and makes an amazing basis for study for years to come. I can already see 3 or 4 new lines of research coming from this dataset alone.
  
• Community engagement: We've made a special effort to reach out to the community, with weekly updates and attempting to be active on the forums and on twitter. We're a small project from a small lab in a small school. While it can always be better, we'd like to think this is has been a plus. Additionally, we've received support from the BOINC developers and general community as a whole via the BOINCNetwork Discord server and BOINC mailing lists, which were fantastically helpful in getting the project up and running in the first place.
  
• The technology: We're leveraging the BOINC infrastructure, and its working despite our WUs being a little out-of-the ordinary for it. We support 2 of the three major operating systems, on x86 and ARM, and GPUs from NVidia and AMD. Are they perfect? hardly. There's plenty of room for improvement and we're trying to make them better. But for 6 months in, we'll claim that's a very good start overall. The server side has gone well to, as the new server we moved to a few months ago has been a real help. Fun fact, for the first few months this project was run off an old thinkpad.. you make do with what you have.
  

Areas to Improve:

• Publish papers and release a dataset: You deserve a timely release of the dataset you helped create. We really wanted to have that out by now, and we feel we have enough data now to do it. But it takes time to write and time to curate/prepare/to document the dataset for release, and frankly that's taking longer than it should. That's on us, and we're working on it. We should be looking at days to weeks, not months.
  
• More supported platforms: We seek developers with C++ experience who can help us support new platforms, especially OSX and Android.
  
• More collaborators on the science side: COVID has really made this extra hard. I'll be honest, many ML researchers I pitched this idea too weren't all that interested early on because a) they were skeptical that the system would work and/or that people would volunteer, b) we didn't support GPUs, and or c) were interested but had no time. Now that we do, and our volunteer pool has swelled in general, we need to re-engage. Published results will help our case.
  
• This is still a 1-person operation: If I may get personal a minute, most of the above is due to the fact this is still a 1 person operation for the most part, and a person who is only a part time doctoral candidate with a full time job and family in addition to running this project. It's not like I'm a regular grad student who can sit for hours a day in a lab and write, either code or papers. I basically steal whatever time I can on the night and weekends. Doing a part-time doctorate is hard enough, choosing to create, develop, monitor, and moderate an entire volunteer computing project on top of that is frankly a bit nuts. But I'm passionate about the work we're doing here, and want to see it succeed. And I'm confident it will. Note: I try to say "we" when acting as project admin because I do feel eventually there will be more than just me, however that's not the case (yet).
  

What the future holds

• MLDS: Publish, both the first rounds of datasets and a paper or two. This is our top priority.
  
• MLDS: DS4 will switch to training CNNs for image classification, leading to a whole new set of insights and questions.
  
• It's important if MLC is to grown and continue that it grown beyond the MLDS application. Therefore, engage with new researchers and research thrusts that are amenable to our unique project architecture. Hyperparameter and architecture search seem extremely well suited for our system.
  

There are plenty more things we could mention, like tweaks to validations (some good, some .. err..less so), having a podcast made about the project, the generosity we've received from other project admins helping us getting our feet under us. And, of course, badges. We, as admins, have learned a lot about how to run a project, and despite some stumbles we hope you find the science compelling, the project interesting, and the community welcoming.

At 6 months in, we're on the cusp of real results, which should feed lots more interesting science. Thanks to all our volunteers for helping push the science of machine learning forward, and we hope to give you reason to continue to support us for many years to come.

If you have feedback, please leave it in the comments below.


Project status snapshot:
(note these numbers are approximations)






Last week's TWIM Notes: Dec 28 2020

Thanks again to all our volunteers!

-- The [email protected] Admins(s)
Homepage: https://www.mlcathome.org/
Twitter: @MLCHome2

You're making a GREAT work!!

A One-Man-Show?
Very brave to start this! I really hope that you will get help soon!
Great work, really!

What are neural networks? What do they do? What is the purpose of creating them and what is it all for?

Under Plans an Goals after your primary goals are implemented please consider adding a 32 bit application for Windows. There are a lot of older systems are still powered up looking for work, after Seti, on a Science related project. Run Time might be about 60 minutes.

Bill F
Dallas TX

---

@Bill F, I'm not sure that pytorch, our underlying library, supports 32 bit builds... but its definitely worth considering. And hand coding a neural network isn't hard per-se, but will get harder as we test more and more complex networks in the future.

@David Mecha : to find out more about the project, please see the main website at https://www.mlcathome.org/, specifically the homepage, the MLDS Datasets page, and the FAQ. You can also get an introduction by listening to the BOINC Radio podcast for [email protected] available here: https://boinc.network/boinc-radio/mlchome/ , and browse the Science area of our forums.

A lot of places link to the BOINC project page https://www.mlcathome.org/mlcathome/ , and many people miss the main site at https://www.mlcathome.org/.

Thank you. I think I understand it a little more now.

BRAVO for your commitment and courage, keep up the good job and community animation as you are doing, this is fundamental for a *good* boinc project :)

I hope you'll find tech & admin & scientific help for your project, but I'm afraid this is the most difficult part of all, and I know that many boinc project died just because of this, the one-man-show turns out to be exhausting after some time.

I wish you the best of luck with all this !

Great job! In which journal or journals do you plan to publish the papers?

---

I am glad that I am retired and don't have to do 1% of what you do. It makes the head spin just reading about it.
But there are few projects where you can be in the top 1% (at the moment) with just two GTX 1060's on Ubuntu. I like efficiency, and the work matches well to the cards.

OK,DONE

Happy new year to you as well!

Great read! Enjoyed the journey so far as a volunteer of this project. Most of all, the technical discussions on the forums about the science behind and the differences between the various experiments. Currently off to exploring Tensorflow/Keras for the first time myself, trying myself at my very first "big" data science projects doing NN-regressions. Model tuning (Hyperparameter) is such a labour-some job, …. Not coding whole applications, but rather just working with the Keras API for R. Still nice, seeing that the learning output of the stderr files of the WUs is oddly familiar! I'll happily bring back the GPUs in the meantime after testing.

Overall, great progress! Good luck with your paper and looking forward to the new runs.

Happy 2021!

I didn't know this was a one man project, wow. Hats off to you John for what this project has already accomplished in half a year.

I am no scientist and glad to help a tiny little bit with the processing power in front of me.

Thanks for shouting out the Discord and the BOINC radio, I didn't know about those.

Here is to another awesome six months and hopefully more ML researchers being excited about collaborating with MLC.

---