|
81)
Message boards :
News :
[TWIM Notes] Apr 8 2021
(Message 1144)
Posted 9 Apr 2021 by pianoman [MLC@Home Admin] Post: This Week in MLC@Home Notes for Apr 8 2021 A weekly summary of news and notes for MLC@Home Summary Lots of things going on behind the scenes these past few weeks. First, the Discord server is up and running, with some nice discussions starting there. We've also made progress on the next client release, getting static builds working for CPU clients on amd64, armv7, and aarch64. We've also started work on DS4, as we talked about in the last news post. The big news this week is that the 2021 BOINC workshop is going on next week, Wednesday Apr 14th, and MLC@Home will be presenting @2:10pm eastern time. You can see the agenda and more information at main website or our local forum thread. We have big plans for our 15 minutes. We consider this to be our chance to introduce this project to a much wider audience. Not only will we be giving an introduction to the science, we'll also be releasing the paper to arXiv on Wednesday and talking about some of the results, as well as our plans for the future. None of this will be a surprise to the people here, but those who aren't following the project will have a chance to hear what we're doing and judge us on our merits. The presentation will be recorded and posted to youtube and here. I'm really looking forward to Wednesday, and I hope many of you can join us! Other News
|
|
82)
Message boards :
News :
New client released with ARM support!
(Message 1143)
Posted 9 Apr 2021 by pianoman [MLC@Home Admin] Post: Now that pytorch allows static linking, an android client is a possibility again... admittedly lower on the priority list at the moment, but its at least possible now. |
|
83)
Message boards :
News :
MLC@Home Discord server
(Message 1132)
Posted 25 Mar 2021 by pianoman [MLC@Home Admin] Post: Members of the MLC@Home community: We have created a Discord server for MLC@Home! Discord is a web-based chat application that, like similar services Slack and Mattermost, re-invents IRC but with less features. However, it does provide an accessible interface for everyone, and makes it easy to create new communities and bots to keeping things up to date. It provides for real-time text-based communication between the community. It compliments (but does not replace) the project web forums, and is, of course completely optional for participating in the project. We chose Discord over other options since there is already a BOINC community project presence on Discord, and writing private bots is relatively easy. Originally this was created for personal use, as it is a single place to collect notifications for new forum posts, the twitter feed, GitLab CI updates, etc. But we're opening it to the larger community if there's interest. We're still learning how to run a public Discord server so expect some growing pains. The Discord server invite link is below if you'd like to join: https://discord.gg/BdE4PGpX2y -- MLC@Home Admins |
|
84)
Questions and Answers :
Windows :
Problems with Win 10 and NVidia 1080
(Message 1131)
Posted 25 Mar 2021 by pianoman [MLC@Home Admin] Post: Note you shouldn't need to download the runtimes, they should be included in the windows download. But windows it a neverending source of random configs that work for most people but fail for some. In this case, your computer seems to be erroring out with <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> app_version download error: couldn't get input files: <file_xfer_error> <file_name>libcudnn_adv_train.so.8-pt17cuda102000</file_name> <error_code>-200 (wrong size)</error_code> </file_xfer_error> </message> ]]> So it looks like there was a problem downloading your client. I realize we're a month late replying, but if you could reset the MLC@Home project, which will re-download the client, that might help clear the error. |
|
85)
Message boards :
News :
[TWIM Notes] Mar 23 2021
(Message 1129)
Posted 24 Mar 2021 by pianoman [MLC@Home Admin] Post: We've been focused on several things recently, and TWIM Notes hasn't been a priority since there hasn't been too much to talk about and, frankly, they take a while to put together. Going forward, TWIM Notes will be intermittent for a few more weeks, but we aim to be back every week in about a month. This won't be a full TWIM Notes, but here's a few tidbits of news: * If you follow the progress on https://gitlab.com/mlcathome/mlds, you'll see that progress on v9.90 continues. We've successfully upgraded to PyTorch v1.8, and just today managed to *finally* get the client statically linked. That will allow us to drop appimage, makes the client a bit smaller overall, and should vastly help compatibility between linux releases. Hopefully the same can be done for windows, ARM, and CUDA/ROCM. * You can follow v9.90 progress here: https://gitlab.com/mlcathome/mlds/-/milestones * We've signed up to present at the 2021 BOINC Workshop next month, and will use that time to launch the paper and highlight the release of the datasets so far. * We're setting up a MLC@Home discord server for real-time chat to compliment the forum and twitter, stay tuned for the invite when it goes live! |
|
86)
Questions and Answers :
Unix/Linux :
Linux for Windows Users
(Message 1128)
Posted 24 Mar 2021 by pianoman [MLC@Home Admin] Post: I keep missing these posts. Ugh. One of a few things needs to happen to for AMD support under windows: * AMD ports ROCm to Windows (unlikely anytime soon) * PyTorch expands their Vulcan and/or OpenCL support to support training as well as inference. ROCm on linux is actually pretty nice, and works fine with the stock kernel driver (4.19+). Polaris support will give a nice speedup over CPU, but for right now both vega and polaris support are lagging behind equivalent nvidia cards. There's an [url= https://gitlab.com/mlcathome/mlds/-/issues/8]issue on gitlab[/url] to look into why. Maybe the move to PyTorch 1.8 (in progress) might help. |
|
87)
Message boards :
News :
[TWIM Notes] Mar 9 2021 posted
(Message 1124)
Posted 10 Mar 2021 by pianoman [MLC@Home Admin] Post: MLC@Home has posted the Mar 9 2021 edition of its weekly "This Week In MLC@Home" newsletter! Work on an updated client for DS4 started in earnest. Supports PyTorch v1.8 and DS4! Read the update and join the discussion here. |
|
88)
Message boards :
News :
[TWIM Notes] Mar 9 2021
(Message 1123)
Posted 10 Mar 2021 by pianoman [MLC@Home Admin] Post: This Week in MLC@Home Notes for Mar 9 2021 A weekly summary of news and notes for MLC@Home Summary The paper is written is now being internally edited/reviewed. It'll take a round or two of that before we post to arXiv and look for a conference to submit. There was some puzzling results with one of the DS2 machines which may require us to re-compute some of those WUs with updated training data, but those were fast to compute and it won't take long for our volunteers to redo them if necessary. That means we've started looking at client updates and DS4! We started this weekend on client v9.90, for which we are targeting the following features:
|
|
89)
Message boards :
News :
[TWIM Notes] Feb 23 2021
(Message 1121)
Posted 9 Mar 2021 by pianoman [MLC@Home Admin] Post: There's been no change to either CPU or GPU WUs, sorry. |
|
90)
Message boards :
News :
[TWIM Notes] Feb 1 2021
(Message 1118)
Posted 5 Mar 2021 by pianoman [MLC@Home Admin] Post: The AMD client *really* needs an update. Maybe I'll try and give it some love this weekend. PyTorch 1.8 with official ROCm support was just released today, so maybe its time to refresh all the clients. Plus pytorch updated their static compilation options, so maybe we can go that route this time too. That would be a huge win. The current AMD client won't support polaris (rx5xx) due to a bug in the version of rocm its linked against (3.8). Also, I know of only one other person who has gotten the rocm client to work, and their WUs revealed there's still a library trying to link against their system's version of miopen (a rocm library) instead of the one we ship (not a huge deal, it'll work, but it should not be doing that). Let me see what I can do this weekend. |
|
91)
Message boards :
News :
[TWIM Notes] Feb 23 2021
(Message 1117)
Posted 5 Mar 2021 by pianoman [MLC@Home Admin] Post: From a volunteer's point of view, no, there's only upside. From a project admin's point of view, there's *mostly* upsides, but there are a few things to consider: * There would be a financial incentive to cheat * Simple mistakes in validation (which we've made on this project in the past) now have financial consequences for users. * Associating "cryptocoin" with a project complicates the narrative in certain quarters. Overall I'm leaning towards requesting whitelisting, but there are still some more anti-cheat measures we'd like to implement first. I'm being purposely vague here for obvious reasons. |
|
92)
Message boards :
News :
[TWIM Notes] Feb 23 2021 posted
(Message 1105)
Posted 24 Feb 2021 by pianoman [MLC@Home Admin] Post: MLC@Home has posted the Feb 23 2021 edition of its weekly "This Week In MLC@Home" newsletter! Paper updates, a note about the larger datasets, and more... Read the update and join the discussion here. |
|
93)
Message boards :
News :
[TWIM Notes] Feb 23 2021
(Message 1104)
Posted 24 Feb 2021 by pianoman [MLC@Home Admin] Post: This Week in MLC@Home Notes for Feb 23 2021 A weekly summary of news and notes for MLC@Home Summary A small update this week since there's not much news overall. Honestly, yours truly got a little swamped with other things in life and needed to take care of some other things last week. Buy we're back this week, and other than an unexpectedly empty CPU queue for about half a day last week (since corrected), the project has continued and you volunteers have carved off another large chunk of datasets 1, 2 and 3. The paper is coming along nicely, and we're finally happy with the data and analysis we want at this point, so it's just writing. Some tables and graphs have been posted to Twitter if you'd like to follow along. Honestly, we're about 3 months overdue to get this out to at least arXiv, but we hope it'll be worth the wait. Most of that time was spent tweaking algorithms to get better results, and the datasets/feature vectors are too large fit on an 8GB GPU, so it's mostly CPU training. 18 hours for an epoch is not uncommon. On the bright side, what we're learning doing this will help DS4 when it's available. DS3-500 and DS3-1000 are complete and ready to upload, but we're still deciding the best place to host such large files. Seeding a torrent off the main server would cut into the already limited bandwidth. We'll need a better solution, for now they exist on a backed-up data drive, and available by individual request. Other News
|
|
94)
Questions and Answers :
Issue Discussion :
No tasks sent
(Message 1097)
Posted 20 Feb 2021 by pianoman [MLC@Home Admin] Post: The CPU task queue temporarily ran dry for about half a day before we noticed and dropped the next batch yesterday afternoon. However, nothing has changed on the arm client side, so I'm not sure why they would suddenly be failing... |
|
95)
Message boards :
News :
[TWIM Notes] Feb 8 2021 posted
(Message 1086)
Posted 9 Feb 2021 by pianoman [MLC@Home Admin] Post: MLC@Home has posted the Feb 8 2021 edition of its weekly "This Week In MLC@Home" newsletter! Its Dataset week! This week MLC@Home releases the first round of datasets computed by our volunteers. They are available for download at the https://www.mlcathome.org/ website. These datasets will help scientists better understand neural networks. Read the update and join the discussion here. |
|
96)
Message boards :
News :
[TWIM Notes] Feb 8 2021
(Message 1085)
Posted 9 Feb 2021 by pianoman [MLC@Home Admin] Post: This Week in MLC@Home Notes for Feb 8 2021 A weekly summary of news and notes for MLC@Home Summary It's Dataset week! We've been working behind the scenes on a paper for analysis of the results you, our volunteers have created so far. The paper is still in progress, but we've waited long enough to release the datasets themselves. Over the next few days, you'll see more and more datasets come available for download at https://www.mlcathome.org/mlds.html. Each dataset archive contains a README.md file with details on what is included and some information on how to use it. Each trained network is in its own directory, with both the native PyTorch version of the saved model, a JSON file that contains the learned weights, and some metadata about each trained network, including a training history. Currently available datasets are: - MLDS-DS1: 100/ea, 500/ea, 1000/ea. (5000/ea and 10000/ea still computing) - MLDS-DS2: 100/ea, 500/ea, 1000/ea. (5000/ea and 10000/ea still computing) - MLDS-DS3: 100/ea (500/ea and 1000/ea zipping/uploading, 5000/ea and 10000/ea still computing) The latter datasets are so large they would be better as torrents, so we may make them available only as torrents. For researchers, if you use these datasets, I ask that you a) let us know, and b) cite our paper when it comes available. Detailed News
|
|
97)
Message boards :
Science :
Datasets starting to be posted
(Message 1084)
Posted 7 Feb 2021 by pianoman [MLC@Home Admin] Post: I'm still working on the paper (and tweaking the classification performance on the DS3 data to get better results for classifying which automata a network was trained to mimic... ). But I promised I'd start posting the datasets that you've helped create, so I've started posting them for download. A more formal announcement is coming, but if you want a preview, you can view them now at: https://www.mlcathome.org/mlds.html . The DS3 ones are getting posted now, but are taking a long time to compress/tar.. DS3-100 took a few hours to tar/xz into a 12GB file. DS3-500 is currently tarring up now. And DS3-1000 might need to move off the SSDs and onto spinny disks to have enough space... meaning it will take even longer. |
|
98)
Questions and Answers :
Issue Discussion :
GPU Utilization and Resource Requirements
(Message 1083)
Posted 7 Feb 2021 by pianoman [MLC@Home Admin] Post: I'm am positive the GPU client could be optimized, but it's a complex issue with a lot of variables (OS, environment, etc..) and unless there's someone who can tackle this quickly, it's lower priority than releasing the dataset, preparing for DS4, analyzing DS1/2/3 results, writing the paper. I did open https://gitlab.com/mlcathome/mlds/-/issues/12 to track this though, if someone wants to tackle it. |
|
99)
Message boards :
News :
[TWIM Notes] Feb 1 2021
(Message 1076)
Posted 4 Feb 2021 by pianoman [MLC@Home Admin] Post: GPUs need 2GB ram min. RX550 is the minimum AMD GPU that will be supported, as I think the RX540 is not POLARIS-based (gfx803 in AMD nomeclature). POLARIS needs an updated client compiled against a later version of rocm (rocm 3.8 has a bug that keeps it from running on POLARIS). It's on the todo list but not ready yet. For now, the only AMD GPUs supported are VEGA-based (gfx900/906/908, not gfx902 APUs) and must be running Linux. Last night I just got pytorch working with rocm4., so that should be easier in the future. There's *some* support for NAVI in rocm, but its untested. Note AMD APUs are not supported by rocm, and thus aren't supported by pytorch. Yes that's sad. On the CUDA side, you need 2GB ram and compute capability 3.5 or higher. That's,... most GTX 700 series and up I think? |
|
100)
Message boards :
News :
[TWIM Notes] Feb 1 2021 posted
(Message 1071)
Posted 2 Feb 2021 by pianoman [MLC@Home Admin] Post: MLC@Home has posted the Feb 1 2021 edition of its weekly "This Week In MLC@Home" newsletter! Updates on dataset release, the paper, and we've now surpassed 1000 examples for each of D1/D2, meaning a greater mix of WUs in the future. Read the update and join the discussion here. |
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)