Validation error on GPU task done on a 1650

Questions and Answers : Windows : Validation error on GPU task done on a 1650
Message board moderation

To post messages, you must log in.

AuthorMessage
TheFanMan64

Send message
Joined: 22 Mar 21
Posts: 2
Credit: 89,586
RAC: 0
Message 1220 - Posted: 11 Jun 2021, 17:32:10 UTC

about 3 out of 6 tasks done had a validation error.
ID: 1220 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bozz4science

Send message
Joined: 9 Jul 20
Posts: 142
Credit: 11,536,204
RAC: 3
Message 1223 - Posted: 11 Jun 2021, 21:08:40 UTC - in response to Message 1220.  

Your validation rate already looks better now. A certain quota of invalids is to be expected and due to the stochastic nature of neural network learning. The admin John Clemens is already aware of this issue being caused mainly due to NaN errors in midst of the training process and will roll out a fix to periodically check the nature of the computed network weights to catch any NaN weight values and revert/reinitialize the weights to avoid some of the otherwise invalid results.
ID: 1223 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Windows : Validation error on GPU task done on a 1650

©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)