Task 7402072

Name ParityModified-1631729418-17208-1_0
Workunit 5075999
Created 16 Sep 2021, 19:52:59 UTC
Sent 21 Sep 2021, 14:42:20 UTC
Report deadline 28 Sep 2021, 14:42:20 UTC
Received 24 Sep 2021, 3:31:10 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 449
Run time 13 min 3 sec
CPU time 12 min 25 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 6,467.43 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 1.64 GB
Peak swap size 3.64 GB
Peak disk usage 1.53 GB

Stderr output

<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
-24 11:04:05	                main:574]	:	INFO	:	Epoch 134 | loss: 0.0311337 | val_loss: 0.0311623 | Time: 2154.8 ms
[2021-09-24 11:04:07	                main:574]	:	INFO	:	Epoch 135 | loss: 0.0311322 | val_loss: 0.0311589 | Time: 2199.77 ms
[2021-09-24 11:04:09	                main:574]	:	INFO	:	Epoch 136 | loss: 0.0311367 | val_loss: 0.031164 | Time: 2123.82 ms
[2021-09-24 11:04:11	                main:574]	:	INFO	:	Epoch 137 | loss: 0.0311365 | val_loss: 0.031164 | Time: 2164.61 ms
[2021-09-24 11:04:14	                main:574]	:	INFO	:	Epoch 138 | loss: 0.0311313 | val_loss: 0.0311645 | Time: 2109.47 ms
[2021-09-24 11:04:16	                main:574]	:	INFO	:	Epoch 139 | loss: 0.0311317 | val_loss: 0.0311648 | Time: 2132.99 ms
[2021-09-24 11:04:18	                main:574]	:	INFO	:	Epoch 140 | loss: 0.0311294 | val_loss: 0.0311615 | Time: 2121.52 ms
[2021-09-24 11:04:20	                main:574]	:	INFO	:	Epoch 141 | loss: 0.0311322 | val_loss: 0.0311632 | Time: 2134.37 ms
[2021-09-24 11:04:22	                main:574]	:	INFO	:	Epoch 142 | loss: 0.031128 | val_loss: 0.0311661 | Time: 2158.97 ms
[2021-09-24 11:04:24	                main:574]	:	INFO	:	Epoch 143 | loss: 0.0311265 | val_loss: 0.0311633 | Time: 2145.51 ms
[2021-09-24 11:04:27	                main:574]	:	INFO	:	Epoch 144 | loss: 0.0311235 | val_loss: 0.031164 | Time: 2172.54 ms
[2021-09-24 11:04:29	                main:574]	:	INFO	:	Epoch 145 | loss: 0.0311232 | val_loss: 0.031165 | Time: 2141.22 ms
[2021-09-24 11:04:31	                main:574]	:	INFO	:	Epoch 146 | loss: 0.0311227 | val_loss: 0.0311634 | Time: 2125.85 ms
[2021-09-24 11:04:33	                main:574]	:	INFO	:	Epoch 147 | loss: 0.0311274 | val_loss: 0.0311621 | Time: 2184.61 ms
[2021-09-24 11:04:35	                main:574]	:	INFO	:	Epoch 148 | loss: 0.0311232 | val_loss: 0.0311634 | Time: 2140.92 ms
[2021-09-24 11:04:38	                main:574]	:	INFO	:	Epoch 149 | loss: 0.0311217 | val_loss: 0.031162 | Time: 2015.65 ms
[2021-09-24 11:04:40	                main:574]	:	INFO	:	Epoch 150 | loss: 0.0311178 | val_loss: 0.0311649 | Time: 1998.82 ms
[2021-09-24 11:04:42	                main:574]	:	INFO	:	Epoch 151 | loss: 0.0311174 | val_loss: 0.0311624 | Time: 2203.57 ms
[2021-09-24 11:04:44	                main:574]	:	INFO	:	Epoch 152 | loss: 0.031117 | val_loss: 0.0311654 | Time: 2139.02 ms
[2021-09-24 11:04:46	                main:574]	:	INFO	:	Epoch 153 | loss: 0.0311168 | val_loss: 0.0311644 | Time: 2148.96 ms
[2021-09-24 11:04:48	                main:574]	:	INFO	:	Epoch 154 | loss: 0.0311142 | val_loss: 0.031165 | Time: 2107.32 ms
[2021-09-24 11:04:51	                main:574]	:	INFO	:	Epoch 155 | loss: 0.0311161 | val_loss: 0.0311674 | Time: 2196.1 ms
[2021-09-24 11:04:53	                main:574]	:	INFO	:	Epoch 156 | loss: 0.031114 | val_loss: 0.0311649 | Time: 2204.62 ms
[2021-09-24 11:04:55	                main:574]	:	INFO	:	Epoch 157 | loss: 0.0311124 | val_loss: 0.0311663 | Time: 2146.7 ms
[2021-09-24 11:04:57	                main:574]	:	INFO	:	Epoch 158 | loss: 0.0311126 | val_loss: 0.0311636 | Time: 2165.69 ms
[2021-09-24 11:04:59	                main:574]	:	INFO	:	Epoch 159 | loss: 0.0311142 | val_loss: 0.0311662 | Time: 2100.73 ms
[2021-09-24 11:05:01	                main:574]	:	INFO	:	Epoch 160 | loss: 0.031115 | val_loss: 0.0311675 | Time: 2098.4 ms
[2021-09-24 11:05:04	                main:574]	:	INFO	:	Epoch 161 | loss: 0.0311125 | val_loss: 0.0311692 | Time: 2017.18 ms
[2021-09-24 11:05:06	                main:574]	:	INFO	:	Epoch 162 | loss: 0.0311157 | val_loss: 0.0311683 | Time: 2103.08 ms
[2021-09-24 11:05:08	                main:574]	:	INFO	:	Epoch 163 | loss: 0.0311114 | val_loss: 0.0311667 | Time: 2120.96 ms
[2021-09-24 11:05:10	                main:574]	:	INFO	:	Epoch 164 | loss: 0.0311104 | val_loss: 0.0311655 | Time: 2185.6 ms
[2021-09-24 11:05:12	                main:574]	:	INFO	:	Epoch 165 | loss: 0.0311169 | val_loss: 0.0311654 | Time: 2122.29 ms
[2021-09-24 11:05:14	                main:574]	:	INFO	:	Epoch 166 | loss: 0.0311148 | val_loss: 0.0311717 | Time: 2118.14 ms
[2021-09-24 11:05:17	                main:574]	:	INFO	:	Epoch 167 | loss: 0.0311102 | val_loss: 0.0311632 | Time: 2205.66 ms
[2021-09-24 11:05:19	                main:574]	:	INFO	:	Epoch 168 | loss: 0.0311101 | val_loss: 0.0311666 | Time: 2180.54 ms
[2021-09-24 11:05:21	                main:574]	:	INFO	:	Epoch 169 | loss: 0.03111 | val_loss: 0.0311684 | Time: 2154.18 ms
[2021-09-24 11:05:23	                main:574]	:	INFO	:	Epoch 170 | loss: 0.0311093 | val_loss: 0.0311704 | Time: 2060.58 ms
[2021-09-24 11:05:25	                main:574]	:	INFO	:	Epoch 171 | loss: 0.0311111 | val_loss: 0.0311678 | Time: 2147.52 ms
[2021-09-24 11:05:27	                main:574]	:	INFO	:	Epoch 172 | loss: 0.0311104 | val_loss: 0.0311742 | Time: 1991.57 ms
[2021-09-24 11:05:30	                main:574]	:	INFO	:	Epoch 173 | loss: 0.0311101 | val_loss: 0.0311641 | Time: 2079.91 ms
[2021-09-24 11:05:32	                main:574]	:	INFO	:	Epoch 174 | loss: 0.0311119 | val_loss: 0.0311708 | Time: 2079.73 ms
[2021-09-24 11:05:34	                main:574]	:	INFO	:	Epoch 175 | loss: 0.0311125 | val_loss: 0.0311679 | Time: 2127.21 ms
[2021-09-24 11:05:36	                main:574]	:	INFO	:	Epoch 176 | loss: 0.0311087 | val_loss: 0.0311723 | Time: 2144.25 ms
[2021-09-24 11:05:38	                main:574]	:	INFO	:	Epoch 177 | loss: 0.0311084 | val_loss: 0.0311718 | Time: 2179.85 ms
[2021-09-24 11:05:40	                main:574]	:	INFO	:	Epoch 178 | loss: 0.0311063 | val_loss: 0.0311741 | Time: 2132.97 ms
[2021-09-24 11:05:43	                main:574]	:	INFO	:	Epoch 179 | loss: 0.0311075 | val_loss: 0.0311718 | Time: 2100.2 ms
[2021-09-24 11:05:45	                main:574]	:	INFO	:	Epoch 180 | loss: 0.0311065 | val_loss: 0.0311739 | Time: 2147.9 ms
[2021-09-24 11:05:47	                main:574]	:	INFO	:	Epoch 181 | loss: 0.0311053 | val_loss: 0.0311641 | Time: 2110.59 ms
[2021-09-24 11:05:49	                main:574]	:	INFO	:	Epoch 182 | loss: 0.031103 | val_loss: 0.0311688 | Time: 2151.93 ms
[2021-09-24 11:05:51	                main:574]	:	INFO	:	Epoch 183 | loss: 0.0311053 | val_loss: 0.0311675 | Time: 2114.82 ms
[2021-09-24 11:05:53	                main:574]	:	INFO	:	Epoch 184 | loss: 0.0311037 | val_loss: 0.0311674 | Time: 2120.88 ms
[2021-09-24 11:05:56	                main:574]	:	INFO	:	Epoch 185 | loss: 0.0311005 | val_loss: 0.0311781 | Time: 2163.32 ms
[2021-09-24 11:05:58	                main:574]	:	INFO	:	Epoch 186 | loss: 0.0310999 | val_loss: 0.0311713 | Time: 2095.39 ms
[2021-09-24 11:06:00	                main:574]	:	INFO	:	Epoch 187 | loss: 0.0311045 | val_loss: 0.0311665 | Time: 2154.39 ms
[2021-09-24 11:06:02	                main:574]	:	INFO	:	Epoch 188 | loss: 0.0311063 | val_loss: 0.0311666 | Time: 2100.26 ms
[2021-09-24 11:06:04	                main:574]	:	INFO	:	Epoch 189 | loss: 0.0311052 | val_loss: 0.0311646 | Time: 2177.65 ms
[2021-09-24 11:06:06	                main:574]	:	INFO	:	Epoch 190 | loss: 0.0311029 | val_loss: 0.0311693 | Time: 2100.33 ms
[2021-09-24 11:06:09	                main:574]	:	INFO	:	Epoch 191 | loss: 0.0311015 | val_loss: 0.0311754 | Time: 2156.17 ms
[2021-09-24 11:06:11	                main:574]	:	INFO	:	Epoch 192 | loss: 0.0311016 | val_loss: 0.0311698 | Time: 2152.71 ms
[2021-09-24 11:06:13	                main:574]	:	INFO	:	Epoch 193 | loss: 0.0311034 | val_loss: 0.0311669 | Time: 2140.44 ms
[2021-09-24 11:06:15	                main:574]	:	INFO	:	Epoch 194 | loss: 0.0311048 | val_loss: 0.0311779 | Time: 2165.49 ms
[2021-09-24 11:06:17	                main:574]	:	INFO	:	Epoch 195 | loss: 0.0311036 | val_loss: 0.0311659 | Time: 2099.39 ms
[2021-09-24 11:06:20	                main:574]	:	INFO	:	Epoch 196 | loss: 0.0311041 | val_loss: 0.0311729 | Time: 2124.03 ms
[2021-09-24 11:06:22	                main:574]	:	INFO	:	Epoch 197 | loss: 0.0311039 | val_loss: 0.0311812 | Time: 2130.72 ms
[2021-09-24 11:06:24	                main:574]	:	INFO	:	Epoch 198 | loss: 0.031102 | val_loss: 0.0311679 | Time: 2122.01 ms
[2021-09-24 11:06:26	                main:574]	:	INFO	:	Epoch 199 | loss: 0.0311003 | val_loss: 0.0311736 | Time: 2115.45 ms
[2021-09-24 11:06:28	                main:574]	:	INFO	:	Epoch 200 | loss: 0.0310978 | val_loss: 0.0311708 | Time: 2157.07 ms
[2021-09-24 11:06:30	                main:574]	:	INFO	:	Epoch 201 | loss: 0.0311012 | val_loss: 0.0311766 | Time: 2072.27 ms
[2021-09-24 11:06:33	                main:574]	:	INFO	:	Epoch 202 | loss: 0.0311 | val_loss: 0.0311691 | Time: 2201.78 ms
[2021-09-24 11:06:35	                main:574]	:	INFO	:	Epoch 203 | loss: 0.0310989 | val_loss: 0.0311782 | Time: 2114.16 ms
[2021-09-24 11:06:37	                main:574]	:	INFO	:	Epoch 204 | loss: 0.031098 | val_loss: 0.0311747 | Time: 2182.02 ms
[2021-09-24 11:06:39	                main:574]	:	INFO	:	Epoch 205 | loss: 0.0310974 | val_loss: 0.0311736 | Time: 2115.91 ms
[2021-09-24 11:06:41	                main:574]	:	INFO	:	Epoch 206 | loss: 0.0310987 | val_loss: 0.0311742 | Time: 2178.83 ms
[2021-09-24 11:06:44	                main:574]	:	INFO	:	Epoch 207 | loss: 0.0310984 | val_loss: 0.0311701 | Time: 2174.88 ms
[2021-09-24 11:06:46	                main:574]	:	INFO	:	Epoch 208 | loss: 0.031114 | val_loss: 0.0311729 | Time: 2144.52 ms
[2021-09-24 11:06:48	                main:574]	:	INFO	:	Epoch 209 | loss: 0.0311176 | val_loss: 0.0311752 | Time: 2158.14 ms
[2021-09-24 11:06:50	                main:574]	:	INFO	:	Epoch 210 | loss: 0.0311091 | val_loss: 0.0311762 | Time: 2065.91 ms
[2021-09-24 11:06:52	                main:574]	:	INFO	:	Epoch 211 | loss: 0.0311081 | val_loss: 0.0311702 | Time: 2136.94 ms
[2021-09-24 11:06:54	                main:574]	:	INFO	:	Epoch 212 | loss: 0.0311039 | val_loss: 0.0311709 | Time: 2196.48 ms
[2021-09-24 11:06:57	                main:574]	:	INFO	:	Epoch 213 | loss: 0.0311036 | val_loss: 0.0311673 | Time: 2136.63 ms
[2021-09-24 11:06:59	                main:574]	:	INFO	:	Epoch 214 | loss: 0.0311008 | val_loss: 0.0311678 | Time: 2062.43 ms
[2021-09-24 11:07:01	                main:574]	:	INFO	:	Epoch 215 | loss: 0.0310985 | val_loss: 0.0311706 | Time: 2143.05 ms
[2021-09-24 11:07:03	                main:574]	:	INFO	:	Epoch 216 | loss: 0.0311008 | val_loss: 0.031168 | Time: 2137.77 ms
[2021-09-24 11:07:05	                main:574]	:	INFO	:	Epoch 217 | loss: 0.0311007 | val_loss: 0.0311687 | Time: 2123.48 ms
[2021-09-24 11:07:07	                main:574]	:	INFO	:	Epoch 218 | loss: 0.0311091 | val_loss: 0.0311789 | Time: 2093.49 ms
[2021-09-24 11:07:10	                main:574]	:	INFO	:	Epoch 219 | loss: 0.031106 | val_loss: 0.0311686 | Time: 2118.22 ms
[2021-09-24 11:07:12	                main:574]	:	INFO	:	Epoch 220 | loss: 0.0311002 | val_loss: 0.0311701 | Time: 2157.86 ms
[2021-09-24 11:07:14	                main:574]	:	INFO	:	Epoch 221 | loss: 0.0311009 | val_loss: 0.0311838 | Time: 2118.38 ms
[2021-09-24 11:07:16	                main:574]	:	INFO	:	Epoch 222 | loss: 0.0311027 | val_loss: 0.0311736 | Time: 2070.78 ms
[2021-09-24 11:07:18	                main:574]	:	INFO	:	Epoch 223 | loss: 0.0311012 | val_loss: 0.0311695 | Time: 2097.14 ms
[2021-09-24 11:07:20	                main:574]	:	INFO	:	Epoch 224 | loss: 0.031096 | val_loss: 0.0311732 | Time: 2162.51 ms
[2021-09-24 11:07:22	                main:574]	:	INFO	:	Epoch 225 | loss: 0.0310935 | val_loss: 0.031175 | Time: 2107.05 ms
[2021-09-24 11:07:25	                main:574]	:	INFO	:	Epoch 226 | loss: 0.0310928 | val_loss: 0.0311677 | Time: 2110.04 ms
[2021-09-24 11:07:27	                main:574]	:	INFO	:	Epoch 227 | loss: 0.0310945 | val_loss: 0.0311756 | Time: 2087.11 ms
[2021-09-24 11:07:29	                main:574]	:	INFO	:	Epoch 228 | loss: 0.0311016 | val_loss: 0.0311806 | Time: 2198.04 ms
[2021-09-24 11:07:31	                main:574]	:	INFO	:	Epoch 229 | loss: 0.031096 | val_loss: 0.0311815 | Time: 2071.66 ms
[2021-09-24 11:07:33	                main:574]	:	INFO	:	Epoch 230 | loss: 0.0310922 | val_loss: 0.0311833 | Time: 2106.16 ms
[2021-09-24 11:07:35	                main:574]	:	INFO	:	Epoch 231 | loss: 0.0310933 | val_loss: 0.0311766 | Time: 2126.52 ms
[2021-09-24 11:07:38	                main:574]	:	INFO	:	Epoch 232 | loss: 0.031101 | val_loss: 0.0311721 | Time: 2084.67 ms
[2021-09-24 11:07:40	                main:574]	:	INFO	:	Epoch 233 | loss: 0.0311028 | val_loss: 0.0311783 | Time: 2110.7 ms
[2021-09-24 11:07:42	                main:574]	:	INFO	:	Epoch 234 | loss: 0.0310982 | val_loss: 0.0311782 | Time: 2085.09 ms
[2021-09-24 11:07:44	                main:574]	:	INFO	:	Epoch 235 | loss: 0.0311054 | val_loss: 0.031175 | Time: 2161.59 ms
[2021-09-24 11:07:46	                main:574]	:	INFO	:	Epoch 236 | loss: 0.0311034 | val_loss: 0.0311754 | Time: 2029.54 ms
[2021-09-24 11:07:48	                main:574]	:	INFO	:	Epoch 237 | loss: 0.0311093 | val_loss: 0.0311684 | Time: 2219.27 ms
[2021-09-24 11:07:50	                main:574]	:	INFO	:	Epoch 238 | loss: 0.0311086 | val_loss: 0.03117 | Time: 2168.78 ms
[2021-09-24 11:07:53	                main:574]	:	INFO	:	Epoch 239 | loss: 0.0311035 | val_loss: 0.0311807 | Time: 2080.03 ms
[2021-09-24 11:07:55	                main:574]	:	INFO	:	Epoch 240 | loss: 0.0311034 | val_loss: 0.0311795 | Time: 2149.27 ms
[2021-09-24 11:07:57	                main:574]	:	INFO	:	Epoch 241 | loss: 0.0311001 | val_loss: 0.0311757 | Time: 2143.17 ms
[2021-09-24 11:07:59	                main:574]	:	INFO	:	Epoch 242 | loss: 0.0311019 | val_loss: 0.0311704 | Time: 2152.57 ms
[2021-09-24 11:08:01	                main:574]	:	INFO	:	Epoch 243 | loss: 0.0310958 | val_loss: 0.0311805 | Time: 2000.48 ms
[2021-09-24 11:08:03	                main:574]	:	INFO	:	Epoch 244 | loss: 0.0310938 | val_loss: 0.031172 | Time: 2125.65 ms
[2021-09-24 11:08:05	                main:574]	:	INFO	:	Epoch 245 | loss: 0.0310954 | val_loss: 0.0311771 | Time: 2090.3 ms
[2021-09-24 11:08:08	                main:574]	:	INFO	:	Epoch 246 | loss: 0.0310965 | val_loss: 0.0311664 | Time: 2073 ms
[2021-09-24 11:08:10	                main:574]	:	INFO	:	Epoch 247 | loss: 0.0310935 | val_loss: 0.0311794 | Time: 2161.03 ms
[2021-09-24 11:08:12	                main:574]	:	INFO	:	Epoch 248 | loss: 0.0310889 | val_loss: 0.0311738 | Time: 2156.9 ms
[2021-09-24 11:08:14	                main:574]	:	INFO	:	Epoch 249 | loss: 0.0310923 | val_loss: 0.0311817 | Time: 2180.53 ms
[2021-09-24 11:08:16	                main:574]	:	INFO	:	Epoch 250 | loss: 0.0311006 | val_loss: 0.031169 | Time: 2134.92 ms
[2021-09-24 11:08:18	                main:574]	:	INFO	:	Epoch 251 | loss: 0.0311052 | val_loss: 0.031172 | Time: 2159.29 ms
[2021-09-24 11:08:21	                main:574]	:	INFO	:	Epoch 252 | loss: 0.0311032 | val_loss: 0.0311764 | Time: 2116.93 ms
[2021-09-24 11:08:23	                main:574]	:	INFO	:	Epoch 253 | loss: 0.0310985 | val_loss: 0.0311731 | Time: 2089.96 ms
[2021-09-24 11:08:25	                main:574]	:	INFO	:	Epoch 254 | loss: 0.0310967 | val_loss: 0.0311759 | Time: 2141.17 ms
[2021-09-24 11:08:27	                main:574]	:	INFO	:	Epoch 255 | loss: 0.0310915 | val_loss: 0.0311814 | Time: 2113.34 ms
[2021-09-24 11:08:29	                main:574]	:	INFO	:	Epoch 256 | loss: 0.0310965 | val_loss: 0.031172 | Time: 2128.36 ms
[2021-09-24 11:08:31	                main:574]	:	INFO	:	Epoch 257 | loss: 0.0310955 | val_loss: 0.031184 | Time: 2108.46 ms
[2021-09-24 11:08:34	                main:574]	:	INFO	:	Epoch 258 | loss: 0.0310978 | val_loss: 0.0311724 | Time: 2122.73 ms
[2021-09-24 11:08:36	                main:574]	:	INFO	:	Epoch 259 | loss: 0.0310918 | val_loss: 0.0311748 | Time: 2106.24 ms
[2021-09-24 11:08:38	                main:574]	:	INFO	:	Epoch 260 | loss: 0.0310934 | val_loss: 0.0311737 | Time: 2155.56 ms
[2021-09-24 11:08:40	                main:574]	:	INFO	:	Epoch 261 | loss: 0.0310933 | val_loss: 0.0311834 | Time: 2099.81 ms
[2021-09-24 11:08:42	                main:574]	:	INFO	:	Epoch 262 | loss: 0.0310912 | val_loss: 0.0311786 | Time: 2108.11 ms
[2021-09-24 11:08:44	                main:574]	:	INFO	:	Epoch 263 | loss: 0.0310887 | val_loss: 0.0311791 | Time: 2200.63 ms
[2021-09-24 11:08:47	                main:574]	:	INFO	:	Epoch 264 | loss: 0.031089 | val_loss: 0.0311841 | Time: 2241.6 ms
[2021-09-24 11:08:49	                main:574]	:	INFO	:	Epoch 265 | loss: 0.0310874 | val_loss: 0.0311756 | Time: 2109.77 ms
[2021-09-24 11:08:51	                main:574]	:	INFO	:	Epoch 266 | loss: 0.0310892 | val_loss: 0.031171 | Time: 2202.66 ms
[2021-09-24 11:08:53	                main:574]	:	INFO	:	Epoch 267 | loss: 0.031093 | val_loss: 0.0311755 | Time: 2030.73 ms
[2021-09-24 11:08:55	                main:574]	:	INFO	:	Epoch 268 | loss: 0.0310968 | val_loss: 0.0311673 | Time: 2086.09 ms
[2021-09-24 11:08:57	                main:574]	:	INFO	:	Epoch 269 | loss: 0.0310989 | val_loss: 0.0311834 | Time: 2118.34 ms
[2021-09-24 11:09:00	                main:574]	:	INFO	:	Epoch 270 | loss: 0.0310953 | val_loss: 0.0311787 | Time: 2175.58 ms
[2021-09-24 11:09:02	                main:574]	:	INFO	:	Epoch 271 | loss: 0.0310905 | val_loss: 0.0311749 | Time: 2185.85 ms
[2021-09-24 11:09:04	                main:574]	:	INFO	:	Epoch 272 | loss: 0.0310915 | val_loss: 0.0311862 | Time: 2135.67 ms
[2021-09-24 11:09:06	                main:574]	:	INFO	:	Epoch 273 | loss: 0.0310909 | val_loss: 0.0311694 | Time: 2136.99 ms
[2021-09-24 11:09:08	                main:574]	:	INFO	:	Epoch 274 | loss: 0.0311011 | val_loss: 0.031172 | Time: 2035.28 ms
[2021-09-24 11:09:10	                main:574]	:	INFO	:	Epoch 275 | loss: 0.0311061 | val_loss: 0.0311731 | Time: 2152.89 ms
[2021-09-24 11:09:13	                main:574]	:	INFO	:	Epoch 276 | loss: 0.0311007 | val_loss: 0.031173 | Time: 2167.82 ms
[2021-09-24 11:09:15	                main:574]	:	INFO	:	Epoch 277 | loss: 0.0310953 | val_loss: 0.0311747 | Time: 2166.72 ms
[2021-09-24 11:09:17	                main:574]	:	INFO	:	Epoch 278 | loss: 0.031096 | val_loss: 0.0311774 | Time: 2130.51 ms
[2021-09-24 11:09:19	                main:574]	:	INFO	:	Epoch 279 | loss: 0.0310935 | val_loss: 0.0311781 | Time: 2136.09 ms
[2021-09-24 11:09:21	                main:574]	:	INFO	:	Epoch 280 | loss: 0.0310917 | val_loss: 0.0311856 | Time: 2071.08 ms
[2021-09-24 11:09:23	                main:574]	:	INFO	:	Epoch 281 | loss: 0.0310893 | val_loss: 0.0311821 | Time: 2123.53 ms
[2021-09-24 11:09:26	                main:574]	:	INFO	:	Epoch 282 | loss: 0.0310908 | val_loss: 0.031185 | Time: 2204.19 ms
[2021-09-24 11:09:28	                main:574]	:	INFO	:	Epoch 283 | loss: 0.0310958 | val_loss: 0.0311791 | Time: 2156.37 ms
[2021-09-24 11:09:30	                main:574]	:	INFO	:	Epoch 284 | loss: 0.0310932 | val_loss: 0.0311761 | Time: 2126.47 ms
[2021-09-24 11:09:32	                main:574]	:	INFO	:	Epoch 285 | loss: 0.0310915 | val_loss: 0.0311894 | Time: 2166.32 ms
[2021-09-24 11:09:34	                main:574]	:	INFO	:	Epoch 286 | loss: 0.0310959 | val_loss: 0.0311827 | Time: 2121.74 ms
[2021-09-24 11:09:36	                main:574]	:	INFO	:	Epoch 287 | loss: 0.0310966 | val_loss: 0.0311695 | Time: 2139.57 ms
[2021-09-24 11:09:39	                main:574]	:	INFO	:	Epoch 288 | loss: 0.0310965 | val_loss: 0.0311728 | Time: 2117.26 ms
[2021-09-24 11:09:41	                main:574]	:	INFO	:	Epoch 289 | loss: 0.0310946 | val_loss: 0.0311808 | Time: 2147.55 ms
[2021-09-24 11:09:43	                main:574]	:	INFO	:	Epoch 290 | loss: 0.0310901 | val_loss: 0.031173 | Time: 2171.44 ms
[2021-09-24 11:09:45	                main:574]	:	INFO	:	Epoch 291 | loss: 0.0310866 | val_loss: 0.0311804 | Time: 2164.26 ms
[2021-09-24 11:09:47	                main:574]	:	INFO	:	Epoch 292 | loss: 0.03109 | val_loss: 0.0311792 | Time: 2109.46 ms
[2021-09-24 11:09:50	                main:574]	:	INFO	:	Epoch 293 | loss: 0.0310869 | val_loss: 0.0311824 | Time: 2170.53 ms
[2021-09-24 11:09:52	                main:574]	:	INFO	:	Epoch 294 | loss: 0.0310897 | val_loss: 0.0311779 | Time: 2020.59 ms
[2021-09-24 11:09:54	                main:574]	:	INFO	:	Epoch 295 | loss: 0.0310851 | val_loss: 0.0311863 | Time: 2093.24 ms
[2021-09-24 11:09:56	                main:574]	:	INFO	:	Epoch 296 | loss: 0.031088 | val_loss: 0.0311831 | Time: 2076.56 ms
[2021-09-24 11:09:58	                main:574]	:	INFO	:	Epoch 297 | loss: 0.0310908 | val_loss: 0.0311742 | Time: 2153.69 ms
[2021-09-24 11:10:00	                main:574]	:	INFO	:	Epoch 298 | loss: 0.031099 | val_loss: 0.0311821 | Time: 2075.9 ms
[2021-09-24 11:10:02	                main:574]	:	INFO	:	Epoch 299 | loss: 0.0310988 | val_loss: 0.0311691 | Time: 2188.55 ms
[2021-09-24 11:10:04	                main:574]	:	INFO	:	Epoch 300 | loss: 0.031095 | val_loss: 0.03117 | Time: 2109.81 ms
[2021-09-24 11:10:07	                main:574]	:	INFO	:	Epoch 301 | loss: 0.0311074 | val_loss: 0.0311809 | Time: 2100.2 ms
[2021-09-24 11:10:09	                main:574]	:	INFO	:	Epoch 302 | loss: 0.0311022 | val_loss: 0.0311919 | Time: 2130.58 ms
[2021-09-24 11:10:11	                main:574]	:	INFO	:	Epoch 303 | loss: 0.031094 | val_loss: 0.031175 | Time: 2076.01 ms
[2021-09-24 11:10:13	                main:574]	:	INFO	:	Epoch 304 | loss: 0.0310949 | val_loss: 0.0311822 | Time: 2161.28 ms
[2021-09-24 11:10:15	                main:574]	:	INFO	:	Epoch 305 | loss: 0.0310919 | val_loss: 0.031177 | Time: 2134.99 ms
[2021-09-24 11:10:17	                main:574]	:	INFO	:	Epoch 306 | loss: 0.0310912 | val_loss: 0.0311746 | Time: 2144.26 ms
[2021-09-24 11:10:20	                main:574]	:	INFO	:	Epoch 307 | loss: 0.0310895 | val_loss: 0.0311756 | Time: 2173.94 ms
[2021-09-24 11:10:22	                main:574]	:	INFO	:	Epoch 308 | loss: 0.0310888 | val_loss: 0.0311817 | Time: 2176.28 ms
[2021-09-24 11:10:24	                main:574]	:	INFO	:	Epoch 309 | loss: 0.031094 | val_loss: 0.0311754 | Time: 2173.93 ms
[2021-09-24 11:10:26	                main:574]	:	INFO	:	Epoch 310 | loss: 0.0310976 | val_loss: 0.0311731 | Time: 2170.43 ms
[2021-09-24 11:10:28	                main:574]	:	INFO	:	Epoch 311 | loss: 0.0311103 | val_loss: 0.0311692 | Time: 2171.44 ms
[2021-09-24 11:10:31	                main:574]	:	INFO	:	Epoch 312 | loss: 0.0311057 | val_loss: 0.0311784 | Time: 2155.64 ms
[2021-09-24 11:10:33	                main:574]	:	INFO	:	Epoch 313 | loss: 0.0311066 | val_loss: 0.031172 | Time: 2084.49 ms
[2021-09-24 11:10:35	                main:574]	:	INFO	:	Epoch 314 | loss: 0.0311035 | val_loss: 0.031175 | Time: 2185.13 ms
[2021-09-24 11:10:37	                main:574]	:	INFO	:	Epoch 315 | loss: 0.0310981 | val_loss: 0.0311786 | Time: 2111.79 ms
[2021-09-24 11:10:39	                main:574]	:	INFO	:	Epoch 316 | loss: 0.0310947 | val_loss: 0.031172 | Time: 2199.92 ms
[2021-09-24 11:10:42	                main:574]	:	INFO	:	Epoch 317 | loss: 0.031096 | val_loss: 0.0311813 | Time: 2160.58 ms
[2021-09-24 11:10:44	                main:574]	:	INFO	:	Epoch 318 | loss: 0.0310956 | val_loss: 0.0311727 | Time: 2061.33 ms
[2021-09-24 11:10:46	                main:574]	:	INFO	:	Epoch 319 | loss: 0.0310947 | val_loss: 0.0311735 | Time: 2197.41 ms
[2021-09-24 11:10:48	                main:574]	:	INFO	:	Epoch 320 | loss: 0.0310937 | val_loss: 0.0311823 | Time: 2132.87 ms
[2021-09-24 11:10:50	                main:574]	:	INFO	:	Epoch 321 | loss: 0.031091 | val_loss: 0.0311789 | Time: 2168.51 ms
[2021-09-24 11:10:52	                main:574]	:	INFO	:	Epoch 322 | loss: 0.0310927 | val_loss: 0.031172 | Time: 2170.2 ms
[2021-09-24 11:10:55	                main:574]	:	INFO	:	Epoch 323 | loss: 0.0310907 | val_loss: 0.0311724 | Time: 2184.45 ms
[2021-09-24 11:10:57	                main:574]	:	INFO	:	Epoch 324 | loss: 0.0310878 | val_loss: 0.0311799 | Time: 2171.17 ms
[2021-09-24 11:10:59	                main:574]	:	INFO	:	Epoch 325 | loss: 0.0310933 | val_loss: 0.0311687 | Time: 2105.5 ms
[2021-09-24 11:11:01	                main:574]	:	INFO	:	Epoch 326 | loss: 0.0310976 | val_loss: 0.0311738 | Time: 2271.13 ms
[2021-09-24 11:11:03	                main:574]	:	INFO	:	Epoch 327 | loss: 0.0310954 | val_loss: 0.0311782 | Time: 2164.17 ms
[2021-09-24 11:11:06	                main:574]	:	INFO	:	Epoch 328 | loss: 0.0310977 | val_loss: 0.0311648 | Time: 2116.86 ms
[2021-09-24 11:11:08	                main:574]	:	INFO	:	Epoch 329 | loss: 0.0310946 | val_loss: 0.0311716 | Time: 2189.21 ms
[2021-09-24 11:11:10	                main:574]	:	INFO	:	Epoch 330 | loss: 0.0311009 | val_loss: 0.0311738 | Time: 2103.32 ms
[2021-09-24 11:11:12	                main:574]	:	INFO	:	Epoch 331 | loss: 0.031089 | val_loss: 0.0311802 | Time: 2123.01 ms
[2021-09-24 11:11:14	                main:574]	:	INFO	:	Epoch 332 | loss: 0.0310884 | val_loss: 0.0311728 | Time: 2128.84 ms
[2021-09-24 11:11:16	                main:574]	:	INFO	:	Epoch 333 | loss: 0.0310944 | val_loss: 0.0311785 | Time: 2166.16 ms
[2021-09-24 11:11:18	                main:574]	:	INFO	:	Epoch 334 | loss: 0.0310884 | val_loss: 0.0311758 | Time: 2053.95 ms
[2021-09-24 11:11:21	                main:574]	:	INFO	:	Epoch 335 | loss: 0.0310954 | val_loss: 0.0311738 | Time: 2159.2 ms
[2021-09-24 11:11:23	                main:574]	:	INFO	:	Epoch 336 | loss: 0.0310968 | val_loss: 0.0311747 | Time: 2141.97 ms
[2021-09-24 11:11:25	                main:574]	:	INFO	:	Epoch 337 | loss: 0.0310921 | val_loss: 0.0311729 | Time: 2149.11 ms
[2021-09-24 11:11:27	                main:574]	:	INFO	:	Epoch 338 | loss: 0.0310896 | val_loss: 0.0311713 | Time: 2160.03 ms
[2021-09-24 11:11:29	                main:574]	:	INFO	:	Epoch 339 | loss: 0.0310881 | val_loss: 0.0311798 | Time: 2174.77 ms
[2021-09-24 11:11:32	                main:574]	:	INFO	:	Epoch 340 | loss: 0.0310883 | val_loss: 0.0311753 | Time: 2183.52 ms
[2021-09-24 11:11:34	                main:574]	:	INFO	:	Epoch 341 | loss: 0.0310853 | val_loss: 0.0311779 | Time: 2023.09 ms
[2021-09-24 11:11:36	                main:574]	:	INFO	:	Epoch 342 | loss: 0.0310904 | val_loss: 0.0311806 | Time: 2118.61 ms
[2021-09-24 11:11:38	                main:574]	:	INFO	:	Epoch 343 | loss: 0.0310947 | val_loss: 0.0311694 | Time: 2208.88 ms
[2021-09-24 11:11:40	                main:574]	:	INFO	:	Epoch 344 | loss: 0.0310906 | val_loss: 0.0311841 | Time: 2144.21 ms
[2021-09-24 11:11:42	                main:574]	:	INFO	:	Epoch 345 | loss: 0.0310876 | val_loss: 0.0311687 | Time: 2129.29 ms
[2021-09-24 11:11:45	                main:574]	:	INFO	:	Epoch 346 | loss: 0.031083 | val_loss: 0.0311771 | Time: 2172.84 ms
[2021-09-24 11:11:47	                main:574]	:	INFO	:	Epoch 347 | loss: 0.0310836 | val_loss: 0.0311788 | Time: 2119.13 ms
[2021-09-24 11:11:49	                main:574]	:	INFO	:	Epoch 348 | loss: 0.0310828 | val_loss: 0.0311768 | Time: 2129.24 ms
[2021-09-24 11:11:51	                main:574]	:	INFO	:	Epoch 349 | loss: 0.0310844 | val_loss: 0.0311787 | Time: 2191.27 ms
[2021-09-24 11:11:53	                main:574]	:	INFO	:	Epoch 350 | loss: 0.0310872 | val_loss: 0.031174 | Time: 2120.81 ms
THCudaCheck FAIL file=..\..\aten\src\THC\THCCachingHostAllocator.cpp line=278 error=719 : unspecified launch failure


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFDE5DA4F38

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 09/24/21 11:11:55
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\7;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 000000006ee90000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e8e60000 00000000001cf000 C:\Windows\SYSTEM32\ntdll.dll (6.2.14393.4530) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.14393.4283 (rs1_release.210303-1802)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4283

ModLoad: 00000000e6460000 00000000000ad000 C:\Windows\System32\KERNEL32.DLL (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.14393.4283 (rs1_release.210303-1802)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4283

ModLoad: 00000000e5d70000 000000000021e000 C:\Windows\System32\KERNELBASE.dll (6.2.14393.4350) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.14393.4283 (rs1_release.210303-1802)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4283

ModLoad: 00000000e6dc0000 0000000000166000 C:\Windows\System32\USER32.dll (6.2.14393.4169) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5cb0000 000000000001e000 C:\Windows\System32\win32u.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6520000 0000000000034000 C:\Windows\System32\GDI32.dll (6.2.14393.4169) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.14393.4169 (rs1_release.210107-1130)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4169

ModLoad: 00000000d82a0000 0000000000058000 C:\ProgramData\BOINC\slots\7\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e61c0000 0000000000183000 C:\Windows\System32\gdi32full.dll (6.2.14393.4530) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.14393.4530 (rs1_release.210705-0736)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4530

ModLoad: 00000000b2b20000 000000000abe7000 C:\ProgramData\BOINC\slots\7\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e8750000 00000000000a2000 C:\Windows\System32\ADVAPI32.dll (6.2.14393.4467) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6560000 000000000009e000 C:\Windows\System32\msvcrt.dll (7.0.14393.2457) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.14393.2457 (rs1_release_inmarket.180822-1743)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.14393.2457

ModLoad: 00000000e6880000 0000000000059000 C:\Windows\System32\sechost.dll (6.2.14393.3808) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6600000 000000000011f000 C:\Windows\System32\RPCRT4.dll (6.2.14393.4467) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5bb0000 00000000000f4000 C:\Windows\System32\ucrtbase.dll (6.2.14393.3659) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.14393.3659 (rs1_release_1.200410-1813)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.3659

ModLoad: 00000000bd710000 0000000000091000 C:\Windows\SYSTEM32\MSVCP140.dll (14.25.28508.3) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\8\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.25.28508.3 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.25.28508.3

ModLoad: 00000000d5a00000 0000000000192000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.14321.1024) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.14321.1024 (rs1_release.160715-1616)
    Company Name          : Microsoft
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14321.1024

ModLoad: 00000000c4280000 0000000000019000 C:\Windows\SYSTEM32\VCRUNTIME140.dll (14.25.28508.3) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\8\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.25.28508.3 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.25.28508.3

ModLoad: 00000000d7cc0000 000000000000c000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll (14.25.28508.3) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\8\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.25.28508.3 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.25.28508.3

ModLoad: 0000000044b40000 0000000024b49000 C:\ProgramData\BOINC\slots\7\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000d2220000 000000000020d000 C:\ProgramData\BOINC\slots\7\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 00000000d1f40000 00000000002db000 C:\ProgramData\BOINC\slots\7\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000d6bd0000 0000000000040000 C:\ProgramData\BOINC\slots\7\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000d1ef0000 0000000000045000 C:\ProgramData\BOINC\slots\7\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000bd9d0000 00000000030a5000 C:\ProgramData\BOINC\slots\7\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 00000000c0a80000 00000000034b2000 C:\ProgramData\BOINC\slots\7\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000d3840000 000000001b3a4000 C:\ProgramData\BOINC\slots\7\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 00000000d96b0000 0000000000010000 C:\ProgramData\BOINC\slots\7\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 00000000a3d50000 000000000919f000 C:\ProgramData\BOINC\slots\7\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000aedc0000 0000000003d5b000 C:\ProgramData\BOINC\slots\7\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000acef0000 0000000001eca000 C:\ProgramData\BOINC\slots\7\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000e8510000 000000000002e000 C:\Windows\System32\IMM32.DLL (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e52d0000 000000000000f000 C:\Windows\System32\kernel.appcore.dll (6.2.14393.2312) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.14393.2312 (rs1_release.180607-1919)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.2312

ModLoad: 00000000e88f0000 0000000000429000 C:\Windows\System32\Setupapi.dll (6.2.14393.2608) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5360000 0000000000042000 C:\Windows\System32\cfgmgr32.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e3fa0000 0000000000028000 C:\Windows\SYSTEM32\DEVOBJ.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5f90000 0000000000055000 C:\Windows\System32\WINTRUST.dll (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.14393.4651 (rs1_release.210911-1554)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4651

ModLoad: 00000000e5300000 0000000000010000 C:\Windows\System32\MSASN1.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5ff0000 00000000001c9000 C:\Windows\System32\CRYPT32.dll (6.2.14393.4350) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6f30000 0000000001505000 C:\Windows\System32\Shell32.dll (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e54d0000 00000000006d3000 C:\Windows\System32\windows.storage.dll (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6af0000 00000000002c6000 C:\Windows\System32\combase.dll (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.14393.4651 (rs1_release.210911-1554)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4651

ModLoad: 00000000e53b0000 000000000006c000 C:\Windows\System32\bcryptPrimitives.dll (6.2.14393.4583) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.14393.4583 (rs1_release.210730-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4583

ModLoad: 00000000e5310000 000000000004c000 C:\Windows\System32\powrprof.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : powrprof.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e6820000 0000000000052000 C:\Windows\System32\shlwapi.dll (6.2.14393.4169) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e5420000 00000000000a8000 C:\Windows\System32\shcore.dll (6.2.14393.4169) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e52e0000 0000000000014000 C:\Windows\System32\profapi.dll (6.2.14393.0) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : profapi.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 0000000000200000 000000000029d000 C:\Windows\system32\nvcuda.dll (27.21.14.6089) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r460_74\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
    File Version          : 27.21.14.6089
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.2.66 driver
    Product Version       : 27.21.14.6089

ModLoad: 00000000e0790000 000000000000a000 C:\Windows\SYSTEM32\VERSION.DLL (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000fec30000 00000000015c2000 C:\Windows\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_2635d5c616c804dc\nvcuda64.dll (27.21.14.6089) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r460_74\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
    File Version          : 27.21.14.6089
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.2.66 driver
    Product Version       : 27.21.14.6089

ModLoad: 00000000e68e0000 000000000006a000 C:\Windows\System32\WS2_32.dll (6.2.14393.3241) (-exported- Symbols Loaded)
    Linked PDB Filename   : ws2_32.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e0750000 000000000002f000 C:\Windows\SYSTEM32\cryptnet.dll (6.2.14393.2035) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptnet.pdb
    File Version          : 10.0.14393.2035 (rs1_release_inmarket.180110-1910)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.2035

ModLoad: 00000000e4d40000 000000000000b000 C:\Windows\SYSTEM32\cryptbase.dll (6.2.14393.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000e3b50000 0000000000095000 C:\Windows\system32\uxtheme.dll (6.2.14393.4169) (-exported- Symbols Loaded)
    Linked PDB Filename   : UxTheme.pdb
    File Version          : 10.0.14393.0 (rs1_release.160715-1616)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.0

ModLoad: 00000000def80000 00000000006f0000 C:\Windows\SYSTEM32\nvapi64.dll (27.21.14.6089) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r460_74\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
    File Version          : 27.21.14.6089
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA Windows drivers
    Product Version       : 27.21.14.6089

ModLoad: 00000000e8d20000 0000000000139000 C:\Windows\System32\OLE32.DLL (6.2.14393.4651) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 10.0.14393.4651 (rs1_release.210911-1554)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.14393.4651

ModLoad: 00000000d7ca0000 0000000000009000 C:\ProgramData\BOINC\slots\7\caffe2_nvrtc.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000a2d80000 0000000000fca000 C:\ProgramData\BOINC\slots\7\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6.14.11.9000
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.89 NVRTC Library
    Product Version       : 6.14.11.9000



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3975, Write: 1624, Other 6232

- I/O Transfers Counters -
Read: 6537719, Write: 142012, Other 81726

- Paged Pool Usage -
QuotaPagedPoolUsage: 3519760, QuotaPeakPagedPoolUsage: 3519760
QuotaNonPagedPoolUsage: 50712, QuotaPeakNonPagedPoolUsage: 51248

- Virtual Memory Usage -
VirtualSize: -389242880, PeakVirtualSize: 199704576

- Pagefile Usage -
PagefileUsage: -389242880, PeakPagefileUsage: -389242880

- Working Set Size -
WorkingSetSize: 1761652736, PeakWorkingSetSize: 1764720640, PageFaultCount: 631356

*** Dump of thread ID 5308 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 148281248.000000, User Time: 4164218624.000000, Wait Time: 8339837.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFDE5DA4F38

- Registers -
rax=0000000000000000 rbx=00000000bcf962f0 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000015ee4400 rdi=0000000019930520
r8=00000000e5bc56ed r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000006000 r13=0000000000000000
r14=0000000015ee5140 r15=0000000015ee50c0 rip=00000000e5da4f38 rsp=0000000015ee42a0 rbp=0000000015ee4ca8
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
15ee4370 c42861e0 00000002 00000000 15ee5140 b80ea014 KERNELBASE!RaiseException+0x0 
15ee43d0 b86e5272 00000002 e5bc6e24 000007cb 00000116 VCRUNTIME140!_CxxThrowException+0x0 
15ee4420 b86e56d4 00000800 00000000 bb70d290 483a5f68 torch_cpu!at::Tensor::zero_+0x0 
15ee4c80 47969ebe 15ee4eb0 483a6850 483a6880 000002cf torch_cpu!_THError+0x0 
15ee4cc0 47965412 15ee4eb0 15ee4e10 c6176410 487da720 torch_cuda!__THCudaCheck+0x0 
15ee4d00 b83959ff 15ee5000 15ee4eb0 15ee5380 d82d97b0 torch_cuda!at::native::set_storage_cuda_+0x0 
15ee5090 b853726f 47fdf630 b80eea63 00010001 d82bef8f torch_cpu!at::native::empty_cpu+0x0 
15ee50d0 b84f0615 15ee0000 15ee5148 15ee5340 d82a7ce6 torch_cpu!at::native::mkldnn_sigmoid_+0x0 
15ee5110 b84f204a 15ee0000 15ee5100 c6152800 46ca6ae5 torch_cpu!at::native::mkldnn_sigmoid_+0x0 
15ee51a0 b84f0615 00000000 b80eea63 15ee5280 15ee5380 torch_cpu!at::native::mkldnn_sigmoid_+0x0 
15ee51e0 b85c1eb0 15ee0000 00000000 15ee5368 00010001 torch_cpu!at::native::mkldnn_sigmoid_+0x0 
15ee5280 b85d2eec 0a5e232d d82d9c50 15ee53e0 d82a3bd6 torch_cpu!at::bucketize_out+0x0 
15ee52d0 4706f160 15ee0000 00000000 15ee53e0 00000000 torch_cpu!at::empty+0x0 
15ee5670 47072de8 d82d9c50 b80f0528 c7066f60 d82d9c50 torch_cuda!at::native::cummin_helper_cuda+0x0 
15ee5850 47074379 15ee5960 2219bd50 15ee5d60 2219bd68 torch_cuda!at::native::cummin_helper_cuda+0x0 
15ee5a90 47073d64 15ee0000 15ee5d60 15ee5ba0 15ee5d60 torch_cuda!at::native::cat_out_cuda+0x0 
15ee5c90 47911dd2 15ee5ce8 00010001 15ee5ce8 15ee5d60 torch_cuda!at::native::cat_cuda+0x0 
15ee5d10 47924179 00000001 15ee5d50 c70656f8 04000000 torch_cuda!at::native::set_storage_cuda_+0x0 
15ee5d50 b85c1d7d 0f7265a0 15ee60c0 15ee6090 00000000 torch_cuda!at::native::set_storage_cuda_+0x0 
15ee5dd0 b85a0d81 00000000 d82c3994 15ee65a0 22196f50 torch_cpu!at::bucketize_out+0x0 
15ee5e10 b83ab4a5 15ee5e50 c7066b60 15ee5f20 15ee5f90 torch_cpu!at::_cat+0x0 
15ee5ee0 b86927bd 15ee5fb0 00010101 15ee5f38 d82f1ea8 torch_cpu!at::native::cat+0x0 
15ee5f60 b81529ae 00000001 d1ef23c5 c57f0840 15ee5f18 torch_cpu!at::zeros_out+0x0 
15ee5fa0 b85c1d7d 15ee5fe0 15ee6050 00000000 e5bbde0b torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0 
15ee6020 b85c75d1 15ee65c0 bd440640 15ee65a0 22196f50 torch_cpu!at::bucketize_out+0x0 
15ee6060 b83b7d4c 15ee62f0 00000000 15ee62f0 00000000 torch_cpu!at::cat+0x0 
15ee6220 b86ac38d 15ee6278 00010001 15ee6278 15ee62f0 torch_cpu!at::native::stack+0x0 
15ee62a0 b81529ae 15ee65a0 22196f50 00000000 00000000 torch_cpu!at::zeros_out+0x0 
15ee62e0 b85c1d7d 15ee6320 15ee6390 00000400 bb361c8f torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0 
15ee6360 b86102e1 15ee6490 d82c3720 15ee65c0 bd440640 torch_cpu!at::bucketize_out+0x0 
15ee63a0 b99a3839 15ee6420 c706d0e0 c706d0e0 15ee64b0 torch_cpu!at::stack+0x0 
15ee6570 b81529ae 00000000 15ee66c0 00000000 15ee6660 torch_cpu!torch::autograd::GraphRoot::apply+0x0 
15ee65b0 b85c1d7d 15ee65f0 15ee6660 00000400 00000000 torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0 
15ee6630 b86102e1 15ee67a0 b80e9cac 1dad93f0 22196f50 torch_cpu!at::bucketize_out+0x0 
15ee6670 6ee9ff7e 15ee6738 226342a0 226342a0 15ee6880 torch_cpu!at::stack+0x0 
15ee6710 6eea2ef8 6ee9fe30 15ee6750 15ee6750 15ee67a0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::addCallback+0x0 
15ee6770 6eea3ea4 0f9b5290 00000000 15ee6910 22196f50 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::getSlot+0x0 
15ee68b0 6ee9d3d2 03157fe8 21526300 15ee6910 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0 
15ee68e0 6eea4541 00000000 0142d8f0 00000000 e5bbde0b mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::operator=+0x0 
15ee6a90 6eea743a 00000000 00000000 15ee75a0 0000000c mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0 
15ee6f90 6eeab950 15ee7090 15ee7090 00000000 a412ecb0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::then+0x0 
15effd30 6f0ec6f8 c70cd360 00000000 00000000 c55a9840 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
15effd70 e64684d4 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
15effda0 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
15effdf0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 2304 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 8339832.000000

- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000163ffcf0 rsi=0000000000000064 rdi=0000000000000000
r8=00000000163ffcc8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=00000000163ffcf0 r15=0000000000000000 rip=00000000e8f062f4 rsp=00000000163ffcc8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
163ffcc0 e5db8df7 163ffd88 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0 
163ffd60 6eeca07f 00000000 00000000 00000000 00000000 KERNELBASE!SleepEx+0x0 
163ffd90 e64684d4 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
163ffdc0 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
163ffe10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 3052 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 8339837.000000

- Registers -
rax=000000000000005b rbx=0000000000000008 rcx=0000000000000008 rdx=00000000164ffa90 rsi=0000000000000000 rdi=0000000000000008
r8=00000000c7540090 r9=0000000000000000 r10=0000000000000000 r11=00000000164ffa00 r12=0000000000000064 r13=00000000164ffa90
r14=00000000164ff780 r15=0000000000000000 rip=00000000e8f067c4 rsp=00000000164ff728 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
164ff720 e5db8c1f 00000002 e8e7859f e8d20000 00000002 ntdll!ZwWaitForMultipleObjects+0x0 
164ffa20 e5db8b1e 003a0043 006e0069 00000008 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
164ffa60 fede66d4 c7284330 c55b3af0 ff1381f8 e8e91cb6 KERNELBASE!WaitForMultipleObjects+0x0 
164ffca0 fee7e51d 00000000 c727f0b0 00000000 00000001 nvcuda64!+0x0 
164ffd50 fede60a3 766534c6 76562feb c78de8d0 00000000 nvcuda64!cuProfilerStop+0x0 
164ffd80 ff0fd794 c78de8d0 00000000 00000000 00000000 nvcuda64!+0x0 
164ffdb0 e64684d4 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
164ffde0 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
164ffe30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 5356 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 8290555.000000

- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=0000000000000224 rdx=0000000000000000 rsi=00000000ffffffff rdi=0000000000000224
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000e8f05cf4 rsp=00000000165ff7e8 rbp=00000000c72ee4d0
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
165ff7e0 e5da4daf 00000000 00000000 00000000 00000000 ntdll!NtWaitForSingleObject+0x0 
165ff880 feffcd6e c78de810 e5d96d82 00000000 00000224 KERNELBASE!WaitForSingleObjectEx+0x0 
165ff8f0 fede60a3 c72ee4d0 00000000 c78de810 00000000 nvcuda64!cuProfilerStop+0x0 
165ff920 ff0fd794 c78de810 00000000 00000000 00000000 nvcuda64!+0x0 
165ff950 e64684d4 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
165ff980 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
165ff9d0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 3860 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 8339810.000000

- Registers -
rax=00000000000001bc rbx=0000000000000000 rcx=00000000c7543298 rdx=00000000166ff9d8 rsi=0000000000000000 rdi=0000000000000000
r8=00000000166ff9d8 r9=0000000000000f14 r10=00000000c7547a10 r11=0000000046f83cc0 r12=00000000166ff9d8 r13=0000000000000000
r14=00000000c7543298 r15=00000000c7547a10 rip=00000000e8f093e4 rsp=00000000166ff918 rbp=00000000166ff970
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
166ff910 e8ec25c9 00000000 00000000 00000000 00000000 ntdll!NtWaitForAlertByThreadId+0x0 
166ff980 e5dd9b78 c75451f0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0 
166ff9b0 fed64a5c 00000001 c75451f0 00000001 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0 
166ffa10 fede60a3 00000004 765ab00a 765ab00b 00000000 nvcuda64!+0x0 
166ffa40 ff0fd794 c78deba0 00000000 00000000 00000000 nvcuda64!+0x0 
166ffa70 e64684d4 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
166ffaa0 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
166ffaf0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 5768 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 8291123.000000

- Registers -
rax=00000000000001bc rbx=0000000000000000 rcx=0000000000f3af48 rdx=0000000000000000 rsi=0000000000f3af48 rdi=0000000000000000
r8=0000000000000000 r9=00000000167ffcf0 r10=000000002218f010 r11=00000000167ff8e0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000f3aef8 rip=00000000e8f093e4 rsp=00000000167ff7e8 rbp=00000000167ff840
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
167ff7e0 e8ec2465 167ff940 2218f010 00000000 d1efc43b ntdll!NtWaitForAlertByThreadId+0x0 
167ff860 e5dd9b28 00f3aef0 00f3af38 00000000 167ffa20 ntdll!RtlSleepConditionVariableSRW+0x0 
167ff8a0 bd74bc2d 00000000 e8ebdb52 167ffd40 00000000 KERNELBASE!SleepConditionVariableSRW+0x0 
167ff8e0 bd7228c8 167ffc50 167ffaf8 00000000 c55bba20 MSVCP140!__crtSetThreadpoolWait+0x0 
167ff910 bd722790 00f3af38 bd722f97 00000000 00000000 MSVCP140!std::_Winerror_message+0x0 
167ff980 b9d0064b e5bcfb40 00000000 00f3aef0 00f3aef0 MSVCP140!std::_Winerror_message+0x0 
167ff9c0 b9d025b1 22182680 00000001 00000000 00000801 torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0 
167ffcb0 b9d02471 bd226d10 167ffcf0 bd226d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0 
167ffd00 b9cf9744 e5c95b38 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0 
167ffd30 e5bcfb80 00000000 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0 
167ffd60 e64684d4 00000000 00000000 00000000 00000000 ucrtbase!_o__realloc_base+0x0 
167ffd90 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
167ffde0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 6048 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 78593752.000000, User Time: 3073593856.000000, Wait Time: 8339836.000000

- Registers -
rax=00000000000001bc rbx=0000000000000000 rcx=0000000000f3b008 rdx=0000000000000000 rsi=0000000000f3b008 rdi=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000f3b008 r11=0000000000f3afb8 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000f3afb8 rip=00000000e8f093e4 rsp=00000000168ff698 rbp=00000000168ff6f0
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
168ff690 e8ec2465 1c69ef28 1c69ef28 00000003 00000000 ntdll!NtWaitForAlertByThreadId+0x0 
168ff710 e5dd9b28 00f3afb0 00f3aff8 00000000 00000000 ntdll!RtlSleepConditionVariableSRW+0x0 
168ff750 bd74bc2d b4ac1460 b4ac1460 b4ac1460 0f742500 KERNELBASE!SleepConditionVariableSRW+0x0 
168ff790 bd7228c8 0f7425b0 00000000 00000001 b9d005b4 MSVCP140!__crtSetThreadpoolWait+0x0 
168ff7c0 bd722790 00f3aff8 bd722f97 00000000 0f7425b0 MSVCP140!std::_Winerror_message+0x0 
168ff830 b9d0064b 0f742590 000001c0 00f3afb0 00f3afb0 MSVCP140!std::_Winerror_message+0x0 
168ff870 b9d025b1 22182f20 ffffffff 0f742590 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0 
168ffb60 b9d02471 bd226d10 168ffba0 bd226d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0 
168ffbb0 b9cf9744 00000000 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0 
168ffbe0 e5bcfb80 00000000 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0 
168ffc10 e64684d4 00000000 00000000 00000000 00000000 ucrtbase!_o__realloc_base+0x0 
168ffc40 e8eb1781 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
168ffc90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)