Task 14151864
| Name | ParityModified-1647048621-21619-1-0_2 |
| Workunit | 10612573 |
| Created | 31 Mar 2022, 17:46:40 UTC |
| Sent | 31 Mar 2022, 18:34:51 UTC |
| Report deadline | 8 Apr 2022, 18:34:51 UTC |
| Received | 31 Mar 2022, 21:17:01 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 18789 |
| Run time | 55 min 29 sec |
| CPU time | 55 min 11 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 11,249.19 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.57 GB |
| Peak swap size | 4.35 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
me: 1514.8 ms
[2022-03-31 16:04:36 main:574] : INFO : Epoch 309 | loss: nan | val_loss: nan | Time: 1559.11 ms
[2022-03-31 16:04:38 main:574] : INFO : Epoch 310 | loss: nan | val_loss: nan | Time: 1533.58 ms
[2022-03-31 16:04:39 main:574] : INFO : Epoch 311 | loss: nan | val_loss: nan | Time: 1558.82 ms
[2022-03-31 16:04:41 main:574] : INFO : Epoch 312 | loss: nan | val_loss: nan | Time: 1549.71 ms
[2022-03-31 16:04:42 main:574] : INFO : Epoch 313 | loss: nan | val_loss: nan | Time: 1390.98 ms
[2022-03-31 16:04:44 main:574] : INFO : Epoch 314 | loss: nan | val_loss: nan | Time: 1327.96 ms
[2022-03-31 16:04:45 main:574] : INFO : Epoch 315 | loss: nan | val_loss: nan | Time: 1570.2 ms
[2022-03-31 16:04:47 main:574] : INFO : Epoch 316 | loss: nan | val_loss: nan | Time: 1503.85 ms
[2022-03-31 16:04:48 main:574] : INFO : Epoch 317 | loss: nan | val_loss: nan | Time: 1561.58 ms
[2022-03-31 16:04:50 main:574] : INFO : Epoch 318 | loss: nan | val_loss: nan | Time: 1542.93 ms
[2022-03-31 16:04:51 main:574] : INFO : Epoch 319 | loss: nan | val_loss: nan | Time: 1543.47 ms
[2022-03-31 16:04:53 main:574] : INFO : Epoch 320 | loss: nan | val_loss: nan | Time: 1265.85 ms
[2022-03-31 16:04:54 main:574] : INFO : Epoch 321 | loss: nan | val_loss: nan | Time: 1440.61 ms
[2022-03-31 16:04:56 main:574] : INFO : Epoch 322 | loss: nan | val_loss: nan | Time: 1586.71 ms
[2022-03-31 16:04:57 main:574] : INFO : Epoch 323 | loss: nan | val_loss: nan | Time: 1550.58 ms
[2022-03-31 16:04:59 main:574] : INFO : Epoch 324 | loss: nan | val_loss: nan | Time: 1516.7 ms
[2022-03-31 16:05:00 main:574] : INFO : Epoch 325 | loss: nan | val_loss: nan | Time: 1573.26 ms
[2022-03-31 16:05:02 main:574] : INFO : Epoch 326 | loss: nan | val_loss: nan | Time: 1576.39 ms
[2022-03-31 16:05:03 main:574] : INFO : Epoch 327 | loss: nan | val_loss: nan | Time: 1086.24 ms
[2022-03-31 16:05:05 main:574] : INFO : Epoch 328 | loss: nan | val_loss: nan | Time: 1560.86 ms
[2022-03-31 16:05:06 main:574] : INFO : Epoch 329 | loss: nan | val_loss: nan | Time: 1543.91 ms
[2022-03-31 16:05:08 main:574] : INFO : Epoch 330 | loss: nan | val_loss: nan | Time: 1597.85 ms
[2022-03-31 16:05:09 main:574] : INFO : Epoch 331 | loss: nan | val_loss: nan | Time: 1538.18 ms
[2022-03-31 16:05:11 main:574] : INFO : Epoch 332 | loss: nan | val_loss: nan | Time: 1550.39 ms
[2022-03-31 16:05:12 main:574] : INFO : Epoch 333 | loss: nan | val_loss: nan | Time: 1458.17 ms
[2022-03-31 16:05:14 main:574] : INFO : Epoch 334 | loss: nan | val_loss: nan | Time: 1232.9 ms
[2022-03-31 16:05:15 main:574] : INFO : Epoch 335 | loss: nan | val_loss: nan | Time: 1556.63 ms
[2022-03-31 16:05:17 main:574] : INFO : Epoch 336 | loss: nan | val_loss: nan | Time: 1568.79 ms
[2022-03-31 16:05:18 main:574] : INFO : Epoch 337 | loss: nan | val_loss: nan | Time: 1542.22 ms
[2022-03-31 16:05:20 main:574] : INFO : Epoch 338 | loss: nan | val_loss: nan | Time: 1549.62 ms
[2022-03-31 16:05:21 main:574] : INFO : Epoch 339 | loss: nan | val_loss: nan | Time: 1556.07 ms
[2022-03-31 16:05:23 main:574] : INFO : Epoch 340 | loss: nan | val_loss: nan | Time: 1378.48 ms
[2022-03-31 16:05:24 main:574] : INFO : Epoch 341 | loss: nan | val_loss: nan | Time: 1389.73 ms
[2022-03-31 16:05:26 main:574] : INFO : Epoch 342 | loss: nan | val_loss: nan | Time: 1541.48 ms
[2022-03-31 16:05:27 main:574] : INFO : Epoch 343 | loss: nan | val_loss: nan | Time: 1568.62 ms
[2022-03-31 16:05:29 main:574] : INFO : Epoch 344 | loss: nan | val_loss: nan | Time: 1554.91 ms
[2022-03-31 16:05:30 main:574] : INFO : Epoch 345 | loss: nan | val_loss: nan | Time: 1548.95 ms
[2022-03-31 16:05:32 main:574] : INFO : Epoch 346 | loss: nan | val_loss: nan | Time: 1567.66 ms
[2022-03-31 16:05:33 main:574] : INFO : Epoch 347 | loss: nan | val_loss: nan | Time: 1110.92 ms
[2022-03-31 16:05:35 main:574] : INFO : Epoch 348 | loss: nan | val_loss: nan | Time: 1554.09 ms
[2022-03-31 16:05:36 main:574] : INFO : Epoch 349 | loss: nan | val_loss: nan | Time: 1550.67 ms
[2022-03-31 16:05:38 main:574] : INFO : Epoch 350 | loss: nan | val_loss: nan | Time: 1583.94 ms
[2022-03-31 16:05:39 main:574] : INFO : Epoch 351 | loss: nan | val_loss: nan | Time: 1571.98 ms
[2022-03-31 16:05:41 main:574] : INFO : Epoch 352 | loss: nan | val_loss: nan | Time: 1532.97 ms
[2022-03-31 16:05:42 main:574] : INFO : Epoch 353 | loss: nan | val_loss: nan | Time: 1493.2 ms
[2022-03-31 16:05:44 main:574] : INFO : Epoch 354 | loss: nan | val_loss: nan | Time: 1245.85 ms
[2022-03-31 16:05:45 main:574] : INFO : Epoch 355 | loss: nan | val_loss: nan | Time: 1558.13 ms
[2022-03-31 16:05:47 main:574] : INFO : Epoch 356 | loss: nan | val_loss: nan | Time: 1578.14 ms
[2022-03-31 16:05:48 main:574] : INFO : Epoch 357 | loss: nan | val_loss: nan | Time: 1554.28 ms
[2022-03-31 16:05:50 main:574] : INFO : Epoch 358 | loss: nan | val_loss: nan | Time: 1561.08 ms
[2022-03-31 16:05:51 main:574] : INFO : Epoch 359 | loss: nan | val_loss: nan | Time: 1574.25 ms
[2022-03-31 16:05:53 main:574] : INFO : Epoch 360 | loss: nan | val_loss: nan | Time: 1258.1 ms
[2022-03-31 16:05:54 main:574] : INFO : Epoch 361 | loss: nan | val_loss: nan | Time: 1396.51 ms
[2022-03-31 16:05:56 main:574] : INFO : Epoch 362 | loss: nan | val_loss: nan | Time: 1604.21 ms
[2022-03-31 16:05:57 main:574] : INFO : Epoch 363 | loss: nan | val_loss: nan | Time: 1561.92 ms
[2022-03-31 16:05:59 main:574] : INFO : Epoch 364 | loss: nan | val_loss: nan | Time: 1570.4 ms
[2022-03-31 16:06:00 main:574] : INFO : Epoch 365 | loss: nan | val_loss: nan | Time: 1577.32 ms
[2022-03-31 16:06:02 main:574] : INFO : Epoch 366 | loss: nan | val_loss: nan | Time: 1537.32 ms
[2022-03-31 16:06:03 main:574] : INFO : Epoch 367 | loss: nan | val_loss: nan | Time: 1186.89 ms
[2022-03-31 16:06:05 main:574] : INFO : Epoch 368 | loss: nan | val_loss: nan | Time: 1581.59 ms
[2022-03-31 16:06:06 main:574] : INFO : Epoch 369 | loss: nan | val_loss: nan | Time: 1544.14 ms
[2022-03-31 16:06:08 main:574] : INFO : Epoch 370 | loss: nan | val_loss: nan | Time: 1552.17 ms
[2022-03-31 16:06:09 main:574] : INFO : Epoch 371 | loss: nan | val_loss: nan | Time: 1578.47 ms
[2022-03-31 16:06:11 main:574] : INFO : Epoch 372 | loss: nan | val_loss: nan | Time: 1537.59 ms
[2022-03-31 16:06:12 main:574] : INFO : Epoch 373 | loss: nan | val_loss: nan | Time: 1484.36 ms
[2022-03-31 16:06:14 main:574] : INFO : Epoch 374 | loss: nan | val_loss: nan | Time: 1262.3 ms
[2022-03-31 16:06:15 main:574] : INFO : Epoch 375 | loss: nan | val_loss: nan | Time: 1563.96 ms
[2022-03-31 16:06:17 main:574] : INFO : Epoch 376 | loss: nan | val_loss: nan | Time: 1556.9 ms
[2022-03-31 16:06:18 main:574] : INFO : Epoch 377 | loss: nan | val_loss: nan | Time: 1543.24 ms
[2022-03-31 16:06:20 main:574] : INFO : Epoch 378 | loss: nan | val_loss: nan | Time: 1568.79 ms
[2022-03-31 16:06:21 main:574] : INFO : Epoch 379 | loss: nan | val_loss: nan | Time: 1604.58 ms
[2022-03-31 16:06:23 main:574] : INFO : Epoch 380 | loss: nan | val_loss: nan | Time: 1274.83 ms
[2022-03-31 16:06:24 main:574] : INFO : Epoch 381 | loss: nan | val_loss: nan | Time: 1387.95 ms
[2022-03-31 16:06:26 main:574] : INFO : Epoch 382 | loss: nan | val_loss: nan | Time: 1607.7 ms
[2022-03-31 16:06:27 main:574] : INFO : Epoch 383 | loss: nan | val_loss: nan | Time: 1549.45 ms
[2022-03-31 16:06:29 main:574] : INFO : Epoch 384 | loss: nan | val_loss: nan | Time: 1537.78 ms
[2022-03-31 16:06:30 main:574] : INFO : Epoch 385 | loss: nan | val_loss: nan | Time: 1559.72 ms
[2022-03-31 16:06:32 main:574] : INFO : Epoch 386 | loss: nan | val_loss: nan | Time: 1549.4 ms
[2022-03-31 16:06:33 main:574] : INFO : Epoch 387 | loss: nan | val_loss: nan | Time: 1157.85 ms
[2022-03-31 16:06:35 main:574] : INFO : Epoch 388 | loss: nan | val_loss: nan | Time: 1678.16 ms
[2022-03-31 16:06:37 main:574] : INFO : Epoch 389 | loss: nan | val_loss: nan | Time: 1708.21 ms
[2022-03-31 16:06:38 main:574] : INFO : Epoch 390 | loss: nan | val_loss: nan | Time: 1597.8 ms
[2022-03-31 16:06:40 main:574] : INFO : Epoch 391 | loss: nan | val_loss: nan | Time: 1550.47 ms
[2022-03-31 16:06:41 main:574] : INFO : Epoch 392 | loss: nan | val_loss: nan | Time: 1568.08 ms
[2022-03-31 16:06:43 main:574] : INFO : Epoch 393 | loss: nan | val_loss: nan | Time: 1501.93 ms
[2022-03-31 16:06:44 main:574] : INFO : Epoch 394 | loss: nan | val_loss: nan | Time: 1376.4 ms
[2022-03-31 16:06:46 main:574] : INFO : Epoch 395 | loss: nan | val_loss: nan | Time: 1559.62 ms
[2022-03-31 16:06:47 main:574] : INFO : Epoch 396 | loss: nan | val_loss: nan | Time: 1524.38 ms
[2022-03-31 16:06:49 main:574] : INFO : Epoch 397 | loss: nan | val_loss: nan | Time: 1560.1 ms
[2022-03-31 16:06:50 main:574] : INFO : Epoch 398 | loss: nan | val_loss: nan | Time: 1546.16 ms
[2022-03-31 16:06:52 main:574] : INFO : Epoch 399 | loss: nan | val_loss: nan | Time: 1562.13 ms
[2022-03-31 16:06:53 main:574] : INFO : Epoch 400 | loss: nan | val_loss: nan | Time: 1218.71 ms
[2022-03-31 16:06:55 main:574] : INFO : Epoch 401 | loss: nan | val_loss: nan | Time: 1550.87 ms
[2022-03-31 16:06:56 main:574] : INFO : Epoch 402 | loss: nan | val_loss: nan | Time: 1560.84 ms
[2022-03-31 16:06:58 main:574] : INFO : Epoch 403 | loss: nan | val_loss: nan | Time: 1573.13 ms
[2022-03-31 16:06:59 main:574] : INFO : Epoch 404 | loss: nan | val_loss: nan | Time: 1539.61 ms
[2022-03-31 16:07:01 main:574] : INFO : Epoch 405 | loss: nan | val_loss: nan | Time: 1579.62 ms
[2022-03-31 16:07:02 main:574] : INFO : Epoch 406 | loss: nan | val_loss: nan | Time: 1502.98 ms
[2022-03-31 16:07:04 main:574] : INFO : Epoch 407 | loss: nan | val_loss: nan | Time: 1213.22 ms
[2022-03-31 16:07:05 main:574] : INFO : Epoch 408 | loss: nan | val_loss: nan | Time: 1592.19 ms
[2022-03-31 16:07:07 main:574] : INFO : Epoch 409 | loss: nan | val_loss: nan | Time: 1562.87 ms
[2022-03-31 16:07:08 main:574] : INFO : Epoch 410 | loss: nan | val_loss: nan | Time: 1541.86 ms
[2022-03-31 16:07:10 main:574] : INFO : Epoch 411 | loss: nan | val_loss: nan | Time: 1542.13 ms
[2022-03-31 16:07:11 main:574] : INFO : Epoch 412 | loss: nan | val_loss: nan | Time: 1553.74 ms
[2022-03-31 16:07:13 main:574] : INFO : Epoch 413 | loss: nan | val_loss: nan | Time: 1370.47 ms
[2022-03-31 16:07:14 main:574] : INFO : Epoch 414 | loss: nan | val_loss: nan | Time: 1410.92 ms
[2022-03-31 16:07:16 main:574] : INFO : Epoch 415 | loss: nan | val_loss: nan | Time: 1548.3 ms
[2022-03-31 16:07:17 main:574] : INFO : Epoch 416 | loss: nan | val_loss: nan | Time: 1551.18 ms
[2022-03-31 16:07:19 main:574] : INFO : Epoch 417 | loss: nan | val_loss: nan | Time: 1571.66 ms
[2022-03-31 16:07:20 main:574] : INFO : Epoch 418 | loss: nan | val_loss: nan | Time: 1535.86 ms
[2022-03-31 16:07:22 main:574] : INFO : Epoch 419 | loss: nan | val_loss: nan | Time: 1547.75 ms
[2022-03-31 16:07:24 main:574] : INFO : Epoch 420 | loss: nan | val_loss: nan | Time: 1527.55 ms
[2022-03-31 16:07:25 main:574] : INFO : Epoch 421 | loss: nan | val_loss: nan | Time: 1548.63 ms
[2022-03-31 16:07:27 main:574] : INFO : Epoch 422 | loss: nan | val_loss: nan | Time: 1543.98 ms
[2022-03-31 16:07:28 main:574] : INFO : Epoch 423 | loss: nan | val_loss: nan | Time: 1559.18 ms
[2022-03-31 16:07:30 main:574] : INFO : Epoch 424 | loss: nan | val_loss: nan | Time: 1563.38 ms
[2022-03-31 16:07:31 main:574] : INFO : Epoch 425 | loss: nan | val_loss: nan | Time: 1527.09 ms
[2022-03-31 16:07:33 main:574] : INFO : Epoch 426 | loss: nan | val_loss: nan | Time: 1393.29 ms
[2022-03-31 16:07:34 main:574] : INFO : Epoch 427 | loss: nan | val_loss: nan | Time: 1276.73 ms
[2022-03-31 16:07:35 main:574] : INFO : Epoch 428 | loss: nan | val_loss: nan | Time: 1558 ms
[2022-03-31 16:07:37 main:574] : INFO : Epoch 429 | loss: nan | val_loss: nan | Time: 1556.39 ms
[2022-03-31 16:07:39 main:574] : INFO : Epoch 430 | loss: nan | val_loss: nan | Time: 1559.21 ms
[2022-03-31 16:07:40 main:574] : INFO : Epoch 431 | loss: nan | val_loss: nan | Time: 1579.93 ms
[2022-03-31 16:07:42 main:574] : INFO : Epoch 432 | loss: nan | val_loss: nan | Time: 1547.46 ms
[2022-03-31 16:07:43 main:574] : INFO : Epoch 433 | loss: nan | val_loss: nan | Time: 1268.34 ms
[2022-03-31 16:07:44 main:574] : INFO : Epoch 434 | loss: nan | val_loss: nan | Time: 1473.79 ms
[2022-03-31 16:07:46 main:574] : INFO : Epoch 435 | loss: nan | val_loss: nan | Time: 1571.95 ms
[2022-03-31 16:07:48 main:574] : INFO : Epoch 436 | loss: nan | val_loss: nan | Time: 1584.01 ms
[2022-03-31 16:07:49 main:574] : INFO : Epoch 437 | loss: nan | val_loss: nan | Time: 1554.55 ms
[2022-03-31 16:07:51 main:574] : INFO : Epoch 438 | loss: nan | val_loss: nan | Time: 1545.48 ms
[2022-03-31 16:07:52 main:574] : INFO : Epoch 439 | loss: nan | val_loss: nan | Time: 1578.43 ms
[2022-03-31 16:07:53 main:574] : INFO : Epoch 440 | loss: nan | val_loss: nan | Time: 1119.48 ms
[2022-03-31 16:07:55 main:574] : INFO : Epoch 441 | loss: nan | val_loss: nan | Time: 1553.37 ms
[2022-03-31 16:07:57 main:574] : INFO : Epoch 442 | loss: nan | val_loss: nan | Time: 1532.73 ms
[2022-03-31 16:07:58 main:574] : INFO : Epoch 443 | loss: nan | val_loss: nan | Time: 1577.37 ms
[2022-03-31 16:08:00 main:574] : INFO : Epoch 444 | loss: nan | val_loss: nan | Time: 1603.64 ms
[2022-03-31 16:08:01 main:574] : INFO : Epoch 445 | loss: nan | val_loss: nan | Time: 1537.74 ms
[2022-03-31 16:08:03 main:574] : INFO : Epoch 446 | loss: nan | val_loss: nan | Time: 1379.74 ms
[2022-03-31 16:08:04 main:574] : INFO : Epoch 447 | loss: nan | val_loss: nan | Time: 1328.62 ms
[2022-03-31 16:08:06 main:574] : INFO : Epoch 448 | loss: nan | val_loss: nan | Time: 1543.77 ms
[2022-03-31 16:08:07 main:574] : INFO : Epoch 449 | loss: nan | val_loss: nan | Time: 1581.66 ms
[2022-03-31 16:08:09 main:574] : INFO : Epoch 450 | loss: nan | val_loss: nan | Time: 1579.74 ms
[2022-03-31 16:08:10 main:574] : INFO : Epoch 451 | loss: nan | val_loss: nan | Time: 1538.41 ms
[2022-03-31 16:08:12 main:574] : INFO : Epoch 452 | loss: nan | val_loss: nan | Time: 1535.64 ms
[2022-03-31 16:08:13 main:574] : INFO : Epoch 453 | loss: nan | val_loss: nan | Time: 1308.42 ms
[2022-03-31 16:08:15 main:574] : INFO : Epoch 454 | loss: nan | val_loss: nan | Time: 1425.99 ms
[2022-03-31 16:08:16 main:574] : INFO : Epoch 455 | loss: nan | val_loss: nan | Time: 1566.74 ms
[2022-03-31 16:08:18 main:574] : INFO : Epoch 456 | loss: nan | val_loss: nan | Time: 1577.15 ms
[2022-03-31 16:08:19 main:574] : INFO : Epoch 457 | loss: nan | val_loss: nan | Time: 1560.21 ms
[2022-03-31 16:08:21 main:574] : INFO : Epoch 458 | loss: nan | val_loss: nan | Time: 1515.72 ms
[2022-03-31 16:08:22 main:574] : INFO : Epoch 459 | loss: nan | val_loss: nan | Time: 1555.14 ms
[2022-03-31 16:08:24 main:574] : INFO : Epoch 460 | loss: nan | val_loss: nan | Time: 1359.88 ms
[2022-03-31 16:08:25 main:574] : INFO : Epoch 461 | loss: nan | val_loss: nan | Time: 1540.43 ms
[2022-03-31 16:08:27 main:574] : INFO : Epoch 462 | loss: nan | val_loss: nan | Time: 1570.91 ms
[2022-03-31 16:08:28 main:574] : INFO : Epoch 463 | loss: nan | val_loss: nan | Time: 1578.65 ms
[2022-03-31 16:08:30 main:574] : INFO : Epoch 464 | loss: nan | val_loss: nan | Time: 1532.83 ms
[2022-03-31 16:08:31 main:574] : INFO : Epoch 465 | loss: nan | val_loss: nan | Time: 1558.68 ms
[2022-03-31 16:08:33 main:574] : INFO : Epoch 466 | loss: nan | val_loss: nan | Time: 1367.92 ms
[2022-03-31 16:08:34 main:574] : INFO : Epoch 467 | loss: nan | val_loss: nan | Time: 1328.59 ms
[2022-03-31 16:08:36 main:574] : INFO : Epoch 468 | loss: nan | val_loss: nan | Time: 1555.17 ms
[2022-03-31 16:08:37 main:574] : INFO : Epoch 469 | loss: nan | val_loss: nan | Time: 1556.88 ms
[2022-03-31 16:08:39 main:574] : INFO : Epoch 470 | loss: nan | val_loss: nan | Time: 1570.78 ms
[2022-03-31 16:08:40 main:574] : INFO : Epoch 471 | loss: nan | val_loss: nan | Time: 1518.86 ms
[2022-03-31 16:08:42 main:574] : INFO : Epoch 472 | loss: nan | val_loss: nan | Time: 1555.24 ms
[2022-03-31 16:08:43 main:574] : INFO : Epoch 473 | loss: nan | val_loss: nan | Time: 1237.05 ms
[2022-03-31 16:08:45 main:574] : INFO : Epoch 474 | loss: nan | val_loss: nan | Time: 1477.97 ms
[2022-03-31 16:08:46 main:574] : INFO : Epoch 475 | loss: nan | val_loss: nan | Time: 1583.84 ms
[2022-03-31 16:08:48 main:574] : INFO : Epoch 476 | loss: nan | val_loss: nan | Time: 1563.35 ms
[2022-03-31 16:08:49 main:574] : INFO : Epoch 477 | loss: nan | val_loss: nan | Time: 1582.33 ms
[2022-03-31 16:08:51 main:574] : INFO : Epoch 478 | loss: nan | val_loss: nan | Time: 1542.45 ms
[2022-03-31 16:08:52 main:574] : INFO : Epoch 479 | loss: nan | val_loss: nan | Time: 1531.53 ms
[2022-03-31 16:08:54 main:574] : INFO : Epoch 480 | loss: nan | val_loss: nan | Time: 1135.61 ms
[2022-03-31 16:08:55 main:574] : INFO : Epoch 481 | loss: nan | val_loss: nan | Time: 1572.19 ms
[2022-03-31 16:08:57 main:574] : INFO : Epoch 482 | loss: nan | val_loss: nan | Time: 1577.32 ms
[2022-03-31 16:08:58 main:574] : INFO : Epoch 483 | loss: nan | val_loss: nan | Time: 1572.39 ms
[2022-03-31 16:09:00 main:574] : INFO : Epoch 484 | loss: nan | val_loss: nan | Time: 1538.75 ms
[2022-03-31 16:09:01 main:574] : INFO : Epoch 485 | loss: nan | val_loss: nan | Time: 1500.5 ms
[2022-03-31 16:09:03 main:574] : INFO : Epoch 486 | loss: nan | val_loss: nan | Time: 1433.65 ms
[2022-03-31 16:09:04 main:574] : INFO : Epoch 487 | loss: nan | val_loss: nan | Time: 1282.88 ms
[2022-03-31 16:09:06 main:574] : INFO : Epoch 488 | loss: nan | val_loss: nan | Time: 1549.53 ms
[2022-03-31 16:09:07 main:574] : INFO : Epoch 489 | loss: nan | val_loss: nan | Time: 1553.63 ms
[2022-03-31 16:09:09 main:574] : INFO : Epoch 490 | loss: nan | val_loss: nan | Time: 1545.37 ms
[2022-03-31 16:09:10 main:574] : INFO : Epoch 491 | loss: nan | val_loss: nan | Time: 1564.88 ms
[2022-03-31 16:09:12 main:574] : INFO : Epoch 492 | loss: nan | val_loss: nan | Time: 1586.15 ms
[2022-03-31 16:09:13 main:574] : INFO : Epoch 493 | loss: nan | val_loss: nan | Time: 1224.55 ms
[2022-03-31 16:09:15 main:574] : INFO : Epoch 494 | loss: nan | val_loss: nan | Time: 1403.54 ms
[2022-03-31 16:09:16 main:574] : INFO : Epoch 495 | loss: nan | val_loss: nan | Time: 1578.47 ms
[2022-03-31 16:09:18 main:574] : INFO : Epoch 496 | loss: nan | val_loss: nan | Time: 1553.13 ms
[2022-03-31 16:09:19 main:574] : INFO : Epoch 497 | loss: nan | val_loss: nan | Time: 1566.34 ms
[2022-03-31 16:09:21 main:574] : INFO : Epoch 498 | loss: nan | val_loss: nan | Time: 1544.35 ms
[2022-03-31 16:09:22 main:574] : INFO : Epoch 499 | loss: nan | val_loss: nan | Time: 1513.38 ms
[2022-03-31 16:09:24 main:574] : INFO : Epoch 500 | loss: nan | val_loss: nan | Time: 1223.46 ms
[2022-03-31 16:09:25 main:574] : INFO : Epoch 501 | loss: nan | val_loss: nan | Time: 1549.42 ms
[2022-03-31 16:09:27 main:574] : INFO : Epoch 502 | loss: nan | val_loss: nan | Time: 1574.42 ms
[2022-03-31 16:09:28 main:574] : INFO : Epoch 503 | loss: nan | val_loss: nan | Time: 1561.94 ms
[2022-03-31 16:09:30 main:574] : INFO : Epoch 504 | loss: nan | val_loss: nan | Time: 1564.17 ms
[2022-03-31 16:09:31 main:574] : INFO : Epoch 505 | loss: nan | val_loss: nan | Time: 1553.8 ms
[2022-03-31 16:09:33 main:574] : INFO : Epoch 506 | loss: nan | val_loss: nan | Time: 1494.08 ms
[2022-03-31 16:09:34 main:574] : INFO : Epoch 507 | loss: nan | val_loss: nan | Time: 1269.37 ms
[2022-03-31 16:09:36 main:574] : INFO : Epoch 508 | loss: nan | val_loss: nan | Time: 1598.52 ms
[2022-03-31 16:09:37 main:574] : INFO : Epoch 509 | loss: nan | val_loss: nan | Time: 1547.46 ms
[2022-03-31 16:09:39 main:574] : INFO : Epoch 510 | loss: nan | val_loss: nan | Time: 1558.75 ms
[2022-03-31 16:09:40 main:574] : INFO : Epoch 511 | loss: nan | val_loss: nan | Time: 1579.53 ms
[2022-03-31 16:09:42 main:574] : INFO : Epoch 512 | loss: nan | val_loss: nan | Time: 1577.33 ms
[2022-03-31 16:09:43 main:574] : INFO : Epoch 513 | loss: nan | val_loss: nan | Time: 1234.37 ms
[2022-03-31 16:09:45 main:574] : INFO : Epoch 514 | loss: nan | val_loss: nan | Time: 1486.42 ms
[2022-03-31 16:09:46 main:574] : INFO : Epoch 515 | loss: nan | val_loss: nan | Time: 1556.45 ms
[2022-03-31 16:09:48 main:574] : INFO : Epoch 516 | loss: nan | val_loss: nan | Time: 1562.48 ms
[2022-03-31 16:09:49 main:574] : INFO : Epoch 517 | loss: nan | val_loss: nan | Time: 1547.84 ms
[2022-03-31 16:09:51 main:574] : INFO : Epoch 518 | loss: nan | val_loss: nan | Time: 1574.83 ms
[2022-03-31 16:09:52 main:574] : INFO : Epoch 519 | loss: nan | val_loss: nan | Time: 1553.04 ms
[2022-03-31 16:09:54 main:574] : INFO : Epoch 520 | loss: nan | val_loss: nan | Time: 1161.06 ms
[2022-03-31 16:09:55 main:574] : INFO : Epoch 521 | loss: nan | val_loss: nan | Time: 1553.66 ms
[2022-03-31 16:09:57 main:574] : INFO : Epoch 522 | loss: nan | val_loss: nan | Time: 1567.45 ms
[2022-03-31 16:09:58 main:574] : INFO : Epoch 523 | loss: nan | val_loss: nan | Time: 1582.27 ms
[2022-03-31 16:10:00 main:574] : INFO : Epoch 524 | loss: nan | val_loss: nan | Time: 1586.69 ms
[2022-03-31 16:10:02 main:574] : INFO : Epoch 525 | loss: nan | val_loss: nan | Time: 1598.51 ms
[2022-03-31 16:10:03 main:574] : INFO : Epoch 526 | loss: nan | val_loss: nan | Time: 1433.96 ms
[2022-03-31 16:10:04 main:574] : INFO : Epoch 527 | loss: nan | val_loss: nan | Time: 1289.79 ms
[2022-03-31 16:10:06 main:574] : INFO : Epoch 528 | loss: nan | val_loss: nan | Time: 1586.08 ms
[2022-03-31 16:10:07 main:574] : INFO : Epoch 529 | loss: nan | val_loss: nan | Time: 1557.03 ms
[2022-03-31 16:10:09 main:574] : INFO : Epoch 530 | loss: nan | val_loss: nan | Time: 1530.93 ms
[2022-03-31 16:10:11 main:574] : INFO : Epoch 531 | loss: nan | val_loss: nan | Time: 1553.17 ms
[2022-03-31 16:10:12 main:574] : INFO : Epoch 532 | loss: nan | val_loss: nan | Time: 1608.8 ms
[2022-03-31 16:10:13 main:574] : INFO : Epoch 533 | loss: nan | val_loss: nan | Time: 1247.92 ms
[2022-03-31 16:10:15 main:574] : INFO : Epoch 534 | loss: nan | val_loss: nan | Time: 1540.64 ms
[2022-03-31 16:10:17 main:574] : INFO : Epoch 535 | loss: nan | val_loss: nan | Time: 1581.1 ms
[2022-03-31 16:10:18 main:574] : INFO : Epoch 536 | loss: nan | val_loss: nan | Time: 1560.45 ms
[2022-03-31 16:10:20 main:574] : INFO : Epoch 537 | loss: nan | val_loss: nan | Time: 1560.28 ms
[2022-03-31 16:10:21 main:574] : INFO : Epoch 538 | loss: nan | val_loss: nan | Time: 1506.75 ms
[2022-03-31 16:10:23 main:574] : INFO : Epoch 539 | loss: nan | val_loss: nan | Time: 1526.14 ms
[2022-03-31 16:10:24 main:574] : INFO : Epoch 540 | loss: nan | val_loss: nan | Time: 1217.77 ms
[2022-03-31 16:10:25 main:574] : INFO : Epoch 541 | loss: nan | val_loss: nan | Time: 1562.1 ms
[2022-03-31 16:10:27 main:574] : INFO : Epoch 542 | loss: nan | val_loss: nan | Time: 1540.55 ms
[2022-03-31 16:10:29 main:574] : INFO : Epoch 543 | loss: nan | val_loss: nan | Time: 1563.37 ms
[2022-03-31 16:10:30 main:574] : INFO : Epoch 544 | loss: nan | val_loss: nan | Time: 1564.33 ms
[2022-03-31 16:10:32 main:574] : INFO : Epoch 545 | loss: nan | val_loss: nan | Time: 1555.16 ms
[2022-03-31 16:10:33 main:574] : INFO : Epoch 546 | loss: nan | val_loss: nan | Time: 1351.04 ms
[2022-03-31 16:10:34 main:574] : INFO : Epoch 547 | loss: nan | val_loss: nan | Time: 1335.35 ms
[2022-03-31 16:10:36 main:574] : INFO : Epoch 548 | loss: nan | val_loss: nan | Time: 1554.37 ms
[2022-03-31 16:10:37 main:574] : INFO : Epoch 549 | loss: nan | val_loss: nan | Time: 1551.25 ms
[2022-03-31 16:10:39 main:574] : INFO : Epoch 550 | loss: nan | val_loss: nan | Time: 1558.66 ms
[2022-03-31 16:10:41 main:574] : INFO : Epoch 551 | loss: nan | val_loss: nan | Time: 1518.07 ms
[2022-03-31 16:10:42 main:574] : INFO : Epoch 552 | loss: nan | val_loss: nan | Time: 1551.84 ms
[2022-03-31 16:10:43 main:574] : INFO : Epoch 553 | loss: nan | val_loss: nan | Time: 1202.47 ms
[2022-03-31 16:10:45 main:574] : INFO : Epoch 554 | loss: nan | val_loss: nan | Time: 1475.66 ms
[2022-03-31 16:10:46 main:574] : INFO : Epoch 555 | loss: nan | val_loss: nan | Time: 1535.66 ms
[2022-03-31 16:10:48 main:574] : INFO : Epoch 556 | loss: nan | val_loss: nan | Time: 1559.79 ms
[2022-03-31 16:10:49 main:574] : INFO : Epoch 557 | loss: nan | val_loss: nan | Time: 1557.64 ms
[2022-03-31 16:10:51 main:574] : INFO : Epoch 558 | loss: nan | val_loss: nan | Time: 1555.82 ms
[2022-03-31 16:10:53 main:574] : INFO : Epoch 559 | loss: nan | val_loss: nan | Time: 1570.95 ms
[2022-03-31 16:10:54 main:574] : INFO : Epoch 560 | loss: nan | val_loss: nan | Time: 1136.38 ms
[2022-03-31 16:10:55 main:574] : INFO : Epoch 561 | loss: nan | val_loss: nan | Time: 1580.75 ms
[2022-03-31 16:10:57 main:574] : INFO : Epoch 562 | loss: nan | val_loss: nan | Time: 1552.88 ms
[2022-03-31 16:10:58 main:574] : INFO : Epoch 563 | loss: nan | val_loss: nan | Time: 1538.15 ms
[2022-03-31 16:11:00 main:574] : INFO : Epoch 564 | loss: nan | val_loss: nan | Time: 1564.84 ms
[2022-03-31 16:11:02 main:574] : INFO : Epoch 565 | loss: nan | val_loss: nan | Time: 1548.38 ms
[2022-03-31 16:11:03 main:574] : INFO : Epoch 566 | loss: nan | val_loss: nan | Time: 1450.96 ms
[2022-03-31 16:11:04 main:574] : INFO : Epoch 567 | loss: nan | val_loss: nan | Time: 1294.2 ms
[2022-03-31 16:11:06 main:574] : INFO : Epoch 568 | loss: nan | val_loss: nan | Time: 1574.93 ms
[2022-03-31 16:11:07 main:574] : INFO : Epoch 569 | loss: nan | val_loss: nan | Time: 1562.52 ms
[2022-03-31 16:11:09 main:574] : INFO : Epoch 570 | loss: nan | val_loss: nan | Time: 1600.62 ms
[2022-03-31 16:11:11 main:574] : INFO : Epoch 571 | loss: nan | val_loss: nan | Time: 1582.23 ms
[2022-03-31 16:11:12 main:574] : INFO : Epoch 572 | loss: nan | val_loss: nan | Time: 1536.09 ms
[2022-03-31 16:11:13 main:574] : INFO : Epoch 573 | loss: nan | val_loss: nan | Time: 1253.04 ms
[2022-03-31 16:11:15 main:574] : INFO : Epoch 574 | loss: nan | val_loss: nan | Time: 1472.79 ms
[2022-03-31 16:11:16 main:574] : INFO : Epoch 575 | loss: nan | val_loss: nan | Time: 1575.49 ms
[2022-03-31 16:11:18 main:574] : INFO : Epoch 576 | loss: nan | val_loss: nan | Time: 1594.18 ms
[2022-03-31 16:11:20 main:574] : INFO : Epoch 577 | loss: nan | val_loss: nan | Time: 1553.16 ms
[2022-03-31 16:11:21 main:574] : INFO : Epoch 578 | loss: nan | val_loss: nan | Time: 1564.79 ms
[2022-03-31 16:11:23 main:574] : INFO : Epoch 579 | loss: nan | val_loss: nan | Time: 1553.76 ms
[2022-03-31 16:11:24 main:574] : INFO : Epoch 580 | loss: nan | val_loss: nan | Time: 1191.16 ms
[2022-03-31 16:11:25 main:574] : INFO : Epoch 581 | loss: nan | val_loss: nan | Time: 1540.93 ms
[2022-03-31 16:11:27 main:574] : INFO : Epoch 582 | loss: nan | val_loss: nan | Time: 1571.36 ms
[2022-03-31 16:11:29 main:574] : INFO : Epoch 583 | loss: nan | val_loss: nan | Time: 1565.41 ms
[2022-03-31 16:11:30 main:574] : INFO : Epoch 584 | loss: nan | val_loss: nan | Time: 1572.49 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFF32944F69
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 03/31/22 16:11:32
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\11;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 000000006e2e0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000034eb0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000332d0000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000032910000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000032fa0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000032ee0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1586 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1586
ModLoad: 0000000032f70000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000032760000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000032870000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000032d20000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000034c10000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000034d20000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000034dc0000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000034600000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026770000 0000000000058000 C:\ProgramData\BOINC\slots\11\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000012200000 000000000008d000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.29.30135.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\4\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.29.30135.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30135.0
ModLoad: 0000000029f50000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 00000000122c0000 000000000001b000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.29.30135.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\4\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.29.30135.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30135.0
ModLoad: 00000000121f0000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.29.30135.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\4\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.29.30135.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30135.0
ModLoad: 000000008ac90000 0000000024b49000 C:\ProgramData\BOINC\slots\11\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000026390000 0000000000045000 C:\ProgramData\BOINC\slots\11\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000de730000 00000000034b2000 C:\ProgramData\BOINC\slots\11\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000800a0000 000000000abe7000 C:\ProgramData\BOINC\slots\11\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000002d4d0000 0000000000010000 C:\ProgramData\BOINC\slots\11\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000c0bf0000 00000000030a5000 C:\ProgramData\BOINC\slots\11\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000064cf0000 000000001b3a4000 C:\ProgramData\BOINC\slots\11\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000068e0000 000000000020d000 C:\ProgramData\BOINC\slots\11\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000041b0000 00000000002db000 C:\ProgramData\BOINC\slots\11\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000002afa0000 0000000000040000 C:\ProgramData\BOINC\slots\11\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bce90000 0000000003d5b000 C:\ProgramData\BOINC\slots\11\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000b2550000 000000000919f000 C:\ProgramData\BOINC\slots\11\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 000000005ab80000 0000000001eca000 C:\ProgramData\BOINC\slots\11\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000033a10000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000304c0000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000335a0000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000032e90000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000032730000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000032370000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000032e20000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000325d0000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000321b0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000033c40000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 00000000306c0000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000034730000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000031f50000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000033140000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000033540000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000ea6f0000 00000000002cd000 C:\WINDOWS\system32\nvcuda.dll (30.0.15.1215) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r512_08\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1215
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.127 driver
Product Version : 30.0.15.1215
ModLoad: 00000000594e0000 000000000169f000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_c0e159863e7afdde\nvcuda64.dll (30.0.15.1215) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r512_08\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1215
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.127 driver
Product Version : 30.0.15.1215
ModLoad: 000000002bb80000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000002bb40000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 000000002b5c0000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000031ec0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000026fb0000 0000000000768000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.15.1215) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r512_08\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1215
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1215
ModLoad: 00000000e9980000 00000000008b3000 C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_c0e159863e7afdde\nvptxJitCompiler64.dll (30.0.15.1215) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 30.0.15.1215
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows PTX JIT Compiler
Product Version : 30.0.15.1215
ModLoad: 000000002d4c0000 0000000000009000 C:\ProgramData\BOINC\slots\11\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d9840000 0000000000fca000 C:\ProgramData\BOINC\slots\11\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000032c90000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 5695, Write: 5975, Other 86223
- I/O Transfers Counters -
Read: 19364114, Write: -1927201613, Other 278528
- Paged Pool Usage -
QuotaPagedPoolUsage: 3536416, QuotaPeakPagedPoolUsage: 3552800
QuotaNonPagedPoolUsage: 58064, QuotaPeakNonPagedPoolUsage: 78056
- Virtual Memory Usage -
VirtualSize: 340701184, PeakVirtualSize: 1393270784
- Pagefile Usage -
PagefileUsage: 340701184, PeakPagefileUsage: 420134912
- Working Set Size -
WorkingSetSize: -1905799168, PeakWorkingSetSize: -1488834560, PageFaultCount: 41940174
*** Dump of thread ID 11968 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 447968736.000000, User Time: 28858437632.000000, Wait Time: 17659760.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFF32944F69
- Registers -
rax=0000000000000020 rbx=000000008e70b320 rcx=000000007a2c3f50 rdx=000000008da60000 rsi=000000001e6e4bc0 rdi=0000000019930520
r8=0000000024d40000 r9=00000000c36826c0 r10=000000002678f630 r11=00000000c36826c0 r12=000000001521e200 r13=000000006b1ac3b0
r14=0000000015214800 r15=0000000000633450 rip=0000000032944f69 rsp=000000001e6e46e0 rbp=000000001e6e4950
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
1e6e47b0 122c6480 8e4e1fc8 8e4e17f0 00000321 00000000 KERNELBASE!RaiseException+0x0
1e6e4810 8da08f53 8e4ddbb8 39a00000 1e6e49b8 39300000 VCRUNTIME140!_CxxThrowException+0x0
1e6e4cf0 8da625ea 1e6e4db8 00000001 1e6e4db8 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
1e6e4dd0 8da70cbf 1e6e54c0 1e6e6cb8 1e6e5e10 00000001 torch_cuda!at::native::set_storage_cuda_+0x0
1e6e4eb0 85aeb9fc 1e6e54c0 1e6e5ba0 00000000 1e6e4fc8 torch_cuda!at::native::set_storage_cuda_+0x0
1e6e5030 85b231a2 1e6e53c0 00000001 00000000 00000000 torch_cpu!at::native::mkldnn_sigmoid_+0x0
1e6e5310 86dd2613 00000000 1e6e5e20 1e6e5e20 1e6e5420 torch_cpu!at::_cudnn_rnn+0x0
1e6e5810 86de080b 4bdc9db7 98e1e450 1e6e5940 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
1e6e58f0 85aeb9fc 1e6e5e70 86f05878 1e6e5a50 59507a77 torch_cpu!torch::autograd::GraphRoot::apply+0x0
1e6e5a70 85b231a2 69fbedc0 00000000 4bdc9227 8e92a3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
1e6e5d50 8da069a1 1e6e0001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
1e6e5f20 8da10117 00000000 1e6e6cb8 26392e0d 1e6e6040 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
1e6e5fb0 8587107b 8a9c1ac0 1e6e62d0 e96228a9 596d7783 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
1e6e6220 858a3b16 00000001 1e6e6cb8 00000080 6b1ac4a8 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
1e6e63b0 85c1ae89 1e6e6428 00000001 1e6e6428 00000001 torch_cpu!at::native::gru+0x0
1e6e6440 86f09fa3 1e6e6540 00000001 1e6e66c0 1e6e6cb8 torch_cpu!at::zeros_out+0x0
1e6e64d0 85c0dd35 267c1ea8 262cef50 00000000 267c1ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
1e6e6550 85ae88ec 8a9c75c8 87d85420 9d5402e0 1e6e6770 torch_cpu!at::zeros_out+0x0
1e6e6650 85b5c1bb 0000000c 00000080 00000004 98e11000 torch_cpu!at::native::mkldnn_sigmoid_+0x0
1e6e66d0 87647294 1e6e67f0 87d85420 1e6e67e0 1e6e6988 torch_cpu!at::gru+0x0
1e6e6900 876462ab 1e6e6c98 85a7030e 1e6e6c98 1e6e6cb8 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
1e6e69f0 6e2f2621 00000002 11ff0480 00000000 267c1ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
1e6e6bf0 6e2f7e98 1e6e6c48 1e6e0000 11ff0601 11ff0480 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
1e6e7090 6e2fb964 1e6e7190 1e6e7190 00000000 75c51b50 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::unsafeRemoveSlot+0x0
1e6ffe30 6e53c6f8 26775710 00000000 00000000 263bd950 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
1e6ffe70 332e7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
1e6ffea0 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1e6fff20 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 14660 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 17659760.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=000000001ebff800 rsi=0000000000000000 rdi=0000000000000064
r8=000000001ebff7d8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000034f4d3f4 rsp=000000001ebff7d8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
1ebff7d0 3295962e 1ebff898 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
1ebff870 6e31a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
1ebff8a0 332e7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
1ebff8d0 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1ebff950 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 22484 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 17659760.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=000000001ecffa00 rsi=0000000000000000 rdi=000000000000000d
r8=0000000000000001 r9=000000002692a900 r10=0000000000000000 r11=000000001ecff970 r12=0000000000000064 r13=000000001ecffa00
r14=000000001ecff700 r15=0000000000000000 rip=0000000034f4d8c4 rsp=000000001ecff6a8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
1ecff6a0 3295cb20 00000000 1e432000 3501a4d0 1e432000 ntdll!ZwWaitForMultipleObjects+0x0
1ecff990 3295ca1e 24b846c0 004e0049 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
1ecff9d0 596bd554 2692dbc0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
1ecffc10 597658cd 00000000 2692a9e0 00000000 00000000 nvcuda64!+0x0
1ecffcc0 596bcf23 773ccf84 76def868 26f01e40 00000000 nvcuda64!cuProfilerStop+0x0
1ecffcf0 599f41e4 26f01e40 00000000 00000000 00000000 nvcuda64!+0x0
1ecffd20 332e7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
1ecffd50 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1ecffdd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 1508 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 17659756.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002b8 rdx=0000000000000000 rsi=000000001edff698 rdi=00000000000002b8
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=000000002690abe0 r13=0000000000000000
r14=00000000000002b8 r15=0000000000000001 rip=0000000034f4cdf4 rsp=000000001edff668 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
1edff660 32931a5e 0000001b 32fbd489 1edff708 00000000 ntdll!ZwWaitForSingleObject+0x0
1edff700 598e596e 26162840 00000000 00000000 000002b8 KERNELBASE!WaitForSingleObjectEx+0x0
1edff940 596bcf23 00000000 26f01540 594e0000 2697e7a0 nvcuda64!cuProfilerStop+0x0
1edff970 599f41e4 26f01540 00000000 00000000 00000000 nvcuda64!+0x0
1edff9a0 332e7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
1edff9d0 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1edffa50 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 26456 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 135781248.000000, User Time: 3673281280.000000, Wait Time: 17659738.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=000000005b67eb08 rdx=0000000000000000 rsi=0000000000000000 rdi=000000005b67eb08
r8=00000000868d2dd8 r9=0000000060439bb8 r10=00000000594e0000 r11=0000000000000003 r12=0000000000000000 r13=0000000000000000
r14=000000001e7ff9e0 r15=000000005b67eab8 rip=0000000034f50764 rsp=000000001e7ff9b8 rbp=000000001e7ffa10
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
1e7ff9b0 34f14021 1e7ffcb0 00000000 1e7ffab0 00000000 ntdll!NtWaitForAlertByThreadId+0x0
1e7ffa30 3297cfa9 5b67eb00 5b67eaf8 5b67eab0 872805b4 ntdll!RtlSleepConditionVariableSRW+0x0
1e7ffa70 12212959 00000000 b914b600 00000000 00000000 KERNELBASE!SleepConditionVariableSRW+0x0
1e7ffaa0 12212bea fa9000d8 00000000 1e7ffcb0 5b67eaf8 MSVCP140!std::_Winerror_message+0x0
1e7ffad0 8728064b 5b67eab0 1e7ffbd0 5b67eab0 b914b5e0 MSVCP140!_Cnd_wait+0x0
1e7ffb10 872825b1 be815390 ffffffff b914b5e0 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
1e7ffe00 87282471 8a7a6d10 1e7ffe40 8a7a6d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
1e7ffe50 87279744 00080001 d27e4340 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
1e7ffe80 32d41bb2 d27e4340 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
1e7ffeb0 332e7034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
1e7ffee0 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1e7fff60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 23896 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 17659748.000000
- Registers -
rax=00000000000001d3 rbx=000000003455ead0 rcx=0000000000000070 rdx=000000003455e750 rsi=0000000024b73550 rdi=000000003455ead0
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000034f507c4 rsp=000000001e8ff818 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
1e8ff810 34f02dc7 00000000 00000000 24b73550 00000000 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
1e8ffb10 332e7034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
1e8ffb40 34f02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1e8ffbc0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)