Task 7360294
| Name | ParityModified-1631726120-10892-0_0 |
| Workunit | 5036531 |
| Created | 15 Sep 2021, 17:15:22 UTC |
| Sent | 15 Sep 2021, 18:39:39 UTC |
| Report deadline | 23 Sep 2021, 18:39:39 UTC |
| Received | 22 Sep 2021, 20:15:27 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 6549 |
| Run time | 59 min 40 sec |
| CPU time | 47 min 22 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 1,132.07 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.70 GB |
| Peak swap size | 3.58 GB |
| Peak disk usage | 1.53 GB |
Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -529697949 (0xe06d7363)</message>
<stderr_txt>
Time: 6286.54 ms
[2021-09-22 12:46:19 main:574] : INFO : Epoch 295 | loss: 0.0310814 | val_loss: 0.0312359 | Time: 6327.24 ms
[2021-09-22 12:46:25 main:574] : INFO : Epoch 296 | loss: 0.0310786 | val_loss: 0.0312283 | Time: 6314.18 ms
[2021-09-22 12:46:32 main:574] : INFO : Epoch 297 | loss: 0.0310774 | val_loss: 0.0312234 | Time: 6255.86 ms
[2021-09-22 12:46:38 main:574] : INFO : Epoch 298 | loss: 0.0310832 | val_loss: 0.031239 | Time: 6268.41 ms
[2021-09-22 12:46:44 main:574] : INFO : Epoch 299 | loss: 0.031085 | val_loss: 0.0312481 | Time: 6256.93 ms
[2021-09-22 12:46:51 main:574] : INFO : Epoch 300 | loss: 0.0310769 | val_loss: 0.0312408 | Time: 6423.54 ms
[2021-09-22 12:46:57 main:574] : INFO : Epoch 301 | loss: 0.0310736 | val_loss: 0.0312554 | Time: 6323.47 ms
[2021-09-22 12:47:04 main:574] : INFO : Epoch 302 | loss: 0.0310792 | val_loss: 0.031263 | Time: 6271.82 ms
[2021-09-22 12:47:10 main:574] : INFO : Epoch 303 | loss: 0.0310749 | val_loss: 0.0312448 | Time: 6343.76 ms
[2021-09-22 12:47:16 main:574] : INFO : Epoch 304 | loss: 0.0310758 | val_loss: 0.0312408 | Time: 6289.76 ms
[2021-09-22 12:47:23 main:574] : INFO : Epoch 305 | loss: 0.0310823 | val_loss: 0.0312412 | Time: 6273.75 ms
[2021-09-22 12:47:29 main:574] : INFO : Epoch 306 | loss: 0.0310762 | val_loss: 0.0312318 | Time: 6300.16 ms
[2021-09-22 12:47:36 main:574] : INFO : Epoch 307 | loss: 0.0310734 | val_loss: 0.0312245 | Time: 6392.74 ms
[2021-09-22 12:47:42 main:574] : INFO : Epoch 308 | loss: 0.0310773 | val_loss: 0.0312486 | Time: 6334.44 ms
[2021-09-22 12:47:48 main:574] : INFO : Epoch 309 | loss: 0.0310721 | val_loss: 0.0312402 | Time: 6280.82 ms
[2021-09-22 12:47:55 main:574] : INFO : Epoch 310 | loss: 0.0310729 | val_loss: 0.0312401 | Time: 6297.87 ms
[2021-09-22 12:48:01 main:574] : INFO : Epoch 311 | loss: 0.0310742 | val_loss: 0.0312543 | Time: 6393.76 ms
[2021-09-22 12:48:08 main:574] : INFO : Epoch 312 | loss: 0.0310737 | val_loss: 0.0312418 | Time: 6503.89 ms
[2021-09-22 12:48:14 main:574] : INFO : Epoch 313 | loss: 0.0310705 | val_loss: 0.0312461 | Time: 6282.91 ms
[2021-09-22 12:48:21 main:574] : INFO : Epoch 314 | loss: 0.0310754 | val_loss: 0.0312377 | Time: 6498.59 ms
[2021-09-22 12:48:27 main:574] : INFO : Epoch 315 | loss: 0.0310789 | val_loss: 0.0312404 | Time: 6301.37 ms
[2021-09-22 12:48:34 main:574] : INFO : Epoch 316 | loss: 0.0310741 | val_loss: 0.0312481 | Time: 6246.38 ms
[2021-09-22 12:48:41 main:574] : INFO : Epoch 317 | loss: 0.0310697 | val_loss: 0.0312482 | Time: 6275.98 ms
[2021-09-22 12:48:47 main:574] : INFO : Epoch 318 | loss: 0.0310761 | val_loss: 0.0312514 | Time: 6253.64 ms
[2021-09-22 12:48:53 main:574] : INFO : Epoch 319 | loss: 0.031073 | val_loss: 0.0312495 | Time: 6245.99 ms
[2021-09-22 12:49:00 main:574] : INFO : Epoch 320 | loss: 0.0310731 | val_loss: 0.0312221 | Time: 6217.26 ms
[2021-09-22 12:49:06 main:574] : INFO : Epoch 321 | loss: 0.0310712 | val_loss: 0.0312464 | Time: 6243.59 ms
[2021-09-22 12:49:12 main:574] : INFO : Epoch 322 | loss: 0.0310665 | val_loss: 0.0312346 | Time: 6220.46 ms
[2021-09-22 12:49:18 main:574] : INFO : Epoch 323 | loss: 0.0310687 | val_loss: 0.0312571 | Time: 6205.51 ms
[2021-09-22 12:49:25 main:574] : INFO : Epoch 324 | loss: 0.0310741 | val_loss: 0.0312443 | Time: 6270.99 ms
[2021-09-22 12:49:31 main:574] : INFO : Epoch 325 | loss: 0.0310705 | val_loss: 0.0312395 | Time: 6195.32 ms
[2021-09-22 12:49:37 main:574] : INFO : Epoch 326 | loss: 0.0310735 | val_loss: 0.031241 | Time: 6183.85 ms
[2021-09-22 12:49:44 main:574] : INFO : Epoch 327 | loss: 0.0310678 | val_loss: 0.0312555 | Time: 6319.85 ms
[2021-09-22 12:49:50 main:574] : INFO : Epoch 328 | loss: 0.031066 | val_loss: 0.0312602 | Time: 6305.44 ms
[2021-09-22 12:49:56 main:574] : INFO : Epoch 329 | loss: 0.0310669 | val_loss: 0.0312536 | Time: 6238.11 ms
[2021-09-22 12:50:03 main:574] : INFO : Epoch 330 | loss: 0.0310736 | val_loss: 0.0312406 | Time: 6293.66 ms
[2021-09-22 12:50:09 main:574] : INFO : Epoch 331 | loss: 0.031074 | val_loss: 0.0312295 | Time: 6252.41 ms
[2021-09-22 12:50:15 main:574] : INFO : Epoch 332 | loss: 0.031072 | val_loss: 0.0312388 | Time: 6236.72 ms
[2021-09-22 12:50:22 main:574] : INFO : Epoch 333 | loss: 0.0310724 | val_loss: 0.0312577 | Time: 6206.66 ms
[2021-09-22 12:50:28 main:574] : INFO : Epoch 334 | loss: 0.0310705 | val_loss: 0.0312362 | Time: 6263.85 ms
[2021-09-22 12:50:34 main:574] : INFO : Epoch 335 | loss: 0.0310707 | val_loss: 0.0312614 | Time: 6258.35 ms
[2021-09-22 12:50:40 main:574] : INFO : Epoch 336 | loss: 0.0310739 | val_loss: 0.0312504 | Time: 6253.38 ms
[2021-09-22 12:50:47 main:574] : INFO : Epoch 337 | loss: 0.0310698 | val_loss: 0.0312495 | Time: 6210.15 ms
[2021-09-22 12:50:53 main:574] : INFO : Epoch 338 | loss: 0.0310665 | val_loss: 0.0312461 | Time: 6216.04 ms
[2021-09-22 12:50:59 main:574] : INFO : Epoch 339 | loss: 0.0310613 | val_loss: 0.0312464 | Time: 6224.5 ms
[2021-09-22 12:51:14 main:574] : INFO : Epoch 340 | loss: 0.0310655 | val_loss: 0.0312439 | Time: 6255.55 ms
[2021-09-22 12:51:21 main:574] : INFO : Epoch 341 | loss: 0.0310616 | val_loss: 0.0312493 | Time: 6277.71 ms
[2021-09-22 12:51:27 main:574] : INFO : Epoch 342 | loss: 0.0310658 | val_loss: 0.0312502 | Time: 6326.9 ms
[2021-09-22 12:51:34 main:574] : INFO : Epoch 343 | loss: 0.0310663 | val_loss: 0.0312618 | Time: 6267.02 ms
[2021-09-22 12:51:40 main:574] : INFO : Epoch 344 | loss: 0.0310638 | val_loss: 0.0312653 | Time: 6322.44 ms
[2021-09-22 12:51:46 main:574] : INFO : Epoch 345 | loss: 0.0310663 | val_loss: 0.0312465 | Time: 6284.8 ms
[2021-09-22 12:51:53 main:574] : INFO : Epoch 346 | loss: 0.0310697 | val_loss: 0.0312475 | Time: 6217.33 ms
[2021-09-22 12:51:59 main:574] : INFO : Epoch 347 | loss: 0.031062 | val_loss: 0.031257 | Time: 6219.57 ms
[2021-09-22 12:52:05 main:574] : INFO : Epoch 348 | loss: 0.0310584 | val_loss: 0.0312535 | Time: 6216.14 ms
[2021-09-22 12:52:12 main:574] : INFO : Epoch 349 | loss: 0.0310587 | val_loss: 0.0312541 | Time: 6349.08 ms
[2021-09-22 12:52:18 main:574] : INFO : Epoch 350 | loss: 0.0310645 | val_loss: 0.0312401 | Time: 6230.49 ms
[2021-09-22 12:52:24 main:574] : INFO : Epoch 351 | loss: 0.0310668 | val_loss: 0.0312373 | Time: 6260.64 ms
[2021-09-22 12:52:31 main:574] : INFO : Epoch 352 | loss: 0.031062 | val_loss: 0.0312495 | Time: 6237.46 ms
[2021-09-22 12:52:37 main:574] : INFO : Epoch 353 | loss: 0.031064 | val_loss: 0.0312422 | Time: 6194.71 ms
[2021-09-22 12:52:43 main:574] : INFO : Epoch 354 | loss: 0.0310637 | val_loss: 0.0312636 | Time: 6276.66 ms
[2021-09-22 12:52:50 main:574] : INFO : Epoch 355 | loss: 0.0310593 | val_loss: 0.0312496 | Time: 6209.58 ms
[2021-09-22 12:52:56 main:574] : INFO : Epoch 356 | loss: 0.0310594 | val_loss: 0.0312488 | Time: 6251.99 ms
[2021-09-22 12:53:02 main:574] : INFO : Epoch 357 | loss: 0.0310637 | val_loss: 0.0312508 | Time: 6291.27 ms
[2021-09-22 12:53:08 main:574] : INFO : Epoch 358 | loss: 0.0310632 | val_loss: 0.031253 | Time: 6246.48 ms
[2021-09-22 12:53:15 main:574] : INFO : Epoch 359 | loss: 0.0310685 | val_loss: 0.0312635 | Time: 6349.03 ms
[2021-09-22 12:53:21 main:574] : INFO : Epoch 360 | loss: 0.0310661 | val_loss: 0.0312535 | Time: 6275.58 ms
[2021-09-22 12:53:28 main:574] : INFO : Epoch 361 | loss: 0.0310577 | val_loss: 0.0312463 | Time: 6261.2 ms
[2021-09-22 12:53:34 main:574] : INFO : Epoch 362 | loss: 0.0310575 | val_loss: 0.0312524 | Time: 6344.08 ms
[2021-09-22 12:53:40 main:574] : INFO : Epoch 363 | loss: 0.0310639 | val_loss: 0.0312419 | Time: 6410.48 ms
[2021-09-22 12:53:47 main:574] : INFO : Epoch 364 | loss: 0.031064 | val_loss: 0.0312415 | Time: 6253.07 ms
[2021-09-22 12:53:53 main:574] : INFO : Epoch 365 | loss: 0.031061 | val_loss: 0.0312444 | Time: 6226.3 ms
[2021-09-22 12:53:59 main:574] : INFO : Epoch 366 | loss: 0.0310616 | val_loss: 0.0312463 | Time: 6270.21 ms
[2021-09-22 12:54:06 main:574] : INFO : Epoch 367 | loss: 0.0310614 | val_loss: 0.0312531 | Time: 6280.28 ms
[2021-09-22 12:54:12 main:574] : INFO : Epoch 368 | loss: 0.0310623 | val_loss: 0.0312602 | Time: 6188.09 ms
[2021-09-22 12:54:18 main:574] : INFO : Epoch 369 | loss: 0.0310598 | val_loss: 0.0312573 | Time: 6255.26 ms
[2021-09-22 12:54:25 main:574] : INFO : Epoch 370 | loss: 0.031061 | val_loss: 0.031252 | Time: 6295.84 ms
[2021-09-22 12:54:31 main:574] : INFO : Epoch 371 | loss: 0.0310608 | val_loss: 0.0312482 | Time: 6345.76 ms
[2021-09-22 12:54:37 main:574] : INFO : Epoch 372 | loss: 0.0310634 | val_loss: 0.0312505 | Time: 6329.59 ms
[2021-09-22 12:54:44 main:574] : INFO : Epoch 373 | loss: 0.0310583 | val_loss: 0.0312685 | Time: 6330.22 ms
[2021-09-22 12:54:50 main:574] : INFO : Epoch 374 | loss: 0.0310558 | val_loss: 0.0312474 | Time: 6354.2 ms
[2021-09-22 12:54:57 main:574] : INFO : Epoch 375 | loss: 0.0310608 | val_loss: 0.0312472 | Time: 6230.01 ms
[2021-09-22 12:55:03 main:574] : INFO : Epoch 376 | loss: 0.0310577 | val_loss: 0.0312434 | Time: 6326.71 ms
[2021-09-22 12:55:09 main:574] : INFO : Epoch 377 | loss: 0.0310618 | val_loss: 0.0312509 | Time: 6343.36 ms
[2021-09-22 12:55:16 main:574] : INFO : Epoch 378 | loss: 0.0310641 | val_loss: 0.0312594 | Time: 6363.35 ms
[2021-09-22 12:55:22 main:574] : INFO : Epoch 379 | loss: 0.0310606 | val_loss: 0.031254 | Time: 6400.85 ms
[2021-09-22 12:55:29 main:574] : INFO : Epoch 380 | loss: 0.0310604 | val_loss: 0.0312638 | Time: 6302.51 ms
[2021-09-22 12:55:35 main:574] : INFO : Epoch 381 | loss: 0.031058 | val_loss: 0.031263 | Time: 6240.69 ms
[2021-09-22 12:55:41 main:574] : INFO : Epoch 382 | loss: 0.0310711 | val_loss: 0.0312701 | Time: 6322.38 ms
[2021-09-22 12:55:48 main:574] : INFO : Epoch 383 | loss: 0.03106 | val_loss: 0.0312591 | Time: 6315.23 ms
[2021-09-22 12:55:54 main:574] : INFO : Epoch 384 | loss: 0.0310617 | val_loss: 0.0312407 | Time: 6225.01 ms
[2021-09-22 12:56:00 main:574] : INFO : Epoch 385 | loss: 0.0310592 | val_loss: 0.0312362 | Time: 6270.82 ms
[2021-09-22 12:56:07 main:574] : INFO : Epoch 386 | loss: 0.0310533 | val_loss: 0.0312534 | Time: 6327.72 ms
[2021-09-22 12:56:13 main:574] : INFO : Epoch 387 | loss: 0.0310562 | val_loss: 0.0312737 | Time: 6275.23 ms
[2021-09-22 12:56:19 main:574] : INFO : Epoch 388 | loss: 0.0310566 | val_loss: 0.0312647 | Time: 6330.2 ms
[2021-09-22 12:56:26 main:574] : INFO : Epoch 389 | loss: 0.031056 | val_loss: 0.031268 | Time: 6285.67 ms
[2021-09-22 12:56:32 main:574] : INFO : Epoch 390 | loss: 0.0310599 | val_loss: 0.0312455 | Time: 6389.9 ms
[2021-09-22 12:56:39 main:574] : INFO : Epoch 391 | loss: 0.0310545 | val_loss: 0.0312524 | Time: 6316.72 ms
[2021-09-22 12:56:45 main:574] : INFO : Epoch 392 | loss: 0.0310569 | val_loss: 0.031276 | Time: 6320.23 ms
[2021-09-22 12:56:52 main:574] : INFO : Epoch 393 | loss: 0.0310615 | val_loss: 0.0312672 | Time: 6371.23 ms
[2021-09-22 12:56:58 main:574] : INFO : Epoch 394 | loss: 0.0310557 | val_loss: 0.0312548 | Time: 6426.43 ms
[2021-09-22 12:57:04 main:574] : INFO : Epoch 395 | loss: 0.0310616 | val_loss: 0.0312436 | Time: 6421.53 ms
[2021-09-22 12:57:11 main:574] : INFO : Epoch 396 | loss: 0.0310641 | val_loss: 0.0312493 | Time: 6376.14 ms
[2021-09-22 12:57:17 main:574] : INFO : Epoch 397 | loss: 0.0310574 | val_loss: 0.0312606 | Time: 6346.19 ms
[2021-09-22 12:57:24 main:574] : INFO : Epoch 398 | loss: 0.0310521 | val_loss: 0.0312716 | Time: 6418.11 ms
[2021-09-22 12:57:30 main:574] : INFO : Epoch 399 | loss: 0.031053 | val_loss: 0.0312532 | Time: 6277.86 ms
[2021-09-22 12:57:36 main:574] : INFO : Epoch 400 | loss: 0.0310504 | val_loss: 0.0312531 | Time: 6247.8 ms
[2021-09-22 12:57:43 main:574] : INFO : Epoch 401 | loss: 0.031051 | val_loss: 0.0312667 | Time: 6217.43 ms
[2021-09-22 12:57:49 main:574] : INFO : Epoch 402 | loss: 0.031057 | val_loss: 0.031269 | Time: 6228.14 ms
[2021-09-22 12:57:55 main:574] : INFO : Epoch 403 | loss: 0.0310549 | val_loss: 0.0312363 | Time: 6287.54 ms
[2021-09-22 12:58:02 main:574] : INFO : Epoch 404 | loss: 0.0310567 | val_loss: 0.0312281 | Time: 6252.86 ms
[2021-09-22 12:58:08 main:574] : INFO : Epoch 405 | loss: 0.0310606 | val_loss: 0.0312644 | Time: 6252.18 ms
[2021-09-22 12:58:14 main:574] : INFO : Epoch 406 | loss: 0.0310528 | val_loss: 0.0312575 | Time: 6318.61 ms
[2021-09-22 12:58:21 main:574] : INFO : Epoch 407 | loss: 0.0310501 | val_loss: 0.0312653 | Time: 6341.54 ms
[2021-09-22 12:58:27 main:574] : INFO : Epoch 408 | loss: 0.0310597 | val_loss: 0.0312572 | Time: 6344.23 ms
[2021-09-22 12:58:34 main:574] : INFO : Epoch 409 | loss: 0.0310569 | val_loss: 0.03126 | Time: 6320.7 ms
[2021-09-22 12:58:40 main:574] : INFO : Epoch 410 | loss: 0.0310503 | val_loss: 0.0312645 | Time: 6257.18 ms
[2021-09-22 12:58:46 main:574] : INFO : Epoch 411 | loss: 0.0310486 | val_loss: 0.0312706 | Time: 6232.14 ms
[2021-09-22 12:58:53 main:574] : INFO : Epoch 412 | loss: 0.0310527 | val_loss: 0.0312498 | Time: 6307.04 ms
[2021-09-22 12:58:59 main:574] : INFO : Epoch 413 | loss: 0.0310579 | val_loss: 0.0312532 | Time: 6344.99 ms
[2021-09-22 12:59:06 main:574] : INFO : Epoch 414 | loss: 0.0310551 | val_loss: 0.031252 | Time: 6344.3 ms
[2021-09-22 12:59:12 main:574] : INFO : Epoch 415 | loss: 0.0310622 | val_loss: 0.0312513 | Time: 6344.5 ms
[2021-09-22 12:59:18 main:574] : INFO : Epoch 416 | loss: 0.0310528 | val_loss: 0.0312705 | Time: 6254.53 ms
[2021-09-22 12:59:25 main:574] : INFO : Epoch 417 | loss: 0.0310508 | val_loss: 0.0312424 | Time: 6218.94 ms
[2021-09-22 12:59:31 main:574] : INFO : Epoch 418 | loss: 0.0310516 | val_loss: 0.0312394 | Time: 6299.66 ms
[2021-09-22 12:59:37 main:574] : INFO : Epoch 419 | loss: 0.0310564 | val_loss: 0.031249 | Time: 6349.15 ms
[2021-09-22 12:59:44 main:574] : INFO : Epoch 420 | loss: 0.0310502 | val_loss: 0.0312596 | Time: 6366.12 ms
[2021-09-22 12:59:50 main:574] : INFO : Epoch 421 | loss: 0.0310479 | val_loss: 0.0312549 | Time: 6277.76 ms
[2021-09-22 12:59:57 main:574] : INFO : Epoch 422 | loss: 0.0310501 | val_loss: 0.0312685 | Time: 6326.48 ms
[2021-09-22 13:00:03 main:574] : INFO : Epoch 423 | loss: 0.0310486 | val_loss: 0.031262 | Time: 6275.91 ms
[2021-09-22 13:00:09 main:574] : INFO : Epoch 424 | loss: 0.0310521 | val_loss: 0.0312691 | Time: 6286.03 ms
[2021-09-22 13:00:16 main:574] : INFO : Epoch 425 | loss: 0.0310553 | val_loss: 0.0312573 | Time: 6253.9 ms
[2021-09-22 13:00:22 main:574] : INFO : Epoch 426 | loss: 0.0310523 | val_loss: 0.0312501 | Time: 6321.26 ms
[2021-09-22 13:00:28 main:574] : INFO : Epoch 427 | loss: 0.0310495 | val_loss: 0.0312662 | Time: 6230.48 ms
[2021-09-22 13:00:35 main:574] : INFO : Epoch 428 | loss: 0.0310505 | val_loss: 0.03127 | Time: 6291.06 ms
[2021-09-22 13:00:41 main:574] : INFO : Epoch 429 | loss: 0.0310506 | val_loss: 0.0312643 | Time: 6180.15 ms
[2021-09-22 13:00:47 main:574] : INFO : Epoch 430 | loss: 0.0310516 | val_loss: 0.0312746 | Time: 6276.92 ms
[2021-09-22 13:00:54 main:574] : INFO : Epoch 431 | loss: 0.0310552 | val_loss: 0.0312645 | Time: 6255.2 ms
[2021-09-22 13:01:00 main:574] : INFO : Epoch 432 | loss: 0.031056 | val_loss: 0.0312627 | Time: 6226.9 ms
[2021-09-22 13:01:06 main:574] : INFO : Epoch 433 | loss: 0.03105 | val_loss: 0.0312592 | Time: 6252.46 ms
[2021-09-22 13:01:13 main:574] : INFO : Epoch 434 | loss: 0.0310587 | val_loss: 0.0312466 | Time: 6253.36 ms
[2021-09-22 13:01:19 main:574] : INFO : Epoch 435 | loss: 0.0310584 | val_loss: 0.0312478 | Time: 6230.65 ms
[2021-09-22 13:01:25 main:574] : INFO : Epoch 436 | loss: 0.031046 | val_loss: 0.0312501 | Time: 6328.18 ms
[2021-09-22 13:01:32 main:574] : INFO : Epoch 437 | loss: 0.0310515 | val_loss: 0.0312663 | Time: 6217.64 ms
[2021-09-22 13:01:38 main:574] : INFO : Epoch 438 | loss: 0.0310484 | val_loss: 0.0312683 | Time: 6238.68 ms
[2021-09-22 13:01:45 main:574] : INFO : Epoch 439 | loss: 0.0310497 | val_loss: 0.0312514 | Time: 6215.83 ms
[2021-09-22 13:01:51 main:574] : INFO : Epoch 440 | loss: 0.0310491 | val_loss: 0.0312697 | Time: 6219.61 ms
[2021-09-22 13:01:58 main:574] : INFO : Epoch 441 | loss: 0.0310448 | val_loss: 0.0312648 | Time: 6250.1 ms
[2021-09-22 13:02:04 main:574] : INFO : Epoch 442 | loss: 0.0310497 | val_loss: 0.0312597 | Time: 6288.16 ms
[2021-09-22 13:02:10 main:574] : INFO : Epoch 443 | loss: 0.0310562 | val_loss: 0.031261 | Time: 6435.73 ms
[2021-09-22 13:02:17 main:574] : INFO : Epoch 444 | loss: 0.0310519 | val_loss: 0.0312651 | Time: 6268.6 ms
[2021-09-22 13:02:23 main:574] : INFO : Epoch 445 | loss: 0.0310509 | val_loss: 0.0312773 | Time: 6212.84 ms
[2021-09-22 13:02:29 main:574] : INFO : Epoch 446 | loss: 0.0310478 | val_loss: 0.0312588 | Time: 6249.77 ms
[2021-09-22 13:02:36 main:574] : INFO : Epoch 447 | loss: 0.0310533 | val_loss: 0.0312635 | Time: 6249.89 ms
[2021-09-22 13:02:42 main:574] : INFO : Epoch 448 | loss: 0.0310543 | val_loss: 0.0312603 | Time: 6259.28 ms
[2021-09-22 13:02:48 main:574] : INFO : Epoch 449 | loss: 0.0310528 | val_loss: 0.0312589 | Time: 6264.35 ms
[2021-09-22 13:02:55 main:574] : INFO : Epoch 450 | loss: 0.0310493 | val_loss: 0.0312755 | Time: 6313.11 ms
[2021-09-22 13:03:01 main:574] : INFO : Epoch 451 | loss: 0.0310486 | val_loss: 0.0312612 | Time: 6283.58 ms
[2021-09-22 13:03:08 main:574] : INFO : Epoch 452 | loss: 0.0310436 | val_loss: 0.0312751 | Time: 6277.8 ms
[2021-09-22 13:03:14 main:574] : INFO : Epoch 453 | loss: 0.0310424 | val_loss: 0.0312686 | Time: 6275.81 ms
[2021-09-22 13:03:20 main:574] : INFO : Epoch 454 | loss: 0.0310435 | val_loss: 0.0312641 | Time: 6236.76 ms
[2021-09-22 13:03:27 main:574] : INFO : Epoch 455 | loss: 0.0310463 | val_loss: 0.0312595 | Time: 6453.44 ms
[2021-09-22 13:03:33 main:574] : INFO : Epoch 456 | loss: 0.0310487 | val_loss: 0.031265 | Time: 6315.5 ms
[2021-09-22 13:03:39 main:574] : INFO : Epoch 457 | loss: 0.0310515 | val_loss: 0.0312623 | Time: 6294.25 ms
[2021-09-22 13:03:46 main:574] : INFO : Epoch 458 | loss: 0.0310497 | val_loss: 0.0312766 | Time: 6269.86 ms
[2021-09-22 13:03:52 main:574] : INFO : Epoch 459 | loss: 0.031047 | val_loss: 0.031261 | Time: 6213.66 ms
[2021-09-22 13:03:59 main:574] : INFO : Epoch 460 | loss: 0.0310436 | val_loss: 0.0312583 | Time: 6358.57 ms
[2021-09-22 13:04:05 main:574] : INFO : Epoch 461 | loss: 0.031043 | val_loss: 0.0312695 | Time: 6308.87 ms
[2021-09-22 13:04:11 main:574] : INFO : Epoch 462 | loss: 0.0310473 | val_loss: 0.031259 | Time: 6289.85 ms
[2021-09-22 13:04:18 main:574] : INFO : Epoch 463 | loss: 0.0310495 | val_loss: 0.0312546 | Time: 6261.9 ms
[2021-09-22 13:04:24 main:574] : INFO : Epoch 464 | loss: 0.0310466 | val_loss: 0.0312718 | Time: 6231.21 ms
[2021-09-22 13:04:30 main:574] : INFO : Epoch 465 | loss: 0.0310525 | val_loss: 0.0312767 | Time: 6373.86 ms
[2021-09-22 13:04:37 main:574] : INFO : Epoch 466 | loss: 0.0310479 | val_loss: 0.0312578 | Time: 6258.8 ms
[2021-09-22 13:04:43 main:574] : INFO : Epoch 467 | loss: 0.0310503 | val_loss: 0.0312457 | Time: 6327.24 ms
[2021-09-22 13:04:49 main:574] : INFO : Epoch 468 | loss: 0.0310469 | val_loss: 0.0312542 | Time: 6387.27 ms
[2021-09-22 13:04:56 main:574] : INFO : Epoch 469 | loss: 0.0310526 | val_loss: 0.0312511 | Time: 6351.67 ms
[2021-09-22 13:05:02 main:574] : INFO : Epoch 470 | loss: 0.0310491 | val_loss: 0.0312762 | Time: 6321.06 ms
[2021-09-22 13:05:09 main:574] : INFO : Epoch 471 | loss: 0.0310489 | val_loss: 0.0312698 | Time: 6247.33 ms
[2021-09-22 13:05:15 main:574] : INFO : Epoch 472 | loss: 0.0310417 | val_loss: 0.0312579 | Time: 6417.56 ms
[2021-09-22 13:05:22 main:574] : INFO : Epoch 473 | loss: 0.0310441 | val_loss: 0.0312795 | Time: 6404.78 ms
[2021-09-22 13:05:28 main:574] : INFO : Epoch 474 | loss: 0.0310488 | val_loss: 0.0312694 | Time: 6334.32 ms
[2021-09-22 13:05:35 main:574] : INFO : Epoch 475 | loss: 0.0310478 | val_loss: 0.0312585 | Time: 6602.03 ms
[2021-09-22 13:05:43 main:574] : INFO : Epoch 476 | loss: 0.0310481 | val_loss: 0.031262 | Time: 8734.16 ms
[2021-09-22 13:05:50 main:574] : INFO : Epoch 477 | loss: 0.0310619 | val_loss: 0.0312451 | Time: 6347.77 ms
[2021-09-22 13:05:56 main:574] : INFO : Epoch 478 | loss: 0.0310663 | val_loss: 0.0312501 | Time: 6356.63 ms
[2021-09-22 13:06:03 main:574] : INFO : Epoch 479 | loss: 0.0310532 | val_loss: 0.0312444 | Time: 6273.72 ms
[2021-09-22 13:06:07 main:574] : INFO : Epoch 480 | loss: 0.0310472 | val_loss: 0.0312575 | Time: 4445.23 ms
[2021-09-22 13:06:11 main:574] : INFO : Epoch 481 | loss: 0.0310472 | val_loss: 0.031266 | Time: 3660.66 ms
[2021-09-22 13:06:15 main:574] : INFO : Epoch 482 | loss: 0.0310472 | val_loss: 0.0312648 | Time: 3701.25 ms
[2021-09-22 13:06:18 main:574] : INFO : Epoch 483 | loss: 0.0310549 | val_loss: 0.0312601 | Time: 3689.65 ms
[2021-09-22 13:06:28 main:574] : INFO : Epoch 484 | loss: 0.0310507 | val_loss: 0.0312724 | Time: 9288.18 ms
[2021-09-22 13:06:43 main:574] : INFO : Epoch 485 | loss: 0.0310516 | val_loss: 0.0312616 | Time: 15614.6 ms
[2021-09-22 13:06:55 main:574] : INFO : Epoch 486 | loss: 0.0310455 | val_loss: 0.0312565 | Time: 11213.8 ms
[2021-09-22 13:07:06 main:574] : INFO : Epoch 487 | loss: 0.0310453 | val_loss: 0.031255 | Time: 10852.2 ms
[2021-09-22 13:07:15 main:574] : INFO : Epoch 488 | loss: 0.0310514 | val_loss: 0.031267 | Time: 9369.82 ms
[2021-09-22 13:07:25 main:574] : INFO : Epoch 489 | loss: 0.0310517 | val_loss: 0.0312639 | Time: 9646.81 ms
[2021-09-22 13:07:35 main:574] : INFO : Epoch 490 | loss: 0.0310436 | val_loss: 0.031269 | Time: 9734.85 ms
[2021-09-22 13:07:44 main:574] : INFO : Epoch 491 | loss: 0.0310434 | val_loss: 0.0312664 | Time: 9528.69 ms
[2021-09-22 13:07:54 main:574] : INFO : Epoch 492 | loss: 0.0310433 | val_loss: 0.0312558 | Time: 9351.71 ms
[2021-09-22 13:08:03 main:574] : INFO : Epoch 493 | loss: 0.0310396 | val_loss: 0.0312811 | Time: 9221.82 ms
[2021-09-22 13:08:12 main:574] : INFO : Epoch 494 | loss: 0.0310439 | val_loss: 0.0312678 | Time: 9288.54 ms
[2021-09-22 13:08:22 main:574] : INFO : Epoch 495 | loss: 0.0310455 | val_loss: 0.0312649 | Time: 9319.19 ms
[2021-09-22 13:08:31 main:574] : INFO : Epoch 496 | loss: 0.0310523 | val_loss: 0.0312466 | Time: 9138.84 ms
[2021-09-22 13:08:40 main:574] : INFO : Epoch 497 | loss: 0.031051 | val_loss: 0.0312653 | Time: 9237.18 ms
[2021-09-22 13:08:49 main:574] : INFO : Epoch 498 | loss: 0.0310431 | val_loss: 0.0312712 | Time: 9300.92 ms
[2021-09-22 13:08:59 main:574] : INFO : Epoch 499 | loss: 0.0310437 | val_loss: 0.0312821 | Time: 9320.68 ms
[2021-09-22 13:09:08 main:574] : INFO : Epoch 500 | loss: 0.0310446 | val_loss: 0.031257 | Time: 9111.57 ms
[2021-09-22 13:09:17 main:574] : INFO : Epoch 501 | loss: 0.0310436 | val_loss: 0.0312592 | Time: 8797.66 ms
[2021-09-22 13:09:26 main:574] : INFO : Epoch 502 | loss: 0.031044 | val_loss: 0.0312633 | Time: 8837.95 ms
[2021-09-22 13:09:36 main:574] : INFO : Epoch 503 | loss: 0.0310467 | val_loss: 0.0312544 | Time: 9963.84 ms
[2021-09-22 13:09:47 main:574] : INFO : Epoch 504 | loss: 0.0310427 | val_loss: 0.0312617 | Time: 10855.9 ms
[2021-09-22 13:09:57 main:574] : INFO : Epoch 505 | loss: 0.0310419 | val_loss: 0.0312814 | Time: 9686.74 ms
[2021-09-22 13:10:06 main:574] : INFO : Epoch 506 | loss: 0.0310444 | val_loss: 0.0312747 | Time: 9123 ms
[2021-09-22 13:10:16 main:574] : INFO : Epoch 507 | loss: 0.0310419 | val_loss: 0.0312611 | Time: 9978.35 ms
[2021-09-22 13:10:26 main:574] : INFO : Epoch 508 | loss: 0.0310449 | val_loss: 0.0312757 | Time: 9651.41 ms
[2021-09-22 13:10:36 main:574] : INFO : Epoch 509 | loss: 0.0310427 | val_loss: 0.0312611 | Time: 10115.3 ms
[2021-09-22 13:10:46 main:574] : INFO : Epoch 510 | loss: 0.0310424 | val_loss: 0.0312852 | Time: 9647.21 ms
[2021-09-22 13:10:55 main:574] : INFO : Epoch 511 | loss: 0.031038 | val_loss: 0.0312598 | Time: 9768.92 ms
[2021-09-22 13:11:05 main:574] : INFO : Epoch 512 | loss: 0.0310526 | val_loss: 0.0312488 | Time: 9796.09 ms
[2021-09-22 13:11:15 main:574] : INFO : Epoch 513 | loss: 0.0310509 | val_loss: 0.0312628 | Time: 9757.63 ms
[2021-09-22 13:11:25 main:574] : INFO : Epoch 514 | loss: 0.0310417 | val_loss: 0.0312584 | Time: 9284.38 ms
[2021-09-22 13:11:34 main:574] : INFO : Epoch 515 | loss: 0.0310481 | val_loss: 0.0312757 | Time: 9421.12 ms
[2021-09-22 13:11:44 main:574] : INFO : Epoch 516 | loss: 0.031044 | val_loss: 0.0312596 | Time: 10033.2 ms
[2021-09-22 13:11:54 main:574] : INFO : Epoch 517 | loss: 0.0310447 | val_loss: 0.0312602 | Time: 9827.73 ms
[2021-09-22 13:12:04 main:574] : INFO : Epoch 518 | loss: 0.0310447 | val_loss: 0.0312538 | Time: 9640.53 ms
[2021-09-22 13:12:13 main:574] : INFO : Epoch 519 | loss: 0.0310447 | val_loss: 0.0312676 | Time: 9527.59 ms
[2021-09-22 13:12:23 main:574] : INFO : Epoch 520 | loss: 0.0310464 | val_loss: 0.0312523 | Time: 9662.91 ms
[2021-09-22 13:12:33 main:574] : INFO : Epoch 521 | loss: 0.0310451 | val_loss: 0.0312597 | Time: 9572.62 ms
[2021-09-22 13:12:42 main:574] : INFO : Epoch 522 | loss: 0.0310464 | val_loss: 0.0312794 | Time: 9704.8 ms
[2021-09-22 13:12:52 main:574] : INFO : Epoch 523 | loss: 0.0310487 | val_loss: 0.0312679 | Time: 9603.12 ms
[2021-09-22 13:13:02 main:574] : INFO : Epoch 524 | loss: 0.0310438 | val_loss: 0.0312883 | Time: 9609.62 ms
[2021-09-22 13:13:11 main:574] : INFO : Epoch 525 | loss: 0.031045 | val_loss: 0.0312641 | Time: 9606.02 ms
[2021-09-22 13:13:21 main:574] : INFO : Epoch 526 | loss: 0.0310433 | val_loss: 0.0312748 | Time: 9620.15 ms
[2021-09-22 13:13:31 main:574] : INFO : Epoch 527 | loss: 0.0310465 | val_loss: 0.0312499 | Time: 9569.01 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFB18E44F99
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 09/22/21 13:13:35
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\34;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000de480000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000001b310000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1023 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1023
ModLoad: 0000000014690000 0000000000015000 C:\Program Files\AVAST Software\Avast\aswhook.dll (21.7.6523.0) (-nosymbols- Symbols Loaded)
Linked PDB Filename : D:\work\616216c47067779c\idp\build\avgast\RU_NTDLL-lH-Y2A!x64_vs142\bin\Release_Unicode_NTDLL_vs142\x64\aswhook.pdb
File Version : 21.7.6523.0
Company Name : AVAST Software
Product Name : Avast
Product Version : 21.7.6523.0
ModLoad: 0000000019410000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000018e10000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 000000001a8f0000 00000000001a1000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000018de0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1237) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1237 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1237
ModLoad: 0000000019ac0000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000019250000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1110
ModLoad: 0000000018d40000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000018a30000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000194d0000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1052) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019890000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 000000001a7a0000 000000000009b000 C:\WINDOWS\System32\sechost.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019930000 000000000012a000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d0680000 0000000000058000 C:\ProgramData\BOINC\slots\34\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000ec6a0000 000000000008d000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 0000000018410000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 00000000ec7a0000 000000000001a000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000faa70000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000a72b0000 000000000abe7000 C:\ProgramData\BOINC\slots\34\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bca80000 00000000002db000 C:\ProgramData\BOINC\slots\34\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000e3960000 0000000000040000 C:\ProgramData\BOINC\slots\34\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bc650000 000000000020d000 C:\ProgramData\BOINC\slots\34\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 0000000082760000 0000000024b49000 C:\ProgramData\BOINC\slots\34\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d0590000 0000000000045000 C:\ProgramData\BOINC\slots\34\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000007f6b0000 00000000030a5000 C:\ProgramData\BOINC\slots\34\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000fc2f0000 0000000000010000 C:\ProgramData\BOINC\slots\34\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 000000006f360000 00000000034b2000 C:\ProgramData\BOINC\slots\34\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000591f0000 000000000919f000 C:\ProgramData\BOINC\slots\34\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000055490000 0000000003d5b000 C:\ProgramData\BOINC\slots\34\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000006d490000 0000000001eca000 C:\ProgramData\BOINC\slots\34\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000000cce0000 000000001b3a4000 C:\ProgramData\BOINC\slots\34\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 0000000019b60000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000015c40000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001a170000 0000000000472000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1237) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000018cf0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000019170000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000018800000 0000000000034000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000018c90000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1237) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1237 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1237
ModLoad: 0000000018b30000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000018370000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001aab0000 000000000073f000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 00000000164c0000 0000000000790000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019e10000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000017cd0000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000001a840000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019b00000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000bc3e0000 0000000000267000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.5671) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 27.21.14.5671
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.96 driver
Product Version : 27.21.14.5671
ModLoad: 00000000157c0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000006bf60000 000000000152a000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_60daf66a00f2e0b6\nvcuda64.dll (27.21.14.5671) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 27.21.14.5671
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.96 driver
Product Version : 27.21.14.5671
ModLoad: 0000000019c80000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.19041.1081 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1081
ModLoad: 000000000eba0000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 0000000017b10000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000156a0000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000000a880000 00000000006d5000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.5671) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 27.21.14.5671
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 27.21.14.5671
ModLoad: 0000000019620000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 00000000fc230000 0000000000009000 C:\ProgramData\BOINC\slots\34\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000046530000 0000000000fca000 C:\ProgramData\BOINC\slots\34\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 00000000190e0000 0000000000083000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 4137, Write: 2702, Other 6362
- I/O Transfers Counters -
Read: 6516350, Write: 2491303, Other 48760
- Paged Pool Usage -
QuotaPagedPoolUsage: 3506144, QuotaPeakPagedPoolUsage: 3522528
QuotaNonPagedPoolUsage: 53440, QuotaPeakNonPagedPoolUsage: 53440
- Virtual Memory Usage -
VirtualSize: -437051392, PeakVirtualSize: 181927936
- Pagefile Usage -
PagefileUsage: -437051392, PeakPagefileUsage: -436985856
- Working Set Size -
WorkingSetSize: 462733312, PeakWorkingSetSize: 1827008512, PageFaultCount: 750460
*** Dump of thread ID 36500 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 13702968320.000000, User Time: 9671718912.000000, Wait Time: 6605242.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFB18E44F99
- Registers -
rax=00000000ec6d7893 rbx=00000000861db320 rcx=00000000d1a60860 rdx=00000000d0591999 rsi=0000000018374a80 rdi=0000000019930520
r8=000000000b6a3e70 r9=00000000d05c9008 r10=0000000018374a60 r11=0000000000000002 r12=0000000000e1e200 r13=0000000010200000
r14=0000000000e14800 r15=0000000000633450 rip=0000000018e44f99 rsp=00000000183745a0 rbp=0000000018374810
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
18374670 ec7a6480 85fb1fc8 85fb17f0 00000312 00000000 KERNELBASE!RaiseException+0x0
183746d0 854d9724 85fadbb8 0f400000 18374878 0ee00000 VCRUNTIME140!_CxxThrowException+0x0
18374bb0 855325ea 18374c78 00000001 18374c78 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
18374c90 85540cbf 18375380 18376b88 18375cd0 1b33b86b torch_cuda!at::native::set_storage_cuda_+0x0
18374d70 accfb9fc 18375380 00000000 0b6a5e70 0b6a5e40 torch_cuda!at::native::set_storage_cuda_+0x0
18374ef0 acd331a2 00000001 ae4a67c2 18374f24 1b33b86b torch_cpu!at::native::mkldnn_sigmoid_+0x0
183751d0 adfe2613 00000000 00000000 00000003 043c0688 torch_cpu!at::_cudnn_rnn+0x0
183756d0 adff080b 0f5387e8 0c079d70 18375800 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
183757b0 accfb9fc 18375d30 6bf85a07 183759e8 18375860 torch_cpu!torch::autograd::GraphRoot::apply+0x0
18375930 acd331a2 1cb7cef0 00000000 0f538578 863fa3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
18375c10 854d69a1 18370001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
18375de0 854e0117 00000000 18376b88 d0592e0d 18375f00 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
18375e70 aca8107b b1bd1ac0 18376190 88796d0c d32e9750 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
183760e0 acab3b16 00000001 18376b88 00000080 1e1bc138 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
18376270 ace2ae89 183762e8 00000001 183762e8 00000001 torch_cpu!at::native::gru+0x0
18376300 ae119fa3 18376400 00000001 18376580 18376b88 torch_cpu!at::zeros_out+0x0
18376390 ace1dd35 d06d1ea8 d2c3a0d0 00000000 d06d1ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
18376410 accf88ec b1bd75c8 adf1b130 1d1720b0 18376630 torch_cpu!at::zeros_out+0x0
18376510 acd6c1bb 0000000c 00000080 00000004 0c079600 torch_cpu!at::native::mkldnn_sigmoid_+0x0
18376590 ae857294 183766b0 adf1b130 183766a0 18376848 torch_cpu!at::gru+0x0
183767c0 ae8562ab 18376b60 acc8030e 18376b60 18376b88 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
183768b0 de492621 00000007 183775c0 00000000 d06d1ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
18376ab0 de49705a 00000000 18370000 18370601 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
18376fb0 de49b950 de4cd95f de4cd95f 00000000 aedc6ac0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::then+0x0
1838fd50 de6dc6f8 d314d3b0 00000000 00000000 d30ce560 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
1838fd90 19427034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
1838fdc0 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
1838fe40 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 22756 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 312500.000000, User Time: 937500.000000, Wait Time: 6605240.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000018affb50 rsi=0000000000000000 rdi=0000000000000064
r8=0000000018affb28 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000001b3ad3f4 rsp=0000000018affb28 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
18affb20 18e5967e 18affbe8 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
18affbc0 de4ba07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
18affbf0 19427034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
18affc20 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
18affca0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 32036 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 468750.000000, User Time: 468750.000000, Wait Time: 6605240.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=0000000018bff410 rsi=0000000000000000 rdi=000000000000000d
r8=0000000018bff0b8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000064 r13=0000000018bff410
r14=0000000018bff110 r15=0000000000000000 rip=000000001b3ad8c4 rsp=0000000018bff0b8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
18bff0b0 18e5cb70 18bff470 1b32dc6b 00000000 18bff1d0 ntdll!ZwWaitForMultipleObjects+0x0
18bff3a0 18e5ca6e d16c6140 00700070 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
18bff3e0 6c10ba44 d32d3470 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
18bff620 6c1a08dd 00000000 00000004 00000000 00000000 nvcuda64!+0x0
18bff6d0 6c10b413 4ca1795d 4bee22e4 d3b702a0 00000000 nvcuda64!cuProfilerStop+0x0
18bff700 6c419200 d3b702a0 00000000 00000000 00000000 nvcuda64!+0x0
18bff730 19427034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
18bff760 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
18bff7e0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 2768 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6605240.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002b8 rdx=0000000000000000 rsi=0000000018cff638 rdi=00000000000002b8
r8=0000000018cff608 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=00000000d32b5310 r13=0000000000000000
r14=00000000000002b8 r15=0000000000000001 rip=000000001b3acdf4 rsp=0000000018cff608 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
18cff600 18e31a8e 00000000 00000000 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
18cff6a0 6c31ab97 00000000 18e3a3c5 00000000 000002b8 KERNELBASE!WaitForSingleObjectEx+0x0
18cff710 6c10b413 00000001 d3661b00 d3b70240 00000000 nvcuda64!cuProfilerStop+0x0
18cff740 6c419200 d3b70240 00000000 00000000 00000000 nvcuda64!+0x0
18cff770 19427034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
18cff7a0 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
18cff820 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 38464 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 6605239.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000d3552728 rdx=0000000018dffac8 rsi=0000000018dffa30 rdi=0000000000000000
r8=0000000018dffa08 r9=0000000018dffa60 r10=0000000000000000 r11=0000000000000246 r12=0000000018dffac8 r13=0000000000000000
r14=00000000d3552728 r15=00000000d3555a98 rip=000000001b3b0764 rsp=0000000018dffa08 rbp=0000000018dffa60
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
18dffa00 1b374295 00000102 6c1d953d 00000000 18dffa30 ntdll!NtWaitForAlertByThreadId+0x0
18dffa70 18e7ce39 d3553280 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
18dffaa0 6c0921da d3553280 d3553280 d3553280 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
18dffb00 6c10b413 00000002 4c688b99 4943896f 00000000 nvcuda64!+0x0
18dffb30 6c419200 d3b70b10 00000000 00000000 00000000 nvcuda64!+0x0
18dffb60 19427034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
18dffb90 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
18dffc10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 18900 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 383281248.000000, User Time: 4649218560.000000, Wait Time: 6605215.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=0000000004bf0428 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000004bf0428
r8=0000000018efe970 r9=0000000018efe9e0 r10=00000000ade0f1c8 r11=00000000d31cbcd0 r12=0000000000000000 r13=0000000000000000
r14=0000000018eff700 r15=0000000004bf03d8 rip=000000001b3b0764 rsp=0000000018eff6d8 rbp=0000000018eff730
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
18eff6d0 1b374021 18eff9d0 00000000 18eff7d0 00000000 ntdll!NtWaitForAlertByThreadId+0x0
18eff750 18e7cee9 04bf0420 04bf0418 04bf03d0 ae4905b4 ntdll!RtlSleepConditionVariableSRW+0x0
18eff790 ec6b2549 00000000 19d40c70 00000000 00000000 KERNELBASE!SleepConditionVariableSRW+0x0
18eff7c0 ec6b27da f1ea6744 00000000 18eff9d0 04bf0418 MSVCP140!std::_Winerror_message+0x0
18eff7f0 ae49064b 04bf03d0 18eff8f0 04bf03d0 19d40c50 MSVCP140!_Cnd_wait+0x0
18eff830 ae4925b1 1e2a2e70 ffffffff 19d40c50 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
18effb20 ae492471 b19b6d10 18effb60 b19b6d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
18effb70 ae489744 00080001 1e19a480 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
18effba0 18a51bb2 1e19a480 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
18effbd0 19427034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
18effc00 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
18effc80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 35440 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 6605221.000000
- Registers -
rax=00000000000001d3 rbx=00000000cfc0ec00 rcx=000000000000007c rdx=00000000cfc0e880 rsi=00000000d16b3bc0 rdi=00000000cfc0ec00
r8=0000000000000398 r9=0000000001010101 r10=0000000000000000 r11=0000000000000001 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000001b3b07c4 rsp=00000000186ff808 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
186ff800 1b362dc7 00000000 00000000 d16b3bc0 00000000 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
186ffb00 19427034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
186ffb30 1b362651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
186ffbb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)