Task 4866844
| Name | ParityModified-1617244792-24232-1_1 |
| Workunit | 2621772 |
| Created | 11 Apr 2021, 7:32:02 UTC |
| Sent | 11 Apr 2021, 8:10:51 UTC |
| Report deadline | 18 Apr 2021, 8:10:51 UTC |
| Received | 11 Apr 2021, 22:51:52 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 157 |
| Run time | 3 hours 57 min 55 sec |
| CPU time | 34 min 17 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 4,474.06 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.65 GB |
| Peak swap size | 3.63 GB |
| Peak disk usage | 1.53 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
: nan | Time: 29829.2 ms
[2021-04-11 14:24:42 main:574] : INFO : Epoch 410 | loss: nan | val_loss: nan | Time: 20766.7 ms
[2021-04-11 14:25:13 main:574] : INFO : Epoch 411 | loss: nan | val_loss: nan | Time: 29924 ms
[2021-04-11 14:25:43 main:574] : INFO : Epoch 412 | loss: nan | val_loss: nan | Time: 29995.3 ms
[2021-04-11 14:26:13 main:574] : INFO : Epoch 413 | loss: nan | val_loss: nan | Time: 30048.3 ms
[2021-04-11 14:26:43 main:574] : INFO : Epoch 414 | loss: nan | val_loss: nan | Time: 29975 ms
[2021-04-11 14:27:13 main:574] : INFO : Epoch 415 | loss: nan | val_loss: nan | Time: 30087.6 ms
[2021-04-11 14:27:43 main:574] : INFO : Epoch 416 | loss: nan | val_loss: nan | Time: 29857.2 ms
[2021-04-11 14:28:12 main:574] : INFO : Epoch 417 | loss: nan | val_loss: nan | Time: 29565.4 ms
[2021-04-11 14:28:42 main:574] : INFO : Epoch 418 | loss: nan | val_loss: nan | Time: 29911.5 ms
[2021-04-11 14:29:13 main:574] : INFO : Epoch 419 | loss: nan | val_loss: nan | Time: 30071.1 ms
[2021-04-11 14:29:43 main:574] : INFO : Epoch 420 | loss: nan | val_loss: nan | Time: 30027.2 ms
[2021-04-11 14:30:13 main:574] : INFO : Epoch 421 | loss: nan | val_loss: nan | Time: 29869.2 ms
[2021-04-11 14:30:34 main:574] : INFO : Epoch 422 | loss: nan | val_loss: nan | Time: 20915.6 ms
[2021-04-11 14:31:04 main:574] : INFO : Epoch 423 | loss: nan | val_loss: nan | Time: 25096 ms
[2021-04-11 14:31:34 main:574] : INFO : Epoch 424 | loss: nan | val_loss: nan | Time: 30047.8 ms
[2021-04-11 14:32:04 main:574] : INFO : Epoch 425 | loss: nan | val_loss: nan | Time: 30031.4 ms
[2021-04-11 14:32:34 main:574] : INFO : Epoch 426 | loss: nan | val_loss: nan | Time: 29839.7 ms
[2021-04-11 14:33:04 main:574] : INFO : Epoch 427 | loss: nan | val_loss: nan | Time: 29924.7 ms
[2021-04-11 14:33:33 main:574] : INFO : Epoch 428 | loss: nan | val_loss: nan | Time: 29742.9 ms
[2021-04-11 14:34:04 main:574] : INFO : Epoch 429 | loss: nan | val_loss: nan | Time: 30040.9 ms
[2021-04-11 14:34:34 main:574] : INFO : Epoch 430 | loss: nan | val_loss: nan | Time: 29967.4 ms
[2021-04-11 14:35:04 main:574] : INFO : Epoch 431 | loss: nan | val_loss: nan | Time: 30018.5 ms
[2021-04-11 14:35:34 main:574] : INFO : Epoch 432 | loss: nan | val_loss: nan | Time: 29907 ms
[2021-04-11 14:36:04 main:574] : INFO : Epoch 433 | loss: nan | val_loss: nan | Time: 29936.7 ms
[2021-04-11 14:36:34 main:574] : INFO : Epoch 434 | loss: nan | val_loss: nan | Time: 30022.2 ms
[2021-04-11 14:37:04 main:574] : INFO : Epoch 435 | loss: nan | val_loss: nan | Time: 29971.9 ms
[2021-04-11 14:37:34 main:574] : INFO : Epoch 436 | loss: nan | val_loss: nan | Time: 30000.8 ms
[2021-04-11 14:37:55 main:574] : INFO : Epoch 437 | loss: nan | val_loss: nan | Time: 20705.9 ms
[2021-04-11 14:38:25 main:574] : INFO : Epoch 438 | loss: nan | val_loss: nan | Time: 29790.5 ms
[2021-04-11 14:38:54 main:574] : INFO : Epoch 439 | loss: nan | val_loss: nan | Time: 29725.2 ms
[2021-04-11 14:39:24 main:574] : INFO : Epoch 440 | loss: nan | val_loss: nan | Time: 29896.9 ms
[2021-04-11 14:39:54 main:574] : INFO : Epoch 441 | loss: nan | val_loss: nan | Time: 30027.2 ms
[2021-04-11 14:40:24 main:574] : INFO : Epoch 442 | loss: nan | val_loss: nan | Time: 29952.9 ms
[2021-04-11 14:40:54 main:574] : INFO : Epoch 443 | loss: nan | val_loss: nan | Time: 29891.7 ms
[2021-04-11 14:41:15 main:574] : INFO : Epoch 444 | loss: nan | val_loss: nan | Time: 20836.3 ms
[2021-04-11 14:41:45 main:574] : INFO : Epoch 445 | loss: nan | val_loss: nan | Time: 29777.9 ms
[2021-04-11 14:42:15 main:574] : INFO : Epoch 446 | loss: nan | val_loss: nan | Time: 29867.5 ms
[2021-04-11 14:42:45 main:574] : INFO : Epoch 447 | loss: nan | val_loss: nan | Time: 29855.6 ms
[2021-04-11 14:43:06 main:574] : INFO : Epoch 448 | loss: nan | val_loss: nan | Time: 20983.6 ms
[2021-04-11 14:43:36 main:574] : INFO : Epoch 449 | loss: nan | val_loss: nan | Time: 24263.7 ms
[2021-04-11 14:44:05 main:574] : INFO : Epoch 450 | loss: nan | val_loss: nan | Time: 29794.9 ms
[2021-04-11 14:44:36 main:574] : INFO : Epoch 451 | loss: nan | val_loss: nan | Time: 30068.3 ms
[2021-04-11 14:44:56 main:574] : INFO : Epoch 452 | loss: nan | val_loss: nan | Time: 20799.3 ms
[2021-04-11 14:45:26 main:574] : INFO : Epoch 453 | loss: nan | val_loss: nan | Time: 24090.9 ms
[2021-04-11 14:45:56 main:574] : INFO : Epoch 454 | loss: nan | val_loss: nan | Time: 30087.6 ms
[2021-04-11 14:46:26 main:574] : INFO : Epoch 455 | loss: nan | val_loss: nan | Time: 29839.1 ms
[2021-04-11 14:46:56 main:574] : INFO : Epoch 456 | loss: nan | val_loss: nan | Time: 29952.6 ms
[2021-04-11 14:47:17 main:574] : INFO : Epoch 457 | loss: nan | val_loss: nan | Time: 20699.8 ms
[2021-04-11 14:47:47 main:574] : INFO : Epoch 458 | loss: nan | val_loss: nan | Time: 29876.2 ms
[2021-04-11 14:48:17 main:574] : INFO : Epoch 459 | loss: nan | val_loss: nan | Time: 29846.7 ms
[2021-04-11 14:48:46 main:574] : INFO : Epoch 460 | loss: nan | val_loss: nan | Time: 29783.3 ms
[2021-04-11 14:49:16 main:574] : INFO : Epoch 461 | loss: nan | val_loss: nan | Time: 29830.2 ms
[2021-04-11 14:49:37 main:574] : INFO : Epoch 462 | loss: nan | val_loss: nan | Time: 20818.4 ms
[2021-04-11 14:50:07 main:574] : INFO : Epoch 463 | loss: nan | val_loss: nan | Time: 29701.3 ms
[2021-04-11 14:50:37 main:574] : INFO : Epoch 464 | loss: nan | val_loss: nan | Time: 29775.9 ms
[2021-04-11 14:51:07 main:574] : INFO : Epoch 465 | loss: nan | val_loss: nan | Time: 29930.8 ms
[2021-04-11 14:51:28 main:574] : INFO : Epoch 466 | loss: nan | val_loss: nan | Time: 20635.5 ms
[2021-04-11 14:51:58 main:574] : INFO : Epoch 467 | loss: nan | val_loss: nan | Time: 29990.1 ms
[2021-04-11 14:52:28 main:574] : INFO : Epoch 468 | loss: nan | val_loss: nan | Time: 29909.1 ms
[2021-04-11 14:52:58 main:574] : INFO : Epoch 469 | loss: nan | val_loss: nan | Time: 30077.5 ms
[2021-04-11 14:53:28 main:574] : INFO : Epoch 470 | loss: nan | val_loss: nan | Time: 29939.7 ms
[2021-04-11 14:53:58 main:574] : INFO : Epoch 471 | loss: nan | val_loss: nan | Time: 29893.6 ms
[2021-04-11 14:54:28 main:574] : INFO : Epoch 472 | loss: nan | val_loss: nan | Time: 29834.1 ms
[2021-04-11 14:54:58 main:574] : INFO : Epoch 473 | loss: nan | val_loss: nan | Time: 29977.4 ms
[2021-04-11 14:55:28 main:574] : INFO : Epoch 474 | loss: nan | val_loss: nan | Time: 29876.9 ms
[2021-04-11 14:55:58 main:574] : INFO : Epoch 475 | loss: nan | val_loss: nan | Time: 29967.4 ms
[2021-04-11 14:56:19 main:574] : INFO : Epoch 476 | loss: nan | val_loss: nan | Time: 20740.5 ms
[2021-04-11 14:56:48 main:574] : INFO : Epoch 477 | loss: nan | val_loss: nan | Time: 24749.3 ms
[2021-04-11 14:57:14 main:574] : INFO : Epoch 478 | loss: nan | val_loss: nan | Time: 26219.2 ms
[2021-04-11 14:57:39 main:574] : INFO : Epoch 479 | loss: nan | val_loss: nan | Time: 20918.2 ms
[2021-04-11 14:58:09 main:574] : INFO : Epoch 480 | loss: nan | val_loss: nan | Time: 29834.5 ms
[2021-04-11 14:58:39 main:574] : INFO : Epoch 481 | loss: nan | val_loss: nan | Time: 29835 ms
[2021-04-11 14:59:09 main:574] : INFO : Epoch 482 | loss: nan | val_loss: nan | Time: 29681.9 ms
[2021-04-11 14:59:39 main:574] : INFO : Epoch 483 | loss: nan | val_loss: nan | Time: 30081.6 ms
[2021-04-11 15:00:09 main:574] : INFO : Epoch 484 | loss: nan | val_loss: nan | Time: 30050 ms
[2021-04-11 15:00:39 main:574] : INFO : Epoch 485 | loss: nan | val_loss: nan | Time: 29928.3 ms
[2021-04-11 15:01:00 main:574] : INFO : Epoch 486 | loss: nan | val_loss: nan | Time: 20736.4 ms
[2021-04-11 15:01:29 main:574] : INFO : Epoch 487 | loss: nan | val_loss: nan | Time: 24084.5 ms
[2021-04-11 15:01:59 main:574] : INFO : Epoch 488 | loss: nan | val_loss: nan | Time: 29942.5 ms
[2021-04-11 15:02:29 main:574] : INFO : Epoch 489 | loss: nan | val_loss: nan | Time: 30081 ms
[2021-04-11 15:02:55 main:574] : INFO : Epoch 490 | loss: nan | val_loss: nan | Time: 26114 ms
[2021-04-11 15:03:20 main:574] : INFO : Epoch 491 | loss: nan | val_loss: nan | Time: 20786.5 ms
[2021-04-11 15:03:50 main:574] : INFO : Epoch 492 | loss: nan | val_loss: nan | Time: 30036.3 ms
[2021-04-11 15:04:20 main:574] : INFO : Epoch 493 | loss: nan | val_loss: nan | Time: 30036.3 ms
[2021-04-11 15:04:50 main:574] : INFO : Epoch 494 | loss: nan | val_loss: nan | Time: 29972.2 ms
[2021-04-11 15:05:20 main:574] : INFO : Epoch 495 | loss: nan | val_loss: nan | Time: 29916.5 ms
[2021-04-11 15:05:50 main:574] : INFO : Epoch 496 | loss: nan | val_loss: nan | Time: 29719.8 ms
[2021-04-11 15:06:20 main:574] : INFO : Epoch 497 | loss: nan | val_loss: nan | Time: 29792 ms
[2021-04-11 15:06:50 main:574] : INFO : Epoch 498 | loss: nan | val_loss: nan | Time: 30000.5 ms
[2021-04-11 15:07:11 main:574] : INFO : Epoch 499 | loss: nan | val_loss: nan | Time: 20802.7 ms
[2021-04-11 15:07:40 main:574] : INFO : Epoch 500 | loss: nan | val_loss: nan | Time: 23516.2 ms
[2021-04-11 15:08:10 main:574] : INFO : Epoch 501 | loss: nan | val_loss: nan | Time: 29829.2 ms
[2021-04-11 15:08:31 main:574] : INFO : Epoch 502 | loss: nan | val_loss: nan | Time: 21036.6 ms
[2021-04-11 15:09:00 main:574] : INFO : Epoch 503 | loss: nan | val_loss: nan | Time: 24322.2 ms
[2021-04-11 15:09:30 main:574] : INFO : Epoch 504 | loss: nan | val_loss: nan | Time: 29944.8 ms
[2021-04-11 15:10:00 main:574] : INFO : Epoch 505 | loss: nan | val_loss: nan | Time: 29784.7 ms
[2021-04-11 15:10:31 main:574] : INFO : Epoch 506 | loss: nan | val_loss: nan | Time: 30739.8 ms
[2021-04-11 15:11:01 main:574] : INFO : Epoch 507 | loss: nan | val_loss: nan | Time: 29610.5 ms
[2021-04-11 15:11:31 main:574] : INFO : Epoch 508 | loss: nan | val_loss: nan | Time: 30125 ms
[2021-04-11 15:12:01 main:574] : INFO : Epoch 509 | loss: nan | val_loss: nan | Time: 29971 ms
[2021-04-11 15:12:22 main:574] : INFO : Epoch 510 | loss: nan | val_loss: nan | Time: 20759.4 ms
[2021-04-11 15:12:51 main:574] : INFO : Epoch 511 | loss: nan | val_loss: nan | Time: 23563.8 ms
[2021-04-11 15:13:21 main:574] : INFO : Epoch 512 | loss: nan | val_loss: nan | Time: 29872.5 ms
[2021-04-11 15:13:47 main:574] : INFO : Epoch 513 | loss: nan | val_loss: nan | Time: 25956.5 ms
[2021-04-11 15:14:12 main:574] : INFO : Epoch 514 | loss: nan | val_loss: nan | Time: 20975.3 ms
[2021-04-11 15:14:42 main:574] : INFO : Epoch 515 | loss: nan | val_loss: nan | Time: 24677.1 ms
[2021-04-11 15:15:11 main:574] : INFO : Epoch 516 | loss: nan | val_loss: nan | Time: 29740.2 ms
[2021-04-11 15:15:41 main:574] : INFO : Epoch 517 | loss: nan | val_loss: nan | Time: 30011.4 ms
[2021-04-11 15:16:12 main:574] : INFO : Epoch 518 | loss: nan | val_loss: nan | Time: 30031.1 ms
[2021-04-11 15:16:32 main:574] : INFO : Epoch 519 | loss: nan | val_loss: nan | Time: 20710.3 ms
[2021-04-11 15:17:02 main:574] : INFO : Epoch 520 | loss: nan | val_loss: nan | Time: 30002.1 ms
[2021-04-11 15:17:32 main:574] : INFO : Epoch 521 | loss: nan | val_loss: nan | Time: 29976.1 ms
[2021-04-11 15:18:02 main:574] : INFO : Epoch 522 | loss: nan | val_loss: nan | Time: 29946.7 ms
[2021-04-11 15:18:33 main:574] : INFO : Epoch 523 | loss: nan | val_loss: nan | Time: 30003.9 ms
[2021-04-11 15:19:02 main:574] : INFO : Epoch 524 | loss: nan | val_loss: nan | Time: 29836.8 ms
[2021-04-11 15:19:32 main:574] : INFO : Epoch 525 | loss: nan | val_loss: nan | Time: 29968.9 ms
[2021-04-11 15:20:03 main:574] : INFO : Epoch 526 | loss: nan | val_loss: nan | Time: 30055.8 ms
[2021-04-11 15:20:33 main:574] : INFO : Epoch 527 | loss: nan | val_loss: nan | Time: 29857.8 ms
[2021-04-11 15:21:03 main:574] : INFO : Epoch 528 | loss: nan | val_loss: nan | Time: 30157.5 ms
[2021-04-11 15:21:33 main:574] : INFO : Epoch 529 | loss: nan | val_loss: nan | Time: 29951.3 ms
[2021-04-11 15:22:03 main:574] : INFO : Epoch 530 | loss: nan | val_loss: nan | Time: 30075.1 ms
[2021-04-11 15:22:33 main:574] : INFO : Epoch 531 | loss: nan | val_loss: nan | Time: 29989.8 ms
[2021-04-11 15:23:03 main:574] : INFO : Epoch 532 | loss: nan | val_loss: nan | Time: 29839.9 ms
[2021-04-11 15:23:24 main:574] : INFO : Epoch 533 | loss: nan | val_loss: nan | Time: 20732 ms
[2021-04-11 15:23:53 main:574] : INFO : Epoch 534 | loss: nan | val_loss: nan | Time: 23799.8 ms
[2021-04-11 15:24:14 main:574] : INFO : Epoch 535 | loss: nan | val_loss: nan | Time: 20793.1 ms
[2021-04-11 15:24:44 main:574] : INFO : Epoch 536 | loss: nan | val_loss: nan | Time: 30044.7 ms
[2021-04-11 15:25:14 main:574] : INFO : Epoch 537 | loss: nan | val_loss: nan | Time: 30036.4 ms
[2021-04-11 15:25:44 main:574] : INFO : Epoch 538 | loss: nan | val_loss: nan | Time: 29780.1 ms
[2021-04-11 15:26:14 main:574] : INFO : Epoch 539 | loss: nan | val_loss: nan | Time: 29904.9 ms
[2021-04-11 15:26:44 main:574] : INFO : Epoch 540 | loss: nan | val_loss: nan | Time: 29750.5 ms
[2021-04-11 15:27:10 main:574] : INFO : Epoch 541 | loss: nan | val_loss: nan | Time: 26248.1 ms
[2021-04-11 15:27:34 main:574] : INFO : Epoch 542 | loss: nan | val_loss: nan | Time: 20772.1 ms
[2021-04-11 15:28:04 main:574] : INFO : Epoch 543 | loss: nan | val_loss: nan | Time: 30015.6 ms
[2021-04-11 15:28:34 main:574] : INFO : Epoch 544 | loss: nan | val_loss: nan | Time: 30035.3 ms
[2021-04-11 15:29:04 main:574] : INFO : Epoch 545 | loss: nan | val_loss: nan | Time: 29615.6 ms
[2021-04-11 15:29:34 main:574] : INFO : Epoch 546 | loss: nan | val_loss: nan | Time: 29899.9 ms
[2021-04-11 15:30:04 main:574] : INFO : Epoch 547 | loss: nan | val_loss: nan | Time: 29992.7 ms
[2021-04-11 15:30:34 main:574] : INFO : Epoch 548 | loss: nan | val_loss: nan | Time: 30017.8 ms
[2021-04-11 15:31:04 main:574] : INFO : Epoch 549 | loss: nan | val_loss: nan | Time: 29836.3 ms
[2021-04-11 15:31:25 main:574] : INFO : Epoch 550 | loss: nan | val_loss: nan | Time: 20865 ms
[2021-04-11 15:31:55 main:574] : INFO : Epoch 551 | loss: nan | val_loss: nan | Time: 25111.6 ms
[2021-04-11 15:32:25 main:574] : INFO : Epoch 552 | loss: nan | val_loss: nan | Time: 29762.9 ms
[2021-04-11 15:32:55 main:574] : INFO : Epoch 553 | loss: nan | val_loss: nan | Time: 29835.4 ms
[2021-04-11 15:33:25 main:574] : INFO : Epoch 554 | loss: nan | val_loss: nan | Time: 29975 ms
[2021-04-11 15:33:55 main:574] : INFO : Epoch 555 | loss: nan | val_loss: nan | Time: 29885.8 ms
[2021-04-11 15:34:25 main:574] : INFO : Epoch 556 | loss: nan | val_loss: nan | Time: 29821.2 ms
[2021-04-11 15:34:46 main:574] : INFO : Epoch 557 | loss: nan | val_loss: nan | Time: 20994.3 ms
[2021-04-11 15:35:15 main:574] : INFO : Epoch 558 | loss: nan | val_loss: nan | Time: 23253.7 ms
[2021-04-11 15:35:45 main:574] : INFO : Epoch 559 | loss: nan | val_loss: nan | Time: 29986.7 ms
[2021-04-11 15:36:15 main:574] : INFO : Epoch 560 | loss: nan | val_loss: nan | Time: 30028.7 ms
[2021-04-11 15:36:36 main:574] : INFO : Epoch 561 | loss: nan | val_loss: nan | Time: 20712.6 ms
[2021-04-11 15:37:06 main:574] : INFO : Epoch 562 | loss: nan | val_loss: nan | Time: 29756.9 ms
[2021-04-11 15:37:36 main:574] : INFO : Epoch 563 | loss: nan | val_loss: nan | Time: 30007.4 ms
[2021-04-11 15:38:06 main:574] : INFO : Epoch 564 | loss: nan | val_loss: nan | Time: 30018.8 ms
[2021-04-11 15:38:36 main:574] : INFO : Epoch 565 | loss: nan | val_loss: nan | Time: 29784.2 ms
[2021-04-11 15:39:06 main:574] : INFO : Epoch 566 | loss: nan | val_loss: nan | Time: 29940.9 ms
[2021-04-11 15:39:36 main:574] : INFO : Epoch 567 | loss: nan | val_loss: nan | Time: 29729.5 ms
[2021-04-11 15:40:06 main:574] : INFO : Epoch 568 | loss: nan | val_loss: nan | Time: 29977.2 ms
[2021-04-11 15:40:27 main:574] : INFO : Epoch 569 | loss: nan | val_loss: nan | Time: 20898.6 ms
[2021-04-11 15:40:56 main:574] : INFO : Epoch 570 | loss: nan | val_loss: nan | Time: 23404.5 ms
[2021-04-11 15:41:26 main:574] : INFO : Epoch 571 | loss: nan | val_loss: nan | Time: 29894.2 ms
[2021-04-11 15:41:56 main:574] : INFO : Epoch 572 | loss: nan | val_loss: nan | Time: 30102 ms
[2021-04-11 15:42:26 main:574] : INFO : Epoch 573 | loss: nan | val_loss: nan | Time: 30019.6 ms
[2021-04-11 15:42:56 main:574] : INFO : Epoch 574 | loss: nan | val_loss: nan | Time: 29933 ms
[2021-04-11 15:43:26 main:574] : INFO : Epoch 575 | loss: nan | val_loss: nan | Time: 30011.5 ms
[2021-04-11 15:43:56 main:574] : INFO : Epoch 576 | loss: nan | val_loss: nan | Time: 29934.5 ms
[2021-04-11 15:44:17 main:574] : INFO : Epoch 577 | loss: nan | val_loss: nan | Time: 20731.9 ms
[2021-04-11 15:44:47 main:574] : INFO : Epoch 578 | loss: nan | val_loss: nan | Time: 29811.8 ms
[2021-04-11 15:45:17 main:574] : INFO : Epoch 579 | loss: nan | val_loss: nan | Time: 30046.9 ms
[2021-04-11 15:45:47 main:574] : INFO : Epoch 580 | loss: nan | val_loss: nan | Time: 29729.6 ms
[2021-04-11 15:46:17 main:574] : INFO : Epoch 581 | loss: nan | val_loss: nan | Time: 29793 ms
[2021-04-11 15:46:48 main:574] : INFO : Epoch 582 | loss: nan | val_loss: nan | Time: 30722.3 ms
[2021-04-11 15:47:18 main:574] : INFO : Epoch 583 | loss: nan | val_loss: nan | Time: 29860 ms
[2021-04-11 15:47:48 main:574] : INFO : Epoch 584 | loss: nan | val_loss: nan | Time: 29956 ms
[2021-04-11 15:48:18 main:574] : INFO : Epoch 585 | loss: nan | val_loss: nan | Time: 30351.6 ms
[2021-04-11 15:48:48 main:574] : INFO : Epoch 586 | loss: nan | val_loss: nan | Time: 29845.2 ms
[2021-04-11 15:49:18 main:574] : INFO : Epoch 587 | loss: nan | val_loss: nan | Time: 29900.1 ms
[2021-04-11 15:49:48 main:574] : INFO : Epoch 588 | loss: nan | val_loss: nan | Time: 29960.1 ms
[2021-04-11 15:50:18 main:574] : INFO : Epoch 589 | loss: nan | val_loss: nan | Time: 29773.4 ms
[2021-04-11 15:50:39 main:574] : INFO : Epoch 590 | loss: nan | val_loss: nan | Time: 20886.1 ms
[2021-04-11 15:51:08 main:574] : INFO : Epoch 591 | loss: nan | val_loss: nan | Time: 23810.2 ms
[2021-04-11 15:51:38 main:574] : INFO : Epoch 592 | loss: nan | val_loss: nan | Time: 29857.2 ms
[2021-04-11 15:51:59 main:574] : INFO : Epoch 593 | loss: nan | val_loss: nan | Time: 20945.8 ms
[2021-04-11 15:52:28 main:574] : INFO : Epoch 594 | loss: nan | val_loss: nan | Time: 24544.5 ms
[2021-04-11 15:52:58 main:574] : INFO : Epoch 595 | loss: nan | val_loss: nan | Time: 29961.2 ms
[2021-04-11 15:53:19 main:574] : INFO : Epoch 596 | loss: nan | val_loss: nan | Time: 20844.8 ms
[2021-04-11 15:53:49 main:574] : INFO : Epoch 597 | loss: nan | val_loss: nan | Time: 24033 ms
[2021-04-11 15:54:19 main:574] : INFO : Epoch 598 | loss: nan | val_loss: nan | Time: 29883.8 ms
[2021-04-11 15:54:39 main:574] : INFO : Epoch 599 | loss: nan | val_loss: nan | Time: 20719.7 ms
[2021-04-11 15:55:09 main:574] : INFO : Epoch 600 | loss: nan | val_loss: nan | Time: 29811.5 ms
[2021-04-11 15:55:45 main:574] : INFO : Epoch 601 | loss: nan | val_loss: nan | Time: 35381.5 ms
[2021-04-11 15:56:09 main:574] : INFO : Epoch 602 | loss: nan | val_loss: nan | Time: 24202.1 ms
[2021-04-11 15:56:36 main:574] : INFO : Epoch 603 | loss: nan | val_loss: nan | Time: 26722.9 ms
[2021-04-11 15:56:59 main:574] : INFO : Epoch 604 | loss: nan | val_loss: nan | Time: 23283 ms
[2021-04-11 15:57:29 main:574] : INFO : Epoch 605 | loss: nan | val_loss: nan | Time: 29972.1 ms
[2021-04-11 15:57:50 main:574] : INFO : Epoch 606 | loss: nan | val_loss: nan | Time: 20920.8 ms
[2021-04-11 15:58:19 main:574] : INFO : Epoch 607 | loss: nan | val_loss: nan | Time: 24190.6 ms
[2021-04-11 15:58:49 main:574] : INFO : Epoch 608 | loss: nan | val_loss: nan | Time: 29872.4 ms
[2021-04-11 15:59:19 main:574] : INFO : Epoch 609 | loss: nan | val_loss: nan | Time: 30002.4 ms
[2021-04-11 15:59:40 main:574] : INFO : Epoch 610 | loss: nan | val_loss: nan | Time: 20819.4 ms
[2021-04-11 16:00:10 main:574] : INFO : Epoch 611 | loss: nan | val_loss: nan | Time: 29879.3 ms
[2021-04-11 16:00:40 main:574] : INFO : Epoch 612 | loss: nan | val_loss: nan | Time: 29702.4 ms
[2021-04-11 16:01:10 main:574] : INFO : Epoch 613 | loss: nan | val_loss: nan | Time: 29974.1 ms
[2021-04-11 16:01:40 main:574] : INFO : Epoch 614 | loss: nan | val_loss: nan | Time: 29898.5 ms
[2021-04-11 16:02:10 main:574] : INFO : Epoch 615 | loss: nan | val_loss: nan | Time: 29940.9 ms
[2021-04-11 16:02:31 main:574] : INFO : Epoch 616 | loss: nan | val_loss: nan | Time: 20747.5 ms
[2021-04-11 16:03:01 main:574] : INFO : Epoch 617 | loss: nan | val_loss: nan | Time: 24267.5 ms
[2021-04-11 16:03:31 main:574] : INFO : Epoch 618 | loss: nan | val_loss: nan | Time: 29845.1 ms
[2021-04-11 16:04:01 main:574] : INFO : Epoch 619 | loss: nan | val_loss: nan | Time: 29891.2 ms
[2021-04-11 16:04:31 main:574] : INFO : Epoch 620 | loss: nan | val_loss: nan | Time: 30063.1 ms
[2021-04-11 16:05:01 main:574] : INFO : Epoch 621 | loss: nan | val_loss: nan | Time: 29884.2 ms
[2021-04-11 16:05:31 main:574] : INFO : Epoch 622 | loss: nan | val_loss: nan | Time: 29994.3 ms
[2021-04-11 16:06:01 main:574] : INFO : Epoch 623 | loss: nan | val_loss: nan | Time: 29772.5 ms
[2021-04-11 16:06:22 main:574] : INFO : Epoch 624 | loss: nan | val_loss: nan | Time: 20911.9 ms
[2021-04-11 16:06:52 main:574] : INFO : Epoch 625 | loss: nan | val_loss: nan | Time: 29845.9 ms
[2021-04-11 16:07:21 main:574] : INFO : Epoch 626 | loss: nan | val_loss: nan | Time: 29722.5 ms
[2021-04-11 16:07:52 main:574] : INFO : Epoch 627 | loss: nan | val_loss: nan | Time: 30044.2 ms
[2021-04-11 16:08:21 main:574] : INFO : Epoch 628 | loss: nan | val_loss: nan | Time: 29863.4 ms
[2021-04-11 16:08:52 main:574] : INFO : Epoch 629 | loss: nan | val_loss: nan | Time: 30005.1 ms
[2021-04-11 16:09:22 main:574] : INFO : Epoch 630 | loss: nan | val_loss: nan | Time: 29888.6 ms
[2021-04-11 16:09:52 main:574] : INFO : Epoch 631 | loss: nan | val_loss: nan | Time: 29953.8 ms
[2021-04-11 16:10:22 main:574] : INFO : Epoch 632 | loss: nan | val_loss: nan | Time: 30310.3 ms
[2021-04-11 16:10:52 main:574] : INFO : Epoch 633 | loss: nan | val_loss: nan | Time: 29715.9 ms
[2021-04-11 16:11:13 main:574] : INFO : Epoch 634 | loss: nan | val_loss: nan | Time: 20855.5 ms
[2021-04-11 16:11:42 main:574] : INFO : Epoch 635 | loss: nan | val_loss: nan | Time: 24056.5 ms
[2021-04-11 16:12:12 main:574] : INFO : Epoch 636 | loss: nan | val_loss: nan | Time: 29838.7 ms
[2021-04-11 16:12:39 main:574] : INFO : Epoch 637 | loss: nan | val_loss: nan | Time: 26690.1 ms
[2021-04-11 16:13:02 main:574] : INFO : Epoch 638 | loss: nan | val_loss: nan | Time: 23397.8 ms
[2021-04-11 16:13:32 main:574] : INFO : Epoch 639 | loss: nan | val_loss: nan | Time: 30035.8 ms
[2021-04-11 16:14:02 main:574] : INFO : Epoch 640 | loss: nan | val_loss: nan | Time: 29968.9 ms
[2021-04-11 16:14:23 main:574] : INFO : Epoch 641 | loss: nan | val_loss: nan | Time: 20878.2 ms
[2021-04-11 16:14:53 main:574] : INFO : Epoch 642 | loss: nan | val_loss: nan | Time: 29871.9 ms
[2021-04-11 16:15:23 main:574] : INFO : Epoch 643 | loss: nan | val_loss: nan | Time: 30099.3 ms
[2021-04-11 16:15:53 main:574] : INFO : Epoch 644 | loss: nan | val_loss: nan | Time: 29911.3 ms
[2021-04-11 16:16:23 main:574] : INFO : Epoch 645 | loss: nan | val_loss: nan | Time: 29959.5 ms
[2021-04-11 16:16:53 main:574] : INFO : Epoch 646 | loss: nan | val_loss: nan | Time: 30008.8 ms
[2021-04-11 16:17:24 main:574] : INFO : Epoch 647 | loss: nan | val_loss: nan | Time: 30076.4 ms
[2021-04-11 16:17:54 main:574] : INFO : Epoch 648 | loss: nan | val_loss: nan | Time: 29920.1 ms
[2021-04-11 16:18:24 main:574] : INFO : Epoch 649 | loss: nan | val_loss: nan | Time: 30091.9 ms
[2021-04-11 16:18:54 main:574] : INFO : Epoch 650 | loss: nan | val_loss: nan | Time: 30055.1 ms
[2021-04-11 16:19:24 main:574] : INFO : Epoch 651 | loss: nan | val_loss: nan | Time: 29691.1 ms
[2021-04-11 16:19:54 main:574] : INFO : Epoch 652 | loss: nan | val_loss: nan | Time: 30300.8 ms
[2021-04-11 16:20:24 main:574] : INFO : Epoch 653 | loss: nan | val_loss: nan | Time: 29854.9 ms
[2021-04-11 16:20:45 main:574] : INFO : Epoch 654 | loss: nan | val_loss: nan | Time: 20813.6 ms
[2021-04-11 16:21:14 main:574] : INFO : Epoch 655 | loss: nan | val_loss: nan | Time: 23818.3 ms
[2021-04-11 16:21:44 main:574] : INFO : Epoch 656 | loss: nan | val_loss: nan | Time: 29954.4 ms
[2021-04-11 16:22:14 main:574] : INFO : Epoch 657 | loss: nan | val_loss: nan | Time: 29946 ms
[2021-04-11 16:22:41 main:574] : INFO : Epoch 658 | loss: nan | val_loss: nan | Time: 26497.8 ms
[2021-04-11 16:23:05 main:574] : INFO : Epoch 659 | loss: nan | val_loss: nan | Time: 23648.1 ms
[2021-04-11 16:23:35 main:574] : INFO : Epoch 660 | loss: nan | val_loss: nan | Time: 29982.2 ms
[2021-04-11 16:24:05 main:574] : INFO : Epoch 661 | loss: nan | val_loss: nan | Time: 29891.7 ms
[2021-04-11 16:24:35 main:574] : INFO : Epoch 662 | loss: nan | val_loss: nan | Time: 29978.1 ms
[2021-04-11 16:24:56 main:574] : INFO : Epoch 663 | loss: nan | val_loss: nan | Time: 20886.1 ms
[2021-04-11 16:25:26 main:574] : INFO : Epoch 664 | loss: nan | val_loss: nan | Time: 24727.4 ms
[2021-04-11 16:25:56 main:574] : INFO : Epoch 665 | loss: nan | val_loss: nan | Time: 25026.8 ms
[2021-04-11 16:26:26 main:574] : INFO : Epoch 666 | loss: nan | val_loss: nan | Time: 24338.2 ms
[2021-04-11 16:26:55 main:574] : INFO : Epoch 667 | loss: nan | val_loss: nan | Time: 23764.4 ms
[2021-04-11 16:27:25 main:574] : INFO : Epoch 668 | loss: nan | val_loss: nan | Time: 30134.1 ms
[2021-04-11 16:27:55 main:574] : INFO : Epoch 669 | loss: nan | val_loss: nan | Time: 29963.8 ms
[2021-04-11 16:28:25 main:574] : INFO : Epoch 670 | loss: nan | val_loss: nan | Time: 29954.5 ms
[2021-04-11 16:28:46 main:574] : INFO : Epoch 671 | loss: nan | val_loss: nan | Time: 20899.6 ms
[2021-04-11 16:29:16 main:574] : INFO : Epoch 672 | loss: nan | val_loss: nan | Time: 24302 ms
[2021-04-11 16:29:46 main:574] : INFO : Epoch 673 | loss: nan | val_loss: nan | Time: 30054.2 ms
[2021-04-11 16:30:16 main:574] : INFO : Epoch 674 | loss: nan | val_loss: nan | Time: 29947.6 ms
[2021-04-11 16:30:46 main:574] : INFO : Epoch 675 | loss: nan | val_loss: nan | Time: 29876.3 ms
[2021-04-11 16:31:07 main:574] : INFO : Epoch 676 | loss: nan | val_loss: nan | Time: 20885.4 ms
[2021-04-11 16:31:37 main:574] : INFO : Epoch 677 | loss: nan | val_loss: nan | Time: 29928.3 ms
[2021-04-11 16:32:07 main:574] : INFO : Epoch 678 | loss: nan | val_loss: nan | Time: 29994.1 ms
[2021-04-11 16:32:37 main:574] : INFO : Epoch 679 | loss: nan | val_loss: nan | Time: 29814.3 ms
[2021-04-11 16:33:06 main:574] : INFO : Epoch 680 | loss: nan | val_loss: nan | Time: 29840.4 ms
[2021-04-11 16:33:37 main:574] : INFO : Epoch 681 | loss: nan | val_loss: nan | Time: 30066.2 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1060 6GB)
[2021-04-11 16:51:16 main:435] : INFO : Set logging level to 1
[2021-04-11 16:51:16 main:441] : INFO : Running in BOINC Client mode
[2021-04-11 16:51:16 main:444] : INFO : Resolving all filenames
[2021-04-11 16:51:16 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 (exists = 1)
[2021-04-11 16:51:16 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1617244792-24232-1_1_r896183044_1 (exists = 1)
[2021-04-11 16:51:16 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1617244792-24232-1_1_r896183044_0 (exists = 0)
[2021-04-11 16:51:16 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1617244792-24232-1 (exists = 1)
[2021-04-11 16:51:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-04-11 16:51:16 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5
[2021-04-11 16:51:16 main:474] : INFO : Configuration:
[2021-04-11 16:51:16 main:475] : INFO : Model type: GRU
[2021-04-11 16:51:16 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-04-11 16:51:16 main:477] : INFO : Max Epochs: 2048
[2021-04-11 16:51:16 main:478] : INFO : Batch Size: 128
[2021-04-11 16:51:16 main:479] : INFO : Learning Rate: 0.01
[2021-04-11 16:51:16 main:480] : INFO : Patience: 10
[2021-04-11 16:51:16 main:481] : INFO : Hidden Width: 12
[2021-04-11 16:51:17 main:482] : INFO : # Recurrent Layers: 4
[2021-04-11 16:51:17 main:483] : INFO : # Backend Layers: 4
[2021-04-11 16:51:17 main:484] : INFO : # Threads: 1
[2021-04-11 16:51:17 main:486] : INFO : Preparing Dataset
[2021-04-11 16:51:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-04-11 16:51:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-04-11 16:51:26 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-04-11 16:51:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-04-11 16:51:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-04-11 16:51:27 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-04-11 16:51:27 main:494] : INFO : Creating Model
[2021-04-11 16:51:28 main:507] : INFO : Preparing config file
[2021-04-11 16:51:28 main:511] : INFO : Found checkpoint, attempting to load...
[2021-04-11 16:51:28 main:512] : INFO : Loading config
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFC7585D759
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 04/11/21 16:51:28
Install Directory : D:\Program Files\
Data Directory : D:\ProgramData
Project Symstore :
LoadLibraryA( D:\ProgramData\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( D:\ProgramData\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: D:\ProgramData\slots\4;D:\ProgramData\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000b61a0000 00000000003df000 D:\ProgramData\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000077b50000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 00000000778d0000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.804) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 0000000075830000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.804) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 0000000070b50000 00000000000f1000 C:\Program Files\Bitdefender\Bitdefender Security\atcuf\265221302066127204\atcuf64.dll (1.40.291.0) (-exported- Symbols Loaded)
Linked PDB Filename : E:\builds\ATC-DEFAULT-SOURCES\bin\x64\Release\atcuf64\atcuf64.pdb
File Version : 1.40.291.0 #0xd59eaa1
Company Name : Bitdefender S.R.L. Bucharest, ROMANIA
Product Name : Bitdefender® ATC
Product Version : 13
ModLoad: 0000000075ce0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000075290000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 0000000076790000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.746 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.746
ModLoad: 00000000756c0000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.746 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.746
ModLoad: 0000000075440000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000075340000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000767c0000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000077570000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000077990000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000075f60000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000003fc70000 0000000000058000 D:\ProgramData\slots\4\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000059c50000 0000000024b49000 D:\ProgramData\slots\4\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000ec150000 000000000abe7000 D:\ProgramData\slots\4\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000637b0000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 000000006d280000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 0000000063790000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 0000000064f40000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 0000000032b40000 0000000000045000 D:\ProgramData\slots\4\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000e90a0000 00000000030a5000 D:\ProgramData\slots\4\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000e5be0000 00000000034b2000 D:\ProgramData\slots\4\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000ca830000 000000001b3a4000 D:\ProgramData\slots\4\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 000000006ef60000 0000000000010000 D:\ProgramData\slots\4\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000bab80000 000000000919f000 D:\ProgramData\slots\4\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 00000000b6e20000 0000000003d5b000 D:\ProgramData\slots\4\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000027f30000 000000000020d000 D:\ProgramData\slots\4\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 0000000027c50000 00000000002db000 D:\ProgramData\slots\4\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000258a0000 0000000001eca000 D:\ProgramData\slots\4\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000003d750000 0000000000040000 D:\ProgramData\slots\4\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000075f30000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000073210000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000076320000 0000000000469000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000754e0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000075530000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000074ff0000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000757d0000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.804) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 0000000075560000 000000000015f000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000074e10000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000076950000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.329 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.329
ModLoad: 0000000073410000 0000000000790000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000077210000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000074c90000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000077160000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000077620000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000031c50000 000000000029d000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.2.135 driver
Product Version : 27.21.14.6140
ModLoad: 000000006f810000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000028590000 00000000015c2000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispsi.inf_amd64_5e5ddf14f45dc8f6\nvcuda64.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.2.135 driver
Product Version : 27.21.14.6140
ModLoad: 0000000077680000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000006ce10000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000074c00000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000072d50000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000069da0000 00000000006f1000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 27.21.14.6140
ModLoad: 0000000075bb0000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000752c0000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3624, Write: 306, Other 886
- I/O Transfers Counters -
Read: 6542042, Write: 7303, Other 38980
- Paged Pool Usage -
QuotaPagedPoolUsage: 3448240, QuotaPeakPagedPoolUsage: 3448240
QuotaNonPagedPoolUsage: 40792, QuotaPeakNonPagedPoolUsage: 40920
- Virtual Memory Usage -
VirtualSize: -1419317248, PeakVirtualSize: -811753472
- Pagefile Usage -
PagefileUsage: -1419317248, PeakPagefileUsage: -1419317248
- Working Set Size -
WorkingSetSize: 1169551360, PeakWorkingSetSize: 1169555456, PageFaultCount: 398445
*** Dump of thread ID 12768 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 9531250.000000, User Time: 32187500.000000, Wait Time: 10754630.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFC7585D759
- Registers -
rax=0000000059010150 rbx=00000000b6504910 rcx=0000000000000000 rdx=00000000000003ff rsi=00000000f12e6490 rdi=0000000019930520
r8=000000000000002e r9=0000000000000040 r10=0000000058fe0cc0 r11=0000000059010320 r12=000000003bcc5a30 r13=00000000f12e66a0
r14=0000000083d323a0 r15=0000000000000000 rip=000000007585d759 rsp=00000000f12e6330 rbp=00000000f12e6620
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
- Callstack -
ChildEBP RetAddr Args to Child
f12e6400 b641a884 00000040 0000004f 3bcc6200 f12e64c8 KERNELBASE!RaiseException+0x0
f12e6460 b61c1ca6 30312440 b2976f7f 30312460 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12e6510 b61c593b 00000074 f12e6778 00000800 00000001 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12e6660 b61c1a78 00000000 00000000 f12e6990 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12e6730 b61c1cfc f12e6990 00000000 00000080 f12e6c60 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12e67f0 b61cc4dd f12e6c60 f12e6c60 f12e6c60 f12e6c60 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12e6ae0 b61bb2fe f12e70f8 f12e6c60 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12ff880 b63fc6f8 002b2860 00000000 00000000 002dd5b0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12ff8c0 778e7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f12ff8f0 77ba2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f12ff970 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 3376 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 10754623.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000f14ff940 rsi=0000000000000000 rdi=0000000000000064
r8=00000000f14fdf6e r9=000000000000006e r10=0000000000000004 r11=00000000f14fe642 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000077bed294 rsp=00000000f14ff918 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f14ff910 7587d84e f14ff9d8 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0
f14ff9b0 b61da07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
f14ff9e0 778e7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f14ffa10 77ba2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f14ffa90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 6912 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 10754628.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=00000000f15ff9e0 rsi=0000000000000000 rdi=000000000000000d
r8=00000000f15ff688 r9=00000000004c8760 r10=0000000000000000 r11=0000000000000246 r12=0000000000000064 r13=00000000f15ff9e0
r14=00000000f15ff6e0 r15=0000000000000000 rip=0000000077bed764 rsp=00000000f15ff688 rbp=00000000004c8760
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f15ff680 75880d40 f15ffbb0 591f0000 70c32160 70b61487 ntdll!ZwWaitForMultipleObjects+0x0
f15ff970 75880c3e 59020770 00000000 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
f15ff9b0 28746714 004cdc80 00000000 00000000 5985ced0 KERNELBASE!WaitForMultipleObjects+0x0
f15ffbf0 287de55d 00000000 004c8760 00000000 00000000 nvcuda64!+0x0
f15ffca0 287460e3 40f9ee90 3ad63a38 00aa6df0 00000000 nvcuda64!cuProfilerStop+0x0
f15ffcd0 28a5d9d4 00aa6df0 00000000 00000000 00000000 nvcuda64!+0x0
f15ffd00 778e7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
f15ffd30 77ba2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f15ffdb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 1876 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 10754020.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002bc rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002bc
r8=00000000591fd600 r9=0000000000000082 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=00000000000002bc r15=0000000000000000 rip=0000000077becc94 rsp=00000000f16ff708 rbp=00000000004ad4b0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f16ff700 7584a34e 00aa7390 28590000 00000001 000002a8 ntdll!NtWaitForSingleObject+0x0
f16ff7a0 2895cfae 00aa7390 00000000 00000000 000002bc KERNELBASE!WaitForSingleObjectEx+0x0
f16ff810 287460e3 004ad4b0 00c05a90 00aa7390 00000000 nvcuda64!cuProfilerStop+0x0
f16ff840 28a5d9d4 00aa7390 00000000 00000000 00000000 nvcuda64!+0x0
f16ff870 778e7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
f16ff8a0 77ba2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f16ff920 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 10956 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 10754613.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=000000000074f298 rdx=00000000f17ff888 rsi=00000000f17ff7f0 rdi=0000000000000000
r8=0000000000000001 r9=0000000000002acc r10=0000000000753e70 r11=00000000f30c83f3 r12=00000000f17ff888 r13=0000000000000000
r14=000000000074f298 r15=0000000000753e70 rip=0000000077bf0604 rsp=00000000f17ff7c8 rbp=00000000f17ff820
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f17ff7c0 77bb4295 00000102 00000102 00000000 f17ff7f0 ntdll!ZwWaitForAlertByThreadId+0x0
f17ff830 7589b9d9 00751650 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
f17ff860 286c4a9c 00751650 00751650 00751650 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
f17ff8c0 287460e3 00000004 403af9c8 40d837ef 00000000 nvcuda64!+0x0
f17ff8f0 28a5d9d4 00aa6f40 00000000 00000000 00000000 nvcuda64!+0x0
f17ff920 778e7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
f17ff950 77ba2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f17ff9d0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)