Task 3656859
| Name | ParityModified-1609263483-27120-5_0 |
| Workunit | 1854668 |
| Created | 13 Jan 2021, 17:14:41 UTC |
| Sent | 14 Jan 2021, 22:43:52 UTC |
| Report deadline | 21 Jan 2021, 22:43:52 UTC |
| Received | 15 Jan 2021, 17:49:50 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 157 |
| Run time | 1 hours 26 min 3 sec |
| CPU time | 41 min 1 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 4,474.03 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.64 GB |
| Peak swap size | 3.62 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
01-15 10:23:05 main:574] : INFO : Epoch 527 | loss: 0.03118 | val_loss: 0.0311826 | Time: 6793.31 ms
[2021-01-15 10:23:12 main:574] : INFO : Epoch 528 | loss: 0.0311799 | val_loss: 0.0311823 | Time: 6952.29 ms
[2021-01-15 10:23:18 main:574] : INFO : Epoch 529 | loss: 0.0311802 | val_loss: 0.0311807 | Time: 5881.32 ms
[2021-01-15 10:23:25 main:574] : INFO : Epoch 530 | loss: 0.0311812 | val_loss: 0.0311824 | Time: 7244.35 ms
[2021-01-15 10:23:32 main:574] : INFO : Epoch 531 | loss: 0.0311808 | val_loss: 0.0311826 | Time: 6689.31 ms
[2021-01-15 10:23:38 main:574] : INFO : Epoch 532 | loss: 0.0311786 | val_loss: 0.0311799 | Time: 6094.17 ms
[2021-01-15 10:23:46 main:574] : INFO : Epoch 533 | loss: 0.0311793 | val_loss: 0.0311813 | Time: 7161.28 ms
[2021-01-15 10:23:52 main:574] : INFO : Epoch 534 | loss: 0.0311783 | val_loss: 0.0311812 | Time: 6088.1 ms
[2021-01-15 10:23:57 main:574] : INFO : Epoch 535 | loss: 0.031178 | val_loss: 0.0311797 | Time: 5318.43 ms
[2021-01-15 10:24:05 main:574] : INFO : Epoch 536 | loss: 0.0311782 | val_loss: 0.0311817 | Time: 7437.47 ms
[2021-01-15 10:24:11 main:574] : INFO : Epoch 537 | loss: 0.03118 | val_loss: 0.0311814 | Time: 6615.01 ms
[2021-01-15 10:24:18 main:574] : INFO : Epoch 538 | loss: 0.031181 | val_loss: 0.0311813 | Time: 6585.05 ms
[2021-01-15 10:24:25 main:574] : INFO : Epoch 539 | loss: 0.0311809 | val_loss: 0.0311843 | Time: 6839.47 ms
[2021-01-15 10:24:32 main:574] : INFO : Epoch 540 | loss: 0.0311802 | val_loss: 0.0311829 | Time: 6889 ms
[2021-01-15 10:24:38 main:574] : INFO : Epoch 541 | loss: 0.0311791 | val_loss: 0.0311835 | Time: 6401.53 ms
[2021-01-15 10:24:46 main:574] : INFO : Epoch 542 | loss: 0.0311785 | val_loss: 0.0311819 | Time: 7218.34 ms
[2021-01-15 10:24:53 main:574] : INFO : Epoch 543 | loss: 0.0311793 | val_loss: 0.0311852 | Time: 7112.38 ms
[2021-01-15 10:24:57 main:574] : INFO : Epoch 544 | loss: 0.031179 | val_loss: 0.0311828 | Time: 4590.28 ms
[2021-01-15 10:25:05 main:574] : INFO : Epoch 545 | loss: 0.0311788 | val_loss: 0.0311822 | Time: 7240.28 ms
[2021-01-15 10:25:11 main:574] : INFO : Epoch 546 | loss: 0.0311801 | val_loss: 0.03118 | Time: 6197.4 ms
[2021-01-15 10:25:19 main:574] : INFO : Epoch 547 | loss: 0.031178 | val_loss: 0.0311797 | Time: 7901.49 ms
[2021-01-15 10:25:25 main:574] : INFO : Epoch 548 | loss: 0.0311783 | val_loss: 0.0311858 | Time: 6237.03 ms
[2021-01-15 10:25:33 main:574] : INFO : Epoch 549 | loss: 0.0311804 | val_loss: 0.0311817 | Time: 7450.19 ms
[2021-01-15 10:25:40 main:574] : INFO : Epoch 550 | loss: 0.0311791 | val_loss: 0.03118 | Time: 6568.11 ms
[2021-01-15 10:25:46 main:574] : INFO : Epoch 551 | loss: 0.0311788 | val_loss: 0.0311823 | Time: 6094.59 ms
[2021-01-15 10:25:53 main:574] : INFO : Epoch 552 | loss: 0.0311791 | val_loss: 0.0311813 | Time: 7112.28 ms
[2021-01-15 10:25:59 main:574] : INFO : Epoch 553 | loss: 0.0311781 | val_loss: 0.0311794 | Time: 6492.58 ms
[2021-01-15 10:26:05 main:574] : INFO : Epoch 554 | loss: 0.0311777 | val_loss: 0.0311786 | Time: 5064.93 ms
[2021-01-15 10:26:12 main:574] : INFO : Epoch 555 | loss: 0.0311775 | val_loss: 0.0311806 | Time: 7157.59 ms
[2021-01-15 10:26:19 main:574] : INFO : Epoch 556 | loss: 0.0311767 | val_loss: 0.0311802 | Time: 6819.14 ms
[2021-01-15 10:26:25 main:574] : INFO : Epoch 557 | loss: 0.0311767 | val_loss: 0.0311791 | Time: 6601.33 ms
[2021-01-15 10:26:32 main:574] : INFO : Epoch 558 | loss: 0.0311764 | val_loss: 0.0311782 | Time: 6784.18 ms
[2021-01-15 10:26:38 main:574] : INFO : Epoch 559 | loss: 0.0311766 | val_loss: 0.0311787 | Time: 5949.89 ms
[2021-01-15 10:26:46 main:574] : INFO : Epoch 560 | loss: 0.031177 | val_loss: 0.0311871 | Time: 7131.88 ms
[2021-01-15 10:26:53 main:574] : INFO : Epoch 561 | loss: 0.0311799 | val_loss: 0.0311825 | Time: 7233.74 ms
[2021-01-15 10:27:00 main:574] : INFO : Epoch 562 | loss: 0.0311804 | val_loss: 0.0311829 | Time: 6682.75 ms
[2021-01-15 10:27:05 main:574] : INFO : Epoch 563 | loss: 0.03118 | val_loss: 0.0311884 | Time: 5762.82 ms
[2021-01-15 10:27:12 main:574] : INFO : Epoch 564 | loss: 0.0311806 | val_loss: 0.0311827 | Time: 6193.11 ms
[2021-01-15 10:27:18 main:574] : INFO : Epoch 565 | loss: 0.0311786 | val_loss: 0.0311833 | Time: 6268.52 ms
[2021-01-15 10:27:26 main:574] : INFO : Epoch 566 | loss: 0.0311786 | val_loss: 0.0311842 | Time: 7576.73 ms
[2021-01-15 10:27:32 main:574] : INFO : Epoch 567 | loss: 0.0311796 | val_loss: 0.031183 | Time: 6301.84 ms
[2021-01-15 10:27:39 main:574] : INFO : Epoch 568 | loss: 0.0311784 | val_loss: 0.0311827 | Time: 6512.44 ms
[2021-01-15 10:27:45 main:574] : INFO : Epoch 569 | loss: 0.0311781 | val_loss: 0.0311828 | Time: 6748.79 ms
[2021-01-15 10:27:52 main:574] : INFO : Epoch 570 | loss: 0.031178 | val_loss: 0.0311812 | Time: 6675.94 ms
[2021-01-15 10:27:59 main:574] : INFO : Epoch 571 | loss: 0.0311778 | val_loss: 0.031182 | Time: 6402.52 ms
[2021-01-15 10:28:06 main:574] : INFO : Epoch 572 | loss: 0.031178 | val_loss: 0.0311816 | Time: 7285.69 ms
[2021-01-15 10:28:11 main:574] : INFO : Epoch 573 | loss: 0.0311792 | val_loss: 0.0311804 | Time: 4767.94 ms
[2021-01-15 10:28:16 main:574] : INFO : Epoch 574 | loss: 0.031181 | val_loss: 0.0311805 | Time: 5536.19 ms
[2021-01-15 10:28:25 main:574] : INFO : Epoch 575 | loss: 0.0311785 | val_loss: 0.0311801 | Time: 8429.55 ms
[2021-01-15 10:28:32 main:574] : INFO : Epoch 576 | loss: 0.0311782 | val_loss: 0.0311799 | Time: 7202.74 ms
[2021-01-15 10:28:39 main:574] : INFO : Epoch 577 | loss: 0.0311778 | val_loss: 0.0311801 | Time: 6641.18 ms
[2021-01-15 10:28:46 main:574] : INFO : Epoch 578 | loss: 0.0311772 | val_loss: 0.031179 | Time: 7167.4 ms
[2021-01-15 10:28:53 main:574] : INFO : Epoch 579 | loss: 0.0311767 | val_loss: 0.0311792 | Time: 6863.03 ms
[2021-01-15 10:29:02 main:574] : INFO : Epoch 580 | loss: 0.0311776 | val_loss: 0.0311825 | Time: 8407.48 ms
[2021-01-15 10:29:08 main:574] : INFO : Epoch 581 | loss: 0.0311814 | val_loss: 0.0311827 | Time: 6077.44 ms
[2021-01-15 10:29:14 main:574] : INFO : Epoch 582 | loss: 0.0311793 | val_loss: 0.0311822 | Time: 6199.25 ms
[2021-01-15 10:29:20 main:574] : INFO : Epoch 583 | loss: 0.031179 | val_loss: 0.0311827 | Time: 6362.43 ms
[2021-01-15 10:29:27 main:574] : INFO : Epoch 584 | loss: 0.0311792 | val_loss: 0.0311815 | Time: 6328.87 ms
[2021-01-15 10:29:35 main:574] : INFO : Epoch 585 | loss: 0.0311773 | val_loss: 0.0311824 | Time: 7757.54 ms
[2021-01-15 10:29:41 main:574] : INFO : Epoch 586 | loss: 0.0311766 | val_loss: 0.0311796 | Time: 6446.48 ms
[2021-01-15 10:29:48 main:574] : INFO : Epoch 587 | loss: 0.0311765 | val_loss: 0.0311803 | Time: 6778.05 ms
[2021-01-15 10:29:55 main:574] : INFO : Epoch 588 | loss: 0.031176 | val_loss: 0.0311806 | Time: 7132.56 ms
[2021-01-15 10:30:02 main:574] : INFO : Epoch 589 | loss: 0.0311763 | val_loss: 0.0311791 | Time: 6706.01 ms
[2021-01-15 10:30:09 main:574] : INFO : Epoch 590 | loss: 0.0311766 | val_loss: 0.0311804 | Time: 6716.61 ms
[2021-01-15 10:30:15 main:574] : INFO : Epoch 591 | loss: 0.0311758 | val_loss: 0.0311795 | Time: 6412.35 ms
[2021-01-15 10:30:21 main:574] : INFO : Epoch 592 | loss: 0.031177 | val_loss: 0.0311809 | Time: 5705.55 ms
[2021-01-15 10:30:27 main:574] : INFO : Epoch 593 | loss: 0.0311765 | val_loss: 0.0311849 | Time: 6015.88 ms
[2021-01-15 10:30:35 main:574] : INFO : Epoch 594 | loss: 0.0311794 | val_loss: 0.0311774 | Time: 7769.35 ms
[2021-01-15 10:30:41 main:574] : INFO : Epoch 595 | loss: 0.031178 | val_loss: 0.0311782 | Time: 6089.47 ms
[2021-01-15 10:30:49 main:574] : INFO : Epoch 596 | loss: 0.0311783 | val_loss: 0.0311808 | Time: 7153.81 ms
[2021-01-15 10:30:56 main:574] : INFO : Epoch 597 | loss: 0.031178 | val_loss: 0.0311804 | Time: 7493.46 ms
[2021-01-15 10:31:04 main:574] : INFO : Epoch 598 | loss: 0.0311784 | val_loss: 0.0311811 | Time: 8037.95 ms
[2021-01-15 10:31:11 main:574] : INFO : Epoch 599 | loss: 0.0311791 | val_loss: 0.0311803 | Time: 6952.6 ms
[2021-01-15 10:31:17 main:574] : INFO : Epoch 600 | loss: 0.0311817 | val_loss: 0.0311804 | Time: 5571 ms
[2021-01-15 10:31:23 main:574] : INFO : Epoch 601 | loss: 0.0311813 | val_loss: 0.0311808 | Time: 6181.62 ms
[2021-01-15 10:31:30 main:574] : INFO : Epoch 602 | loss: 0.0311807 | val_loss: 0.031183 | Time: 6504.61 ms
[2021-01-15 10:31:36 main:574] : INFO : Epoch 603 | loss: 0.0311817 | val_loss: 0.031181 | Time: 6288.31 ms
[2021-01-15 10:31:44 main:574] : INFO : Epoch 604 | loss: 0.0311803 | val_loss: 0.0311806 | Time: 7745.38 ms
[2021-01-15 10:31:52 main:574] : INFO : Epoch 605 | loss: 0.0311801 | val_loss: 0.0311809 | Time: 7708.09 ms
[2021-01-15 10:31:59 main:574] : INFO : Epoch 606 | loss: 0.0311804 | val_loss: 0.0311804 | Time: 6821.89 ms
[2021-01-15 10:32:06 main:574] : INFO : Epoch 607 | loss: 0.0311809 | val_loss: 0.0311816 | Time: 7547.06 ms
[2021-01-15 10:32:13 main:574] : INFO : Epoch 608 | loss: 0.0311814 | val_loss: 0.0311862 | Time: 6759.37 ms
[2021-01-15 10:32:20 main:574] : INFO : Epoch 609 | loss: 0.0311817 | val_loss: 0.0311807 | Time: 6394.72 ms
[2021-01-15 10:32:26 main:574] : INFO : Epoch 610 | loss: 0.0311809 | val_loss: 0.0311853 | Time: 5830.9 ms
[2021-01-15 10:32:32 main:574] : INFO : Epoch 611 | loss: 0.0311788 | val_loss: 0.0311812 | Time: 6507.21 ms
[2021-01-15 10:32:39 main:574] : INFO : Epoch 612 | loss: 0.0311799 | val_loss: 0.03118 | Time: 6811.22 ms
[2021-01-15 10:32:46 main:574] : INFO : Epoch 613 | loss: 0.0311776 | val_loss: 0.0311802 | Time: 6860.45 ms
[2021-01-15 10:32:52 main:574] : INFO : Epoch 614 | loss: 0.0311781 | val_loss: 0.031181 | Time: 6322.92 ms
[2021-01-15 10:33:00 main:574] : INFO : Epoch 615 | loss: 0.0311775 | val_loss: 0.0311804 | Time: 7034.26 ms
[2021-01-15 10:33:06 main:574] : INFO : Epoch 616 | loss: 0.031177 | val_loss: 0.0311814 | Time: 6860.75 ms
[2021-01-15 10:33:14 main:574] : INFO : Epoch 617 | loss: 0.0311786 | val_loss: 0.0311794 | Time: 6992.38 ms
[2021-01-15 10:33:20 main:574] : INFO : Epoch 618 | loss: 0.0311769 | val_loss: 0.0311802 | Time: 6718.56 ms
[2021-01-15 10:33:27 main:574] : INFO : Epoch 619 | loss: 0.0311775 | val_loss: 0.0311799 | Time: 6197.38 ms
[2021-01-15 10:33:33 main:574] : INFO : Epoch 620 | loss: 0.0311772 | val_loss: 0.0311816 | Time: 5767.35 ms
[2021-01-15 10:33:39 main:574] : INFO : Epoch 621 | loss: 0.0311769 | val_loss: 0.0311778 | Time: 6520.39 ms
[2021-01-15 10:33:47 main:574] : INFO : Epoch 622 | loss: 0.0311776 | val_loss: 0.0311848 | Time: 7400.37 ms
[2021-01-15 10:33:53 main:574] : INFO : Epoch 623 | loss: 0.0311797 | val_loss: 0.031179 | Time: 6484.19 ms
[2021-01-15 10:33:59 main:574] : INFO : Epoch 624 | loss: 0.0311777 | val_loss: 0.0311803 | Time: 6122.07 ms
[2021-01-15 10:34:06 main:574] : INFO : Epoch 625 | loss: 0.0311758 | val_loss: 0.0311774 | Time: 6845.77 ms
[2021-01-15 10:34:13 main:574] : INFO : Epoch 626 | loss: 0.0311752 | val_loss: 0.0311805 | Time: 6695.5 ms
[2021-01-15 10:34:20 main:574] : INFO : Epoch 627 | loss: 0.0311764 | val_loss: 0.0311795 | Time: 6434.04 ms
[2021-01-15 10:34:27 main:574] : INFO : Epoch 628 | loss: 0.031174 | val_loss: 0.0311807 | Time: 7325.17 ms
[2021-01-15 10:34:32 main:574] : INFO : Epoch 629 | loss: 0.0311759 | val_loss: 0.0311797 | Time: 5075.67 ms
[2021-01-15 10:34:38 main:574] : INFO : Epoch 630 | loss: 0.0311761 | val_loss: 0.0311839 | Time: 5545.15 ms
[2021-01-15 10:34:45 main:574] : INFO : Epoch 631 | loss: 0.0311784 | val_loss: 0.0311814 | Time: 7453.73 ms
[2021-01-15 10:34:52 main:574] : INFO : Epoch 632 | loss: 0.0311775 | val_loss: 0.0311808 | Time: 6440.33 ms
[2021-01-15 10:34:58 main:574] : INFO : Epoch 633 | loss: 0.0311765 | val_loss: 0.031179 | Time: 6090.59 ms
[2021-01-15 10:35:06 main:574] : INFO : Epoch 634 | loss: 0.0311798 | val_loss: 0.0311851 | Time: 7432.72 ms
[2021-01-15 10:35:12 main:574] : INFO : Epoch 635 | loss: 0.0311869 | val_loss: 0.0311845 | Time: 6007.51 ms
[2021-01-15 10:35:18 main:574] : INFO : Epoch 636 | loss: 0.0311847 | val_loss: 0.0311808 | Time: 6236.17 ms
[2021-01-15 10:35:26 main:574] : INFO : Epoch 637 | loss: 0.0311822 | val_loss: 0.0311826 | Time: 7711.96 ms
[2021-01-15 10:35:32 main:574] : INFO : Epoch 638 | loss: 0.0311799 | val_loss: 0.0311791 | Time: 6345.48 ms
[2021-01-15 10:35:37 main:574] : INFO : Epoch 639 | loss: 0.031179 | val_loss: 0.0311788 | Time: 4772.47 ms
[2021-01-15 10:35:45 main:574] : INFO : Epoch 640 | loss: 0.0311784 | val_loss: 0.0311787 | Time: 7471.32 ms
[2021-01-15 10:35:51 main:574] : INFO : Epoch 641 | loss: 0.0311784 | val_loss: 0.0311798 | Time: 6340.19 ms
[2021-01-15 10:35:57 main:574] : INFO : Epoch 642 | loss: 0.0311791 | val_loss: 0.031179 | Time: 5748.96 ms
[2021-01-15 10:36:05 main:574] : INFO : Epoch 643 | loss: 0.0311782 | val_loss: 0.0311837 | Time: 7806.74 ms
[2021-01-15 10:36:11 main:574] : INFO : Epoch 644 | loss: 0.0311786 | val_loss: 0.0311784 | Time: 6211.65 ms
[2021-01-15 10:36:17 main:574] : INFO : Epoch 645 | loss: 0.0311776 | val_loss: 0.0311781 | Time: 6210.97 ms
[2021-01-15 10:36:25 main:574] : INFO : Epoch 646 | loss: 0.0311758 | val_loss: 0.0311774 | Time: 8063.6 ms
[2021-01-15 10:36:32 main:574] : INFO : Epoch 647 | loss: 0.031174 | val_loss: 0.0311781 | Time: 6066.82 ms
[2021-01-15 10:36:36 main:574] : INFO : Epoch 648 | loss: 0.0311741 | val_loss: 0.0311785 | Time: 4777.4 ms
[2021-01-15 10:36:44 main:574] : INFO : Epoch 649 | loss: 0.0311744 | val_loss: 0.0311798 | Time: 7318.34 ms
[2021-01-15 10:36:50 main:574] : INFO : Epoch 650 | loss: 0.0311815 | val_loss: 0.0311819 | Time: 6186.83 ms
[2021-01-15 10:36:57 main:574] : INFO : Epoch 651 | loss: 0.0311818 | val_loss: 0.0311845 | Time: 6828.81 ms
[2021-01-15 10:37:04 main:574] : INFO : Epoch 652 | loss: 0.0311815 | val_loss: 0.03118 | Time: 7335.01 ms
[2021-01-15 10:37:11 main:574] : INFO : Epoch 653 | loss: 0.0311798 | val_loss: 0.0311844 | Time: 6238.93 ms
[2021-01-15 10:37:17 main:574] : INFO : Epoch 654 | loss: 0.0311787 | val_loss: 0.0311793 | Time: 5815.37 ms
[2021-01-15 10:37:25 main:574] : INFO : Epoch 655 | loss: 0.0311777 | val_loss: 0.0311816 | Time: 8015.1 ms
[2021-01-15 10:37:32 main:574] : INFO : Epoch 656 | loss: 0.0311776 | val_loss: 0.0311836 | Time: 6758.08 ms
[2021-01-15 10:37:37 main:574] : INFO : Epoch 657 | loss: 0.0311769 | val_loss: 0.0311785 | Time: 5658.15 ms
[2021-01-15 10:37:43 main:574] : INFO : Epoch 658 | loss: 0.0311771 | val_loss: 0.0311799 | Time: 6040.17 ms
[2021-01-15 10:37:50 main:574] : INFO : Epoch 659 | loss: 0.0311759 | val_loss: 0.031178 | Time: 6356.23 ms
[2021-01-15 10:37:57 main:574] : INFO : Epoch 660 | loss: 0.0311757 | val_loss: 0.0311792 | Time: 7354.99 ms
[2021-01-15 10:38:04 main:574] : INFO : Epoch 661 | loss: 0.0311751 | val_loss: 0.0311791 | Time: 7152.18 ms
[2021-01-15 10:38:11 main:574] : INFO : Epoch 662 | loss: 0.0311747 | val_loss: 0.0311791 | Time: 6252.37 ms
[2021-01-15 10:38:17 main:574] : INFO : Epoch 663 | loss: 0.0311743 | val_loss: 0.0311804 | Time: 6217.11 ms
[2021-01-15 10:38:25 main:574] : INFO : Epoch 664 | loss: 0.0311753 | val_loss: 0.0311787 | Time: 7797.24 ms
[2021-01-15 10:38:31 main:574] : INFO : Epoch 665 | loss: 0.0311755 | val_loss: 0.0311811 | Time: 6191.88 ms
[2021-01-15 10:38:38 main:574] : INFO : Epoch 666 | loss: 0.0311769 | val_loss: 0.0311804 | Time: 6434.43 ms
[2021-01-15 10:38:43 main:574] : INFO : Epoch 667 | loss: 0.0311758 | val_loss: 0.0311809 | Time: 5117.96 ms
[2021-01-15 10:38:48 main:574] : INFO : Epoch 668 | loss: 0.0311765 | val_loss: 0.0311787 | Time: 5391.99 ms
[2021-01-15 10:38:56 main:574] : INFO : Epoch 669 | loss: 0.0311757 | val_loss: 0.0311783 | Time: 7834.72 ms
[2021-01-15 10:39:02 main:574] : INFO : Epoch 670 | loss: 0.0311744 | val_loss: 0.0311787 | Time: 6001.77 ms
[2021-01-15 10:39:09 main:574] : INFO : Epoch 671 | loss: 0.0311748 | val_loss: 0.0311788 | Time: 6192.96 ms
[2021-01-15 10:39:17 main:574] : INFO : Epoch 672 | loss: 0.0311757 | val_loss: 0.031177 | Time: 8105.52 ms
[2021-01-15 10:39:24 main:574] : INFO : Epoch 673 | loss: 0.0311744 | val_loss: 0.0311776 | Time: 6782.95 ms
[2021-01-15 10:39:30 main:574] : INFO : Epoch 674 | loss: 0.0311763 | val_loss: 0.0311898 | Time: 6116.85 ms
[2021-01-15 10:39:37 main:574] : INFO : Epoch 675 | loss: 0.0311811 | val_loss: 0.031181 | Time: 6686.51 ms
[2021-01-15 10:39:44 main:574] : INFO : Epoch 676 | loss: 0.0311795 | val_loss: 0.0311825 | Time: 7276.07 ms
[2021-01-15 10:39:49 main:574] : INFO : Epoch 677 | loss: 0.0311788 | val_loss: 0.0311816 | Time: 4487.91 ms
[2021-01-15 10:39:56 main:574] : INFO : Epoch 678 | loss: 0.0311789 | val_loss: 0.0311813 | Time: 7799.04 ms
[2021-01-15 10:40:03 main:574] : INFO : Epoch 679 | loss: 0.0311787 | val_loss: 0.0311806 | Time: 6328.69 ms
[2021-01-15 10:40:09 main:574] : INFO : Epoch 680 | loss: 0.0311785 | val_loss: 0.0311802 | Time: 6443.47 ms
[2021-01-15 10:40:17 main:574] : INFO : Epoch 681 | loss: 0.0311776 | val_loss: 0.0311843 | Time: 7278.42 ms
[2021-01-15 10:40:24 main:574] : INFO : Epoch 682 | loss: 0.0311788 | val_loss: 0.0311791 | Time: 6671.65 ms
[2021-01-15 10:40:30 main:574] : INFO : Epoch 683 | loss: 0.0311779 | val_loss: 0.0311802 | Time: 6353.3 ms
[2021-01-15 10:40:37 main:574] : INFO : Epoch 684 | loss: 0.0311767 | val_loss: 0.031179 | Time: 7126.47 ms
[2021-01-15 10:40:44 main:574] : INFO : Epoch 685 | loss: 0.0311769 | val_loss: 0.031179 | Time: 6494.54 ms
[2021-01-15 10:40:49 main:574] : INFO : Epoch 686 | loss: 0.0311778 | val_loss: 0.0311791 | Time: 5133.73 ms
[2021-01-15 10:40:55 main:574] : INFO : Epoch 687 | loss: 0.0311751 | val_loss: 0.0311823 | Time: 6003.08 ms
[2021-01-15 10:41:03 main:574] : INFO : Epoch 688 | loss: 0.0311806 | val_loss: 0.0311813 | Time: 7755.5 ms
[2021-01-15 10:41:09 main:574] : INFO : Epoch 689 | loss: 0.0311819 | val_loss: 0.0311838 | Time: 6484.97 ms
[2021-01-15 10:41:17 main:574] : INFO : Epoch 690 | loss: 0.0311798 | val_loss: 0.0311825 | Time: 7272.96 ms
[2021-01-15 10:41:24 main:574] : INFO : Epoch 691 | loss: 0.0311811 | val_loss: 0.0311798 | Time: 7218.74 ms
[2021-01-15 10:41:31 main:574] : INFO : Epoch 692 | loss: 0.0311804 | val_loss: 0.0311796 | Time: 6345.05 ms
[2021-01-15 10:41:37 main:574] : INFO : Epoch 693 | loss: 0.0311812 | val_loss: 0.0311816 | Time: 6664.8 ms
[2021-01-15 10:41:45 main:574] : INFO : Epoch 694 | loss: 0.0311804 | val_loss: 0.0311826 | Time: 7221.66 ms
[2021-01-15 10:41:51 main:574] : INFO : Epoch 695 | loss: 0.0311814 | val_loss: 0.0311804 | Time: 6567.08 ms
[2021-01-15 10:41:57 main:574] : INFO : Epoch 696 | loss: 0.0311797 | val_loss: 0.0311813 | Time: 5331.48 ms
[2021-01-15 10:42:04 main:574] : INFO : Epoch 697 | loss: 0.0311791 | val_loss: 0.0311823 | Time: 6941.26 ms
[2021-01-15 10:42:10 main:574] : INFO : Epoch 698 | loss: 0.0311796 | val_loss: 0.0311836 | Time: 6660.43 ms
[2021-01-15 10:42:17 main:574] : INFO : Epoch 699 | loss: 0.0311791 | val_loss: 0.0311806 | Time: 6945.58 ms
[2021-01-15 10:42:24 main:574] : INFO : Epoch 700 | loss: 0.0311781 | val_loss: 0.0311823 | Time: 6393.85 ms
[2021-01-15 10:42:30 main:574] : INFO : Epoch 701 | loss: 0.0311779 | val_loss: 0.0311799 | Time: 6320.06 ms
[2021-01-15 10:42:37 main:574] : INFO : Epoch 702 | loss: 0.0311777 | val_loss: 0.031181 | Time: 6491.44 ms
[2021-01-15 10:42:45 main:574] : INFO : Epoch 703 | loss: 0.0311773 | val_loss: 0.031179 | Time: 7991.42 ms
[2021-01-15 10:42:51 main:574] : INFO : Epoch 704 | loss: 0.0311778 | val_loss: 0.0311808 | Time: 6421.67 ms
[2021-01-15 10:42:57 main:574] : INFO : Epoch 705 | loss: 0.0311777 | val_loss: 0.0311821 | Time: 5403.31 ms
[2021-01-15 10:43:03 main:574] : INFO : Epoch 706 | loss: 0.0311792 | val_loss: 0.0311831 | Time: 6407.86 ms
[2021-01-15 10:43:10 main:574] : INFO : Epoch 707 | loss: 0.0311773 | val_loss: 0.0311822 | Time: 6336.83 ms
[2021-01-15 10:43:17 main:574] : INFO : Epoch 708 | loss: 0.031177 | val_loss: 0.0311858 | Time: 7446.81 ms
[2021-01-15 10:43:25 main:574] : INFO : Epoch 709 | loss: 0.0311783 | val_loss: 0.0311809 | Time: 7463.66 ms
[2021-01-15 10:43:31 main:574] : INFO : Epoch 710 | loss: 0.0311778 | val_loss: 0.0311816 | Time: 6245.99 ms
[2021-01-15 10:43:38 main:574] : INFO : Epoch 711 | loss: 0.0311772 | val_loss: 0.0311821 | Time: 6568.13 ms
[2021-01-15 10:43:45 main:574] : INFO : Epoch 712 | loss: 0.031181 | val_loss: 0.0311852 | Time: 7285.56 ms
[2021-01-15 10:43:52 main:574] : INFO : Epoch 713 | loss: 0.0311907 | val_loss: 0.0311869 | Time: 6957.73 ms
[2021-01-15 10:43:59 main:574] : INFO : Epoch 714 | loss: 0.0311893 | val_loss: 0.0311868 | Time: 6888.69 ms
[2021-01-15 10:44:05 main:574] : INFO : Epoch 715 | loss: 0.0311883 | val_loss: 0.0311861 | Time: 6162.86 ms
[2021-01-15 10:44:12 main:574] : INFO : Epoch 716 | loss: 0.0311875 | val_loss: 0.0311856 | Time: 6514.55 ms
[2021-01-15 10:44:18 main:574] : INFO : Epoch 717 | loss: 0.0311879 | val_loss: 0.0311862 | Time: 6025.69 ms
[2021-01-15 10:44:27 main:574] : INFO : Epoch 718 | loss: 0.0311874 | val_loss: 0.0311868 | Time: 8829.01 ms
[2021-01-15 10:44:33 main:574] : INFO : Epoch 719 | loss: 0.0311874 | val_loss: 0.031185 | Time: 6302.63 ms
[2021-01-15 10:44:40 main:574] : INFO : Epoch 720 | loss: 0.0311877 | val_loss: 0.0311898 | Time: 6406.93 ms
[2021-01-15 10:44:47 main:574] : INFO : Epoch 721 | loss: 0.0311873 | val_loss: 0.0311881 | Time: 7140.4 ms
[2021-01-15 10:44:54 main:574] : INFO : Epoch 722 | loss: 0.031187 | val_loss: 0.0311867 | Time: 7030.77 ms
[2021-01-15 10:45:01 main:574] : INFO : Epoch 723 | loss: 0.0311867 | val_loss: 0.0311847 | Time: 6514.15 ms
[2021-01-15 10:45:07 main:574] : INFO : Epoch 724 | loss: 0.0311873 | val_loss: 0.0311849 | Time: 5673.06 ms
[2021-01-15 10:45:13 main:574] : INFO : Epoch 725 | loss: 0.0311864 | val_loss: 0.0311855 | Time: 6444.4 ms
[2021-01-15 10:45:20 main:574] : INFO : Epoch 726 | loss: 0.0311858 | val_loss: 0.0311852 | Time: 6852.93 ms
[2021-01-15 10:45:27 main:574] : INFO : Epoch 727 | loss: 0.0311864 | val_loss: 0.0311889 | Time: 7186.08 ms
[2021-01-15 10:45:34 main:574] : INFO : Epoch 728 | loss: 0.0311857 | val_loss: 0.031185 | Time: 6867.3 ms
[2021-01-15 10:45:41 main:574] : INFO : Epoch 729 | loss: 0.0311856 | val_loss: 0.0311853 | Time: 6276.13 ms
[2021-01-15 10:45:48 main:574] : INFO : Epoch 730 | loss: 0.0311861 | val_loss: 0.0311853 | Time: 6905.64 ms
[2021-01-15 10:45:55 main:574] : INFO : Epoch 731 | loss: 0.0311849 | val_loss: 0.0311859 | Time: 7073.92 ms
[2021-01-15 10:46:03 main:574] : INFO : Epoch 732 | loss: 0.0311849 | val_loss: 0.0311853 | Time: 7547.49 ms
[2021-01-15 10:46:08 main:574] : INFO : Epoch 733 | loss: 0.0311848 | val_loss: 0.0311881 | Time: 5662.26 ms
[2021-01-15 10:46:16 main:574] : INFO : Epoch 734 | loss: 0.031185 | val_loss: 0.0311857 | Time: 7169.84 ms
[2021-01-15 10:46:22 main:574] : INFO : Epoch 735 | loss: 0.0311844 | val_loss: 0.0311854 | Time: 5819.15 ms
[2021-01-15 10:46:28 main:574] : INFO : Epoch 736 | loss: 0.0311855 | val_loss: 0.0311881 | Time: 6350.67 ms
[2021-01-15 10:46:36 main:574] : INFO : Epoch 737 | loss: 0.0311874 | val_loss: 0.0311853 | Time: 7627.54 ms
[2021-01-15 10:46:44 main:574] : INFO : Epoch 738 | loss: 0.0311877 | val_loss: 0.0311893 | Time: 8131.21 ms
[2021-01-15 10:46:51 main:574] : INFO : Epoch 739 | loss: 0.0311897 | val_loss: 0.0311861 | Time: 6621.13 ms
[2021-01-15 10:46:59 main:574] : INFO : Epoch 740 | loss: 0.0311889 | val_loss: 0.0311831 | Time: 7975.12 ms
[2021-01-15 10:47:06 main:574] : INFO : Epoch 741 | loss: 0.0311861 | val_loss: 0.0311831 | Time: 7558.81 ms
[2021-01-15 10:47:12 main:574] : INFO : Epoch 742 | loss: 0.0311864 | val_loss: 0.0311834 | Time: 5711.39 ms
[2021-01-15 10:47:18 main:574] : INFO : Epoch 743 | loss: 0.0311855 | val_loss: 0.031184 | Time: 5820.04 ms
[2021-01-15 10:47:25 main:574] : INFO : Epoch 744 | loss: 0.0311841 | val_loss: 0.0311826 | Time: 7133.68 ms
[2021-01-15 10:47:32 main:574] : INFO : Epoch 745 | loss: 0.031185 | val_loss: 0.0311839 | Time: 6298.98 ms
[2021-01-15 10:47:38 main:574] : INFO : Epoch 746 | loss: 0.0311856 | val_loss: 0.0311825 | Time: 6202.21 ms
[2021-01-15 10:47:46 main:574] : INFO : Epoch 747 | loss: 0.0311841 | val_loss: 0.0311841 | Time: 8284.07 ms
[2021-01-15 10:47:53 main:574] : INFO : Epoch 748 | loss: 0.0311852 | val_loss: 0.0311841 | Time: 6601.82 ms
[2021-01-15 10:47:58 main:574] : INFO : Epoch 749 | loss: 0.0311866 | val_loss: 0.031191 | Time: 5615.06 ms
[2021-01-15 10:48:06 main:574] : INFO : Epoch 750 | loss: 0.0311873 | val_loss: 0.0311827 | Time: 7939.39 ms
[2021-01-15 10:48:13 main:574] : INFO : Epoch 751 | loss: 0.0311835 | val_loss: 0.0311827 | Time: 6113.85 ms
[2021-01-15 10:48:17 main:574] : INFO : Epoch 752 | loss: 0.0311831 | val_loss: 0.0311825 | Time: 4721.19 ms
[2021-01-15 10:48:24 main:574] : INFO : Epoch 753 | loss: 0.0311842 | val_loss: 0.0311866 | Time: 6481.75 ms
[2021-01-15 10:48:28 main:574] : INFO : Epoch 754 | loss: 0.0311832 | val_loss: 0.0311829 | Time: 4206.98 ms
[2021-01-15 10:48:34 main:574] : INFO : Epoch 755 | loss: 0.0311831 | val_loss: 0.0311829 | Time: 5150.27 ms
[2021-01-15 10:48:38 main:574] : INFO : Epoch 756 | loss: 0.0311832 | val_loss: 0.0311831 | Time: 4523.71 ms
[2021-01-15 10:48:43 main:574] : INFO : Epoch 757 | loss: 0.0311827 | val_loss: 0.0311848 | Time: 4848.55 ms
[2021-01-15 10:48:48 main:574] : INFO : Epoch 758 | loss: 0.031184 | val_loss: 0.0311828 | Time: 4610.6 ms
[2021-01-15 10:48:53 main:574] : INFO : Epoch 759 | loss: 0.0311839 | val_loss: 0.031183 | Time: 4760.51 ms
[2021-01-15 10:48:58 main:574] : INFO : Epoch 760 | loss: 0.0311847 | val_loss: 0.0311846 | Time: 4973.63 ms
[2021-01-15 10:49:02 main:574] : INFO : Epoch 761 | loss: 0.0311846 | val_loss: 0.0311837 | Time: 4561.64 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFD1341D759
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 01/15/21 10:49:03
Install Directory : D:\Program Files\
Data Directory : D:\ProgramData
Project Symstore :
LoadLibraryA( D:\ProgramData\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( D:\ProgramData\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: D:\ProgramData\slots\2;D:\ProgramData\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000a8fd0000 00000000003df000 D:\ProgramData\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000015a70000 00000000001f6000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
ModLoad: 0000000015880000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
ModLoad: 00000000133f0000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
ModLoad: 00000000f1ed0000 00000000000d4000 C:\Program Files\Bitdefender\Bitdefender Security\atcuf\264968492627633626\atcuf64.dll (1.34.268.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\Bamboo\home\xml-data\build-dir\ATC-DEFAULT-SOURCES\bin\x64\Release\atcuf64\atcuf64.pdb
File Version : 1.34.268.0 #0x011f0aa
Company Name : Bitdefender S.R.L. Bucharest, ROMANIA
Product Name : Bitdefender® ATC
Product Version : 13
ModLoad: 00000000154d0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.685) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000131b0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
ModLoad: 0000000015720000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.685) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.685 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.685
ModLoad: 00000000138c0000 0000000000109000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.685) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.685 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.685
ModLoad: 0000000013820000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000139d0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000015940000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000157e0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000015670000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000014dc0000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000f76b0000 0000000000058000 D:\ProgramData\slots\2\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000add50000 0000000024b49000 D:\ProgramData\slots\2\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000007c900000 000000000abe7000 D:\ProgramData\slots\2\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000002be0000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 000000000c1d0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.488) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.488 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.488
ModLoad: 0000000002bc0000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 0000000004590000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29301.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\agent\_work\26\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29301.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29301.0
ModLoad: 00000000038e0000 0000000000045000 D:\ProgramData\slots\2\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000b1af0000 00000000030a5000 D:\ProgramData\slots\2\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000ae630000 00000000034b2000 D:\ProgramData\slots\2\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 0000000061550000 000000001b3a4000 D:\ProgramData\slots\2\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 000000000f0c0000 0000000000010000 D:\ProgramData\slots\2\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000583b0000 000000000919f000 D:\ProgramData\slots\2\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000054650000 0000000003d5b000 D:\ProgramData\slots\2\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000e6820000 000000000020d000 D:\ProgramData\slots\2\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000c27e0000 00000000002db000 D:\ProgramData\slots\2\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000052780000 0000000001eca000 D:\ProgramData\slots\2\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000f7b30000 0000000000040000 D:\ProgramData\slots\2\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000015a00000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000011120000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000013ad0000 0000000000467000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000013770000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000013740000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000012ef0000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000137c0000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
ModLoad: 00000000131e0000 000000000015d000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000012d30000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000013ff0000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.329 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.329
ModLoad: 0000000011320000 0000000000795000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000015170000 0000000000356000 C:\WINDOWS\System32\combase.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000012b50000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000013f40000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000014bd0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000c1c00000 0000000000267000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.5638) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 27.21.14.5638
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.70 driver
Product Version : 27.21.14.5638
ModLoad: 000000000dee0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000bc920000 000000000152d000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispsi.inf_amd64_98b8327e9bd736d9\nvcuda64.dll (27.21.14.5638) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 27.21.14.5638
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.70 driver
Product Version : 27.21.14.5638
ModLoad: 0000000014a30000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000010c60000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.610) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000003200000 00000000006d3000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.5638) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 27.21.14.5638
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 27.21.14.5638
ModLoad: 0000000014aa0000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000000f0b0000 0000000000009000 D:\ProgramData\slots\2\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bb950000 0000000000fca000 D:\ProgramData\slots\2\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 00000000136c0000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 4399, Write: 3838, Other 10177
- I/O Transfers Counters -
Read: 6587292, Write: 4024149, Other 138606
- Paged Pool Usage -
QuotaPagedPoolUsage: 3506400, QuotaPeakPagedPoolUsage: 3507200
QuotaNonPagedPoolUsage: 55008, QuotaPeakNonPagedPoolUsage: 55152
- Virtual Memory Usage -
VirtualSize: -403107840, PeakVirtualSize: 217260032
- Pagefile Usage -
PagefileUsage: -403107840, PeakPagefileUsage: -403107840
- Working Set Size -
WorkingSetSize: 1727930368, PeakWorkingSetSize: 1764057088, PageFaultCount: 637340
*** Dump of thread ID 13316 (state: Ready): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 861249984.000000, User Time: 13444843520.000000, Wait Time: 12351411.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFD1341D759
- Registers -
rax=0000000000000000 rbx=0000000086d762f0 rcx=0000000000000000 rdx=0000000000000000 rsi=00000000a8753fe0 rdi=0000000019930520
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000006000 r13=0000000000000000
r14=00000000a8754d20 r15=00000000a8754ca0 rip=000000001341d759 rsp=00000000a8753e80 rbp=00000000a8754888
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
a8753f50 02bc6220 957eb7f8 00000000 a8754d20 81eca014 KERNELBASE!RaiseException+0x0
a8753fb0 824c5272 957eb798 ad950af0 000007cb 00000116 VCRUNTIME140!_CxxThrowException+0x0
a8754000 824c56d4 00000001 00000000 854ed290 b15b5f68 torch_cpu!at::Tensor::zero_+0x0
a8754860 b0b79ebe a8754a90 b15b6850 b15b6880 000002cf torch_cpu!_THError+0x0
a87548a0 b0b75412 a8754a90 a87549f0 d3262040 b19ea720 torch_cuda!__THCudaCheck+0x0
a87548e0 821759ff a8754c00 a8754a90 a8754f60 f76e97b0 torch_cuda!at::native::set_storage_cuda_+0x0
a8754c70 8231726f b11ef630 81ecea63 00000001 f76cef8f torch_cpu!at::native::empty_cpu+0x0
a8754cb0 822d0615 a8750000 a8754d28 a8754f20 f76b7ce6 torch_cpu!at::native::mkldnn_sigmoid_+0x0
a8754cf0 822d204a a8750000 a8754d00 d3248000 afeb6ae5 torch_cpu!at::native::mkldnn_sigmoid_+0x0
a8754d80 822d0615 00000000 81ecea63 a8754e60 a8754f60 torch_cpu!at::native::mkldnn_sigmoid_+0x0
a8754dc0 823a1eb0 a8750000 00000000 a8754f48 00000001 torch_cpu!at::native::mkldnn_sigmoid_+0x0
a8754e60 823b2eec c5d50746 f76e9c50 a8754fc0 f76b3bd6 torch_cpu!at::bucketize_out+0x0
a8754eb0 b027f160 a8750000 00000000 a8754fc0 00000000 torch_cpu!at::empty+0x0
a8755250 b0282de8 f76e9c50 81ed0528 d41351a0 f76e9c50 torch_cuda!at::native::cummin_helper_cuda+0x0
a8755430 b0284379 a8755540 46b24a90 a8755940 46b24aa8 torch_cuda!at::native::cummin_helper_cuda+0x0
a8755670 b0283d64 a8750000 a8755940 a8755780 a8755940 torch_cuda!at::native::cat_out_cuda+0x0
a8755870 b0b21dd2 a87558c8 00000001 a87558c8 a8755940 torch_cuda!at::native::cat_cuda+0x0
a87558f0 b0b34179 a8750001 a8755930 d4132d08 04000000 torch_cuda!at::native::set_storage_cuda_+0x0
a8755930 823a1d7d 33963d90 00000000 00200000 00000000 torch_cuda!at::native::set_storage_cuda_+0x0
a87559b0 82380d81 00000000 f76d3994 a8756180 46b245f0 torch_cpu!at::bucketize_out+0x0
a87559f0 8218b4a5 a8755a30 d4134da0 a8755b00 a8755b70 torch_cpu!at::_cat+0x0
a8755ac0 824727bd a8755b90 00000101 a8755b18 f7701ea8 torch_cpu!at::native::cat+0x0
a8755b40 81f329ae 46b24d30 46b24d40 46b24d48 a8755af8 torch_cpu!at::zeros_out+0x0
a8755b80 823a1d7d a8755bc0 a8755c30 cbcf0000 a8755ed8 torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
a8755c00 823a75d1 a87561a0 87220640 a8756180 46b245f0 torch_cpu!at::bucketize_out+0x0
a8755c40 82197d4c a8755ed0 00000000 a8755ed0 00000000 torch_cpu!at::cat+0x0
a8755e00 8248c38d a8755e58 00000001 a8755e58 a8755ed0 torch_cpu!at::native::stack+0x0
a8755e80 81f329ae 00000000 00000000 00200000 00000000 torch_cpu!at::zeros_out+0x0
a8755ec0 823a1d7d a8755f00 a8755f70 00000000 85141c8f torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
a8755f40 823f02e1 a8756070 f76d3720 a87561a0 87220640 torch_cpu!at::bucketize_out+0x0
a8755f80 83783839 a8756000 d4130880 d4130880 a8756090 torch_cpu!at::stack+0x0
a8756150 81f329ae 46b24d30 00000002 00000000 a8756240 torch_cpu!torch::autograd::GraphRoot::apply+0x0
a8756190 823a1d7d a87561d0 a8756240 d4134160 0000041b torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
a8756210 823f02e1 a8756380 81ec9cac 25bbad10 46b245f0 torch_cpu!at::bucketize_out+0x0
a8756250 a8fdff7e a8756318 33e061b0 33e061b0 a8756460 torch_cpu!at::stack+0x0
a87562f0 a8fe2ef8 a8fdfe30 a8756330 a8756330 a8756380 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::addCallback+0x0
a8756350 a8fe3ea4 0332d2a0 15a85d21 a87564f0 ad130000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::getSlot+0x0
a8756490 a8fdd3d2 41d65c40 45c4d130 a87564f0 15a895f1 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0
a87564c0 a8fe4541 00000000 45c5db50 188f2540 15a895f1 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::operator=+0x0
a8756670 a8fe743a a8756740 00000000 a8757180 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0
a8756b70 a8feb950 a8756c70 a8756c70 00000000 03512580 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::then+0x0
a876f910 a922c6f8 d41c5e20 00000000 00000000 d4209870 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
a876f950 15897034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
a876f980 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a876fa00 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8316 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 12351405.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000a8bff830 rsi=0000000000000000 rdi=0000000000000064
r8=00000000a8bff808 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000015b0c634 rsp=00000000a8bff808 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
a8bff800 1343d84e a8bff8c8 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0
a8bff8a0 a900a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
a8bff8d0 15897034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
a8bff900 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a8bff980 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 13892 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 12351410.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=00000000a8cffaa0 rsi=0000000000000000 rdi=000000000000000d
r8=0000000000000001 r9=0000000000000000 r10=0000000000000000 r11=00000000a8cffa10 r12=0000000000000064 r13=00000000a8cffaa0
r14=00000000a8cff7a0 r15=0000000000000000 rip=0000000015b0cb04 rsp=00000000a8cff748 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
a8cff740 13440d40 00000003 00000004 00000001 00000000 ntdll!NtWaitForMultipleObjects+0x0
a8cffa30 13440c3e 00000000 15ab630d 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
a8cffa70 bcacb8d4 00000000 00000000 a8cffb38 d463d858 KERNELBASE!WaitForMultipleObjects+0x0
a8cffcb0 bcb6076d 00000000 00000004 00000000 00000000 nvcuda64!+0x0
a8cffd60 bcacb2a3 581ca0c1 5790cf35 d4cec5e0 00000000 nvcuda64!cuProfilerStop+0x0
a8cffd90 bcdd8de0 d4cec5e0 00000000 00000000 00000000 nvcuda64!+0x0
a8cffdc0 15897034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
a8cffdf0 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a8cffe70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 12180 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 12351408.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=0000000000000284 rdx=0000000000000000 rsi=00000000a8dff9f8 rdi=0000000000000284
r8=00000000a8dff9c8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=00000000d43a0b60 r13=0000000000000000
r14=0000000000000284 r15=0000000000000001 rip=0000000015b0c034 rsp=00000000a8dff9c8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
a8dff9c0 1340a34e 00000000 00000000 00000000 00000000 ntdll!NtWaitForSingleObject+0x0
a8dffa60 bccda797 00000000 00000000 00000000 00000284 KERNELBASE!WaitForSingleObjectEx+0x0
a8dffad0 bcacb2a3 00000001 d480eef0 d4cec670 00000000 nvcuda64!cuProfilerStop+0x0
a8dffb00 bcdd8de0 d4cec670 00000000 00000000 00000000 nvcuda64!+0x0
a8dffb30 15897034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
a8dffb60 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a8dffbe0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 2432 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 12351411.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000d4639258 rdx=00000000a8effc58 rsi=00000000a8effbc0 rdi=0000000000000000
r8=0000000000000001 r9=0000000000000980 r10=00000000d463d9d8 r11=00000000f27fdaed r12=00000000a8effc58 r13=0000000000000000
r14=00000000d4639258 r15=00000000d463d9d8 rip=0000000015b0f9a4 rsp=00000000a8effb98 rbp=00000000a8effbf0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
a8effb90 15ad54f5 00000102 bcb993cd 00000000 a8effbc0 ntdll!NtWaitForAlertByThreadId+0x0
a8effc00 1345b9d9 d463b1c0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
a8effc30 bca5209a d463b1c0 d463b1c0 d463b1c0 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
a8effc90 bcacb2a3 00000002 582dbe75 5791f003 00000000 nvcuda64!+0x0
a8effcc0 bcdd8de0 d4cec730 00000000 00000000 00000000 nvcuda64!+0x0
a8effcf0 15897034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
a8effd20 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a8effda0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 7740 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 549218752.000000, User Time: 9752655872.000000, Wait Time: 12351407.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=000000002d12f598 rdx=0000000000000000 rsi=0000000000000000 rdi=000000002d12f598
r8=0000000000000001 r9=0000000000000001 r10=0000000000000000 r11=0000000085e15460 r12=0000000000000000 r13=0000000000000000
r14=00000000a8fff250 r15=000000002d12f548 rip=0000000015b0f9a4 rsp=00000000a8fff228 rbp=00000000a8fff280
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
a8fff220 15ad5281 33de1dd8 02bf2f97 33ae2c80 33ae2c70 ntdll!NtWaitForAlertByThreadId+0x0
a8fff2a0 1345ba89 2d12f590 2d12f588 2d12f540 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
a8fff2e0 02c1bf3d 33ae2ca0 00000000 00000000 83ae05b4 KERNELBASE!SleepConditionVariableSRW+0x0
a8fff320 02bf2798 00000000 33ae2ca0 00000000 00000000 MSVCP140!__crtSetThreadpoolWait+0x0
a8fff350 02bf2a4a 229ab87f 00000000 a8fff560 2d12f588 MSVCP140!std::_Winerror_message+0x0
a8fff380 83ae064b 2d12f540 a8fff480 2d12f540 33ae2c80 MSVCP140!_Cnd_wait+0x0
a8fff3c0 83ae25b1 45e304a0 ffffffff 33ae2c80 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
a8fff6b0 83ae2471 87006d10 a8fff6f0 87006d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
a8fff700 83ad9744 00080001 45c26510 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
a8fff730 139f14c2 45c26510 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
a8fff760 15897034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
a8fff790 15abd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
a8fff810 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)