Task 13238738
| Name | ParityModified-1645996248-18601-3-0_0 |
| Workunit | 10357703 |
| Created | 15 Mar 2022, 16:59:01 UTC |
| Sent | 22 Mar 2022, 11:39:48 UTC |
| Report deadline | 30 Mar 2022, 11:39:48 UTC |
| Received | 23 Mar 2022, 17:57:02 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 563 |
| Run time | 37 min 55 sec |
| CPU time | 36 min 16 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 5,442.56 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.10 GB |
| Peak swap size | 4.33 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
loss: 0.0311746 | Time: 2098.97 ms
[2022-03-23 18:47:39 main:574] : INFO : Epoch 828 | loss: 0.0311676 | val_loss: 0.0311749 | Time: 2148.04 ms
[2022-03-23 18:47:41 main:574] : INFO : Epoch 829 | loss: 0.0311718 | val_loss: 0.031179 | Time: 2096.18 ms
[2022-03-23 18:47:43 main:574] : INFO : Epoch 830 | loss: 0.0311691 | val_loss: 0.0311775 | Time: 2061.92 ms
[2022-03-23 18:47:45 main:574] : INFO : Epoch 831 | loss: 0.0311693 | val_loss: 0.0311756 | Time: 2101.72 ms
[2022-03-23 18:47:48 main:574] : INFO : Epoch 832 | loss: 0.0311719 | val_loss: 0.0311783 | Time: 2107.36 ms
[2022-03-23 18:47:50 main:574] : INFO : Epoch 833 | loss: 0.0311677 | val_loss: 0.0311778 | Time: 2069.6 ms
[2022-03-23 18:47:52 main:574] : INFO : Epoch 834 | loss: 0.0311698 | val_loss: 0.031175 | Time: 2048.26 ms
[2022-03-23 18:47:54 main:574] : INFO : Epoch 835 | loss: 0.0311689 | val_loss: 0.0311761 | Time: 2071.15 ms
[2022-03-23 18:47:56 main:574] : INFO : Epoch 836 | loss: 0.031169 | val_loss: 0.0311772 | Time: 2085.05 ms
[2022-03-23 18:47:58 main:574] : INFO : Epoch 837 | loss: 0.0311698 | val_loss: 0.0311792 | Time: 2067.31 ms
[2022-03-23 18:48:00 main:574] : INFO : Epoch 838 | loss: 0.0311694 | val_loss: 0.0311751 | Time: 2089.28 ms
[2022-03-23 18:48:02 main:574] : INFO : Epoch 839 | loss: 0.0311708 | val_loss: 0.0311765 | Time: 2081.98 ms
[2022-03-23 18:48:04 main:574] : INFO : Epoch 840 | loss: 0.0311703 | val_loss: 0.031175 | Time: 2080.9 ms
[2022-03-23 18:48:06 main:574] : INFO : Epoch 841 | loss: 0.0311686 | val_loss: 0.031173 | Time: 2081.49 ms
[2022-03-23 18:48:09 main:574] : INFO : Epoch 842 | loss: 0.0311714 | val_loss: 0.031176 | Time: 2119.16 ms
[2022-03-23 18:48:11 main:574] : INFO : Epoch 843 | loss: 0.0311706 | val_loss: 0.0311752 | Time: 2089.23 ms
[2022-03-23 18:48:13 main:574] : INFO : Epoch 844 | loss: 0.0311697 | val_loss: 0.0311803 | Time: 2084.3 ms
[2022-03-23 18:48:15 main:574] : INFO : Epoch 845 | loss: 0.0311709 | val_loss: 0.0311772 | Time: 2083.8 ms
[2022-03-23 18:48:17 main:574] : INFO : Epoch 846 | loss: 0.0311716 | val_loss: 0.0311784 | Time: 2096.95 ms
[2022-03-23 18:48:19 main:574] : INFO : Epoch 847 | loss: 0.0311731 | val_loss: 0.0311793 | Time: 2079.9 ms
[2022-03-23 18:48:21 main:574] : INFO : Epoch 848 | loss: 0.0311704 | val_loss: 0.031178 | Time: 2076.41 ms
[2022-03-23 18:48:23 main:574] : INFO : Epoch 849 | loss: 0.0311742 | val_loss: 0.0311763 | Time: 2108.38 ms
[2022-03-23 18:48:26 main:574] : INFO : Epoch 850 | loss: 0.0311757 | val_loss: 0.0311769 | Time: 2095.69 ms
[2022-03-23 18:48:28 main:574] : INFO : Epoch 851 | loss: 0.0311715 | val_loss: 0.0311798 | Time: 2083.44 ms
[2022-03-23 18:48:30 main:574] : INFO : Epoch 852 | loss: 0.0311707 | val_loss: 0.0311815 | Time: 2088.57 ms
[2022-03-23 18:48:32 main:574] : INFO : Epoch 853 | loss: 0.031172 | val_loss: 0.0311776 | Time: 2127.94 ms
[2022-03-23 18:48:34 main:574] : INFO : Epoch 854 | loss: 0.031169 | val_loss: 0.0311779 | Time: 2086.33 ms
[2022-03-23 18:48:36 main:574] : INFO : Epoch 855 | loss: 0.0311708 | val_loss: 0.031179 | Time: 2104.64 ms
[2022-03-23 18:48:38 main:574] : INFO : Epoch 856 | loss: 0.0311705 | val_loss: 0.0311763 | Time: 2148.65 ms
[2022-03-23 18:48:40 main:574] : INFO : Epoch 857 | loss: 0.0311736 | val_loss: 0.0311755 | Time: 2089.06 ms
[2022-03-23 18:48:43 main:574] : INFO : Epoch 858 | loss: 0.0311744 | val_loss: 0.0311762 | Time: 2088.87 ms
[2022-03-23 18:48:45 main:574] : INFO : Epoch 859 | loss: 0.0311725 | val_loss: 0.0311768 | Time: 2080.61 ms
[2022-03-23 18:48:47 main:574] : INFO : Epoch 860 | loss: 0.0311706 | val_loss: 0.0311753 | Time: 2097.19 ms
[2022-03-23 18:48:49 main:574] : INFO : Epoch 861 | loss: 0.0311688 | val_loss: 0.0311755 | Time: 2066.21 ms
[2022-03-23 18:48:51 main:574] : INFO : Epoch 862 | loss: 0.0311696 | val_loss: 0.0311819 | Time: 2085.83 ms
[2022-03-23 18:48:53 main:574] : INFO : Epoch 863 | loss: 0.03117 | val_loss: 0.0311751 | Time: 2126.76 ms
[2022-03-23 18:48:55 main:574] : INFO : Epoch 864 | loss: 0.031174 | val_loss: 0.0311798 | Time: 2075.85 ms
[2022-03-23 18:48:57 main:574] : INFO : Epoch 865 | loss: 0.0311693 | val_loss: 0.0311788 | Time: 2080.89 ms
[2022-03-23 18:49:00 main:574] : INFO : Epoch 866 | loss: 0.0311679 | val_loss: 0.0311789 | Time: 2078.75 ms
[2022-03-23 18:49:02 main:574] : INFO : Epoch 867 | loss: 0.0311692 | val_loss: 0.0311792 | Time: 2079.23 ms
[2022-03-23 18:49:04 main:574] : INFO : Epoch 868 | loss: 0.0311692 | val_loss: 0.0311818 | Time: 2057.05 ms
[2022-03-23 18:49:06 main:574] : INFO : Epoch 869 | loss: 0.0311722 | val_loss: 0.0311844 | Time: 2090.69 ms
[2022-03-23 18:49:08 main:574] : INFO : Epoch 870 | loss: 0.0311718 | val_loss: 0.0311791 | Time: 2062.31 ms
[2022-03-23 18:49:10 main:574] : INFO : Epoch 871 | loss: 0.0311706 | val_loss: 0.0311849 | Time: 2036.27 ms
[2022-03-23 18:49:12 main:574] : INFO : Epoch 872 | loss: 0.0373452 | val_loss: 0.0414776 | Time: 2015.67 ms
[2022-03-23 18:49:14 main:574] : INFO : Epoch 873 | loss: 0.0341363 | val_loss: 0.0317986 | Time: 2050.32 ms
[2022-03-23 18:49:16 main:574] : INFO : Epoch 874 | loss: 0.0314478 | val_loss: 0.0313788 | Time: 2055.79 ms
[2022-03-23 18:49:18 main:574] : INFO : Epoch 875 | loss: 0.0312878 | val_loss: 0.0312593 | Time: 2074.53 ms
[2022-03-23 18:49:21 main:574] : INFO : Epoch 876 | loss: 0.0312554 | val_loss: 0.0312489 | Time: 2088.93 ms
[2022-03-23 18:49:23 main:574] : INFO : Epoch 877 | loss: 0.0312487 | val_loss: 0.0312459 | Time: 2057.45 ms
[2022-03-23 18:49:25 main:574] : INFO : Epoch 878 | loss: 0.0312456 | val_loss: 0.0312431 | Time: 2104.77 ms
[2022-03-23 18:49:27 main:574] : INFO : Epoch 879 | loss: 0.0312417 | val_loss: 0.0312389 | Time: 2080.97 ms
[2022-03-23 18:49:29 main:574] : INFO : Epoch 880 | loss: 0.0312376 | val_loss: 0.0312337 | Time: 2126.79 ms
[2022-03-23 18:49:31 main:574] : INFO : Epoch 881 | loss: 0.0312323 | val_loss: 0.0312281 | Time: 2070.38 ms
[2022-03-23 18:49:33 main:574] : INFO : Epoch 882 | loss: 0.031227 | val_loss: 0.031222 | Time: 2111.33 ms
[2022-03-23 18:49:35 main:574] : INFO : Epoch 883 | loss: 0.0312207 | val_loss: 0.0312152 | Time: 2083.88 ms
[2022-03-23 18:49:38 main:574] : INFO : Epoch 884 | loss: 0.0312139 | val_loss: 0.0312085 | Time: 2075.81 ms
[2022-03-23 18:49:40 main:574] : INFO : Epoch 885 | loss: 0.0312076 | val_loss: 0.0312025 | Time: 2102.25 ms
[2022-03-23 18:49:42 main:574] : INFO : Epoch 886 | loss: 0.0312022 | val_loss: 0.0311953 | Time: 2123.06 ms
[2022-03-23 18:49:44 main:574] : INFO : Epoch 887 | loss: 0.031198 | val_loss: 0.031193 | Time: 2092.09 ms
[2022-03-23 18:49:46 main:574] : INFO : Epoch 888 | loss: 0.0311958 | val_loss: 0.0311905 | Time: 2103.47 ms
[2022-03-23 18:49:48 main:574] : INFO : Epoch 889 | loss: 0.0311934 | val_loss: 0.0311884 | Time: 2095.44 ms
[2022-03-23 18:49:50 main:574] : INFO : Epoch 890 | loss: 0.0311909 | val_loss: 0.0311864 | Time: 2095.41 ms
[2022-03-23 18:49:53 main:574] : INFO : Epoch 891 | loss: 0.0311892 | val_loss: 0.0311853 | Time: 2147.21 ms
[2022-03-23 18:49:55 main:574] : INFO : Epoch 892 | loss: 0.0311873 | val_loss: 0.0311854 | Time: 2103.41 ms
[2022-03-23 18:49:57 main:574] : INFO : Epoch 893 | loss: 0.0311865 | val_loss: 0.0311835 | Time: 2081.14 ms
[2022-03-23 18:49:59 main:574] : INFO : Epoch 894 | loss: 0.0311865 | val_loss: 0.031183 | Time: 2105.76 ms
[2022-03-23 18:50:01 main:574] : INFO : Epoch 895 | loss: 0.0311855 | val_loss: 0.031183 | Time: 2199.17 ms
[2022-03-23 18:50:03 main:574] : INFO : Epoch 896 | loss: 0.0311849 | val_loss: 0.0311828 | Time: 2133.42 ms
[2022-03-23 18:50:06 main:574] : INFO : Epoch 897 | loss: 0.0311843 | val_loss: 0.0311828 | Time: 2054.77 ms
[2022-03-23 18:50:08 main:574] : INFO : Epoch 898 | loss: 0.0311848 | val_loss: 0.0311834 | Time: 2052.27 ms
[2022-03-23 18:50:10 main:574] : INFO : Epoch 899 | loss: 0.031184 | val_loss: 0.0311821 | Time: 2088.52 ms
[2022-03-23 18:50:12 main:574] : INFO : Epoch 900 | loss: 0.0311831 | val_loss: 0.0311817 | Time: 2138.52 ms
[2022-03-23 18:50:14 main:574] : INFO : Epoch 901 | loss: 0.0311833 | val_loss: 0.0311824 | Time: 2117.47 ms
[2022-03-23 18:50:16 main:574] : INFO : Epoch 902 | loss: 0.0311828 | val_loss: 0.0311813 | Time: 2080.93 ms
[2022-03-23 18:50:18 main:574] : INFO : Epoch 903 | loss: 0.0311818 | val_loss: 0.0311807 | Time: 2115.74 ms
[2022-03-23 18:50:20 main:574] : INFO : Epoch 904 | loss: 0.0311815 | val_loss: 0.0311811 | Time: 2102.74 ms
[2022-03-23 18:50:23 main:574] : INFO : Epoch 905 | loss: 0.0311812 | val_loss: 0.0311808 | Time: 2075.09 ms
[2022-03-23 18:50:25 main:574] : INFO : Epoch 906 | loss: 0.0311813 | val_loss: 0.0311828 | Time: 2093.95 ms
[2022-03-23 18:50:27 main:574] : INFO : Epoch 907 | loss: 0.0311813 | val_loss: 0.0311803 | Time: 2062.09 ms
[2022-03-23 18:50:29 main:574] : INFO : Epoch 908 | loss: 0.0311807 | val_loss: 0.0311819 | Time: 2100.08 ms
[2022-03-23 18:50:31 main:574] : INFO : Epoch 909 | loss: 0.0311812 | val_loss: 0.031181 | Time: 2102.86 ms
[2022-03-23 18:50:33 main:574] : INFO : Epoch 910 | loss: 0.0311796 | val_loss: 0.0311796 | Time: 2107.5 ms
[2022-03-23 18:50:35 main:574] : INFO : Epoch 911 | loss: 0.0311794 | val_loss: 0.0311799 | Time: 2061.17 ms
[2022-03-23 18:50:37 main:574] : INFO : Epoch 912 | loss: 0.0311791 | val_loss: 0.0311806 | Time: 2077.37 ms
[2022-03-23 18:50:40 main:574] : INFO : Epoch 913 | loss: 0.031179 | val_loss: 0.0311819 | Time: 2067.38 ms
[2022-03-23 18:50:42 main:574] : INFO : Epoch 914 | loss: 0.0311791 | val_loss: 0.0311809 | Time: 2044.96 ms
[2022-03-23 18:50:44 main:574] : INFO : Epoch 915 | loss: 0.0311792 | val_loss: 0.0311795 | Time: 2061.61 ms
[2022-03-23 18:50:46 main:574] : INFO : Epoch 916 | loss: 0.0311785 | val_loss: 0.0311811 | Time: 2117.03 ms
[2022-03-23 18:50:48 main:574] : INFO : Epoch 917 | loss: 0.0311785 | val_loss: 0.0311795 | Time: 2113.48 ms
[2022-03-23 18:50:50 main:574] : INFO : Epoch 918 | loss: 0.0311781 | val_loss: 0.031179 | Time: 2074.7 ms
[2022-03-23 18:50:52 main:574] : INFO : Epoch 919 | loss: 0.0311773 | val_loss: 0.0311786 | Time: 2091.88 ms
[2022-03-23 18:50:54 main:574] : INFO : Epoch 920 | loss: 0.0311771 | val_loss: 0.0311789 | Time: 2108.21 ms
[2022-03-23 18:50:56 main:574] : INFO : Epoch 921 | loss: 0.0311768 | val_loss: 0.0311786 | Time: 2069.38 ms
[2022-03-23 18:50:59 main:574] : INFO : Epoch 922 | loss: 0.0311773 | val_loss: 0.0311803 | Time: 2047.83 ms
[2022-03-23 18:51:01 main:574] : INFO : Epoch 923 | loss: 0.0311775 | val_loss: 0.0311813 | Time: 2074.48 ms
[2022-03-23 18:51:03 main:574] : INFO : Epoch 924 | loss: 0.0311777 | val_loss: 0.0311783 | Time: 2096.39 ms
[2022-03-23 18:51:05 main:574] : INFO : Epoch 925 | loss: 0.0311766 | val_loss: 0.0311816 | Time: 2087.56 ms
[2022-03-23 18:51:07 main:574] : INFO : Epoch 926 | loss: 0.0311782 | val_loss: 0.0311778 | Time: 2051.05 ms
[2022-03-23 18:51:09 main:574] : INFO : Epoch 927 | loss: 0.0311765 | val_loss: 0.031178 | Time: 2079.77 ms
[2022-03-23 18:51:11 main:574] : INFO : Epoch 928 | loss: 0.0311762 | val_loss: 0.031178 | Time: 2081.11 ms
[2022-03-23 18:51:13 main:574] : INFO : Epoch 929 | loss: 0.0311759 | val_loss: 0.0311779 | Time: 2105.07 ms
[2022-03-23 18:51:15 main:574] : INFO : Epoch 930 | loss: 0.0311778 | val_loss: 0.0311777 | Time: 2056.93 ms
[2022-03-23 18:51:18 main:574] : INFO : Epoch 931 | loss: 0.0311771 | val_loss: 0.0311777 | Time: 2078.85 ms
[2022-03-23 18:51:20 main:574] : INFO : Epoch 932 | loss: 0.0311754 | val_loss: 0.0311785 | Time: 2048.85 ms
[2022-03-23 18:51:22 main:574] : INFO : Epoch 933 | loss: 0.0311757 | val_loss: 0.0311774 | Time: 2020.17 ms
[2022-03-23 18:51:24 main:574] : INFO : Epoch 934 | loss: 0.0311749 | val_loss: 0.0311779 | Time: 2068.54 ms
[2022-03-23 18:51:26 main:574] : INFO : Epoch 935 | loss: 0.0311747 | val_loss: 0.0311768 | Time: 2059.77 ms
[2022-03-23 18:51:28 main:574] : INFO : Epoch 936 | loss: 0.0311751 | val_loss: 0.0311778 | Time: 2046.96 ms
[2022-03-23 18:51:30 main:574] : INFO : Epoch 937 | loss: 0.031175 | val_loss: 0.031176 | Time: 2078.24 ms
[2022-03-23 18:51:32 main:574] : INFO : Epoch 938 | loss: 0.0311743 | val_loss: 0.0311769 | Time: 2077.42 ms
[2022-03-23 18:51:34 main:574] : INFO : Epoch 939 | loss: 0.0311743 | val_loss: 0.0311776 | Time: 2059.61 ms
[2022-03-23 18:51:36 main:574] : INFO : Epoch 940 | loss: 0.0311764 | val_loss: 0.0311796 | Time: 2089.04 ms
[2022-03-23 18:51:38 main:574] : INFO : Epoch 941 | loss: 0.0311754 | val_loss: 0.0311777 | Time: 2075.08 ms
[2022-03-23 18:51:41 main:574] : INFO : Epoch 942 | loss: 0.0311766 | val_loss: 0.0311774 | Time: 2027.97 ms
[2022-03-23 18:51:43 main:574] : INFO : Epoch 943 | loss: 0.0311749 | val_loss: 0.0311757 | Time: 2097.17 ms
[2022-03-23 18:51:45 main:574] : INFO : Epoch 944 | loss: 0.0311744 | val_loss: 0.0311765 | Time: 2082.31 ms
[2022-03-23 18:51:47 main:574] : INFO : Epoch 945 | loss: 0.0311746 | val_loss: 0.0311768 | Time: 2075.41 ms
[2022-03-23 18:51:49 main:574] : INFO : Epoch 946 | loss: 0.0311739 | val_loss: 0.031177 | Time: 2031.51 ms
[2022-03-23 18:51:51 main:574] : INFO : Epoch 947 | loss: 0.0311738 | val_loss: 0.0311755 | Time: 2056.74 ms
[2022-03-23 18:51:53 main:574] : INFO : Epoch 948 | loss: 0.0311734 | val_loss: 0.0311763 | Time: 2023.47 ms
[2022-03-23 18:51:55 main:574] : INFO : Epoch 949 | loss: 0.0311746 | val_loss: 0.0311801 | Time: 2092.63 ms
[2022-03-23 18:51:57 main:574] : INFO : Epoch 950 | loss: 0.0311737 | val_loss: 0.0311766 | Time: 2065.74 ms
[2022-03-23 18:51:59 main:574] : INFO : Epoch 951 | loss: 0.0311735 | val_loss: 0.0311779 | Time: 2096.26 ms
[2022-03-23 18:52:02 main:574] : INFO : Epoch 952 | loss: 0.0311747 | val_loss: 0.0311764 | Time: 2099.24 ms
[2022-03-23 18:52:04 main:574] : INFO : Epoch 953 | loss: 0.0311731 | val_loss: 0.0311757 | Time: 2105.99 ms
[2022-03-23 18:52:06 main:574] : INFO : Epoch 954 | loss: 0.0311732 | val_loss: 0.0311762 | Time: 2102.22 ms
[2022-03-23 18:52:08 main:574] : INFO : Epoch 955 | loss: 0.0311732 | val_loss: 0.0311776 | Time: 2095.51 ms
[2022-03-23 18:52:10 main:574] : INFO : Epoch 956 | loss: 0.0311735 | val_loss: 0.0311766 | Time: 2094.06 ms
[2022-03-23 18:52:12 main:574] : INFO : Epoch 957 | loss: 0.0311747 | val_loss: 0.0311779 | Time: 2110.79 ms
[2022-03-23 18:52:14 main:574] : INFO : Epoch 958 | loss: 0.0311735 | val_loss: 0.0311756 | Time: 2086.89 ms
[2022-03-23 18:52:17 main:574] : INFO : Epoch 959 | loss: 0.0311741 | val_loss: 0.0311755 | Time: 2066.82 ms
[2022-03-23 18:52:19 main:574] : INFO : Epoch 960 | loss: 0.0311722 | val_loss: 0.0311763 | Time: 2111.65 ms
[2022-03-23 18:52:21 main:574] : INFO : Epoch 961 | loss: 0.0311727 | val_loss: 0.0311769 | Time: 2108.84 ms
[2022-03-23 18:52:23 main:574] : INFO : Epoch 962 | loss: 0.0311738 | val_loss: 0.0311762 | Time: 2073 ms
[2022-03-23 18:52:25 main:574] : INFO : Epoch 963 | loss: 0.0311733 | val_loss: 0.0311762 | Time: 2126.27 ms
[2022-03-23 18:52:27 main:574] : INFO : Epoch 964 | loss: 0.0311732 | val_loss: 0.0311781 | Time: 2107.98 ms
[2022-03-23 18:52:29 main:574] : INFO : Epoch 965 | loss: 0.0311731 | val_loss: 0.031177 | Time: 2084.76 ms
[2022-03-23 18:52:32 main:574] : INFO : Epoch 966 | loss: 0.0311733 | val_loss: 0.031179 | Time: 2124.59 ms
[2022-03-23 18:52:34 main:574] : INFO : Epoch 967 | loss: 0.0311743 | val_loss: 0.031178 | Time: 2071.81 ms
[2022-03-23 18:52:36 main:574] : INFO : Epoch 968 | loss: 0.0311738 | val_loss: 0.0311772 | Time: 2099.39 ms
[2022-03-23 18:52:38 main:574] : INFO : Epoch 969 | loss: 0.0311727 | val_loss: 0.0311808 | Time: 2140.32 ms
[2022-03-23 18:52:40 main:574] : INFO : Epoch 970 | loss: 0.0311732 | val_loss: 0.0311766 | Time: 2105.82 ms
[2022-03-23 18:52:42 main:574] : INFO : Epoch 971 | loss: 0.0311727 | val_loss: 0.0311789 | Time: 2102.47 ms
[2022-03-23 18:52:44 main:574] : INFO : Epoch 972 | loss: 0.0311735 | val_loss: 0.0311773 | Time: 2074.54 ms
[2022-03-23 18:52:47 main:574] : INFO : Epoch 973 | loss: 0.0311719 | val_loss: 0.0311765 | Time: 2108.93 ms
[2022-03-23 18:52:49 main:574] : INFO : Epoch 974 | loss: 0.0311724 | val_loss: 0.0311782 | Time: 2107.07 ms
[2022-03-23 18:52:51 main:574] : INFO : Epoch 975 | loss: 0.0311738 | val_loss: 0.0311785 | Time: 2093.25 ms
[2022-03-23 18:52:53 main:574] : INFO : Epoch 976 | loss: 0.0311744 | val_loss: 0.0311793 | Time: 2092.97 ms
[2022-03-23 18:52:55 main:574] : INFO : Epoch 977 | loss: 0.0311762 | val_loss: 0.0311788 | Time: 2057.42 ms
[2022-03-23 18:52:57 main:574] : INFO : Epoch 978 | loss: 0.0311761 | val_loss: 0.0311772 | Time: 2100.39 ms
[2022-03-23 18:53:00 main:574] : INFO : Epoch 979 | loss: 0.0311734 | val_loss: 0.0311773 | Time: 2123.67 ms
[2022-03-23 18:53:02 main:574] : INFO : Epoch 980 | loss: 0.0311754 | val_loss: 0.0311783 | Time: 2138.18 ms
[2022-03-23 18:53:04 main:574] : INFO : Epoch 981 | loss: 0.0311724 | val_loss: 0.0311763 | Time: 2077.41 ms
[2022-03-23 18:53:06 main:574] : INFO : Epoch 982 | loss: 0.0311731 | val_loss: 0.0311803 | Time: 2139.32 ms
[2022-03-23 18:53:08 main:574] : INFO : Epoch 983 | loss: 0.0311726 | val_loss: 0.0311792 | Time: 2146.57 ms
[2022-03-23 18:53:10 main:574] : INFO : Epoch 984 | loss: 0.0311723 | val_loss: 0.0311779 | Time: 2066.41 ms
[2022-03-23 18:53:13 main:574] : INFO : Epoch 985 | loss: 0.0311717 | val_loss: 0.0311775 | Time: 2085.57 ms
[2022-03-23 18:53:15 main:574] : INFO : Epoch 986 | loss: 0.0311718 | val_loss: 0.0311762 | Time: 2128.64 ms
[2022-03-23 18:53:17 main:574] : INFO : Epoch 987 | loss: 0.0311724 | val_loss: 0.0311761 | Time: 2123.68 ms
[2022-03-23 18:53:19 main:574] : INFO : Epoch 988 | loss: 0.0311718 | val_loss: 0.0311759 | Time: 2148.21 ms
[2022-03-23 18:53:22 main:574] : INFO : Epoch 989 | loss: 0.0311722 | val_loss: 0.0311766 | Time: 2116.63 ms
[2022-03-23 18:53:24 main:574] : INFO : Epoch 990 | loss: 0.0311724 | val_loss: 0.0311794 | Time: 2124.61 ms
[2022-03-23 18:53:26 main:574] : INFO : Epoch 991 | loss: 0.0311741 | val_loss: 0.0311757 | Time: 2089.04 ms
[2022-03-23 18:53:28 main:574] : INFO : Epoch 992 | loss: 0.0311723 | val_loss: 0.0311762 | Time: 2084.36 ms
[2022-03-23 18:53:30 main:574] : INFO : Epoch 993 | loss: 0.0311715 | val_loss: 0.0311767 | Time: 2108.95 ms
[2022-03-23 18:53:32 main:574] : INFO : Epoch 994 | loss: 0.0311722 | val_loss: 0.0311784 | Time: 2114.45 ms
[2022-03-23 18:53:34 main:574] : INFO : Epoch 995 | loss: 0.0311754 | val_loss: 0.0311775 | Time: 2111.34 ms
[2022-03-23 18:53:37 main:574] : INFO : Epoch 996 | loss: 0.0311748 | val_loss: 0.0311775 | Time: 2096.87 ms
[2022-03-23 18:53:39 main:574] : INFO : Epoch 997 | loss: 0.0311763 | val_loss: 0.0311759 | Time: 2104.06 ms
[2022-03-23 18:53:41 main:574] : INFO : Epoch 998 | loss: 0.0311752 | val_loss: 0.0311776 | Time: 2110.74 ms
[2022-03-23 18:53:43 main:574] : INFO : Epoch 999 | loss: 0.0311736 | val_loss: 0.0311757 | Time: 2128.93 ms
[2022-03-23 18:53:45 main:574] : INFO : Epoch 1000 | loss: 0.0311719 | val_loss: 0.0311747 | Time: 2066.36 ms
[2022-03-23 18:53:47 main:574] : INFO : Epoch 1001 | loss: 0.0311733 | val_loss: 0.0311765 | Time: 2063.15 ms
[2022-03-23 18:53:49 main:574] : INFO : Epoch 1002 | loss: 0.0311723 | val_loss: 0.0311769 | Time: 2082.93 ms
[2022-03-23 18:53:52 main:574] : INFO : Epoch 1003 | loss: 0.0311737 | val_loss: 0.0311759 | Time: 2109.9 ms
[2022-03-23 18:53:54 main:574] : INFO : Epoch 1004 | loss: 0.0311726 | val_loss: 0.0311774 | Time: 2079.3 ms
[2022-03-23 18:53:56 main:574] : INFO : Epoch 1005 | loss: 0.0311712 | val_loss: 0.0311766 | Time: 2126.17 ms
[2022-03-23 18:53:58 main:574] : INFO : Epoch 1006 | loss: 0.0311723 | val_loss: 0.0311795 | Time: 2101.74 ms
[2022-03-23 18:54:00 main:574] : INFO : Epoch 1007 | loss: 0.0311746 | val_loss: 0.031174 | Time: 2131.79 ms
[2022-03-23 18:54:02 main:574] : INFO : Epoch 1008 | loss: 0.0311717 | val_loss: 0.0311759 | Time: 2088.26 ms
[2022-03-23 18:54:05 main:574] : INFO : Epoch 1009 | loss: 0.0311722 | val_loss: 0.0311762 | Time: 2162.35 ms
[2022-03-23 18:54:07 main:574] : INFO : Epoch 1010 | loss: 0.0311714 | val_loss: 0.0311753 | Time: 2111.22 ms
[2022-03-23 18:54:09 main:574] : INFO : Epoch 1011 | loss: 0.0311713 | val_loss: 0.031176 | Time: 2116.95 ms
[2022-03-23 18:54:11 main:574] : INFO : Epoch 1012 | loss: 0.0311702 | val_loss: 0.0311752 | Time: 2121.82 ms
[2022-03-23 18:54:13 main:574] : INFO : Epoch 1013 | loss: 0.0311704 | val_loss: 0.0311769 | Time: 2121.55 ms
[2022-03-23 18:54:15 main:574] : INFO : Epoch 1014 | loss: 0.031173 | val_loss: 0.0311749 | Time: 2109.71 ms
[2022-03-23 18:54:18 main:574] : INFO : Epoch 1015 | loss: 0.0311719 | val_loss: 0.0311755 | Time: 2118.33 ms
[2022-03-23 18:54:20 main:574] : INFO : Epoch 1016 | loss: 0.0311731 | val_loss: 0.031177 | Time: 2110.55 ms
[2022-03-23 18:54:22 main:574] : INFO : Epoch 1017 | loss: 0.0311723 | val_loss: 0.0311754 | Time: 2102.54 ms
[2022-03-23 18:54:24 main:574] : INFO : Epoch 1018 | loss: 0.0311746 | val_loss: 0.0311743 | Time: 2146.62 ms
[2022-03-23 18:54:26 main:574] : INFO : Epoch 1019 | loss: 0.0311711 | val_loss: 0.0311736 | Time: 2123.9 ms
[2022-03-23 18:54:28 main:574] : INFO : Epoch 1020 | loss: 0.0311728 | val_loss: 0.0311781 | Time: 2140.64 ms
[2022-03-23 18:54:31 main:574] : INFO : Epoch 1021 | loss: 0.0311715 | val_loss: 0.0311738 | Time: 2112.29 ms
[2022-03-23 18:54:33 main:574] : INFO : Epoch 1022 | loss: 0.0311699 | val_loss: 0.0311732 | Time: 2125.45 ms
[2022-03-23 18:54:35 main:574] : INFO : Epoch 1023 | loss: 0.0311688 | val_loss: 0.0311749 | Time: 2084.74 ms
[2022-03-23 18:54:37 main:574] : INFO : Epoch 1024 | loss: 0.0311686 | val_loss: 0.0311732 | Time: 2130.87 ms
[2022-03-23 18:54:39 main:574] : INFO : Epoch 1025 | loss: 0.031168 | val_loss: 0.0311746 | Time: 2141.75 ms
[2022-03-23 18:54:42 main:574] : INFO : Epoch 1026 | loss: 0.0311701 | val_loss: 0.0311753 | Time: 2142.77 ms
[2022-03-23 18:54:44 main:574] : INFO : Epoch 1027 | loss: 0.031169 | val_loss: 0.0311775 | Time: 2105.34 ms
[2022-03-23 18:54:46 main:574] : INFO : Epoch 1028 | loss: 0.0311718 | val_loss: 0.0311834 | Time: 2116.39 ms
[2022-03-23 18:54:48 main:574] : INFO : Epoch 1029 | loss: 0.0311698 | val_loss: 0.0311768 | Time: 2121.9 ms
[2022-03-23 18:54:50 main:574] : INFO : Epoch 1030 | loss: 0.0311696 | val_loss: 0.0311749 | Time: 2080.67 ms
[2022-03-23 18:54:52 main:574] : INFO : Epoch 1031 | loss: 0.0311694 | val_loss: 0.031174 | Time: 2110.84 ms
[2022-03-23 18:54:54 main:574] : INFO : Epoch 1032 | loss: 0.0311692 | val_loss: 0.0311743 | Time: 2078.17 ms
[2022-03-23 18:54:57 main:574] : INFO : Epoch 1033 | loss: 0.0311676 | val_loss: 0.0311745 | Time: 2089.81 ms
[2022-03-23 18:54:59 main:574] : INFO : Epoch 1034 | loss: 0.031168 | val_loss: 0.0311734 | Time: 2098.76 ms
[2022-03-23 18:55:01 main:574] : INFO : Epoch 1035 | loss: 0.0311675 | val_loss: 0.0311764 | Time: 2203.81 ms
[2022-03-23 18:55:03 main:574] : INFO : Epoch 1036 | loss: 0.0311671 | val_loss: 0.0311726 | Time: 2116.33 ms
[2022-03-23 18:55:06 main:574] : INFO : Epoch 1037 | loss: 0.0311682 | val_loss: 0.0311767 | Time: 2142.75 ms
[2022-03-23 18:55:08 main:574] : INFO : Epoch 1038 | loss: 0.0311734 | val_loss: 0.0311825 | Time: 2116.04 ms
[2022-03-23 18:55:10 main:574] : INFO : Epoch 1039 | loss: 0.0311746 | val_loss: 0.0311855 | Time: 2115.1 ms
[2022-03-23 18:55:12 main:574] : INFO : Epoch 1040 | loss: 0.031174 | val_loss: 0.0311824 | Time: 2131.43 ms
[2022-03-23 18:55:14 main:574] : INFO : Epoch 1041 | loss: 0.0311729 | val_loss: 0.0311751 | Time: 2097.3 ms
[2022-03-23 18:55:16 main:574] : INFO : Epoch 1042 | loss: 0.03117 | val_loss: 0.031177 | Time: 2090.48 ms
[2022-03-23 18:55:18 main:574] : INFO : Epoch 1043 | loss: 0.031169 | val_loss: 0.0311732 | Time: 2105.64 ms
[2022-03-23 18:55:20 main:574] : INFO : Epoch 1044 | loss: 0.0311719 | val_loss: 0.0311769 | Time: 1966.25 ms
[2022-03-23 18:55:23 main:574] : INFO : Epoch 1045 | loss: 0.0311673 | val_loss: 0.0311795 | Time: 2096.94 ms
[2022-03-23 18:55:25 main:574] : INFO : Epoch 1046 | loss: 0.031168 | val_loss: 0.031178 | Time: 2125.08 ms
THCudaCheck FAIL file=..\..\aten\src\THC\THCCachingHostAllocator.cpp line=278 error=719 : unspecified launch failure
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFBF36C3B29
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 03/23/22 18:55:25
Install Directory : H:\Program Files\BOINC\
Data Directory : H:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( H:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( H:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( H:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( H:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: H:\ProgramData\BOINC\slots\28;H:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000d50e0000 00000000003df000 H:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000f6520000 00000000001f0000 C:\Windows\SYSTEM32\ntdll.dll (6.2.18362.1171) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.18362.1171 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.1171
ModLoad: 00000000f5020000 00000000000b2000 C:\Windows\System32\KERNEL32.DLL (6.2.18362.1110) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.18362.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.1110
ModLoad: 00000000f3680000 00000000002a5000 C:\Windows\System32\KERNELBASE.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.18362.1171 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.1171
ModLoad: 00000000f45d0000 0000000000194000 C:\Windows\System32\USER32.dll (6.2.18362.1256) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f45a0000 0000000000021000 C:\Windows\System32\win32u.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.18362.1237 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1237
ModLoad: 00000000f4ff0000 0000000000026000 C:\Windows\System32\GDI32.dll (6.2.18362.1256) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.18362.1256 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1256
ModLoad: 00000000f3a30000 0000000000196000 C:\Windows\System32\gdi32full.dll (6.2.18362.1256) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.18362.1256 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1256
ModLoad: 00000000f4500000 000000000009e000 C:\Windows\System32\msvcp_win.dll (6.2.18362.1110) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.18362.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1110
ModLoad: 00000000f3520000 00000000000fa000 C:\Windows\System32\ucrtbase.dll (6.2.18362.1110) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.18362.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1110
ModLoad: 00000000f5d90000 00000000000a3000 C:\Windows\System32\ADVAPI32.dll (6.2.18362.752) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f63e0000 000000000009e000 C:\Windows\System32\msvcrt.dll (7.0.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.18362.1
ModLoad: 00000000f4ea0000 0000000000097000 C:\Windows\System32\sechost.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000f5e40000 0000000000120000 C:\Windows\System32\RPCRT4.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000e80d0000 0000000000058000 H:\ProgramData\BOINC\slots\28\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000dac30000 0000000000091000 C:\Windows\SYSTEM32\MSVCP140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000dd0a0000 00000000001f4000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.18362.1049) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.18362.1049 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1049
ModLoad: 00000000f12d0000 0000000000019000 C:\Windows\SYSTEM32\VCRUNTIME140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000f3080000 000000000000c000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000af5e0000 000000000abe7000 H:\ProgramData\BOINC\slots\28\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000ce6f0000 00000000002db000 H:\ProgramData\BOINC\slots\28\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d48a0000 000000000020d000 H:\ProgramData\BOINC\slots\28\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000f07b0000 0000000000040000 H:\ProgramData\BOINC\slots\28\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000003af10000 0000000024b49000 H:\ProgramData\BOINC\slots\28\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000e9e00000 0000000000045000 H:\ProgramData\BOINC\slots\28\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bfab0000 00000000034b2000 H:\ProgramData\BOINC\slots\28\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000f2ae0000 0000000000010000 H:\ProgramData\BOINC\slots\28\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000bca00000 00000000030a5000 H:\ProgramData\BOINC\slots\28\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000ab880000 0000000003d5b000 H:\ProgramData\BOINC\slots\28\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000a99b0000 0000000001eca000 H:\ProgramData\BOINC\slots\28\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000007c470000 000000001b3a4000 H:\ProgramData\BOINC\slots\28\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000a0810000 000000000919f000 H:\ProgramData\BOINC\slots\28\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 00000000f3490000 0000000000081000 C:\Windows\System32\bcryptPrimitives.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.18362.1237 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1237
ModLoad: 00000000f4d10000 000000000002e000 C:\Windows\System32\IMM32.DLL (6.2.18362.387) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.18362.387 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.387
ModLoad: 00000000f33f0000 0000000000011000 C:\Windows\System32\kernel.appcore.dll (6.2.18362.1171) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.18362.1171 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1171
ModLoad: 00000000f1990000 0000000000020000 C:\Windows\SYSTEM32\dxcore.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : DXCore.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000f39e0000 000000000004a000 C:\Windows\System32\cfgmgr32.dll (6.2.18362.387) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.18362.387 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.387
ModLoad: 00000000f60a0000 0000000000336000 C:\Windows\System32\combase.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f4790000 0000000000470000 C:\Windows\System32\Setupapi.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f3bf0000 0000000000026000 C:\Windows\System32\bcrypt.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f31b0000 000000000002a000 C:\Windows\SYSTEM32\DEVOBJ.dll (6.2.18362.387) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.18362.387 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.387
ModLoad: 00000000f3620000 000000000005c000 C:\Windows\System32\WINTRUST.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.18362.1237 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1237
ModLoad: 00000000f33d0000 0000000000012000 C:\Windows\System32\MSASN1.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000f3c20000 0000000000149000 C:\Windows\System32\CRYPT32.dll (6.2.18362.592) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f55d0000 00000000006e7000 C:\Windows\System32\Shell32.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.18362.900 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.900
ModLoad: 00000000f4f40000 00000000000a9000 C:\Windows\System32\shcore.dll (6.2.18362.1237) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f3d70000 0000000000781000 C:\Windows\System32\windows.storage.dll (6.2.18362.1171) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f3410000 0000000000023000 C:\Windows\System32\profapi.dll (6.2.18362.693) (-nosymbols- Symbols Loaded)
Linked PDB Filename : profapi.pdb
File Version : 10.0.18362.693 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.693
ModLoad: 00000000f3440000 000000000004a000 C:\Windows\System32\powrprof.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : powrprof.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f33c0000 0000000000010000 C:\Windows\System32\UMPDC.dll (-exported- Symbols Loaded)
Linked PDB Filename : UMPDC.pdb
ModLoad: 00000000f5f60000 0000000000052000 C:\Windows\System32\shlwapi.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f3bd0000 0000000000017000 C:\Windows\System32\cryptsp.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : cryptsp.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000ce480000 0000000000267000 C:\Windows\system32\nvcuda.dll (27.21.14.5709) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_87\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 27.21.14.5709
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.106 driver
Product Version : 27.21.14.5709
ModLoad: 00000000edcb0000 000000000000a000 C:\Windows\SYSTEM32\VERSION.DLL (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 000000009f2e0000 000000000152c000 C:\Windows\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_983b03c2be43c272\nvcuda64.dll (27.21.14.5709) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_87\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 27.21.14.5709
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.1.106 driver
Product Version : 27.21.14.5709
ModLoad: 00000000f6030000 000000000006f000 C:\Windows\System32\WS2_32.dll (6.2.18362.387) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000edae0000 000000000002f000 C:\Windows\SYSTEM32\cryptnet.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000f27d0000 000000000000c000 C:\Windows\SYSTEM32\cryptbase.dll (6.2.18362.1) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.18362.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.18362.1
ModLoad: 00000000f29f0000 0000000000099000 C:\Windows\system32\uxtheme.dll (6.2.18362.449) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000e1ba0000 00000000006d6000 C:\Windows\SYSTEM32\nvapi64.dll (27.21.14.5709) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_87\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 27.21.14.5709
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 27.21.14.5709
ModLoad: 00000000f5290000 0000000000157000 C:\Windows\System32\OLE32.DLL (6.2.18362.1082) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.18362.530 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.18362.530
ModLoad: 00000000f0960000 0000000000009000 H:\ProgramData\BOINC\slots\28\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000c8c80000 0000000000fca000 H:\ProgramData\BOINC\slots\28\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 4667, Write: 3929, Other 10118
- I/O Transfers Counters -
Read: 10828704, Write: 466861, Other 129396
- Paged Pool Usage -
QuotaPagedPoolUsage: 3536488, QuotaPeakPagedPoolUsage: 3537208
QuotaNonPagedPoolUsage: 59288, QuotaPeakNonPagedPoolUsage: 59696
- Virtual Memory Usage -
VirtualSize: 360108032, PeakVirtualSize: 977440768
- Pagefile Usage -
PagefileUsage: 360108032, PeakPagefileUsage: 360108032
- Working Set Size -
WorkingSetSize: -2043150336, PeakWorkingSetSize: -2033872896, PageFaultCount: 851991
*** Dump of thread ID 9664 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 3361562624.000000, User Time: 10250781696.000000, Wait Time: 1871711232.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFBF36C3B29
- Registers -
rax=0000000000000000 rbx=00000000b9a562f0 rcx=0000000000000000 rdx=0000000000000000 rsi=00000000250e43d0 rdi=0000000019930520
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000006000 r13=0000000000000000
r14=00000000250e5110 r15=00000000250e5090 rip=00000000f36c3b29 rsp=00000000250e4270 rbp=00000000250e4c78
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
- Callstack -
ChildEBP RetAddr Args to Child
250e4340 f12d6220 2dd794c4 00000000 250e5110 b4baa014 KERNELBASE!RaiseException+0x0
250e43a0 b51a5272 2dd79464 f3527de0 000007cb 00000116 VCRUNTIME140!_CxxThrowException+0x0
250e43f0 b51a56d4 00000002 f352e6e1 b81cd290 3e775f68 torch_cpu!at::Tensor::zero_+0x0
250e4c50 3dd39ebe 250e4e80 3e776850 3e776880 000002cf torch_cpu!_THError+0x0
250e4c90 3dd35412 250e4e80 250e4de0 62c743a0 3ebaa720 torch_cuda!__THCudaCheck+0x0
250e4cd0 b4e559ff 250e5000 250e4e80 250e5350 e81097b0 torch_cuda!at::native::set_storage_cuda_+0x0
250e5060 b4ff726f 3e3af630 b4baea63 00000001 e80eef8f torch_cpu!at::native::empty_cpu+0x0
250e50a0 b4fb0615 250e0000 250e5118 250e5310 e80d7ce6 torch_cpu!at::native::mkldnn_sigmoid_+0x0
250e50e0 b4fb204a 250e0000 250e5100 62c56100 3d076ae5 torch_cpu!at::native::mkldnn_sigmoid_+0x0
250e5170 b4fb0615 00000000 b4baea63 250e5250 250e5350 torch_cpu!at::native::mkldnn_sigmoid_+0x0
250e51b0 b5081eb0 250e0000 00000000 250e5338 00000001 torch_cpu!at::native::mkldnn_sigmoid_+0x0
250e5250 b5092eec 2e4332fe e8109c50 250e53b0 e80d3bd6 torch_cpu!at::bucketize_out+0x0
250e52a0 3d43f160 250e0000 00000000 250e53b0 00000000 torch_cpu!at::empty+0x0
250e5640 3d442de8 e8109c50 b4bb0528 a2a6d020 e8109c50 torch_cuda!at::native::cummin_helper_cuda+0x0
250e5820 3d444379 250e5930 cd878530 250e5d30 cd878548 torch_cuda!at::native::cummin_helper_cuda+0x0
250e5a60 3d443d64 250e0000 250e5d30 250e5b70 250e5d30 torch_cuda!at::native::cat_out_cuda+0x0
250e5c60 3dce1dd2 250e5cb8 00000001 250e5cb8 250e5d30 torch_cuda!at::native::cat_cuda+0x0
250e5ce0 3dcf4179 e9e00001 250e5d20 a2a672a8 04000000 torch_cuda!at::native::set_storage_cuda_+0x0
250e5d20 b5081d7d baab7950 250e6090 250e6060 00000000 torch_cuda!at::native::set_storage_cuda_+0x0
250e5da0 b5060d81 00000000 e80f3994 250e6570 cd86cb10 torch_cpu!at::bucketize_out+0x0
250e5de0 b4e6b4a5 250e5e20 a2a6cc20 250e5ef0 250e5f60 torch_cpu!at::_cat+0x0
250e5eb0 b51527bd 250e5f80 00000101 250e5f08 e8121ea8 torch_cpu!at::native::cat+0x0
250e5f30 b4c129ae 250e6590 b9f00640 6407adb0 e80d7ce6 torch_cpu!at::zeros_out+0x0
250e5f70 b5081d7d 250e5fb0 250e6020 00000000 f65606e1 torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
250e5ff0 b50875d1 250e6590 b9f00640 250e6570 cd86cb10 torch_cpu!at::bucketize_out+0x0
250e6030 b4e77d4c 250e62c0 00000000 250e62c0 00000000 torch_cpu!at::cat+0x0
250e61f0 b516c38d 250e6248 00000001 250e6248 250e62c0 torch_cpu!at::native::stack+0x0
250e6270 b4c129ae 250e6570 cd86cb10 00000000 00000000 torch_cpu!at::zeros_out+0x0
250e62b0 b5081d7d 250e62f0 250e6360 00000000 b7e21c8f torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
250e6330 b50d02e1 250e6460 e80f3720 250e6590 b9f00640 torch_cpu!at::bucketize_out+0x0
250e6370 b6463839 250e63f0 a2a6c800 a2a6c800 250e6480 torch_cpu!at::stack+0x0
250e6540 b4c129ae 00000000 b4baea63 00000000 250e6630 torch_cpu!torch::autograd::GraphRoot::apply+0x0
250e6580 b5081d7d 250e65c0 250e6630 b9f069e8 00000000 torch_cpu!torch::nn::functional::BatchNormFuncOptions::~BatchNormFuncOptions+0x0
250e6600 b50d02e1 250e6770 b4ba9cac c83b55c0 cd86cb10 torch_cpu!at::bucketize_out+0x0
250e6640 d50effb2 250e6670 bae533a0 bae533a0 250e6850 torch_cpu!at::stack+0x0
250e66e0 d50f2ef8 d50efe30 250e6720 250e6720 baabcc50 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::addCallback+0x0
250e6740 d50f3ea4 417e8010 f65606e1 250e68e0 62180000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::getSlot+0x0
250e6880 d50ed3d2 cdc95e50 f65606e1 250e68e0 d057b020 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0
250e68b0 d50f4541 00000000 d058a9f0 00000000 f35314cb mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::operator=+0x0
250e6a60 d50f743a 250e6b30 00000000 250e7570 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::markCompleted+0x0
250e6f60 d50fb950 250e7060 250e7060 00000000 1fc0e6b0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::then+0x0
250ffd00 d533c6f8 63bc4370 00000000 00000000 636bbb80 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
250ffd40 f5037c24 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
250ffd70 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
250ffdf0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8612 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1871711232.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000255ffe20 rsi=0000000000000000 rdi=0000000000000064
r8=0000000000000007 r9=0000000000000000 r10=00000000255feb10 r11=0000000062451401 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000f65bcd74 rsp=00000000255ffdf8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
255ffdf0 f36cfc21 255ffeb8 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0
255ffe90 d511a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
255ffec0 f5037c24 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
255ffef0 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
255fff70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 3168 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1871711232.000000
- Registers -
rax=000000000000005b rbx=000000000000000b rcx=000000000000000b rdx=00000000256ff4a0 rsi=0000000000000000 rdi=000000000000000b
r8=0000000000000001 r9=0000000000000000 r10=0000000000000000 r11=00000000256ff410 r12=0000000000000064 r13=00000000256ff4a0
r14=00000000256ff190 r15=0000000000000000 rip=00000000f65bd244 rsp=00000000256ff138 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
256ff130 f36d2ae7 f52b3880 f6593e54 00000000 24f88000 ntdll!NtWaitForMultipleObjects+0x0
256ff430 f36d29ce 9f7d4b48 62191ef0 0000000b 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
256ff470 9f48bb54 63decf10 00000000 63de7ed0 00000000 KERNELBASE!WaitForMultipleObjects+0x0
256ff6b0 9f5209ed 00000000 00000004 00000000 00000000 nvcuda64!+0x0
256ff760 9f48b523 75ebd510 75bb324d 642e1990 00000000 nvcuda64!cuProfilerStop+0x0
256ff790 9f79a750 642e1990 00000000 00000000 00000000 nvcuda64!+0x0
256ff7c0 f5037c24 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
256ff7f0 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
256ff870 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 7068 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1871711232.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002c8 rdx=0000000000000000 rsi=00000000257ff8d8 rdi=00000000000002c8
r8=00000000642e1360 r9=000000009f79a700 r10=0000000000000000 r11=0000000000000000 r12=0000000063dd0870 r13=0000000000000000
r14=00000000000002c8 r15=0000000000000001 rip=00000000f65bc774 rsp=00000000257ff8a8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
257ff8a0 f36c1e93 00000000 00000000 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
257ff940 9f69c0e7 00000000 f36add05 00000000 000002c8 KERNELBASE!WaitForSingleObjectEx+0x0
257ff9b0 9f48b523 00000001 648c5980 642e1360 00000000 nvcuda64!cuProfilerStop+0x0
257ff9e0 9f79a750 642e1360 00000000 00000000 00000000 nvcuda64!+0x0
257ffa10 f5037c24 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
257ffa40 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
257ffac0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 5324 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1871711104.000000
- Registers -
rax=00000000000001ca rbx=0000000000000000 rcx=000000006407a258 rdx=00000000258ffa18 rsi=00000000258ff980 rdi=0000000000000000
r8=0000000000000000 r9=000000000000004a r10=000000007fe40101 r11=00000000258ff9b0 r12=00000000258ffa18 r13=0000000000000000
r14=000000006407a258 r15=000000006407d5c8 rip=00000000f65c0024 rsp=00000000258ff958 rbp=00000000258ff9b0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
258ff950 f6580e95 00000102 00000102 00000000 258ff980 ntdll!ZwWaitForAlertByThreadId+0x0
258ff9c0 f36fc849 6407adb0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
258ff9f0 9f41223a 6407adb0 6407adb0 6407adb0 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
258ffa50 9f48b523 00000004 75c3b77f 75c3b77f 00000000 nvcuda64!+0x0
258ffa80 9f79a750 642e17e0 00000000 00000000 00000000 nvcuda64!+0x0
258ffab0 f5037c24 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
258ffae0 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
258ffb60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 14916 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 502031264.000000, User Time: 7647968768.000000, Wait Time: 1871711232.000000
- Registers -
rax=00000000000001ca rbx=0000000000000000 rcx=00000000b0659668 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
r8=000000005f0ccf60 r9=0000000000000080 r10=0000000000000000 r11=000000000000ffff r12=0000000000000003 r13=00000000000001c0
r14=00000000b0659618 r15=0000000000000000 rip=00000000f65c0024 rsp=00000000259ff208 rbp=00000000b0659668
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
259ff200 f6580c10 259ff330 bae2d378 259ff570 b67c092a ntdll!ZwWaitForAlertByThreadId+0x0
259ff270 f36aec8d b0659610 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
259ff2b0 dac6ba8d baac9670 baac9670 baac9670 00000000 KERNELBASE!SleepConditionVariableSRW+0x0
259ff2f0 dac42848 c8646110 00000000 00000000 b67c05b4 MSVCP140!__crtSetThreadpoolWait+0x0
259ff320 dac42710 00000000 dac42f17 00000000 c8646110 MSVCP140!std::_Winerror_message+0x0
259ff390 b67c064b c86460f0 000001c0 b0659610 b0659610 MSVCP140!std::_Winerror_message+0x0
259ff3d0 b67c25b1 d04594d0 ffffffff c86460f0 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
259ff6c0 b67c2471 b9ce6d10 259ff700 b9ce6d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
259ff710 b67b9744 00080001 d064bec0 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
259ff740 f35410b2 d064bec0 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
259ff770 f5037c24 00000000 00000000 00000000 00000000 ucrtbase!_beginthreadex+0x0
259ff7a0 f658d4d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
259ff820 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)