Task 13627859
| Name | ParityModified-1647045759-28115-2-0_0 |
| Workunit | 10721131 |
| Created | 23 Mar 2022, 4:24:38 UTC |
| Sent | 30 Mar 2022, 5:01:33 UTC |
| Report deadline | 7 Apr 2022, 5:01:33 UTC |
| Received | 13 Apr 2022, 7:35:19 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 12436 |
| Run time | 1 hours 24 min 58 sec |
| CPU time | 1 hours 23 min 25 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 2,470.64 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.63 GB |
| Peak swap size | 3.59 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> INFO : Epoch 1903 | loss: 0.0311884 | val_loss: 0.0312027 | Time: 2791.98 ms [2022-04-11 15:02:56 main:574] : INFO : Epoch 1904 | loss: 0.0311862 | val_loss: 0.0312051 | Time: 2507.29 ms [2022-04-11 15:02:59 main:574] : INFO : Epoch 1905 | loss: 0.0311872 | val_loss: 0.0312025 | Time: 2815.03 ms [2022-04-11 15:03:02 main:574] : INFO : Epoch 1906 | loss: 0.0311864 | val_loss: 0.0312032 | Time: 2590.88 ms [2022-04-11 15:03:04 main:574] : INFO : Epoch 1907 | loss: 0.031185 | val_loss: 0.0312014 | Time: 2398.75 ms [2022-04-11 15:03:07 main:574] : INFO : Epoch 1908 | loss: 0.0311817 | val_loss: 0.031202 | Time: 2559.98 ms [2022-04-11 15:03:09 main:574] : INFO : Epoch 1909 | loss: 0.0311814 | val_loss: 0.0312011 | Time: 2513.67 ms [2022-04-11 15:03:12 main:574] : INFO : Epoch 1910 | loss: 0.0311797 | val_loss: 0.0312022 | Time: 2584.16 ms [2022-04-11 15:03:15 main:574] : INFO : Epoch 1911 | loss: 0.0311811 | val_loss: 0.0312036 | Time: 2677.63 ms [2022-04-11 15:03:17 main:574] : INFO : Epoch 1912 | loss: 0.0311804 | val_loss: 0.0311999 | Time: 2503.78 ms [2022-04-11 15:03:20 main:574] : INFO : Epoch 1913 | loss: 0.0311808 | val_loss: 0.0312004 | Time: 2539.58 ms [2022-04-11 15:03:32 main:574] : INFO : Epoch 1914 | loss: 0.0311803 | val_loss: 0.0312013 | Time: 12717 ms [2022-04-11 15:03:35 main:574] : INFO : Epoch 1915 | loss: 0.0311813 | val_loss: 0.0312002 | Time: 2802.04 ms [2022-04-11 15:03:38 main:574] : INFO : Epoch 1916 | loss: 0.0311833 | val_loss: 0.0312012 | Time: 2399.52 ms [2022-04-11 15:03:40 main:574] : INFO : Epoch 1917 | loss: 0.0311854 | val_loss: 0.0312019 | Time: 2509.82 ms [2022-04-11 15:03:43 main:574] : INFO : Epoch 1918 | loss: 0.0311833 | val_loss: 0.0312002 | Time: 2402.32 ms [2022-04-11 15:03:45 main:574] : INFO : Epoch 1919 | loss: 0.0311829 | val_loss: 0.0312032 | Time: 2779.97 ms [2022-04-11 15:03:48 main:574] : INFO : Epoch 1920 | loss: 0.0311841 | val_loss: 0.0312015 | Time: 2950.5 ms [2022-04-11 15:03:51 main:574] : INFO : Epoch 1921 | loss: 0.0311868 | val_loss: 0.0312069 | Time: 2486.16 ms [2022-04-11 15:03:53 main:574] : INFO : Epoch 1922 | loss: 0.0311927 | val_loss: 0.0312028 | Time: 2611.21 ms [2022-04-11 15:03:56 main:574] : INFO : Epoch 1923 | loss: 0.031194 | val_loss: 0.0312161 | Time: 3048.26 ms [2022-04-11 15:03:59 main:574] : INFO : Epoch 1924 | loss: 0.0311912 | val_loss: 0.0312033 | Time: 2822.55 ms [2022-04-11 15:04:02 main:574] : INFO : Epoch 1925 | loss: 0.0311863 | val_loss: 0.0312029 | Time: 2764.86 ms [2022-04-11 15:04:05 main:574] : INFO : Epoch 1926 | loss: 0.0311869 | val_loss: 0.0312022 | Time: 2846.97 ms [2022-04-11 15:04:07 main:574] : INFO : Epoch 1927 | loss: 0.0311867 | val_loss: 0.0312016 | Time: 2471.59 ms [2022-04-11 15:04:10 main:574] : INFO : Epoch 1928 | loss: 0.0311854 | val_loss: 0.0312021 | Time: 2478.09 ms [2022-04-11 15:04:12 main:574] : INFO : Epoch 1929 | loss: 0.0311847 | val_loss: 0.0312012 | Time: 2605.28 ms [2022-04-11 15:04:15 main:574] : INFO : Epoch 1930 | loss: 0.0311847 | val_loss: 0.0311998 | Time: 2385.98 ms [2022-04-11 15:04:18 main:574] : INFO : Epoch 1931 | loss: 0.031187 | val_loss: 0.0311993 | Time: 2695.4 ms [2022-04-11 15:04:20 main:574] : INFO : Epoch 1932 | loss: 0.031184 | val_loss: 0.0311998 | Time: 2841.57 ms [2022-04-11 15:04:23 main:574] : INFO : Epoch 1933 | loss: 0.0311849 | val_loss: 0.0312011 | Time: 2442.5 ms [2022-04-11 15:04:26 main:574] : INFO : Epoch 1934 | loss: 0.0311858 | val_loss: 0.0312011 | Time: 2698.76 ms [2022-04-11 15:04:28 main:574] : INFO : Epoch 1935 | loss: 0.0311871 | val_loss: 0.0312005 | Time: 2610.97 ms [2022-04-11 15:04:31 main:574] : INFO : Epoch 1936 | loss: 0.0311893 | val_loss: 0.0312028 | Time: 2399.34 ms [2022-04-11 15:04:33 main:574] : INFO : Epoch 1937 | loss: 0.0311884 | val_loss: 0.0312004 | Time: 2736.54 ms [2022-04-11 15:04:36 main:574] : INFO : Epoch 1938 | loss: 0.0311874 | val_loss: 0.0312033 | Time: 3184.49 ms [2022-04-11 15:04:39 main:574] : INFO : Epoch 1939 | loss: 0.0311882 | val_loss: 0.0312024 | Time: 2357.41 ms [2022-04-11 15:04:41 main:574] : INFO : Epoch 1940 | loss: 0.0311894 | val_loss: 0.0311995 | Time: 2599.39 ms [2022-04-11 15:04:44 main:574] : INFO : Epoch 1941 | loss: 0.0311876 | val_loss: 0.0312018 | Time: 2614.36 ms [2022-04-11 15:04:47 main:574] : INFO : Epoch 1942 | loss: 0.0311889 | val_loss: 0.0312024 | Time: 2703.27 ms [2022-04-11 15:04:49 main:574] : INFO : Epoch 1943 | loss: 0.0311875 | val_loss: 0.0312048 | Time: 2604.76 ms [2022-04-11 15:04:52 main:574] : INFO : Epoch 1944 | loss: 0.0311894 | val_loss: 0.0312 | Time: 2719.01 ms [2022-04-11 15:04:54 main:574] : INFO : Epoch 1945 | loss: 0.0311877 | val_loss: 0.0312013 | Time: 2367.95 ms [2022-04-11 15:04:57 main:574] : INFO : Epoch 1946 | loss: 0.0311856 | val_loss: 0.0312005 | Time: 2849.56 ms [2022-04-11 15:05:00 main:574] : INFO : Epoch 1947 | loss: 0.0311846 | val_loss: 0.0312006 | Time: 2688.53 ms [2022-04-11 15:05:02 main:574] : INFO : Epoch 1948 | loss: 0.0311868 | val_loss: 0.0312037 | Time: 2341.96 ms [2022-04-11 15:05:05 main:574] : INFO : Epoch 1949 | loss: 0.0311877 | val_loss: 0.0312052 | Time: 2702.87 ms [2022-04-11 15:05:07 main:574] : INFO : Epoch 1950 | loss: 0.0311873 | val_loss: 0.0312031 | Time: 2391.34 ms [2022-04-11 15:05:10 main:574] : INFO : Epoch 1951 | loss: 0.0311863 | val_loss: 0.0312008 | Time: 2531.47 ms [2022-04-11 15:05:13 main:574] : INFO : Epoch 1952 | loss: 0.0311862 | val_loss: 0.0312066 | Time: 2682.6 ms [2022-04-11 15:05:15 main:574] : INFO : Epoch 1953 | loss: 0.0311871 | val_loss: 0.0312064 | Time: 2576.01 ms [2022-04-11 15:05:18 main:574] : INFO : Epoch 1954 | loss: 0.031185 | val_loss: 0.0312037 | Time: 2782.47 ms [2022-04-11 15:05:21 main:574] : INFO : Epoch 1955 | loss: 0.0311847 | val_loss: 0.0312013 | Time: 2596.67 ms [2022-04-11 15:05:23 main:574] : INFO : Epoch 1956 | loss: 0.0311837 | val_loss: 0.0312032 | Time: 2335.68 ms [2022-04-11 15:05:26 main:574] : INFO : Epoch 1957 | loss: 0.0311837 | val_loss: 0.0312019 | Time: 2730.06 ms [2022-04-11 15:05:29 main:574] : INFO : Epoch 1958 | loss: 0.0311829 | val_loss: 0.0312007 | Time: 2964.47 ms [2022-04-11 15:05:31 main:574] : INFO : Epoch 1959 | loss: 0.0311841 | val_loss: 0.0312057 | Time: 2342.89 ms [2022-04-11 15:05:34 main:574] : INFO : Epoch 1960 | loss: 0.0311872 | val_loss: 0.0312043 | Time: 3212.49 ms [2022-04-11 15:05:37 main:574] : INFO : Epoch 1961 | loss: 0.0311853 | val_loss: 0.0312074 | Time: 3110.44 ms [2022-04-11 15:05:40 main:574] : INFO : Epoch 1962 | loss: 0.0311867 | val_loss: 0.0312089 | Time: 2644.71 ms [2022-04-11 15:05:43 main:574] : INFO : Epoch 1963 | loss: 0.0311898 | val_loss: 0.0312059 | Time: 2501.39 ms [2022-04-11 15:05:45 main:574] : INFO : Epoch 1964 | loss: 0.0311888 | val_loss: 0.031204 | Time: 2614.22 ms [2022-04-11 15:05:48 main:574] : INFO : Epoch 1965 | loss: 0.0311874 | val_loss: 0.031203 | Time: 2539.94 ms [2022-04-11 15:05:50 main:574] : INFO : Epoch 1966 | loss: 0.0311863 | val_loss: 0.0312039 | Time: 2586.57 ms [2022-04-11 15:05:53 main:574] : INFO : Epoch 1967 | loss: 0.0311852 | val_loss: 0.031202 | Time: 2604.72 ms [2022-04-11 15:05:55 main:574] : INFO : Epoch 1968 | loss: 0.0311849 | val_loss: 0.0312028 | Time: 2347.63 ms [2022-04-11 15:05:58 main:574] : INFO : Epoch 1969 | loss: 0.0311855 | val_loss: 0.0312036 | Time: 2578.44 ms [2022-04-11 15:06:01 main:574] : INFO : Epoch 1970 | loss: 0.0311868 | val_loss: 0.0312076 | Time: 2805.51 ms [2022-04-11 15:06:03 main:574] : INFO : Epoch 1971 | loss: 0.0311866 | val_loss: 0.0312035 | Time: 2316.35 ms [2022-04-11 15:06:06 main:574] : INFO : Epoch 1972 | loss: 0.0311836 | val_loss: 0.0312032 | Time: 2603.23 ms [2022-04-11 15:06:08 main:574] : INFO : Epoch 1973 | loss: 0.0311823 | val_loss: 0.0312024 | Time: 2666.7 ms [2022-04-11 15:06:11 main:574] : INFO : Epoch 1974 | loss: 0.0311826 | val_loss: 0.0312014 | Time: 2529.9 ms [2022-04-11 15:06:14 main:574] : INFO : Epoch 1975 | loss: 0.0311828 | val_loss: 0.0312041 | Time: 2832.8 ms [2022-04-11 15:06:16 main:574] : INFO : Epoch 1976 | loss: 0.0311837 | val_loss: 0.0312037 | Time: 2612.22 ms [2022-04-11 15:06:19 main:574] : INFO : Epoch 1977 | loss: 0.031188 | val_loss: 0.0312069 | Time: 2577.36 ms [2022-04-11 15:06:22 main:574] : INFO : Epoch 1978 | loss: 0.0311894 | val_loss: 0.0312065 | Time: 2808.77 ms [2022-04-11 15:06:24 main:574] : INFO : Epoch 1979 | loss: 0.0311903 | val_loss: 0.0312067 | Time: 2824.04 ms [2022-04-11 15:06:27 main:574] : INFO : Epoch 1980 | loss: 0.0311906 | val_loss: 0.0312067 | Time: 2520.55 ms [2022-04-11 15:06:29 main:574] : INFO : Epoch 1981 | loss: 0.0311882 | val_loss: 0.0312077 | Time: 2521.34 ms [2022-04-11 15:06:33 main:574] : INFO : Epoch 1982 | loss: 0.031186 | val_loss: 0.031203 | Time: 3076.58 ms [2022-04-11 15:06:36 main:574] : INFO : Epoch 1983 | loss: 0.0311868 | val_loss: 0.0312044 | Time: 3454.81 ms [2022-04-11 15:06:39 main:574] : INFO : Epoch 1984 | loss: 0.0311891 | val_loss: 0.0312033 | Time: 2786.26 ms [2022-04-11 15:06:42 main:574] : INFO : Epoch 1985 | loss: 0.0311884 | val_loss: 0.0312018 | Time: 3298.78 ms [2022-04-11 15:06:45 main:574] : INFO : Epoch 1986 | loss: 0.0311872 | val_loss: 0.031204 | Time: 2934.65 ms [2022-04-11 15:06:48 main:574] : INFO : Epoch 1987 | loss: 0.0311867 | val_loss: 0.0312038 | Time: 2511.41 ms [2022-04-11 15:06:50 main:574] : INFO : Epoch 1988 | loss: 0.0311861 | val_loss: 0.0312017 | Time: 2726.58 ms [2022-04-11 15:06:53 main:574] : INFO : Epoch 1989 | loss: 0.0311852 | val_loss: 0.0312092 | Time: 2510.04 ms [2022-04-11 15:06:55 main:574] : INFO : Epoch 1990 | loss: 0.0311836 | val_loss: 0.0312035 | Time: 2441.07 ms [2022-04-11 15:06:58 main:574] : INFO : Epoch 1991 | loss: 0.0311844 | val_loss: 0.0312038 | Time: 2649.16 ms [2022-04-11 15:07:01 main:574] : INFO : Epoch 1992 | loss: 0.0311831 | val_loss: 0.0312033 | Time: 2780.67 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-12 12:49:56 main:435] : INFO : Set logging level to 1 [2022-04-12 12:49:56 main:441] : INFO : Running in BOINC Client mode [2022-04-12 12:49:56 main:444] : INFO : Resolving all filenames [2022-04-12 12:49:56 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-12 12:49:56 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-12 12:49:56 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-12 12:49:56 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-12 12:49:56 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-12 12:49:56 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-12 12:49:56 main:474] : INFO : Configuration: [2022-04-12 12:49:56 main:475] : INFO : Model type: GRU [2022-04-12 12:49:56 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-12 12:49:56 main:477] : INFO : Max Epochs: 2048 [2022-04-12 12:49:56 main:478] : INFO : Batch Size: 128 [2022-04-12 12:49:56 main:479] : INFO : Learning Rate: 0.01 [2022-04-12 12:49:56 main:480] : INFO : Patience: 10 [2022-04-12 12:49:56 main:481] : INFO : Hidden Width: 12 [2022-04-12 12:49:56 main:482] : INFO : # Recurrent Layers: 4 [2022-04-12 12:49:56 main:483] : INFO : # Backend Layers: 4 [2022-04-12 12:49:56 main:484] : INFO : # Threads: 1 [2022-04-12 12:49:56 main:486] : INFO : Preparing Dataset [2022-04-12 12:49:56 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-12 12:49:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-12 12:49:59 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-12 12:49:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-12 12:49:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-12 12:49:59 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-12 12:49:59 main:494] : INFO : Creating Model [2022-04-12 12:49:59 main:507] : INFO : Preparing config file [2022-04-12 12:49:59 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-12 12:49:59 main:512] : INFO : Loading config [2022-04-12 12:49:59 main:514] : INFO : Loading state [2022-04-12 12:50:11 main:559] : INFO : Loading DataLoader into Memory [2022-04-12 12:50:11 main:562] : INFO : Starting Training [2022-04-12 12:50:13 main:574] : INFO : Epoch 1968 | loss: 0.0312129 | val_loss: 0.0312083 | Time: 2772.35 ms [2022-04-12 12:50:16 main:574] : INFO : Epoch 1969 | loss: 0.0311895 | val_loss: 0.0312033 | Time: 2194.82 ms [2022-04-12 12:50:18 main:574] : INFO : Epoch 1970 | loss: 0.0311879 | val_loss: 0.0312042 | Time: 2203.69 ms [2022-04-12 12:50:20 main:574] : INFO : Epoch 1971 | loss: 0.0311856 | val_loss: 0.0312005 | Time: 2309.62 ms [2022-04-12 12:50:23 main:574] : INFO : Epoch 1972 | loss: 0.0311844 | val_loss: 0.0312007 | Time: 2423.73 ms [2022-04-12 12:50:25 main:574] : INFO : Epoch 1973 | loss: 0.0311848 | val_loss: 0.0312017 | Time: 2340.33 ms [2022-04-12 12:50:27 main:574] : INFO : Epoch 1974 | loss: 0.0311846 | val_loss: 0.0312011 | Time: 2416.17 ms [2022-04-12 12:50:30 main:574] : INFO : Epoch 1975 | loss: 0.0311828 | val_loss: 0.0312037 | Time: 2428.08 ms [2022-04-12 12:50:32 main:574] : INFO : Epoch 1976 | loss: 0.0311859 | val_loss: 0.0312068 | Time: 2356.98 ms [2022-04-12 12:50:34 main:574] : INFO : Epoch 1977 | loss: 0.0311845 | val_loss: 0.0312025 | Time: 2385.42 ms [2022-04-12 12:50:37 main:574] : INFO : Epoch 1978 | loss: 0.0311837 | val_loss: 0.031211 | Time: 2309.58 ms [2022-04-12 12:50:39 main:574] : INFO : Epoch 1979 | loss: 0.0311868 | val_loss: 0.0312054 | Time: 2367.62 ms [2022-04-12 12:50:42 main:574] : INFO : Epoch 1980 | loss: 0.0311859 | val_loss: 0.0312002 | Time: 2442.32 ms [2022-04-12 12:50:44 main:574] : INFO : Epoch 1981 | loss: 0.0311847 | val_loss: 0.0312015 | Time: 2283.57 ms [2022-04-12 12:50:46 main:574] : INFO : Epoch 1982 | loss: 0.0311833 | val_loss: 0.0312005 | Time: 2520.08 ms [2022-04-12 12:50:49 main:574] : INFO : Epoch 1983 | loss: 0.0311823 | val_loss: 0.0312003 | Time: 2355.04 ms [2022-04-12 12:50:51 main:574] : INFO : Epoch 1984 | loss: 0.0311832 | val_loss: 0.0312044 | Time: 2488.63 ms [2022-04-12 12:50:54 main:574] : INFO : Epoch 1985 | loss: 0.0311832 | val_loss: 0.0312052 | Time: 2505.49 ms [2022-04-12 12:50:58 main:574] : INFO : Epoch 1986 | loss: 0.0311834 | val_loss: 0.0312028 | Time: 4084.66 ms [2022-04-12 12:51:22 main:574] : INFO : Epoch 1987 | loss: 0.0311855 | val_loss: 0.0312042 | Time: 24104.5 ms [2022-04-12 12:51:24 main:574] : INFO : Epoch 1988 | loss: 0.031185 | val_loss: 0.0312032 | Time: 2405.43 ms [2022-04-12 12:51:27 main:574] : INFO : Epoch 1989 | loss: 0.0311833 | val_loss: 0.0312046 | Time: 2577.19 ms [2022-04-12 12:51:30 main:574] : INFO : Epoch 1990 | loss: 0.0311849 | val_loss: 0.0312083 | Time: 2656.15 ms [2022-04-12 12:51:32 main:574] : INFO : Epoch 1991 | loss: 0.0311844 | val_loss: 0.0312026 | Time: 2380.24 ms [2022-04-12 12:51:35 main:574] : INFO : Epoch 1992 | loss: 0.031186 | val_loss: 0.0312036 | Time: 2730.58 ms [2022-04-12 12:51:37 main:574] : INFO : Epoch 1993 | loss: 0.0311852 | val_loss: 0.0312034 | Time: 2452.99 ms [2022-04-12 12:51:40 main:574] : INFO : Epoch 1994 | loss: 0.0311857 | val_loss: 0.0312019 | Time: 2440.2 ms [2022-04-12 12:51:42 main:574] : INFO : Epoch 1995 | loss: 0.031184 | val_loss: 0.0312009 | Time: 2796.7 ms [2022-04-12 12:51:45 main:574] : INFO : Epoch 1996 | loss: 0.0311846 | val_loss: 0.0312013 | Time: 2516.12 ms [2022-04-12 12:51:48 main:574] : INFO : Epoch 1997 | loss: 0.0311847 | val_loss: 0.031203 | Time: 2542.09 ms [2022-04-12 12:51:50 main:574] : INFO : Epoch 1998 | loss: 0.0311849 | val_loss: 0.0312056 | Time: 2596.11 ms [2022-04-12 12:51:53 main:574] : INFO : Epoch 1999 | loss: 0.031187 | val_loss: 0.0312023 | Time: 2551.41 ms [2022-04-12 12:51:56 main:574] : INFO : Epoch 2000 | loss: 0.0311866 | val_loss: 0.0312027 | Time: 3276.6 ms [2022-04-12 12:51:59 main:574] : INFO : Epoch 2001 | loss: 0.0311865 | val_loss: 0.0312031 | Time: 3134.08 ms [2022-04-12 12:52:02 main:574] : INFO : Epoch 2002 | loss: 0.0311871 | val_loss: 0.0312085 | Time: 2686.95 ms [2022-04-12 12:52:04 main:574] : INFO : Epoch 2003 | loss: 0.0311882 | val_loss: 0.0312029 | Time: 2519.38 ms [2022-04-12 12:52:07 main:574] : INFO : Epoch 2004 | loss: 0.031185 | val_loss: 0.0312026 | Time: 2876.87 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-12 13:52:26 main:435] : INFO : Set logging level to 1 [2022-04-12 13:52:26 main:441] : INFO : Running in BOINC Client mode [2022-04-12 13:52:26 main:444] : INFO : Resolving all filenames [2022-04-12 13:52:26 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-12 13:52:26 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-12 13:52:26 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-12 13:52:26 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-12 13:52:26 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-12 13:52:26 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-12 13:52:26 main:474] : INFO : Configuration: [2022-04-12 13:52:26 main:475] : INFO : Model type: GRU [2022-04-12 13:52:26 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-12 13:52:26 main:477] : INFO : Max Epochs: 2048 [2022-04-12 13:52:26 main:478] : INFO : Batch Size: 128 [2022-04-12 13:52:26 main:479] : INFO : Learning Rate: 0.01 [2022-04-12 13:52:26 main:480] : INFO : Patience: 10 [2022-04-12 13:52:26 main:481] : INFO : Hidden Width: 12 [2022-04-12 13:52:26 main:482] : INFO : # Recurrent Layers: 4 [2022-04-12 13:52:26 main:483] : INFO : # Backend Layers: 4 [2022-04-12 13:52:26 main:484] : INFO : # Threads: 1 [2022-04-12 13:52:26 main:486] : INFO : Preparing Dataset [2022-04-12 13:52:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-12 13:52:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-12 13:52:28 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-12 13:52:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-12 13:52:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-12 13:52:28 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-12 13:52:28 main:494] : INFO : Creating Model [2022-04-12 13:52:28 main:507] : INFO : Preparing config file [2022-04-12 13:52:28 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-12 13:52:28 main:512] : INFO : Loading config [2022-04-12 13:52:28 main:514] : INFO : Loading state [2022-04-12 13:52:28 main:559] : INFO : Loading DataLoader into Memory [2022-04-12 13:52:28 main:562] : INFO : Starting Training [2022-04-12 13:52:31 main:574] : INFO : Epoch 1968 | loss: 0.0312154 | val_loss: 0.0312106 | Time: 2397.6 ms [2022-04-12 13:52:33 main:574] : INFO : Epoch 1969 | loss: 0.0311895 | val_loss: 0.031204 | Time: 2190.27 ms [2022-04-12 13:52:35 main:574] : INFO : Epoch 1970 | loss: 0.0311865 | val_loss: 0.0312036 | Time: 2206.19 ms [2022-04-12 13:52:38 main:574] : INFO : Epoch 1971 | loss: 0.0311857 | val_loss: 0.0312045 | Time: 2286.32 ms [2022-04-12 13:52:40 main:574] : INFO : Epoch 1972 | loss: 0.0311851 | val_loss: 0.0312023 | Time: 2329.04 ms [2022-04-12 13:52:42 main:574] : INFO : Epoch 1973 | loss: 0.0311833 | val_loss: 0.0312047 | Time: 2383.19 ms [2022-04-12 13:52:45 main:574] : INFO : Epoch 1974 | loss: 0.0311844 | val_loss: 0.0312054 | Time: 2303.48 ms [2022-04-12 13:52:47 main:574] : INFO : Epoch 1975 | loss: 0.0311829 | val_loss: 0.0312039 | Time: 2330.1 ms [2022-04-12 13:52:49 main:574] : INFO : Epoch 1976 | loss: 0.0311819 | val_loss: 0.0312007 | Time: 2317.81 ms [2022-04-12 13:52:52 main:574] : INFO : Epoch 1977 | loss: 0.0311811 | val_loss: 0.0312071 | Time: 2361.76 ms [2022-04-12 13:52:54 main:574] : INFO : Epoch 1978 | loss: 0.0311816 | val_loss: 0.0312007 | Time: 2397.42 ms [2022-04-12 13:52:56 main:574] : INFO : Epoch 1979 | loss: 0.031182 | val_loss: 0.0312 | Time: 2310.07 ms [2022-04-12 13:52:59 main:574] : INFO : Epoch 1980 | loss: 0.0311812 | val_loss: 0.0311993 | Time: 2398.83 ms [2022-04-12 13:53:01 main:574] : INFO : Epoch 1981 | loss: 0.0311823 | val_loss: 0.0312009 | Time: 2337.21 ms [2022-04-12 13:53:03 main:574] : INFO : Epoch 1982 | loss: 0.0311819 | val_loss: 0.0311985 | Time: 2431.52 ms [2022-04-12 13:53:16 main:574] : INFO : Epoch 1983 | loss: 0.0311814 | val_loss: 0.0312021 | Time: 12406.3 ms [2022-04-12 13:53:18 main:574] : INFO : Epoch 1984 | loss: 0.0311823 | val_loss: 0.0311997 | Time: 2291.12 ms [2022-04-12 13:53:21 main:574] : INFO : Epoch 1985 | loss: 0.0311807 | val_loss: 0.0311982 | Time: 2318.72 ms [2022-04-12 13:53:23 main:574] : INFO : Epoch 1986 | loss: 0.0311797 | val_loss: 0.0312003 | Time: 2354.82 ms [2022-04-12 13:53:25 main:574] : INFO : Epoch 1987 | loss: 0.0311804 | val_loss: 0.0312019 | Time: 2565.42 ms [2022-04-12 13:53:28 main:574] : INFO : Epoch 1988 | loss: 0.03118 | val_loss: 0.0312006 | Time: 2536 ms [2022-04-12 13:53:30 main:574] : INFO : Epoch 1989 | loss: 0.0311798 | val_loss: 0.0312032 | Time: 2491.43 ms [2022-04-12 13:53:33 main:574] : INFO : Epoch 1990 | loss: 0.0311818 | val_loss: 0.031202 | Time: 2592.38 ms [2022-04-12 13:53:35 main:574] : INFO : Epoch 1991 | loss: 0.0311824 | val_loss: 0.0312016 | Time: 2358.41 ms [2022-04-12 13:53:38 main:574] : INFO : Epoch 1992 | loss: 0.0311819 | val_loss: 0.0312024 | Time: 2622.19 ms [2022-04-12 13:53:41 main:574] : INFO : Epoch 1993 | loss: 0.0311821 | val_loss: 0.0312033 | Time: 2590.26 ms [2022-04-12 13:53:43 main:574] : INFO : Epoch 1994 | loss: 0.0311828 | val_loss: 0.0312013 | Time: 2420.04 ms [2022-04-12 13:53:46 main:574] : INFO : Epoch 1995 | loss: 0.0311811 | val_loss: 0.0311997 | Time: 2548.37 ms [2022-04-12 13:53:48 main:574] : INFO : Epoch 1996 | loss: 0.0311796 | val_loss: 0.0311994 | Time: 2480.31 ms [2022-04-12 13:53:50 main:574] : INFO : Epoch 1997 | loss: 0.031179 | val_loss: 0.031201 | Time: 2284.97 ms [2022-04-12 13:53:53 main:574] : INFO : Epoch 1998 | loss: 0.0311787 | val_loss: 0.0311998 | Time: 2552.02 ms [2022-04-12 13:53:55 main:574] : INFO : Epoch 1999 | loss: 0.0311831 | val_loss: 0.0312008 | Time: 2298 ms [2022-04-12 13:53:58 main:574] : INFO : Epoch 2000 | loss: 0.0311871 | val_loss: 0.0312011 | Time: 2518.63 ms [2022-04-12 13:54:00 main:574] : INFO : Epoch 2001 | loss: 0.0311832 | val_loss: 0.0311997 | Time: 2603.79 ms [2022-04-12 13:54:03 main:574] : INFO : Epoch 2002 | loss: 0.0311851 | val_loss: 0.0311989 | Time: 2335.07 ms [2022-04-12 13:54:16 main:574] : INFO : Epoch 2003 | loss: 0.0311836 | val_loss: 0.0312011 | Time: 12877.7 ms [2022-04-12 13:54:18 main:574] : INFO : Epoch 2004 | loss: 0.0311839 | val_loss: 0.0311987 | Time: 2602.01 ms [2022-04-12 13:54:21 main:574] : INFO : Epoch 2005 | loss: 0.0311832 | val_loss: 0.0311979 | Time: 2337.49 ms [2022-04-12 13:54:23 main:574] : INFO : Epoch 2006 | loss: 0.0311821 | val_loss: 0.0311987 | Time: 2606.26 ms [2022-04-12 13:54:26 main:574] : INFO : Epoch 2007 | loss: 0.0311814 | val_loss: 0.0311973 | Time: 2720.85 ms [2022-04-12 13:54:28 main:574] : INFO : Epoch 2008 | loss: 0.0311817 | val_loss: 0.0311976 | Time: 2581.44 ms [2022-04-12 13:54:31 main:574] : INFO : Epoch 2009 | loss: 0.0311813 | val_loss: 0.0311966 | Time: 2713.21 ms [2022-04-12 13:54:34 main:574] : INFO : Epoch 2010 | loss: 0.0311798 | val_loss: 0.0311972 | Time: 2346.9 ms [2022-04-12 13:54:36 main:574] : INFO : Epoch 2011 | loss: 0.0311794 | val_loss: 0.0311969 | Time: 2641.56 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-12 13:57:39 main:435] : INFO : Set logging level to 1 [2022-04-12 13:57:39 main:441] : INFO : Running in BOINC Client mode [2022-04-12 13:57:39 main:444] : INFO : Resolving all filenames [2022-04-12 13:57:39 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-12 13:57:39 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-12 13:57:39 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-12 13:57:39 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-12 13:57:39 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-12 13:57:39 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-12 13:57:39 main:474] : INFO : Configuration: [2022-04-12 13:57:39 main:475] : INFO : Model type: GRU [2022-04-12 13:57:39 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-12 13:57:39 main:477] : INFO : Max Epochs: 2048 [2022-04-12 13:57:39 main:478] : INFO : Batch Size: 128 [2022-04-12 13:57:39 main:479] : INFO : Learning Rate: 0.01 [2022-04-12 13:57:39 main:480] : INFO : Patience: 10 [2022-04-12 13:57:39 main:481] : INFO : Hidden Width: 12 [2022-04-12 13:57:39 main:482] : INFO : # Recurrent Layers: 4 [2022-04-12 13:57:39 main:483] : INFO : # Backend Layers: 4 [2022-04-12 13:57:39 main:484] : INFO : # Threads: 1 [2022-04-12 13:57:39 main:486] : INFO : Preparing Dataset [2022-04-12 13:57:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-12 13:57:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-12 13:57:41 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-12 13:57:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-12 13:57:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-12 13:57:41 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-12 13:57:41 main:494] : INFO : Creating Model [2022-04-12 13:57:41 main:507] : INFO : Preparing config file [2022-04-12 13:57:41 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-12 13:57:41 main:512] : INFO : Loading config [2022-04-12 13:57:41 main:514] : INFO : Loading state [2022-04-12 13:57:42 main:559] : INFO : Loading DataLoader into Memory [2022-04-12 13:57:42 main:562] : INFO : Starting Training [2022-04-12 13:57:44 main:574] : INFO : Epoch 1968 | loss: 0.031214 | val_loss: 0.0312074 | Time: 2388.33 ms [2022-04-12 13:57:46 main:574] : INFO : Epoch 1969 | loss: 0.031187 | val_loss: 0.0312035 | Time: 2201.86 ms [2022-04-12 13:57:49 main:574] : INFO : Epoch 1970 | loss: 0.0311851 | val_loss: 0.0312027 | Time: 2207.11 ms [2022-04-12 13:57:51 main:574] : INFO : Epoch 1971 | loss: 0.0311832 | val_loss: 0.0312039 | Time: 2293.83 ms [2022-04-12 13:57:53 main:574] : INFO : Epoch 1972 | loss: 0.0311827 | val_loss: 0.0312024 | Time: 2410.43 ms [2022-04-12 13:57:56 main:574] : INFO : Epoch 1973 | loss: 0.0311817 | val_loss: 0.0312027 | Time: 2321.73 ms [2022-04-12 13:57:58 main:574] : INFO : Epoch 1974 | loss: 0.0311825 | val_loss: 0.0312036 | Time: 2441.37 ms [2022-04-12 13:58:00 main:574] : INFO : Epoch 1975 | loss: 0.0311828 | val_loss: 0.0312019 | Time: 2340.53 ms [2022-04-12 13:58:03 main:574] : INFO : Epoch 1976 | loss: 0.0311812 | val_loss: 0.0312004 | Time: 2602.3 ms [2022-04-12 13:58:06 main:574] : INFO : Epoch 1977 | loss: 0.0311819 | val_loss: 0.0312013 | Time: 2783 ms [2022-04-12 13:58:18 main:574] : INFO : Epoch 1978 | loss: 0.0311818 | val_loss: 0.0312005 | Time: 12404.4 ms [2022-04-12 13:58:21 main:574] : INFO : Epoch 1979 | loss: 0.0311851 | val_loss: 0.031201 | Time: 2358.01 ms [2022-04-12 13:58:23 main:574] : INFO : Epoch 1980 | loss: 0.0311811 | val_loss: 0.0312005 | Time: 2432.99 ms [2022-04-12 13:58:25 main:574] : INFO : Epoch 1981 | loss: 0.0311806 | val_loss: 0.0312002 | Time: 2329.54 ms [2022-04-12 13:58:28 main:574] : INFO : Epoch 1982 | loss: 0.0311801 | val_loss: 0.0312034 | Time: 2423.54 ms [2022-04-12 13:58:30 main:574] : INFO : Epoch 1983 | loss: 0.0311846 | val_loss: 0.0312028 | Time: 2342.72 ms [2022-04-12 13:58:33 main:574] : INFO : Epoch 1984 | loss: 0.0311839 | val_loss: 0.0312011 | Time: 2546.89 ms [2022-04-12 13:58:35 main:574] : INFO : Epoch 1985 | loss: 0.0311829 | val_loss: 0.0312045 | Time: 2594.05 ms [2022-04-12 13:58:38 main:574] : INFO : Epoch 1986 | loss: 0.0311846 | val_loss: 0.0311999 | Time: 2357.47 ms [2022-04-12 13:58:40 main:574] : INFO : Epoch 1987 | loss: 0.0311836 | val_loss: 0.0311996 | Time: 2461.07 ms [2022-04-12 13:58:43 main:574] : INFO : Epoch 1988 | loss: 0.0311827 | val_loss: 0.0312044 | Time: 2614.98 ms [2022-04-12 13:58:45 main:574] : INFO : Epoch 1989 | loss: 0.0311823 | val_loss: 0.0312009 | Time: 2424.26 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-13 09:15:28 main:435] : INFO : Set logging level to 1 [2022-04-13 09:15:28 main:441] : INFO : Running in BOINC Client mode [2022-04-13 09:15:28 main:444] : INFO : Resolving all filenames [2022-04-13 09:15:28 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 09:15:28 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 09:15:28 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 09:15:28 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 09:15:28 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 09:15:28 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 09:15:28 main:474] : INFO : Configuration: [2022-04-13 09:15:28 main:475] : INFO : Model type: GRU [2022-04-13 09:15:28 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 09:15:28 main:477] : INFO : Max Epochs: 2048 [2022-04-13 09:15:28 main:478] : INFO : Batch Size: 128 [2022-04-13 09:15:28 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 09:15:28 main:480] : INFO : Patience: 10 [2022-04-13 09:15:28 main:481] : INFO : Hidden Width: 12 [2022-04-13 09:15:28 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 09:15:28 main:483] : INFO : # Backend Layers: 4 [2022-04-13 09:15:28 main:484] : INFO : # Threads: 1 [2022-04-13 09:15:28 main:486] : INFO : Preparing Dataset [2022-04-13 09:15:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 09:15:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 09:15:31 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 09:15:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 09:15:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 09:15:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 09:15:31 main:494] : INFO : Creating Model [2022-04-13 09:15:31 main:507] : INFO : Preparing config file [2022-04-13 09:15:31 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 09:15:31 main:512] : INFO : Loading config [2022-04-13 09:15:31 main:514] : INFO : Loading state [2022-04-13 09:15:32 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 09:15:32 main:562] : INFO : Starting Training [2022-04-13 09:15:35 main:574] : INFO : Epoch 1968 | loss: 0.0312143 | val_loss: 0.0312118 | Time: 2794.5 ms [2022-04-13 09:15:37 main:574] : INFO : Epoch 1969 | loss: 0.0311894 | val_loss: 0.0312028 | Time: 2189.75 ms [2022-04-13 09:15:39 main:574] : INFO : Epoch 1970 | loss: 0.031186 | val_loss: 0.0312026 | Time: 2309.31 ms [2022-04-13 09:15:42 main:574] : INFO : Epoch 1971 | loss: 0.031185 | val_loss: 0.0312016 | Time: 2240.92 ms [2022-04-13 09:15:44 main:574] : INFO : Epoch 1972 | loss: 0.0311852 | val_loss: 0.0312027 | Time: 2309.79 ms [2022-04-13 09:15:46 main:574] : INFO : Epoch 1973 | loss: 0.0311847 | val_loss: 0.0312014 | Time: 2296.83 ms [2022-04-13 09:15:49 main:574] : INFO : Epoch 1974 | loss: 0.0311837 | val_loss: 0.0312024 | Time: 2354.92 ms [2022-04-13 09:15:51 main:574] : INFO : Epoch 1975 | loss: 0.0311838 | val_loss: 0.0312034 | Time: 2303.88 ms [2022-04-13 09:15:53 main:574] : INFO : Epoch 1976 | loss: 0.0311823 | val_loss: 0.0312002 | Time: 2310.38 ms [2022-04-13 09:15:56 main:574] : INFO : Epoch 1977 | loss: 0.0311819 | val_loss: 0.0311986 | Time: 2397.67 ms [2022-04-13 09:15:58 main:574] : INFO : Epoch 1978 | loss: 0.0311815 | val_loss: 0.0311997 | Time: 2289.85 ms [2022-04-13 09:16:00 main:574] : INFO : Epoch 1979 | loss: 0.0311818 | val_loss: 0.0312023 | Time: 2398.53 ms [2022-04-13 09:16:03 main:574] : INFO : Epoch 1980 | loss: 0.0311815 | val_loss: 0.0312028 | Time: 2322.98 ms [2022-04-13 09:16:05 main:574] : INFO : Epoch 1981 | loss: 0.0311851 | val_loss: 0.0312051 | Time: 2320.41 ms [2022-04-13 09:16:07 main:574] : INFO : Epoch 1982 | loss: 0.031189 | val_loss: 0.0312012 | Time: 2396.02 ms [2022-04-13 09:16:10 main:574] : INFO : Epoch 1983 | loss: 0.0311885 | val_loss: 0.0312008 | Time: 2256.26 ms [2022-04-13 09:16:12 main:574] : INFO : Epoch 1984 | loss: 0.0311882 | val_loss: 0.0312025 | Time: 2401.01 ms [2022-04-13 09:16:14 main:574] : INFO : Epoch 1985 | loss: 0.0311866 | val_loss: 0.0312029 | Time: 2317.07 ms [2022-04-13 09:16:17 main:574] : INFO : Epoch 1986 | loss: 0.031185 | val_loss: 0.0312005 | Time: 2375.09 ms [2022-04-13 09:16:19 main:574] : INFO : Epoch 1987 | loss: 0.0311839 | val_loss: 0.0312045 | Time: 2390.03 ms [2022-04-13 09:16:21 main:574] : INFO : Epoch 1988 | loss: 0.0311833 | val_loss: 0.0311998 | Time: 2261.89 ms [2022-04-13 09:16:24 main:574] : INFO : Epoch 1989 | loss: 0.0311823 | val_loss: 0.0311998 | Time: 2395.65 ms [2022-04-13 09:16:26 main:574] : INFO : Epoch 1990 | loss: 0.0311826 | val_loss: 0.0311998 | Time: 2235.95 ms [2022-04-13 09:16:31 main:574] : INFO : Epoch 1991 | loss: 0.0311803 | val_loss: 0.0311996 | Time: 5033.32 ms [2022-04-13 09:16:55 main:574] : INFO : Epoch 1992 | loss: 0.0311795 | val_loss: 0.0312006 | Time: 23756.7 ms [2022-04-13 09:16:57 main:574] : INFO : Epoch 1993 | loss: 0.0311823 | val_loss: 0.0312076 | Time: 2407.7 ms [2022-04-13 09:17:00 main:574] : INFO : Epoch 1994 | loss: 0.0311827 | val_loss: 0.0312013 | Time: 2463.54 ms [2022-04-13 09:17:02 main:574] : INFO : Epoch 1995 | loss: 0.0311792 | val_loss: 0.0312038 | Time: 2336.76 ms [2022-04-13 09:17:15 main:574] : INFO : Epoch 1996 | loss: 0.0311784 | val_loss: 0.0312085 | Time: 12970.9 ms [2022-04-13 09:17:18 main:574] : INFO : Epoch 1997 | loss: 0.0311827 | val_loss: 0.0312026 | Time: 2671.25 ms [2022-04-13 09:17:20 main:574] : INFO : Epoch 1998 | loss: 0.0311803 | val_loss: 0.0312017 | Time: 2334.55 ms [2022-04-13 09:17:22 main:574] : INFO : Epoch 1999 | loss: 0.0311789 | val_loss: 0.0312003 | Time: 2425.83 ms [2022-04-13 09:17:25 main:574] : INFO : Epoch 2000 | loss: 0.0311786 | val_loss: 0.0312011 | Time: 2443.38 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-13 10:26:14 main:435] : INFO : Set logging level to 1 [2022-04-13 10:26:14 main:441] : INFO : Running in BOINC Client mode [2022-04-13 10:26:14 main:444] : INFO : Resolving all filenames [2022-04-13 10:26:14 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 10:26:14 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 10:26:14 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 10:26:14 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 10:26:14 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 10:26:14 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 10:26:14 main:474] : INFO : Configuration: [2022-04-13 10:26:14 main:475] : INFO : Model type: GRU [2022-04-13 10:26:14 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 10:26:14 main:477] : INFO : Max Epochs: 2048 [2022-04-13 10:26:14 main:478] : INFO : Batch Size: 128 [2022-04-13 10:26:14 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 10:26:14 main:480] : INFO : Patience: 10 [2022-04-13 10:26:14 main:481] : INFO : Hidden Width: 12 [2022-04-13 10:26:14 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 10:26:14 main:483] : INFO : # Backend Layers: 4 [2022-04-13 10:26:14 main:484] : INFO : # Threads: 1 [2022-04-13 10:26:14 main:486] : INFO : Preparing Dataset [2022-04-13 10:26:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 10:26:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 10:26:16 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 10:26:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 10:26:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 10:26:16 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 10:26:16 main:494] : INFO : Creating Model [2022-04-13 10:26:16 main:507] : INFO : Preparing config file [2022-04-13 10:26:16 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 10:26:16 main:512] : INFO : Loading config [2022-04-13 10:26:16 main:514] : INFO : Loading state [2022-04-13 10:26:17 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 10:26:17 main:562] : INFO : Starting Training [2022-04-13 10:26:19 main:574] : INFO : Epoch 1968 | loss: 0.0312112 | val_loss: 0.031211 | Time: 2414.13 ms [2022-04-13 10:26:21 main:574] : INFO : Epoch 1969 | loss: 0.0311865 | val_loss: 0.0312018 | Time: 2189.65 ms [2022-04-13 10:26:23 main:574] : INFO : Epoch 1970 | loss: 0.0311842 | val_loss: 0.0312004 | Time: 2195.05 ms [2022-04-13 10:26:26 main:574] : INFO : Epoch 1971 | loss: 0.0311819 | val_loss: 0.0312009 | Time: 2236.53 ms [2022-04-13 10:26:28 main:574] : INFO : Epoch 1972 | loss: 0.0311827 | val_loss: 0.031201 | Time: 2385.73 ms [2022-04-13 10:26:30 main:574] : INFO : Epoch 1973 | loss: 0.031182 | val_loss: 0.0312018 | Time: 2287.2 ms [2022-04-13 10:26:33 main:574] : INFO : Epoch 1974 | loss: 0.0311817 | val_loss: 0.0312034 | Time: 2360.29 ms [2022-04-13 10:26:35 main:574] : INFO : Epoch 1975 | loss: 0.0311811 | val_loss: 0.0311995 | Time: 2286.14 ms [2022-04-13 10:26:37 main:574] : INFO : Epoch 1976 | loss: 0.031183 | val_loss: 0.031202 | Time: 2254.55 ms [2022-04-13 10:26:40 main:574] : INFO : Epoch 1977 | loss: 0.0311816 | val_loss: 0.0311994 | Time: 2353.47 ms [2022-04-13 10:26:42 main:574] : INFO : Epoch 1978 | loss: 0.0311812 | val_loss: 0.0312015 | Time: 2237.92 ms [2022-04-13 10:26:44 main:574] : INFO : Epoch 1979 | loss: 0.0311807 | val_loss: 0.0312027 | Time: 2299.37 ms [2022-04-13 10:26:47 main:574] : INFO : Epoch 1980 | loss: 0.0311802 | val_loss: 0.0312046 | Time: 2395.34 ms [2022-04-13 10:26:49 main:574] : INFO : Epoch 1981 | loss: 0.031182 | val_loss: 0.0312009 | Time: 2295.9 ms [2022-04-13 10:26:51 main:574] : INFO : Epoch 1982 | loss: 0.0311804 | val_loss: 0.0312 | Time: 2370.13 ms [2022-04-13 10:26:54 main:574] : INFO : Epoch 1983 | loss: 0.0311794 | val_loss: 0.0312001 | Time: 2303.59 ms [2022-04-13 10:26:56 main:574] : INFO : Epoch 1984 | loss: 0.0311803 | val_loss: 0.0312014 | Time: 2303.63 ms [2022-04-13 10:26:58 main:574] : INFO : Epoch 1985 | loss: 0.03118 | val_loss: 0.0312068 | Time: 2282.42 ms [2022-04-13 10:27:01 main:574] : INFO : Epoch 1986 | loss: 0.0311808 | val_loss: 0.0312018 | Time: 2540.66 ms [2022-04-13 10:27:03 main:574] : INFO : Epoch 1987 | loss: 0.0311791 | val_loss: 0.0312011 | Time: 2588.23 ms [2022-04-13 10:27:06 main:574] : INFO : Epoch 1988 | loss: 0.0311796 | val_loss: 0.031201 | Time: 2247.67 ms [2022-04-13 10:27:08 main:574] : INFO : Epoch 1989 | loss: 0.031179 | val_loss: 0.0312005 | Time: 2475.37 ms [2022-04-13 10:27:10 main:574] : INFO : Epoch 1990 | loss: 0.031179 | val_loss: 0.0312034 | Time: 2330.24 ms [2022-04-13 10:27:13 main:574] : INFO : Epoch 1991 | loss: 0.0311796 | val_loss: 0.0312002 | Time: 2326.21 ms [2022-04-13 10:27:16 main:574] : INFO : Epoch 1992 | loss: 0.031179 | val_loss: 0.0312024 | Time: 3034.76 ms [2022-04-13 10:27:18 main:574] : INFO : Epoch 1993 | loss: 0.0311819 | val_loss: 0.0312033 | Time: 2406.57 ms [2022-04-13 10:27:21 main:574] : INFO : Epoch 1994 | loss: 0.0311812 | val_loss: 0.0312022 | Time: 2561.05 ms [2022-04-13 10:27:23 main:574] : INFO : Epoch 1995 | loss: 0.0311806 | val_loss: 0.0312028 | Time: 2640.23 ms [2022-04-13 10:27:26 main:574] : INFO : Epoch 1996 | loss: 0.0311838 | val_loss: 0.0311997 | Time: 2301.13 ms [2022-04-13 10:27:28 main:574] : INFO : Epoch 1997 | loss: 0.0311819 | val_loss: 0.0312028 | Time: 2498.48 ms [2022-04-13 10:27:31 main:574] : INFO : Epoch 1998 | loss: 0.0311806 | val_loss: 0.0312018 | Time: 2374.04 ms [2022-04-13 10:27:33 main:574] : INFO : Epoch 1999 | loss: 0.03118 | val_loss: 0.0312029 | Time: 2383.53 ms [2022-04-13 10:27:35 main:574] : INFO : Epoch 2000 | loss: 0.0311792 | val_loss: 0.0312034 | Time: 2460.35 ms [2022-04-13 10:27:38 main:574] : INFO : Epoch 2001 | loss: 0.0311828 | val_loss: 0.0312033 | Time: 2337.29 ms [2022-04-13 10:27:40 main:574] : INFO : Epoch 2002 | loss: 0.031184 | val_loss: 0.0312025 | Time: 2472.22 ms [2022-04-13 10:27:43 main:574] : INFO : Epoch 2003 | loss: 0.0311835 | val_loss: 0.0312043 | Time: 2510.78 ms [2022-04-13 10:27:45 main:574] : INFO : Epoch 2004 | loss: 0.0311848 | val_loss: 0.0312087 | Time: 2493.34 ms [2022-04-13 10:27:48 main:574] : INFO : Epoch 2005 | loss: 0.0311852 | val_loss: 0.031201 | Time: 2414.28 ms [2022-04-13 10:27:50 main:574] : INFO : Epoch 2006 | loss: 0.0311816 | val_loss: 0.0312044 | Time: 2258.65 ms [2022-04-13 10:27:52 main:574] : INFO : Epoch 2007 | loss: 0.0311794 | val_loss: 0.0312035 | Time: 2499.83 ms [2022-04-13 10:27:55 main:574] : INFO : Epoch 2008 | loss: 0.0311793 | val_loss: 0.0312032 | Time: 2366.99 ms [2022-04-13 10:27:57 main:574] : INFO : Epoch 2009 | loss: 0.0311781 | val_loss: 0.0312032 | Time: 2345.15 ms [2022-04-13 10:28:00 main:574] : INFO : Epoch 2010 | loss: 0.0311785 | val_loss: 0.0312035 | Time: 2575.67 ms [2022-04-13 10:28:02 main:574] : INFO : Epoch 2011 | loss: 0.0311828 | val_loss: 0.0312025 | Time: 2428.51 ms [2022-04-13 10:28:15 main:574] : INFO : Epoch 2012 | loss: 0.0311802 | val_loss: 0.0312077 | Time: 12814.6 ms [2022-04-13 10:28:17 main:574] : INFO : Epoch 2013 | loss: 0.0311837 | val_loss: 0.0312035 | Time: 2490 ms [2022-04-13 10:28:20 main:574] : INFO : Epoch 2014 | loss: 0.0311881 | val_loss: 0.0312027 | Time: 2330.03 ms [2022-04-13 10:28:22 main:574] : INFO : Epoch 2015 | loss: 0.0311883 | val_loss: 0.0312037 | Time: 2400.91 ms [2022-04-13 10:28:25 main:574] : INFO : Epoch 2016 | loss: 0.0311886 | val_loss: 0.0312009 | Time: 2394.61 ms [2022-04-13 10:28:27 main:574] : INFO : Epoch 2017 | loss: 0.0311872 | val_loss: 0.0312009 | Time: 2425.21 ms [2022-04-13 10:28:30 main:574] : INFO : Epoch 2018 | loss: 0.031186 | val_loss: 0.031204 | Time: 2626.37 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: Quadro P2000) [2022-04-13 10:31:31 main:435] : INFO : Set logging level to 1 [2022-04-13 10:31:31 main:441] : INFO : Running in BOINC Client mode [2022-04-13 10:31:31 main:444] : INFO : Resolving all filenames [2022-04-13 10:31:31 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 10:31:31 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 10:31:31 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 10:31:31 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 10:31:31 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 10:31:31 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 10:31:31 main:474] : INFO : Configuration: [2022-04-13 10:31:31 main:475] : INFO : Model type: GRU [2022-04-13 10:31:31 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 10:31:31 main:477] : INFO : Max Epochs: 2048 [2022-04-13 10:31:31 main:478] : INFO : Batch Size: 128 [2022-04-13 10:31:31 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 10:31:31 main:480] : INFO : Patience: 10 [2022-04-13 10:31:31 main:481] : INFO : Hidden Width: 12 [2022-04-13 10:31:31 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 10:31:31 main:483] : INFO : # Backend Layers: 4 [2022-04-13 10:31:31 main:484] : INFO : # Threads: 1 [2022-04-13 10:31:31 main:486] : INFO : Preparing Dataset [2022-04-13 10:31:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 10:31:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 10:31:33 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 10:31:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 10:31:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 10:31:34 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 10:31:34 main:494] : INFO : Creating Model [2022-04-13 10:31:34 main:507] : INFO : Preparing config file [2022-04-13 10:31:34 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 10:31:34 main:512] : INFO : Loading config [2022-04-13 10:31:34 main:514] : INFO : Loading state [2022-04-13 10:31:34 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 10:31:34 main:562] : INFO : Starting Training [2022-04-13 10:31:37 main:574] : INFO : Epoch 1968 | loss: 0.0312142 | val_loss: 0.0312013 | Time: 2370.68 ms [2022-04-13 10:31:39 main:574] : INFO : Epoch 1969 | loss: 0.0311876 | val_loss: 0.031202 | Time: 2213.89 ms [2022-04-13 10:31:41 main:574] : INFO : Epoch 1970 | loss: 0.0311872 | val_loss: 0.0312002 | Time: 2184.98 ms [2022-04-13 10:31:43 main:574] : INFO : Epoch 1971 | loss: 0.0311852 | val_loss: 0.0312023 | Time: 2308.3 ms [2022-04-13 10:31:46 main:574] : INFO : Epoch 1972 | loss: 0.0311843 | val_loss: 0.0311987 | Time: 2333.91 ms [2022-04-13 10:31:48 main:574] : INFO : Epoch 1973 | loss: 0.0311838 | val_loss: 0.0312004 | Time: 2290.18 ms [2022-04-13 10:31:51 main:574] : INFO : Epoch 1974 | loss: 0.0311817 | val_loss: 0.0312038 | Time: 2509.04 ms [2022-04-13 10:31:53 main:574] : INFO : Epoch 1975 | loss: 0.0311847 | val_loss: 0.0312037 | Time: 2322.48 ms [2022-04-13 10:31:55 main:574] : INFO : Epoch 1976 | loss: 0.0311865 | val_loss: 0.0312024 | Time: 2440.44 ms [2022-04-13 10:31:58 main:574] : INFO : Epoch 1977 | loss: 0.0311871 | val_loss: 0.0312025 | Time: 2451.2 ms [2022-04-13 10:32:00 main:574] : INFO : Epoch 1978 | loss: 0.0311857 | val_loss: 0.0312025 | Time: 2346.91 ms [2022-04-13 10:32:03 main:574] : INFO : Epoch 1979 | loss: 0.0311852 | val_loss: 0.0312007 | Time: 2497.81 ms [2022-04-13 10:32:15 main:574] : INFO : Epoch 1980 | loss: 0.0311837 | val_loss: 0.0312007 | Time: 12448.1 ms [2022-04-13 10:32:18 main:574] : INFO : Epoch 1981 | loss: 0.0311843 | val_loss: 0.0311982 | Time: 2389.42 ms [2022-04-13 10:32:20 main:574] : INFO : Epoch 1982 | loss: 0.0311841 | val_loss: 0.0311976 | Time: 2346.84 ms [2022-04-13 10:32:22 main:574] : INFO : Epoch 1983 | loss: 0.0311826 | val_loss: 0.0311991 | Time: 2428.19 ms [2022-04-13 10:32:25 main:574] : INFO : Epoch 1984 | loss: 0.0311835 | val_loss: 0.0312012 | Time: 2380.7 ms [2022-04-13 10:32:27 main:574] : INFO : Epoch 1985 | loss: 0.031185 | val_loss: 0.0311995 | Time: 2299.66 ms [2022-04-13 10:32:29 main:574] : INFO : Epoch 1986 | loss: 0.0311842 | val_loss: 0.0311982 | Time: 2472.35 ms [2022-04-13 10:32:32 main:574] : INFO : Epoch 1987 | loss: 0.0311838 | val_loss: 0.0311996 | Time: 2353.66 ms [2022-04-13 10:32:34 main:574] : INFO : Epoch 1988 | loss: 0.031182 | val_loss: 0.0311988 | Time: 2380.22 ms [2022-04-13 10:32:37 main:574] : INFO : Epoch 1989 | loss: 0.0311825 | val_loss: 0.0311973 | Time: 2461.91 ms [2022-04-13 10:32:39 main:574] : INFO : Epoch 1990 | loss: 0.0311822 | val_loss: 0.0312024 | Time: 2245.39 ms [2022-04-13 10:32:41 main:574] : INFO : Epoch 1991 | loss: 0.0311815 | val_loss: 0.0311975 | Time: 2413.48 ms [2022-04-13 10:32:44 main:574] : INFO : Epoch 1992 | loss: 0.0311799 | val_loss: 0.0311976 | Time: 2417.82 ms [2022-04-13 10:32:46 main:574] : INFO : Epoch 1993 | loss: 0.0311818 | val_loss: 0.0312031 | Time: 2418.67 ms [2022-04-13 10:32:49 main:574] : INFO : Epoch 1994 | loss: 0.031182 | val_loss: 0.0311983 | Time: 2436.2 ms [2022-04-13 10:32:51 main:574] : INFO : Epoch 1995 | loss: 0.0311797 | val_loss: 0.0311988 | Time: 2319.2 ms [2022-04-13 10:32:53 main:574] : INFO : Epoch 1996 | loss: 0.0311791 | val_loss: 0.0312007 | Time: 2418.66 ms [2022-04-13 10:32:56 main:574] : INFO : Epoch 1997 | loss: 0.0311806 | val_loss: 0.0312005 | Time: 2299.44 ms [2022-04-13 10:32:58 main:574] : INFO : Epoch 1998 | loss: 0.0311832 | val_loss: 0.0312009 | Time: 2495.04 ms [2022-04-13 10:33:01 main:574] : INFO : Epoch 1999 | loss: 0.0311839 | val_loss: 0.0312036 | Time: 2461.73 ms [2022-04-13 10:33:03 main:574] : INFO : Epoch 2000 | loss: 0.0311827 | val_loss: 0.0312012 | Time: 2420.61 ms [2022-04-13 10:33:06 main:574] : INFO : Epoch 2001 | loss: 0.0311817 | val_loss: 0.0312008 | Time: 2473.24 ms [2022-04-13 10:33:18 main:574] : INFO : Epoch 2002 | loss: 0.0311811 | val_loss: 0.0312005 | Time: 12539.8 ms [2022-04-13 10:33:21 main:574] : INFO : Epoch 2003 | loss: 0.0311822 | val_loss: 0.0312016 | Time: 2418.11 ms [2022-04-13 10:33:23 main:574] : INFO : Epoch 2004 | loss: 0.0311824 | val_loss: 0.0311986 | Time: 2387.72 ms [2022-04-13 10:33:25 main:574] : INFO : Epoch 2005 | loss: 0.0311817 | val_loss: 0.0312001 | Time: 2315.78 ms [2022-04-13 10:33:28 main:574] : INFO : Epoch 2006 | loss: 0.0311828 | val_loss: 0.0312026 | Time: 2429.02 ms [2022-04-13 10:33:30 main:574] : INFO : Epoch 2007 | loss: 0.0311824 | val_loss: 0.0312028 | Time: 2371.87 ms [2022-04-13 10:33:32 main:574] : INFO : Epoch 2008 | loss: 0.0311811 | val_loss: 0.0311978 | Time: 2328.87 ms [2022-04-13 10:33:35 main:574] : INFO : Epoch 2009 | loss: 0.0311805 | val_loss: 0.0312017 | Time: 2579.67 ms [2022-04-13 10:33:37 main:574] : INFO : Epoch 2010 | loss: 0.0311808 | val_loss: 0.0312022 | Time: 2297.93 ms [2022-04-13 10:33:40 main:574] : INFO : Epoch 2011 | loss: 0.03118 | val_loss: 0.0312007 | Time: 2412.45 ms [2022-04-13 10:33:42 main:574] : INFO : Epoch 2012 | loss: 0.0311813 | val_loss: 0.0312032 | Time: 2327.76 ms [2022-04-13 10:33:44 main:574] : INFO : Epoch 2013 | loss: 0.0311821 | val_loss: 0.0312039 | Time: 2343.71 ms [2022-04-13 10:33:47 main:574] : INFO : Epoch 2014 | loss: 0.0311809 | val_loss: 0.0312 | Time: 2413.04 ms [2022-04-13 10:33:49 main:574] : INFO : Epoch 2015 | loss: 0.0311807 | val_loss: 0.0312011 | Time: 2271.8 ms [2022-04-13 10:33:52 main:574] : INFO : Epoch 2016 | loss: 0.0311798 | val_loss: 0.0312001 | Time: 2547.68 ms [2022-04-13 10:33:54 main:574] : INFO : Epoch 2017 | loss: 0.0311804 | val_loss: 0.0312007 | Time: 2267.75 ms [2022-04-13 10:33:56 main:574] : INFO : Epoch 2018 | loss: 0.0311791 | val_loss: 0.0311998 | Time: 2497.24 ms [2022-04-13 10:33:59 main:574] : INFO : Epoch 2019 | loss: 0.0311782 | val_loss: 0.031199 | Time: 2409.97 ms [2022-04-13 10:34:01 main:574] : INFO : Epoch 2020 | loss: 0.0311806 | val_loss: 0.0311979 | Time: 2269.14 ms [2022-04-13 10:34:03 main:574] : INFO : Epoch 2021 | loss: 0.0311808 | val_loss: 0.0312057 | Time: 2425.49 ms [2022-04-13 10:34:06 main:574] : INFO : Epoch 2022 | loss: 0.0311807 | val_loss: 0.0311992 | Time: 2241.13 ms [2022-04-13 10:34:08 main:574] : INFO : Epoch 2023 | loss: 0.0311777 | val_loss: 0.0311998 | Time: 2409.23 ms [2022-04-13 10:34:11 main:574] : INFO : Epoch 2024 | loss: 0.0311775 | val_loss: 0.0312012 | Time: 2428.22 ms [2022-04-13 10:34:13 main:574] : INFO : Epoch 2025 | loss: 0.0311769 | val_loss: 0.0311988 | Time: 2293.62 ms [2022-04-13 10:34:15 main:574] : INFO : Epoch 2026 | loss: 0.031177 | val_loss: 0.0311984 | Time: 2409.18 ms [2022-04-13 10:34:18 main:574] : INFO : Epoch 2027 | loss: 0.0311779 | val_loss: 0.0312031 | Time: 2503.06 ms [2022-04-13 10:34:20 main:574] : INFO : Epoch 2028 | loss: 0.0311795 | val_loss: 0.0312007 | Time: 2424.48 ms [2022-04-13 10:34:23 main:574] : INFO : Epoch 2029 | loss: 0.0311783 | val_loss: 0.0311996 | Time: 2447.14 ms [2022-04-13 10:34:25 main:574] : INFO : Epoch 2030 | loss: 0.0311778 | val_loss: 0.0311987 | Time: 2321.85 ms [2022-04-13 10:34:27 main:574] : INFO : Epoch 2031 | loss: 0.0311796 | val_loss: 0.0312026 | Time: 2380.43 ms [2022-04-13 10:34:30 main:574] : INFO : Epoch 2032 | loss: 0.0311797 | val_loss: 0.0312009 | Time: 2280.48 ms [2022-04-13 10:34:32 main:574] : INFO : Epoch 2033 | loss: 0.031178 | val_loss: 0.0312022 | Time: 2448.66 ms [2022-04-13 10:34:34 main:574] : INFO : Epoch 2034 | loss: 0.0311784 | val_loss: 0.0312015 | Time: 2327.6 ms [2022-04-13 10:34:37 main:574] : INFO : Epoch 2035 | loss: 0.0311793 | val_loss: 0.0312027 | Time: 2425.37 ms [2022-04-13 10:34:39 main:574] : INFO : Epoch 2036 | loss: 0.0311781 | val_loss: 0.0312017 | Time: 2360.13 ms [2022-04-13 10:34:42 main:574] : INFO : Epoch 2037 | loss: 0.0311769 | val_loss: 0.0312009 | Time: 2279.92 ms [2022-04-13 10:34:44 main:574] : INFO : Epoch 2038 | loss: 0.0311782 | val_loss: 0.031201 | Time: 2431.43 ms [2022-04-13 10:34:46 main:574] : INFO : Epoch 2039 | loss: 0.0311772 | val_loss: 0.0312006 | Time: 2328.38 ms [2022-04-13 10:34:49 main:574] : INFO : Epoch 2040 | loss: 0.0311776 | val_loss: 0.0312046 | Time: 2386.91 ms [2022-04-13 10:34:51 main:574] : INFO : Epoch 2041 | loss: 0.0311799 | val_loss: 0.0311986 | Time: 2476.81 ms [2022-04-13 10:34:53 main:574] : INFO : Epoch 2042 | loss: 0.0311782 | val_loss: 0.031201 | Time: 2274.02 ms [2022-04-13 10:34:56 main:574] : INFO : Epoch 2043 | loss: 0.0311787 | val_loss: 0.0311988 | Time: 2408.5 ms [2022-04-13 10:34:58 main:574] : INFO : Epoch 2044 | loss: 0.031179 | val_loss: 0.0311979 | Time: 2290.41 ms [2022-04-13 10:35:01 main:574] : INFO : Epoch 2045 | loss: 0.0311773 | val_loss: 0.0311999 | Time: 2451.06 ms [2022-04-13 10:35:03 main:574] : INFO : Epoch 2046 | loss: 0.0311773 | val_loss: 0.031201 | Time: 2427.33 ms [2022-04-13 10:35:05 main:574] : INFO : Epoch 2047 | loss: 0.0311766 | val_loss: 0.0312016 | Time: 2351.1 ms [2022-04-13 10:35:08 main:574] : INFO : Epoch 2048 | loss: 0.0311772 | val_loss: 0.0312008 | Time: 2399.11 ms [2022-04-13 10:35:08 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0312008 [2022-04-13 10:35:08 main:603] : INFO : Saving end state to config to file [2022-04-13 10:35:08 main:608] : INFO : Success, exiting.. 10:35:08 (3140): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)