Task 12706612
| Name | ParityModified-1645994450-5533-2-0_0 |
| Workunit | 9857696 |
| Created | 7 Mar 2022, 21:24:37 UTC |
| Sent | 9 Mar 2022, 17:34:26 UTC |
| Report deadline | 17 Mar 2022, 17:34:26 UTC |
| Received | 2 May 2022, 17:09:13 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 19252 |
| Run time | 55 min 32 sec |
| CPU time | 53 min 39 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 4,569.32 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.63 GB |
| Peak swap size | 3.62 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 9.33 ms [2022-03-11 12:37:37 main:574] : INFO : Epoch 1989 | loss: 0.03109 | val_loss: 0.0311681 | Time: 1506.87 ms [2022-03-11 12:37:39 main:574] : INFO : Epoch 1990 | loss: 0.0310917 | val_loss: 0.0311737 | Time: 1487.79 ms [2022-03-11 12:37:40 main:574] : INFO : Epoch 1991 | loss: 0.0310917 | val_loss: 0.0311534 | Time: 1535.79 ms [2022-03-11 12:37:42 main:574] : INFO : Epoch 1992 | loss: 0.0310921 | val_loss: 0.0311705 | Time: 1505.22 ms [2022-03-11 12:37:43 main:574] : INFO : Epoch 1993 | loss: 0.0310887 | val_loss: 0.0311612 | Time: 1530.6 ms [2022-03-11 12:37:45 main:574] : INFO : Epoch 1994 | loss: 0.0310866 | val_loss: 0.0311765 | Time: 1511.09 ms [2022-03-11 12:37:46 main:574] : INFO : Epoch 1995 | loss: 0.0310844 | val_loss: 0.0311676 | Time: 1491.17 ms [2022-03-11 12:37:48 main:574] : INFO : Epoch 1996 | loss: 0.0310844 | val_loss: 0.0311703 | Time: 1488.96 ms [2022-03-11 12:37:49 main:574] : INFO : Epoch 1997 | loss: 0.0310885 | val_loss: 0.0311638 | Time: 1502.2 ms [2022-03-11 12:37:51 main:574] : INFO : Epoch 1998 | loss: 0.0310877 | val_loss: 0.0311616 | Time: 1497.51 ms [2022-03-11 12:37:52 main:574] : INFO : Epoch 1999 | loss: 0.0310833 | val_loss: 0.0311816 | Time: 1521.32 ms [2022-03-11 12:37:54 main:574] : INFO : Epoch 2000 | loss: 0.0310833 | val_loss: 0.0311743 | Time: 1504.27 ms [2022-03-11 12:37:55 main:574] : INFO : Epoch 2001 | loss: 0.0310831 | val_loss: 0.0311692 | Time: 1549.57 ms [2022-03-11 12:37:57 main:574] : INFO : Epoch 2002 | loss: 0.031085 | val_loss: 0.0311836 | Time: 1518.38 ms [2022-03-11 12:37:58 main:574] : INFO : Epoch 2003 | loss: 0.0310856 | val_loss: 0.031167 | Time: 1517.96 ms [2022-03-11 12:38:00 main:574] : INFO : Epoch 2004 | loss: 0.0310837 | val_loss: 0.0311774 | Time: 1490.75 ms [2022-03-11 12:38:01 main:574] : INFO : Epoch 2005 | loss: 0.0310844 | val_loss: 0.0311702 | Time: 1523.29 ms [2022-03-11 12:38:03 main:574] : INFO : Epoch 2006 | loss: 0.031082 | val_loss: 0.031177 | Time: 1523.6 ms [2022-03-11 12:38:04 main:574] : INFO : Epoch 2007 | loss: 0.0310817 | val_loss: 0.0311784 | Time: 1497.66 ms [2022-03-11 12:38:06 main:574] : INFO : Epoch 2008 | loss: 0.0310869 | val_loss: 0.0311597 | Time: 1492.11 ms [2022-03-11 12:38:07 main:574] : INFO : Epoch 2009 | loss: 0.0310884 | val_loss: 0.0311706 | Time: 1490.89 ms [2022-03-11 12:38:09 main:574] : INFO : Epoch 2010 | loss: 0.0310946 | val_loss: 0.0311689 | Time: 1542.94 ms [2022-03-11 12:38:10 main:574] : INFO : Epoch 2011 | loss: 0.0310952 | val_loss: 0.0311648 | Time: 1515.9 ms [2022-03-11 12:38:12 main:574] : INFO : Epoch 2012 | loss: 0.0310922 | val_loss: 0.0311687 | Time: 1537.94 ms [2022-03-11 12:38:14 main:574] : INFO : Epoch 2013 | loss: 0.0310959 | val_loss: 0.0311682 | Time: 1670.29 ms [2022-03-11 12:38:15 main:574] : INFO : Epoch 2014 | loss: 0.0310889 | val_loss: 0.0311659 | Time: 1526.24 ms [2022-03-11 12:38:17 main:574] : INFO : Epoch 2015 | loss: 0.0310881 | val_loss: 0.0311692 | Time: 1584.03 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-03-11 12:41:18 main:435] : INFO : Set logging level to 1 [2022-03-11 12:41:18 main:441] : INFO : Running in BOINC Client mode [2022-03-11 12:41:18 main:444] : INFO : Resolving all filenames [2022-03-11 12:41:18 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-11 12:41:18 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-11 12:41:18 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-11 12:41:18 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-11 12:41:18 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-11 12:41:18 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-11 12:41:18 main:474] : INFO : Configuration: [2022-03-11 12:41:18 main:475] : INFO : Model type: GRU [2022-03-11 12:41:18 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-11 12:41:18 main:477] : INFO : Max Epochs: 2048 [2022-03-11 12:41:18 main:478] : INFO : Batch Size: 128 [2022-03-11 12:41:18 main:479] : INFO : Learning Rate: 0.01 [2022-03-11 12:41:18 main:480] : INFO : Patience: 10 [2022-03-11 12:41:18 main:481] : INFO : Hidden Width: 12 [2022-03-11 12:41:18 main:482] : INFO : # Recurrent Layers: 4 [2022-03-11 12:41:18 main:483] : INFO : # Backend Layers: 4 [2022-03-11 12:41:18 main:484] : INFO : # Threads: 1 [2022-03-11 12:41:18 main:486] : INFO : Preparing Dataset [2022-03-11 12:41:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-11 12:41:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-11 12:41:20 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-11 12:41:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-11 12:41:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-11 12:41:20 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-11 12:41:20 main:494] : INFO : Creating Model [2022-03-11 12:41:20 main:507] : INFO : Preparing config file [2022-03-11 12:41:20 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-11 12:41:20 main:512] : INFO : Loading config [2022-03-11 12:41:20 main:514] : INFO : Loading state [2022-03-11 12:41:20 main:559] : INFO : Loading DataLoader into Memory [2022-03-11 12:41:20 main:562] : INFO : Starting Training [2022-03-11 12:41:22 main:574] : INFO : Epoch 1939 | loss: 0.031141 | val_loss: 0.0311693 | Time: 1662.72 ms [2022-03-11 12:41:23 main:574] : INFO : Epoch 1940 | loss: 0.0311045 | val_loss: 0.0311661 | Time: 1449.44 ms [2022-03-11 12:41:25 main:574] : INFO : Epoch 1941 | loss: 0.0311024 | val_loss: 0.031162 | Time: 1463.28 ms [2022-03-11 12:41:26 main:574] : INFO : Epoch 1942 | loss: 0.0311018 | val_loss: 0.0311611 | Time: 1445.16 ms [2022-03-11 12:41:28 main:574] : INFO : Epoch 1943 | loss: 0.0310966 | val_loss: 0.0311644 | Time: 1430.92 ms [2022-03-11 12:41:29 main:574] : INFO : Epoch 1944 | loss: 0.0310952 | val_loss: 0.031175 | Time: 1462.15 ms [2022-03-11 12:41:31 main:574] : INFO : Epoch 1945 | loss: 0.0310973 | val_loss: 0.0311753 | Time: 1436.45 ms [2022-03-11 12:41:32 main:574] : INFO : Epoch 1946 | loss: 0.0310972 | val_loss: 0.0311727 | Time: 1449.5 ms [2022-03-11 12:41:34 main:574] : INFO : Epoch 1947 | loss: 0.0310957 | val_loss: 0.0311688 | Time: 1457.55 ms [2022-03-11 12:41:35 main:574] : INFO : Epoch 1948 | loss: 0.0310967 | val_loss: 0.031162 | Time: 1453.42 ms [2022-03-11 12:41:37 main:574] : INFO : Epoch 1949 | loss: 0.0310981 | val_loss: 0.0311702 | Time: 1435.63 ms [2022-03-11 12:41:38 main:574] : INFO : Epoch 1950 | loss: 0.0311046 | val_loss: 0.0311579 | Time: 1441.34 ms [2022-03-11 12:41:39 main:574] : INFO : Epoch 1951 | loss: 0.0311016 | val_loss: 0.0311579 | Time: 1433.37 ms [2022-03-11 12:41:41 main:574] : INFO : Epoch 1952 | loss: 0.0310938 | val_loss: 0.0311622 | Time: 1458.55 ms [2022-03-11 12:41:42 main:574] : INFO : Epoch 1953 | loss: 0.0310924 | val_loss: 0.0311583 | Time: 1425.28 ms [2022-03-11 12:41:44 main:574] : INFO : Epoch 1954 | loss: 0.0311004 | val_loss: 0.031166 | Time: 1420.52 ms [2022-03-11 12:41:45 main:574] : INFO : Epoch 1955 | loss: 0.0311039 | val_loss: 0.0311619 | Time: 1468.87 ms [2022-03-11 12:41:47 main:574] : INFO : Epoch 1956 | loss: 0.0310994 | val_loss: 0.0311642 | Time: 1436.7 ms [2022-03-11 12:41:48 main:574] : INFO : Epoch 1957 | loss: 0.031095 | val_loss: 0.0311675 | Time: 1440.77 ms [2022-03-11 12:41:50 main:574] : INFO : Epoch 1958 | loss: 0.0310933 | val_loss: 0.031165 | Time: 1442.39 ms [2022-03-11 12:41:51 main:574] : INFO : Epoch 1959 | loss: 0.0310932 | val_loss: 0.0311658 | Time: 1445.43 ms [2022-03-11 12:41:52 main:574] : INFO : Epoch 1960 | loss: 0.0310907 | val_loss: 0.0311706 | Time: 1437.77 ms [2022-03-11 12:41:54 main:574] : INFO : Epoch 1961 | loss: 0.031089 | val_loss: 0.03116 | Time: 1452.35 ms [2022-03-11 12:41:55 main:574] : INFO : Epoch 1962 | loss: 0.0310909 | val_loss: 0.0311833 | Time: 1442.34 ms [2022-03-11 12:41:57 main:574] : INFO : Epoch 1963 | loss: 0.0310892 | val_loss: 0.0311741 | Time: 1455.98 ms [2022-03-11 12:41:58 main:574] : INFO : Epoch 1964 | loss: 0.0310881 | val_loss: 0.0311764 | Time: 1442.29 ms [2022-03-11 12:42:00 main:574] : INFO : Epoch 1965 | loss: 0.0310941 | val_loss: 0.0311787 | Time: 1442.12 ms [2022-03-11 12:42:01 main:574] : INFO : Epoch 1966 | loss: 0.0310905 | val_loss: 0.031175 | Time: 1430.77 ms [2022-03-11 12:42:03 main:574] : INFO : Epoch 1967 | loss: 0.0310915 | val_loss: 0.0311612 | Time: 1442.37 ms [2022-03-11 12:42:04 main:574] : INFO : Epoch 1968 | loss: 0.0310947 | val_loss: 0.0311685 | Time: 1451.24 ms [2022-03-11 12:42:06 main:574] : INFO : Epoch 1969 | loss: 0.0310998 | val_loss: 0.0311624 | Time: 1454.85 ms [2022-03-11 12:42:07 main:574] : INFO : Epoch 1970 | loss: 0.0310971 | val_loss: 0.0311603 | Time: 1444.7 ms [2022-03-11 12:42:08 main:574] : INFO : Epoch 1971 | loss: 0.0310946 | val_loss: 0.0311617 | Time: 1466.26 ms [2022-03-11 12:42:10 main:574] : INFO : Epoch 1972 | loss: 0.0310962 | val_loss: 0.0311692 | Time: 1451.85 ms [2022-03-11 12:42:11 main:574] : INFO : Epoch 1973 | loss: 0.0310938 | val_loss: 0.0311623 | Time: 1439.61 ms [2022-03-11 12:42:13 main:574] : INFO : Epoch 1974 | loss: 0.0310973 | val_loss: 0.0311749 | Time: 1432.02 ms [2022-03-11 12:42:14 main:574] : INFO : Epoch 1975 | loss: 0.0311133 | val_loss: 0.0311622 | Time: 1448.79 ms [2022-03-11 12:42:16 main:574] : INFO : Epoch 1976 | loss: 0.0311076 | val_loss: 0.031161 | Time: 1434.88 ms [2022-03-11 12:42:17 main:574] : INFO : Epoch 1977 | loss: 0.0311052 | val_loss: 0.0311664 | Time: 1450.65 ms [2022-03-11 12:42:19 main:574] : INFO : Epoch 1978 | loss: 0.0311042 | val_loss: 0.0311662 | Time: 1821.76 ms [2022-03-11 12:42:20 main:574] : INFO : Epoch 1979 | loss: 0.0311004 | val_loss: 0.031171 | Time: 1455.16 ms [2022-03-11 12:42:22 main:574] : INFO : Epoch 1980 | loss: 0.0310986 | val_loss: 0.0311644 | Time: 1460.15 ms [2022-03-11 12:42:23 main:574] : INFO : Epoch 1981 | loss: 0.0310974 | val_loss: 0.0311737 | Time: 1439.33 ms [2022-03-11 12:42:25 main:574] : INFO : Epoch 1982 | loss: 0.0310944 | val_loss: 0.0311634 | Time: 1465.19 ms [2022-03-11 12:42:26 main:574] : INFO : Epoch 1983 | loss: 0.031097 | val_loss: 0.0311659 | Time: 1423.18 ms [2022-03-11 12:42:28 main:574] : INFO : Epoch 1984 | loss: 0.0310992 | val_loss: 0.0311623 | Time: 1451.1 ms [2022-03-11 12:42:29 main:574] : INFO : Epoch 1985 | loss: 0.0310985 | val_loss: 0.0311665 | Time: 1430.52 ms [2022-03-11 12:42:31 main:574] : INFO : Epoch 1986 | loss: 0.0310925 | val_loss: 0.0311705 | Time: 1430.29 ms [2022-03-11 12:42:32 main:574] : INFO : Epoch 1987 | loss: 0.031089 | val_loss: 0.0311754 | Time: 1450.58 ms [2022-03-11 12:42:33 main:574] : INFO : Epoch 1988 | loss: 0.0310891 | val_loss: 0.0311849 | Time: 1438.17 ms [2022-03-11 12:42:35 main:574] : INFO : Epoch 1989 | loss: 0.031087 | val_loss: 0.0311714 | Time: 1483.24 ms [2022-03-11 12:42:36 main:574] : INFO : Epoch 1990 | loss: 0.0310885 | val_loss: 0.0311702 | Time: 1425.83 ms [2022-03-11 12:42:38 main:574] : INFO : Epoch 1991 | loss: 0.0310862 | val_loss: 0.0311634 | Time: 1444.76 ms [2022-03-11 12:42:39 main:574] : INFO : Epoch 1992 | loss: 0.0310862 | val_loss: 0.0311673 | Time: 1450.91 ms [2022-03-11 12:42:41 main:574] : INFO : Epoch 1993 | loss: 0.0310882 | val_loss: 0.0311726 | Time: 1433.61 ms [2022-03-11 12:42:42 main:574] : INFO : Epoch 1994 | loss: 0.0310954 | val_loss: 0.0311677 | Time: 1442.12 ms [2022-03-11 12:42:44 main:574] : INFO : Epoch 1995 | loss: 0.031099 | val_loss: 0.0311658 | Time: 1436.43 ms [2022-03-11 12:42:45 main:574] : INFO : Epoch 1996 | loss: 0.0310994 | val_loss: 0.0311637 | Time: 1452.27 ms [2022-03-11 12:42:47 main:574] : INFO : Epoch 1997 | loss: 0.0310991 | val_loss: 0.0311695 | Time: 1436.07 ms [2022-03-11 12:42:48 main:574] : INFO : Epoch 1998 | loss: 0.0310924 | val_loss: 0.0311652 | Time: 1425.67 ms [2022-03-11 12:42:49 main:574] : INFO : Epoch 1999 | loss: 0.0310905 | val_loss: 0.0311711 | Time: 1453.05 ms [2022-03-11 12:42:51 main:574] : INFO : Epoch 2000 | loss: 0.0310939 | val_loss: 0.0311791 | Time: 1913.45 ms [2022-03-11 12:42:53 main:574] : INFO : Epoch 2001 | loss: 0.0310928 | val_loss: 0.0311616 | Time: 1499.7 ms [2022-03-11 12:42:54 main:574] : INFO : Epoch 2002 | loss: 0.031091 | val_loss: 0.0311698 | Time: 1456.86 ms [2022-03-11 12:42:56 main:574] : INFO : Epoch 2003 | loss: 0.0310874 | val_loss: 0.031181 | Time: 1454.97 ms [2022-03-11 12:42:57 main:574] : INFO : Epoch 2004 | loss: 0.0310878 | val_loss: 0.0311688 | Time: 1431.38 ms [2022-03-11 12:42:59 main:574] : INFO : Epoch 2005 | loss: 0.0310848 | val_loss: 0.0311737 | Time: 1454.34 ms [2022-03-11 12:43:00 main:574] : INFO : Epoch 2006 | loss: 0.0310916 | val_loss: 0.031171 | Time: 1464.03 ms [2022-03-11 12:43:02 main:574] : INFO : Epoch 2007 | loss: 0.0310893 | val_loss: 0.0311712 | Time: 1460.11 ms [2022-03-11 12:43:03 main:574] : INFO : Epoch 2008 | loss: 0.0310863 | val_loss: 0.0311652 | Time: 1447.55 ms [2022-03-11 12:43:05 main:574] : INFO : Epoch 2009 | loss: 0.0310855 | val_loss: 0.031167 | Time: 1456.3 ms [2022-03-11 12:43:06 main:574] : INFO : Epoch 2010 | loss: 0.0310838 | val_loss: 0.031167 | Time: 1437.76 ms [2022-03-11 12:43:08 main:574] : INFO : Epoch 2011 | loss: 0.0310829 | val_loss: 0.0311669 | Time: 1443.89 ms [2022-03-11 12:43:09 main:574] : INFO : Epoch 2012 | loss: 0.0310835 | val_loss: 0.0311632 | Time: 1427.02 ms [2022-03-11 12:43:10 main:574] : INFO : Epoch 2013 | loss: 0.0310884 | val_loss: 0.031169 | Time: 1425 ms [2022-03-11 12:43:12 main:574] : INFO : Epoch 2014 | loss: 0.0310887 | val_loss: 0.0311688 | Time: 1452.48 ms [2022-03-11 12:43:13 main:574] : INFO : Epoch 2015 | loss: 0.031113 | val_loss: 0.0311636 | Time: 1443.61 ms [2022-03-11 12:43:15 main:574] : INFO : Epoch 2016 | loss: 0.0311084 | val_loss: 0.0311667 | Time: 1453.4 ms [2022-03-11 12:43:16 main:574] : INFO : Epoch 2017 | loss: 0.031103 | val_loss: 0.0311568 | Time: 1442.41 ms [2022-03-11 12:43:18 main:574] : INFO : Epoch 2018 | loss: 0.0310974 | val_loss: 0.0311717 | Time: 1555.37 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-03-11 14:01:35 main:435] : INFO : Set logging level to 1 [2022-03-11 14:01:35 main:441] : INFO : Running in BOINC Client mode [2022-03-11 14:01:35 main:444] : INFO : Resolving all filenames [2022-03-11 14:01:35 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-11 14:01:35 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-11 14:01:35 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-11 14:01:35 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-11 14:01:35 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-11 14:01:35 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-11 14:01:35 main:474] : INFO : Configuration: [2022-03-11 14:01:35 main:475] : INFO : Model type: GRU [2022-03-11 14:01:35 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-11 14:01:35 main:477] : INFO : Max Epochs: 2048 [2022-03-11 14:01:35 main:478] : INFO : Batch Size: 128 [2022-03-11 14:01:35 main:479] : INFO : Learning Rate: 0.01 [2022-03-11 14:01:35 main:480] : INFO : Patience: 10 [2022-03-11 14:01:35 main:481] : INFO : Hidden Width: 12 [2022-03-11 14:01:35 main:482] : INFO : # Recurrent Layers: 4 [2022-03-11 14:01:35 main:483] : INFO : # Backend Layers: 4 [2022-03-11 14:01:35 main:484] : INFO : # Threads: 1 [2022-03-11 14:01:35 main:486] : INFO : Preparing Dataset [2022-03-11 14:01:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-11 14:01:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-11 14:01:36 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-11 14:01:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-11 14:01:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-11 14:01:36 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-11 14:01:36 main:494] : INFO : Creating Model [2022-03-11 14:01:36 main:507] : INFO : Preparing config file [2022-03-11 14:01:36 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-11 14:01:36 main:512] : INFO : Loading config [2022-03-11 14:01:36 main:514] : INFO : Loading state [2022-03-11 14:01:37 main:559] : INFO : Loading DataLoader into Memory [2022-03-11 14:01:37 main:562] : INFO : Starting Training [2022-03-11 14:01:39 main:574] : INFO : Epoch 1939 | loss: 0.0311451 | val_loss: 0.0311694 | Time: 1667.99 ms [2022-03-11 14:01:40 main:574] : INFO : Epoch 1940 | loss: 0.0311016 | val_loss: 0.0311682 | Time: 1433.94 ms [2022-03-11 14:01:42 main:574] : INFO : Epoch 1941 | loss: 0.0310996 | val_loss: 0.0311703 | Time: 1438.51 ms [2022-03-11 14:01:43 main:574] : INFO : Epoch 1942 | loss: 0.0310999 | val_loss: 0.0311611 | Time: 1441.07 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-03-11 14:12:11 main:435] : INFO : Set logging level to 1 [2022-03-11 14:12:11 main:441] : INFO : Running in BOINC Client mode [2022-03-11 14:12:11 main:444] : INFO : Resolving all filenames [2022-03-11 14:12:11 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-11 14:12:11 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-11 14:12:11 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-11 14:12:11 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-11 14:12:11 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-11 14:12:11 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-11 14:12:11 main:474] : INFO : Configuration: [2022-03-11 14:12:11 main:475] : INFO : Model type: GRU [2022-03-11 14:12:11 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-11 14:12:11 main:477] : INFO : Max Epochs: 2048 [2022-03-11 14:12:11 main:478] : INFO : Batch Size: 128 [2022-03-11 14:12:11 main:479] : INFO : Learning Rate: 0.01 [2022-03-11 14:12:11 main:480] : INFO : Patience: 10 [2022-03-11 14:12:11 main:481] : INFO : Hidden Width: 12 [2022-03-11 14:12:11 main:482] : INFO : # Recurrent Layers: 4 [2022-03-11 14:12:11 main:483] : INFO : # Backend Layers: 4 [2022-03-11 14:12:11 main:484] : INFO : # Threads: 1 [2022-03-11 14:12:11 main:486] : INFO : Preparing Dataset [2022-03-11 14:12:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-11 14:12:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-11 14:12:12 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-11 14:12:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-11 14:12:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-11 14:12:13 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-11 14:12:13 main:494] : INFO : Creating Model [2022-03-11 14:12:13 main:507] : INFO : Preparing config file [2022-03-11 14:12:13 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-11 14:12:13 main:512] : INFO : Loading config [2022-03-11 14:12:13 main:514] : INFO : Loading state [2022-03-11 14:12:13 main:559] : INFO : Loading DataLoader into Memory [2022-03-11 14:12:13 main:562] : INFO : Starting Training [2022-03-11 14:12:15 main:574] : INFO : Epoch 1939 | loss: 0.0311418 | val_loss: 0.0311674 | Time: 1793.22 ms [2022-03-11 14:12:17 main:574] : INFO : Epoch 1940 | loss: 0.0311036 | val_loss: 0.0311778 | Time: 1624.81 ms [2022-03-11 14:12:19 main:574] : INFO : Epoch 1941 | loss: 0.0310981 | val_loss: 0.0311643 | Time: 2305.79 ms [2022-03-11 14:12:22 main:574] : INFO : Epoch 1942 | loss: 0.0310965 | val_loss: 0.031178 | Time: 3227.4 ms [2022-03-11 14:12:25 main:574] : INFO : Epoch 1943 | loss: 0.0310988 | val_loss: 0.0311677 | Time: 3261.48 ms [2022-03-11 14:12:29 main:574] : INFO : Epoch 1944 | loss: 0.0310958 | val_loss: 0.0311733 | Time: 3282.8 ms [2022-03-11 14:12:32 main:574] : INFO : Epoch 1945 | loss: 0.031097 | val_loss: 0.0311664 | Time: 3278.99 ms [2022-03-11 14:12:35 main:574] : INFO : Epoch 1946 | loss: 0.0310959 | val_loss: 0.0311614 | Time: 3301.66 ms [2022-03-11 14:12:39 main:574] : INFO : Epoch 1947 | loss: 0.0310956 | val_loss: 0.0311641 | Time: 3308.45 ms [2022-03-11 14:12:42 main:574] : INFO : Epoch 1948 | loss: 0.0310947 | val_loss: 0.0311701 | Time: 3319 ms [2022-03-11 14:12:45 main:574] : INFO : Epoch 1949 | loss: 0.0310927 | val_loss: 0.0311723 | Time: 3311.32 ms [2022-03-11 14:12:49 main:574] : INFO : Epoch 1950 | loss: 0.0310926 | val_loss: 0.0311672 | Time: 3312.43 ms [2022-03-11 14:12:52 main:574] : INFO : Epoch 1951 | loss: 0.0310908 | val_loss: 0.0311642 | Time: 3311.98 ms [2022-03-11 14:12:55 main:574] : INFO : Epoch 1952 | loss: 0.031089 | val_loss: 0.03117 | Time: 3319.26 ms [2022-03-11 14:12:59 main:574] : INFO : Epoch 1953 | loss: 0.0310984 | val_loss: 0.031168 | Time: 3317.87 ms [2022-03-11 14:13:02 main:574] : INFO : Epoch 1954 | loss: 0.0310963 | val_loss: 0.0311802 | Time: 3316.14 ms [2022-03-11 14:13:05 main:574] : INFO : Epoch 1955 | loss: 0.0310923 | val_loss: 0.0311678 | Time: 3317.73 ms [2022-03-11 14:13:09 main:574] : INFO : Epoch 1956 | loss: 0.0310927 | val_loss: 0.0311754 | Time: 3317.41 ms [2022-03-11 14:13:12 main:574] : INFO : Epoch 1957 | loss: 0.0310943 | val_loss: 0.0311678 | Time: 3317.97 ms [2022-03-11 14:13:15 main:574] : INFO : Epoch 1958 | loss: 0.0310926 | val_loss: 0.0311632 | Time: 3314 ms [2022-03-11 14:13:19 main:574] : INFO : Epoch 1959 | loss: 0.0310961 | val_loss: 0.031177 | Time: 3314.8 ms [2022-03-11 14:13:22 main:574] : INFO : Epoch 1960 | loss: 0.0310927 | val_loss: 0.0311717 | Time: 3319.54 ms [2022-03-11 14:13:25 main:574] : INFO : Epoch 1961 | loss: 0.0310912 | val_loss: 0.031176 | Time: 3312.52 ms [2022-03-11 14:13:28 main:574] : INFO : Epoch 1962 | loss: 0.0310934 | val_loss: 0.0311772 | Time: 3317.13 ms [2022-03-11 14:13:32 main:574] : INFO : Epoch 1963 | loss: 0.0310937 | val_loss: 0.0311719 | Time: 3316.06 ms [2022-03-11 14:13:35 main:574] : INFO : Epoch 1964 | loss: 0.0310929 | val_loss: 0.0311645 | Time: 3323.73 ms [2022-03-11 14:13:38 main:574] : INFO : Epoch 1965 | loss: 0.0310898 | val_loss: 0.0311672 | Time: 3314.38 ms [2022-03-11 14:13:42 main:574] : INFO : Epoch 1966 | loss: 0.0310887 | val_loss: 0.0311819 | Time: 3324.07 ms [2022-03-11 14:13:45 main:574] : INFO : Epoch 1967 | loss: 0.0310896 | val_loss: 0.0311801 | Time: 3314.57 ms [2022-03-11 14:13:48 main:574] : INFO : Epoch 1968 | loss: 0.0310953 | val_loss: 0.0311744 | Time: 3314.59 ms [2022-03-11 14:13:52 main:574] : INFO : Epoch 1969 | loss: 0.0310912 | val_loss: 0.0311662 | Time: 3316.68 ms [2022-03-11 14:13:55 main:574] : INFO : Epoch 1970 | loss: 0.0310883 | val_loss: 0.0311745 | Time: 3321.31 ms [2022-03-11 14:13:58 main:574] : INFO : Epoch 1971 | loss: 0.0310884 | val_loss: 0.0311624 | Time: 3304.75 ms [2022-03-11 14:14:02 main:574] : INFO : Epoch 1972 | loss: 0.0310887 | val_loss: 0.0311772 | Time: 3316.54 ms [2022-03-11 14:14:05 main:574] : INFO : Epoch 1973 | loss: 0.0310878 | val_loss: 0.0311835 | Time: 3323.24 ms [2022-03-11 14:14:08 main:574] : INFO : Epoch 1974 | loss: 0.0310857 | val_loss: 0.031178 | Time: 3309.07 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-04-23 15:42:04 main:435] : INFO : Set logging level to 1 [2022-04-23 15:42:04 main:441] : INFO : Running in BOINC Client mode [2022-04-23 15:42:04 main:444] : INFO : Resolving all filenames [2022-04-23 15:42:04 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-23 15:42:04 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-23 15:42:04 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-23 15:42:04 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-23 15:42:04 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-23 15:42:04 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-23 15:42:04 main:474] : INFO : Configuration: [2022-04-23 15:42:04 main:475] : INFO : Model type: GRU [2022-04-23 15:42:04 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-23 15:42:04 main:477] : INFO : Max Epochs: 2048 [2022-04-23 15:42:04 main:478] : INFO : Batch Size: 128 [2022-04-23 15:42:04 main:479] : INFO : Learning Rate: 0.01 [2022-04-23 15:42:04 main:480] : INFO : Patience: 10 [2022-04-23 15:42:04 main:481] : INFO : Hidden Width: 12 [2022-04-23 15:42:04 main:482] : INFO : # Recurrent Layers: 4 [2022-04-23 15:42:04 main:483] : INFO : # Backend Layers: 4 [2022-04-23 15:42:04 main:484] : INFO : # Threads: 1 [2022-04-23 15:42:04 main:486] : INFO : Preparing Dataset [2022-04-23 15:42:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-23 15:42:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-23 15:42:07 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-23 15:42:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-23 15:42:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-23 15:42:07 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-23 15:42:07 main:494] : INFO : Creating Model [2022-04-23 15:42:07 main:507] : INFO : Preparing config file [2022-04-23 15:42:07 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-23 15:42:07 main:512] : INFO : Loading config [2022-04-23 15:42:07 main:514] : INFO : Loading state [2022-04-23 15:42:08 main:559] : INFO : Loading DataLoader into Memory [2022-04-23 15:42:08 main:562] : INFO : Starting Training [2022-04-23 15:42:10 main:574] : INFO : Epoch 1939 | loss: 0.0311335 | val_loss: 0.0311702 | Time: 2058.36 ms [2022-04-23 15:42:12 main:574] : INFO : Epoch 1940 | loss: 0.031102 | val_loss: 0.0311656 | Time: 1455.79 ms [2022-04-23 15:42:13 main:574] : INFO : Epoch 1941 | loss: 0.0310962 | val_loss: 0.0311651 | Time: 1472.55 ms [2022-04-23 15:42:15 main:574] : INFO : Epoch 1942 | loss: 0.0310935 | val_loss: 0.0311733 | Time: 1490.44 ms [2022-04-23 15:42:16 main:574] : INFO : Epoch 1943 | loss: 0.0310929 | val_loss: 0.0311648 | Time: 1489.29 ms [2022-04-23 15:42:18 main:574] : INFO : Epoch 1944 | loss: 0.031093 | val_loss: 0.0311688 | Time: 1463.88 ms [2022-04-23 15:42:19 main:574] : INFO : Epoch 1945 | loss: 0.0310982 | val_loss: 0.0311682 | Time: 1491.32 ms [2022-04-23 15:42:21 main:574] : INFO : Epoch 1946 | loss: 0.0310995 | val_loss: 0.0311691 | Time: 1480.46 ms [2022-04-23 15:42:22 main:574] : INFO : Epoch 1947 | loss: 0.0310954 | val_loss: 0.0311667 | Time: 1466.24 ms [2022-04-23 15:42:24 main:574] : INFO : Epoch 1948 | loss: 0.0310952 | val_loss: 0.0311718 | Time: 1486.34 ms [2022-04-23 15:42:25 main:574] : INFO : Epoch 1949 | loss: 0.0310922 | val_loss: 0.0311788 | Time: 1476.03 ms [2022-04-23 15:42:27 main:574] : INFO : Epoch 1950 | loss: 0.0310951 | val_loss: 0.0311708 | Time: 1468.64 ms [2022-04-23 15:42:28 main:574] : INFO : Epoch 1951 | loss: 0.031096 | val_loss: 0.0311774 | Time: 1477.82 ms [2022-04-23 15:42:30 main:574] : INFO : Epoch 1952 | loss: 0.0310935 | val_loss: 0.031173 | Time: 1474.69 ms [2022-04-23 15:42:31 main:574] : INFO : Epoch 1953 | loss: 0.0310934 | val_loss: 0.0311688 | Time: 1463.44 ms [2022-04-23 15:42:33 main:574] : INFO : Epoch 1954 | loss: 0.0310929 | val_loss: 0.0311845 | Time: 1478.78 ms [2022-04-23 15:42:34 main:574] : INFO : Epoch 1955 | loss: 0.031089 | val_loss: 0.0311657 | Time: 1479.23 ms [2022-04-23 15:42:36 main:574] : INFO : Epoch 1956 | loss: 0.0310943 | val_loss: 0.0311779 | Time: 1464.38 ms [2022-04-23 15:42:37 main:574] : INFO : Epoch 1957 | loss: 0.0310904 | val_loss: 0.0311813 | Time: 1456.19 ms [2022-04-23 15:42:38 main:574] : INFO : Epoch 1958 | loss: 0.0310914 | val_loss: 0.031168 | Time: 1452.73 ms [2022-04-23 15:42:40 main:574] : INFO : Epoch 1959 | loss: 0.0310929 | val_loss: 0.0311858 | Time: 1450.11 ms [2022-04-23 15:42:41 main:574] : INFO : Epoch 1960 | loss: 0.0310899 | val_loss: 0.0311762 | Time: 1496.54 ms [2022-04-23 15:42:43 main:574] : INFO : Epoch 1961 | loss: 0.0310876 | val_loss: 0.0311754 | Time: 1480.63 ms [2022-04-23 15:42:44 main:574] : INFO : Epoch 1962 | loss: 0.0310884 | val_loss: 0.031176 | Time: 1481.48 ms [2022-04-23 15:42:46 main:574] : INFO : Epoch 1963 | loss: 0.0310957 | val_loss: 0.0311793 | Time: 1485.03 ms [2022-04-23 15:42:47 main:574] : INFO : Epoch 1964 | loss: 0.0310945 | val_loss: 0.0311695 | Time: 1479.29 ms [2022-04-23 15:42:49 main:574] : INFO : Epoch 1965 | loss: 0.0310927 | val_loss: 0.0311649 | Time: 1463.54 ms [2022-04-23 15:42:50 main:574] : INFO : Epoch 1966 | loss: 0.0310895 | val_loss: 0.0311866 | Time: 1507.32 ms [2022-04-23 15:42:52 main:574] : INFO : Epoch 1967 | loss: 0.0310858 | val_loss: 0.0311802 | Time: 1483.93 ms [2022-04-23 15:42:53 main:574] : INFO : Epoch 1968 | loss: 0.0310889 | val_loss: 0.0311695 | Time: 1502.12 ms [2022-04-23 15:42:55 main:574] : INFO : Epoch 1969 | loss: 0.0310945 | val_loss: 0.0311798 | Time: 1493.88 ms [2022-04-23 15:42:56 main:574] : INFO : Epoch 1970 | loss: 0.031093 | val_loss: 0.0311826 | Time: 1491.84 ms [2022-04-23 15:42:58 main:574] : INFO : Epoch 1971 | loss: 0.0310903 | val_loss: 0.0311734 | Time: 1473.52 ms [2022-04-23 15:42:59 main:574] : INFO : Epoch 1972 | loss: 0.0310876 | val_loss: 0.0311802 | Time: 1536.43 ms [2022-04-23 15:43:01 main:574] : INFO : Epoch 1973 | loss: 0.0310876 | val_loss: 0.0311846 | Time: 1491.76 ms [2022-04-23 15:43:02 main:574] : INFO : Epoch 1974 | loss: 0.0311038 | val_loss: 0.0311552 | Time: 1476.58 ms [2022-04-23 15:43:04 main:574] : INFO : Epoch 1975 | loss: 0.0311202 | val_loss: 0.0311567 | Time: 1504.15 ms [2022-04-23 15:43:05 main:574] : INFO : Epoch 1976 | loss: 0.0311184 | val_loss: 0.0311544 | Time: 1472.64 ms [2022-04-23 15:43:07 main:574] : INFO : Epoch 1977 | loss: 0.0311115 | val_loss: 0.0311608 | Time: 1544.2 ms [2022-04-23 15:43:08 main:574] : INFO : Epoch 1978 | loss: 0.031109 | val_loss: 0.0311583 | Time: 1645.37 ms [2022-04-23 15:43:10 main:574] : INFO : Epoch 1979 | loss: 0.0311042 | val_loss: 0.031158 | Time: 1515.53 ms [2022-04-23 15:43:11 main:574] : INFO : Epoch 1980 | loss: 0.0310997 | val_loss: 0.0311598 | Time: 1455.63 ms [2022-04-23 15:43:13 main:574] : INFO : Epoch 1981 | loss: 0.0310987 | val_loss: 0.031161 | Time: 1437.2 ms [2022-04-23 15:43:14 main:574] : INFO : Epoch 1982 | loss: 0.031099 | val_loss: 0.0311615 | Time: 1419.82 ms [2022-04-23 15:43:16 main:574] : INFO : Epoch 1983 | loss: 0.0310991 | val_loss: 0.0311672 | Time: 1445.72 ms [2022-04-23 15:43:17 main:574] : INFO : Epoch 1984 | loss: 0.0310956 | val_loss: 0.0311731 | Time: 1453.72 ms [2022-04-23 15:43:19 main:574] : INFO : Epoch 1985 | loss: 0.0310942 | val_loss: 0.0311673 | Time: 1459.83 ms [2022-04-23 15:43:20 main:574] : INFO : Epoch 1986 | loss: 0.0310942 | val_loss: 0.0311632 | Time: 1434.62 ms [2022-04-23 15:43:22 main:574] : INFO : Epoch 1987 | loss: 0.0310928 | val_loss: 0.0311657 | Time: 1437.97 ms [2022-04-23 15:43:23 main:574] : INFO : Epoch 1988 | loss: 0.0310934 | val_loss: 0.0311666 | Time: 1434.55 ms [2022-04-23 15:43:24 main:574] : INFO : Epoch 1989 | loss: 0.0310924 | val_loss: 0.0311708 | Time: 1449.63 ms [2022-04-23 15:43:26 main:574] : INFO : Epoch 1990 | loss: 0.0310958 | val_loss: 0.0311708 | Time: 1446.71 ms [2022-04-23 15:43:27 main:574] : INFO : Epoch 1991 | loss: 0.031092 | val_loss: 0.0311656 | Time: 1433.89 ms [2022-04-23 15:43:29 main:574] : INFO : Epoch 1992 | loss: 0.0310908 | val_loss: 0.0311648 | Time: 1455.16 ms [2022-04-23 15:43:30 main:574] : INFO : Epoch 1993 | loss: 0.0310938 | val_loss: 0.0311697 | Time: 1446.72 ms [2022-04-23 15:43:32 main:574] : INFO : Epoch 1994 | loss: 0.0310915 | val_loss: 0.0311697 | Time: 1471.16 ms [2022-04-23 15:43:33 main:574] : INFO : Epoch 1995 | loss: 0.0310882 | val_loss: 0.0311662 | Time: 1427.38 ms [2022-04-23 15:43:35 main:574] : INFO : Epoch 1996 | loss: 0.0310881 | val_loss: 0.031185 | Time: 1430.75 ms [2022-04-23 15:43:36 main:574] : INFO : Epoch 1997 | loss: 0.0310863 | val_loss: 0.0311723 | Time: 1450.06 ms [2022-04-23 15:43:38 main:574] : INFO : Epoch 1998 | loss: 0.0310927 | val_loss: 0.0311826 | Time: 1453.13 ms [2022-04-23 15:43:39 main:574] : INFO : Epoch 1999 | loss: 0.0310911 | val_loss: 0.0311806 | Time: 1450.83 ms [2022-04-23 15:43:41 main:574] : INFO : Epoch 2000 | loss: 0.0310906 | val_loss: 0.0311755 | Time: 1539.55 ms [2022-04-23 15:43:42 main:574] : INFO : Epoch 2001 | loss: 0.0311002 | val_loss: 0.0311704 | Time: 1704.09 ms [2022-04-23 15:43:44 main:574] : INFO : Epoch 2002 | loss: 0.0310949 | val_loss: 0.0311836 | Time: 1635.61 ms [2022-04-23 15:43:45 main:574] : INFO : Epoch 2003 | loss: 0.0310934 | val_loss: 0.0311646 | Time: 1492.16 ms [2022-04-23 15:43:47 main:574] : INFO : Epoch 2004 | loss: 0.0310896 | val_loss: 0.0311795 | Time: 1537.63 ms [2022-04-23 15:43:49 main:574] : INFO : Epoch 2005 | loss: 0.0310901 | val_loss: 0.0311713 | Time: 1617.85 ms [2022-04-23 15:43:50 main:574] : INFO : Epoch 2006 | loss: 0.031091 | val_loss: 0.031167 | Time: 1715.7 ms [2022-04-23 15:43:52 main:574] : INFO : Epoch 2007 | loss: 0.03109 | val_loss: 0.0311916 | Time: 1452.79 ms [2022-04-23 15:43:53 main:574] : INFO : Epoch 2008 | loss: 0.0310915 | val_loss: 0.0311773 | Time: 1431.03 ms [2022-04-23 15:43:55 main:574] : INFO : Epoch 2009 | loss: 0.0310883 | val_loss: 0.0311791 | Time: 1449.84 ms [2022-04-23 15:43:56 main:574] : INFO : Epoch 2010 | loss: 0.0310873 | val_loss: 0.0311793 | Time: 1480.72 ms [2022-04-23 15:43:58 main:574] : INFO : Epoch 2011 | loss: 0.0310857 | val_loss: 0.0311903 | Time: 1722.84 ms [2022-04-23 15:43:59 main:574] : INFO : Epoch 2012 | loss: 0.0310872 | val_loss: 0.0311767 | Time: 1454.91 ms [2022-04-23 15:44:01 main:574] : INFO : Epoch 2013 | loss: 0.0310877 | val_loss: 0.0311742 | Time: 1437.75 ms [2022-04-23 15:44:02 main:574] : INFO : Epoch 2014 | loss: 0.0310995 | val_loss: 0.0311827 | Time: 1461.56 ms [2022-04-23 15:44:04 main:574] : INFO : Epoch 2015 | loss: 0.0310935 | val_loss: 0.0311813 | Time: 1446.52 ms [2022-04-23 15:44:05 main:574] : INFO : Epoch 2016 | loss: 0.0310893 | val_loss: 0.0311716 | Time: 1439.42 ms [2022-04-23 15:44:07 main:574] : INFO : Epoch 2017 | loss: 0.0310902 | val_loss: 0.0311747 | Time: 1444.21 ms [2022-04-23 15:44:08 main:574] : INFO : Epoch 2018 | loss: 0.0310921 | val_loss: 0.0311701 | Time: 1464.58 ms [2022-04-23 15:44:09 main:574] : INFO : Epoch 2019 | loss: 0.031092 | val_loss: 0.0311708 | Time: 1466.59 ms [2022-04-23 15:44:11 main:574] : INFO : Epoch 2020 | loss: 0.031089 | val_loss: 0.0311736 | Time: 1450.37 ms [2022-04-23 15:44:12 main:574] : INFO : Epoch 2021 | loss: 0.0310897 | val_loss: 0.0311712 | Time: 1445.41 ms [2022-04-23 15:44:14 main:574] : INFO : Epoch 2022 | loss: 0.03109 | val_loss: 0.0311783 | Time: 1433.32 ms [2022-04-23 15:44:15 main:574] : INFO : Epoch 2023 | loss: 0.031087 | val_loss: 0.0311689 | Time: 1589.05 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-04-30 20:47:29 main:435] : INFO : Set logging level to 1 [2022-04-30 20:47:29 main:441] : INFO : Running in BOINC Client mode [2022-04-30 20:47:29 main:444] : INFO : Resolving all filenames [2022-04-30 20:47:29 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-30 20:47:29 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-30 20:47:29 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-30 20:47:29 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-30 20:47:29 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-30 20:47:29 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-30 20:47:29 main:474] : INFO : Configuration: [2022-04-30 20:47:29 main:475] : INFO : Model type: GRU [2022-04-30 20:47:29 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-30 20:47:29 main:477] : INFO : Max Epochs: 2048 [2022-04-30 20:47:29 main:478] : INFO : Batch Size: 128 [2022-04-30 20:47:29 main:479] : INFO : Learning Rate: 0.01 [2022-04-30 20:47:29 main:480] : INFO : Patience: 10 [2022-04-30 20:47:29 main:481] : INFO : Hidden Width: 12 [2022-04-30 20:47:29 main:482] : INFO : # Recurrent Layers: 4 [2022-04-30 20:47:29 main:483] : INFO : # Backend Layers: 4 [2022-04-30 20:47:29 main:484] : INFO : # Threads: 1 [2022-04-30 20:47:29 main:486] : INFO : Preparing Dataset [2022-04-30 20:47:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-30 20:47:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-30 20:47:31 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-30 20:47:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-30 20:47:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-30 20:47:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-30 20:47:31 main:494] : INFO : Creating Model [2022-04-30 20:47:31 main:507] : INFO : Preparing config file [2022-04-30 20:47:31 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-30 20:47:31 main:512] : INFO : Loading config [2022-04-30 20:47:31 main:514] : INFO : Loading state [2022-04-30 20:47:33 main:559] : INFO : Loading DataLoader into Memory [2022-04-30 20:47:33 main:562] : INFO : Starting Training [2022-04-30 20:47:35 main:574] : INFO : Epoch 1939 | loss: 0.0311485 | val_loss: 0.0311782 | Time: 2144.58 ms [2022-04-30 20:47:36 main:574] : INFO : Epoch 1940 | loss: 0.0311049 | val_loss: 0.0311742 | Time: 1502.57 ms [2022-04-30 20:47:38 main:574] : INFO : Epoch 1941 | loss: 0.0311005 | val_loss: 0.0311642 | Time: 1494.99 ms [2022-04-30 20:47:39 main:574] : INFO : Epoch 1942 | loss: 0.0310965 | val_loss: 0.0311813 | Time: 1496.33 ms [2022-04-30 20:47:41 main:574] : INFO : Epoch 1943 | loss: 0.0310928 | val_loss: 0.0311785 | Time: 1495.6 ms [2022-04-30 20:47:42 main:574] : INFO : Epoch 1944 | loss: 0.031092 | val_loss: 0.0311671 | Time: 1527.24 ms [2022-04-30 20:47:44 main:574] : INFO : Epoch 1945 | loss: 0.0310962 | val_loss: 0.0311712 | Time: 1487.35 ms [2022-04-30 20:47:45 main:574] : INFO : Epoch 1946 | loss: 0.0310916 | val_loss: 0.0311733 | Time: 1487.38 ms [2022-04-30 20:47:47 main:574] : INFO : Epoch 1947 | loss: 0.0310937 | val_loss: 0.0311824 | Time: 1495.52 ms [2022-04-30 20:47:48 main:574] : INFO : Epoch 1948 | loss: 0.0310914 | val_loss: 0.0311647 | Time: 1494.77 ms [2022-04-30 20:47:50 main:574] : INFO : Epoch 1949 | loss: 0.0310898 | val_loss: 0.0311772 | Time: 1513.78 ms [2022-04-30 20:47:51 main:574] : INFO : Epoch 1950 | loss: 0.0310885 | val_loss: 0.0311672 | Time: 1488.06 ms [2022-04-30 20:47:53 main:574] : INFO : Epoch 1951 | loss: 0.0310916 | val_loss: 0.0311755 | Time: 1490.67 ms [2022-04-30 20:47:54 main:574] : INFO : Epoch 1952 | loss: 0.0310929 | val_loss: 0.0311841 | Time: 1508.14 ms [2022-04-30 20:47:56 main:574] : INFO : Epoch 1953 | loss: 0.0310948 | val_loss: 0.0311679 | Time: 1514.55 ms [2022-04-30 20:47:57 main:574] : INFO : Epoch 1954 | loss: 0.031092 | val_loss: 0.0311656 | Time: 1502.33 ms [2022-04-30 20:47:59 main:574] : INFO : Epoch 1955 | loss: 0.031089 | val_loss: 0.0311651 | Time: 1514.7 ms [2022-04-30 20:48:00 main:574] : INFO : Epoch 1956 | loss: 0.0310887 | val_loss: 0.0311626 | Time: 1508.59 ms [2022-04-30 20:48:02 main:574] : INFO : Epoch 1957 | loss: 0.0310935 | val_loss: 0.0311661 | Time: 1506.89 ms [2022-04-30 20:48:04 main:574] : INFO : Epoch 1958 | loss: 0.0310976 | val_loss: 0.0311747 | Time: 1513.04 ms [2022-04-30 20:48:05 main:574] : INFO : Epoch 1959 | loss: 0.0310968 | val_loss: 0.031171 | Time: 1519.88 ms [2022-04-30 20:48:07 main:574] : INFO : Epoch 1960 | loss: 0.0310965 | val_loss: 0.0311677 | Time: 1496.85 ms [2022-04-30 20:48:08 main:574] : INFO : Epoch 1961 | loss: 0.0310977 | val_loss: 0.0311641 | Time: 1501.78 ms [2022-04-30 20:48:10 main:574] : INFO : Epoch 1962 | loss: 0.0310969 | val_loss: 0.0311641 | Time: 1517.05 ms [2022-04-30 20:48:11 main:574] : INFO : Epoch 1963 | loss: 0.0310994 | val_loss: 0.031166 | Time: 1504.86 ms [2022-04-30 20:48:13 main:574] : INFO : Epoch 1964 | loss: 0.0311041 | val_loss: 0.0311586 | Time: 1496.61 ms [2022-04-30 20:48:14 main:574] : INFO : Epoch 1965 | loss: 0.0311054 | val_loss: 0.0311589 | Time: 1511.21 ms [2022-04-30 20:48:16 main:574] : INFO : Epoch 1966 | loss: 0.0310999 | val_loss: 0.0311592 | Time: 1524.42 ms [2022-04-30 20:48:17 main:574] : INFO : Epoch 1967 | loss: 0.0310953 | val_loss: 0.0311591 | Time: 1512.91 ms [2022-04-30 20:48:19 main:574] : INFO : Epoch 1968 | loss: 0.0310933 | val_loss: 0.031161 | Time: 1568.28 ms [2022-04-30 20:48:20 main:574] : INFO : Epoch 1969 | loss: 0.0310932 | val_loss: 0.0311652 | Time: 1717.74 ms [2022-04-30 20:48:22 main:574] : INFO : Epoch 1970 | loss: 0.0310931 | val_loss: 0.0311728 | Time: 1576.22 ms [2022-04-30 20:48:24 main:574] : INFO : Epoch 1971 | loss: 0.0310957 | val_loss: 0.0311612 | Time: 1498.8 ms [2022-04-30 20:48:25 main:574] : INFO : Epoch 1972 | loss: 0.0311006 | val_loss: 0.0311657 | Time: 1504.69 ms [2022-04-30 20:48:27 main:574] : INFO : Epoch 1973 | loss: 0.0310964 | val_loss: 0.0311609 | Time: 1513.58 ms [2022-04-30 20:54:12 main:574] : INFO : Epoch 1974 | loss: 0.0310918 | val_loss: 0.0311735 | Time: 345712 ms [2022-04-30 20:54:14 main:574] : INFO : Epoch 1975 | loss: 0.0310946 | val_loss: 0.0311738 | Time: 1845.47 ms [2022-04-30 20:54:16 main:574] : INFO : Epoch 1976 | loss: 0.0310938 | val_loss: 0.0311631 | Time: 1819.54 ms [2022-04-30 20:54:18 main:574] : INFO : Epoch 1977 | loss: 0.0310928 | val_loss: 0.0311795 | Time: 1868.38 ms [2022-04-30 20:54:20 main:574] : INFO : Epoch 1978 | loss: 0.0310936 | val_loss: 0.0311721 | Time: 2111.09 ms [2022-04-30 20:54:42 main:574] : INFO : Epoch 1979 | loss: 0.0310915 | val_loss: 0.031164 | Time: 22420.4 ms [2022-04-30 20:54:44 main:574] : INFO : Epoch 1980 | loss: 0.0310986 | val_loss: 0.0311621 | Time: 1946.78 ms [2022-04-30 20:54:46 main:574] : INFO : Epoch 1981 | loss: 0.0310961 | val_loss: 0.031163 | Time: 1854.36 ms [2022-04-30 20:54:48 main:574] : INFO : Epoch 1982 | loss: 0.0310953 | val_loss: 0.0311698 | Time: 1959.23 ms [2022-04-30 20:54:50 main:574] : INFO : Epoch 1983 | loss: 0.0310974 | val_loss: 0.0311603 | Time: 1882.54 ms [2022-04-30 20:54:52 main:574] : INFO : Epoch 1984 | loss: 0.0310958 | val_loss: 0.0311707 | Time: 1918.26 ms [2022-04-30 20:54:54 main:574] : INFO : Epoch 1985 | loss: 0.0310999 | val_loss: 0.0311663 | Time: 1860.95 ms [2022-04-30 20:54:56 main:574] : INFO : Epoch 1986 | loss: 0.0310976 | val_loss: 0.0311747 | Time: 1963.47 ms [2022-04-30 20:54:58 main:574] : INFO : Epoch 1987 | loss: 0.0310958 | val_loss: 0.0311692 | Time: 2062.67 ms [2022-04-30 20:55:00 main:574] : INFO : Epoch 1988 | loss: 0.0310997 | val_loss: 0.0311703 | Time: 2071.23 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2022-05-02 12:57:03 main:435] : INFO : Set logging level to 1 [2022-05-02 12:57:03 main:441] : INFO : Running in BOINC Client mode [2022-05-02 12:57:03 main:444] : INFO : Resolving all filenames [2022-05-02 12:57:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-05-02 12:57:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-05-02 12:57:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-05-02 12:57:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-05-02 12:57:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-05-02 12:57:03 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-05-02 12:57:03 main:474] : INFO : Configuration: [2022-05-02 12:57:03 main:475] : INFO : Model type: GRU [2022-05-02 12:57:03 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-05-02 12:57:03 main:477] : INFO : Max Epochs: 2048 [2022-05-02 12:57:03 main:478] : INFO : Batch Size: 128 [2022-05-02 12:57:03 main:479] : INFO : Learning Rate: 0.01 [2022-05-02 12:57:03 main:480] : INFO : Patience: 10 [2022-05-02 12:57:03 main:481] : INFO : Hidden Width: 12 [2022-05-02 12:57:03 main:482] : INFO : # Recurrent Layers: 4 [2022-05-02 12:57:03 main:483] : INFO : # Backend Layers: 4 [2022-05-02 12:57:03 main:484] : INFO : # Threads: 1 [2022-05-02 12:57:03 main:486] : INFO : Preparing Dataset [2022-05-02 12:57:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-05-02 12:57:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-05-02 12:57:06 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-05-02 12:57:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-05-02 12:57:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-05-02 12:57:06 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-05-02 12:57:06 main:494] : INFO : Creating Model [2022-05-02 12:57:06 main:507] : INFO : Preparing config file [2022-05-02 12:57:06 main:511] : INFO : Found checkpoint, attempting to load... [2022-05-02 12:57:06 main:512] : INFO : Loading config [2022-05-02 12:57:06 main:514] : INFO : Loading state [2022-05-02 12:57:07 main:559] : INFO : Loading DataLoader into Memory [2022-05-02 12:57:07 main:562] : INFO : Starting Training [2022-05-02 12:57:09 main:574] : INFO : Epoch 1975 | loss: 0.0311334 | val_loss: 0.0311763 | Time: 1964.28 ms [2022-05-02 12:57:11 main:574] : INFO : Epoch 1976 | loss: 0.031104 | val_loss: 0.0311611 | Time: 1448.11 ms [2022-05-02 12:57:12 main:574] : INFO : Epoch 1977 | loss: 0.0311002 | val_loss: 0.0311651 | Time: 1449.08 ms [2022-05-02 12:57:13 main:574] : INFO : Epoch 1978 | loss: 0.0310975 | val_loss: 0.031165 | Time: 1451.74 ms [2022-05-02 12:57:15 main:574] : INFO : Epoch 1979 | loss: 0.0310943 | val_loss: 0.0311633 | Time: 1444.18 ms [2022-05-02 12:57:16 main:574] : INFO : Epoch 1980 | loss: 0.0310919 | val_loss: 0.0311556 | Time: 1433.33 ms [2022-05-02 12:57:18 main:574] : INFO : Epoch 1981 | loss: 0.0310949 | val_loss: 0.0311584 | Time: 1459.96 ms [2022-05-02 12:57:19 main:574] : INFO : Epoch 1982 | loss: 0.0310928 | val_loss: 0.0311627 | Time: 1437.54 ms [2022-05-02 12:57:21 main:574] : INFO : Epoch 1983 | loss: 0.0310882 | val_loss: 0.0311665 | Time: 1455.43 ms [2022-05-02 12:57:22 main:574] : INFO : Epoch 1984 | loss: 0.0310904 | val_loss: 0.0311662 | Time: 1448.84 ms [2022-05-02 12:57:24 main:574] : INFO : Epoch 1985 | loss: 0.0310946 | val_loss: 0.0311632 | Time: 1447.25 ms [2022-05-02 12:57:25 main:574] : INFO : Epoch 1986 | loss: 0.0310951 | val_loss: 0.0311667 | Time: 1434.45 ms [2022-05-02 12:57:27 main:574] : INFO : Epoch 1987 | loss: 0.031093 | val_loss: 0.0311698 | Time: 1447.86 ms [2022-05-02 12:57:28 main:574] : INFO : Epoch 1988 | loss: 0.0310883 | val_loss: 0.0311747 | Time: 1450.27 ms [2022-05-02 12:57:29 main:574] : INFO : Epoch 1989 | loss: 0.0310909 | val_loss: 0.0311737 | Time: 1439.33 ms [2022-05-02 12:57:31 main:574] : INFO : Epoch 1990 | loss: 0.0310959 | val_loss: 0.0311661 | Time: 1443.41 ms [2022-05-02 12:57:32 main:574] : INFO : Epoch 1991 | loss: 0.0310962 | val_loss: 0.0311753 | Time: 1455.48 ms [2022-05-02 12:57:34 main:574] : INFO : Epoch 1992 | loss: 0.031091 | val_loss: 0.0311781 | Time: 1435.76 ms [2022-05-02 12:57:35 main:574] : INFO : Epoch 1993 | loss: 0.0310894 | val_loss: 0.0311652 | Time: 1465.84 ms [2022-05-02 12:57:37 main:574] : INFO : Epoch 1994 | loss: 0.0310931 | val_loss: 0.03118 | Time: 1471.02 ms [2022-05-02 12:57:38 main:574] : INFO : Epoch 1995 | loss: 0.0310942 | val_loss: 0.0311733 | Time: 1464.15 ms [2022-05-02 12:57:40 main:574] : INFO : Epoch 1996 | loss: 0.0310945 | val_loss: 0.0311682 | Time: 1455.95 ms [2022-05-02 12:57:41 main:574] : INFO : Epoch 1997 | loss: 0.0311173 | val_loss: 0.0311484 | Time: 1463.84 ms [2022-05-02 12:57:43 main:574] : INFO : Epoch 1998 | loss: 0.0311212 | val_loss: 0.0311541 | Time: 1447.67 ms [2022-05-02 12:57:44 main:574] : INFO : Epoch 1999 | loss: 0.0311164 | val_loss: 0.0311562 | Time: 1466.19 ms [2022-05-02 12:57:45 main:574] : INFO : Epoch 2000 | loss: 0.031112 | val_loss: 0.0311542 | Time: 1430.99 ms [2022-05-02 12:57:47 main:574] : INFO : Epoch 2001 | loss: 0.0311092 | val_loss: 0.0311625 | Time: 1444 ms [2022-05-02 12:57:48 main:574] : INFO : Epoch 2002 | loss: 0.0311046 | val_loss: 0.0311599 | Time: 1443.95 ms [2022-05-02 12:57:50 main:574] : INFO : Epoch 2003 | loss: 0.0311022 | val_loss: 0.0311627 | Time: 1452.17 ms [2022-05-02 12:57:51 main:574] : INFO : Epoch 2004 | loss: 0.0311016 | val_loss: 0.0311634 | Time: 1450.32 ms [2022-05-02 12:57:53 main:574] : INFO : Epoch 2005 | loss: 0.0311004 | val_loss: 0.031164 | Time: 1452.27 ms [2022-05-02 12:57:54 main:574] : INFO : Epoch 2006 | loss: 0.0310965 | val_loss: 0.0311675 | Time: 1441.53 ms [2022-05-02 12:57:56 main:574] : INFO : Epoch 2007 | loss: 0.0310966 | val_loss: 0.0311692 | Time: 1459.49 ms [2022-05-02 12:57:57 main:574] : INFO : Epoch 2008 | loss: 0.0310937 | val_loss: 0.031174 | Time: 1439.47 ms [2022-05-02 12:57:59 main:574] : INFO : Epoch 2009 | loss: 0.0310921 | val_loss: 0.0311683 | Time: 1440.59 ms [2022-05-02 12:58:00 main:574] : INFO : Epoch 2010 | loss: 0.0310922 | val_loss: 0.0311669 | Time: 1439.3 ms [2022-05-02 12:58:02 main:574] : INFO : Epoch 2011 | loss: 0.031095 | val_loss: 0.0311618 | Time: 2019.15 ms [2022-05-02 12:58:06 main:574] : INFO : Epoch 2012 | loss: 0.0310983 | val_loss: 0.0311708 | Time: 3700.9 ms [2022-05-02 12:58:08 main:574] : INFO : Epoch 2013 | loss: 0.0310954 | val_loss: 0.0311597 | Time: 2719.87 ms [2022-05-02 12:58:11 main:574] : INFO : Epoch 2014 | loss: 0.0310936 | val_loss: 0.0311679 | Time: 2444.2 ms [2022-05-02 12:58:14 main:574] : INFO : Epoch 2015 | loss: 0.0310928 | val_loss: 0.0311676 | Time: 2747.42 ms [2022-05-02 12:58:36 main:574] : INFO : Epoch 2016 | loss: 0.0310918 | val_loss: 0.0311691 | Time: 22528.8 ms [2022-05-02 12:58:38 main:574] : INFO : Epoch 2017 | loss: 0.0310897 | val_loss: 0.0311627 | Time: 1760.88 ms [2022-05-02 12:58:40 main:574] : INFO : Epoch 2018 | loss: 0.0310914 | val_loss: 0.0311837 | Time: 1772.82 ms [2022-05-02 12:58:42 main:574] : INFO : Epoch 2019 | loss: 0.0310907 | val_loss: 0.0311638 | Time: 2091.27 ms [2022-05-02 12:58:44 main:574] : INFO : Epoch 2020 | loss: 0.0310895 | val_loss: 0.0311661 | Time: 2533.75 ms [2022-05-02 13:07:33 main:574] : INFO : Epoch 2021 | loss: 0.0310928 | val_loss: 0.0311699 | Time: 528701 ms [2022-05-02 13:07:35 main:574] : INFO : Epoch 2022 | loss: 0.031091 | val_loss: 0.0311766 | Time: 1454.19 ms [2022-05-02 13:07:36 main:574] : INFO : Epoch 2023 | loss: 0.0310918 | val_loss: 0.0311674 | Time: 1447.82 ms [2022-05-02 13:07:37 main:574] : INFO : Epoch 2024 | loss: 0.0310953 | val_loss: 0.0311714 | Time: 1455.02 ms [2022-05-02 13:07:39 main:574] : INFO : Epoch 2025 | loss: 0.0310935 | val_loss: 0.0311679 | Time: 1457.78 ms [2022-05-02 13:07:40 main:574] : INFO : Epoch 2026 | loss: 0.0311008 | val_loss: 0.0311718 | Time: 1447.82 ms [2022-05-02 13:07:42 main:574] : INFO : Epoch 2027 | loss: 0.0310998 | val_loss: 0.0311704 | Time: 1438.1 ms [2022-05-02 13:07:43 main:574] : INFO : Epoch 2028 | loss: 0.0310956 | val_loss: 0.0311699 | Time: 1448.74 ms [2022-05-02 13:07:45 main:574] : INFO : Epoch 2029 | loss: 0.0310959 | val_loss: 0.0311677 | Time: 1427.3 ms [2022-05-02 13:07:46 main:574] : INFO : Epoch 2030 | loss: 0.0310949 | val_loss: 0.0311782 | Time: 1449.34 ms [2022-05-02 13:07:48 main:574] : INFO : Epoch 2031 | loss: 0.0311034 | val_loss: 0.0311593 | Time: 1466.45 ms [2022-05-02 13:07:49 main:574] : INFO : Epoch 2032 | loss: 0.0311012 | val_loss: 0.0311567 | Time: 1433.64 ms [2022-05-02 13:07:51 main:574] : INFO : Epoch 2033 | loss: 0.0310959 | val_loss: 0.0311639 | Time: 1453.37 ms [2022-05-02 13:07:52 main:574] : INFO : Epoch 2034 | loss: 0.0310981 | val_loss: 0.031157 | Time: 1447.54 ms [2022-05-02 13:07:53 main:574] : INFO : Epoch 2035 | loss: 0.0310987 | val_loss: 0.0311734 | Time: 1440.25 ms [2022-05-02 13:07:55 main:574] : INFO : Epoch 2036 | loss: 0.0310937 | val_loss: 0.0311644 | Time: 1454.85 ms [2022-05-02 13:07:56 main:574] : INFO : Epoch 2037 | loss: 0.0310911 | val_loss: 0.031162 | Time: 1453.52 ms [2022-05-02 13:07:58 main:574] : INFO : Epoch 2038 | loss: 0.03109 | val_loss: 0.0311786 | Time: 1463.55 ms [2022-05-02 13:07:59 main:574] : INFO : Epoch 2039 | loss: 0.0310976 | val_loss: 0.0311635 | Time: 1446.03 ms [2022-05-02 13:08:01 main:574] : INFO : Epoch 2040 | loss: 0.0311041 | val_loss: 0.0311759 | Time: 1450.94 ms [2022-05-02 13:08:02 main:574] : INFO : Epoch 2041 | loss: 0.0310987 | val_loss: 0.0311605 | Time: 1431.46 ms [2022-05-02 13:08:04 main:574] : INFO : Epoch 2042 | loss: 0.0310942 | val_loss: 0.0311769 | Time: 1432.21 ms [2022-05-02 13:08:05 main:574] : INFO : Epoch 2043 | loss: 0.0310934 | val_loss: 0.031165 | Time: 1462.61 ms [2022-05-02 13:08:06 main:574] : INFO : Epoch 2044 | loss: 0.0310944 | val_loss: 0.0311742 | Time: 1454.06 ms [2022-05-02 13:08:08 main:574] : INFO : Epoch 2045 | loss: 0.0310935 | val_loss: 0.0311733 | Time: 1461.98 ms [2022-05-02 13:08:09 main:574] : INFO : Epoch 2046 | loss: 0.0310893 | val_loss: 0.0311776 | Time: 1453.43 ms [2022-05-02 13:08:11 main:574] : INFO : Epoch 2047 | loss: 0.0310877 | val_loss: 0.0311675 | Time: 1470.03 ms [2022-05-02 13:08:12 main:574] : INFO : Epoch 2048 | loss: 0.0310873 | val_loss: 0.0311829 | Time: 1434.27 ms [2022-05-02 13:08:12 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311829 [2022-05-02 13:08:12 main:603] : INFO : Saving end state to config to file [2022-05-02 13:08:12 main:608] : INFO : Success, exiting.. 13:08:12 (10840): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)