Task 9258395
| Name | ParityModified-1636179330-15374-3-0_0 |
| Workunit | 6689449 |
| Created | 29 Nov 2021, 11:19:02 UTC |
| Sent | 5 Dec 2021, 0:36:31 UTC |
| Report deadline | 13 Dec 2021, 0:36:31 UTC |
| Received | 6 Dec 2021, 23:26:09 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 157 |
| Run time | 9 hours 40 min 12 sec |
| CPU time | 6 hours 47 min 43 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 4,474.06 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.61 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> ms [2021-12-06 13:44:11 main:574] : INFO : Epoch 1987 | loss: 0.0312425 | val_loss: 0.031242 | Time: 20056.4 ms [2021-12-06 13:44:31 main:574] : INFO : Epoch 1988 | loss: 0.0312414 | val_loss: 0.0312404 | Time: 19621.6 ms [2021-12-06 13:44:50 main:574] : INFO : Epoch 1989 | loss: 0.0312403 | val_loss: 0.0312395 | Time: 19335.7 ms [2021-12-06 13:45:09 main:574] : INFO : Epoch 1990 | loss: 0.0312392 | val_loss: 0.0312385 | Time: 19129.6 ms [2021-12-06 13:45:29 main:574] : INFO : Epoch 1991 | loss: 0.0312379 | val_loss: 0.0312371 | Time: 19219.3 ms [2021-12-06 13:45:48 main:574] : INFO : Epoch 1992 | loss: 0.0312367 | val_loss: 0.031236 | Time: 19485.1 ms [2021-12-06 13:46:08 main:574] : INFO : Epoch 1993 | loss: 0.0312355 | val_loss: 0.0312347 | Time: 19946.7 ms [2021-12-06 13:46:27 main:574] : INFO : Epoch 1994 | loss: 0.0312343 | val_loss: 0.0312331 | Time: 18937.2 ms [2021-12-06 13:46:47 main:574] : INFO : Epoch 1995 | loss: 0.0312326 | val_loss: 0.0312318 | Time: 19374.4 ms [2021-12-06 13:47:07 main:574] : INFO : Epoch 1996 | loss: 0.0312307 | val_loss: 0.0312295 | Time: 19839.3 ms [2021-12-06 13:47:26 main:574] : INFO : Epoch 1997 | loss: 0.0312279 | val_loss: 0.0312259 | Time: 18996.4 ms [2021-12-06 13:47:45 main:574] : INFO : Epoch 1998 | loss: 0.0312228 | val_loss: 0.0312182 | Time: 19029.5 ms [2021-12-06 13:48:05 main:574] : INFO : Epoch 1999 | loss: 0.0312142 | val_loss: 0.0312101 | Time: 19765.8 ms [2021-12-06 13:48:24 main:574] : INFO : Epoch 2000 | loss: 0.0312067 | val_loss: 0.0312042 | Time: 19239.7 ms [2021-12-06 13:48:43 main:574] : INFO : Epoch 2001 | loss: 0.0312011 | val_loss: 0.0311995 | Time: 19082 ms [2021-12-06 13:49:03 main:574] : INFO : Epoch 2002 | loss: 0.0311957 | val_loss: 0.0311937 | Time: 19616.5 ms [2021-12-06 13:49:22 main:574] : INFO : Epoch 2003 | loss: 0.0311886 | val_loss: 0.0311879 | Time: 19234.8 ms [2021-12-06 13:49:41 main:574] : INFO : Epoch 2004 | loss: 0.0311764 | val_loss: 0.0311773 | Time: 19192.3 ms [2021-12-06 13:50:01 main:574] : INFO : Epoch 2005 | loss: 0.0311639 | val_loss: 0.0311669 | Time: 19603 ms [2021-12-06 13:50:21 main:574] : INFO : Epoch 2006 | loss: 0.0311555 | val_loss: 0.0311599 | Time: 19852.1 ms [2021-12-06 13:50:41 main:574] : INFO : Epoch 2007 | loss: 0.0311515 | val_loss: 0.0311562 | Time: 20123.2 ms [2021-12-06 13:51:01 main:574] : INFO : Epoch 2008 | loss: 0.0311482 | val_loss: 0.031154 | Time: 19360.4 ms [2021-12-06 13:51:20 main:574] : INFO : Epoch 2009 | loss: 0.0311466 | val_loss: 0.0311519 | Time: 19460.2 ms [2021-12-06 13:51:39 main:574] : INFO : Epoch 2010 | loss: 0.0311447 | val_loss: 0.0311514 | Time: 18799.5 ms [2021-12-06 13:51:58 main:574] : INFO : Epoch 2011 | loss: 0.031143 | val_loss: 0.0311468 | Time: 19376.8 ms [2021-12-06 13:52:18 main:574] : INFO : Epoch 2012 | loss: 0.0311393 | val_loss: 0.0311461 | Time: 19373.9 ms [2021-12-06 13:52:38 main:574] : INFO : Epoch 2013 | loss: 0.0311362 | val_loss: 0.0311438 | Time: 20341.9 ms [2021-12-06 13:52:57 main:574] : INFO : Epoch 2014 | loss: 0.0311333 | val_loss: 0.0311438 | Time: 18880.2 ms [2021-12-06 13:53:16 main:574] : INFO : Epoch 2015 | loss: 0.0311306 | val_loss: 0.0311405 | Time: 19246.6 ms [2021-12-06 13:53:36 main:574] : INFO : Epoch 2016 | loss: 0.0311287 | val_loss: 0.0311384 | Time: 19170.8 ms [2021-12-06 13:53:54 main:574] : INFO : Epoch 2017 | loss: 0.0311251 | val_loss: 0.0311369 | Time: 18355.7 ms [2021-12-06 13:54:14 main:574] : INFO : Epoch 2018 | loss: 0.031119 | val_loss: 0.0311313 | Time: 19716.2 ms [2021-12-06 13:54:33 main:574] : INFO : Epoch 2019 | loss: 0.0311137 | val_loss: 0.0311305 | Time: 19409.4 ms [2021-12-06 13:54:53 main:574] : INFO : Epoch 2020 | loss: 0.03111 | val_loss: 0.031128 | Time: 20031.6 ms [2021-12-06 13:55:12 main:574] : INFO : Epoch 2021 | loss: 0.0311066 | val_loss: 0.0311221 | Time: 19122.8 ms [2021-12-06 13:55:32 main:574] : INFO : Epoch 2022 | loss: 0.0311021 | val_loss: 0.0311214 | Time: 19799.5 ms [2021-12-06 13:55:52 main:574] : INFO : Epoch 2023 | loss: 0.0310998 | val_loss: 0.03112 | Time: 19399.5 ms [2021-12-06 13:56:11 main:574] : INFO : Epoch 2024 | loss: 0.0310985 | val_loss: 0.0311195 | Time: 19340.2 ms [2021-12-06 13:56:31 main:574] : INFO : Epoch 2025 | loss: 0.031095 | val_loss: 0.0311226 | Time: 19623.1 ms [2021-12-06 13:56:50 main:574] : INFO : Epoch 2026 | loss: 0.0310962 | val_loss: 0.031119 | Time: 18772.8 ms [2021-12-06 13:57:09 main:574] : INFO : Epoch 2027 | loss: 0.0310912 | val_loss: 0.0311229 | Time: 18890.2 ms [2021-12-06 13:57:28 main:574] : INFO : Epoch 2028 | loss: 0.031089 | val_loss: 0.0311207 | Time: 18808.6 ms [2021-12-06 13:57:47 main:574] : INFO : Epoch 2029 | loss: 0.0310876 | val_loss: 0.0311167 | Time: 19082.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 14:07:12 main:435] : INFO : Set logging level to 1 [2021-12-06 14:07:12 main:441] : INFO : Running in BOINC Client mode [2021-12-06 14:07:13 main:444] : INFO : Resolving all filenames [2021-12-06 14:07:13 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 14:07:13 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 14:07:13 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 14:07:13 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 14:07:13 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 14:07:13 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 14:07:13 main:474] : INFO : Configuration: [2021-12-06 14:07:13 main:475] : INFO : Model type: GRU [2021-12-06 14:07:13 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 14:07:13 main:477] : INFO : Max Epochs: 2048 [2021-12-06 14:07:13 main:478] : INFO : Batch Size: 128 [2021-12-06 14:07:13 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 14:07:13 main:480] : INFO : Patience: 10 [2021-12-06 14:07:13 main:481] : INFO : Hidden Width: 12 [2021-12-06 14:07:13 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 14:07:13 main:483] : INFO : # Backend Layers: 4 [2021-12-06 14:07:13 main:484] : INFO : # Threads: 1 [2021-12-06 14:07:13 main:486] : INFO : Preparing Dataset [2021-12-06 14:07:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 14:07:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 14:07:21 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 14:07:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 14:07:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 14:07:22 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 14:07:22 main:494] : INFO : Creating Model [2021-12-06 14:07:22 main:507] : INFO : Preparing config file [2021-12-06 14:07:22 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 14:07:22 main:512] : INFO : Loading config [2021-12-06 14:07:22 main:514] : INFO : Loading state [2021-12-06 14:07:24 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 14:07:25 main:562] : INFO : Starting Training [2021-12-06 14:07:45 main:574] : INFO : Epoch 1931 | loss: 0.031114 | val_loss: 0.0311341 | Time: 20462.7 ms [2021-12-06 14:08:04 main:574] : INFO : Epoch 1932 | loss: 0.0310619 | val_loss: 0.0311277 | Time: 19071.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 14:28:18 main:435] : INFO : Set logging level to 1 [2021-12-06 14:28:18 main:441] : INFO : Running in BOINC Client mode [2021-12-06 14:28:18 main:444] : INFO : Resolving all filenames [2021-12-06 14:28:18 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 14:28:18 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 14:28:18 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 14:28:18 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 14:28:18 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 14:28:18 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 14:28:18 main:474] : INFO : Configuration: [2021-12-06 14:28:18 main:475] : INFO : Model type: GRU [2021-12-06 14:28:18 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 14:28:18 main:477] : INFO : Max Epochs: 2048 [2021-12-06 14:28:19 main:478] : INFO : Batch Size: 128 [2021-12-06 14:28:19 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 14:28:19 main:480] : INFO : Patience: 10 [2021-12-06 14:28:19 main:481] : INFO : Hidden Width: 12 [2021-12-06 14:28:19 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 14:28:19 main:483] : INFO : # Backend Layers: 4 [2021-12-06 14:28:19 main:484] : INFO : # Threads: 1 [2021-12-06 14:28:19 main:486] : INFO : Preparing Dataset [2021-12-06 14:28:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 14:28:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 14:28:25 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 14:28:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 14:28:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 14:28:25 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 14:28:25 main:494] : INFO : Creating Model [2021-12-06 14:28:25 main:507] : INFO : Preparing config file [2021-12-06 14:28:25 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 14:28:25 main:512] : INFO : Loading config [2021-12-06 14:28:25 main:514] : INFO : Loading state [2021-12-06 14:28:27 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 14:28:28 main:562] : INFO : Starting Training [2021-12-06 14:28:44 main:574] : INFO : Epoch 1931 | loss: 0.0311172 | val_loss: 0.0311341 | Time: 16882.3 ms [2021-12-06 14:29:02 main:574] : INFO : Epoch 1932 | loss: 0.0310665 | val_loss: 0.0311172 | Time: 17156.6 ms [2021-12-06 14:29:17 main:574] : INFO : Epoch 1933 | loss: 0.0310586 | val_loss: 0.0311214 | Time: 15621.4 ms [2021-12-06 14:29:33 main:574] : INFO : Epoch 1934 | loss: 0.0310581 | val_loss: 0.0311246 | Time: 15464.7 ms [2021-12-06 14:29:48 main:574] : INFO : Epoch 1935 | loss: 0.0310534 | val_loss: 0.0311295 | Time: 15259.9 ms [2021-12-06 14:30:04 main:574] : INFO : Epoch 1936 | loss: 0.0310524 | val_loss: 0.0311181 | Time: 15644.3 ms [2021-12-06 14:30:20 main:574] : INFO : Epoch 1937 | loss: 0.0310552 | val_loss: 0.0311195 | Time: 16175.4 ms [2021-12-06 14:30:36 main:574] : INFO : Epoch 1938 | loss: 0.0310508 | val_loss: 0.0311271 | Time: 15557 ms [2021-12-06 14:30:51 main:574] : INFO : Epoch 1939 | loss: 0.0310492 | val_loss: 0.0311253 | Time: 15503.2 ms [2021-12-06 14:31:06 main:574] : INFO : Epoch 1940 | loss: 0.031052 | val_loss: 0.0311182 | Time: 14784.8 ms [2021-12-06 14:31:22 main:574] : INFO : Epoch 1941 | loss: 0.0310527 | val_loss: 0.0311283 | Time: 15654.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 14:33:25 main:435] : INFO : Set logging level to 1 [2021-12-06 14:33:25 main:441] : INFO : Running in BOINC Client mode [2021-12-06 14:33:25 main:444] : INFO : Resolving all filenames [2021-12-06 14:33:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 14:33:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 14:33:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 14:33:25 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 14:33:25 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 14:33:25 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 14:33:25 main:474] : INFO : Configuration: [2021-12-06 14:33:25 main:475] : INFO : Model type: GRU [2021-12-06 14:33:25 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 14:33:25 main:477] : INFO : Max Epochs: 2048 [2021-12-06 14:33:25 main:478] : INFO : Batch Size: 128 [2021-12-06 14:33:25 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 14:33:25 main:480] : INFO : Patience: 10 [2021-12-06 14:33:25 main:481] : INFO : Hidden Width: 12 [2021-12-06 14:33:25 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 14:33:25 main:483] : INFO : # Backend Layers: 4 [2021-12-06 14:33:26 main:484] : INFO : # Threads: 1 [2021-12-06 14:33:26 main:486] : INFO : Preparing Dataset [2021-12-06 14:33:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 14:33:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 14:33:30 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 14:33:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 14:33:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 14:33:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 14:33:31 main:494] : INFO : Creating Model [2021-12-06 14:33:31 main:507] : INFO : Preparing config file [2021-12-06 14:33:31 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 14:33:31 main:512] : INFO : Loading config [2021-12-06 14:33:31 main:514] : INFO : Loading state [2021-12-06 14:33:34 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 14:33:34 main:562] : INFO : Starting Training [2021-12-06 14:33:50 main:574] : INFO : Epoch 1931 | loss: 0.031105 | val_loss: 0.0311283 | Time: 16109.9 ms [2021-12-06 14:34:05 main:574] : INFO : Epoch 1932 | loss: 0.0310618 | val_loss: 0.0311212 | Time: 15324.8 ms [2021-12-06 14:34:21 main:574] : INFO : Epoch 1933 | loss: 0.0310569 | val_loss: 0.0311172 | Time: 15783 ms [2021-12-06 14:34:36 main:574] : INFO : Epoch 1934 | loss: 0.031053 | val_loss: 0.0311227 | Time: 15034.6 ms [2021-12-06 14:34:52 main:574] : INFO : Epoch 1935 | loss: 0.0310499 | val_loss: 0.0311211 | Time: 15420.6 ms [2021-12-06 14:35:07 main:574] : INFO : Epoch 1936 | loss: 0.0310456 | val_loss: 0.0311238 | Time: 14885.9 ms [2021-12-06 14:35:22 main:574] : INFO : Epoch 1937 | loss: 0.0310486 | val_loss: 0.031116 | Time: 15502 ms [2021-12-06 14:35:38 main:574] : INFO : Epoch 1938 | loss: 0.0310529 | val_loss: 0.0311271 | Time: 15425.3 ms [2021-12-06 14:35:53 main:574] : INFO : Epoch 1939 | loss: 0.0310523 | val_loss: 0.0311197 | Time: 15305.7 ms [2021-12-06 14:36:09 main:574] : INFO : Epoch 1940 | loss: 0.0310576 | val_loss: 0.0311226 | Time: 15367.6 ms [2021-12-06 14:36:24 main:574] : INFO : Epoch 1941 | loss: 0.0310574 | val_loss: 0.0311292 | Time: 15607.7 ms [2021-12-06 14:36:40 main:574] : INFO : Epoch 1942 | loss: 0.0310522 | val_loss: 0.0311273 | Time: 15321 ms [2021-12-06 14:36:55 main:574] : INFO : Epoch 1943 | loss: 0.0310518 | val_loss: 0.0311327 | Time: 15488.2 ms [2021-12-06 14:37:11 main:574] : INFO : Epoch 1944 | loss: 0.0310482 | val_loss: 0.0311287 | Time: 15266.1 ms [2021-12-06 14:37:27 main:574] : INFO : Epoch 1945 | loss: 0.0310464 | val_loss: 0.0311273 | Time: 16546.9 ms [2021-12-06 14:37:43 main:574] : INFO : Epoch 1946 | loss: 0.0310468 | val_loss: 0.031124 | Time: 15866.7 ms [2021-12-06 14:37:58 main:574] : INFO : Epoch 1947 | loss: 0.0310518 | val_loss: 0.0311213 | Time: 15102 ms [2021-12-06 14:38:14 main:574] : INFO : Epoch 1948 | loss: 0.0310514 | val_loss: 0.031135 | Time: 15616.4 ms [2021-12-06 14:38:30 main:574] : INFO : Epoch 1949 | loss: 0.0310523 | val_loss: 0.0311215 | Time: 15639.7 ms [2021-12-06 14:38:45 main:574] : INFO : Epoch 1950 | loss: 0.0310462 | val_loss: 0.0311292 | Time: 15561.9 ms [2021-12-06 14:39:01 main:574] : INFO : Epoch 1951 | loss: 0.0310517 | val_loss: 0.0311381 | Time: 15447.1 ms [2021-12-06 14:39:16 main:574] : INFO : Epoch 1952 | loss: 0.0310533 | val_loss: 0.0311251 | Time: 15447.1 ms [2021-12-06 14:39:32 main:574] : INFO : Epoch 1953 | loss: 0.0310573 | val_loss: 0.0311246 | Time: 15335.9 ms [2021-12-06 14:39:47 main:574] : INFO : Epoch 1954 | loss: 0.0310521 | val_loss: 0.0311192 | Time: 15245.5 ms [2021-12-06 14:40:03 main:574] : INFO : Epoch 1955 | loss: 0.0310511 | val_loss: 0.0311261 | Time: 15951.4 ms [2021-12-06 14:40:18 main:574] : INFO : Epoch 1956 | loss: 0.0310473 | val_loss: 0.0311234 | Time: 15144.5 ms [2021-12-06 14:40:34 main:574] : INFO : Epoch 1957 | loss: 0.0310487 | val_loss: 0.0311338 | Time: 15423.5 ms [2021-12-06 14:40:48 main:574] : INFO : Epoch 1958 | loss: 0.0310495 | val_loss: 0.0311244 | Time: 14899.3 ms [2021-12-06 14:41:04 main:574] : INFO : Epoch 1959 | loss: 0.0310496 | val_loss: 0.0311346 | Time: 15654.4 ms [2021-12-06 14:41:19 main:574] : INFO : Epoch 1960 | loss: 0.0310459 | val_loss: 0.0311169 | Time: 14958.5 ms [2021-12-06 14:41:35 main:574] : INFO : Epoch 1961 | loss: 0.0310415 | val_loss: 0.0311255 | Time: 15392.9 ms [2021-12-06 14:41:49 main:574] : INFO : Epoch 1962 | loss: 0.0310419 | val_loss: 0.0311263 | Time: 14548.4 ms [2021-12-06 14:42:06 main:574] : INFO : Epoch 1963 | loss: 0.031046 | val_loss: 0.0311288 | Time: 16985.8 ms [2021-12-06 14:42:22 main:574] : INFO : Epoch 1964 | loss: 0.0310457 | val_loss: 0.0311244 | Time: 15743.1 ms [2021-12-06 14:42:38 main:574] : INFO : Epoch 1965 | loss: 0.0310475 | val_loss: 0.0311243 | Time: 15576.8 ms [2021-12-06 14:42:54 main:574] : INFO : Epoch 1966 | loss: 0.0310466 | val_loss: 0.0311245 | Time: 16395.1 ms [2021-12-06 14:43:10 main:574] : INFO : Epoch 1967 | loss: 0.0310505 | val_loss: 0.0311209 | Time: 16214.3 ms [2021-12-06 14:43:30 main:574] : INFO : Epoch 1968 | loss: 0.0310494 | val_loss: 0.0311173 | Time: 19747.9 ms [2021-12-06 14:43:47 main:574] : INFO : Epoch 1969 | loss: 0.0310554 | val_loss: 0.0311244 | Time: 16798.3 ms [2021-12-06 14:44:06 main:574] : INFO : Epoch 1970 | loss: 0.0310551 | val_loss: 0.0311243 | Time: 18557.3 ms [2021-12-06 14:44:22 main:574] : INFO : Epoch 1971 | loss: 0.031052 | val_loss: 0.0311163 | Time: 15973.9 ms [2021-12-06 14:44:38 main:574] : INFO : Epoch 1972 | loss: 0.0310519 | val_loss: 0.0311232 | Time: 16567.7 ms [2021-12-06 14:44:55 main:574] : INFO : Epoch 1973 | loss: 0.0310516 | val_loss: 0.0311194 | Time: 16618.7 ms [2021-12-06 14:45:11 main:574] : INFO : Epoch 1974 | loss: 0.0310491 | val_loss: 0.0311227 | Time: 15821.7 ms [2021-12-06 14:45:27 main:574] : INFO : Epoch 1975 | loss: 0.0310496 | val_loss: 0.0311249 | Time: 16471 ms [2021-12-06 14:45:43 main:574] : INFO : Epoch 1976 | loss: 0.0310603 | val_loss: 0.0311265 | Time: 15910.9 ms [2021-12-06 14:46:00 main:574] : INFO : Epoch 1977 | loss: 0.0310564 | val_loss: 0.0311164 | Time: 16202.2 ms [2021-12-06 14:46:16 main:574] : INFO : Epoch 1978 | loss: 0.0310557 | val_loss: 0.0311222 | Time: 16725.3 ms [2021-12-06 14:46:35 main:574] : INFO : Epoch 1979 | loss: 0.0310507 | val_loss: 0.0311206 | Time: 18367.3 ms [2021-12-06 14:46:53 main:574] : INFO : Epoch 1980 | loss: 0.031047 | val_loss: 0.0311229 | Time: 17573.7 ms [2021-12-06 14:47:08 main:574] : INFO : Epoch 1981 | loss: 0.0310482 | val_loss: 0.031123 | Time: 15344 ms [2021-12-06 14:47:24 main:574] : INFO : Epoch 1982 | loss: 0.0310464 | val_loss: 0.0311223 | Time: 16258.1 ms [2021-12-06 14:47:40 main:574] : INFO : Epoch 1983 | loss: 0.0310468 | val_loss: 0.0311318 | Time: 15992.9 ms [2021-12-06 14:47:57 main:574] : INFO : Epoch 1984 | loss: 0.0310456 | val_loss: 0.0311278 | Time: 16274.6 ms [2021-12-06 14:48:13 main:574] : INFO : Epoch 1985 | loss: 0.0310468 | val_loss: 0.0311227 | Time: 16107.3 ms [2021-12-06 14:48:28 main:574] : INFO : Epoch 1986 | loss: 0.0310464 | val_loss: 0.0311257 | Time: 15341.5 ms [2021-12-06 14:48:44 main:574] : INFO : Epoch 1987 | loss: 0.0310499 | val_loss: 0.0311346 | Time: 16128.7 ms [2021-12-06 14:49:01 main:574] : INFO : Epoch 1988 | loss: 0.0310484 | val_loss: 0.0311427 | Time: 16251.8 ms [2021-12-06 14:49:17 main:574] : INFO : Epoch 1989 | loss: 0.0310472 | val_loss: 0.0311325 | Time: 16344.1 ms [2021-12-06 14:49:33 main:574] : INFO : Epoch 1990 | loss: 0.0310427 | val_loss: 0.0311338 | Time: 16122.3 ms [2021-12-06 14:49:50 main:574] : INFO : Epoch 1991 | loss: 0.0310453 | val_loss: 0.0311401 | Time: 16451.4 ms [2021-12-06 14:50:06 main:574] : INFO : Epoch 1992 | loss: 0.0310441 | val_loss: 0.0311352 | Time: 16264.4 ms [2021-12-06 14:50:22 main:574] : INFO : Epoch 1993 | loss: 0.031045 | val_loss: 0.0311404 | Time: 16105.1 ms [2021-12-06 14:50:38 main:574] : INFO : Epoch 1994 | loss: 0.0310579 | val_loss: 0.0311342 | Time: 15933.7 ms [2021-12-06 14:50:55 main:574] : INFO : Epoch 1995 | loss: 0.0310542 | val_loss: 0.0311299 | Time: 16609.2 ms [2021-12-06 14:51:11 main:574] : INFO : Epoch 1996 | loss: 0.0310532 | val_loss: 0.0311186 | Time: 15920.8 ms [2021-12-06 14:51:27 main:574] : INFO : Epoch 1997 | loss: 0.0310504 | val_loss: 0.0311289 | Time: 16363.9 ms [2021-12-06 14:51:43 main:574] : INFO : Epoch 1998 | loss: 0.0310457 | val_loss: 0.0311206 | Time: 15978.4 ms [2021-12-06 14:51:59 main:574] : INFO : Epoch 1999 | loss: 0.031042 | val_loss: 0.031123 | Time: 15907.2 ms [2021-12-06 14:52:16 main:574] : INFO : Epoch 2000 | loss: 0.0310435 | val_loss: 0.031121 | Time: 16613.9 ms [2021-12-06 14:52:32 main:574] : INFO : Epoch 2001 | loss: 0.031044 | val_loss: 0.0311237 | Time: 16239.2 ms [2021-12-06 14:52:48 main:574] : INFO : Epoch 2002 | loss: 0.0310481 | val_loss: 0.0311252 | Time: 16174.3 ms [2021-12-06 14:53:05 main:574] : INFO : Epoch 2003 | loss: 0.0310427 | val_loss: 0.0311261 | Time: 16271 ms [2021-12-06 14:53:21 main:574] : INFO : Epoch 2004 | loss: 0.0310408 | val_loss: 0.0311306 | Time: 16061.7 ms [2021-12-06 14:53:37 main:574] : INFO : Epoch 2005 | loss: 0.031047 | val_loss: 0.0311324 | Time: 16148.1 ms [2021-12-06 14:53:53 main:574] : INFO : Epoch 2006 | loss: 0.0310446 | val_loss: 0.0311274 | Time: 15943.4 ms [2021-12-06 14:54:10 main:574] : INFO : Epoch 2007 | loss: 0.0310408 | val_loss: 0.0311348 | Time: 16170.5 ms [2021-12-06 14:54:26 main:574] : INFO : Epoch 2008 | loss: 0.0310473 | val_loss: 0.0311169 | Time: 16349.3 ms [2021-12-06 14:54:42 main:574] : INFO : Epoch 2009 | loss: 0.0310429 | val_loss: 0.0311202 | Time: 15700.6 ms [2021-12-06 14:54:58 main:574] : INFO : Epoch 2010 | loss: 0.0310418 | val_loss: 0.0311268 | Time: 16290 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 15:00:59 main:435] : INFO : Set logging level to 1 [2021-12-06 15:00:59 main:441] : INFO : Running in BOINC Client mode [2021-12-06 15:00:59 main:444] : INFO : Resolving all filenames [2021-12-06 15:00:59 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 15:01:00 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 15:01:00 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 15:01:00 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 15:01:00 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 15:01:00 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 15:01:00 main:474] : INFO : Configuration: [2021-12-06 15:01:00 main:475] : INFO : Model type: GRU [2021-12-06 15:01:00 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 15:01:00 main:477] : INFO : Max Epochs: 2048 [2021-12-06 15:01:00 main:478] : INFO : Batch Size: 128 [2021-12-06 15:01:00 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 15:01:00 main:480] : INFO : Patience: 10 [2021-12-06 15:01:00 main:481] : INFO : Hidden Width: 12 [2021-12-06 15:01:00 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 15:01:00 main:483] : INFO : # Backend Layers: 4 [2021-12-06 15:01:00 main:484] : INFO : # Threads: 1 [2021-12-06 15:01:00 main:486] : INFO : Preparing Dataset [2021-12-06 15:01:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 15:01:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 15:01:08 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 15:01:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 15:01:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 15:01:09 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 15:01:09 main:494] : INFO : Creating Model [2021-12-06 15:01:09 main:507] : INFO : Preparing config file [2021-12-06 15:01:09 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 15:01:09 main:512] : INFO : Loading config [2021-12-06 15:01:09 main:514] : INFO : Loading state [2021-12-06 15:01:13 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 15:01:13 main:562] : INFO : Starting Training [2021-12-06 15:01:36 main:574] : INFO : Epoch 1931 | loss: 0.0311071 | val_loss: 0.0311364 | Time: 22223.5 ms [2021-12-06 15:01:57 main:574] : INFO : Epoch 1932 | loss: 0.0310644 | val_loss: 0.0311188 | Time: 20907.3 ms [2021-12-06 15:02:18 main:574] : INFO : Epoch 1933 | loss: 0.0310591 | val_loss: 0.0311283 | Time: 21110 ms [2021-12-06 15:02:39 main:574] : INFO : Epoch 1934 | loss: 0.0310571 | val_loss: 0.0311259 | Time: 20656.7 ms [2021-12-06 15:03:00 main:574] : INFO : Epoch 1935 | loss: 0.0310569 | val_loss: 0.031134 | Time: 21377.8 ms [2021-12-06 15:03:22 main:574] : INFO : Epoch 1936 | loss: 0.0310566 | val_loss: 0.0311264 | Time: 21791.7 ms [2021-12-06 15:03:43 main:574] : INFO : Epoch 1937 | loss: 0.0310491 | val_loss: 0.0311254 | Time: 20945.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 15:06:07 main:435] : INFO : Set logging level to 1 [2021-12-06 15:06:07 main:441] : INFO : Running in BOINC Client mode [2021-12-06 15:06:07 main:444] : INFO : Resolving all filenames [2021-12-06 15:06:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 15:06:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 15:06:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 15:06:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 15:06:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 15:06:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 15:06:07 main:474] : INFO : Configuration: [2021-12-06 15:06:07 main:475] : INFO : Model type: GRU [2021-12-06 15:06:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 15:06:07 main:477] : INFO : Max Epochs: 2048 [2021-12-06 15:06:07 main:478] : INFO : Batch Size: 128 [2021-12-06 15:06:07 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 15:06:07 main:480] : INFO : Patience: 10 [2021-12-06 15:06:08 main:481] : INFO : Hidden Width: 12 [2021-12-06 15:06:08 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 15:06:08 main:483] : INFO : # Backend Layers: 4 [2021-12-06 15:06:08 main:484] : INFO : # Threads: 1 [2021-12-06 15:06:08 main:486] : INFO : Preparing Dataset [2021-12-06 15:06:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 15:06:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 15:06:14 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 15:06:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 15:06:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 15:06:14 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 15:06:15 main:494] : INFO : Creating Model [2021-12-06 15:06:15 main:507] : INFO : Preparing config file [2021-12-06 15:06:15 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 15:06:15 main:512] : INFO : Loading config [2021-12-06 15:06:15 main:514] : INFO : Loading state [2021-12-06 15:06:18 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 15:06:18 main:562] : INFO : Starting Training [2021-12-06 15:06:39 main:574] : INFO : Epoch 1931 | loss: 0.0310957 | val_loss: 0.0311224 | Time: 20488.9 ms [2021-12-06 15:07:00 main:574] : INFO : Epoch 1932 | loss: 0.031065 | val_loss: 0.0311238 | Time: 21294.1 ms [2021-12-06 15:07:22 main:574] : INFO : Epoch 1933 | loss: 0.0310595 | val_loss: 0.0311219 | Time: 21552.5 ms [2021-12-06 15:07:45 main:574] : INFO : Epoch 1934 | loss: 0.0310613 | val_loss: 0.0311152 | Time: 23398.6 ms [2021-12-06 15:08:07 main:574] : INFO : Epoch 1935 | loss: 0.0310551 | val_loss: 0.0311135 | Time: 21885.4 ms [2021-12-06 15:08:28 main:574] : INFO : Epoch 1936 | loss: 0.0310579 | val_loss: 0.0311169 | Time: 20938.6 ms [2021-12-06 15:08:49 main:574] : INFO : Epoch 1937 | loss: 0.0310564 | val_loss: 0.0311312 | Time: 20604.3 ms [2021-12-06 15:09:10 main:574] : INFO : Epoch 1938 | loss: 0.0310607 | val_loss: 0.0311205 | Time: 21347.6 ms [2021-12-06 15:09:32 main:574] : INFO : Epoch 1939 | loss: 0.0310556 | val_loss: 0.031128 | Time: 21271.9 ms [2021-12-06 15:09:52 main:574] : INFO : Epoch 1940 | loss: 0.0310528 | val_loss: 0.0311179 | Time: 20636.4 ms [2021-12-06 15:10:14 main:574] : INFO : Epoch 1941 | loss: 0.0310546 | val_loss: 0.0311338 | Time: 21693.1 ms [2021-12-06 15:10:36 main:574] : INFO : Epoch 1942 | loss: 0.0310509 | val_loss: 0.0311189 | Time: 21477.4 ms [2021-12-06 15:10:58 main:574] : INFO : Epoch 1943 | loss: 0.0310481 | val_loss: 0.0311224 | Time: 22190.7 ms [2021-12-06 15:11:19 main:574] : INFO : Epoch 1944 | loss: 0.0310471 | val_loss: 0.0311155 | Time: 20614.8 ms [2021-12-06 15:11:40 main:574] : INFO : Epoch 1945 | loss: 0.0310474 | val_loss: 0.0311294 | Time: 20904.1 ms [2021-12-06 15:12:00 main:574] : INFO : Epoch 1946 | loss: 0.03105 | val_loss: 0.0311183 | Time: 20230.2 ms [2021-12-06 15:12:20 main:574] : INFO : Epoch 1947 | loss: 0.0310545 | val_loss: 0.0311275 | Time: 20526.9 ms [2021-12-06 15:12:40 main:574] : INFO : Epoch 1948 | loss: 0.0310572 | val_loss: 0.0311207 | Time: 19920.7 ms [2021-12-06 15:13:01 main:574] : INFO : Epoch 1949 | loss: 0.0310581 | val_loss: 0.0311233 | Time: 20909.5 ms [2021-12-06 15:13:22 main:574] : INFO : Epoch 1950 | loss: 0.0310583 | val_loss: 0.0311347 | Time: 20709.4 ms [2021-12-06 15:13:44 main:574] : INFO : Epoch 1951 | loss: 0.0310578 | val_loss: 0.0311213 | Time: 21809.7 ms [2021-12-06 15:14:06 main:574] : INFO : Epoch 1952 | loss: 0.0310519 | val_loss: 0.03112 | Time: 21499.6 ms [2021-12-06 15:14:27 main:574] : INFO : Epoch 1953 | loss: 0.0310485 | val_loss: 0.0311277 | Time: 20928.2 ms [2021-12-06 15:14:48 main:574] : INFO : Epoch 1954 | loss: 0.0310476 | val_loss: 0.0311242 | Time: 21146 ms [2021-12-06 15:15:09 main:574] : INFO : Epoch 1955 | loss: 0.0310449 | val_loss: 0.0311229 | Time: 21196 ms [2021-12-06 15:15:29 main:574] : INFO : Epoch 1956 | loss: 0.0310446 | val_loss: 0.0311173 | Time: 20033.1 ms [2021-12-06 15:15:50 main:574] : INFO : Epoch 1957 | loss: 0.0310444 | val_loss: 0.0311159 | Time: 20261.6 ms [2021-12-06 15:16:11 main:574] : INFO : Epoch 1958 | loss: 0.0310437 | val_loss: 0.0311216 | Time: 20947.9 ms [2021-12-06 15:16:30 main:574] : INFO : Epoch 1959 | loss: 0.0310434 | val_loss: 0.0311182 | Time: 19507.3 ms [2021-12-06 15:16:52 main:574] : INFO : Epoch 1960 | loss: 0.0310539 | val_loss: 0.0311248 | Time: 22134.2 ms [2021-12-06 15:17:14 main:574] : INFO : Epoch 1961 | loss: 0.0310546 | val_loss: 0.03112 | Time: 21685.9 ms [2021-12-06 15:17:35 main:574] : INFO : Epoch 1962 | loss: 0.0310566 | val_loss: 0.0311231 | Time: 20519.2 ms [2021-12-06 15:17:56 main:574] : INFO : Epoch 1963 | loss: 0.0310518 | val_loss: 0.031122 | Time: 20993.6 ms [2021-12-06 15:18:18 main:574] : INFO : Epoch 1964 | loss: 0.031047 | val_loss: 0.031127 | Time: 22011.4 ms [2021-12-06 15:18:39 main:574] : INFO : Epoch 1965 | loss: 0.0310449 | val_loss: 0.0311234 | Time: 21311.8 ms [2021-12-06 15:19:00 main:574] : INFO : Epoch 1966 | loss: 0.0310437 | val_loss: 0.031122 | Time: 20381.9 ms [2021-12-06 15:19:21 main:574] : INFO : Epoch 1967 | loss: 0.0310435 | val_loss: 0.0311151 | Time: 20875.6 ms [2021-12-06 15:19:41 main:574] : INFO : Epoch 1968 | loss: 0.0310554 | val_loss: 0.0311195 | Time: 20813.4 ms [2021-12-06 15:20:03 main:574] : INFO : Epoch 1969 | loss: 0.0310541 | val_loss: 0.0311133 | Time: 21154.7 ms [2021-12-06 15:20:24 main:574] : INFO : Epoch 1970 | loss: 0.0310532 | val_loss: 0.0311157 | Time: 21192.1 ms [2021-12-06 15:20:46 main:574] : INFO : Epoch 1971 | loss: 0.031053 | val_loss: 0.0311212 | Time: 21535.5 ms [2021-12-06 15:21:09 main:574] : INFO : Epoch 1972 | loss: 0.0310588 | val_loss: 0.0311171 | Time: 22996.4 ms [2021-12-06 15:21:30 main:574] : INFO : Epoch 1973 | loss: 0.0310581 | val_loss: 0.0311171 | Time: 20956.8 ms [2021-12-06 15:21:50 main:574] : INFO : Epoch 1974 | loss: 0.0310669 | val_loss: 0.0311233 | Time: 20378.3 ms [2021-12-06 15:22:10 main:574] : INFO : Epoch 1975 | loss: 0.0310626 | val_loss: 0.0311263 | Time: 19877.7 ms [2021-12-06 15:22:30 main:574] : INFO : Epoch 1976 | loss: 0.0310634 | val_loss: 0.0311176 | Time: 20015.8 ms [2021-12-06 15:22:52 main:574] : INFO : Epoch 1977 | loss: 0.0310583 | val_loss: 0.0311267 | Time: 22046.5 ms [2021-12-06 15:23:14 main:574] : INFO : Epoch 1978 | loss: 0.0310582 | val_loss: 0.0311146 | Time: 21778.4 ms [2021-12-06 15:23:35 main:574] : INFO : Epoch 1979 | loss: 0.0310578 | val_loss: 0.0311176 | Time: 21165.8 ms [2021-12-06 15:23:56 main:574] : INFO : Epoch 1980 | loss: 0.0310597 | val_loss: 0.0311174 | Time: 20697.4 ms [2021-12-06 15:24:17 main:574] : INFO : Epoch 1981 | loss: 0.0310606 | val_loss: 0.0311182 | Time: 21105.4 ms [2021-12-06 15:24:40 main:574] : INFO : Epoch 1982 | loss: 0.0310539 | val_loss: 0.031119 | Time: 22526.6 ms [2021-12-06 15:25:00 main:574] : INFO : Epoch 1983 | loss: 0.0310514 | val_loss: 0.0311209 | Time: 20346 ms [2021-12-06 15:25:21 main:574] : INFO : Epoch 1984 | loss: 0.0310517 | val_loss: 0.0311183 | Time: 20357.8 ms [2021-12-06 15:25:41 main:574] : INFO : Epoch 1985 | loss: 0.0310477 | val_loss: 0.0311267 | Time: 20699 ms [2021-12-06 15:26:02 main:574] : INFO : Epoch 1986 | loss: 0.031047 | val_loss: 0.0311284 | Time: 20363.5 ms [2021-12-06 15:26:23 main:574] : INFO : Epoch 1987 | loss: 0.031054 | val_loss: 0.0311287 | Time: 21181.6 ms [2021-12-06 15:26:44 main:574] : INFO : Epoch 1988 | loss: 0.031056 | val_loss: 0.0311225 | Time: 20569.7 ms [2021-12-06 15:27:05 main:574] : INFO : Epoch 1989 | loss: 0.0310544 | val_loss: 0.0311212 | Time: 20872.2 ms [2021-12-06 15:27:26 main:574] : INFO : Epoch 1990 | loss: 0.0310571 | val_loss: 0.0311153 | Time: 21078.4 ms [2021-12-06 15:27:46 main:574] : INFO : Epoch 1991 | loss: 0.0310534 | val_loss: 0.0311197 | Time: 20320.2 ms [2021-12-06 15:28:07 main:574] : INFO : Epoch 1992 | loss: 0.0310568 | val_loss: 0.0311179 | Time: 21082.1 ms [2021-12-06 15:28:29 main:574] : INFO : Epoch 1993 | loss: 0.0310763 | val_loss: 0.0311235 | Time: 21301.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 15:36:18 main:435] : INFO : Set logging level to 1 [2021-12-06 15:36:18 main:441] : INFO : Running in BOINC Client mode [2021-12-06 15:36:18 main:444] : INFO : Resolving all filenames [2021-12-06 15:36:18 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 15:36:18 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 15:36:18 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 15:36:18 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 15:36:18 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 15:36:18 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 15:36:18 main:474] : INFO : Configuration: [2021-12-06 15:36:18 main:475] : INFO : Model type: GRU [2021-12-06 15:36:18 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 15:36:18 main:477] : INFO : Max Epochs: 2048 [2021-12-06 15:36:18 main:478] : INFO : Batch Size: 128 [2021-12-06 15:36:18 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 15:36:19 main:480] : INFO : Patience: 10 [2021-12-06 15:36:19 main:481] : INFO : Hidden Width: 12 [2021-12-06 15:36:19 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 15:36:19 main:483] : INFO : # Backend Layers: 4 [2021-12-06 15:36:19 main:484] : INFO : # Threads: 1 [2021-12-06 15:36:19 main:486] : INFO : Preparing Dataset [2021-12-06 15:36:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 15:36:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 15:36:27 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 15:36:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 15:36:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 15:36:27 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 15:36:27 main:494] : INFO : Creating Model [2021-12-06 15:36:27 main:507] : INFO : Preparing config file [2021-12-06 15:36:28 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 15:36:28 main:512] : INFO : Loading config [2021-12-06 15:36:28 main:514] : INFO : Loading state [2021-12-06 15:36:31 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 15:36:31 main:562] : INFO : Starting Training [2021-12-06 15:36:52 main:574] : INFO : Epoch 1931 | loss: 0.0311121 | val_loss: 0.0311422 | Time: 20839.9 ms [2021-12-06 15:37:12 main:574] : INFO : Epoch 1932 | loss: 0.0310704 | val_loss: 0.0311171 | Time: 19839.2 ms [2021-12-06 15:37:33 main:574] : INFO : Epoch 1933 | loss: 0.031063 | val_loss: 0.031125 | Time: 20836.1 ms [2021-12-06 15:37:53 main:574] : INFO : Epoch 1934 | loss: 0.0310551 | val_loss: 0.0311307 | Time: 20539.8 ms [2021-12-06 15:38:15 main:574] : INFO : Epoch 1935 | loss: 0.0310578 | val_loss: 0.031116 | Time: 21607.2 ms [2021-12-06 15:38:36 main:574] : INFO : Epoch 1936 | loss: 0.0310581 | val_loss: 0.0311176 | Time: 20954 ms [2021-12-06 15:38:57 main:574] : INFO : Epoch 1937 | loss: 0.0310584 | val_loss: 0.0311188 | Time: 20706.8 ms [2021-12-06 15:39:20 main:574] : INFO : Epoch 1938 | loss: 0.0310603 | val_loss: 0.0311263 | Time: 23060.9 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-06 15:41:25 main:435] : INFO : Set logging level to 1 [2021-12-06 15:41:25 main:441] : INFO : Running in BOINC Client mode [2021-12-06 15:41:25 main:444] : INFO : Resolving all filenames [2021-12-06 15:41:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-06 15:41:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-06 15:41:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-06 15:41:26 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-06 15:41:26 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-06 15:41:26 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-06 15:41:26 main:474] : INFO : Configuration: [2021-12-06 15:41:26 main:475] : INFO : Model type: GRU [2021-12-06 15:41:26 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-06 15:41:26 main:477] : INFO : Max Epochs: 2048 [2021-12-06 15:41:26 main:478] : INFO : Batch Size: 128 [2021-12-06 15:41:26 main:479] : INFO : Learning Rate: 0.01 [2021-12-06 15:41:26 main:480] : INFO : Patience: 10 [2021-12-06 15:41:26 main:481] : INFO : Hidden Width: 12 [2021-12-06 15:41:26 main:482] : INFO : # Recurrent Layers: 4 [2021-12-06 15:41:26 main:483] : INFO : # Backend Layers: 4 [2021-12-06 15:41:26 main:484] : INFO : # Threads: 1 [2021-12-06 15:41:26 main:486] : INFO : Preparing Dataset [2021-12-06 15:41:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-06 15:41:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-06 15:41:32 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-06 15:41:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-06 15:41:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-06 15:41:32 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-06 15:41:32 main:494] : INFO : Creating Model [2021-12-06 15:41:32 main:507] : INFO : Preparing config file [2021-12-06 15:41:32 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-06 15:41:32 main:512] : INFO : Loading config [2021-12-06 15:41:32 main:514] : INFO : Loading state [2021-12-06 15:41:37 main:559] : INFO : Loading DataLoader into Memory [2021-12-06 15:41:38 main:562] : INFO : Starting Training [2021-12-06 15:42:01 main:574] : INFO : Epoch 1931 | loss: 0.0311075 | val_loss: 0.0311312 | Time: 23338.1 ms [2021-12-06 15:42:22 main:574] : INFO : Epoch 1932 | loss: 0.0310666 | val_loss: 0.0311236 | Time: 20747.6 ms [2021-12-06 15:42:43 main:574] : INFO : Epoch 1933 | loss: 0.0310613 | val_loss: 0.0311238 | Time: 20610.8 ms [2021-12-06 15:43:05 main:574] : INFO : Epoch 1934 | loss: 0.0310574 | val_loss: 0.0311282 | Time: 21722.8 ms [2021-12-06 15:43:28 main:574] : INFO : Epoch 1935 | loss: 0.0310545 | val_loss: 0.0311164 | Time: 23425.9 ms [2021-12-06 15:43:58 main:574] : INFO : Epoch 1936 | loss: 0.0310617 | val_loss: 0.0311195 | Time: 29878.8 ms [2021-12-06 15:44:26 main:574] : INFO : Epoch 1937 | loss: 0.0310603 | val_loss: 0.0311197 | Time: 27692.1 ms [2021-12-06 15:44:52 main:574] : INFO : Epoch 1938 | loss: 0.0310587 | val_loss: 0.0311312 | Time: 26272.3 ms [2021-12-06 15:45:13 main:574] : INFO : Epoch 1939 | loss: 0.0310594 | val_loss: 0.0311201 | Time: 21023.9 ms [2021-12-06 15:45:36 main:574] : INFO : Epoch 1940 | loss: 0.0310536 | val_loss: 0.031118 | Time: 22924.4 ms [2021-12-06 15:45:58 main:574] : INFO : Epoch 1941 | loss: 0.0310494 | val_loss: 0.0311224 | Time: 21988.7 ms [2021-12-06 15:46:21 main:574] : INFO : Epoch 1942 | loss: 0.0310475 | val_loss: 0.0311209 | Time: 22429.7 ms [2021-12-06 15:46:42 main:574] : INFO : Epoch 1943 | loss: 0.0310457 | val_loss: 0.0311238 | Time: 21349.4 ms [2021-12-06 15:47:03 main:574] : INFO : Epoch 1944 | loss: 0.0310448 | val_loss: 0.0311305 | Time: 20942.5 ms [2021-12-06 15:47:25 main:574] : INFO : Epoch 1945 | loss: 0.0310481 | val_loss: 0.0311385 | Time: 21247 ms [2021-12-06 15:47:47 main:574] : INFO : Epoch 1946 | loss: 0.0310486 | val_loss: 0.0311318 | Time: 22198.9 ms [2021-12-06 15:48:10 main:574] : INFO : Epoch 1947 | loss: 0.0310468 | val_loss: 0.0311252 | Time: 22627.5 ms [2021-12-06 15:48:31 main:574] : INFO : Epoch 1948 | loss: 0.0310437 | val_loss: 0.03113 | Time: 21450.1 ms [2021-12-06 15:48:52 main:574] : INFO : Epoch 1949 | loss: 0.0310453 | val_loss: 0.0311303 | Time: 21205.4 ms [2021-12-06 15:49:13 main:574] : INFO : Epoch 1950 | loss: 0.0310485 | val_loss: 0.0311264 | Time: 20941.2 ms [2021-12-06 15:49:36 main:574] : INFO : Epoch 1951 | loss: 0.0310578 | val_loss: 0.0311247 | Time: 22093.4 ms [2021-12-06 15:49:59 main:574] : INFO : Epoch 1952 | loss: 0.0310569 | val_loss: 0.0311208 | Time: 22878.9 ms [2021-12-06 15:50:21 main:574] : INFO : Epoch 1953 | loss: 0.0310558 | val_loss: 0.0311185 | Time: 22406.3 ms [2021-12-06 15:50:42 main:574] : INFO : Epoch 1954 | loss: 0.0310546 | val_loss: 0.0311181 | Time: 21223 ms [2021-12-06 15:51:03 main:574] : INFO : Epoch 1955 | loss: 0.0310562 | val_loss: 0.0311198 | Time: 20521.4 ms [2021-12-06 15:51:24 main:574] : INFO : Epoch 1956 | loss: 0.0310527 | val_loss: 0.0311235 | Time: 20720.5 ms [2021-12-06 15:51:46 main:574] : INFO : Epoch 1957 | loss: 0.0310499 | val_loss: 0.0311223 | Time: 21793.1 ms [2021-12-06 15:52:08 main:574] : INFO : Epoch 1958 | loss: 0.0310518 | val_loss: 0.0311245 | Time: 22289.6 ms [2021-12-06 15:52:30 main:574] : INFO : Epoch 1959 | loss: 0.0310512 | val_loss: 0.0311147 | Time: 22112.2 ms [2021-12-06 15:52:53 main:574] : INFO : Epoch 1960 | loss: 0.0310604 | val_loss: 0.0311253 | Time: 22226.3 ms [2021-12-06 15:53:14 main:574] : INFO : Epoch 1961 | loss: 0.0310561 | val_loss: 0.0311248 | Time: 20973.6 ms [2021-12-06 15:53:35 main:574] : INFO : Epoch 1962 | loss: 0.0310507 | val_loss: 0.031121 | Time: 21737.6 ms [2021-12-06 15:53:58 main:574] : INFO : Epoch 1963 | loss: 0.0310549 | val_loss: 0.0311152 | Time: 22195.9 ms [2021-12-06 15:54:20 main:574] : INFO : Epoch 1964 | loss: 0.0310517 | val_loss: 0.0311203 | Time: 22679.9 ms [2021-12-06 15:54:43 main:574] : INFO : Epoch 1965 | loss: 0.031049 | val_loss: 0.0311198 | Time: 22529.9 ms [2021-12-06 15:55:04 main:574] : INFO : Epoch 1966 | loss: 0.0310472 | val_loss: 0.031121 | Time: 21183.2 ms [2021-12-06 15:55:27 main:574] : INFO : Epoch 1967 | loss: 0.0310622 | val_loss: 0.0311153 | Time: 22223.6 ms [2021-12-06 15:55:49 main:574] : INFO : Epoch 1968 | loss: 0.031069 | val_loss: 0.0311127 | Time: 22528.6 ms [2021-12-06 15:56:12 main:574] : INFO : Epoch 1969 | loss: 0.031064 | val_loss: 0.0311181 | Time: 22433.3 ms [2021-12-06 15:56:33 main:574] : INFO : Epoch 1970 | loss: 0.0310624 | val_loss: 0.0311134 | Time: 21434.5 ms [2021-12-06 15:56:54 main:574] : INFO : Epoch 1971 | loss: 0.0310527 | val_loss: 0.0311224 | Time: 21163.5 ms [2021-12-06 15:57:17 main:574] : INFO : Epoch 1972 | loss: 0.0310549 | val_loss: 0.0311177 | Time: 22536 ms [2021-12-06 15:57:39 main:574] : INFO : Epoch 1973 | loss: 0.0310563 | val_loss: 0.0311198 | Time: 22398.9 ms [2021-12-06 15:58:02 main:574] : INFO : Epoch 1974 | loss: 0.0310513 | val_loss: 0.0311137 | Time: 22143.7 ms [2021-12-06 15:58:23 main:574] : INFO : Epoch 1975 | loss: 0.0310526 | val_loss: 0.0311184 | Time: 21523.6 ms [2021-12-06 15:58:45 main:574] : INFO : Epoch 1976 | loss: 0.0310545 | val_loss: 0.0311176 | Time: 21207.6 ms [2021-12-06 15:59:07 main:574] : INFO : Epoch 1977 | loss: 0.0310528 | val_loss: 0.0311203 | Time: 22489.4 ms [2021-12-06 15:59:29 main:574] : INFO : Epoch 1978 | loss: 0.0310507 | val_loss: 0.0311201 | Time: 21988 ms [2021-12-06 15:59:52 main:574] : INFO : Epoch 1979 | loss: 0.0310506 | val_loss: 0.0311176 | Time: 22438.7 ms [2021-12-06 16:00:14 main:574] : INFO : Epoch 1980 | loss: 0.031049 | val_loss: 0.0311186 | Time: 22583.5 ms [2021-12-06 16:00:38 main:574] : INFO : Epoch 1981 | loss: 0.031053 | val_loss: 0.0311116 | Time: 23305.7 ms [2021-12-06 16:01:00 main:574] : INFO : Epoch 1982 | loss: 0.0310552 | val_loss: 0.0311081 | Time: 22112.7 ms [2021-12-06 16:01:22 main:574] : INFO : Epoch 1983 | loss: 0.0310492 | val_loss: 0.0311169 | Time: 22263.2 ms [2021-12-06 16:01:44 main:574] : INFO : Epoch 1984 | loss: 0.0310495 | val_loss: 0.031119 | Time: 21437.8 ms [2021-12-06 16:02:05 main:574] : INFO : Epoch 1985 | loss: 0.0310524 | val_loss: 0.0311171 | Time: 20781.3 ms [2021-12-06 16:02:27 main:574] : INFO : Epoch 1986 | loss: 0.0310493 | val_loss: 0.0311238 | Time: 21950.3 ms [2021-12-06 16:02:49 main:574] : INFO : Epoch 1987 | loss: 0.031043 | val_loss: 0.0311172 | Time: 22007.2 ms [2021-12-06 16:03:11 main:574] : INFO : Epoch 1988 | loss: 0.0310409 | val_loss: 0.0311225 | Time: 22497 ms [2021-12-06 16:03:33 main:574] : INFO : Epoch 1989 | loss: 0.0310402 | val_loss: 0.0311443 | Time: 21964.1 ms [2021-12-06 16:03:54 main:574] : INFO : Epoch 1990 | loss: 0.0310432 | val_loss: 0.0311239 | Time: 20797.3 ms [2021-12-06 16:04:18 main:574] : INFO : Epoch 1991 | loss: 0.0310376 | val_loss: 0.0311254 | Time: 23353.2 ms [2021-12-06 16:04:41 main:574] : INFO : Epoch 1992 | loss: 0.0310395 | val_loss: 0.0311118 | Time: 23000.9 ms [2021-12-06 16:05:03 main:574] : INFO : Epoch 1993 | loss: 0.0310394 | val_loss: 0.0311245 | Time: 22500 ms [2021-12-06 16:05:24 main:574] : INFO : Epoch 1994 | loss: 0.0310413 | val_loss: 0.0311264 | Time: 20799.9 ms [2021-12-06 16:05:45 main:574] : INFO : Epoch 1995 | loss: 0.0310379 | val_loss: 0.0311202 | Time: 20951.6 ms [2021-12-06 16:06:06 main:574] : INFO : Epoch 1996 | loss: 0.0310365 | val_loss: 0.0311236 | Time: 21209.8 ms [2021-12-06 16:06:29 main:574] : INFO : Epoch 1997 | loss: 0.0310435 | val_loss: 0.0311145 | Time: 22721 ms [2021-12-06 16:06:51 main:574] : INFO : Epoch 1998 | loss: 0.0310488 | val_loss: 0.0311166 | Time: 21631.2 ms [2021-12-06 16:07:14 main:574] : INFO : Epoch 1999 | loss: 0.0310576 | val_loss: 0.031114 | Time: 22756.1 ms [2021-12-06 16:07:37 main:574] : INFO : Epoch 2000 | loss: 0.0310549 | val_loss: 0.03112 | Time: 23397.6 ms [2021-12-06 16:07:59 main:574] : INFO : Epoch 2001 | loss: 0.031049 | val_loss: 0.0311358 | Time: 21673.4 ms [2021-12-06 16:08:21 main:574] : INFO : Epoch 2002 | loss: 0.0310452 | val_loss: 0.0311228 | Time: 21751.1 ms [2021-12-06 16:08:43 main:574] : INFO : Epoch 2003 | loss: 0.0310521 | val_loss: 0.0311228 | Time: 22485.2 ms [2021-12-06 16:09:05 main:574] : INFO : Epoch 2004 | loss: 0.0310481 | val_loss: 0.0311243 | Time: 21288.7 ms [2021-12-06 16:09:27 main:574] : INFO : Epoch 2005 | loss: 0.0310452 | val_loss: 0.0311202 | Time: 22483.6 ms [2021-12-06 16:09:50 main:574] : INFO : Epoch 2006 | loss: 0.0310516 | val_loss: 0.0311152 | Time: 22451.8 ms [2021-12-06 16:10:12 main:574] : INFO : Epoch 2007 | loss: 0.0310586 | val_loss: 0.0311179 | Time: 22256.9 ms [2021-12-06 16:10:34 main:574] : INFO : Epoch 2008 | loss: 0.031061 | val_loss: 0.0311207 | Time: 21665.7 ms [2021-12-06 16:10:55 main:574] : INFO : Epoch 2009 | loss: 0.0310679 | val_loss: 0.0311196 | Time: 21095.4 ms [2021-12-06 16:11:16 main:574] : INFO : Epoch 2010 | loss: 0.0310632 | val_loss: 0.0311278 | Time: 21054.2 ms [2021-12-06 16:11:39 main:574] : INFO : Epoch 2011 | loss: 0.0310584 | val_loss: 0.0311192 | Time: 23141.2 ms [2021-12-06 16:12:02 main:574] : INFO : Epoch 2012 | loss: 0.0310554 | val_loss: 0.0311128 | Time: 22222.5 ms [2021-12-06 16:12:24 main:574] : INFO : Epoch 2013 | loss: 0.0310529 | val_loss: 0.031118 | Time: 22396.2 ms [2021-12-06 16:12:46 main:574] : INFO : Epoch 2014 | loss: 0.0310551 | val_loss: 0.0311244 | Time: 21533.8 ms [2021-12-06 16:13:07 main:574] : INFO : Epoch 2015 | loss: 0.0310621 | val_loss: 0.0311194 | Time: 21841.4 ms [2021-12-06 16:13:31 main:574] : INFO : Epoch 2016 | loss: 0.0310729 | val_loss: 0.0311256 | Time: 22945.5 ms [2021-12-06 16:13:53 main:574] : INFO : Epoch 2017 | loss: 0.031074 | val_loss: 0.0311195 | Time: 22141.5 ms [2021-12-06 16:14:15 main:574] : INFO : Epoch 2018 | loss: 0.0310742 | val_loss: 0.0311233 | Time: 22097.4 ms [2021-12-06 16:14:36 main:574] : INFO : Epoch 2019 | loss: 0.0310808 | val_loss: 0.0311276 | Time: 21072.1 ms [2021-12-06 16:14:58 main:574] : INFO : Epoch 2020 | loss: 0.0310691 | val_loss: 0.031131 | Time: 22181.9 ms [2021-12-06 16:15:21 main:574] : INFO : Epoch 2021 | loss: 0.0310677 | val_loss: 0.0311305 | Time: 22862.8 ms [2021-12-06 16:15:44 main:574] : INFO : Epoch 2022 | loss: 0.0310697 | val_loss: 0.0311214 | Time: 22767.4 ms [2021-12-06 16:16:05 main:574] : INFO : Epoch 2023 | loss: 0.0310644 | val_loss: 0.0311306 | Time: 20985.9 ms [2021-12-06 16:16:27 main:574] : INFO : Epoch 2024 | loss: 0.0310589 | val_loss: 0.031124 | Time: 21414.2 ms [2021-12-06 16:16:49 main:574] : INFO : Epoch 2025 | loss: 0.0310574 | val_loss: 0.0311236 | Time: 22566.1 ms [2021-12-06 16:17:12 main:574] : INFO : Epoch 2026 | loss: 0.0310572 | val_loss: 0.0311172 | Time: 22447.1 ms [2021-12-06 16:17:34 main:574] : INFO : Epoch 2027 | loss: 0.0310688 | val_loss: 0.0311269 | Time: 22243.9 ms [2021-12-06 16:17:55 main:574] : INFO : Epoch 2028 | loss: 0.0310645 | val_loss: 0.0311203 | Time: 21080.9 ms [2021-12-06 16:18:16 main:574] : INFO : Epoch 2029 | loss: 0.0310589 | val_loss: 0.0311279 | Time: 20740.8 ms [2021-12-06 16:18:38 main:574] : INFO : Epoch 2030 | loss: 0.0310598 | val_loss: 0.0311203 | Time: 21938.7 ms [2021-12-06 16:19:00 main:574] : INFO : Epoch 2031 | loss: 0.0310623 | val_loss: 0.0311274 | Time: 22093.3 ms [2021-12-06 16:19:22 main:574] : INFO : Epoch 2032 | loss: 0.0310548 | val_loss: 0.0311215 | Time: 21995.2 ms [2021-12-06 16:19:44 main:574] : INFO : Epoch 2033 | loss: 0.0310611 | val_loss: 0.0311417 | Time: 21886.2 ms [2021-12-06 16:20:06 main:574] : INFO : Epoch 2034 | loss: 0.0310957 | val_loss: 0.0311358 | Time: 21377.6 ms [2021-12-06 16:20:31 main:574] : INFO : Epoch 2035 | loss: 0.0310859 | val_loss: 0.0311321 | Time: 24739 ms [2021-12-06 16:20:53 main:574] : INFO : Epoch 2036 | loss: 0.0310826 | val_loss: 0.031131 | Time: 22356.7 ms [2021-12-06 16:21:15 main:574] : INFO : Epoch 2037 | loss: 0.0310758 | val_loss: 0.0311278 | Time: 22270.9 ms [2021-12-06 16:21:36 main:574] : INFO : Epoch 2038 | loss: 0.0310746 | val_loss: 0.0311237 | Time: 20983.2 ms [2021-12-06 16:21:58 main:574] : INFO : Epoch 2039 | loss: 0.0310733 | val_loss: 0.0311221 | Time: 21761.2 ms [2021-12-06 16:22:21 main:574] : INFO : Epoch 2040 | loss: 0.0310717 | val_loss: 0.0311258 | Time: 22169.8 ms [2021-12-06 16:22:43 main:574] : INFO : Epoch 2041 | loss: 0.0310743 | val_loss: 0.0311286 | Time: 22806.9 ms [2021-12-06 16:23:05 main:574] : INFO : Epoch 2042 | loss: 0.0310756 | val_loss: 0.0311289 | Time: 21888.3 ms [2021-12-06 16:23:26 main:574] : INFO : Epoch 2043 | loss: 0.0310742 | val_loss: 0.0311271 | Time: 20920.5 ms [2021-12-06 16:23:48 main:574] : INFO : Epoch 2044 | loss: 0.0310704 | val_loss: 0.0311272 | Time: 21536.6 ms [2021-12-06 16:24:11 main:574] : INFO : Epoch 2045 | loss: 0.0310667 | val_loss: 0.0311241 | Time: 23431.7 ms [2021-12-06 16:24:35 main:574] : INFO : Epoch 2046 | loss: 0.0310656 | val_loss: 0.0311245 | Time: 23132.8 ms [2021-12-06 16:24:56 main:574] : INFO : Epoch 2047 | loss: 0.0310663 | val_loss: 0.0311319 | Time: 21558.8 ms [2021-12-06 16:25:17 main:574] : INFO : Epoch 2048 | loss: 0.0310716 | val_loss: 0.0311371 | Time: 20931.4 ms [2021-12-06 16:25:17 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311371 [2021-12-06 16:25:18 main:603] : INFO : Saving end state to config to file [2021-12-06 16:25:18 main:608] : INFO : Success, exiting.. 16:25:18 (22972): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)