Task 9706966
| Name | ParityModified-1638632750-18959-1-0_0 |
| Workunit | 7104241 |
| Created | 11 Dec 2021, 19:26:18 UTC |
| Sent | 17 Dec 2021, 14:07:35 UTC |
| Report deadline | 25 Dec 2021, 14:07:35 UTC |
| Received | 18 Dec 2021, 21:33:13 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 157 |
| Run time | 12 hours 30 min 38 sec |
| CPU time | 6 hours 55 min 39 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 4,474.06 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.61 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> :15 main:512] : INFO : Loading config [2021-12-18 11:29:15 main:514] : INFO : Loading state [2021-12-18 11:29:17 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 11:29:17 main:562] : INFO : Starting Training [2021-12-18 11:29:35 main:574] : INFO : Epoch 1812 | loss: 0.0311263 | val_loss: 0.0311859 | Time: 18508.1 ms [2021-12-18 11:29:53 main:574] : INFO : Epoch 1813 | loss: 0.0310916 | val_loss: 0.031185 | Time: 17394.6 ms [2021-12-18 11:30:10 main:574] : INFO : Epoch 1814 | loss: 0.0310832 | val_loss: 0.0311869 | Time: 17313 ms [2021-12-18 11:30:27 main:574] : INFO : Epoch 1815 | loss: 0.0310831 | val_loss: 0.031175 | Time: 17193.7 ms [2021-12-18 11:30:46 main:574] : INFO : Epoch 1816 | loss: 0.0310819 | val_loss: 0.0311802 | Time: 17994.5 ms [2021-12-18 11:31:03 main:574] : INFO : Epoch 1817 | loss: 0.031081 | val_loss: 0.0311768 | Time: 17664.3 ms [2021-12-18 11:31:20 main:574] : INFO : Epoch 1818 | loss: 0.0310854 | val_loss: 0.0311791 | Time: 17028.1 ms [2021-12-18 11:31:37 main:574] : INFO : Epoch 1819 | loss: 0.0310871 | val_loss: 0.0311853 | Time: 16744.4 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 11:39:16 main:435] : INFO : Set logging level to 1 [2021-12-18 11:39:16 main:441] : INFO : Running in BOINC Client mode [2021-12-18 11:39:16 main:444] : INFO : Resolving all filenames [2021-12-18 11:39:16 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 11:39:16 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 11:39:16 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 11:39:16 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 11:39:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 11:39:16 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 11:39:16 main:474] : INFO : Configuration: [2021-12-18 11:39:16 main:475] : INFO : Model type: GRU [2021-12-18 11:39:16 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 11:39:16 main:477] : INFO : Max Epochs: 2048 [2021-12-18 11:39:16 main:478] : INFO : Batch Size: 128 [2021-12-18 11:39:17 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 11:39:17 main:480] : INFO : Patience: 10 [2021-12-18 11:39:17 main:481] : INFO : Hidden Width: 12 [2021-12-18 11:39:17 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 11:39:17 main:483] : INFO : # Backend Layers: 4 [2021-12-18 11:39:17 main:484] : INFO : # Threads: 1 [2021-12-18 11:39:17 main:486] : INFO : Preparing Dataset [2021-12-18 11:39:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 11:39:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 11:39:23 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 11:39:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 11:39:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 11:39:23 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 11:39:24 main:494] : INFO : Creating Model [2021-12-18 11:39:24 main:507] : INFO : Preparing config file [2021-12-18 11:39:24 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 11:39:24 main:512] : INFO : Loading config [2021-12-18 11:39:24 main:514] : INFO : Loading state [2021-12-18 11:39:27 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 11:39:27 main:562] : INFO : Starting Training Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 11:49:03 main:435] : INFO : Set logging level to 1 [2021-12-18 11:49:03 main:441] : INFO : Running in BOINC Client mode [2021-12-18 11:49:03 main:444] : INFO : Resolving all filenames [2021-12-18 11:49:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 11:49:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 11:49:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 11:49:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 11:49:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 11:49:03 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 11:49:04 main:474] : INFO : Configuration: [2021-12-18 11:49:04 main:475] : INFO : Model type: GRU [2021-12-18 11:49:04 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 11:49:04 main:477] : INFO : Max Epochs: 2048 [2021-12-18 11:49:04 main:478] : INFO : Batch Size: 128 [2021-12-18 11:49:04 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 11:49:04 main:480] : INFO : Patience: 10 [2021-12-18 11:49:04 main:481] : INFO : Hidden Width: 12 [2021-12-18 11:49:04 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 11:49:04 main:483] : INFO : # Backend Layers: 4 [2021-12-18 11:49:04 main:484] : INFO : # Threads: 1 [2021-12-18 11:49:05 main:486] : INFO : Preparing Dataset [2021-12-18 11:49:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 11:49:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 11:49:09 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 11:49:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 11:49:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 11:49:10 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 11:49:10 main:494] : INFO : Creating Model [2021-12-18 11:49:10 main:507] : INFO : Preparing config file [2021-12-18 11:49:10 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 11:49:10 main:512] : INFO : Loading config [2021-12-18 11:49:10 main:514] : INFO : Loading state [2021-12-18 11:49:13 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 11:49:13 main:562] : INFO : Starting Training [2021-12-18 11:49:29 main:574] : INFO : Epoch 1812 | loss: 0.0311229 | val_loss: 0.0311698 | Time: 15963.6 ms [2021-12-18 11:49:45 main:574] : INFO : Epoch 1813 | loss: 0.0310882 | val_loss: 0.0311851 | Time: 15667.3 ms [2021-12-18 11:50:00 main:574] : INFO : Epoch 1814 | loss: 0.0310864 | val_loss: 0.0311771 | Time: 15416.9 ms [2021-12-18 11:50:16 main:574] : INFO : Epoch 1815 | loss: 0.0310855 | val_loss: 0.0311796 | Time: 15958.8 ms [2021-12-18 11:50:32 main:574] : INFO : Epoch 1816 | loss: 0.0310852 | val_loss: 0.031183 | Time: 15349.3 ms [2021-12-18 11:50:47 main:574] : INFO : Epoch 1817 | loss: 0.0310854 | val_loss: 0.0311832 | Time: 15264.9 ms [2021-12-18 11:51:03 main:574] : INFO : Epoch 1818 | loss: 0.0310827 | val_loss: 0.031179 | Time: 15745.4 ms [2021-12-18 11:51:19 main:574] : INFO : Epoch 1819 | loss: 0.0310826 | val_loss: 0.0311753 | Time: 16055.9 ms [2021-12-18 11:51:35 main:574] : INFO : Epoch 1820 | loss: 0.0310975 | val_loss: 0.0311677 | Time: 15647.3 ms [2021-12-18 11:51:50 main:574] : INFO : Epoch 1821 | loss: 0.0311274 | val_loss: 0.031157 | Time: 15425.2 ms [2021-12-18 11:52:07 main:574] : INFO : Epoch 1822 | loss: 0.0311214 | val_loss: 0.0311742 | Time: 16263.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 12:14:20 main:435] : INFO : Set logging level to 1 [2021-12-18 12:14:20 main:441] : INFO : Running in BOINC Client mode [2021-12-18 12:14:20 main:444] : INFO : Resolving all filenames [2021-12-18 12:14:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 12:14:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 12:14:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 12:14:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 12:14:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 12:14:21 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 12:14:21 main:474] : INFO : Configuration: [2021-12-18 12:14:21 main:475] : INFO : Model type: GRU [2021-12-18 12:14:21 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 12:14:21 main:477] : INFO : Max Epochs: 2048 [2021-12-18 12:14:21 main:478] : INFO : Batch Size: 128 [2021-12-18 12:14:21 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 12:14:21 main:480] : INFO : Patience: 10 [2021-12-18 12:14:21 main:481] : INFO : Hidden Width: 12 [2021-12-18 12:14:21 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 12:14:21 main:483] : INFO : # Backend Layers: 4 [2021-12-18 12:14:21 main:484] : INFO : # Threads: 1 [2021-12-18 12:14:22 main:486] : INFO : Preparing Dataset [2021-12-18 12:14:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 12:14:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 12:14:28 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 12:14:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 12:14:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 12:14:28 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 12:14:28 main:494] : INFO : Creating Model [2021-12-18 12:14:28 main:507] : INFO : Preparing config file [2021-12-18 12:14:29 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 12:14:29 main:512] : INFO : Loading config [2021-12-18 12:14:29 main:514] : INFO : Loading state [2021-12-18 12:14:31 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 12:14:31 main:562] : INFO : Starting Training [2021-12-18 12:14:51 main:574] : INFO : Epoch 1812 | loss: 0.0311292 | val_loss: 0.0311863 | Time: 19883.8 ms [2021-12-18 12:15:09 main:574] : INFO : Epoch 1813 | loss: 0.0310906 | val_loss: 0.0311706 | Time: 17975.4 ms [2021-12-18 12:15:29 main:574] : INFO : Epoch 1814 | loss: 0.0310925 | val_loss: 0.0311796 | Time: 19524.2 ms [2021-12-18 12:15:47 main:574] : INFO : Epoch 1815 | loss: 0.0310857 | val_loss: 0.0311813 | Time: 18401.4 ms [2021-12-18 12:16:07 main:574] : INFO : Epoch 1816 | loss: 0.031085 | val_loss: 0.0311791 | Time: 19395.8 ms [2021-12-18 12:16:28 main:574] : INFO : Epoch 1817 | loss: 0.0310912 | val_loss: 0.0311701 | Time: 21123 ms [2021-12-18 12:16:47 main:574] : INFO : Epoch 1818 | loss: 0.0311028 | val_loss: 0.0311712 | Time: 18828.9 ms [2021-12-18 12:17:06 main:574] : INFO : Epoch 1819 | loss: 0.0310945 | val_loss: 0.0311814 | Time: 19009.8 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 12:19:27 main:435] : INFO : Set logging level to 1 [2021-12-18 12:19:27 main:441] : INFO : Running in BOINC Client mode [2021-12-18 12:19:27 main:444] : INFO : Resolving all filenames [2021-12-18 12:19:27 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 12:19:28 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 12:19:28 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 12:19:28 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 12:19:28 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 12:19:28 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 12:19:28 main:474] : INFO : Configuration: [2021-12-18 12:19:29 main:475] : INFO : Model type: GRU [2021-12-18 12:19:29 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 12:19:29 main:477] : INFO : Max Epochs: 2048 [2021-12-18 12:19:29 main:478] : INFO : Batch Size: 128 [2021-12-18 12:19:29 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 12:19:29 main:480] : INFO : Patience: 10 [2021-12-18 12:19:29 main:481] : INFO : Hidden Width: 12 [2021-12-18 12:19:29 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 12:19:29 main:483] : INFO : # Backend Layers: 4 [2021-12-18 12:19:29 main:484] : INFO : # Threads: 1 [2021-12-18 12:19:29 main:486] : INFO : Preparing Dataset [2021-12-18 12:19:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 12:19:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 12:19:34 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 12:19:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 12:19:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 12:19:34 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 12:19:35 main:494] : INFO : Creating Model [2021-12-18 12:19:35 main:507] : INFO : Preparing config file [2021-12-18 12:19:35 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 12:19:35 main:512] : INFO : Loading config [2021-12-18 12:19:35 main:514] : INFO : Loading state [2021-12-18 12:19:39 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 12:19:39 main:562] : INFO : Starting Training [2021-12-18 12:19:59 main:574] : INFO : Epoch 1812 | loss: 0.0311291 | val_loss: 0.0311773 | Time: 19925 ms [2021-12-18 12:20:19 main:574] : INFO : Epoch 1813 | loss: 0.0310905 | val_loss: 0.0311857 | Time: 19510.6 ms [2021-12-18 12:20:38 main:574] : INFO : Epoch 1814 | loss: 0.0310855 | val_loss: 0.0311785 | Time: 19138.4 ms [2021-12-18 12:20:57 main:574] : INFO : Epoch 1815 | loss: 0.0310839 | val_loss: 0.0311724 | Time: 19274.5 ms [2021-12-18 12:21:18 main:574] : INFO : Epoch 1816 | loss: 0.0310833 | val_loss: 0.0311808 | Time: 20563.4 ms [2021-12-18 12:21:37 main:574] : INFO : Epoch 1817 | loss: 0.031082 | val_loss: 0.0311819 | Time: 19417.9 ms [2021-12-18 12:21:57 main:574] : INFO : Epoch 1818 | loss: 0.0310818 | val_loss: 0.0311769 | Time: 19285.6 ms [2021-12-18 12:22:16 main:574] : INFO : Epoch 1819 | loss: 0.0310855 | val_loss: 0.0311771 | Time: 19501.1 ms [2021-12-18 12:22:36 main:574] : INFO : Epoch 1820 | loss: 0.0310835 | val_loss: 0.0311877 | Time: 19716.8 ms [2021-12-18 12:22:56 main:574] : INFO : Epoch 1821 | loss: 0.0310795 | val_loss: 0.0311839 | Time: 19832 ms [2021-12-18 12:23:16 main:574] : INFO : Epoch 1822 | loss: 0.0310818 | val_loss: 0.0311822 | Time: 19645.7 ms [2021-12-18 12:23:36 main:574] : INFO : Epoch 1823 | loss: 0.0310801 | val_loss: 0.0311939 | Time: 19679.5 ms [2021-12-18 12:23:55 main:574] : INFO : Epoch 1824 | loss: 0.0310807 | val_loss: 0.0311853 | Time: 19760.5 ms [2021-12-18 12:24:15 main:574] : INFO : Epoch 1825 | loss: 0.0310827 | val_loss: 0.0311766 | Time: 19240 ms [2021-12-18 12:24:35 main:574] : INFO : Epoch 1826 | loss: 0.0310824 | val_loss: 0.0311801 | Time: 19773.3 ms [2021-12-18 12:24:54 main:574] : INFO : Epoch 1827 | loss: 0.0310826 | val_loss: 0.0311896 | Time: 18944.8 ms [2021-12-18 12:25:13 main:574] : INFO : Epoch 1828 | loss: 0.0310823 | val_loss: 0.0311827 | Time: 18947.4 ms [2021-12-18 12:25:32 main:574] : INFO : Epoch 1829 | loss: 0.031077 | val_loss: 0.0311839 | Time: 18975.6 ms [2021-12-18 12:25:50 main:574] : INFO : Epoch 1830 | loss: 0.0311282 | val_loss: 0.0311659 | Time: 18377.7 ms [2021-12-18 12:26:10 main:574] : INFO : Epoch 1831 | loss: 0.0311434 | val_loss: 0.0311578 | Time: 19306.6 ms [2021-12-18 12:26:30 main:574] : INFO : Epoch 1832 | loss: 0.0311528 | val_loss: 0.0311682 | Time: 19976.3 ms [2021-12-18 12:26:49 main:574] : INFO : Epoch 1833 | loss: 0.0311646 | val_loss: 0.0311711 | Time: 19538 ms [2021-12-18 12:27:09 main:574] : INFO : Epoch 1834 | loss: 0.0311606 | val_loss: 0.0311677 | Time: 19473.9 ms [2021-12-18 12:27:29 main:574] : INFO : Epoch 1835 | loss: 0.0311558 | val_loss: 0.0311677 | Time: 20373.3 ms [2021-12-18 12:27:49 main:574] : INFO : Epoch 1836 | loss: 0.0311518 | val_loss: 0.0311682 | Time: 19389.1 ms [2021-12-18 12:28:08 main:574] : INFO : Epoch 1837 | loss: 0.031152 | val_loss: 0.0311688 | Time: 19224 ms [2021-12-18 12:28:28 main:574] : INFO : Epoch 1838 | loss: 0.0311496 | val_loss: 0.0311746 | Time: 19196.9 ms [2021-12-18 12:28:47 main:574] : INFO : Epoch 1839 | loss: 0.0311487 | val_loss: 0.0311676 | Time: 19095.9 ms [2021-12-18 12:29:06 main:574] : INFO : Epoch 1840 | loss: 0.0311451 | val_loss: 0.0311703 | Time: 18894.2 ms [2021-12-18 12:29:25 main:574] : INFO : Epoch 1841 | loss: 0.0311454 | val_loss: 0.0311681 | Time: 19425.6 ms [2021-12-18 12:29:44 main:574] : INFO : Epoch 1842 | loss: 0.0311434 | val_loss: 0.0311678 | Time: 18787.8 ms [2021-12-18 12:30:04 main:574] : INFO : Epoch 1843 | loss: 0.0311445 | val_loss: 0.03116 | Time: 19827 ms [2021-12-18 12:30:24 main:574] : INFO : Epoch 1844 | loss: 0.0311456 | val_loss: 0.0311637 | Time: 19570 ms [2021-12-18 12:30:43 main:574] : INFO : Epoch 1845 | loss: 0.0311405 | val_loss: 0.0311648 | Time: 18890.5 ms [2021-12-18 12:31:02 main:574] : INFO : Epoch 1846 | loss: 0.0311374 | val_loss: 0.0311627 | Time: 19452.3 ms [2021-12-18 12:31:21 main:574] : INFO : Epoch 1847 | loss: 0.0311372 | val_loss: 0.0311656 | Time: 18532.5 ms [2021-12-18 12:31:40 main:574] : INFO : Epoch 1848 | loss: 0.0311364 | val_loss: 0.0311599 | Time: 19350.8 ms [2021-12-18 12:32:00 main:574] : INFO : Epoch 1849 | loss: 0.031132 | val_loss: 0.0311627 | Time: 19738 ms [2021-12-18 12:32:19 main:574] : INFO : Epoch 1850 | loss: 0.0311302 | val_loss: 0.0311634 | Time: 18245.1 ms [2021-12-18 12:32:38 main:574] : INFO : Epoch 1851 | loss: 0.0311303 | val_loss: 0.0311686 | Time: 19407.8 ms [2021-12-18 12:32:58 main:574] : INFO : Epoch 1852 | loss: 0.0311311 | val_loss: 0.0311623 | Time: 19592.8 ms [2021-12-18 12:33:17 main:574] : INFO : Epoch 1853 | loss: 0.0311282 | val_loss: 0.0311618 | Time: 19081.3 ms [2021-12-18 12:33:36 main:574] : INFO : Epoch 1854 | loss: 0.0311276 | val_loss: 0.0311616 | Time: 18531.2 ms [2021-12-18 12:33:55 main:574] : INFO : Epoch 1855 | loss: 0.031145 | val_loss: 0.03116 | Time: 19262.4 ms [2021-12-18 12:34:14 main:574] : INFO : Epoch 1856 | loss: 0.0311434 | val_loss: 0.0311631 | Time: 18715.4 ms [2021-12-18 12:34:31 main:574] : INFO : Epoch 1857 | loss: 0.0311403 | val_loss: 0.0311611 | Time: 17567.9 ms [2021-12-18 12:34:51 main:574] : INFO : Epoch 1858 | loss: 0.0311397 | val_loss: 0.0311606 | Time: 19132.1 ms [2021-12-18 12:35:10 main:574] : INFO : Epoch 1859 | loss: 0.0311365 | val_loss: 0.0311605 | Time: 19226.5 ms [2021-12-18 12:35:30 main:574] : INFO : Epoch 1860 | loss: 0.0311344 | val_loss: 0.0311591 | Time: 19623.6 ms [2021-12-18 12:35:49 main:574] : INFO : Epoch 1861 | loss: 0.0311325 | val_loss: 0.0311585 | Time: 19492.5 ms [2021-12-18 12:36:09 main:574] : INFO : Epoch 1862 | loss: 0.0311304 | val_loss: 0.0311587 | Time: 19272 ms [2021-12-18 12:36:28 main:574] : INFO : Epoch 1863 | loss: 0.0311276 | val_loss: 0.0311584 | Time: 19059.1 ms [2021-12-18 12:36:47 main:574] : INFO : Epoch 1864 | loss: 0.031129 | val_loss: 0.0311617 | Time: 18784.3 ms [2021-12-18 12:37:06 main:574] : INFO : Epoch 1865 | loss: 0.0311242 | val_loss: 0.0311559 | Time: 19076.2 ms [2021-12-18 12:37:24 main:574] : INFO : Epoch 1866 | loss: 0.0311222 | val_loss: 0.0311628 | Time: 18440.4 ms [2021-12-18 12:37:42 main:574] : INFO : Epoch 1867 | loss: 0.0311265 | val_loss: 0.0311574 | Time: 17186.6 ms [2021-12-18 12:38:01 main:574] : INFO : Epoch 1868 | loss: 0.0311222 | val_loss: 0.0311584 | Time: 19205.2 ms [2021-12-18 12:38:20 main:574] : INFO : Epoch 1869 | loss: 0.0311199 | val_loss: 0.0311594 | Time: 18558.7 ms [2021-12-18 12:38:38 main:574] : INFO : Epoch 1870 | loss: 0.031121 | val_loss: 0.0311642 | Time: 18613.5 ms [2021-12-18 12:38:57 main:574] : INFO : Epoch 1871 | loss: 0.0311207 | val_loss: 0.031162 | Time: 18553.6 ms [2021-12-18 12:39:16 main:574] : INFO : Epoch 1872 | loss: 0.031124 | val_loss: 0.0311602 | Time: 19245.2 ms [2021-12-18 12:39:34 main:574] : INFO : Epoch 1873 | loss: 0.031122 | val_loss: 0.0311614 | Time: 17961.6 ms [2021-12-18 12:39:52 main:574] : INFO : Epoch 1874 | loss: 0.0311185 | val_loss: 0.0311594 | Time: 17357.6 ms [2021-12-18 12:40:11 main:574] : INFO : Epoch 1875 | loss: 0.0311155 | val_loss: 0.0311623 | Time: 18732.2 ms [2021-12-18 12:40:32 main:574] : INFO : Epoch 1876 | loss: 0.0311163 | val_loss: 0.0311603 | Time: 21730.5 ms [2021-12-18 12:40:52 main:574] : INFO : Epoch 1877 | loss: 0.0311177 | val_loss: 0.0311617 | Time: 19719 ms [2021-12-18 12:41:12 main:574] : INFO : Epoch 1878 | loss: 0.031115 | val_loss: 0.0311586 | Time: 19675.3 ms [2021-12-18 12:41:32 main:574] : INFO : Epoch 1879 | loss: 0.0311153 | val_loss: 0.0311589 | Time: 19761.5 ms [2021-12-18 12:41:51 main:574] : INFO : Epoch 1880 | loss: 0.0311161 | val_loss: 0.0311633 | Time: 19512.8 ms [2021-12-18 12:42:11 main:574] : INFO : Epoch 1881 | loss: 0.031116 | val_loss: 0.0311614 | Time: 19857.6 ms [2021-12-18 12:42:31 main:574] : INFO : Epoch 1882 | loss: 0.0311169 | val_loss: 0.0311594 | Time: 19384 ms [2021-12-18 12:42:51 main:574] : INFO : Epoch 1883 | loss: 0.0311227 | val_loss: 0.0311583 | Time: 20228.8 ms [2021-12-18 12:43:16 main:574] : INFO : Epoch 1884 | loss: 0.031118 | val_loss: 0.0311615 | Time: 24617 ms [2021-12-18 12:43:36 main:574] : INFO : Epoch 1885 | loss: 0.0311154 | val_loss: 0.0311585 | Time: 19668.7 ms [2021-12-18 12:43:55 main:574] : INFO : Epoch 1886 | loss: 0.0311149 | val_loss: 0.0311612 | Time: 19669.4 ms [2021-12-18 12:44:15 main:574] : INFO : Epoch 1887 | loss: 0.0311134 | val_loss: 0.0311583 | Time: 19087.6 ms [2021-12-18 12:44:35 main:574] : INFO : Epoch 1888 | loss: 0.0311151 | val_loss: 0.03116 | Time: 20476.4 ms [2021-12-18 12:44:55 main:574] : INFO : Epoch 1889 | loss: 0.0311151 | val_loss: 0.0311617 | Time: 19723.8 ms [2021-12-18 12:45:15 main:574] : INFO : Epoch 1890 | loss: 0.0311108 | val_loss: 0.0311644 | Time: 20151.1 ms [2021-12-18 12:45:35 main:574] : INFO : Epoch 1891 | loss: 0.0311106 | val_loss: 0.0311645 | Time: 19698.7 ms [2021-12-18 12:45:55 main:574] : INFO : Epoch 1892 | loss: 0.0311106 | val_loss: 0.0311665 | Time: 19869.2 ms [2021-12-18 12:46:15 main:574] : INFO : Epoch 1893 | loss: 0.0311096 | val_loss: 0.0311627 | Time: 19744.4 ms [2021-12-18 12:46:35 main:574] : INFO : Epoch 1894 | loss: 0.0311095 | val_loss: 0.031162 | Time: 19689.7 ms [2021-12-18 12:46:55 main:574] : INFO : Epoch 1895 | loss: 0.031107 | val_loss: 0.0311667 | Time: 19903.6 ms [2021-12-18 12:47:14 main:574] : INFO : Epoch 1896 | loss: 0.0311072 | val_loss: 0.03117 | Time: 19741.2 ms [2021-12-18 12:47:34 main:574] : INFO : Epoch 1897 | loss: 0.0311104 | val_loss: 0.0311615 | Time: 19157.1 ms [2021-12-18 12:47:54 main:574] : INFO : Epoch 1898 | loss: 0.0311089 | val_loss: 0.0311722 | Time: 19920.4 ms [2021-12-18 12:48:14 main:574] : INFO : Epoch 1899 | loss: 0.0311106 | val_loss: 0.0311637 | Time: 19819.7 ms [2021-12-18 12:48:33 main:574] : INFO : Epoch 1900 | loss: 0.0311208 | val_loss: 0.0311502 | Time: 19574.8 ms [2021-12-18 12:48:53 main:574] : INFO : Epoch 1901 | loss: 0.0311181 | val_loss: 0.0311576 | Time: 19840.7 ms [2021-12-18 12:49:13 main:574] : INFO : Epoch 1902 | loss: 0.0311191 | val_loss: 0.0311607 | Time: 20017.8 ms [2021-12-18 12:49:33 main:574] : INFO : Epoch 1903 | loss: 0.031124 | val_loss: 0.0311579 | Time: 19971.4 ms [2021-12-18 12:49:53 main:574] : INFO : Epoch 1904 | loss: 0.0311222 | val_loss: 0.03115 | Time: 19966.2 ms [2021-12-18 12:50:13 main:574] : INFO : Epoch 1905 | loss: 0.0311186 | val_loss: 0.0311541 | Time: 19357.3 ms [2021-12-18 12:50:33 main:574] : INFO : Epoch 1906 | loss: 0.0311167 | val_loss: 0.0311482 | Time: 19827.4 ms [2021-12-18 12:50:53 main:574] : INFO : Epoch 1907 | loss: 0.0311133 | val_loss: 0.031152 | Time: 19927.9 ms [2021-12-18 12:51:13 main:574] : INFO : Epoch 1908 | loss: 0.0311124 | val_loss: 0.0311579 | Time: 20150.9 ms [2021-12-18 12:51:33 main:574] : INFO : Epoch 1909 | loss: 0.0311087 | val_loss: 0.0311534 | Time: 19908.3 ms [2021-12-18 12:51:52 main:574] : INFO : Epoch 1910 | loss: 0.0311068 | val_loss: 0.031157 | Time: 19224.8 ms [2021-12-18 12:52:13 main:574] : INFO : Epoch 1911 | loss: 0.0311115 | val_loss: 0.0311646 | Time: 20165.8 ms [2021-12-18 12:52:33 main:574] : INFO : Epoch 1912 | loss: 0.031113 | val_loss: 0.0311677 | Time: 20252.5 ms [2021-12-18 12:52:52 main:574] : INFO : Epoch 1913 | loss: 0.0311132 | val_loss: 0.0311575 | Time: 18993.4 ms [2021-12-18 12:53:12 main:574] : INFO : Epoch 1914 | loss: 0.0311144 | val_loss: 0.0311578 | Time: 19474.6 ms [2021-12-18 12:53:31 main:574] : INFO : Epoch 1915 | loss: 0.0311139 | val_loss: 0.0311654 | Time: 19253.5 ms [2021-12-18 12:53:51 main:574] : INFO : Epoch 1916 | loss: 0.0311127 | val_loss: 0.0311588 | Time: 20007.7 ms [2021-12-18 12:54:11 main:574] : INFO : Epoch 1917 | loss: 0.0311105 | val_loss: 0.0311536 | Time: 19704.1 ms [2021-12-18 12:54:30 main:574] : INFO : Epoch 1918 | loss: 0.0311078 | val_loss: 0.0311605 | Time: 19475.5 ms [2021-12-18 12:54:50 main:574] : INFO : Epoch 1919 | loss: 0.0311058 | val_loss: 0.0311572 | Time: 19712.2 ms [2021-12-18 12:55:10 main:574] : INFO : Epoch 1920 | loss: 0.0311097 | val_loss: 0.0311711 | Time: 19697.9 ms [2021-12-18 12:55:30 main:574] : INFO : Epoch 1921 | loss: 0.0311127 | val_loss: 0.0311597 | Time: 19928.1 ms [2021-12-18 12:55:51 main:574] : INFO : Epoch 1922 | loss: 0.0311105 | val_loss: 0.0311626 | Time: 20553.3 ms [2021-12-18 12:56:11 main:574] : INFO : Epoch 1923 | loss: 0.0311113 | val_loss: 0.0311579 | Time: 19843.2 ms [2021-12-18 12:56:31 main:574] : INFO : Epoch 1924 | loss: 0.0311073 | val_loss: 0.0311539 | Time: 19972.8 ms [2021-12-18 12:56:51 main:574] : INFO : Epoch 1925 | loss: 0.0311077 | val_loss: 0.0311611 | Time: 19852.7 ms [2021-12-18 12:57:11 main:574] : INFO : Epoch 1926 | loss: 0.0311065 | val_loss: 0.0311619 | Time: 20057.4 ms [2021-12-18 12:57:30 main:574] : INFO : Epoch 1927 | loss: 0.0311032 | val_loss: 0.0311655 | Time: 19430.5 ms [2021-12-18 12:57:50 main:574] : INFO : Epoch 1928 | loss: 0.0311008 | val_loss: 0.0311693 | Time: 19609.2 ms [2021-12-18 12:58:10 main:574] : INFO : Epoch 1929 | loss: 0.0310997 | val_loss: 0.0311622 | Time: 19687.1 ms [2021-12-18 12:58:30 main:574] : INFO : Epoch 1930 | loss: 0.0311007 | val_loss: 0.0311615 | Time: 19943.1 ms [2021-12-18 12:58:50 main:574] : INFO : Epoch 1931 | loss: 0.0311056 | val_loss: 0.0311661 | Time: 20295.4 ms [2021-12-18 12:59:10 main:574] : INFO : Epoch 1932 | loss: 0.0311044 | val_loss: 0.0311664 | Time: 19773.6 ms [2021-12-18 12:59:30 main:574] : INFO : Epoch 1933 | loss: 0.0311048 | val_loss: 0.0311568 | Time: 20076.7 ms [2021-12-18 12:59:50 main:574] : INFO : Epoch 1934 | loss: 0.0311052 | val_loss: 0.0311625 | Time: 20157 ms [2021-12-18 13:00:11 main:574] : INFO : Epoch 1935 | loss: 0.0311049 | val_loss: 0.0311561 | Time: 20819 ms [2021-12-18 13:00:31 main:574] : INFO : Epoch 1936 | loss: 0.0311028 | val_loss: 0.0311642 | Time: 19902.9 ms [2021-12-18 13:00:51 main:574] : INFO : Epoch 1937 | loss: 0.0311019 | val_loss: 0.0311729 | Time: 19358.4 ms [2021-12-18 13:01:11 main:574] : INFO : Epoch 1938 | loss: 0.0311021 | val_loss: 0.0311709 | Time: 19564.2 ms [2021-12-18 13:01:30 main:574] : INFO : Epoch 1939 | loss: 0.0311126 | val_loss: 0.0311648 | Time: 19807.2 ms [2021-12-18 13:01:50 main:574] : INFO : Epoch 1940 | loss: 0.03117 | val_loss: 0.0311904 | Time: 20001 ms [2021-12-18 13:02:11 main:574] : INFO : Epoch 1941 | loss: 0.031351 | val_loss: 0.0319729 | Time: 19997.4 ms [2021-12-18 13:02:30 main:574] : INFO : Epoch 1942 | loss: 0.0322578 | val_loss: 0.0315293 | Time: 19795.1 ms [2021-12-18 13:02:51 main:574] : INFO : Epoch 1943 | loss: 0.0316225 | val_loss: 0.0313335 | Time: 20075.1 ms [2021-12-18 13:03:11 main:574] : INFO : Epoch 1944 | loss: 0.0312809 | val_loss: 0.0312391 | Time: 20199.8 ms [2021-12-18 13:03:31 main:574] : INFO : Epoch 1945 | loss: 0.0312171 | val_loss: 0.0311873 | Time: 19829.4 ms [2021-12-18 13:03:51 main:574] : INFO : Epoch 1946 | loss: 0.0311886 | val_loss: 0.0311817 | Time: 19654.1 ms [2021-12-18 13:04:11 main:574] : INFO : Epoch 1947 | loss: 0.0311822 | val_loss: 0.0311751 | Time: 20018.8 ms [2021-12-18 13:04:31 main:574] : INFO : Epoch 1948 | loss: 0.0311794 | val_loss: 0.0311754 | Time: 19915.2 ms [2021-12-18 13:04:51 main:574] : INFO : Epoch 1949 | loss: 0.0311776 | val_loss: 0.0311744 | Time: 19753.6 ms [2021-12-18 13:05:11 main:574] : INFO : Epoch 1950 | loss: 0.0311748 | val_loss: 0.0311716 | Time: 20141.5 ms [2021-12-18 13:05:31 main:574] : INFO : Epoch 1951 | loss: 0.0311704 | val_loss: 0.0311691 | Time: 19733 ms [2021-12-18 13:05:50 main:574] : INFO : Epoch 1952 | loss: 0.031163 | val_loss: 0.0311661 | Time: 19439.3 ms [2021-12-18 13:06:10 main:574] : INFO : Epoch 1953 | loss: 0.0311584 | val_loss: 0.0311654 | Time: 19672.6 ms [2021-12-18 13:06:30 main:574] : INFO : Epoch 1954 | loss: 0.0311578 | val_loss: 0.0311626 | Time: 19832.8 ms [2021-12-18 13:06:50 main:574] : INFO : Epoch 1955 | loss: 0.031151 | val_loss: 0.031157 | Time: 20421.7 ms [2021-12-18 13:07:11 main:574] : INFO : Epoch 1956 | loss: 0.0311444 | val_loss: 0.0311554 | Time: 20474.5 ms [2021-12-18 13:07:31 main:574] : INFO : Epoch 1957 | loss: 0.0311406 | val_loss: 0.0311525 | Time: 19854.4 ms [2021-12-18 13:07:51 main:574] : INFO : Epoch 1958 | loss: 0.0311346 | val_loss: 0.0311505 | Time: 19972 ms [2021-12-18 13:08:11 main:574] : INFO : Epoch 1959 | loss: 0.0311307 | val_loss: 0.0311539 | Time: 19823.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 13:12:57 main:435] : INFO : Set logging level to 1 [2021-12-18 13:12:57 main:441] : INFO : Running in BOINC Client mode [2021-12-18 13:12:57 main:444] : INFO : Resolving all filenames [2021-12-18 13:12:57 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 13:12:57 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 13:12:57 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 13:12:57 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 13:12:58 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 13:12:58 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 13:12:58 main:474] : INFO : Configuration: [2021-12-18 13:12:58 main:475] : INFO : Model type: GRU [2021-12-18 13:12:58 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 13:12:58 main:477] : INFO : Max Epochs: 2048 [2021-12-18 13:12:58 main:478] : INFO : Batch Size: 128 [2021-12-18 13:12:58 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 13:12:58 main:480] : INFO : Patience: 10 [2021-12-18 13:12:58 main:481] : INFO : Hidden Width: 12 [2021-12-18 13:12:58 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 13:12:59 main:483] : INFO : # Backend Layers: 4 [2021-12-18 13:12:59 main:484] : INFO : # Threads: 1 [2021-12-18 13:12:59 main:486] : INFO : Preparing Dataset [2021-12-18 13:12:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 13:13:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 13:13:04 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 13:13:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 13:13:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 13:13:05 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 13:13:05 main:494] : INFO : Creating Model [2021-12-18 13:13:05 main:507] : INFO : Preparing config file [2021-12-18 13:13:05 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 13:13:05 main:512] : INFO : Loading config [2021-12-18 13:13:05 main:514] : INFO : Loading state [2021-12-18 13:13:08 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 13:13:08 main:562] : INFO : Starting Training [2021-12-18 13:13:25 main:574] : INFO : Epoch 1906 | loss: 0.0311499 | val_loss: 0.0311621 | Time: 17105.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 13:43:33 main:435] : INFO : Set logging level to 1 [2021-12-18 13:43:33 main:441] : INFO : Running in BOINC Client mode [2021-12-18 13:43:33 main:444] : INFO : Resolving all filenames [2021-12-18 13:43:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 13:43:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 13:43:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 13:43:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 13:43:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 13:43:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 13:43:33 main:474] : INFO : Configuration: [2021-12-18 13:43:33 main:475] : INFO : Model type: GRU [2021-12-18 13:43:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 13:43:33 main:477] : INFO : Max Epochs: 2048 [2021-12-18 13:43:34 main:478] : INFO : Batch Size: 128 [2021-12-18 13:43:34 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 13:43:34 main:480] : INFO : Patience: 10 [2021-12-18 13:43:34 main:481] : INFO : Hidden Width: 12 [2021-12-18 13:43:34 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 13:43:34 main:483] : INFO : # Backend Layers: 4 [2021-12-18 13:43:34 main:484] : INFO : # Threads: 1 [2021-12-18 13:43:34 main:486] : INFO : Preparing Dataset [2021-12-18 13:43:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 13:43:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 13:43:38 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 13:43:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 13:43:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 13:43:39 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 13:43:39 main:494] : INFO : Creating Model [2021-12-18 13:43:39 main:507] : INFO : Preparing config file [2021-12-18 13:43:39 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 13:43:39 main:512] : INFO : Loading config [2021-12-18 13:43:39 main:514] : INFO : Loading state [2021-12-18 13:43:42 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 13:43:43 main:562] : INFO : Starting Training [2021-12-18 13:43:58 main:574] : INFO : Epoch 1906 | loss: 0.0311597 | val_loss: 0.0311637 | Time: 15676.2 ms [2021-12-18 13:44:14 main:574] : INFO : Epoch 1907 | loss: 0.0311244 | val_loss: 0.0311563 | Time: 15695.3 ms [2021-12-18 13:44:29 main:574] : INFO : Epoch 1908 | loss: 0.0311212 | val_loss: 0.0311557 | Time: 14818.7 ms [2021-12-18 13:44:45 main:574] : INFO : Epoch 1909 | loss: 0.0311227 | val_loss: 0.0311506 | Time: 15577.3 ms [2021-12-18 13:45:00 main:574] : INFO : Epoch 1910 | loss: 0.0311201 | val_loss: 0.0311614 | Time: 15320.3 ms [2021-12-18 13:45:16 main:574] : INFO : Epoch 1911 | loss: 0.0311161 | val_loss: 0.0311541 | Time: 15947.1 ms [2021-12-18 13:45:31 main:574] : INFO : Epoch 1912 | loss: 0.0311148 | val_loss: 0.031155 | Time: 15317 ms [2021-12-18 13:45:47 main:574] : INFO : Epoch 1913 | loss: 0.0311176 | val_loss: 0.031157 | Time: 15894.4 ms [2021-12-18 13:46:03 main:574] : INFO : Epoch 1914 | loss: 0.0311151 | val_loss: 0.0311552 | Time: 15615 ms [2021-12-18 13:46:18 main:574] : INFO : Epoch 1915 | loss: 0.0311129 | val_loss: 0.0311573 | Time: 15248.6 ms [2021-12-18 13:46:34 main:574] : INFO : Epoch 1916 | loss: 0.0311115 | val_loss: 0.031157 | Time: 15970.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB) [2021-12-18 13:48:40 main:435] : INFO : Set logging level to 1 [2021-12-18 13:48:40 main:441] : INFO : Running in BOINC Client mode [2021-12-18 13:48:40 main:444] : INFO : Resolving all filenames [2021-12-18 13:48:40 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-18 13:48:40 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-18 13:48:40 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-18 13:48:40 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-18 13:48:40 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-18 13:48:40 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-18 13:48:41 main:474] : INFO : Configuration: [2021-12-18 13:48:41 main:475] : INFO : Model type: GRU [2021-12-18 13:48:41 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-18 13:48:41 main:477] : INFO : Max Epochs: 2048 [2021-12-18 13:48:41 main:478] : INFO : Batch Size: 128 [2021-12-18 13:48:41 main:479] : INFO : Learning Rate: 0.01 [2021-12-18 13:48:41 main:480] : INFO : Patience: 10 [2021-12-18 13:48:41 main:481] : INFO : Hidden Width: 12 [2021-12-18 13:48:41 main:482] : INFO : # Recurrent Layers: 4 [2021-12-18 13:48:41 main:483] : INFO : # Backend Layers: 4 [2021-12-18 13:48:41 main:484] : INFO : # Threads: 1 [2021-12-18 13:48:41 main:486] : INFO : Preparing Dataset [2021-12-18 13:48:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-18 13:48:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-18 13:48:46 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-18 13:48:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-18 13:48:47 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-18 13:48:47 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-18 13:48:47 main:494] : INFO : Creating Model [2021-12-18 13:48:47 main:507] : INFO : Preparing config file [2021-12-18 13:48:47 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-18 13:48:47 main:512] : INFO : Loading config [2021-12-18 13:48:47 main:514] : INFO : Loading state [2021-12-18 13:48:49 main:559] : INFO : Loading DataLoader into Memory [2021-12-18 13:48:49 main:562] : INFO : Starting Training [2021-12-18 13:49:06 main:574] : INFO : Epoch 1906 | loss: 0.0311492 | val_loss: 0.0311536 | Time: 16277.8 ms [2021-12-18 13:49:21 main:574] : INFO : Epoch 1907 | loss: 0.031122 | val_loss: 0.0311522 | Time: 15419.3 ms [2021-12-18 13:49:37 main:574] : INFO : Epoch 1908 | loss: 0.0311163 | val_loss: 0.0311494 | Time: 15675.9 ms [2021-12-18 13:49:52 main:574] : INFO : Epoch 1909 | loss: 0.0311247 | val_loss: 0.0311553 | Time: 15263.5 ms [2021-12-18 13:50:08 main:574] : INFO : Epoch 1910 | loss: 0.031121 | val_loss: 0.0311494 | Time: 15474 ms [2021-12-18 13:50:23 main:574] : INFO : Epoch 1911 | loss: 0.031115 | val_loss: 0.03115 | Time: 15531.7 ms [2021-12-18 13:50:38 main:574] : INFO : Epoch 1912 | loss: 0.0311126 | val_loss: 0.0311534 | Time: 14921.5 ms [2021-12-18 13:50:54 main:574] : INFO : Epoch 1913 | loss: 0.0311152 | val_loss: 0.0311502 | Time: 15862.4 ms [2021-12-18 13:51:10 main:574] : INFO : Epoch 1914 | loss: 0.0311134 | val_loss: 0.0311549 | Time: 15347.8 ms [2021-12-18 13:51:25 main:574] : INFO : Epoch 1915 | loss: 0.0311133 | val_loss: 0.0311541 | Time: 15645 ms [2021-12-18 13:51:41 main:574] : INFO : Epoch 1916 | loss: 0.0311123 | val_loss: 0.0311551 | Time: 15934.1 ms [2021-12-18 13:51:57 main:574] : INFO : Epoch 1917 | loss: 0.0311108 | val_loss: 0.0311571 | Time: 15779.2 ms [2021-12-18 13:52:13 main:574] : INFO : Epoch 1918 | loss: 0.031116 | val_loss: 0.0311592 | Time: 15446.1 ms [2021-12-18 13:52:28 main:574] : INFO : Epoch 1919 | loss: 0.0311179 | val_loss: 0.0311601 | Time: 15411.1 ms [2021-12-18 13:52:44 main:574] : INFO : Epoch 1920 | loss: 0.0311159 | val_loss: 0.0311532 | Time: 15585.4 ms [2021-12-18 13:52:59 main:574] : INFO : Epoch 1921 | loss: 0.0311139 | val_loss: 0.0311544 | Time: 15292.5 ms [2021-12-18 13:53:15 main:574] : INFO : Epoch 1922 | loss: 0.0311096 | val_loss: 0.0311513 | Time: 15907.2 ms [2021-12-18 13:53:31 main:574] : INFO : Epoch 1923 | loss: 0.0311128 | val_loss: 0.0311561 | Time: 15093.6 ms [2021-12-18 13:53:46 main:574] : INFO : Epoch 1924 | loss: 0.0311201 | val_loss: 0.031152 | Time: 15758.9 ms [2021-12-18 13:54:02 main:574] : INFO : Epoch 1925 | loss: 0.031115 | val_loss: 0.0311537 | Time: 15792.9 ms [2021-12-18 13:54:18 main:574] : INFO : Epoch 1926 | loss: 0.0311177 | val_loss: 0.0311552 | Time: 15915.9 ms [2021-12-18 13:54:34 main:574] : INFO : Epoch 1927 | loss: 0.0311152 | val_loss: 0.0311573 | Time: 15578 ms [2021-12-18 13:54:49 main:574] : INFO : Epoch 1928 | loss: 0.0311145 | val_loss: 0.0311502 | Time: 14946.2 ms [2021-12-18 13:55:05 main:574] : INFO : Epoch 1929 | loss: 0.0311156 | val_loss: 0.0311552 | Time: 15865.9 ms [2021-12-18 13:55:20 main:574] : INFO : Epoch 1930 | loss: 0.0311071 | val_loss: 0.0311579 | Time: 15150.1 ms [2021-12-18 13:55:36 main:574] : INFO : Epoch 1931 | loss: 0.0311067 | val_loss: 0.0311547 | Time: 15637.5 ms [2021-12-18 13:55:51 main:574] : INFO : Epoch 1932 | loss: 0.0311055 | val_loss: 0.0311523 | Time: 15525.9 ms [2021-12-18 13:56:07 main:574] : INFO : Epoch 1933 | loss: 0.0311045 | val_loss: 0.0311525 | Time: 15317.8 ms [2021-12-18 13:56:22 main:574] : INFO : Epoch 1934 | loss: 0.0311017 | val_loss: 0.0311595 | Time: 15367 ms [2021-12-18 13:56:38 main:574] : INFO : Epoch 1935 | loss: 0.0311023 | val_loss: 0.0311545 | Time: 15380.9 ms [2021-12-18 13:56:53 main:574] : INFO : Epoch 1936 | loss: 0.0311007 | val_loss: 0.0311581 | Time: 15277.9 ms [2021-12-18 13:57:09 main:574] : INFO : Epoch 1937 | loss: 0.0311007 | val_loss: 0.0311628 | Time: 15525.2 ms [2021-12-18 13:57:24 main:574] : INFO : Epoch 1938 | loss: 0.0311004 | val_loss: 0.0311514 | Time: 15212.1 ms [2021-12-18 13:57:39 main:574] : INFO : Epoch 1939 | loss: 0.0311044 | val_loss: 0.0311656 | Time: 15248.8 ms [2021-12-18 13:57:55 main:574] : INFO : Epoch 1940 | loss: 0.031101 | val_loss: 0.031153 | Time: 15925.3 ms [2021-12-18 13:58:11 main:574] : INFO : Epoch 1941 | loss: 0.0310985 | val_loss: 0.031158 | Time: 15372.9 ms [2021-12-18 13:58:26 main:574] : INFO : Epoch 1942 | loss: 0.0310983 | val_loss: 0.0311584 | Time: 15682.2 ms [2021-12-18 13:58:42 main:574] : INFO : Epoch 1943 | loss: 0.0310954 | val_loss: 0.0311623 | Time: 15166.3 ms [2021-12-18 13:58:57 main:574] : INFO : Epoch 1944 | loss: 0.0310991 | val_loss: 0.0311576 | Time: 15223.5 ms [2021-12-18 13:59:12 main:574] : INFO : Epoch 1945 | loss: 0.0310985 | val_loss: 0.0311626 | Time: 15395.8 ms [2021-12-18 13:59:28 main:574] : INFO : Epoch 1946 | loss: 0.0311052 | val_loss: 0.0311627 | Time: 15186.7 ms [2021-12-18 13:59:43 main:574] : INFO : Epoch 1947 | loss: 0.0311213 | val_loss: 0.0311621 | Time: 15568.3 ms [2021-12-18 13:59:59 main:574] : INFO : Epoch 1948 | loss: 0.031114 | val_loss: 0.0311595 | Time: 15222.8 ms [2021-12-18 14:00:16 main:574] : INFO : Epoch 1949 | loss: 0.03111 | val_loss: 0.0311628 | Time: 17387.7 ms [2021-12-18 14:00:34 main:574] : INFO : Epoch 1950 | loss: 0.0311088 | val_loss: 0.0311551 | Time: 18182.9 ms [2021-12-18 14:00:53 main:574] : INFO : Epoch 1951 | loss: 0.0311093 | val_loss: 0.0311569 | Time: 18437.5 ms [2021-12-18 14:01:13 main:574] : INFO : Epoch 1952 | loss: 0.0311048 | val_loss: 0.0311611 | Time: 19997.4 ms [2021-12-18 14:01:32 main:574] : INFO : Epoch 1953 | loss: 0.0311032 | val_loss: 0.031156 | Time: 18851.7 ms [2021-12-18 14:01:52 main:574] : INFO : Epoch 1954 | loss: 0.0310996 | val_loss: 0.0311587 | Time: 19445.8 ms [2021-12-18 14:02:10 main:574] : INFO : Epoch 1955 | loss: 0.0310994 | val_loss: 0.031159 | Time: 18805.7 ms [2021-12-18 14:02:29 main:574] : INFO : Epoch 1956 | loss: 0.0310991 | val_loss: 0.0311574 | Time: 18890.8 ms [2021-12-18 14:02:49 main:574] : INFO : Epoch 1957 | loss: 0.0311001 | val_loss: 0.03116 | Time: 19751.8 ms [2021-12-18 14:03:09 main:574] : INFO : Epoch 1958 | loss: 0.0310979 | val_loss: 0.03116 | Time: 19298.6 ms [2021-12-18 14:03:28 main:574] : INFO : Epoch 1959 | loss: 0.0310991 | val_loss: 0.0311643 | Time: 19759 ms [2021-12-18 14:03:47 main:574] : INFO : Epoch 1960 | loss: 0.0311036 | val_loss: 0.0311601 | Time: 18880 ms [2021-12-18 14:04:08 main:574] : INFO : Epoch 1961 | loss: 0.0311066 | val_loss: 0.0311471 | Time: 20271.1 ms [2021-12-18 14:04:27 main:574] : INFO : Epoch 1962 | loss: 0.0311164 | val_loss: 0.0311561 | Time: 19092.3 ms [2021-12-18 14:04:46 main:574] : INFO : Epoch 1963 | loss: 0.0311118 | val_loss: 0.0311522 | Time: 19262 ms [2021-12-18 14:05:06 main:574] : INFO : Epoch 1964 | loss: 0.031107 | val_loss: 0.0311571 | Time: 19159.3 ms [2021-12-18 14:05:25 main:574] : INFO : Epoch 1965 | loss: 0.0311029 | val_loss: 0.0311532 | Time: 18776.4 ms [2021-12-18 14:05:44 main:574] : INFO : Epoch 1966 | loss: 0.0311028 | val_loss: 0.0311586 | Time: 19704.6 ms [2021-12-18 14:06:04 main:574] : INFO : Epoch 1967 | loss: 0.0311044 | val_loss: 0.0311566 | Time: 19045.4 ms [2021-12-18 14:06:23 main:574] : INFO : Epoch 1968 | loss: 0.0311037 | val_loss: 0.031154 | Time: 19653.7 ms [2021-12-18 14:06:43 main:574] : INFO : Epoch 1969 | loss: 0.0311037 | val_loss: 0.0311562 | Time: 19413.9 ms [2021-12-18 14:07:03 main:574] : INFO : Epoch 1970 | loss: 0.0311001 | val_loss: 0.0311571 | Time: 19649.7 ms [2021-12-18 14:07:23 main:574] : INFO : Epoch 1971 | loss: 0.0310988 | val_loss: 0.0311557 | Time: 19885.8 ms [2021-12-18 14:07:42 main:574] : INFO : Epoch 1972 | loss: 0.031097 | val_loss: 0.0311667 | Time: 19341.6 ms [2021-12-18 14:08:02 main:574] : INFO : Epoch 1973 | loss: 0.0310975 | val_loss: 0.0311593 | Time: 19700.2 ms [2021-12-18 14:08:22 main:574] : INFO : Epoch 1974 | loss: 0.0310988 | val_loss: 0.0311537 | Time: 20069.6 ms [2021-12-18 14:08:41 main:574] : INFO : Epoch 1975 | loss: 0.0310981 | val_loss: 0.0311534 | Time: 19039.7 ms [2021-12-18 14:09:00 main:574] : INFO : Epoch 1976 | loss: 0.0311015 | val_loss: 0.0311593 | Time: 18883.8 ms [2021-12-18 14:09:20 main:574] : INFO : Epoch 1977 | loss: 0.0310999 | val_loss: 0.0311547 | Time: 20114.1 ms [2021-12-18 14:09:40 main:574] : INFO : Epoch 1978 | loss: 0.0311042 | val_loss: 0.0311569 | Time: 19442.7 ms [2021-12-18 14:09:59 main:574] : INFO : Epoch 1979 | loss: 0.0310977 | val_loss: 0.0311566 | Time: 19463.9 ms [2021-12-18 14:10:19 main:574] : INFO : Epoch 1980 | loss: 0.031095 | val_loss: 0.0311602 | Time: 19781.4 ms [2021-12-18 14:10:39 main:574] : INFO : Epoch 1981 | loss: 0.031095 | val_loss: 0.0311603 | Time: 19258.2 ms [2021-12-18 14:10:59 main:574] : INFO : Epoch 1982 | loss: 0.0310993 | val_loss: 0.0311558 | Time: 19805.9 ms [2021-12-18 14:11:18 main:574] : INFO : Epoch 1983 | loss: 0.0310982 | val_loss: 0.0311559 | Time: 19160.4 ms [2021-12-18 14:11:37 main:574] : INFO : Epoch 1984 | loss: 0.0310942 | val_loss: 0.0311587 | Time: 19440 ms [2021-12-18 14:11:57 main:574] : INFO : Epoch 1985 | loss: 0.0311025 | val_loss: 0.0311601 | Time: 19835.2 ms [2021-12-18 14:12:17 main:574] : INFO : Epoch 1986 | loss: 0.0311006 | val_loss: 0.0311631 | Time: 19752.4 ms [2021-12-18 14:12:36 main:574] : INFO : Epoch 1987 | loss: 0.0310994 | val_loss: 0.0311674 | Time: 18907.2 ms [2021-12-18 14:12:55 main:574] : INFO : Epoch 1988 | loss: 0.031103 | val_loss: 0.0311666 | Time: 19037.7 ms [2021-12-18 14:13:15 main:574] : INFO : Epoch 1989 | loss: 0.0310972 | val_loss: 0.0311629 | Time: 19452.9 ms [2021-12-18 14:13:34 main:574] : INFO : Epoch 1990 | loss: 0.0310945 | val_loss: 0.031161 | Time: 19284.4 ms [2021-12-18 14:13:54 main:574] : INFO : Epoch 1991 | loss: 0.0310904 | val_loss: 0.0311695 | Time: 19391.5 ms [2021-12-18 14:14:13 main:574] : INFO : Epoch 1992 | loss: 0.0310899 | val_loss: 0.0311629 | Time: 19480.4 ms [2021-12-18 14:14:32 main:574] : INFO : Epoch 1993 | loss: 0.0310884 | val_loss: 0.0311647 | Time: 18134.6 ms [2021-12-18 14:14:51 main:574] : INFO : Epoch 1994 | loss: 0.0310895 | val_loss: 0.0311653 | Time: 19278.9 ms [2021-12-18 14:15:11 main:574] : INFO : Epoch 1995 | loss: 0.0311013 | val_loss: 0.0311543 | Time: 19754.9 ms [2021-12-18 14:15:30 main:574] : INFO : Epoch 1996 | loss: 0.031112 | val_loss: 0.0311686 | Time: 19411.3 ms [2021-12-18 14:15:50 main:574] : INFO : Epoch 1997 | loss: 0.0311043 | val_loss: 0.0311588 | Time: 19442.7 ms [2021-12-18 14:16:10 main:574] : INFO : Epoch 1998 | loss: 0.0310973 | val_loss: 0.0311702 | Time: 19943.5 ms [2021-12-18 14:16:29 main:574] : INFO : Epoch 1999 | loss: 0.0310958 | val_loss: 0.0311653 | Time: 18821.6 ms [2021-12-18 14:16:49 main:574] : INFO : Epoch 2000 | loss: 0.0310921 | val_loss: 0.031157 | Time: 19975.1 ms [2021-12-18 14:17:09 main:574] : INFO : Epoch 2001 | loss: 0.0310934 | val_loss: 0.0311775 | Time: 19529.9 ms [2021-12-18 14:17:27 main:574] : INFO : Epoch 2002 | loss: 0.0310941 | val_loss: 0.0311669 | Time: 18476.8 ms [2021-12-18 14:17:46 main:574] : INFO : Epoch 2003 | loss: 0.031099 | val_loss: 0.0311551 | Time: 18326.7 ms [2021-12-18 14:18:05 main:574] : INFO : Epoch 2004 | loss: 0.0310965 | val_loss: 0.0311624 | Time: 19248.3 ms [2021-12-18 14:18:25 main:574] : INFO : Epoch 2005 | loss: 0.0310943 | val_loss: 0.0311646 | Time: 19750.7 ms [2021-12-18 14:18:44 main:574] : INFO : Epoch 2006 | loss: 0.0310945 | val_loss: 0.031159 | Time: 19308.5 ms [2021-12-18 14:19:04 main:574] : INFO : Epoch 2007 | loss: 0.0310904 | val_loss: 0.031167 | Time: 19197.5 ms [2021-12-18 14:19:22 main:574] : INFO : Epoch 2008 | loss: 0.0310942 | val_loss: 0.0311638 | Time: 18746.3 ms [2021-12-18 14:19:42 main:574] : INFO : Epoch 2009 | loss: 0.0310956 | val_loss: 0.0311592 | Time: 19482.8 ms [2021-12-18 14:20:01 main:574] : INFO : Epoch 2010 | loss: 0.0310956 | val_loss: 0.031171 | Time: 19187.5 ms [2021-12-18 14:20:21 main:574] : INFO : Epoch 2011 | loss: 0.0310909 | val_loss: 0.0311621 | Time: 19423.9 ms [2021-12-18 14:20:41 main:574] : INFO : Epoch 2012 | loss: 0.0310895 | val_loss: 0.0311671 | Time: 19788.4 ms [2021-12-18 14:21:00 main:574] : INFO : Epoch 2013 | loss: 0.0310916 | val_loss: 0.0311687 | Time: 19074.1 ms [2021-12-18 14:21:19 main:574] : INFO : Epoch 2014 | loss: 0.0310895 | val_loss: 0.0311698 | Time: 19474.2 ms [2021-12-18 14:21:39 main:574] : INFO : Epoch 2015 | loss: 0.0310914 | val_loss: 0.031165 | Time: 19283.1 ms [2021-12-18 14:21:57 main:574] : INFO : Epoch 2016 | loss: 0.0310951 | val_loss: 0.0311545 | Time: 18522.4 ms [2021-12-18 14:22:17 main:574] : INFO : Epoch 2017 | loss: 0.0310941 | val_loss: 0.0311651 | Time: 19601.6 ms [2021-12-18 14:22:37 main:574] : INFO : Epoch 2018 | loss: 0.0310958 | val_loss: 0.031155 | Time: 20135.5 ms [2021-12-18 14:22:57 main:574] : INFO : Epoch 2019 | loss: 0.031095 | val_loss: 0.0311609 | Time: 19417.2 ms [2021-12-18 14:23:17 main:574] : INFO : Epoch 2020 | loss: 0.0310918 | val_loss: 0.0311661 | Time: 20041.7 ms [2021-12-18 14:23:37 main:574] : INFO : Epoch 2021 | loss: 0.0311215 | val_loss: 0.0311608 | Time: 19962.8 ms [2021-12-18 14:23:56 main:574] : INFO : Epoch 2022 | loss: 0.0311203 | val_loss: 0.0311535 | Time: 18838 ms [2021-12-18 14:24:16 main:574] : INFO : Epoch 2023 | loss: 0.0311129 | val_loss: 0.0311601 | Time: 19940.2 ms [2021-12-18 14:24:36 main:574] : INFO : Epoch 2024 | loss: 0.031106 | val_loss: 0.0311589 | Time: 19804.5 ms [2021-12-18 14:24:55 main:574] : INFO : Epoch 2025 | loss: 0.0311048 | val_loss: 0.0311691 | Time: 18884.6 ms [2021-12-18 14:25:14 main:574] : INFO : Epoch 2026 | loss: 0.031104 | val_loss: 0.0311664 | Time: 18809.6 ms [2021-12-18 14:25:33 main:574] : INFO : Epoch 2027 | loss: 0.0311016 | val_loss: 0.0311628 | Time: 19086.2 ms [2021-12-18 14:25:53 main:574] : INFO : Epoch 2028 | loss: 0.0311088 | val_loss: 0.0311572 | Time: 19122.4 ms [2021-12-18 14:26:12 main:574] : INFO : Epoch 2029 | loss: 0.0311076 | val_loss: 0.0311582 | Time: 19306.1 ms [2021-12-18 14:26:31 main:574] : INFO : Epoch 2030 | loss: 0.0311243 | val_loss: 0.0312095 | Time: 19212 ms [2021-12-18 14:26:50 main:574] : INFO : Epoch 2031 | loss: 0.0317638 | val_loss: 0.0320252 | Time: 18365.5 ms [2021-12-18 14:27:09 main:574] : INFO : Epoch 2032 | loss: 0.0314197 | val_loss: 0.0312459 | Time: 19361.3 ms [2021-12-18 14:27:29 main:574] : INFO : Epoch 2033 | loss: 0.031172 | val_loss: 0.0311574 | Time: 19334.4 ms [2021-12-18 14:27:47 main:574] : INFO : Epoch 2034 | loss: 0.0311369 | val_loss: 0.0311579 | Time: 18569.7 ms [2021-12-18 14:28:08 main:574] : INFO : Epoch 2035 | loss: 0.0311243 | val_loss: 0.0311473 | Time: 20369.5 ms [2021-12-18 14:28:28 main:574] : INFO : Epoch 2036 | loss: 0.0311198 | val_loss: 0.0311538 | Time: 19467.4 ms [2021-12-18 14:28:47 main:574] : INFO : Epoch 2037 | loss: 0.0311329 | val_loss: 0.0311496 | Time: 19776.3 ms [2021-12-18 14:29:08 main:574] : INFO : Epoch 2038 | loss: 0.0311255 | val_loss: 0.0311472 | Time: 20084 ms [2021-12-18 14:29:27 main:574] : INFO : Epoch 2039 | loss: 0.0311225 | val_loss: 0.0311492 | Time: 18698.1 ms [2021-12-18 14:29:45 main:574] : INFO : Epoch 2040 | loss: 0.0311207 | val_loss: 0.0311477 | Time: 18146.5 ms [2021-12-18 14:30:04 main:574] : INFO : Epoch 2041 | loss: 0.0311181 | val_loss: 0.0311475 | Time: 18798.3 ms [2021-12-18 14:30:23 main:574] : INFO : Epoch 2042 | loss: 0.0311157 | val_loss: 0.0311446 | Time: 18842.8 ms [2021-12-18 14:30:42 main:574] : INFO : Epoch 2043 | loss: 0.031114 | val_loss: 0.0311507 | Time: 19182.7 ms [2021-12-18 14:31:01 main:574] : INFO : Epoch 2044 | loss: 0.0311144 | val_loss: 0.0311457 | Time: 19349 ms [2021-12-18 14:31:21 main:574] : INFO : Epoch 2045 | loss: 0.0311099 | val_loss: 0.0311599 | Time: 19222 ms [2021-12-18 14:31:40 main:574] : INFO : Epoch 2046 | loss: 0.0311106 | val_loss: 0.0311469 | Time: 19608.6 ms [2021-12-18 14:31:59 main:574] : INFO : Epoch 2047 | loss: 0.0311115 | val_loss: 0.0311509 | Time: 19029.8 ms [2021-12-18 14:32:19 main:574] : INFO : Epoch 2048 | loss: 0.0311188 | val_loss: 0.0311564 | Time: 19597.2 ms [2021-12-18 14:32:19 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311564 [2021-12-18 14:32:20 main:603] : INFO : Saving end state to config to file [2021-12-18 14:32:20 main:608] : INFO : Success, exiting.. 14:32:20 (4144): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)