Task 14618048
| Name | ParityModified-1647046517-4117-3-0_0 |
| Workunit | 11638746 |
| Created | 12 Apr 2022, 7:25:30 UTC |
| Sent | 12 Apr 2022, 7:25:35 UTC |
| Report deadline | 20 Apr 2022, 7:25:35 UTC |
| Received | 21 Apr 2022, 15:32:38 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 13966 |
| Run time | 2 hours 32 min 22 sec |
| CPU time | 2 hours 28 min |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 10,956.98 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.37 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> s: 0.0312071 | Time: 4350.58 ms [2022-04-22 00:39:55 main:574] : INFO : Epoch 1821 | loss: 0.031208 | val_loss: 0.0312078 | Time: 4312.88 ms [2022-04-22 00:39:59 main:574] : INFO : Epoch 1822 | loss: 0.0312086 | val_loss: 0.031207 | Time: 4330.76 ms [2022-04-22 00:40:04 main:574] : INFO : Epoch 1823 | loss: 0.0312075 | val_loss: 0.0312067 | Time: 4374.62 ms [2022-04-22 00:40:08 main:574] : INFO : Epoch 1824 | loss: 0.0312072 | val_loss: 0.0312069 | Time: 4392.54 ms [2022-04-22 00:40:12 main:574] : INFO : Epoch 1825 | loss: 0.0312063 | val_loss: 0.0312101 | Time: 4290.25 ms [2022-04-22 00:40:17 main:574] : INFO : Epoch 1826 | loss: 0.0312083 | val_loss: 0.0312061 | Time: 4380.11 ms [2022-04-22 00:40:21 main:574] : INFO : Epoch 1827 | loss: 0.0312053 | val_loss: 0.0312052 | Time: 4323.52 ms [2022-04-22 00:40:25 main:574] : INFO : Epoch 1828 | loss: 0.0312059 | val_loss: 0.031207 | Time: 4379.36 ms [2022-04-22 00:40:30 main:574] : INFO : Epoch 1829 | loss: 0.0312053 | val_loss: 0.0312056 | Time: 4377.75 ms [2022-04-22 00:40:34 main:574] : INFO : Epoch 1830 | loss: 0.0312051 | val_loss: 0.0312057 | Time: 4299.07 ms [2022-04-22 00:40:39 main:574] : INFO : Epoch 1831 | loss: 0.031205 | val_loss: 0.031206 | Time: 4373.69 ms [2022-04-22 00:40:43 main:574] : INFO : Epoch 1832 | loss: 0.0312064 | val_loss: 0.0312058 | Time: 4312.99 ms [2022-04-22 00:40:47 main:574] : INFO : Epoch 1833 | loss: 0.0312071 | val_loss: 0.0312062 | Time: 4361.6 ms [2022-04-22 00:40:52 main:574] : INFO : Epoch 1834 | loss: 0.0312069 | val_loss: 0.0312049 | Time: 4252.37 ms [2022-04-22 00:40:56 main:574] : INFO : Epoch 1835 | loss: 0.0312061 | val_loss: 0.0312056 | Time: 4307.32 ms [2022-04-22 00:41:00 main:574] : INFO : Epoch 1836 | loss: 0.0312059 | val_loss: 0.0312063 | Time: 4287.03 ms [2022-04-22 00:41:05 main:574] : INFO : Epoch 1837 | loss: 0.0312069 | val_loss: 0.0312079 | Time: 4384.58 ms [2022-04-22 00:41:09 main:574] : INFO : Epoch 1838 | loss: 0.0312065 | val_loss: 0.0312063 | Time: 4328.55 ms [2022-04-22 00:41:13 main:574] : INFO : Epoch 1839 | loss: 0.0312053 | val_loss: 0.031205 | Time: 4368.32 ms [2022-04-22 00:41:18 main:574] : INFO : Epoch 1840 | loss: 0.0312051 | val_loss: 0.0312048 | Time: 4350.17 ms [2022-04-22 00:41:22 main:574] : INFO : Epoch 1841 | loss: 0.0312058 | val_loss: 0.0312055 | Time: 4377.66 ms [2022-04-22 00:41:26 main:574] : INFO : Epoch 1842 | loss: 0.0312044 | val_loss: 0.0312043 | Time: 4327.59 ms [2022-04-22 00:41:31 main:574] : INFO : Epoch 1843 | loss: 0.0312046 | val_loss: 0.0312064 | Time: 4364.02 ms [2022-04-22 00:41:35 main:574] : INFO : Epoch 1844 | loss: 0.0312046 | val_loss: 0.0312053 | Time: 4288.92 ms [2022-04-22 00:41:39 main:574] : INFO : Epoch 1845 | loss: 0.0312048 | val_loss: 0.0312047 | Time: 4371.83 ms [2022-04-22 00:41:44 main:574] : INFO : Epoch 1846 | loss: 0.0312054 | val_loss: 0.0312042 | Time: 4371.24 ms [2022-04-22 00:41:48 main:574] : INFO : Epoch 1847 | loss: 0.0312044 | val_loss: 0.0312048 | Time: 4352.99 ms [2022-04-22 00:41:52 main:574] : INFO : Epoch 1848 | loss: 0.0312049 | val_loss: 0.0312094 | Time: 4338.75 ms [2022-04-22 00:41:57 main:574] : INFO : Epoch 1849 | loss: 0.0312068 | val_loss: 0.031205 | Time: 4306.79 ms [2022-04-22 00:42:01 main:574] : INFO : Epoch 1850 | loss: 0.0312041 | val_loss: 0.0312045 | Time: 4377.2 ms [2022-04-22 00:42:05 main:574] : INFO : Epoch 1851 | loss: 0.031204 | val_loss: 0.0312051 | Time: 4341.71 ms [2022-04-22 00:42:10 main:574] : INFO : Epoch 1852 | loss: 0.0312043 | val_loss: 0.0312051 | Time: 4295.55 ms [2022-04-22 00:42:14 main:574] : INFO : Epoch 1853 | loss: 0.0312045 | val_loss: 0.0312095 | Time: 4257.81 ms [2022-04-22 00:42:18 main:574] : INFO : Epoch 1854 | loss: 0.0312058 | val_loss: 0.0312052 | Time: 4345.1 ms [2022-04-22 00:42:23 main:574] : INFO : Epoch 1855 | loss: 0.0312051 | val_loss: 0.0312042 | Time: 4313.75 ms [2022-04-22 00:42:27 main:574] : INFO : Epoch 1856 | loss: 0.0312064 | val_loss: 0.0312107 | Time: 4329.75 ms [2022-04-22 00:42:31 main:574] : INFO : Epoch 1857 | loss: 0.0312049 | val_loss: 0.0312045 | Time: 4390.76 ms [2022-04-22 00:42:36 main:574] : INFO : Epoch 1858 | loss: 0.0312046 | val_loss: 0.0312048 | Time: 4290.41 ms [2022-04-22 00:42:40 main:574] : INFO : Epoch 1859 | loss: 0.0312041 | val_loss: 0.0312042 | Time: 4303.1 ms [2022-04-22 00:42:44 main:574] : INFO : Epoch 1860 | loss: 0.0312036 | val_loss: 0.0312066 | Time: 4350.35 ms [2022-04-22 00:42:49 main:574] : INFO : Epoch 1861 | loss: 0.0312049 | val_loss: 0.0312091 | Time: 4350.5 ms [2022-04-22 00:42:53 main:574] : INFO : Epoch 1862 | loss: 0.0312049 | val_loss: 0.0312068 | Time: 4312.62 ms [2022-04-22 00:42:57 main:574] : INFO : Epoch 1863 | loss: 0.0312034 | val_loss: 0.0312038 | Time: 4339.61 ms [2022-04-22 00:43:02 main:574] : INFO : Epoch 1864 | loss: 0.0312047 | val_loss: 0.0312059 | Time: 4402.45 ms [2022-04-22 00:43:06 main:574] : INFO : Epoch 1865 | loss: 0.0312064 | val_loss: 0.031214 | Time: 4327.16 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 00:46:07 main:435] : INFO : Set logging level to 1 [2022-04-22 00:46:07 main:441] : INFO : Running in BOINC Client mode [2022-04-22 00:46:07 main:444] : INFO : Resolving all filenames [2022-04-22 00:46:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 00:46:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 00:46:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 00:46:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 00:46:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 00:46:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 00:46:07 main:474] : INFO : Configuration: [2022-04-22 00:46:07 main:475] : INFO : Model type: GRU [2022-04-22 00:46:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 00:46:07 main:477] : INFO : Max Epochs: 2048 [2022-04-22 00:46:07 main:478] : INFO : Batch Size: 128 [2022-04-22 00:46:07 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 00:46:07 main:480] : INFO : Patience: 10 [2022-04-22 00:46:07 main:481] : INFO : Hidden Width: 12 [2022-04-22 00:46:07 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 00:46:07 main:483] : INFO : # Backend Layers: 4 [2022-04-22 00:46:07 main:484] : INFO : # Threads: 1 [2022-04-22 00:46:07 main:486] : INFO : Preparing Dataset [2022-04-22 00:46:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 00:46:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 00:46:09 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 00:46:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 00:46:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 00:46:09 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 00:46:09 main:494] : INFO : Creating Model [2022-04-22 00:46:09 main:507] : INFO : Preparing config file [2022-04-22 00:46:09 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 00:46:09 main:512] : INFO : Loading config [2022-04-22 00:46:09 main:514] : INFO : Loading state [2022-04-22 00:46:10 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 00:46:11 main:562] : INFO : Starting Training [2022-04-22 00:46:15 main:574] : INFO : Epoch 1854 | loss: 0.0312277 | val_loss: 0.0312066 | Time: 4715.19 ms [2022-04-22 00:46:20 main:574] : INFO : Epoch 1855 | loss: 0.0312077 | val_loss: 0.0312068 | Time: 4445.19 ms [2022-04-22 00:46:24 main:574] : INFO : Epoch 1856 | loss: 0.0312057 | val_loss: 0.0312063 | Time: 4331.15 ms [2022-04-22 00:46:28 main:574] : INFO : Epoch 1857 | loss: 0.0312044 | val_loss: 0.0312081 | Time: 4350.37 ms [2022-04-22 00:46:33 main:574] : INFO : Epoch 1858 | loss: 0.0312036 | val_loss: 0.0312072 | Time: 4443.28 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 00:49:33 main:435] : INFO : Set logging level to 1 [2022-04-22 00:49:33 main:441] : INFO : Running in BOINC Client mode [2022-04-22 00:49:33 main:444] : INFO : Resolving all filenames [2022-04-22 00:49:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 00:49:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 00:49:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 00:49:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 00:49:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 00:49:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 00:49:33 main:474] : INFO : Configuration: [2022-04-22 00:49:33 main:475] : INFO : Model type: GRU [2022-04-22 00:49:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 00:49:33 main:477] : INFO : Max Epochs: 2048 [2022-04-22 00:49:33 main:478] : INFO : Batch Size: 128 [2022-04-22 00:49:33 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 00:49:33 main:480] : INFO : Patience: 10 [2022-04-22 00:49:33 main:481] : INFO : Hidden Width: 12 [2022-04-22 00:49:33 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 00:49:33 main:483] : INFO : # Backend Layers: 4 [2022-04-22 00:49:33 main:484] : INFO : # Threads: 1 [2022-04-22 00:49:33 main:486] : INFO : Preparing Dataset [2022-04-22 00:49:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 00:49:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 00:49:35 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 00:49:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 00:49:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 00:49:35 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 00:49:35 main:494] : INFO : Creating Model [2022-04-22 00:49:35 main:507] : INFO : Preparing config file [2022-04-22 00:49:35 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 00:49:35 main:512] : INFO : Loading config [2022-04-22 00:49:35 main:514] : INFO : Loading state [2022-04-22 00:49:37 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 00:49:37 main:562] : INFO : Starting Training [2022-04-22 00:49:41 main:574] : INFO : Epoch 1854 | loss: 0.0312274 | val_loss: 0.0312085 | Time: 4728.14 ms [2022-04-22 00:49:46 main:574] : INFO : Epoch 1855 | loss: 0.0312074 | val_loss: 0.0312067 | Time: 4497.75 ms [2022-04-22 00:49:50 main:574] : INFO : Epoch 1856 | loss: 0.0312054 | val_loss: 0.0312071 | Time: 4527.54 ms [2022-04-22 00:49:55 main:574] : INFO : Epoch 1857 | loss: 0.031206 | val_loss: 0.0312051 | Time: 4390.67 ms [2022-04-22 00:49:59 main:574] : INFO : Epoch 1858 | loss: 0.0312074 | val_loss: 0.0312083 | Time: 4453.7 ms [2022-04-22 00:50:04 main:574] : INFO : Epoch 1859 | loss: 0.0312085 | val_loss: 0.0312087 | Time: 4474.61 ms [2022-04-22 00:50:08 main:574] : INFO : Epoch 1860 | loss: 0.0312078 | val_loss: 0.0312076 | Time: 4399.26 ms [2022-04-22 00:50:12 main:574] : INFO : Epoch 1861 | loss: 0.0312077 | val_loss: 0.0312081 | Time: 4346.61 ms [2022-04-22 00:50:17 main:574] : INFO : Epoch 1862 | loss: 0.0312069 | val_loss: 0.0312096 | Time: 4387.34 ms [2022-04-22 00:50:21 main:574] : INFO : Epoch 1863 | loss: 0.0312088 | val_loss: 0.0312085 | Time: 4382.53 ms [2022-04-22 00:50:26 main:574] : INFO : Epoch 1864 | loss: 0.0312087 | val_loss: 0.0312087 | Time: 4484.47 ms [2022-04-22 00:50:30 main:574] : INFO : Epoch 1865 | loss: 0.0312078 | val_loss: 0.0312095 | Time: 4471.47 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 00:53:34 main:435] : INFO : Set logging level to 1 [2022-04-22 00:53:34 main:441] : INFO : Running in BOINC Client mode [2022-04-22 00:53:34 main:444] : INFO : Resolving all filenames [2022-04-22 00:53:34 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 00:53:34 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 00:53:34 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 00:53:34 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 00:53:34 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 00:53:34 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 00:53:34 main:474] : INFO : Configuration: [2022-04-22 00:53:34 main:475] : INFO : Model type: GRU [2022-04-22 00:53:34 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 00:53:34 main:477] : INFO : Max Epochs: 2048 [2022-04-22 00:53:34 main:478] : INFO : Batch Size: 128 [2022-04-22 00:53:34 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 00:53:34 main:480] : INFO : Patience: 10 [2022-04-22 00:53:34 main:481] : INFO : Hidden Width: 12 [2022-04-22 00:53:34 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 00:53:34 main:483] : INFO : # Backend Layers: 4 [2022-04-22 00:53:34 main:484] : INFO : # Threads: 1 [2022-04-22 00:53:34 main:486] : INFO : Preparing Dataset [2022-04-22 00:53:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 00:53:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 00:53:36 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 00:53:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 00:53:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 00:53:36 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 00:53:36 main:494] : INFO : Creating Model [2022-04-22 00:53:36 main:507] : INFO : Preparing config file [2022-04-22 00:53:36 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 00:53:36 main:512] : INFO : Loading config [2022-04-22 00:53:36 main:514] : INFO : Loading state [2022-04-22 00:53:37 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 00:53:37 main:562] : INFO : Starting Training [2022-04-22 00:53:42 main:574] : INFO : Epoch 1854 | loss: 0.0312297 | val_loss: 0.0312121 | Time: 4723.24 ms [2022-04-22 00:53:46 main:574] : INFO : Epoch 1855 | loss: 0.0312082 | val_loss: 0.0312067 | Time: 4350.48 ms [2022-04-22 00:53:51 main:574] : INFO : Epoch 1856 | loss: 0.0312055 | val_loss: 0.0312065 | Time: 4341.39 ms [2022-04-22 00:53:55 main:574] : INFO : Epoch 1857 | loss: 0.0312046 | val_loss: 0.031206 | Time: 4280.61 ms [2022-04-22 00:53:59 main:574] : INFO : Epoch 1858 | loss: 0.0312049 | val_loss: 0.0312068 | Time: 4287.33 ms [2022-04-22 00:54:04 main:574] : INFO : Epoch 1859 | loss: 0.0312057 | val_loss: 0.0312057 | Time: 4424.87 ms [2022-04-22 00:54:08 main:574] : INFO : Epoch 1860 | loss: 0.0312083 | val_loss: 0.0312128 | Time: 4317.29 ms [2022-04-22 00:54:12 main:574] : INFO : Epoch 1861 | loss: 0.0312092 | val_loss: 0.0312101 | Time: 4347.78 ms [2022-04-22 00:54:17 main:574] : INFO : Epoch 1862 | loss: 0.0312112 | val_loss: 0.0312119 | Time: 4369.92 ms [2022-04-22 00:54:21 main:574] : INFO : Epoch 1863 | loss: 0.0312106 | val_loss: 0.031215 | Time: 4374.44 ms [2022-04-22 00:54:25 main:574] : INFO : Epoch 1864 | loss: 0.0312092 | val_loss: 0.031212 | Time: 4314.12 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 01:01:25 main:435] : INFO : Set logging level to 1 [2022-04-22 01:01:25 main:441] : INFO : Running in BOINC Client mode [2022-04-22 01:01:25 main:444] : INFO : Resolving all filenames [2022-04-22 01:01:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 01:01:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 01:01:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 01:01:25 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 01:01:25 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 01:01:25 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 01:01:25 main:474] : INFO : Configuration: [2022-04-22 01:01:25 main:475] : INFO : Model type: GRU [2022-04-22 01:01:25 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 01:01:25 main:477] : INFO : Max Epochs: 2048 [2022-04-22 01:01:25 main:478] : INFO : Batch Size: 128 [2022-04-22 01:01:25 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 01:01:25 main:480] : INFO : Patience: 10 [2022-04-22 01:01:25 main:481] : INFO : Hidden Width: 12 [2022-04-22 01:01:25 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 01:01:25 main:483] : INFO : # Backend Layers: 4 [2022-04-22 01:01:25 main:484] : INFO : # Threads: 1 [2022-04-22 01:01:25 main:486] : INFO : Preparing Dataset [2022-04-22 01:01:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 01:01:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 01:01:27 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 01:01:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 01:01:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 01:01:27 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 01:01:27 main:494] : INFO : Creating Model [2022-04-22 01:01:27 main:507] : INFO : Preparing config file [2022-04-22 01:01:27 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 01:01:27 main:512] : INFO : Loading config [2022-04-22 01:01:27 main:514] : INFO : Loading state [2022-04-22 01:01:28 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 01:01:28 main:562] : INFO : Starting Training [2022-04-22 01:01:32 main:574] : INFO : Epoch 1854 | loss: 0.0312293 | val_loss: 0.0312087 | Time: 4651.04 ms [2022-04-22 01:01:37 main:574] : INFO : Epoch 1855 | loss: 0.0312096 | val_loss: 0.0312058 | Time: 4336.72 ms [2022-04-22 01:01:41 main:574] : INFO : Epoch 1856 | loss: 0.0312063 | val_loss: 0.0312061 | Time: 4391.69 ms [2022-04-22 01:01:46 main:574] : INFO : Epoch 1857 | loss: 0.0312051 | val_loss: 0.0312046 | Time: 4374.98 ms [2022-04-22 01:01:50 main:574] : INFO : Epoch 1858 | loss: 0.0312062 | val_loss: 0.0312034 | Time: 4346.39 ms [2022-04-22 01:01:54 main:574] : INFO : Epoch 1859 | loss: 0.0312056 | val_loss: 0.0312048 | Time: 4326.75 ms [2022-04-22 01:01:59 main:574] : INFO : Epoch 1860 | loss: 0.0312046 | val_loss: 0.0312035 | Time: 4318.99 ms [2022-04-22 01:02:03 main:574] : INFO : Epoch 1861 | loss: 0.0312057 | val_loss: 0.0312093 | Time: 4322.48 ms [2022-04-22 01:02:07 main:574] : INFO : Epoch 1862 | loss: 0.0312059 | val_loss: 0.0312057 | Time: 4364.12 ms [2022-04-22 01:02:12 main:574] : INFO : Epoch 1863 | loss: 0.0312064 | val_loss: 0.0312058 | Time: 4357.63 ms [2022-04-22 01:02:16 main:574] : INFO : Epoch 1864 | loss: 0.0312057 | val_loss: 0.0312038 | Time: 4398.91 ms [2022-04-22 01:02:20 main:574] : INFO : Epoch 1865 | loss: 0.0312044 | val_loss: 0.0312081 | Time: 4380.05 ms [2022-04-22 01:02:25 main:574] : INFO : Epoch 1866 | loss: 0.0312044 | val_loss: 0.0312062 | Time: 4291.99 ms [2022-04-22 01:02:29 main:574] : INFO : Epoch 1867 | loss: 0.0312052 | val_loss: 0.031204 | Time: 4431.17 ms [2022-04-22 01:02:34 main:574] : INFO : Epoch 1868 | loss: 0.0312032 | val_loss: 0.0312034 | Time: 4397.72 ms [2022-04-22 01:02:38 main:574] : INFO : Epoch 1869 | loss: 0.031203 | val_loss: 0.0312044 | Time: 4411.3 ms [2022-04-22 01:02:42 main:574] : INFO : Epoch 1870 | loss: 0.0312045 | val_loss: 0.0312042 | Time: 4427.14 ms [2022-04-22 01:02:47 main:574] : INFO : Epoch 1871 | loss: 0.0312038 | val_loss: 0.0312078 | Time: 4399.99 ms [2022-04-22 01:02:51 main:574] : INFO : Epoch 1872 | loss: 0.0312087 | val_loss: 0.0312092 | Time: 4267.09 ms [2022-04-22 01:02:55 main:574] : INFO : Epoch 1873 | loss: 0.0312092 | val_loss: 0.0312089 | Time: 4265.71 ms [2022-04-22 01:03:00 main:574] : INFO : Epoch 1874 | loss: 0.0312082 | val_loss: 0.0312072 | Time: 4319.72 ms [2022-04-22 01:03:04 main:574] : INFO : Epoch 1875 | loss: 0.0312063 | val_loss: 0.0312068 | Time: 4347.97 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 01:07:27 main:435] : INFO : Set logging level to 1 [2022-04-22 01:07:27 main:441] : INFO : Running in BOINC Client mode [2022-04-22 01:07:27 main:444] : INFO : Resolving all filenames [2022-04-22 01:07:27 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 01:07:27 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 01:07:27 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 01:07:27 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 01:07:27 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 01:07:27 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 01:07:27 main:474] : INFO : Configuration: [2022-04-22 01:07:27 main:475] : INFO : Model type: GRU [2022-04-22 01:07:27 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 01:07:27 main:477] : INFO : Max Epochs: 2048 [2022-04-22 01:07:27 main:478] : INFO : Batch Size: 128 [2022-04-22 01:07:27 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 01:07:27 main:480] : INFO : Patience: 10 [2022-04-22 01:07:27 main:481] : INFO : Hidden Width: 12 [2022-04-22 01:07:27 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 01:07:27 main:483] : INFO : # Backend Layers: 4 [2022-04-22 01:07:27 main:484] : INFO : # Threads: 1 [2022-04-22 01:07:27 main:486] : INFO : Preparing Dataset [2022-04-22 01:07:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 01:07:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 01:07:29 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 01:07:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 01:07:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 01:07:29 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 01:07:29 main:494] : INFO : Creating Model [2022-04-22 01:07:29 main:507] : INFO : Preparing config file [2022-04-22 01:07:29 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 01:07:29 main:512] : INFO : Loading config [2022-04-22 01:07:29 main:514] : INFO : Loading state [2022-04-22 01:07:30 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 01:07:30 main:562] : INFO : Starting Training [2022-04-22 01:07:35 main:574] : INFO : Epoch 1869 | loss: 0.0312256 | val_loss: 0.0312082 | Time: 4725.89 ms [2022-04-22 01:07:40 main:574] : INFO : Epoch 1870 | loss: 0.0312069 | val_loss: 0.0312081 | Time: 4420.73 ms [2022-04-22 01:07:44 main:574] : INFO : Epoch 1871 | loss: 0.0312052 | val_loss: 0.0312071 | Time: 4438.55 ms [2022-04-22 01:07:48 main:574] : INFO : Epoch 1872 | loss: 0.031206 | val_loss: 0.0312072 | Time: 4369.46 ms [2022-04-22 01:07:53 main:574] : INFO : Epoch 1873 | loss: 0.0312059 | val_loss: 0.03121 | Time: 4440.3 ms [2022-04-22 01:07:57 main:574] : INFO : Epoch 1874 | loss: 0.031206 | val_loss: 0.0312069 | Time: 4359.54 ms [2022-04-22 01:08:02 main:574] : INFO : Epoch 1875 | loss: 0.0312071 | val_loss: 0.0312065 | Time: 4375.38 ms [2022-04-22 01:08:06 main:574] : INFO : Epoch 1876 | loss: 0.0312078 | val_loss: 0.0312071 | Time: 4349.67 ms [2022-04-22 01:08:10 main:574] : INFO : Epoch 1877 | loss: 0.031208 | val_loss: 0.0312072 | Time: 4374.02 ms [2022-04-22 01:08:15 main:574] : INFO : Epoch 1878 | loss: 0.0312093 | val_loss: 0.0312097 | Time: 4403.96 ms [2022-04-22 01:08:19 main:574] : INFO : Epoch 1879 | loss: 0.0312107 | val_loss: 0.0312105 | Time: 4369.18 ms [2022-04-22 01:08:23 main:574] : INFO : Epoch 1880 | loss: 0.0312113 | val_loss: 0.0312096 | Time: 4295.33 ms [2022-04-22 01:08:28 main:574] : INFO : Epoch 1881 | loss: 0.0312108 | val_loss: 0.0312093 | Time: 4377.18 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 01:11:33 main:435] : INFO : Set logging level to 1 [2022-04-22 01:11:33 main:441] : INFO : Running in BOINC Client mode [2022-04-22 01:11:33 main:444] : INFO : Resolving all filenames [2022-04-22 01:11:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 01:11:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 01:11:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 01:11:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 01:11:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 01:11:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 01:11:33 main:474] : INFO : Configuration: [2022-04-22 01:11:33 main:475] : INFO : Model type: GRU [2022-04-22 01:11:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 01:11:33 main:477] : INFO : Max Epochs: 2048 [2022-04-22 01:11:33 main:478] : INFO : Batch Size: 128 [2022-04-22 01:11:33 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 01:11:33 main:480] : INFO : Patience: 10 [2022-04-22 01:11:33 main:481] : INFO : Hidden Width: 12 [2022-04-22 01:11:33 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 01:11:33 main:483] : INFO : # Backend Layers: 4 [2022-04-22 01:11:33 main:484] : INFO : # Threads: 1 [2022-04-22 01:11:33 main:486] : INFO : Preparing Dataset [2022-04-22 01:11:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 01:11:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 01:11:35 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 01:11:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 01:11:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 01:11:35 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 01:11:35 main:494] : INFO : Creating Model [2022-04-22 01:11:35 main:507] : INFO : Preparing config file [2022-04-22 01:11:35 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 01:11:35 main:512] : INFO : Loading config [2022-04-22 01:11:35 main:514] : INFO : Loading state [2022-04-22 01:11:36 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 01:11:36 main:562] : INFO : Starting Training [2022-04-22 01:11:41 main:574] : INFO : Epoch 1869 | loss: 0.0312326 | val_loss: 0.0312091 | Time: 4733.53 ms [2022-04-22 01:11:46 main:574] : INFO : Epoch 1870 | loss: 0.031207 | val_loss: 0.0312045 | Time: 4672.51 ms [2022-04-22 01:11:50 main:574] : INFO : Epoch 1871 | loss: 0.0312035 | val_loss: 0.0312045 | Time: 4577.77 ms [2022-04-22 01:11:55 main:574] : INFO : Epoch 1872 | loss: 0.0312034 | val_loss: 0.0312027 | Time: 4494.49 ms [2022-04-22 01:12:00 main:574] : INFO : Epoch 1873 | loss: 0.0312028 | val_loss: 0.0312037 | Time: 4687.52 ms [2022-04-22 01:12:04 main:574] : INFO : Epoch 1874 | loss: 0.0312022 | val_loss: 0.0312029 | Time: 4627 ms [2022-04-22 01:12:09 main:574] : INFO : Epoch 1875 | loss: 0.0312027 | val_loss: 0.0312079 | Time: 4428.81 ms [2022-04-22 01:12:13 main:574] : INFO : Epoch 1876 | loss: 0.0312029 | val_loss: 0.0312037 | Time: 4397.18 ms [2022-04-22 01:12:18 main:574] : INFO : Epoch 1877 | loss: 0.031202 | val_loss: 0.031205 | Time: 4601.11 ms [2022-04-22 01:12:22 main:574] : INFO : Epoch 1878 | loss: 0.0312029 | val_loss: 0.0312036 | Time: 4543.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 01:15:24 main:435] : INFO : Set logging level to 1 [2022-04-22 01:15:24 main:441] : INFO : Running in BOINC Client mode [2022-04-22 01:15:24 main:444] : INFO : Resolving all filenames [2022-04-22 01:15:24 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 01:15:24 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 01:15:24 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 01:15:24 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 01:15:24 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 01:15:24 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 01:15:24 main:474] : INFO : Configuration: [2022-04-22 01:15:24 main:475] : INFO : Model type: GRU [2022-04-22 01:15:24 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 01:15:24 main:477] : INFO : Max Epochs: 2048 [2022-04-22 01:15:24 main:478] : INFO : Batch Size: 128 [2022-04-22 01:15:24 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 01:15:24 main:480] : INFO : Patience: 10 [2022-04-22 01:15:24 main:481] : INFO : Hidden Width: 12 [2022-04-22 01:15:24 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 01:15:24 main:483] : INFO : # Backend Layers: 4 [2022-04-22 01:15:24 main:484] : INFO : # Threads: 1 [2022-04-22 01:15:24 main:486] : INFO : Preparing Dataset [2022-04-22 01:15:24 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 01:15:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 01:15:26 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 01:15:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 01:15:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 01:15:26 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 01:15:26 main:494] : INFO : Creating Model [2022-04-22 01:15:26 main:507] : INFO : Preparing config file [2022-04-22 01:15:26 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 01:15:26 main:512] : INFO : Loading config [2022-04-22 01:15:27 main:514] : INFO : Loading state [2022-04-22 01:15:28 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 01:15:28 main:562] : INFO : Starting Training [2022-04-22 01:15:32 main:574] : INFO : Epoch 1869 | loss: 0.0312317 | val_loss: 0.0312057 | Time: 4761.86 ms [2022-04-22 01:15:37 main:574] : INFO : Epoch 1870 | loss: 0.0312065 | val_loss: 0.0312085 | Time: 4325.51 ms [2022-04-22 01:15:41 main:574] : INFO : Epoch 1871 | loss: 0.0312049 | val_loss: 0.0312062 | Time: 4456.8 ms [2022-04-22 01:15:46 main:574] : INFO : Epoch 1872 | loss: 0.0312034 | val_loss: 0.0312057 | Time: 4390.68 ms [2022-04-22 01:15:50 main:574] : INFO : Epoch 1873 | loss: 0.0312032 | val_loss: 0.0312065 | Time: 4382.17 ms [2022-04-22 01:15:54 main:574] : INFO : Epoch 1874 | loss: 0.0312032 | val_loss: 0.0312053 | Time: 4350.18 ms [2022-04-22 01:15:59 main:574] : INFO : Epoch 1875 | loss: 0.0312031 | val_loss: 0.0312048 | Time: 4374.79 ms [2022-04-22 01:16:03 main:574] : INFO : Epoch 1876 | loss: 0.0312047 | val_loss: 0.0312042 | Time: 4543.34 ms [2022-04-22 01:16:08 main:574] : INFO : Epoch 1877 | loss: 0.0312054 | val_loss: 0.0312068 | Time: 4477.43 ms [2022-04-22 01:16:12 main:574] : INFO : Epoch 1878 | loss: 0.0312058 | val_loss: 0.0312033 | Time: 4379.09 ms [2022-04-22 01:16:17 main:574] : INFO : Epoch 1879 | loss: 0.0312049 | val_loss: 0.0312039 | Time: 4453.89 ms [2022-04-22 01:16:21 main:574] : INFO : Epoch 1880 | loss: 0.0312036 | val_loss: 0.0312061 | Time: 4373.48 ms [2022-04-22 01:16:25 main:574] : INFO : Epoch 1881 | loss: 0.0312033 | val_loss: 0.0312026 | Time: 4413.33 ms [2022-04-22 01:16:30 main:574] : INFO : Epoch 1882 | loss: 0.0312011 | val_loss: 0.0312045 | Time: 4467.28 ms [2022-04-22 01:16:34 main:574] : INFO : Epoch 1883 | loss: 0.0312035 | val_loss: 0.0312036 | Time: 4365.48 ms [2022-04-22 01:16:39 main:574] : INFO : Epoch 1884 | loss: 0.0312032 | val_loss: 0.0312075 | Time: 4373.63 ms [2022-04-22 01:16:43 main:574] : INFO : Epoch 1885 | loss: 0.0312049 | val_loss: 0.0312066 | Time: 4433.86 ms [2022-04-22 01:16:48 main:574] : INFO : Epoch 1886 | loss: 0.0312013 | val_loss: 0.0312053 | Time: 4453.52 ms [2022-04-22 01:16:52 main:574] : INFO : Epoch 1887 | loss: 0.0312018 | val_loss: 0.0312031 | Time: 4415.94 ms [2022-04-22 01:16:56 main:574] : INFO : Epoch 1888 | loss: 0.031204 | val_loss: 0.0312101 | Time: 4430.1 ms [2022-04-22 01:17:01 main:574] : INFO : Epoch 1889 | loss: 0.0312255 | val_loss: 0.0312265 | Time: 4420.32 ms [2022-04-22 01:17:05 main:574] : INFO : Epoch 1890 | loss: 0.0312287 | val_loss: 0.0312309 | Time: 4538.14 ms [2022-04-22 01:17:10 main:574] : INFO : Epoch 1891 | loss: 0.0312291 | val_loss: 0.0312228 | Time: 4576.1 ms [2022-04-22 01:17:14 main:574] : INFO : Epoch 1892 | loss: 0.0312205 | val_loss: 0.031213 | Time: 4312.11 ms [2022-04-22 01:17:19 main:574] : INFO : Epoch 1893 | loss: 0.0312121 | val_loss: 0.0312129 | Time: 4376.92 ms [2022-04-22 01:17:23 main:574] : INFO : Epoch 1894 | loss: 0.0312133 | val_loss: 0.0312135 | Time: 4727.71 ms [2022-04-22 01:17:28 main:574] : INFO : Epoch 1895 | loss: 0.0312126 | val_loss: 0.0312115 | Time: 4446.15 ms [2022-04-22 01:17:32 main:574] : INFO : Epoch 1896 | loss: 0.0312118 | val_loss: 0.0312143 | Time: 4375.07 ms [2022-04-22 01:17:37 main:574] : INFO : Epoch 1897 | loss: 0.0312111 | val_loss: 0.031212 | Time: 4331.42 ms [2022-04-22 01:17:41 main:574] : INFO : Epoch 1898 | loss: 0.0312091 | val_loss: 0.031213 | Time: 4323.06 ms [2022-04-22 01:17:45 main:574] : INFO : Epoch 1899 | loss: 0.0312104 | val_loss: 0.0312107 | Time: 4282.72 ms [2022-04-22 01:17:50 main:574] : INFO : Epoch 1900 | loss: 0.0312096 | val_loss: 0.0312108 | Time: 4406.15 ms [2022-04-22 01:17:54 main:574] : INFO : Epoch 1901 | loss: 0.0312085 | val_loss: 0.0312103 | Time: 4396.5 ms [2022-04-22 01:17:58 main:574] : INFO : Epoch 1902 | loss: 0.031209 | val_loss: 0.0312134 | Time: 4269.66 ms [2022-04-22 01:18:03 main:574] : INFO : Epoch 1903 | loss: 0.031209 | val_loss: 0.0312101 | Time: 4360.57 ms [2022-04-22 01:18:07 main:574] : INFO : Epoch 1904 | loss: 0.0312081 | val_loss: 0.0312096 | Time: 4353.73 ms [2022-04-22 01:18:11 main:574] : INFO : Epoch 1905 | loss: 0.0312084 | val_loss: 0.0312093 | Time: 4361.12 ms [2022-04-22 01:18:16 main:574] : INFO : Epoch 1906 | loss: 0.0312081 | val_loss: 0.03121 | Time: 4348.54 ms [2022-04-22 01:18:20 main:574] : INFO : Epoch 1907 | loss: 0.0312078 | val_loss: 0.0312099 | Time: 4292.25 ms [2022-04-22 01:18:24 main:574] : INFO : Epoch 1908 | loss: 0.0312074 | val_loss: 0.0312089 | Time: 4315.51 ms [2022-04-22 01:18:29 main:574] : INFO : Epoch 1909 | loss: 0.031209 | val_loss: 0.0312091 | Time: 4383.37 ms [2022-04-22 01:18:33 main:574] : INFO : Epoch 1910 | loss: 0.0312102 | val_loss: 0.0312118 | Time: 4400.4 ms [2022-04-22 01:18:38 main:574] : INFO : Epoch 1911 | loss: 0.0312094 | val_loss: 0.0312104 | Time: 4352.19 ms [2022-04-22 01:18:42 main:574] : INFO : Epoch 1912 | loss: 0.0312105 | val_loss: 0.0312113 | Time: 4351.19 ms [2022-04-22 01:18:46 main:574] : INFO : Epoch 1913 | loss: 0.0312101 | val_loss: 0.03121 | Time: 4334.41 ms [2022-04-22 01:18:51 main:574] : INFO : Epoch 1914 | loss: 0.0312105 | val_loss: 0.0312107 | Time: 4367.64 ms [2022-04-22 01:18:55 main:574] : INFO : Epoch 1915 | loss: 0.0312113 | val_loss: 0.0312132 | Time: 4309.76 ms [2022-04-22 01:18:59 main:574] : INFO : Epoch 1916 | loss: 0.0312107 | val_loss: 0.0312113 | Time: 4342.85 ms [2022-04-22 01:19:04 main:574] : INFO : Epoch 1917 | loss: 0.0312122 | val_loss: 0.0312125 | Time: 4451.83 ms [2022-04-22 01:19:08 main:574] : INFO : Epoch 1918 | loss: 0.0312102 | val_loss: 0.031215 | Time: 4368.15 ms [2022-04-22 01:19:12 main:574] : INFO : Epoch 1919 | loss: 0.0312175 | val_loss: 0.0312183 | Time: 4370.86 ms [2022-04-22 01:19:17 main:574] : INFO : Epoch 1920 | loss: 0.0312229 | val_loss: 0.0312241 | Time: 4299.82 ms [2022-04-22 01:19:21 main:574] : INFO : Epoch 1921 | loss: 0.0312265 | val_loss: 0.0312231 | Time: 4298.52 ms [2022-04-22 01:19:25 main:574] : INFO : Epoch 1922 | loss: 0.0312251 | val_loss: 0.031224 | Time: 4322.25 ms [2022-04-22 01:19:30 main:574] : INFO : Epoch 1923 | loss: 0.0312229 | val_loss: 0.0312227 | Time: 4400.1 ms [2022-04-22 01:19:34 main:574] : INFO : Epoch 1924 | loss: 0.0312222 | val_loss: 0.0312222 | Time: 4679.93 ms [2022-04-22 01:19:39 main:574] : INFO : Epoch 1925 | loss: 0.0312226 | val_loss: 0.0312204 | Time: 4570.53 ms [2022-04-22 01:19:44 main:574] : INFO : Epoch 1926 | loss: 0.0312212 | val_loss: 0.0312207 | Time: 4622.45 ms [2022-04-22 01:19:48 main:574] : INFO : Epoch 1927 | loss: 0.0312222 | val_loss: 0.0312227 | Time: 4381.91 ms [2022-04-22 01:19:52 main:574] : INFO : Epoch 1928 | loss: 0.0312216 | val_loss: 0.0312211 | Time: 4391.94 ms [2022-04-22 01:19:57 main:574] : INFO : Epoch 1929 | loss: 0.0312218 | val_loss: 0.0312204 | Time: 4355.86 ms [2022-04-22 01:20:01 main:574] : INFO : Epoch 1930 | loss: 0.031221 | val_loss: 0.0312207 | Time: 4340.73 ms [2022-04-22 01:20:06 main:574] : INFO : Epoch 1931 | loss: 0.0312211 | val_loss: 0.0312205 | Time: 4352.76 ms [2022-04-22 01:20:10 main:574] : INFO : Epoch 1932 | loss: 0.0312213 | val_loss: 0.031224 | Time: 4264.71 ms [2022-04-22 01:20:14 main:574] : INFO : Epoch 1933 | loss: 0.0312215 | val_loss: 0.0312207 | Time: 4412.7 ms [2022-04-22 01:20:19 main:574] : INFO : Epoch 1934 | loss: 0.0312213 | val_loss: 0.0312203 | Time: 4301.64 ms [2022-04-22 01:20:23 main:574] : INFO : Epoch 1935 | loss: 0.0312208 | val_loss: 0.0312201 | Time: 4262.69 ms [2022-04-22 01:20:27 main:574] : INFO : Epoch 1936 | loss: 0.0312208 | val_loss: 0.0312201 | Time: 4363.44 ms [2022-04-22 01:20:32 main:574] : INFO : Epoch 1937 | loss: 0.0312211 | val_loss: 0.031223 | Time: 4384.86 ms [2022-04-22 01:20:36 main:574] : INFO : Epoch 1938 | loss: 0.0312219 | val_loss: 0.0312211 | Time: 4312.6 ms [2022-04-22 01:20:40 main:574] : INFO : Epoch 1939 | loss: 0.031221 | val_loss: 0.0312201 | Time: 4276.25 ms [2022-04-22 01:20:44 main:574] : INFO : Epoch 1940 | loss: 0.0312212 | val_loss: 0.0312208 | Time: 4285.23 ms [2022-04-22 01:20:49 main:574] : INFO : Epoch 1941 | loss: 0.0312212 | val_loss: 0.031224 | Time: 4343.99 ms [2022-04-22 01:20:53 main:574] : INFO : Epoch 1942 | loss: 0.0312238 | val_loss: 0.0312205 | Time: 4496.52 ms [2022-04-22 01:20:58 main:574] : INFO : Epoch 1943 | loss: 0.0312225 | val_loss: 0.0312268 | Time: 4585.54 ms [2022-04-22 01:21:02 main:574] : INFO : Epoch 1944 | loss: 0.0312229 | val_loss: 0.0312205 | Time: 4400.81 ms [2022-04-22 01:21:07 main:574] : INFO : Epoch 1945 | loss: 0.0312216 | val_loss: 0.0312215 | Time: 4392.48 ms [2022-04-22 01:21:11 main:574] : INFO : Epoch 1946 | loss: 0.0312213 | val_loss: 0.0312203 | Time: 4342.07 ms [2022-04-22 01:21:15 main:574] : INFO : Epoch 1947 | loss: 0.0312207 | val_loss: 0.0312206 | Time: 4283.13 ms [2022-04-22 01:21:20 main:574] : INFO : Epoch 1948 | loss: 0.0312207 | val_loss: 0.03122 | Time: 4373.55 ms [2022-04-22 01:21:24 main:574] : INFO : Epoch 1949 | loss: 0.031221 | val_loss: 0.0312201 | Time: 4359.94 ms [2022-04-22 01:21:28 main:574] : INFO : Epoch 1950 | loss: 0.0312222 | val_loss: 0.0312233 | Time: 4333.64 ms [2022-04-22 01:21:33 main:574] : INFO : Epoch 1951 | loss: 0.0312227 | val_loss: 0.0312199 | Time: 4345.77 ms [2022-04-22 01:21:37 main:574] : INFO : Epoch 1952 | loss: 0.0312214 | val_loss: 0.0312198 | Time: 4327.01 ms [2022-04-22 01:21:41 main:574] : INFO : Epoch 1953 | loss: 0.0312212 | val_loss: 0.0312213 | Time: 4323.53 ms [2022-04-22 01:21:46 main:574] : INFO : Epoch 1954 | loss: 0.0312205 | val_loss: 0.0312198 | Time: 4346.18 ms [2022-04-22 01:21:50 main:574] : INFO : Epoch 1955 | loss: 0.0312204 | val_loss: 0.0312199 | Time: 4305.57 ms [2022-04-22 01:21:54 main:574] : INFO : Epoch 1956 | loss: 0.0312203 | val_loss: 0.0312216 | Time: 4343.17 ms [2022-04-22 01:21:59 main:574] : INFO : Epoch 1957 | loss: 0.0312213 | val_loss: 0.0312198 | Time: 4309.62 ms [2022-04-22 01:22:03 main:574] : INFO : Epoch 1958 | loss: 0.0312209 | val_loss: 0.0312198 | Time: 4352.87 ms [2022-04-22 01:22:07 main:574] : INFO : Epoch 1959 | loss: 0.0312208 | val_loss: 0.031223 | Time: 4351.74 ms [2022-04-22 01:22:12 main:574] : INFO : Epoch 1960 | loss: 0.0312204 | val_loss: 0.0312195 | Time: 4327.01 ms [2022-04-22 01:22:16 main:574] : INFO : Epoch 1961 | loss: 0.0312207 | val_loss: 0.0312194 | Time: 4331.3 ms [2022-04-22 01:22:20 main:574] : INFO : Epoch 1962 | loss: 0.0312198 | val_loss: 0.03122 | Time: 4324.28 ms [2022-04-22 01:22:25 main:574] : INFO : Epoch 1963 | loss: 0.031221 | val_loss: 0.0312222 | Time: 4436.64 ms [2022-04-22 01:22:29 main:574] : INFO : Epoch 1964 | loss: 0.0312205 | val_loss: 0.0312194 | Time: 4530.82 ms [2022-04-22 01:22:34 main:574] : INFO : Epoch 1965 | loss: 0.0312204 | val_loss: 0.0312193 | Time: 4339.69 ms [2022-04-22 01:22:38 main:574] : INFO : Epoch 1966 | loss: 0.0312203 | val_loss: 0.0312199 | Time: 4337.33 ms [2022-04-22 01:22:43 main:574] : INFO : Epoch 1967 | loss: 0.0312201 | val_loss: 0.0312198 | Time: 4478.79 ms [2022-04-22 01:22:47 main:574] : INFO : Epoch 1968 | loss: 0.0312202 | val_loss: 0.0312193 | Time: 4409.48 ms [2022-04-22 01:22:51 main:574] : INFO : Epoch 1969 | loss: 0.0312213 | val_loss: 0.03122 | Time: 4391.84 ms [2022-04-22 01:22:56 main:574] : INFO : Epoch 1970 | loss: 0.0312204 | val_loss: 0.031221 | Time: 4341.33 ms [2022-04-22 01:23:00 main:574] : INFO : Epoch 1971 | loss: 0.0312202 | val_loss: 0.0312196 | Time: 4364.77 ms [2022-04-22 01:23:04 main:574] : INFO : Epoch 1972 | loss: 0.0312202 | val_loss: 0.0312193 | Time: 4324 ms [2022-04-22 01:23:09 main:574] : INFO : Epoch 1973 | loss: 0.0312206 | val_loss: 0.031219 | Time: 4437.1 ms [2022-04-22 01:23:13 main:574] : INFO : Epoch 1974 | loss: 0.0312196 | val_loss: 0.0312194 | Time: 4368.97 ms [2022-04-22 01:23:18 main:574] : INFO : Epoch 1975 | loss: 0.0312198 | val_loss: 0.0312203 | Time: 4319.19 ms [2022-04-22 01:23:22 main:574] : INFO : Epoch 1976 | loss: 0.0312203 | val_loss: 0.031221 | Time: 4299.82 ms [2022-04-22 01:23:26 main:574] : INFO : Epoch 1977 | loss: 0.0312215 | val_loss: 0.0312194 | Time: 4311.1 ms [2022-04-22 01:23:31 main:574] : INFO : Epoch 1978 | loss: 0.0312195 | val_loss: 0.0312194 | Time: 4419.39 ms [2022-04-22 01:23:35 main:574] : INFO : Epoch 1979 | loss: 0.0312207 | val_loss: 0.0312196 | Time: 4328.78 ms [2022-04-22 01:23:39 main:574] : INFO : Epoch 1980 | loss: 0.0312208 | val_loss: 0.031223 | Time: 4327.44 ms [2022-04-22 01:23:44 main:574] : INFO : Epoch 1981 | loss: 0.0312244 | val_loss: 0.0312194 | Time: 4284.78 ms [2022-04-22 01:23:48 main:574] : INFO : Epoch 1982 | loss: 0.0312202 | val_loss: 0.0312208 | Time: 4406.32 ms [2022-04-22 01:23:52 main:574] : INFO : Epoch 1983 | loss: 0.03122 | val_loss: 0.0312204 | Time: 4341.26 ms [2022-04-22 01:23:57 main:574] : INFO : Epoch 1984 | loss: 0.0312199 | val_loss: 0.0312223 | Time: 4327.91 ms [2022-04-22 01:24:01 main:574] : INFO : Epoch 1985 | loss: 0.0312198 | val_loss: 0.031219 | Time: 4327.86 ms [2022-04-22 01:24:05 main:574] : INFO : Epoch 1986 | loss: 0.0312197 | val_loss: 0.031219 | Time: 4432.76 ms [2022-04-22 01:24:10 main:574] : INFO : Epoch 1987 | loss: 0.0312197 | val_loss: 0.0312199 | Time: 4401.45 ms [2022-04-22 01:24:14 main:574] : INFO : Epoch 1988 | loss: 0.0312202 | val_loss: 0.0312199 | Time: 4357.74 ms [2022-04-22 01:24:19 main:574] : INFO : Epoch 1989 | loss: 0.0312202 | val_loss: 0.0312205 | Time: 4344.93 ms [2022-04-22 01:24:23 main:574] : INFO : Epoch 1990 | loss: 0.0312196 | val_loss: 0.0312198 | Time: 4392.82 ms [2022-04-22 01:24:27 main:574] : INFO : Epoch 1991 | loss: 0.0312194 | val_loss: 0.0312197 | Time: 4345.57 ms [2022-04-22 01:24:32 main:574] : INFO : Epoch 1992 | loss: 0.0312195 | val_loss: 0.0312196 | Time: 4393.92 ms [2022-04-22 01:24:36 main:574] : INFO : Epoch 1993 | loss: 0.0312199 | val_loss: 0.0312201 | Time: 4342.31 ms [2022-04-22 01:24:40 main:574] : INFO : Epoch 1994 | loss: 0.03122 | val_loss: 0.0312196 | Time: 4294.23 ms [2022-04-22 01:24:45 main:574] : INFO : Epoch 1995 | loss: 0.0312204 | val_loss: 0.0312196 | Time: 4391.05 ms [2022-04-22 01:24:49 main:574] : INFO : Epoch 1996 | loss: 0.0312205 | val_loss: 0.0312197 | Time: 4326.13 ms [2022-04-22 01:24:53 main:574] : INFO : Epoch 1997 | loss: 0.0312198 | val_loss: 0.0312197 | Time: 4354.42 ms [2022-04-22 01:24:58 main:574] : INFO : Epoch 1998 | loss: 0.0312198 | val_loss: 0.0312201 | Time: 4299.18 ms [2022-04-22 01:25:02 main:574] : INFO : Epoch 1999 | loss: 0.0312189 | val_loss: 0.0312209 | Time: 4312.01 ms [2022-04-22 01:25:06 main:574] : INFO : Epoch 2000 | loss: 0.0312193 | val_loss: 0.0312193 | Time: 4361.12 ms [2022-04-22 01:25:11 main:574] : INFO : Epoch 2001 | loss: 0.0312191 | val_loss: 0.0312205 | Time: 4311.49 ms [2022-04-22 01:25:15 main:574] : INFO : Epoch 2002 | loss: 0.0312188 | val_loss: 0.0312226 | Time: 4306.25 ms [2022-04-22 01:25:19 main:574] : INFO : Epoch 2003 | loss: 0.0312201 | val_loss: 0.0312213 | Time: 4335.63 ms [2022-04-22 01:25:24 main:574] : INFO : Epoch 2004 | loss: 0.0312208 | val_loss: 0.0312227 | Time: 4329.39 ms [2022-04-22 01:25:28 main:574] : INFO : Epoch 2005 | loss: 0.0312195 | val_loss: 0.0312191 | Time: 4347.48 ms [2022-04-22 01:25:33 main:574] : INFO : Epoch 2006 | loss: 0.0312184 | val_loss: 0.0312187 | Time: 4500.6 ms [2022-04-22 01:25:37 main:574] : INFO : Epoch 2007 | loss: 0.0312183 | val_loss: 0.031219 | Time: 4428.54 ms [2022-04-22 01:25:42 main:574] : INFO : Epoch 2008 | loss: 0.031219 | val_loss: 0.0312188 | Time: 4522.76 ms [2022-04-22 01:25:46 main:574] : INFO : Epoch 2009 | loss: 0.0312195 | val_loss: 0.0312194 | Time: 4405.51 ms [2022-04-22 01:25:50 main:574] : INFO : Epoch 2010 | loss: 0.0312194 | val_loss: 0.0312217 | Time: 4306.41 ms [2022-04-22 01:25:55 main:574] : INFO : Epoch 2011 | loss: 0.0312196 | val_loss: 0.031219 | Time: 4455.49 ms [2022-04-22 01:25:59 main:574] : INFO : Epoch 2012 | loss: 0.0312188 | val_loss: 0.031219 | Time: 4287.42 ms [2022-04-22 01:26:03 main:574] : INFO : Epoch 2013 | loss: 0.0312186 | val_loss: 0.03122 | Time: 4306.08 ms [2022-04-22 01:26:08 main:574] : INFO : Epoch 2014 | loss: 0.031219 | val_loss: 0.0312199 | Time: 4295.65 ms [2022-04-22 01:26:12 main:574] : INFO : Epoch 2015 | loss: 0.0312202 | val_loss: 0.0312229 | Time: 4304.61 ms [2022-04-22 01:26:16 main:574] : INFO : Epoch 2016 | loss: 0.0312229 | val_loss: 0.031219 | Time: 4384.51 ms [2022-04-22 01:26:21 main:574] : INFO : Epoch 2017 | loss: 0.0312173 | val_loss: 0.0312158 | Time: 4398.58 ms [2022-04-22 01:26:25 main:574] : INFO : Epoch 2018 | loss: 0.0312153 | val_loss: 0.0312145 | Time: 4294.56 ms [2022-04-22 01:26:30 main:574] : INFO : Epoch 2019 | loss: 0.0312151 | val_loss: 0.0312142 | Time: 4401.27 ms [2022-04-22 01:26:34 main:574] : INFO : Epoch 2020 | loss: 0.031215 | val_loss: 0.0312157 | Time: 4347.51 ms [2022-04-22 01:26:38 main:574] : INFO : Epoch 2021 | loss: 0.0312142 | val_loss: 0.031214 | Time: 4325.76 ms [2022-04-22 01:26:43 main:574] : INFO : Epoch 2022 | loss: 0.0312146 | val_loss: 0.031214 | Time: 4308.27 ms [2022-04-22 01:26:47 main:574] : INFO : Epoch 2023 | loss: 0.0312139 | val_loss: 0.0312162 | Time: 4348.5 ms [2022-04-22 01:26:51 main:574] : INFO : Epoch 2024 | loss: 0.0312153 | val_loss: 0.0312142 | Time: 4332.39 ms [2022-04-22 01:26:56 main:574] : INFO : Epoch 2025 | loss: 0.0312159 | val_loss: 0.031215 | Time: 4344.01 ms [2022-04-22 01:27:00 main:574] : INFO : Epoch 2026 | loss: 0.0312143 | val_loss: 0.0312142 | Time: 4303.02 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-22 01:30:04 main:435] : INFO : Set logging level to 1 [2022-04-22 01:30:04 main:441] : INFO : Running in BOINC Client mode [2022-04-22 01:30:04 main:444] : INFO : Resolving all filenames [2022-04-22 01:30:04 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-22 01:30:04 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-22 01:30:04 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-22 01:30:04 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-22 01:30:04 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-22 01:30:04 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-22 01:30:04 main:474] : INFO : Configuration: [2022-04-22 01:30:04 main:475] : INFO : Model type: GRU [2022-04-22 01:30:04 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-22 01:30:04 main:477] : INFO : Max Epochs: 2048 [2022-04-22 01:30:04 main:478] : INFO : Batch Size: 128 [2022-04-22 01:30:04 main:479] : INFO : Learning Rate: 0.01 [2022-04-22 01:30:04 main:480] : INFO : Patience: 10 [2022-04-22 01:30:04 main:481] : INFO : Hidden Width: 12 [2022-04-22 01:30:04 main:482] : INFO : # Recurrent Layers: 4 [2022-04-22 01:30:04 main:483] : INFO : # Backend Layers: 4 [2022-04-22 01:30:04 main:484] : INFO : # Threads: 1 [2022-04-22 01:30:04 main:486] : INFO : Preparing Dataset [2022-04-22 01:30:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-22 01:30:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-22 01:30:05 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-22 01:30:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-22 01:30:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-22 01:30:06 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-22 01:30:06 main:494] : INFO : Creating Model [2022-04-22 01:30:06 main:507] : INFO : Preparing config file [2022-04-22 01:30:06 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-22 01:30:06 main:512] : INFO : Loading config [2022-04-22 01:30:06 main:514] : INFO : Loading state [2022-04-22 01:30:07 main:559] : INFO : Loading DataLoader into Memory [2022-04-22 01:30:07 main:562] : INFO : Starting Training [2022-04-22 01:30:12 main:574] : INFO : Epoch 2018 | loss: 0.0312433 | val_loss: 0.0312197 | Time: 4778.9 ms [2022-04-22 01:30:16 main:574] : INFO : Epoch 2019 | loss: 0.0312185 | val_loss: 0.0312146 | Time: 4407.97 ms [2022-04-22 01:30:20 main:574] : INFO : Epoch 2020 | loss: 0.0312164 | val_loss: 0.0312167 | Time: 4345.94 ms [2022-04-22 01:30:25 main:574] : INFO : Epoch 2021 | loss: 0.0312158 | val_loss: 0.0312121 | Time: 4312.73 ms [2022-04-22 01:30:29 main:574] : INFO : Epoch 2022 | loss: 0.0312116 | val_loss: 0.0312106 | Time: 4439.19 ms [2022-04-22 01:30:33 main:574] : INFO : Epoch 2023 | loss: 0.0312111 | val_loss: 0.0312102 | Time: 4328.34 ms [2022-04-22 01:30:38 main:574] : INFO : Epoch 2024 | loss: 0.0312111 | val_loss: 0.0312088 | Time: 4349.17 ms [2022-04-22 01:30:42 main:574] : INFO : Epoch 2025 | loss: 0.0312107 | val_loss: 0.0312088 | Time: 4360.38 ms [2022-04-22 01:30:46 main:574] : INFO : Epoch 2026 | loss: 0.0312109 | val_loss: 0.0312093 | Time: 4305.54 ms [2022-04-22 01:30:51 main:574] : INFO : Epoch 2027 | loss: 0.0312118 | val_loss: 0.0312094 | Time: 4382.93 ms [2022-04-22 01:30:55 main:574] : INFO : Epoch 2028 | loss: 0.0312104 | val_loss: 0.0312092 | Time: 4337.06 ms [2022-04-22 01:30:59 main:574] : INFO : Epoch 2029 | loss: 0.0312105 | val_loss: 0.031209 | Time: 4335.97 ms [2022-04-22 01:31:04 main:574] : INFO : Epoch 2030 | loss: 0.0312103 | val_loss: 0.0312093 | Time: 4394.25 ms [2022-04-22 01:31:08 main:574] : INFO : Epoch 2031 | loss: 0.0312102 | val_loss: 0.0312103 | Time: 4356.37 ms [2022-04-22 01:31:13 main:574] : INFO : Epoch 2032 | loss: 0.0312114 | val_loss: 0.0312095 | Time: 4357.2 ms [2022-04-22 01:31:17 main:574] : INFO : Epoch 2033 | loss: 0.0312107 | val_loss: 0.0312093 | Time: 4441.88 ms [2022-04-22 01:31:21 main:574] : INFO : Epoch 2034 | loss: 0.0312094 | val_loss: 0.0312101 | Time: 4422.83 ms [2022-04-22 01:31:26 main:574] : INFO : Epoch 2035 | loss: 0.0312098 | val_loss: 0.0312089 | Time: 4333.02 ms [2022-04-22 01:31:30 main:574] : INFO : Epoch 2036 | loss: 0.0312098 | val_loss: 0.0312105 | Time: 4424.97 ms [2022-04-22 01:31:35 main:574] : INFO : Epoch 2037 | loss: 0.0312095 | val_loss: 0.0312105 | Time: 4384.98 ms [2022-04-22 01:31:39 main:574] : INFO : Epoch 2038 | loss: 0.03121 | val_loss: 0.0312096 | Time: 4375.18 ms [2022-04-22 01:31:43 main:574] : INFO : Epoch 2039 | loss: 0.0312091 | val_loss: 0.0312126 | Time: 4357.86 ms [2022-04-22 01:31:48 main:574] : INFO : Epoch 2040 | loss: 0.0312098 | val_loss: 0.0312089 | Time: 4364.76 ms [2022-04-22 01:31:52 main:574] : INFO : Epoch 2041 | loss: 0.0312089 | val_loss: 0.0312092 | Time: 4394.07 ms [2022-04-22 01:31:57 main:574] : INFO : Epoch 2042 | loss: 0.0312093 | val_loss: 0.0312093 | Time: 4453.49 ms [2022-04-22 01:32:01 main:574] : INFO : Epoch 2043 | loss: 0.0312099 | val_loss: 0.031211 | Time: 4630.42 ms [2022-04-22 01:32:06 main:574] : INFO : Epoch 2044 | loss: 0.0312112 | val_loss: 0.0312089 | Time: 4441.18 ms [2022-04-22 01:32:10 main:574] : INFO : Epoch 2045 | loss: 0.03121 | val_loss: 0.0312092 | Time: 4295.46 ms [2022-04-22 01:32:14 main:574] : INFO : Epoch 2046 | loss: 0.0312127 | val_loss: 0.0312108 | Time: 4287.23 ms [2022-04-22 01:32:19 main:574] : INFO : Epoch 2047 | loss: 0.0312112 | val_loss: 0.0312114 | Time: 4350.58 ms [2022-04-22 01:32:23 main:574] : INFO : Epoch 2048 | loss: 0.0312114 | val_loss: 0.0312116 | Time: 4358.54 ms [2022-04-22 01:32:23 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0312116 [2022-04-22 01:32:23 main:603] : INFO : Saving end state to config to file [2022-04-22 01:32:23 main:608] : INFO : Success, exiting.. 01:32:23 (52208): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)