Task 9661898
| Name | ParityModified-1638631104-28713-1-0_0 |
| Workunit | 7061813 |
| Created | 10 Dec 2021, 12:42:02 UTC |
| Sent | 15 Dec 2021, 15:26:25 UTC |
| Report deadline | 23 Dec 2021, 15:26:25 UTC |
| Received | 19 Dec 2021, 7:50:02 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 6741 |
| Run time | 6 hours 29 min 7 sec |
| CPU time | 4 hours 36 min 5 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 786.79 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.69 GB |
| Peak swap size | 3.44 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 14:37:22 main:435] : INFO : Set logging level to 1 [2021-12-19 14:37:22 main:441] : INFO : Running in BOINC Client mode [2021-12-19 14:37:22 main:444] : INFO : Resolving all filenames [2021-12-19 14:37:22 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 14:37:22 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 14:37:22 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 14:37:22 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 14:37:22 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 14:37:22 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 14:37:22 main:474] : INFO : Configuration: [2021-12-19 14:37:22 main:475] : INFO : Model type: GRU [2021-12-19 14:37:22 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 14:37:22 main:477] : INFO : Max Epochs: 2048 [2021-12-19 14:37:22 main:478] : INFO : Batch Size: 128 [2021-12-19 14:37:22 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 14:37:22 main:480] : INFO : Patience: 10 [2021-12-19 14:37:22 main:481] : INFO : Hidden Width: 12 [2021-12-19 14:37:22 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 14:37:22 main:483] : INFO : # Backend Layers: 4 [2021-12-19 14:37:22 main:484] : INFO : # Threads: 1 [2021-12-19 14:37:22 main:486] : INFO : Preparing Dataset [2021-12-19 14:37:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 14:37:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 14:37:25 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 14:37:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 14:37:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 14:37:26 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 14:37:26 main:494] : INFO : Creating Model [2021-12-19 14:37:26 main:507] : INFO : Preparing config file [2021-12-19 14:37:26 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 14:37:26 main:512] : INFO : Loading config [2021-12-19 14:37:26 main:514] : INFO : Loading state [2021-12-19 14:37:27 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 14:37:27 main:562] : INFO : Starting Training [2021-12-19 14:37:33 main:574] : INFO : Epoch 1786 | loss: 0.0310326 | val_loss: 0.030979 | Time: 6707.16 ms [2021-12-19 14:37:42 main:574] : INFO : Epoch 1787 | loss: 0.0309271 | val_loss: 0.0309713 | Time: 9061.73 ms [2021-12-19 14:38:03 main:574] : INFO : Epoch 1788 | loss: 0.0309134 | val_loss: 0.0309748 | Time: 20631.4 ms [2021-12-19 14:38:25 main:574] : INFO : Epoch 1789 | loss: 0.0309067 | val_loss: 0.0309659 | Time: 21430.6 ms [2021-12-19 14:38:44 main:574] : INFO : Epoch 1790 | loss: 0.030895 | val_loss: 0.0309573 | Time: 19664.8 ms [2021-12-19 14:39:05 main:574] : INFO : Epoch 1791 | loss: 0.0308981 | val_loss: 0.0309637 | Time: 20664.9 ms [2021-12-19 14:39:29 main:574] : INFO : Epoch 1792 | loss: 0.0308985 | val_loss: 0.0309571 | Time: 23727.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 14:42:39 main:435] : INFO : Set logging level to 1 [2021-12-19 14:42:39 main:441] : INFO : Running in BOINC Client mode [2021-12-19 14:42:39 main:444] : INFO : Resolving all filenames [2021-12-19 14:42:39 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 14:42:39 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 14:42:39 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 14:42:39 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 14:42:39 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 14:42:39 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 14:42:39 main:474] : INFO : Configuration: [2021-12-19 14:42:39 main:475] : INFO : Model type: GRU [2021-12-19 14:42:39 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 14:42:39 main:477] : INFO : Max Epochs: 2048 [2021-12-19 14:42:39 main:478] : INFO : Batch Size: 128 [2021-12-19 14:42:39 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 14:42:39 main:480] : INFO : Patience: 10 [2021-12-19 14:42:39 main:481] : INFO : Hidden Width: 12 [2021-12-19 14:42:39 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 14:42:39 main:483] : INFO : # Backend Layers: 4 [2021-12-19 14:42:39 main:484] : INFO : # Threads: 1 [2021-12-19 14:42:39 main:486] : INFO : Preparing Dataset [2021-12-19 14:42:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 14:42:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 14:42:43 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 14:42:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 14:42:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 14:42:43 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 14:42:43 main:494] : INFO : Creating Model [2021-12-19 14:42:43 main:507] : INFO : Preparing config file [2021-12-19 14:42:43 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 14:42:43 main:512] : INFO : Loading config [2021-12-19 14:42:43 main:514] : INFO : Loading state [2021-12-19 14:42:45 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 14:42:45 main:562] : INFO : Starting Training [2021-12-19 14:42:52 main:574] : INFO : Epoch 1786 | loss: 0.0310275 | val_loss: 0.0309746 | Time: 6914.33 ms [2021-12-19 14:43:01 main:574] : INFO : Epoch 1787 | loss: 0.0309292 | val_loss: 0.0309737 | Time: 9557.4 ms [2021-12-19 14:43:35 main:574] : INFO : Epoch 1788 | loss: 0.0309127 | val_loss: 0.0309754 | Time: 33369.8 ms [2021-12-19 14:43:57 main:574] : INFO : Epoch 1789 | loss: 0.0309184 | val_loss: 0.030975 | Time: 21824.5 ms [2021-12-19 14:44:20 main:574] : INFO : Epoch 1790 | loss: 0.030909 | val_loss: 0.0309636 | Time: 23114.6 ms [2021-12-19 14:44:41 main:574] : INFO : Epoch 1791 | loss: 0.0308991 | val_loss: 0.0309589 | Time: 21288.6 ms [2021-12-19 14:45:03 main:574] : INFO : Epoch 1792 | loss: 0.0309138 | val_loss: 0.0309721 | Time: 21313 ms [2021-12-19 14:45:23 main:574] : INFO : Epoch 1793 | loss: 0.0309157 | val_loss: 0.0309644 | Time: 20329.8 ms [2021-12-19 14:45:43 main:574] : INFO : Epoch 1794 | loss: 0.0309108 | val_loss: 0.0309575 | Time: 20459.4 ms [2021-12-19 14:46:05 main:574] : INFO : Epoch 1795 | loss: 0.0309021 | val_loss: 0.0309584 | Time: 21104.3 ms [2021-12-19 14:46:25 main:574] : INFO : Epoch 1796 | loss: 0.0309042 | val_loss: 0.0309753 | Time: 20226.2 ms [2021-12-19 14:46:58 main:574] : INFO : Epoch 1797 | loss: 0.0309168 | val_loss: 0.0309726 | Time: 32685.5 ms [2021-12-19 14:47:21 main:574] : INFO : Epoch 1798 | loss: 0.0309172 | val_loss: 0.0309749 | Time: 22867.9 ms [2021-12-19 14:47:43 main:574] : INFO : Epoch 1799 | loss: 0.0309038 | val_loss: 0.0309577 | Time: 22342.4 ms [2021-12-19 14:48:03 main:574] : INFO : Epoch 1800 | loss: 0.0309059 | val_loss: 0.0309625 | Time: 19993.6 ms [2021-12-19 14:48:23 main:574] : INFO : Epoch 1801 | loss: 0.0308935 | val_loss: 0.0309583 | Time: 20159 ms [2021-12-19 14:48:44 main:574] : INFO : Epoch 1802 | loss: 0.0309153 | val_loss: 0.0309665 | Time: 20352.8 ms [2021-12-19 14:49:04 main:574] : INFO : Epoch 1803 | loss: 0.030898 | val_loss: 0.0309575 | Time: 20484.5 ms [2021-12-19 14:49:38 main:574] : INFO : Epoch 1804 | loss: 0.0308965 | val_loss: 0.0309675 | Time: 33691.4 ms [2021-12-19 14:50:00 main:574] : INFO : Epoch 1805 | loss: 0.0309107 | val_loss: 0.0309613 | Time: 21538 ms [2021-12-19 14:50:20 main:574] : INFO : Epoch 1806 | loss: 0.0309003 | val_loss: 0.0309686 | Time: 20132.7 ms [2021-12-19 14:50:41 main:574] : INFO : Epoch 1807 | loss: 0.0308957 | val_loss: 0.0309527 | Time: 21003.7 ms [2021-12-19 14:51:02 main:574] : INFO : Epoch 1808 | loss: 0.0308871 | val_loss: 0.0309523 | Time: 20720.3 ms [2021-12-19 14:51:24 main:574] : INFO : Epoch 1809 | loss: 0.0308773 | val_loss: 0.0309491 | Time: 21539.9 ms [2021-12-19 14:51:43 main:574] : INFO : Epoch 1810 | loss: 0.0308808 | val_loss: 0.0309671 | Time: 19627 ms [2021-12-19 14:52:04 main:574] : INFO : Epoch 1811 | loss: 0.0308771 | val_loss: 0.0309493 | Time: 21220.8 ms [2021-12-19 14:52:25 main:574] : INFO : Epoch 1812 | loss: 0.0308823 | val_loss: 0.030963 | Time: 20493 ms [2021-12-19 14:52:57 main:574] : INFO : Epoch 1813 | loss: 0.0308855 | val_loss: 0.0309611 | Time: 31655.7 ms [2021-12-19 14:53:17 main:574] : INFO : Epoch 1814 | loss: 0.0308768 | val_loss: 0.0309535 | Time: 19938.8 ms [2021-12-19 14:53:37 main:574] : INFO : Epoch 1815 | loss: 0.0308684 | val_loss: 0.0309669 | Time: 20219.8 ms [2021-12-19 14:53:58 main:574] : INFO : Epoch 1816 | loss: 0.0308641 | val_loss: 0.0309525 | Time: 20217 ms [2021-12-19 14:54:19 main:574] : INFO : Epoch 1817 | loss: 0.0308678 | val_loss: 0.0309462 | Time: 21523.5 ms [2021-12-19 14:54:39 main:574] : INFO : Epoch 1818 | loss: 0.0308643 | val_loss: 0.0309439 | Time: 20090.5 ms [2021-12-19 14:55:00 main:574] : INFO : Epoch 1819 | loss: 0.0308748 | val_loss: 0.0309806 | Time: 20486.6 ms [2021-12-19 14:55:20 main:574] : INFO : Epoch 1820 | loss: 0.0308706 | val_loss: 0.0309508 | Time: 19492.3 ms [2021-12-19 14:55:39 main:574] : INFO : Epoch 1821 | loss: 0.0308636 | val_loss: 0.0309805 | Time: 18898.5 ms [2021-12-19 14:55:58 main:574] : INFO : Epoch 1822 | loss: 0.030862 | val_loss: 0.0309658 | Time: 19428.5 ms [2021-12-19 14:56:19 main:574] : INFO : Epoch 1823 | loss: 0.03086 | val_loss: 0.0310155 | Time: 20656.2 ms [2021-12-19 14:56:39 main:574] : INFO : Epoch 1824 | loss: 0.030875 | val_loss: 0.0309409 | Time: 20485 ms [2021-12-19 14:57:00 main:574] : INFO : Epoch 1825 | loss: 0.0308718 | val_loss: 0.0309647 | Time: 19920.2 ms [2021-12-19 14:57:21 main:574] : INFO : Epoch 1826 | loss: 0.030891 | val_loss: 0.0309611 | Time: 21311.5 ms [2021-12-19 14:58:45 main:574] : INFO : Epoch 1827 | loss: 0.0309017 | val_loss: 0.0309601 | Time: 83994.4 ms [2021-12-19 14:59:05 main:574] : INFO : Epoch 1828 | loss: 0.0309334 | val_loss: 0.0309833 | Time: 20070.1 ms [2021-12-19 14:59:26 main:574] : INFO : Epoch 1829 | loss: 0.0309104 | val_loss: 0.0309678 | Time: 20670.9 ms [2021-12-19 14:59:47 main:574] : INFO : Epoch 1830 | loss: 0.0308958 | val_loss: 0.0309513 | Time: 20892.1 ms [2021-12-19 15:00:07 main:574] : INFO : Epoch 1831 | loss: 0.0308812 | val_loss: 0.0309363 | Time: 19803.4 ms [2021-12-19 15:00:27 main:574] : INFO : Epoch 1832 | loss: 0.0308772 | val_loss: 0.030956 | Time: 20046.9 ms [2021-12-19 15:00:47 main:574] : INFO : Epoch 1833 | loss: 0.0308714 | val_loss: 0.0309347 | Time: 19636.8 ms [2021-12-19 15:01:08 main:574] : INFO : Epoch 1834 | loss: 0.0308653 | val_loss: 0.0309548 | Time: 21247.6 ms [2021-12-19 15:01:29 main:574] : INFO : Epoch 1835 | loss: 0.0308654 | val_loss: 0.0309694 | Time: 21028.6 ms [2021-12-19 15:01:50 main:574] : INFO : Epoch 1836 | loss: 0.0308676 | val_loss: 0.0309303 | Time: 20628.7 ms [2021-12-19 15:02:10 main:574] : INFO : Epoch 1837 | loss: 0.030859 | val_loss: 0.0309329 | Time: 20314.1 ms [2021-12-19 15:02:31 main:574] : INFO : Epoch 1838 | loss: 0.0308544 | val_loss: 0.0309337 | Time: 20556 ms [2021-12-19 15:03:07 main:574] : INFO : Epoch 1839 | loss: 0.0308674 | val_loss: 0.030949 | Time: 36315.6 ms [2021-12-19 15:03:28 main:574] : INFO : Epoch 1840 | loss: 0.030863 | val_loss: 0.0309622 | Time: 20651.5 ms [2021-12-19 15:03:48 main:574] : INFO : Epoch 1841 | loss: 0.0308604 | val_loss: 0.0309279 | Time: 20246 ms [2021-12-19 15:04:09 main:574] : INFO : Epoch 1842 | loss: 0.0308499 | val_loss: 0.0309428 | Time: 20801.4 ms [2021-12-19 15:04:28 main:574] : INFO : Epoch 1843 | loss: 0.0308555 | val_loss: 0.0309514 | Time: 19169.7 ms [2021-12-19 15:04:49 main:574] : INFO : Epoch 1844 | loss: 0.0308636 | val_loss: 0.0309348 | Time: 21045 ms [2021-12-19 15:05:09 main:574] : INFO : Epoch 1845 | loss: 0.0308871 | val_loss: 0.0309609 | Time: 19990.9 ms [2021-12-19 15:05:29 main:574] : INFO : Epoch 1846 | loss: 0.0308929 | val_loss: 0.0309256 | Time: 19609.7 ms [2021-12-19 15:05:49 main:574] : INFO : Epoch 1847 | loss: 0.0308737 | val_loss: 0.0309408 | Time: 19910.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 15:08:59 main:435] : INFO : Set logging level to 1 [2021-12-19 15:08:59 main:441] : INFO : Running in BOINC Client mode [2021-12-19 15:08:59 main:444] : INFO : Resolving all filenames [2021-12-19 15:08:59 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 15:08:59 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 15:08:59 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 15:08:59 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 15:08:59 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 15:08:59 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 15:08:59 main:474] : INFO : Configuration: [2021-12-19 15:08:59 main:475] : INFO : Model type: GRU [2021-12-19 15:08:59 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 15:08:59 main:477] : INFO : Max Epochs: 2048 [2021-12-19 15:08:59 main:478] : INFO : Batch Size: 128 [2021-12-19 15:08:59 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 15:08:59 main:480] : INFO : Patience: 10 [2021-12-19 15:08:59 main:481] : INFO : Hidden Width: 12 [2021-12-19 15:08:59 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 15:08:59 main:483] : INFO : # Backend Layers: 4 [2021-12-19 15:08:59 main:484] : INFO : # Threads: 1 [2021-12-19 15:08:59 main:486] : INFO : Preparing Dataset [2021-12-19 15:08:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 15:08:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 15:09:03 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 15:09:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 15:09:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 15:09:03 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 15:09:03 main:494] : INFO : Creating Model [2021-12-19 15:09:03 main:507] : INFO : Preparing config file [2021-12-19 15:09:03 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 15:09:03 main:512] : INFO : Loading config [2021-12-19 15:09:03 main:514] : INFO : Loading state [2021-12-19 15:09:04 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 15:09:04 main:562] : INFO : Starting Training [2021-12-19 15:09:11 main:574] : INFO : Epoch 1841 | loss: 0.0309437 | val_loss: 0.0309799 | Time: 6742.75 ms [2021-12-19 15:09:29 main:574] : INFO : Epoch 1842 | loss: 0.0309156 | val_loss: 0.0309542 | Time: 18685.1 ms [2021-12-19 15:09:39 main:574] : INFO : Epoch 1843 | loss: 0.0308881 | val_loss: 0.0309381 | Time: 9702.9 ms [2021-12-19 15:10:41 main:574] : INFO : Epoch 1844 | loss: 0.0308807 | val_loss: 0.0309468 | Time: 62344.9 ms [2021-12-19 15:11:01 main:574] : INFO : Epoch 1845 | loss: 0.0308564 | val_loss: 0.0309315 | Time: 19757.3 ms [2021-12-19 15:11:21 main:574] : INFO : Epoch 1846 | loss: 0.0308492 | val_loss: 0.0309182 | Time: 19256.8 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 15:14:16 main:435] : INFO : Set logging level to 1 [2021-12-19 15:14:16 main:441] : INFO : Running in BOINC Client mode [2021-12-19 15:14:16 main:444] : INFO : Resolving all filenames [2021-12-19 15:14:16 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 15:14:16 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 15:14:16 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 15:14:16 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 15:14:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 15:14:16 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 15:14:16 main:474] : INFO : Configuration: [2021-12-19 15:14:16 main:475] : INFO : Model type: GRU [2021-12-19 15:14:16 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 15:14:16 main:477] : INFO : Max Epochs: 2048 [2021-12-19 15:14:16 main:478] : INFO : Batch Size: 128 [2021-12-19 15:14:16 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 15:14:16 main:480] : INFO : Patience: 10 [2021-12-19 15:14:16 main:481] : INFO : Hidden Width: 12 [2021-12-19 15:14:16 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 15:14:16 main:483] : INFO : # Backend Layers: 4 [2021-12-19 15:14:16 main:484] : INFO : # Threads: 1 [2021-12-19 15:14:16 main:486] : INFO : Preparing Dataset [2021-12-19 15:14:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 15:14:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 15:14:19 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 15:14:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 15:14:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 15:14:20 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 15:14:20 main:494] : INFO : Creating Model [2021-12-19 15:14:20 main:507] : INFO : Preparing config file [2021-12-19 15:14:20 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 15:14:20 main:512] : INFO : Loading config [2021-12-19 15:14:20 main:514] : INFO : Loading state [2021-12-19 15:14:21 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 15:14:21 main:562] : INFO : Starting Training [2021-12-19 15:14:27 main:574] : INFO : Epoch 1841 | loss: 0.0309213 | val_loss: 0.0309448 | Time: 6754.63 ms [2021-12-19 15:14:36 main:574] : INFO : Epoch 1842 | loss: 0.0308754 | val_loss: 0.0309349 | Time: 8247.35 ms [2021-12-19 15:14:57 main:574] : INFO : Epoch 1843 | loss: 0.0308539 | val_loss: 0.0309368 | Time: 21762.5 ms [2021-12-19 15:15:19 main:574] : INFO : Epoch 1844 | loss: 0.0308483 | val_loss: 0.0309707 | Time: 21850.9 ms [2021-12-19 15:15:39 main:574] : INFO : Epoch 1845 | loss: 0.0308678 | val_loss: 0.0309257 | Time: 19498.3 ms [2021-12-19 15:15:59 main:574] : INFO : Epoch 1846 | loss: 0.0308482 | val_loss: 0.0309309 | Time: 20084.1 ms [2021-12-19 15:16:18 main:574] : INFO : Epoch 1847 | loss: 0.0308404 | val_loss: 0.0309163 | Time: 19458.8 ms [2021-12-19 15:16:38 main:574] : INFO : Epoch 1848 | loss: 0.0308435 | val_loss: 0.0309436 | Time: 19925.6 ms [2021-12-19 15:16:59 main:574] : INFO : Epoch 1849 | loss: 0.0308619 | val_loss: 0.030946 | Time: 20695.2 ms [2021-12-19 15:17:18 main:574] : INFO : Epoch 1850 | loss: 0.0308691 | val_loss: 0.030931 | Time: 19160.8 ms [2021-12-19 15:17:38 main:574] : INFO : Epoch 1851 | loss: 0.0308498 | val_loss: 0.0309136 | Time: 19700.6 ms [2021-12-19 15:17:58 main:574] : INFO : Epoch 1852 | loss: 0.0308425 | val_loss: 0.0309201 | Time: 19752.2 ms [2021-12-19 15:18:21 main:574] : INFO : Epoch 1853 | loss: 0.0308472 | val_loss: 0.030907 | Time: 22525.9 ms [2021-12-19 15:18:42 main:574] : INFO : Epoch 1854 | loss: 0.0308381 | val_loss: 0.0309214 | Time: 21711.4 ms [2021-12-19 15:19:02 main:574] : INFO : Epoch 1855 | loss: 0.0308441 | val_loss: 0.0309305 | Time: 19451.3 ms [2021-12-19 15:19:21 main:574] : INFO : Epoch 1856 | loss: 0.0308461 | val_loss: 0.0309339 | Time: 18999.7 ms [2021-12-19 15:19:42 main:574] : INFO : Epoch 1857 | loss: 0.0308743 | val_loss: 0.030916 | Time: 21067.4 ms [2021-12-19 15:20:03 main:574] : INFO : Epoch 1858 | loss: 0.0308796 | val_loss: 0.0309948 | Time: 20576.2 ms [2021-12-19 15:20:23 main:574] : INFO : Epoch 1859 | loss: 0.0309308 | val_loss: 0.0309294 | Time: 20122.1 ms [2021-12-19 15:20:42 main:574] : INFO : Epoch 1860 | loss: 0.0308796 | val_loss: 0.0309147 | Time: 19307.5 ms [2021-12-19 15:21:03 main:574] : INFO : Epoch 1861 | loss: 0.0308567 | val_loss: 0.0309178 | Time: 20776.9 ms [2021-12-19 15:21:23 main:574] : INFO : Epoch 1862 | loss: 0.0308451 | val_loss: 0.0309072 | Time: 20170.4 ms [2021-12-19 15:21:43 main:574] : INFO : Epoch 1863 | loss: 0.0308294 | val_loss: 0.0309057 | Time: 19441.1 ms [2021-12-19 15:22:03 main:574] : INFO : Epoch 1864 | loss: 0.0308367 | val_loss: 0.0309053 | Time: 20320.5 ms [2021-12-19 15:22:22 main:574] : INFO : Epoch 1865 | loss: 0.0308272 | val_loss: 0.0308975 | Time: 18993.9 ms [2021-12-19 15:22:42 main:574] : INFO : Epoch 1866 | loss: 0.0308238 | val_loss: 0.0309005 | Time: 20025.1 ms [2021-12-19 15:23:03 main:574] : INFO : Epoch 1867 | loss: 0.0308288 | val_loss: 0.030907 | Time: 20777 ms [2021-12-19 15:23:23 main:574] : INFO : Epoch 1868 | loss: 0.030848 | val_loss: 0.0309076 | Time: 19966.9 ms [2021-12-19 15:23:54 main:574] : INFO : Epoch 1869 | loss: 0.0308366 | val_loss: 0.0309126 | Time: 31202.9 ms [2021-12-19 15:24:14 main:574] : INFO : Epoch 1870 | loss: 0.0308206 | val_loss: 0.0308931 | Time: 19502.2 ms [2021-12-19 15:24:33 main:574] : INFO : Epoch 1871 | loss: 0.0308305 | val_loss: 0.0308961 | Time: 18695 ms [2021-12-19 15:24:53 main:574] : INFO : Epoch 1872 | loss: 0.030829 | val_loss: 0.0309165 | Time: 20115.2 ms [2021-12-19 15:25:13 main:574] : INFO : Epoch 1873 | loss: 0.0308544 | val_loss: 0.0309063 | Time: 19603.3 ms [2021-12-19 15:25:32 main:574] : INFO : Epoch 1874 | loss: 0.0308386 | val_loss: 0.0309157 | Time: 19379 ms [2021-12-19 15:25:52 main:574] : INFO : Epoch 1875 | loss: 0.0308329 | val_loss: 0.030919 | Time: 19752.1 ms [2021-12-19 15:26:11 main:574] : INFO : Epoch 1876 | loss: 0.0308557 | val_loss: 0.0309091 | Time: 19364.1 ms [2021-12-19 15:26:32 main:574] : INFO : Epoch 1877 | loss: 0.0308301 | val_loss: 0.0309101 | Time: 20581.8 ms [2021-12-19 15:26:52 main:574] : INFO : Epoch 1878 | loss: 0.0308237 | val_loss: 0.0308811 | Time: 19783.2 ms [2021-12-19 15:27:12 main:574] : INFO : Epoch 1879 | loss: 0.0308306 | val_loss: 0.0309081 | Time: 20005.4 ms [2021-12-19 15:27:31 main:574] : INFO : Epoch 1880 | loss: 0.0308257 | val_loss: 0.0308931 | Time: 19495.1 ms [2021-12-19 15:28:22 main:574] : INFO : Epoch 1881 | loss: 0.0308628 | val_loss: 0.0310458 | Time: 51058.3 ms [2021-12-19 15:28:31 main:574] : INFO : Epoch 1882 | loss: 0.031024 | val_loss: 0.0310416 | Time: 8665.8 ms [2021-12-19 15:29:11 main:574] : INFO : Epoch 1883 | loss: 0.0309624 | val_loss: 0.030936 | Time: 39586.2 ms [2021-12-19 15:29:43 main:574] : INFO : Epoch 1884 | loss: 0.0308673 | val_loss: 0.0309095 | Time: 31831.5 ms [2021-12-19 15:29:52 main:574] : INFO : Epoch 1885 | loss: 0.0308588 | val_loss: 0.0308995 | Time: 9368.23 ms [2021-12-19 15:30:11 main:574] : INFO : Epoch 1886 | loss: 0.0308502 | val_loss: 0.0308905 | Time: 19150.7 ms [2021-12-19 15:30:31 main:574] : INFO : Epoch 1887 | loss: 0.0308265 | val_loss: 0.0309113 | Time: 19093.7 ms [2021-12-19 15:30:50 main:574] : INFO : Epoch 1888 | loss: 0.0308286 | val_loss: 0.0308833 | Time: 19772.1 ms [2021-12-19 15:31:11 main:574] : INFO : Epoch 1889 | loss: 0.0308438 | val_loss: 0.0308958 | Time: 20114.4 ms [2021-12-19 15:31:31 main:574] : INFO : Epoch 1890 | loss: 0.0308379 | val_loss: 0.0308942 | Time: 19969.4 ms [2021-12-19 15:31:51 main:574] : INFO : Epoch 1891 | loss: 0.0308771 | val_loss: 0.0309464 | Time: 19901.6 ms [2021-12-19 15:32:11 main:574] : INFO : Epoch 1892 | loss: 0.0308755 | val_loss: 0.0309144 | Time: 19952.5 ms [2021-12-19 15:32:31 main:574] : INFO : Epoch 1893 | loss: 0.0309141 | val_loss: 0.0310204 | Time: 20172.1 ms [2021-12-19 15:32:52 main:574] : INFO : Epoch 1894 | loss: 0.0310027 | val_loss: 0.0309675 | Time: 20674.4 ms [2021-12-19 15:33:11 main:574] : INFO : Epoch 1895 | loss: 0.0309186 | val_loss: 0.0309235 | Time: 19787.3 ms [2021-12-19 15:33:31 main:574] : INFO : Epoch 1896 | loss: 0.0309081 | val_loss: 0.0309043 | Time: 19268.6 ms [2021-12-19 15:34:06 main:574] : INFO : Epoch 1897 | loss: 0.0308876 | val_loss: 0.0309036 | Time: 34859.6 ms [2021-12-19 15:34:14 main:574] : INFO : Epoch 1898 | loss: 0.0308755 | val_loss: 0.0308895 | Time: 8665.07 ms [2021-12-19 15:34:35 main:574] : INFO : Epoch 1899 | loss: 0.0308384 | val_loss: 0.0307956 | Time: 20189.3 ms [2021-12-19 15:34:54 main:574] : INFO : Epoch 1900 | loss: 0.0308154 | val_loss: 0.0310023 | Time: 19046.4 ms [2021-12-19 15:35:13 main:574] : INFO : Epoch 1901 | loss: 0.030922 | val_loss: 0.030781 | Time: 19509.5 ms [2021-12-19 15:35:33 main:574] : INFO : Epoch 1902 | loss: 0.0306682 | val_loss: 0.0305299 | Time: 19287.3 ms [2021-12-19 15:35:53 main:574] : INFO : Epoch 1903 | loss: 0.0308503 | val_loss: 0.0312678 | Time: 19844.1 ms [2021-12-19 15:36:13 main:574] : INFO : Epoch 1904 | loss: 0.0313769 | val_loss: 0.0313282 | Time: 19948.9 ms [2021-12-19 15:36:32 main:574] : INFO : Epoch 1905 | loss: 0.0313299 | val_loss: 0.0313147 | Time: 19554.5 ms [2021-12-19 15:36:52 main:574] : INFO : Epoch 1906 | loss: 0.0312669 | val_loss: 0.0312494 | Time: 19371.3 ms [2021-12-19 15:37:12 main:574] : INFO : Epoch 1907 | loss: 0.0312465 | val_loss: 0.0312445 | Time: 20103 ms [2021-12-19 15:37:32 main:574] : INFO : Epoch 1908 | loss: 0.031243 | val_loss: 0.0312372 | Time: 20067.4 ms [2021-12-19 15:37:53 main:574] : INFO : Epoch 1909 | loss: 0.0312363 | val_loss: 0.0312288 | Time: 20322.5 ms [2021-12-19 15:38:13 main:574] : INFO : Epoch 1910 | loss: 0.0312327 | val_loss: 0.031222 | Time: 20095 ms [2021-12-19 15:38:32 main:574] : INFO : Epoch 1911 | loss: 0.0312302 | val_loss: 0.0312175 | Time: 19274.2 ms [2021-12-19 15:38:52 main:574] : INFO : Epoch 1912 | loss: 0.0312311 | val_loss: 0.0312266 | Time: 20167.2 ms [2021-12-19 15:39:25 main:574] : INFO : Epoch 1913 | loss: 0.0312198 | val_loss: 0.031209 | Time: 32495.2 ms [2021-12-19 15:40:26 main:574] : INFO : Epoch 1914 | loss: 0.0312183 | val_loss: 0.0312142 | Time: 61251.4 ms [2021-12-19 15:40:46 main:574] : INFO : Epoch 1915 | loss: 0.0312103 | val_loss: 0.0312025 | Time: 20002.8 ms [2021-12-19 15:41:06 main:574] : INFO : Epoch 1916 | loss: 0.0312072 | val_loss: 0.0311911 | Time: 19679.8 ms [2021-12-19 15:41:25 main:574] : INFO : Epoch 1917 | loss: 0.0312003 | val_loss: 0.031198 | Time: 19133 ms [2021-12-19 15:41:45 main:574] : INFO : Epoch 1918 | loss: 0.0311912 | val_loss: 0.0312053 | Time: 19963.2 ms [2021-12-19 15:42:06 main:574] : INFO : Epoch 1919 | loss: 0.0311862 | val_loss: 0.0311712 | Time: 20637.5 ms [2021-12-19 15:42:27 main:574] : INFO : Epoch 1920 | loss: 0.0311705 | val_loss: 0.0311654 | Time: 20713.8 ms [2021-12-19 15:42:46 main:574] : INFO : Epoch 1921 | loss: 0.0311632 | val_loss: 0.0311576 | Time: 18831.8 ms [2021-12-19 15:43:06 main:574] : INFO : Epoch 1922 | loss: 0.0311403 | val_loss: 0.0311318 | Time: 20431.7 ms [2021-12-19 15:43:26 main:574] : INFO : Epoch 1923 | loss: 0.0311221 | val_loss: 0.0311119 | Time: 19843.2 ms [2021-12-19 15:43:45 main:574] : INFO : Epoch 1924 | loss: 0.031104 | val_loss: 0.0310857 | Time: 19186.8 ms [2021-12-19 15:44:05 main:574] : INFO : Epoch 1925 | loss: 0.0310509 | val_loss: 0.0309273 | Time: 19719.6 ms [2021-12-19 15:44:39 main:574] : INFO : Epoch 1926 | loss: 0.0309753 | val_loss: 0.0301882 | Time: 33655.7 ms [2021-12-19 15:44:59 main:574] : INFO : Epoch 1927 | loss: 0.0352592 | val_loss: 0.0325177 | Time: 19641.2 ms [2021-12-19 15:45:18 main:574] : INFO : Epoch 1928 | loss: 0.0323746 | val_loss: 0.0315466 | Time: 19419.1 ms [2021-12-19 15:45:37 main:574] : INFO : Epoch 1929 | loss: 0.0313808 | val_loss: 0.0312744 | Time: 18752.2 ms [2021-12-19 15:45:56 main:574] : INFO : Epoch 1930 | loss: 0.0312297 | val_loss: 0.0311938 | Time: 19335.4 ms [2021-12-19 15:46:16 main:574] : INFO : Epoch 1931 | loss: 0.0311708 | val_loss: 0.0311362 | Time: 19766.8 ms [2021-12-19 15:46:37 main:574] : INFO : Epoch 1932 | loss: 0.0311137 | val_loss: 0.0310324 | Time: 20260.8 ms [2021-12-19 15:46:57 main:574] : INFO : Epoch 1933 | loss: 0.030901 | val_loss: 0.0307716 | Time: 20198.2 ms [2021-12-19 15:47:17 main:574] : INFO : Epoch 1934 | loss: 0.0307341 | val_loss: 0.0306401 | Time: 20074.4 ms [2021-12-19 15:47:37 main:574] : INFO : Epoch 1935 | loss: 0.0306095 | val_loss: 0.0305225 | Time: 20335.9 ms [2021-12-19 15:47:58 main:574] : INFO : Epoch 1936 | loss: 0.0305176 | val_loss: 0.0304504 | Time: 20873.6 ms [2021-12-19 15:48:18 main:574] : INFO : Epoch 1937 | loss: 0.0304505 | val_loss: 0.0303682 | Time: 20112.9 ms [2021-12-19 15:48:38 main:574] : INFO : Epoch 1938 | loss: 0.0303847 | val_loss: 0.030285 | Time: 19894.8 ms [2021-12-19 15:48:58 main:574] : INFO : Epoch 1939 | loss: 0.0300202 | val_loss: 0.0295081 | Time: 20053.4 ms [2021-12-19 15:49:20 main:574] : INFO : Epoch 1940 | loss: 0.029202 | val_loss: 0.0288051 | Time: 21059.4 ms [2021-12-19 15:49:43 main:574] : INFO : Epoch 1941 | loss: 0.0287389 | val_loss: 0.0284884 | Time: 23682.2 ms [2021-12-19 15:50:04 main:574] : INFO : Epoch 1942 | loss: 0.0285235 | val_loss: 0.0283041 | Time: 20226.3 ms [2021-12-19 15:50:23 main:574] : INFO : Epoch 1943 | loss: 0.0283218 | val_loss: 0.0280817 | Time: 19543.9 ms [2021-12-19 15:50:43 main:574] : INFO : Epoch 1944 | loss: 0.0281052 | val_loss: 0.0279102 | Time: 19747.9 ms [2021-12-19 15:51:03 main:574] : INFO : Epoch 1945 | loss: 0.0279555 | val_loss: 0.0277904 | Time: 20339.1 ms [2021-12-19 15:51:23 main:574] : INFO : Epoch 1946 | loss: 0.0278448 | val_loss: 0.0276984 | Time: 19723.1 ms [2021-12-19 15:51:43 main:574] : INFO : Epoch 1947 | loss: 0.0277624 | val_loss: 0.0276861 | Time: 20166 ms [2021-12-19 15:52:04 main:574] : INFO : Epoch 1948 | loss: 0.0277005 | val_loss: 0.0275142 | Time: 20485.8 ms [2021-12-19 15:52:24 main:574] : INFO : Epoch 1949 | loss: 0.0275767 | val_loss: 0.0274606 | Time: 19932.8 ms [2021-12-19 15:52:43 main:574] : INFO : Epoch 1950 | loss: 0.0275396 | val_loss: 0.0274302 | Time: 19324.1 ms [2021-12-19 15:53:04 main:574] : INFO : Epoch 1951 | loss: 0.0274254 | val_loss: 0.0272428 | Time: 20916.3 ms [2021-12-19 15:53:25 main:574] : INFO : Epoch 1952 | loss: 0.0273359 | val_loss: 0.0272064 | Time: 20366 ms [2021-12-19 15:53:44 main:574] : INFO : Epoch 1953 | loss: 0.0272926 | val_loss: 0.0271258 | Time: 18955.5 ms [2021-12-19 15:54:04 main:574] : INFO : Epoch 1954 | loss: 0.0272317 | val_loss: 0.0271197 | Time: 19771.4 ms [2021-12-19 15:54:24 main:574] : INFO : Epoch 1955 | loss: 0.0271785 | val_loss: 0.0270669 | Time: 20019.9 ms [2021-12-19 15:54:58 main:574] : INFO : Epoch 1956 | loss: 0.027087 | val_loss: 0.0268559 | Time: 34464 ms [2021-12-19 15:55:48 main:574] : INFO : Epoch 1957 | loss: 0.0269105 | val_loss: 0.0267369 | Time: 49618.5 ms [2021-12-19 15:56:07 main:574] : INFO : Epoch 1958 | loss: 0.0266516 | val_loss: 0.0264822 | Time: 19182.5 ms [2021-12-19 15:56:28 main:574] : INFO : Epoch 1959 | loss: 0.0263967 | val_loss: 0.0261032 | Time: 20010.7 ms [2021-12-19 15:56:48 main:574] : INFO : Epoch 1960 | loss: 0.0262576 | val_loss: 0.0269707 | Time: 20019 ms [2021-12-19 15:57:08 main:574] : INFO : Epoch 1961 | loss: 0.0267349 | val_loss: 0.0261072 | Time: 19788.2 ms [2021-12-19 15:57:28 main:574] : INFO : Epoch 1962 | loss: 0.0260459 | val_loss: 0.0257267 | Time: 20145.8 ms [2021-12-19 15:58:01 main:574] : INFO : Epoch 1963 | loss: 0.026025 | val_loss: 0.0277341 | Time: 33117.1 ms [2021-12-19 15:58:10 main:574] : INFO : Epoch 1964 | loss: 0.0283533 | val_loss: 0.0272525 | Time: 9134.06 ms [2021-12-19 15:58:30 main:574] : INFO : Epoch 1965 | loss: 0.0264967 | val_loss: 0.0255524 | Time: 19779.9 ms [2021-12-19 15:58:49 main:574] : INFO : Epoch 1966 | loss: 0.0245133 | val_loss: 0.0232027 | Time: 18888.4 ms [2021-12-19 15:59:08 main:574] : INFO : Epoch 1967 | loss: 0.0228293 | val_loss: 0.022257 | Time: 19300.7 ms [2021-12-19 15:59:30 main:574] : INFO : Epoch 1968 | loss: 0.0220843 | val_loss: 0.0216886 | Time: 22169.3 ms [2021-12-19 16:00:10 main:574] : INFO : Epoch 1969 | loss: 0.0216356 | val_loss: 0.0216573 | Time: 39654.1 ms [2021-12-19 16:00:29 main:574] : INFO : Epoch 1970 | loss: 0.0217323 | val_loss: 0.0205093 | Time: 18666.8 ms [2021-12-19 16:00:48 main:574] : INFO : Epoch 1971 | loss: 0.0201892 | val_loss: 0.0195354 | Time: 18574.6 ms [2021-12-19 16:01:08 main:574] : INFO : Epoch 1972 | loss: 0.0218632 | val_loss: 0.0293312 | Time: 19867.6 ms [2021-12-19 16:01:28 main:574] : INFO : Epoch 1973 | loss: 0.0308958 | val_loss: 0.0312274 | Time: 20368.3 ms [2021-12-19 16:01:47 main:574] : INFO : Epoch 1974 | loss: 0.03123 | val_loss: 0.0312163 | Time: 19142.1 ms [2021-12-19 16:02:07 main:574] : INFO : Epoch 1975 | loss: 0.0312153 | val_loss: 0.0312035 | Time: 19588.1 ms [2021-12-19 16:02:27 main:574] : INFO : Epoch 1976 | loss: 0.0311963 | val_loss: 0.0311826 | Time: 19678.7 ms [2021-12-19 16:02:46 main:574] : INFO : Epoch 1977 | loss: 0.0311737 | val_loss: 0.0311599 | Time: 19491.1 ms [2021-12-19 16:03:07 main:574] : INFO : Epoch 1978 | loss: 0.0311476 | val_loss: 0.0311235 | Time: 21033.7 ms [2021-12-19 16:03:28 main:574] : INFO : Epoch 1979 | loss: 0.0310972 | val_loss: 0.0310415 | Time: 20516.3 ms [2021-12-19 16:03:48 main:574] : INFO : Epoch 1980 | loss: 0.0310279 | val_loss: 0.0309701 | Time: 19966.7 ms [2021-12-19 16:04:09 main:574] : INFO : Epoch 1981 | loss: 0.0309644 | val_loss: 0.0308896 | Time: 20538.1 ms [2021-12-19 16:04:28 main:574] : INFO : Epoch 1982 | loss: 0.0308782 | val_loss: 0.0307925 | Time: 19363.7 ms [2021-12-19 16:04:49 main:574] : INFO : Epoch 1983 | loss: 0.0307985 | val_loss: 0.0307 | Time: 21484.5 ms [2021-12-19 16:05:12 main:574] : INFO : Epoch 1984 | loss: 0.0306849 | val_loss: 0.0305751 | Time: 22644.2 ms [2021-12-19 16:05:31 main:574] : INFO : Epoch 1985 | loss: 0.0305598 | val_loss: 0.0304197 | Time: 19101.7 ms [2021-12-19 16:05:51 main:574] : INFO : Epoch 1986 | loss: 0.0304248 | val_loss: 0.0302588 | Time: 19212.8 ms [2021-12-19 16:06:11 main:574] : INFO : Epoch 1987 | loss: 0.0302451 | val_loss: 0.0300544 | Time: 20143.9 ms [2021-12-19 16:06:31 main:574] : INFO : Epoch 1988 | loss: 0.0300418 | val_loss: 0.0298655 | Time: 19918.3 ms [2021-12-19 16:06:51 main:574] : INFO : Epoch 1989 | loss: 0.0299423 | val_loss: 0.0297747 | Time: 19906.9 ms [2021-12-19 16:07:11 main:574] : INFO : Epoch 1990 | loss: 0.0297754 | val_loss: 0.029559 | Time: 20243.3 ms [2021-12-19 16:07:31 main:574] : INFO : Epoch 1991 | loss: 0.0296001 | val_loss: 0.0294056 | Time: 19410.8 ms [2021-12-19 16:07:50 main:574] : INFO : Epoch 1992 | loss: 0.0294415 | val_loss: 0.0292417 | Time: 19272.1 ms [2021-12-19 16:08:10 main:574] : INFO : Epoch 1993 | loss: 0.0292567 | val_loss: 0.0290833 | Time: 19634.4 ms [2021-12-19 16:08:30 main:574] : INFO : Epoch 1994 | loss: 0.0291841 | val_loss: 0.0289746 | Time: 20235.4 ms [2021-12-19 16:08:50 main:574] : INFO : Epoch 1995 | loss: 0.0289717 | val_loss: 0.0287605 | Time: 19566.1 ms [2021-12-19 16:09:09 main:574] : INFO : Epoch 1996 | loss: 0.0288046 | val_loss: 0.0286136 | Time: 19640.4 ms [2021-12-19 16:09:29 main:574] : INFO : Epoch 1997 | loss: 0.0286209 | val_loss: 0.0284024 | Time: 19652.3 ms [2021-12-19 16:09:50 main:574] : INFO : Epoch 1998 | loss: 0.0284154 | val_loss: 0.0282018 | Time: 20615.2 ms [2021-12-19 16:10:22 main:574] : INFO : Epoch 1999 | loss: 0.0282237 | val_loss: 0.0280495 | Time: 31839.7 ms [2021-12-19 16:10:40 main:574] : INFO : Epoch 2000 | loss: 0.0280949 | val_loss: 0.0279208 | Time: 18271.2 ms [2021-12-19 16:11:00 main:574] : INFO : Epoch 2001 | loss: 0.0279546 | val_loss: 0.0277804 | Time: 19531.7 ms [2021-12-19 16:11:09 main:574] : INFO : Epoch 2002 | loss: 0.0278167 | val_loss: 0.0276757 | Time: 8868.41 ms [2021-12-19 16:11:28 main:574] : INFO : Epoch 2003 | loss: 0.027726 | val_loss: 0.0276076 | Time: 19159.5 ms [2021-12-19 16:11:48 main:574] : INFO : Epoch 2004 | loss: 0.0276457 | val_loss: 0.0275014 | Time: 19682.9 ms [2021-12-19 16:12:07 main:574] : INFO : Epoch 2005 | loss: 0.0275457 | val_loss: 0.0274136 | Time: 18696.7 ms [2021-12-19 16:12:27 main:574] : INFO : Epoch 2006 | loss: 0.0274117 | val_loss: 0.0273129 | Time: 19951.4 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 16:16:04 main:435] : INFO : Set logging level to 1 [2021-12-19 16:16:04 main:441] : INFO : Running in BOINC Client mode [2021-12-19 16:16:04 main:444] : INFO : Resolving all filenames [2021-12-19 16:16:04 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 16:16:04 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 16:16:04 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 16:16:04 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 16:16:04 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 16:16:04 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 16:16:04 main:474] : INFO : Configuration: [2021-12-19 16:16:04 main:475] : INFO : Model type: GRU [2021-12-19 16:16:04 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 16:16:04 main:477] : INFO : Max Epochs: 2048 [2021-12-19 16:16:04 main:478] : INFO : Batch Size: 128 [2021-12-19 16:16:04 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 16:16:04 main:480] : INFO : Patience: 10 [2021-12-19 16:16:04 main:481] : INFO : Hidden Width: 12 [2021-12-19 16:16:04 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 16:16:04 main:483] : INFO : # Backend Layers: 4 [2021-12-19 16:16:04 main:484] : INFO : # Threads: 1 [2021-12-19 16:16:04 main:486] : INFO : Preparing Dataset [2021-12-19 16:16:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 16:16:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 16:16:07 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 16:16:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 16:16:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 16:16:07 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 16:16:07 main:494] : INFO : Creating Model [2021-12-19 16:16:07 main:507] : INFO : Preparing config file [2021-12-19 16:16:07 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 16:16:07 main:512] : INFO : Loading config [2021-12-19 16:16:07 main:514] : INFO : Loading state [2021-12-19 16:16:08 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 16:16:08 main:562] : INFO : Starting Training [2021-12-19 16:16:15 main:574] : INFO : Epoch 1999 | loss: 0.0303085 | val_loss: 0.0291491 | Time: 6753.6 ms [2021-12-19 16:16:23 main:574] : INFO : Epoch 2000 | loss: 0.0280405 | val_loss: 0.0272689 | Time: 8228.9 ms [2021-12-19 16:16:45 main:574] : INFO : Epoch 2001 | loss: 0.0270752 | val_loss: 0.0266521 | Time: 21063.6 ms [2021-12-19 16:16:53 main:574] : INFO : Epoch 2002 | loss: 0.0266351 | val_loss: 0.026346 | Time: 8525.41 ms [2021-12-19 16:17:56 main:574] : INFO : Epoch 2003 | loss: 0.0252978 | val_loss: 0.0241165 | Time: 62611.9 ms [2021-12-19 16:18:15 main:574] : INFO : Epoch 2004 | loss: 0.0240631 | val_loss: 0.02371 | Time: 19358.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 16:21:22 main:435] : INFO : Set logging level to 1 [2021-12-19 16:21:22 main:441] : INFO : Running in BOINC Client mode [2021-12-19 16:21:22 main:444] : INFO : Resolving all filenames [2021-12-19 16:21:22 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 16:21:22 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 16:21:22 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 16:21:22 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 16:21:22 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 16:21:22 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 16:21:22 main:474] : INFO : Configuration: [2021-12-19 16:21:22 main:475] : INFO : Model type: GRU [2021-12-19 16:21:22 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 16:21:22 main:477] : INFO : Max Epochs: 2048 [2021-12-19 16:21:22 main:478] : INFO : Batch Size: 128 [2021-12-19 16:21:22 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 16:21:22 main:480] : INFO : Patience: 10 [2021-12-19 16:21:22 main:481] : INFO : Hidden Width: 12 [2021-12-19 16:21:22 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 16:21:22 main:483] : INFO : # Backend Layers: 4 [2021-12-19 16:21:22 main:484] : INFO : # Threads: 1 [2021-12-19 16:21:22 main:486] : INFO : Preparing Dataset [2021-12-19 16:21:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 16:21:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 16:21:25 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 16:21:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 16:21:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 16:21:25 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 16:21:25 main:494] : INFO : Creating Model [2021-12-19 16:21:25 main:507] : INFO : Preparing config file [2021-12-19 16:21:25 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 16:21:25 main:512] : INFO : Loading config [2021-12-19 16:21:25 main:514] : INFO : Loading state [2021-12-19 16:21:26 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 16:21:26 main:562] : INFO : Starting Training [2021-12-19 16:21:33 main:574] : INFO : Epoch 1999 | loss: 0.030418 | val_loss: 0.029313 | Time: 6670.1 ms [2021-12-19 16:21:42 main:574] : INFO : Epoch 2000 | loss: 0.0282191 | val_loss: 0.027275 | Time: 8671.21 ms [2021-12-19 16:22:03 main:574] : INFO : Epoch 2001 | loss: 0.0270741 | val_loss: 0.0266517 | Time: 21092.5 ms [2021-12-19 16:22:25 main:574] : INFO : Epoch 2002 | loss: 0.026598 | val_loss: 0.026755 | Time: 22247.3 ms [2021-12-19 16:22:45 main:574] : INFO : Epoch 2003 | loss: 0.0270923 | val_loss: 0.027025 | Time: 19602.3 ms [2021-12-19 16:23:04 main:574] : INFO : Epoch 2004 | loss: 0.0266689 | val_loss: 0.0259448 | Time: 19394.2 ms [2021-12-19 16:23:23 main:574] : INFO : Epoch 2005 | loss: 0.0263045 | val_loss: 0.0263963 | Time: 19091.3 ms [2021-12-19 16:23:44 main:574] : INFO : Epoch 2006 | loss: 0.0261596 | val_loss: 0.0254851 | Time: 20309.3 ms [2021-12-19 16:24:03 main:574] : INFO : Epoch 2007 | loss: 0.0249349 | val_loss: 0.0242258 | Time: 19253 ms [2021-12-19 16:24:24 main:574] : INFO : Epoch 2008 | loss: 0.0241617 | val_loss: 0.0234181 | Time: 20905.5 ms [2021-12-19 16:24:45 main:574] : INFO : Epoch 2009 | loss: 0.0230853 | val_loss: 0.0233669 | Time: 21052 ms [2021-12-19 16:25:56 main:574] : INFO : Epoch 2010 | loss: 0.023699 | val_loss: 0.0232088 | Time: 70854.7 ms [2021-12-19 16:26:15 main:574] : INFO : Epoch 2011 | loss: 0.0225564 | val_loss: 0.0237336 | Time: 18915.2 ms [2021-12-19 16:26:35 main:574] : INFO : Epoch 2012 | loss: 0.0242537 | val_loss: 0.0240415 | Time: 20352.2 ms [2021-12-19 16:26:55 main:574] : INFO : Epoch 2013 | loss: 0.0247979 | val_loss: 0.0257321 | Time: 19661.3 ms [2021-12-19 16:27:14 main:574] : INFO : Epoch 2014 | loss: 0.024811 | val_loss: 0.0234819 | Time: 19321.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 16:30:54 main:435] : INFO : Set logging level to 1 [2021-12-19 16:30:54 main:441] : INFO : Running in BOINC Client mode [2021-12-19 16:30:54 main:444] : INFO : Resolving all filenames [2021-12-19 16:30:55 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 16:30:55 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 16:30:55 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 16:30:55 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 16:30:55 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 16:30:55 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 16:30:55 main:474] : INFO : Configuration: [2021-12-19 16:30:55 main:475] : INFO : Model type: GRU [2021-12-19 16:30:55 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 16:30:55 main:477] : INFO : Max Epochs: 2048 [2021-12-19 16:30:55 main:478] : INFO : Batch Size: 128 [2021-12-19 16:30:55 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 16:30:55 main:480] : INFO : Patience: 10 [2021-12-19 16:30:55 main:481] : INFO : Hidden Width: 12 [2021-12-19 16:30:55 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 16:30:55 main:483] : INFO : # Backend Layers: 4 [2021-12-19 16:30:55 main:484] : INFO : # Threads: 1 [2021-12-19 16:30:55 main:486] : INFO : Preparing Dataset [2021-12-19 16:30:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 16:30:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 16:30:58 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 16:30:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 16:30:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 16:30:58 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 16:30:58 main:494] : INFO : Creating Model [2021-12-19 16:30:58 main:507] : INFO : Preparing config file [2021-12-19 16:30:58 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 16:30:58 main:512] : INFO : Loading config [2021-12-19 16:30:58 main:514] : INFO : Loading state [2021-12-19 16:30:59 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 16:30:59 main:562] : INFO : Starting Training [2021-12-19 16:31:06 main:574] : INFO : Epoch 2011 | loss: 0.0244532 | val_loss: 0.0235757 | Time: 6805.67 ms [2021-12-19 16:31:13 main:574] : INFO : Epoch 2012 | loss: 0.0231916 | val_loss: 0.0226685 | Time: 7549.23 ms [2021-12-19 16:31:35 main:574] : INFO : Epoch 2013 | loss: 0.0215939 | val_loss: 0.0221992 | Time: 21320.4 ms [2021-12-19 16:31:43 main:574] : INFO : Epoch 2014 | loss: 0.0216206 | val_loss: 0.0209177 | Time: 8396.01 ms [2021-12-19 16:32:03 main:574] : INFO : Epoch 2015 | loss: 0.0199624 | val_loss: 0.0195636 | Time: 19645.2 ms [2021-12-19 16:32:35 main:574] : INFO : Epoch 2016 | loss: 0.0199861 | val_loss: 0.0192599 | Time: 32425.8 ms [2021-12-19 16:32:56 main:574] : INFO : Epoch 2017 | loss: 0.0188738 | val_loss: 0.0185251 | Time: 20698.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2021-12-19 16:36:08 main:435] : INFO : Set logging level to 1 [2021-12-19 16:36:08 main:441] : INFO : Running in BOINC Client mode [2021-12-19 16:36:08 main:444] : INFO : Resolving all filenames [2021-12-19 16:36:08 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2021-12-19 16:36:08 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2021-12-19 16:36:08 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2021-12-19 16:36:08 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2021-12-19 16:36:08 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2021-12-19 16:36:08 main:472] : INFO : Dataset filename: dataset.hdf5 [2021-12-19 16:36:08 main:474] : INFO : Configuration: [2021-12-19 16:36:08 main:475] : INFO : Model type: GRU [2021-12-19 16:36:08 main:476] : INFO : Validation Loss Threshold: 0.0001 [2021-12-19 16:36:08 main:477] : INFO : Max Epochs: 2048 [2021-12-19 16:36:08 main:478] : INFO : Batch Size: 128 [2021-12-19 16:36:08 main:479] : INFO : Learning Rate: 0.01 [2021-12-19 16:36:08 main:480] : INFO : Patience: 10 [2021-12-19 16:36:08 main:481] : INFO : Hidden Width: 12 [2021-12-19 16:36:08 main:482] : INFO : # Recurrent Layers: 4 [2021-12-19 16:36:08 main:483] : INFO : # Backend Layers: 4 [2021-12-19 16:36:08 main:484] : INFO : # Threads: 1 [2021-12-19 16:36:08 main:486] : INFO : Preparing Dataset [2021-12-19 16:36:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2021-12-19 16:36:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2021-12-19 16:36:11 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2021-12-19 16:36:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2021-12-19 16:36:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2021-12-19 16:36:12 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2021-12-19 16:36:12 main:494] : INFO : Creating Model [2021-12-19 16:36:12 main:507] : INFO : Preparing config file [2021-12-19 16:36:12 main:511] : INFO : Found checkpoint, attempting to load... [2021-12-19 16:36:12 main:512] : INFO : Loading config [2021-12-19 16:36:12 main:514] : INFO : Loading state [2021-12-19 16:36:13 main:559] : INFO : Loading DataLoader into Memory [2021-12-19 16:36:13 main:562] : INFO : Starting Training [2021-12-19 16:36:19 main:574] : INFO : Epoch 2011 | loss: 0.0241557 | val_loss: 0.0225668 | Time: 6730.7 ms [2021-12-19 16:36:27 main:574] : INFO : Epoch 2012 | loss: 0.021348 | val_loss: 0.0214603 | Time: 7850.62 ms [2021-12-19 16:36:38 main:574] : INFO : Epoch 2013 | loss: 0.022242 | val_loss: 0.0227498 | Time: 10483.2 ms [2021-12-19 16:36:57 main:574] : INFO : Epoch 2014 | loss: 0.022756 | val_loss: 0.0230238 | Time: 9061.42 ms [2021-12-19 16:37:17 main:574] : INFO : Epoch 2015 | loss: 0.0227406 | val_loss: 0.0221029 | Time: 19513.9 ms [2021-12-19 16:37:36 main:574] : INFO : Epoch 2016 | loss: 0.0216218 | val_loss: 0.0210124 | Time: 19824.7 ms [2021-12-19 16:37:56 main:574] : INFO : Epoch 2017 | loss: 0.0208392 | val_loss: 0.0203143 | Time: 19969.7 ms [2021-12-19 16:38:16 main:574] : INFO : Epoch 2018 | loss: 0.0202053 | val_loss: 0.0197567 | Time: 19540.7 ms [2021-12-19 16:38:36 main:574] : INFO : Epoch 2019 | loss: 0.0196253 | val_loss: 0.0191767 | Time: 19814 ms [2021-12-19 16:38:56 main:574] : INFO : Epoch 2020 | loss: 0.0191526 | val_loss: 0.0189354 | Time: 20293.6 ms [2021-12-19 16:39:17 main:574] : INFO : Epoch 2021 | loss: 0.0193427 | val_loss: 0.019263 | Time: 20331.4 ms [2021-12-19 16:39:40 main:574] : INFO : Epoch 2022 | loss: 0.0200103 | val_loss: 0.0210326 | Time: 23041.9 ms [2021-12-19 16:40:51 main:574] : INFO : Epoch 2023 | loss: 0.0205484 | val_loss: 0.0196277 | Time: 20321.1 ms [2021-12-19 16:41:24 main:574] : INFO : Epoch 2024 | loss: 0.0195826 | val_loss: 0.0202701 | Time: 33078.1 ms [2021-12-19 16:41:43 main:574] : INFO : Epoch 2025 | loss: 0.0216031 | val_loss: 0.0240593 | Time: 19156.7 ms [2021-12-19 16:42:02 main:574] : INFO : Epoch 2026 | loss: 0.0243535 | val_loss: 0.0240317 | Time: 18816.7 ms [2021-12-19 16:42:22 main:574] : INFO : Epoch 2027 | loss: 0.0237368 | val_loss: 0.0237108 | Time: 20481 ms [2021-12-19 16:42:32 main:574] : INFO : Epoch 2028 | loss: 0.0229221 | val_loss: 0.0221267 | Time: 9247.73 ms [2021-12-19 16:43:03 main:574] : INFO : Epoch 2029 | loss: 0.021488 | val_loss: 0.0209564 | Time: 31294 ms [2021-12-19 16:43:24 main:574] : INFO : Epoch 2030 | loss: 0.0218128 | val_loss: 0.0230159 | Time: 21237.8 ms [2021-12-19 16:43:43 main:574] : INFO : Epoch 2031 | loss: 0.022921 | val_loss: 0.0217789 | Time: 18986.7 ms [2021-12-19 16:44:04 main:574] : INFO : Epoch 2032 | loss: 0.0244297 | val_loss: 0.0299045 | Time: 20393.6 ms [2021-12-19 16:44:23 main:574] : INFO : Epoch 2033 | loss: 0.0301222 | val_loss: 0.030162 | Time: 19391.6 ms [2021-12-19 16:44:47 main:574] : INFO : Epoch 2034 | loss: 0.0301313 | val_loss: 0.0299621 | Time: 23538.7 ms [2021-12-19 16:45:06 main:574] : INFO : Epoch 2035 | loss: 0.0298572 | val_loss: 0.0296004 | Time: 19285.4 ms [2021-12-19 16:45:15 main:574] : INFO : Epoch 2036 | loss: 0.0293532 | val_loss: 0.028936 | Time: 8380.21 ms [2021-12-19 16:45:35 main:574] : INFO : Epoch 2037 | loss: 0.0287361 | val_loss: 0.028321 | Time: 20064.7 ms [2021-12-19 16:45:54 main:574] : INFO : Epoch 2038 | loss: 0.0281647 | val_loss: 0.027755 | Time: 19465.4 ms [2021-12-19 16:46:30 main:574] : INFO : Epoch 2039 | loss: 0.0276883 | val_loss: 0.0279124 | Time: 35353.2 ms [2021-12-19 16:46:50 main:574] : INFO : Epoch 2040 | loss: 0.0276245 | val_loss: 0.0269605 | Time: 19874.1 ms [2021-12-19 16:47:10 main:574] : INFO : Epoch 2041 | loss: 0.0269199 | val_loss: 0.0264853 | Time: 20656.1 ms [2021-12-19 16:47:31 main:574] : INFO : Epoch 2042 | loss: 0.0264211 | val_loss: 0.0260377 | Time: 20598 ms [2021-12-19 16:47:51 main:574] : INFO : Epoch 2043 | loss: 0.0260896 | val_loss: 0.0256942 | Time: 20175.4 ms [2021-12-19 16:48:13 main:574] : INFO : Epoch 2044 | loss: 0.0255956 | val_loss: 0.0253153 | Time: 21963.4 ms [2021-12-19 16:48:34 main:574] : INFO : Epoch 2045 | loss: 0.0253509 | val_loss: 0.0253519 | Time: 20859.1 ms [2021-12-19 16:48:55 main:574] : INFO : Epoch 2046 | loss: 0.0246749 | val_loss: 0.0240386 | Time: 20172.2 ms [2021-12-19 16:49:14 main:574] : INFO : Epoch 2047 | loss: 0.0238727 | val_loss: 0.0232805 | Time: 19547.3 ms [2021-12-19 16:49:34 main:574] : INFO : Epoch 2048 | loss: 0.0233116 | val_loss: 0.022681 | Time: 19835.9 ms [2021-12-19 16:49:34 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.022681 [2021-12-19 16:49:34 main:603] : INFO : Saving end state to config to file [2021-12-19 16:49:35 main:608] : INFO : Success, exiting.. 16:49:35 (8084): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)