Task 10914857
| Name | ParityModified-1639951721-8944-2-0_0 |
| Workunit | 8213982 |
| Created | 20 Jan 2022, 6:48:48 UTC |
| Sent | 6 Feb 2022, 20:06:57 UTC |
| Report deadline | 14 Feb 2022, 20:06:57 UTC |
| Received | 28 Feb 2022, 6:06:06 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 17440 |
| Run time | 6 hours 44 min 31 sec |
| CPU time | 6 hours 30 min 21 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 2,492.74 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.50 GB |
| Peak swap size | 3.59 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 5351.6 ms [2022-02-27 20:15:32 main:574] : INFO : Epoch 1779 | loss: 0.0309065 | val_loss: 0.0309236 | Time: 15433.3 ms [2022-02-27 20:15:41 main:574] : INFO : Epoch 1780 | loss: 0.0309022 | val_loss: 0.0309244 | Time: 9325.07 ms [2022-02-27 20:15:51 main:574] : INFO : Epoch 1781 | loss: 0.0309001 | val_loss: 0.030927 | Time: 9391.17 ms [2022-02-27 20:16:03 main:574] : INFO : Epoch 1782 | loss: 0.0308981 | val_loss: 0.0309263 | Time: 12640.9 ms [2022-02-27 20:16:19 main:574] : INFO : Epoch 1783 | loss: 0.0308904 | val_loss: 0.0309205 | Time: 15376.1 ms [2022-02-27 20:16:30 main:574] : INFO : Epoch 1784 | loss: 0.0308833 | val_loss: 0.0309076 | Time: 10875.4 ms [2022-02-27 20:16:39 main:574] : INFO : Epoch 1785 | loss: 0.0308746 | val_loss: 0.0309193 | Time: 9568.13 ms [2022-02-27 20:16:53 main:574] : INFO : Epoch 1786 | loss: 0.0308641 | val_loss: 0.0309145 | Time: 14118.7 ms [2022-02-27 20:17:09 main:574] : INFO : Epoch 1787 | loss: 0.03086 | val_loss: 0.0309051 | Time: 15507.3 ms [2022-02-27 20:17:20 main:574] : INFO : Epoch 1788 | loss: 0.0308929 | val_loss: 0.0309193 | Time: 11147 ms [2022-02-27 20:17:30 main:574] : INFO : Epoch 1789 | loss: 0.030877 | val_loss: 0.0309053 | Time: 9629.47 ms [2022-02-27 20:17:43 main:574] : INFO : Epoch 1790 | loss: 0.030911 | val_loss: 0.030942 | Time: 13600.8 ms [2022-02-27 20:17:59 main:574] : INFO : Epoch 1791 | loss: 0.0309354 | val_loss: 0.0309429 | Time: 15408.8 ms [2022-02-27 20:18:10 main:574] : INFO : Epoch 1792 | loss: 0.0309183 | val_loss: 0.0309164 | Time: 11009.8 ms [2022-02-27 20:18:19 main:574] : INFO : Epoch 1793 | loss: 0.0308733 | val_loss: 0.0308947 | Time: 9683.18 ms [2022-02-27 20:18:33 main:574] : INFO : Epoch 1794 | loss: 0.0308769 | val_loss: 0.0309334 | Time: 13203.5 ms [2022-02-27 20:18:48 main:574] : INFO : Epoch 1795 | loss: 0.0309042 | val_loss: 0.0309245 | Time: 15152 ms [2022-02-27 20:19:01 main:574] : INFO : Epoch 1796 | loss: 0.0308802 | val_loss: 0.030902 | Time: 12781.6 ms [2022-02-27 20:19:10 main:574] : INFO : Epoch 1797 | loss: 0.0308626 | val_loss: 0.0308948 | Time: 9699.1 ms [2022-02-27 20:19:23 main:574] : INFO : Epoch 1798 | loss: 0.0308522 | val_loss: 0.0308886 | Time: 12429.8 ms [2022-02-27 20:19:38 main:574] : INFO : Epoch 1799 | loss: 0.0308483 | val_loss: 0.030892 | Time: 15576.6 ms [2022-02-27 20:19:51 main:574] : INFO : Epoch 1800 | loss: 0.0308459 | val_loss: 0.030899 | Time: 12482 ms [2022-02-27 20:20:00 main:574] : INFO : Epoch 1801 | loss: 0.0308492 | val_loss: 0.0309051 | Time: 9654.16 ms [2022-02-27 20:20:12 main:574] : INFO : Epoch 1802 | loss: 0.030873 | val_loss: 0.0309155 | Time: 11592 ms [2022-02-27 20:20:28 main:574] : INFO : Epoch 1803 | loss: 0.030882 | val_loss: 0.0309094 | Time: 15513 ms [2022-02-27 20:20:41 main:574] : INFO : Epoch 1804 | loss: 0.0308659 | val_loss: 0.0308978 | Time: 13728.7 ms [2022-02-27 20:20:51 main:574] : INFO : Epoch 1805 | loss: 0.0308533 | val_loss: 0.0309027 | Time: 9540.18 ms [2022-02-27 20:21:03 main:574] : INFO : Epoch 1806 | loss: 0.0308839 | val_loss: 0.0309035 | Time: 11689.9 ms [2022-02-27 20:21:18 main:574] : INFO : Epoch 1807 | loss: 0.0308719 | val_loss: 0.0309024 | Time: 15414.3 ms [2022-02-27 20:21:30 main:574] : INFO : Epoch 1808 | loss: 0.0308587 | val_loss: 0.0308849 | Time: 12440.5 ms [2022-02-27 20:21:40 main:574] : INFO : Epoch 1809 | loss: 0.0308513 | val_loss: 0.0308839 | Time: 9943.27 ms [2022-02-27 20:21:53 main:574] : INFO : Epoch 1810 | loss: 0.030846 | val_loss: 0.0308969 | Time: 12248.2 ms [2022-02-27 20:22:08 main:574] : INFO : Epoch 1811 | loss: 0.0309018 | val_loss: 0.0309208 | Time: 15290.8 ms [2022-02-27 20:22:21 main:574] : INFO : Epoch 1812 | loss: 0.0308809 | val_loss: 0.0308978 | Time: 13465.2 ms [2022-02-27 20:22:31 main:574] : INFO : Epoch 1813 | loss: 0.0308582 | val_loss: 0.0309001 | Time: 9356.63 ms [2022-02-27 20:22:43 main:574] : INFO : Epoch 1814 | loss: 0.0310984 | val_loss: 0.0312337 | Time: 11840.2 ms [2022-02-27 20:22:57 main:574] : INFO : Epoch 1815 | loss: 0.031238 | val_loss: 0.0312352 | Time: 14840.2 ms [2022-02-27 20:23:12 main:574] : INFO : Epoch 1816 | loss: 0.031239 | val_loss: 0.0312345 | Time: 14174.9 ms [2022-02-27 20:23:21 main:574] : INFO : Epoch 1817 | loss: 0.03124 | val_loss: 0.0312405 | Time: 9481.33 ms [2022-02-27 20:23:33 main:574] : INFO : Epoch 1818 | loss: 0.0312433 | val_loss: 0.0312406 | Time: 11604.6 ms [2022-02-27 20:23:48 main:574] : INFO : Epoch 1819 | loss: 0.0312404 | val_loss: 0.0312392 | Time: 15252.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 21:15:27 main:435] : INFO : Set logging level to 1 [2022-02-27 21:15:27 main:441] : INFO : Running in BOINC Client mode [2022-02-27 21:15:27 main:444] : INFO : Resolving all filenames [2022-02-27 21:15:27 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 21:15:27 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 21:15:27 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 21:15:27 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 21:15:27 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 21:15:27 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 21:15:27 main:474] : INFO : Configuration: [2022-02-27 21:15:27 main:475] : INFO : Model type: GRU [2022-02-27 21:15:27 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 21:15:27 main:477] : INFO : Max Epochs: 2048 [2022-02-27 21:15:27 main:478] : INFO : Batch Size: 128 [2022-02-27 21:15:27 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 21:15:27 main:480] : INFO : Patience: 10 [2022-02-27 21:15:27 main:481] : INFO : Hidden Width: 12 [2022-02-27 21:15:27 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 21:15:27 main:483] : INFO : # Backend Layers: 4 [2022-02-27 21:15:27 main:484] : INFO : # Threads: 1 [2022-02-27 21:15:27 main:486] : INFO : Preparing Dataset [2022-02-27 21:15:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 21:15:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 21:15:29 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 21:15:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 21:15:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 21:15:30 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 21:15:30 main:494] : INFO : Creating Model [2022-02-27 21:15:30 main:507] : INFO : Preparing config file [2022-02-27 21:15:30 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 21:15:30 main:512] : INFO : Loading config [2022-02-27 21:15:30 main:514] : INFO : Loading state [2022-02-27 21:15:30 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 21:15:30 main:562] : INFO : Starting Training [2022-02-27 21:15:45 main:574] : INFO : Epoch 1820 | loss: 0.0312425 | val_loss: 0.0311939 | Time: 15141.3 ms [2022-02-27 21:15:50 main:574] : INFO : Epoch 1821 | loss: 0.0311634 | val_loss: 0.0311315 | Time: 5040.42 ms [2022-02-27 21:15:58 main:574] : INFO : Epoch 1822 | loss: 0.0310872 | val_loss: 0.0310378 | Time: 7469.36 ms [2022-02-27 21:16:05 main:574] : INFO : Epoch 1823 | loss: 0.0309786 | val_loss: 0.0309364 | Time: 7564.01 ms [2022-02-27 21:16:13 main:574] : INFO : Epoch 1824 | loss: 0.0309327 | val_loss: 0.0309481 | Time: 7565.73 ms [2022-02-27 21:16:22 main:574] : INFO : Epoch 1825 | loss: 0.0309307 | val_loss: 0.0309516 | Time: 8916.23 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 21:41:43 main:435] : INFO : Set logging level to 1 [2022-02-27 21:41:43 main:441] : INFO : Running in BOINC Client mode [2022-02-27 21:41:43 main:444] : INFO : Resolving all filenames [2022-02-27 21:41:43 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 21:41:43 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 21:41:43 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 21:41:43 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 21:41:43 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 21:41:43 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 21:41:43 main:474] : INFO : Configuration: [2022-02-27 21:41:43 main:475] : INFO : Model type: GRU [2022-02-27 21:41:43 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 21:41:43 main:477] : INFO : Max Epochs: 2048 [2022-02-27 21:41:43 main:478] : INFO : Batch Size: 128 [2022-02-27 21:41:43 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 21:41:43 main:480] : INFO : Patience: 10 [2022-02-27 21:41:43 main:481] : INFO : Hidden Width: 12 [2022-02-27 21:41:43 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 21:41:43 main:483] : INFO : # Backend Layers: 4 [2022-02-27 21:41:43 main:484] : INFO : # Threads: 1 [2022-02-27 21:41:43 main:486] : INFO : Preparing Dataset [2022-02-27 21:41:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 21:41:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 21:41:45 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 21:41:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 21:41:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 21:41:45 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 21:41:45 main:494] : INFO : Creating Model [2022-02-27 21:41:45 main:507] : INFO : Preparing config file [2022-02-27 21:41:45 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 21:41:45 main:512] : INFO : Loading config [2022-02-27 21:41:45 main:514] : INFO : Loading state [2022-02-27 21:41:46 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 21:41:46 main:562] : INFO : Starting Training [2022-02-27 21:41:50 main:574] : INFO : Epoch 1820 | loss: 0.0312404 | val_loss: 0.0311759 | Time: 4098.37 ms [2022-02-27 21:41:54 main:574] : INFO : Epoch 1821 | loss: 0.0311581 | val_loss: 0.0311316 | Time: 3854.77 ms [2022-02-27 21:42:01 main:574] : INFO : Epoch 1822 | loss: 0.0310841 | val_loss: 0.0310175 | Time: 7809.02 ms [2022-02-27 21:42:09 main:574] : INFO : Epoch 1823 | loss: 0.0309845 | val_loss: 0.030954 | Time: 7809.08 ms [2022-02-27 21:42:17 main:574] : INFO : Epoch 1824 | loss: 0.0309235 | val_loss: 0.0309117 | Time: 7538.8 ms [2022-02-27 21:42:24 main:574] : INFO : Epoch 1825 | loss: 0.0308904 | val_loss: 0.0309015 | Time: 7502.48 ms [2022-02-27 21:42:31 main:574] : INFO : Epoch 1826 | loss: 0.030919 | val_loss: 0.0309506 | Time: 6988.39 ms [2022-02-27 21:42:39 main:574] : INFO : Epoch 1827 | loss: 0.0309431 | val_loss: 0.030951 | Time: 7447.12 ms [2022-02-27 21:42:47 main:574] : INFO : Epoch 1828 | loss: 0.0309282 | val_loss: 0.0309332 | Time: 8698.08 ms [2022-02-27 21:42:56 main:574] : INFO : Epoch 1829 | loss: 0.0309144 | val_loss: 0.0309193 | Time: 8216.18 ms [2022-02-27 21:43:05 main:574] : INFO : Epoch 1830 | loss: 0.0309056 | val_loss: 0.0309135 | Time: 9229.22 ms [2022-02-27 21:43:13 main:574] : INFO : Epoch 1831 | loss: 0.0308968 | val_loss: 0.0309118 | Time: 8144.75 ms [2022-02-27 21:43:21 main:574] : INFO : Epoch 1832 | loss: 0.030893 | val_loss: 0.0309155 | Time: 7839.74 ms [2022-02-27 21:43:29 main:574] : INFO : Epoch 1833 | loss: 0.0308923 | val_loss: 0.0309123 | Time: 8348.92 ms [2022-02-27 21:43:38 main:574] : INFO : Epoch 1834 | loss: 0.0308914 | val_loss: 0.0309263 | Time: 8394.45 ms [2022-02-27 21:43:49 main:574] : INFO : Epoch 1835 | loss: 0.0309028 | val_loss: 0.0309245 | Time: 11372.3 ms [2022-02-27 21:44:01 main:574] : INFO : Epoch 1836 | loss: 0.0309027 | val_loss: 0.0309179 | Time: 11622.8 ms [2022-02-27 21:44:12 main:574] : INFO : Epoch 1837 | loss: 0.0308964 | val_loss: 0.0309283 | Time: 11142.7 ms [2022-02-27 21:44:22 main:574] : INFO : Epoch 1838 | loss: 0.0309174 | val_loss: 0.0309422 | Time: 10032.8 ms [2022-02-27 21:44:32 main:574] : INFO : Epoch 1839 | loss: 0.0310075 | val_loss: 0.0310563 | Time: 10260.4 ms [2022-02-27 21:44:42 main:574] : INFO : Epoch 1840 | loss: 0.0310429 | val_loss: 0.0310211 | Time: 10091.6 ms [2022-02-27 21:44:52 main:574] : INFO : Epoch 1841 | loss: 0.031004 | val_loss: 0.0309855 | Time: 10382.1 ms [2022-02-27 21:45:05 main:574] : INFO : Epoch 1842 | loss: 0.0309768 | val_loss: 0.030988 | Time: 12561.9 ms [2022-02-27 21:45:15 main:574] : INFO : Epoch 1843 | loss: 0.0309787 | val_loss: 0.0309727 | Time: 10314.9 ms [2022-02-27 21:45:26 main:574] : INFO : Epoch 1844 | loss: 0.0309895 | val_loss: 0.0309953 | Time: 10143.8 ms [2022-02-27 21:45:38 main:574] : INFO : Epoch 1845 | loss: 0.031042 | val_loss: 0.0310933 | Time: 12141 ms [2022-02-27 21:45:50 main:574] : INFO : Epoch 1846 | loss: 0.031139 | val_loss: 0.0311405 | Time: 12299.5 ms [2022-02-27 21:46:00 main:574] : INFO : Epoch 1847 | loss: 0.031133 | val_loss: 0.0311232 | Time: 10290.4 ms [2022-02-27 21:46:20 main:574] : INFO : Epoch 1848 | loss: 0.0311221 | val_loss: 0.0311171 | Time: 19248.9 ms [2022-02-27 21:46:29 main:574] : INFO : Epoch 1849 | loss: 0.0311176 | val_loss: 0.0311111 | Time: 9905.97 ms [2022-02-27 21:46:45 main:574] : INFO : Epoch 1850 | loss: 0.0311027 | val_loss: 0.0311022 | Time: 15594 ms [2022-02-27 21:46:57 main:574] : INFO : Epoch 1851 | loss: 0.03109 | val_loss: 0.0310868 | Time: 12112.7 ms [2022-02-27 21:47:08 main:574] : INFO : Epoch 1852 | loss: 0.0310795 | val_loss: 0.0310793 | Time: 11058.1 ms [2022-02-27 21:47:19 main:574] : INFO : Epoch 1853 | loss: 0.0310758 | val_loss: 0.0310781 | Time: 10289.9 ms [2022-02-27 21:47:33 main:574] : INFO : Epoch 1854 | loss: 0.031069 | val_loss: 0.031068 | Time: 14637.4 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 21:55:38 main:435] : INFO : Set logging level to 1 [2022-02-27 21:55:38 main:441] : INFO : Running in BOINC Client mode [2022-02-27 21:55:38 main:444] : INFO : Resolving all filenames [2022-02-27 21:55:38 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 21:55:38 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 21:55:38 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 21:55:38 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 21:55:38 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 21:55:38 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 21:55:38 main:474] : INFO : Configuration: [2022-02-27 21:55:38 main:475] : INFO : Model type: GRU [2022-02-27 21:55:38 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 21:55:38 main:477] : INFO : Max Epochs: 2048 [2022-02-27 21:55:38 main:478] : INFO : Batch Size: 128 [2022-02-27 21:55:38 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 21:55:38 main:480] : INFO : Patience: 10 [2022-02-27 21:55:38 main:481] : INFO : Hidden Width: 12 [2022-02-27 21:55:38 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 21:55:38 main:483] : INFO : # Backend Layers: 4 [2022-02-27 21:55:38 main:484] : INFO : # Threads: 1 [2022-02-27 21:55:38 main:486] : INFO : Preparing Dataset [2022-02-27 21:55:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 21:55:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 21:55:40 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 21:55:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 21:55:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 21:55:40 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 21:55:40 main:494] : INFO : Creating Model [2022-02-27 21:55:40 main:507] : INFO : Preparing config file [2022-02-27 21:55:40 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 21:55:40 main:512] : INFO : Loading config [2022-02-27 21:55:40 main:514] : INFO : Loading state [2022-02-27 21:55:41 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 21:55:41 main:562] : INFO : Starting Training [2022-02-27 21:55:45 main:574] : INFO : Epoch 1843 | loss: 0.0310082 | val_loss: 0.0309564 | Time: 4229.49 ms [2022-02-27 21:55:50 main:574] : INFO : Epoch 1844 | loss: 0.0309527 | val_loss: 0.0309395 | Time: 4342.55 ms [2022-02-27 21:55:57 main:574] : INFO : Epoch 1845 | loss: 0.0309399 | val_loss: 0.030964 | Time: 6982.55 ms [2022-02-27 21:56:05 main:574] : INFO : Epoch 1846 | loss: 0.0309772 | val_loss: 0.0309739 | Time: 7785.73 ms [2022-02-27 21:56:12 main:574] : INFO : Epoch 1847 | loss: 0.0309732 | val_loss: 0.0309639 | Time: 7206.58 ms [2022-02-27 21:56:19 main:574] : INFO : Epoch 1848 | loss: 0.0309594 | val_loss: 0.0309563 | Time: 7350.22 ms [2022-02-27 21:56:26 main:574] : INFO : Epoch 1849 | loss: 0.0309458 | val_loss: 0.0309509 | Time: 6783.85 ms [2022-02-27 21:56:34 main:574] : INFO : Epoch 1850 | loss: 0.0309351 | val_loss: 0.0309499 | Time: 7567.34 ms [2022-02-27 21:56:41 main:574] : INFO : Epoch 1851 | loss: 0.0309417 | val_loss: 0.0309612 | Time: 7843.89 ms [2022-02-27 21:56:49 main:574] : INFO : Epoch 1852 | loss: 0.0309434 | val_loss: 0.0309526 | Time: 7978.33 ms [2022-02-27 21:56:57 main:574] : INFO : Epoch 1853 | loss: 0.0309335 | val_loss: 0.0309466 | Time: 7855.48 ms [2022-02-27 21:57:06 main:574] : INFO : Epoch 1854 | loss: 0.0309233 | val_loss: 0.0309416 | Time: 8864.28 ms [2022-02-27 21:57:15 main:574] : INFO : Epoch 1855 | loss: 0.0309169 | val_loss: 0.0309437 | Time: 8705.76 ms [2022-02-27 21:57:24 main:574] : INFO : Epoch 1856 | loss: 0.0309135 | val_loss: 0.0309345 | Time: 9409.45 ms [2022-02-27 21:57:34 main:574] : INFO : Epoch 1857 | loss: 0.0309114 | val_loss: 0.0309377 | Time: 9992.52 ms [2022-02-27 21:57:49 main:574] : INFO : Epoch 1858 | loss: 0.0309149 | val_loss: 0.0309466 | Time: 14458.4 ms [2022-02-27 21:58:00 main:574] : INFO : Epoch 1859 | loss: 0.0309252 | val_loss: 0.0309555 | Time: 11247.3 ms [2022-02-27 21:58:11 main:574] : INFO : Epoch 1860 | loss: 0.0309403 | val_loss: 0.0309499 | Time: 10661.9 ms [2022-02-27 21:58:21 main:574] : INFO : Epoch 1861 | loss: 0.0309475 | val_loss: 0.0309565 | Time: 10067.2 ms [2022-02-27 21:58:31 main:574] : INFO : Epoch 1862 | loss: 0.0309501 | val_loss: 0.0309815 | Time: 9894.14 ms [2022-02-27 21:58:46 main:574] : INFO : Epoch 1863 | loss: 0.0309921 | val_loss: 0.0309891 | Time: 15018.6 ms [2022-02-27 21:58:57 main:574] : INFO : Epoch 1864 | loss: 0.0309802 | val_loss: 0.0309733 | Time: 11029 ms [2022-02-27 21:59:08 main:574] : INFO : Epoch 1865 | loss: 0.0309704 | val_loss: 0.0309672 | Time: 10865.8 ms [2022-02-27 21:59:17 main:574] : INFO : Epoch 1866 | loss: 0.0309499 | val_loss: 0.0309514 | Time: 9906.35 ms [2022-02-27 21:59:30 main:574] : INFO : Epoch 1867 | loss: 0.0309449 | val_loss: 0.0309567 | Time: 12487.8 ms [2022-02-27 21:59:44 main:574] : INFO : Epoch 1868 | loss: 0.0309397 | val_loss: 0.0309517 | Time: 14145.4 ms [2022-02-27 21:59:54 main:574] : INFO : Epoch 1869 | loss: 0.0309381 | val_loss: 0.0309472 | Time: 9636.57 ms [2022-02-27 22:00:04 main:574] : INFO : Epoch 1870 | loss: 0.0309341 | val_loss: 0.0309499 | Time: 10737.3 ms [2022-02-27 22:00:16 main:574] : INFO : Epoch 1871 | loss: 0.0309902 | val_loss: 0.0310848 | Time: 11620.7 ms [2022-02-27 22:00:32 main:574] : INFO : Epoch 1872 | loss: 0.0311536 | val_loss: 0.0311798 | Time: 15912.4 ms [2022-02-27 22:00:43 main:574] : INFO : Epoch 1873 | loss: 0.0311845 | val_loss: 0.0311877 | Time: 10547.3 ms [2022-02-27 22:00:52 main:574] : INFO : Epoch 1874 | loss: 0.0311837 | val_loss: 0.0311873 | Time: 9776.35 ms [2022-02-27 22:01:04 main:574] : INFO : Epoch 1875 | loss: 0.031184 | val_loss: 0.0311828 | Time: 11570.1 ms [2022-02-27 22:01:20 main:574] : INFO : Epoch 1876 | loss: 0.0311798 | val_loss: 0.0311783 | Time: 16004.4 ms [2022-02-27 22:01:31 main:574] : INFO : Epoch 1877 | loss: 0.0311777 | val_loss: 0.0311773 | Time: 11153.6 ms [2022-02-27 22:01:41 main:574] : INFO : Epoch 1878 | loss: 0.0311765 | val_loss: 0.0311768 | Time: 9747.38 ms [2022-02-27 22:01:53 main:574] : INFO : Epoch 1879 | loss: 0.0311805 | val_loss: 0.0311864 | Time: 12360 ms [2022-02-27 22:02:09 main:574] : INFO : Epoch 1880 | loss: 0.0311814 | val_loss: 0.0311835 | Time: 15716.2 ms [2022-02-27 22:02:19 main:574] : INFO : Epoch 1881 | loss: 0.031178 | val_loss: 0.0311802 | Time: 10033.9 ms [2022-02-27 22:02:29 main:574] : INFO : Epoch 1882 | loss: 0.0311784 | val_loss: 0.0311824 | Time: 10558 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 22:08:29 main:435] : INFO : Set logging level to 1 [2022-02-27 22:08:29 main:441] : INFO : Running in BOINC Client mode [2022-02-27 22:08:29 main:444] : INFO : Resolving all filenames [2022-02-27 22:08:29 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 22:08:29 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 22:08:29 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 22:08:29 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 22:08:29 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 22:08:29 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 22:08:29 main:474] : INFO : Configuration: [2022-02-27 22:08:29 main:475] : INFO : Model type: GRU [2022-02-27 22:08:29 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 22:08:29 main:477] : INFO : Max Epochs: 2048 [2022-02-27 22:08:29 main:478] : INFO : Batch Size: 128 [2022-02-27 22:08:29 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 22:08:29 main:480] : INFO : Patience: 10 [2022-02-27 22:08:29 main:481] : INFO : Hidden Width: 12 [2022-02-27 22:08:29 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 22:08:29 main:483] : INFO : # Backend Layers: 4 [2022-02-27 22:08:29 main:484] : INFO : # Threads: 1 [2022-02-27 22:08:29 main:486] : INFO : Preparing Dataset [2022-02-27 22:08:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 22:08:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 22:08:31 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 22:08:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 22:08:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 22:08:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 22:08:31 main:494] : INFO : Creating Model [2022-02-27 22:08:31 main:507] : INFO : Preparing config file [2022-02-27 22:08:31 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 22:08:31 main:512] : INFO : Loading config [2022-02-27 22:08:31 main:514] : INFO : Loading state [2022-02-27 22:08:32 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 22:08:32 main:562] : INFO : Starting Training [2022-02-27 22:08:36 main:574] : INFO : Epoch 1882 | loss: 0.0311787 | val_loss: 0.0311294 | Time: 4018.51 ms [2022-02-27 22:08:40 main:574] : INFO : Epoch 1883 | loss: 0.0310991 | val_loss: 0.0310789 | Time: 4232.39 ms [2022-02-27 22:08:47 main:574] : INFO : Epoch 1884 | loss: 0.0310562 | val_loss: 0.0310374 | Time: 6965.45 ms [2022-02-27 22:08:54 main:574] : INFO : Epoch 1885 | loss: 0.0310302 | val_loss: 0.031013 | Time: 7215.41 ms [2022-02-27 22:09:02 main:574] : INFO : Epoch 1886 | loss: 0.0310066 | val_loss: 0.031006 | Time: 7612.68 ms [2022-02-27 22:09:09 main:574] : INFO : Epoch 1887 | loss: 0.0310211 | val_loss: 0.0310271 | Time: 7474.94 ms [2022-02-27 22:09:16 main:574] : INFO : Epoch 1888 | loss: 0.031052 | val_loss: 0.0310726 | Time: 6626.12 ms [2022-02-27 22:09:24 main:574] : INFO : Epoch 1889 | loss: 0.0310591 | val_loss: 0.0310535 | Time: 7496.1 ms [2022-02-27 22:09:32 main:574] : INFO : Epoch 1890 | loss: 0.0310438 | val_loss: 0.031036 | Time: 8260.13 ms [2022-02-27 22:09:40 main:574] : INFO : Epoch 1891 | loss: 0.0310375 | val_loss: 0.0310239 | Time: 8061.04 ms [2022-02-27 22:09:48 main:574] : INFO : Epoch 1892 | loss: 0.0310235 | val_loss: 0.0310187 | Time: 8028.34 ms [2022-02-27 22:09:56 main:574] : INFO : Epoch 1893 | loss: 0.0310434 | val_loss: 0.0310988 | Time: 8378.96 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 22:30:02 main:435] : INFO : Set logging level to 1 [2022-02-27 22:30:02 main:441] : INFO : Running in BOINC Client mode [2022-02-27 22:30:02 main:444] : INFO : Resolving all filenames [2022-02-27 22:30:02 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 22:30:02 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 22:30:02 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 22:30:02 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 22:30:02 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 22:30:02 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 22:30:02 main:474] : INFO : Configuration: [2022-02-27 22:30:02 main:475] : INFO : Model type: GRU [2022-02-27 22:30:02 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 22:30:02 main:477] : INFO : Max Epochs: 2048 [2022-02-27 22:30:02 main:478] : INFO : Batch Size: 128 [2022-02-27 22:30:02 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 22:30:02 main:480] : INFO : Patience: 10 [2022-02-27 22:30:02 main:481] : INFO : Hidden Width: 12 [2022-02-27 22:30:02 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 22:30:02 main:483] : INFO : # Backend Layers: 4 [2022-02-27 22:30:02 main:484] : INFO : # Threads: 1 [2022-02-27 22:30:02 main:486] : INFO : Preparing Dataset [2022-02-27 22:30:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 22:30:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 22:30:04 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 22:30:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 22:30:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 22:30:04 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 22:30:04 main:494] : INFO : Creating Model [2022-02-27 22:30:04 main:507] : INFO : Preparing config file [2022-02-27 22:30:04 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 22:30:04 main:512] : INFO : Loading config [2022-02-27 22:30:04 main:514] : INFO : Loading state [2022-02-27 22:30:05 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 22:30:05 main:562] : INFO : Starting Training [2022-02-27 22:30:09 main:574] : INFO : Epoch 1882 | loss: 0.031188 | val_loss: 0.0311483 | Time: 4016.09 ms [2022-02-27 22:30:13 main:574] : INFO : Epoch 1883 | loss: 0.031108 | val_loss: 0.0310751 | Time: 4330.02 ms [2022-02-27 22:30:20 main:574] : INFO : Epoch 1884 | loss: 0.0310499 | val_loss: 0.0310208 | Time: 6711.21 ms [2022-02-27 22:30:27 main:574] : INFO : Epoch 1885 | loss: 0.0310112 | val_loss: 0.0309944 | Time: 7151.98 ms [2022-02-27 22:30:35 main:574] : INFO : Epoch 1886 | loss: 0.03099 | val_loss: 0.0309709 | Time: 7238.01 ms [2022-02-27 22:30:42 main:574] : INFO : Epoch 1887 | loss: 0.0309798 | val_loss: 0.0309625 | Time: 7125.48 ms [2022-02-27 22:30:49 main:574] : INFO : Epoch 1888 | loss: 0.0309553 | val_loss: 0.0309545 | Time: 6862.54 ms [2022-02-27 22:30:56 main:574] : INFO : Epoch 1889 | loss: 0.0309431 | val_loss: 0.0309404 | Time: 7020.35 ms [2022-02-27 22:31:04 main:574] : INFO : Epoch 1890 | loss: 0.0309273 | val_loss: 0.0309383 | Time: 8880.3 ms [2022-02-27 22:31:12 main:574] : INFO : Epoch 1891 | loss: 0.030953 | val_loss: 0.0309784 | Time: 7684.24 ms [2022-02-27 22:31:20 main:574] : INFO : Epoch 1892 | loss: 0.0309633 | val_loss: 0.0309588 | Time: 7984.8 ms [2022-02-27 22:31:28 main:574] : INFO : Epoch 1893 | loss: 0.0309513 | val_loss: 0.0309739 | Time: 7918.34 ms [2022-02-27 22:31:36 main:574] : INFO : Epoch 1894 | loss: 0.0309387 | val_loss: 0.0309386 | Time: 7735.95 ms [2022-02-27 22:31:44 main:574] : INFO : Epoch 1895 | loss: 0.0309293 | val_loss: 0.030938 | Time: 7923.39 ms [2022-02-27 22:31:53 main:574] : INFO : Epoch 1896 | loss: 0.0309322 | val_loss: 0.0309429 | Time: 8868.25 ms [2022-02-27 22:32:03 main:574] : INFO : Epoch 1897 | loss: 0.0309158 | val_loss: 0.0309164 | Time: 10457.8 ms [2022-02-27 22:32:15 main:574] : INFO : Epoch 1898 | loss: 0.030907 | val_loss: 0.0309399 | Time: 12013.8 ms [2022-02-27 22:32:25 main:574] : INFO : Epoch 1899 | loss: 0.0309279 | val_loss: 0.0309339 | Time: 10099.4 ms [2022-02-27 22:32:36 main:574] : INFO : Epoch 1900 | loss: 0.0309217 | val_loss: 0.0309349 | Time: 11206.8 ms [2022-02-27 22:32:48 main:574] : INFO : Epoch 1901 | loss: 0.0309153 | val_loss: 0.0309316 | Time: 11546.8 ms [2022-02-27 22:32:58 main:574] : INFO : Epoch 1902 | loss: 0.0309234 | val_loss: 0.0309669 | Time: 9961.48 ms [2022-02-27 22:33:09 main:574] : INFO : Epoch 1903 | loss: 0.0309661 | val_loss: 0.0309592 | Time: 11028.4 ms [2022-02-27 22:33:23 main:574] : INFO : Epoch 1904 | loss: 0.0309513 | val_loss: 0.0309674 | Time: 13809.5 ms [2022-02-27 22:33:34 main:574] : INFO : Epoch 1905 | loss: 0.0309531 | val_loss: 0.0309612 | Time: 10986.9 ms [2022-02-27 22:33:43 main:574] : INFO : Epoch 1906 | loss: 0.0309374 | val_loss: 0.0309436 | Time: 9704.59 ms [2022-02-27 22:33:53 main:574] : INFO : Epoch 1907 | loss: 0.0309247 | val_loss: 0.0309417 | Time: 9903.87 ms [2022-02-27 22:34:05 main:574] : INFO : Epoch 1908 | loss: 0.0309244 | val_loss: 0.030931 | Time: 12109.4 ms [2022-02-27 22:34:20 main:574] : INFO : Epoch 1909 | loss: 0.0309763 | val_loss: 0.0309805 | Time: 14211.7 ms [2022-02-27 22:34:29 main:574] : INFO : Epoch 1910 | loss: 0.0309473 | val_loss: 0.0309526 | Time: 9698.78 ms [2022-02-27 22:34:39 main:574] : INFO : Epoch 1911 | loss: 0.0309141 | val_loss: 0.0309364 | Time: 9922.97 ms [2022-02-27 22:34:53 main:574] : INFO : Epoch 1912 | loss: 0.0309288 | val_loss: 0.0309571 | Time: 13248.7 ms [2022-02-27 22:35:08 main:574] : INFO : Epoch 1913 | loss: 0.0309313 | val_loss: 0.0309494 | Time: 15170.5 ms [2022-02-27 22:35:20 main:574] : INFO : Epoch 1914 | loss: 0.0309138 | val_loss: 0.0309513 | Time: 12143.3 ms [2022-02-27 22:35:34 main:574] : INFO : Epoch 1915 | loss: 0.0309052 | val_loss: 0.0309325 | Time: 14243.9 ms [2022-02-27 22:35:48 main:574] : INFO : Epoch 1916 | loss: 0.030896 | val_loss: 0.0309248 | Time: 13362.8 ms [2022-02-27 22:35:57 main:574] : INFO : Epoch 1917 | loss: 0.0308892 | val_loss: 0.0309179 | Time: 9880.01 ms [2022-02-27 22:36:08 main:574] : INFO : Epoch 1918 | loss: 0.0308865 | val_loss: 0.030914 | Time: 10856.9 ms [2022-02-27 22:36:23 main:574] : INFO : Epoch 1919 | loss: 0.0308902 | val_loss: 0.0309147 | Time: 14668.7 ms [2022-02-27 22:36:37 main:574] : INFO : Epoch 1920 | loss: 0.0308865 | val_loss: 0.0309225 | Time: 14106.1 ms [2022-02-27 22:36:47 main:574] : INFO : Epoch 1921 | loss: 0.0308827 | val_loss: 0.0309093 | Time: 9971.36 ms [2022-02-27 22:36:57 main:574] : INFO : Epoch 1922 | loss: 0.0308808 | val_loss: 0.0309122 | Time: 9541.26 ms [2022-02-27 22:37:11 main:574] : INFO : Epoch 1923 | loss: 0.0308811 | val_loss: 0.0309056 | Time: 13890.3 ms [2022-02-27 22:37:26 main:574] : INFO : Epoch 1924 | loss: 0.0308781 | val_loss: 0.0309059 | Time: 15390.3 ms [2022-02-27 22:37:37 main:574] : INFO : Epoch 1925 | loss: 0.0308843 | val_loss: 0.0309294 | Time: 10900 ms [2022-02-27 22:37:47 main:574] : INFO : Epoch 1926 | loss: 0.0309024 | val_loss: 0.0309564 | Time: 9781.79 ms [2022-02-27 22:37:58 main:574] : INFO : Epoch 1927 | loss: 0.0309134 | val_loss: 0.0309188 | Time: 11738.5 ms [2022-02-27 22:38:14 main:574] : INFO : Epoch 1928 | loss: 0.0309012 | val_loss: 0.0309074 | Time: 15888.4 ms [2022-02-27 22:38:26 main:574] : INFO : Epoch 1929 | loss: 0.0308892 | val_loss: 0.0309021 | Time: 11339.6 ms [2022-02-27 22:38:35 main:574] : INFO : Epoch 1930 | loss: 0.0308873 | val_loss: 0.0309113 | Time: 9581.09 ms [2022-02-27 22:38:47 main:574] : INFO : Epoch 1931 | loss: 0.0308786 | val_loss: 0.0308953 | Time: 11402.1 ms [2022-02-27 22:39:02 main:574] : INFO : Epoch 1932 | loss: 0.0308703 | val_loss: 0.030914 | Time: 15559.4 ms [2022-02-27 22:39:14 main:574] : INFO : Epoch 1933 | loss: 0.0309312 | val_loss: 0.0309925 | Time: 11653.6 ms [2022-02-27 22:39:23 main:574] : INFO : Epoch 1934 | loss: 0.0309683 | val_loss: 0.0309608 | Time: 9572.75 ms [2022-02-27 22:39:36 main:574] : INFO : Epoch 1935 | loss: 0.0309316 | val_loss: 0.0309292 | Time: 12861.5 ms [2022-02-27 22:39:52 main:574] : INFO : Epoch 1936 | loss: 0.0309055 | val_loss: 0.0309211 | Time: 16008.8 ms [2022-02-27 22:40:07 main:574] : INFO : Epoch 1937 | loss: 0.0309034 | val_loss: 0.0309208 | Time: 14679.6 ms [2022-02-27 22:40:19 main:574] : INFO : Epoch 1938 | loss: 0.0308942 | val_loss: 0.030917 | Time: 11768.5 ms [2022-02-27 22:40:34 main:574] : INFO : Epoch 1939 | loss: 0.030889 | val_loss: 0.0309025 | Time: 15293.1 ms [2022-02-27 22:40:49 main:574] : INFO : Epoch 1940 | loss: 0.0308846 | val_loss: 0.0309089 | Time: 14713.8 ms [2022-02-27 22:40:58 main:574] : INFO : Epoch 1941 | loss: 0.030877 | val_loss: 0.0308924 | Time: 9382.4 ms [2022-02-27 22:41:08 main:574] : INFO : Epoch 1942 | loss: 0.0308686 | val_loss: 0.0308911 | Time: 9761.28 ms [2022-02-27 22:41:22 main:574] : INFO : Epoch 1943 | loss: 0.0308735 | val_loss: 0.0309004 | Time: 14295.4 ms [2022-02-27 22:41:38 main:574] : INFO : Epoch 1944 | loss: 0.0308727 | val_loss: 0.030893 | Time: 15429.3 ms [2022-02-27 22:41:48 main:574] : INFO : Epoch 1945 | loss: 0.0308668 | val_loss: 0.0308896 | Time: 10675.7 ms [2022-02-27 22:41:58 main:574] : INFO : Epoch 1946 | loss: 0.0310526 | val_loss: 0.0312233 | Time: 9613.27 ms [2022-02-27 22:42:13 main:574] : INFO : Epoch 1947 | loss: 0.0312285 | val_loss: 0.0312318 | Time: 15492.6 ms [2022-02-27 22:42:28 main:574] : INFO : Epoch 1948 | loss: 0.0312324 | val_loss: 0.0312389 | Time: 14640.4 ms [2022-02-27 22:42:38 main:574] : INFO : Epoch 1949 | loss: 0.0312431 | val_loss: 0.0312431 | Time: 9422.17 ms [2022-02-27 22:42:48 main:574] : INFO : Epoch 1950 | loss: 0.0312443 | val_loss: 0.0312472 | Time: 10536 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-27 23:16:25 main:435] : INFO : Set logging level to 1 [2022-02-27 23:16:25 main:441] : INFO : Running in BOINC Client mode [2022-02-27 23:16:25 main:444] : INFO : Resolving all filenames [2022-02-27 23:16:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 23:16:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 23:16:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 23:16:25 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 23:16:25 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 23:16:25 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 23:16:25 main:474] : INFO : Configuration: [2022-02-27 23:16:25 main:475] : INFO : Model type: GRU [2022-02-27 23:16:25 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 23:16:25 main:477] : INFO : Max Epochs: 2048 [2022-02-27 23:16:25 main:478] : INFO : Batch Size: 128 [2022-02-27 23:16:25 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 23:16:25 main:480] : INFO : Patience: 10 [2022-02-27 23:16:25 main:481] : INFO : Hidden Width: 12 [2022-02-27 23:16:25 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 23:16:25 main:483] : INFO : # Backend Layers: 4 [2022-02-27 23:16:25 main:484] : INFO : # Threads: 1 [2022-02-27 23:16:25 main:486] : INFO : Preparing Dataset [2022-02-27 23:16:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 23:16:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 23:16:27 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 23:16:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 23:16:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 23:16:27 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 23:16:27 main:494] : INFO : Creating Model [2022-02-27 23:16:27 main:507] : INFO : Preparing config file [2022-02-27 23:16:27 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 23:16:27 main:512] : INFO : Loading config [2022-02-27 23:16:27 main:514] : INFO : Loading state [2022-02-27 23:16:28 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 23:16:28 main:562] : INFO : Starting Training [2022-02-27 23:16:32 main:574] : INFO : Epoch 1938 | loss: 0.0309406 | val_loss: 0.0309275 | Time: 3836.12 ms [2022-02-27 23:16:36 main:574] : INFO : Epoch 1939 | loss: 0.0308948 | val_loss: 0.0309154 | Time: 4424.51 ms [2022-02-27 23:16:54 main:574] : INFO : Epoch 1940 | loss: 0.0308932 | val_loss: 0.0309077 | Time: 18097.2 ms [2022-02-27 23:17:03 main:574] : INFO : Epoch 1941 | loss: 0.0308878 | val_loss: 0.0309012 | Time: 8341.5 ms [2022-02-27 23:17:10 main:574] : INFO : Epoch 1942 | loss: 0.0308806 | val_loss: 0.0309055 | Time: 7581.37 ms [2022-02-27 23:17:18 main:574] : INFO : Epoch 1943 | loss: 0.0308786 | val_loss: 0.0309009 | Time: 7395.49 ms [2022-02-27 23:17:25 main:574] : INFO : Epoch 1944 | loss: 0.030868 | val_loss: 0.0309068 | Time: 7285.09 ms [2022-02-27 23:17:32 main:574] : INFO : Epoch 1945 | loss: 0.0308686 | val_loss: 0.0309003 | Time: 7495.91 ms [2022-02-27 23:17:40 main:574] : INFO : Epoch 1946 | loss: 0.0308715 | val_loss: 0.0309181 | Time: 8079.43 ms [2022-02-27 23:17:49 main:574] : INFO : Epoch 1947 | loss: 0.030919 | val_loss: 0.0309589 | Time: 8101.93 ms [2022-02-27 23:17:57 main:574] : INFO : Epoch 1948 | loss: 0.0309788 | val_loss: 0.0310899 | Time: 8144.18 ms [2022-02-27 23:18:06 main:574] : INFO : Epoch 1949 | loss: 0.031059 | val_loss: 0.0310611 | Time: 9287.87 ms [2022-02-27 23:18:15 main:574] : INFO : Epoch 1950 | loss: 0.0310888 | val_loss: 0.0311165 | Time: 8971.59 ms [2022-02-27 23:18:24 main:574] : INFO : Epoch 1951 | loss: 0.0311096 | val_loss: 0.0310963 | Time: 9341.45 ms [2022-02-27 23:18:34 main:574] : INFO : Epoch 1952 | loss: 0.0310831 | val_loss: 0.031066 | Time: 9883.82 ms [2022-02-27 23:18:44 main:574] : INFO : Epoch 1953 | loss: 0.0310699 | val_loss: 0.0310639 | Time: 10233 ms [2022-02-27 23:18:55 main:574] : INFO : Epoch 1954 | loss: 0.0310496 | val_loss: 0.0310483 | Time: 10321.7 ms [2022-02-27 23:19:06 main:574] : INFO : Epoch 1955 | loss: 0.0310327 | val_loss: 0.0310209 | Time: 11261.2 ms [2022-02-27 23:19:19 main:574] : INFO : Epoch 1956 | loss: 0.031006 | val_loss: 0.0310051 | Time: 12446.9 ms [2022-02-27 23:19:29 main:574] : INFO : Epoch 1957 | loss: 0.0310099 | val_loss: 0.03102 | Time: 10198.1 ms [2022-02-27 23:19:39 main:574] : INFO : Epoch 1958 | loss: 0.0309892 | val_loss: 0.030987 | Time: 10201.8 ms [2022-02-27 23:19:51 main:574] : INFO : Epoch 1959 | loss: 0.0309718 | val_loss: 0.0309757 | Time: 12245.3 ms [2022-02-27 23:20:03 main:574] : INFO : Epoch 1960 | loss: 0.0309637 | val_loss: 0.0309686 | Time: 12305.8 ms [2022-02-27 23:20:14 main:574] : INFO : Epoch 1961 | loss: 0.0309558 | val_loss: 0.030968 | Time: 10345.2 ms [2022-02-27 23:20:26 main:574] : INFO : Epoch 1962 | loss: 0.0309516 | val_loss: 0.0309661 | Time: 12334.4 ms [2022-02-27 23:20:39 main:574] : INFO : Epoch 1963 | loss: 0.0309462 | val_loss: 0.030956 | Time: 12479.8 ms [2022-02-27 23:20:49 main:574] : INFO : Epoch 1964 | loss: 0.0309387 | val_loss: 0.0309547 | Time: 10088.1 ms [2022-02-27 23:20:59 main:574] : INFO : Epoch 1965 | loss: 0.0309329 | val_loss: 0.0309503 | Time: 10201 ms [2022-02-27 23:21:15 main:574] : INFO : Epoch 1966 | loss: 0.0309278 | val_loss: 0.0309471 | Time: 15940.5 ms [2022-02-27 23:21:26 main:574] : INFO : Epoch 1967 | loss: 0.0309262 | val_loss: 0.0309407 | Time: 11553.2 ms [2022-02-27 23:21:37 main:574] : INFO : Epoch 1968 | loss: 0.0309227 | val_loss: 0.0309401 | Time: 10062.5 ms [2022-02-27 23:21:47 main:574] : INFO : Epoch 1969 | loss: 0.0309191 | val_loss: 0.0309367 | Time: 10488.2 ms [2022-02-27 23:22:01 main:574] : INFO : Epoch 1970 | loss: 0.0309159 | val_loss: 0.0309345 | Time: 14419 ms [2022-02-27 23:22:14 main:574] : INFO : Epoch 1971 | loss: 0.0309121 | val_loss: 0.0309333 | Time: 12401.7 ms [2022-02-27 23:22:24 main:574] : INFO : Epoch 1972 | loss: 0.0309109 | val_loss: 0.0309345 | Time: 10041.1 ms [2022-02-27 23:22:34 main:574] : INFO : Epoch 1973 | loss: 0.0309102 | val_loss: 0.0309312 | Time: 10158.7 ms [2022-02-27 23:22:50 main:574] : INFO : Epoch 1974 | loss: 0.0309064 | val_loss: 0.0309275 | Time: 16221.6 ms [2022-02-27 23:23:02 main:574] : INFO : Epoch 1975 | loss: 0.0309017 | val_loss: 0.030923 | Time: 12082.3 ms [2022-02-27 23:23:13 main:574] : INFO : Epoch 1976 | loss: 0.030896 | val_loss: 0.0309204 | Time: 10718.4 ms [2022-02-27 23:23:26 main:574] : INFO : Epoch 1977 | loss: 0.0309051 | val_loss: 0.0309466 | Time: 12926.6 ms [2022-02-27 23:23:42 main:574] : INFO : Epoch 1978 | loss: 0.030921 | val_loss: 0.0309368 | Time: 15829.1 ms [2022-02-27 23:23:52 main:574] : INFO : Epoch 1979 | loss: 0.0309039 | val_loss: 0.0309216 | Time: 10229.4 ms [2022-02-27 23:24:03 main:574] : INFO : Epoch 1980 | loss: 0.0308942 | val_loss: 0.0309248 | Time: 10370.9 ms [2022-02-27 23:24:18 main:574] : INFO : Epoch 1981 | loss: 0.030895 | val_loss: 0.0309188 | Time: 14976 ms [2022-02-27 23:24:32 main:574] : INFO : Epoch 1982 | loss: 0.0308914 | val_loss: 0.0309194 | Time: 14175.7 ms [2022-02-27 23:24:42 main:574] : INFO : Epoch 1983 | loss: 0.0309023 | val_loss: 0.0310477 | Time: 10070.5 ms [2022-02-27 23:24:53 main:574] : INFO : Epoch 1984 | loss: 0.0311258 | val_loss: 0.0311574 | Time: 11151.5 ms [2022-02-27 23:25:09 main:574] : INFO : Epoch 1985 | loss: 0.0311739 | val_loss: 0.0311655 | Time: 15931.2 ms [2022-02-27 23:25:22 main:574] : INFO : Epoch 1986 | loss: 0.0311746 | val_loss: 0.0311887 | Time: 13018.7 ms [2022-02-27 23:25:32 main:574] : INFO : Epoch 1987 | loss: 0.0311949 | val_loss: 0.0311971 | Time: 9671.61 ms [2022-02-27 23:25:44 main:574] : INFO : Epoch 1988 | loss: 0.0311961 | val_loss: 0.0311944 | Time: 11964.4 ms [2022-02-27 23:26:00 main:574] : INFO : Epoch 1989 | loss: 0.0311929 | val_loss: 0.0311924 | Time: 16237.4 ms [2022-02-27 23:26:12 main:574] : INFO : Epoch 1990 | loss: 0.0311896 | val_loss: 0.0311902 | Time: 12307.7 ms [2022-02-27 23:26:22 main:574] : INFO : Epoch 1991 | loss: 0.0311869 | val_loss: 0.0311892 | Time: 10146.1 ms [2022-02-27 23:26:36 main:574] : INFO : Epoch 1992 | loss: 0.0311859 | val_loss: 0.0311884 | Time: 13710.9 ms [2022-02-27 23:26:52 main:574] : INFO : Epoch 1993 | loss: 0.0311836 | val_loss: 0.0311868 | Time: 16044.7 ms [2022-02-27 23:27:03 main:574] : INFO : Epoch 1994 | loss: 0.0311824 | val_loss: 0.0311832 | Time: 11217.4 ms [2022-02-27 23:27:13 main:574] : INFO : Epoch 1995 | loss: 0.0311784 | val_loss: 0.0311821 | Time: 9709.11 ms [2022-02-27 23:27:27 main:574] : INFO : Epoch 1996 | loss: 0.0311745 | val_loss: 0.0311724 | Time: 14414.4 ms [2022-02-27 23:27:43 main:574] : INFO : Epoch 1997 | loss: 0.0311739 | val_loss: 0.0311743 | Time: 16032.2 ms [2022-02-27 23:27:55 main:574] : INFO : Epoch 1998 | loss: 0.0311701 | val_loss: 0.0311709 | Time: 11247.6 ms [2022-02-27 23:28:05 main:574] : INFO : Epoch 1999 | loss: 0.0311671 | val_loss: 0.0311655 | Time: 10025.4 ms [2022-02-27 23:28:20 main:574] : INFO : Epoch 2000 | loss: 0.0311648 | val_loss: 0.031166 | Time: 15024 ms [2022-02-27 23:28:36 main:574] : INFO : Epoch 2001 | loss: 0.0311627 | val_loss: 0.0311629 | Time: 16210.2 ms [2022-02-27 23:28:46 main:574] : INFO : Epoch 2002 | loss: 0.0311602 | val_loss: 0.0311588 | Time: 10234.8 ms [2022-02-27 23:28:56 main:574] : INFO : Epoch 2003 | loss: 0.0311584 | val_loss: 0.0311639 | Time: 9897.18 ms [2022-02-27 23:29:12 main:574] : INFO : Epoch 2004 | loss: 0.0311564 | val_loss: 0.0311514 | Time: 15417.2 ms [2022-02-27 23:29:26 main:574] : INFO : Epoch 2005 | loss: 0.0311528 | val_loss: 0.031149 | Time: 14723.5 ms [2022-02-27 23:29:36 main:574] : INFO : Epoch 2006 | loss: 0.0311511 | val_loss: 0.0311518 | Time: 9806.89 ms [2022-02-27 23:29:48 main:574] : INFO : Epoch 2007 | loss: 0.0311478 | val_loss: 0.0311477 | Time: 11988.4 ms [2022-02-27 23:30:04 main:574] : INFO : Epoch 2008 | loss: 0.0311454 | val_loss: 0.0311459 | Time: 16102.2 ms [2022-02-27 23:30:19 main:574] : INFO : Epoch 2009 | loss: 0.0311432 | val_loss: 0.031143 | Time: 14806.4 ms [2022-02-27 23:30:29 main:574] : INFO : Epoch 2010 | loss: 0.031144 | val_loss: 0.0311417 | Time: 9945.83 ms [2022-02-27 23:30:41 main:574] : INFO : Epoch 2011 | loss: 0.0311414 | val_loss: 0.0311425 | Time: 11666.2 ms [2022-02-27 23:30:57 main:574] : INFO : Epoch 2012 | loss: 0.0311401 | val_loss: 0.0311391 | Time: 15922.5 ms [2022-02-27 23:31:12 main:574] : INFO : Epoch 2013 | loss: 0.0311408 | val_loss: 0.0311435 | Time: 15317.4 ms [2022-02-27 23:31:22 main:574] : INFO : Epoch 2014 | loss: 0.0311413 | val_loss: 0.0311421 | Time: 9774.47 ms [2022-02-27 23:31:33 main:574] : INFO : Epoch 2015 | loss: 0.0311388 | val_loss: 0.0311356 | Time: 11087.2 ms [2022-02-27 23:31:49 main:574] : INFO : Epoch 2016 | loss: 0.0311381 | val_loss: 0.0311347 | Time: 16448.3 ms [2022-02-27 23:32:04 main:574] : INFO : Epoch 2017 | loss: 0.0311351 | val_loss: 0.0311335 | Time: 14778.3 ms [2022-02-27 23:32:14 main:574] : INFO : Epoch 2018 | loss: 0.0311328 | val_loss: 0.0311355 | Time: 9943.95 ms [2022-02-27 23:32:27 main:574] : INFO : Epoch 2019 | loss: 0.0311341 | val_loss: 0.0311311 | Time: 12777.6 ms [2022-02-27 23:32:43 main:574] : INFO : Epoch 2020 | loss: 0.0311314 | val_loss: 0.0311332 | Time: 16071.1 ms [2022-02-27 23:32:58 main:574] : INFO : Epoch 2021 | loss: 0.0311341 | val_loss: 0.0311324 | Time: 14987.5 ms [2022-02-27 23:33:08 main:574] : INFO : Epoch 2022 | loss: 0.031129 | val_loss: 0.0311286 | Time: 10119.4 ms [2022-02-27 23:33:19 main:574] : INFO : Epoch 2023 | loss: 0.0311286 | val_loss: 0.0311275 | Time: 10948.9 ms [2022-02-27 23:33:35 main:574] : INFO : Epoch 2024 | loss: 0.0311271 | val_loss: 0.0311267 | Time: 15984.5 ms [2022-02-27 23:33:52 main:574] : INFO : Epoch 2025 | loss: 0.0311262 | val_loss: 0.0311261 | Time: 16688.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-28 00:24:49 main:435] : INFO : Set logging level to 1 [2022-02-28 00:24:49 main:441] : INFO : Running in BOINC Client mode [2022-02-28 00:24:49 main:444] : INFO : Resolving all filenames [2022-02-28 00:24:49 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-28 00:24:49 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-28 00:24:49 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-28 00:24:49 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-28 00:24:49 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-28 00:24:49 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-28 00:24:49 main:474] : INFO : Configuration: [2022-02-28 00:24:49 main:475] : INFO : Model type: GRU [2022-02-28 00:24:49 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-28 00:24:50 main:477] : INFO : Max Epochs: 2048 [2022-02-28 00:24:50 main:478] : INFO : Batch Size: 128 [2022-02-28 00:24:50 main:479] : INFO : Learning Rate: 0.01 [2022-02-28 00:24:50 main:480] : INFO : Patience: 10 [2022-02-28 00:24:50 main:481] : INFO : Hidden Width: 12 [2022-02-28 00:24:50 main:482] : INFO : # Recurrent Layers: 4 [2022-02-28 00:24:50 main:483] : INFO : # Backend Layers: 4 [2022-02-28 00:24:50 main:484] : INFO : # Threads: 1 [2022-02-28 00:24:50 main:486] : INFO : Preparing Dataset [2022-02-28 00:24:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-28 00:24:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-02-28 01:02:01 main:435] : INFO : Set logging level to 1 [2022-02-28 01:02:01 main:441] : INFO : Running in BOINC Client mode [2022-02-28 01:02:01 main:444] : INFO : Resolving all filenames [2022-02-28 01:02:01 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-28 01:02:01 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-28 01:02:01 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-28 01:02:01 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-28 01:02:01 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-28 01:02:01 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-28 01:02:01 main:474] : INFO : Configuration: [2022-02-28 01:02:01 main:475] : INFO : Model type: GRU [2022-02-28 01:02:01 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-28 01:02:01 main:477] : INFO : Max Epochs: 2048 [2022-02-28 01:02:01 main:478] : INFO : Batch Size: 128 [2022-02-28 01:02:01 main:479] : INFO : Learning Rate: 0.01 [2022-02-28 01:02:01 main:480] : INFO : Patience: 10 [2022-02-28 01:02:01 main:481] : INFO : Hidden Width: 12 [2022-02-28 01:02:01 main:482] : INFO : # Recurrent Layers: 4 [2022-02-28 01:02:01 main:483] : INFO : # Backend Layers: 4 [2022-02-28 01:02:01 main:484] : INFO : # Threads: 1 [2022-02-28 01:02:01 main:486] : INFO : Preparing Dataset [2022-02-28 01:02:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-28 01:02:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-28 01:02:04 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-28 01:02:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-28 01:02:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-28 01:02:04 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-28 01:02:04 main:494] : INFO : Creating Model [2022-02-28 01:02:04 main:507] : INFO : Preparing config file [2022-02-28 01:02:04 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-28 01:02:04 main:512] : INFO : Loading config [2022-02-28 01:02:04 main:514] : INFO : Loading state [2022-02-28 01:02:04 main:559] : INFO : Loading DataLoader into Memory [2022-02-28 01:02:04 main:562] : INFO : Starting Training [2022-02-28 01:02:08 main:574] : INFO : Epoch 2025 | loss: 0.0311561 | val_loss: 0.0311081 | Time: 3959.88 ms [2022-02-28 01:02:13 main:574] : INFO : Epoch 2026 | loss: 0.0310951 | val_loss: 0.0310824 | Time: 4566.32 ms [2022-02-28 01:02:20 main:574] : INFO : Epoch 2027 | loss: 0.0310699 | val_loss: 0.031075 | Time: 7201.15 ms [2022-02-28 01:02:27 main:574] : INFO : Epoch 2028 | loss: 0.0310659 | val_loss: 0.0310672 | Time: 7273.51 ms [2022-02-28 01:02:35 main:574] : INFO : Epoch 2029 | loss: 0.0310621 | val_loss: 0.0310532 | Time: 7342.18 ms [2022-02-28 01:02:42 main:574] : INFO : Epoch 2030 | loss: 0.0310498 | val_loss: 0.0310467 | Time: 7401.75 ms [2022-02-28 01:02:49 main:574] : INFO : Epoch 2031 | loss: 0.0310469 | val_loss: 0.0310451 | Time: 7045.79 ms [2022-02-28 01:02:57 main:574] : INFO : Epoch 2032 | loss: 0.0310425 | val_loss: 0.0310486 | Time: 7986.76 ms [2022-02-28 01:03:08 main:574] : INFO : Epoch 2033 | loss: 0.0310376 | val_loss: 0.0310332 | Time: 10620.6 ms [2022-02-28 01:03:17 main:574] : INFO : Epoch 2034 | loss: 0.031025 | val_loss: 0.0310393 | Time: 8625.75 ms [2022-02-28 01:03:25 main:574] : INFO : Epoch 2035 | loss: 0.0310597 | val_loss: 0.0310639 | Time: 8200.72 ms [2022-02-28 01:03:33 main:574] : INFO : Epoch 2036 | loss: 0.031063 | val_loss: 0.0310508 | Time: 8321.46 ms [2022-02-28 01:03:42 main:574] : INFO : Epoch 2037 | loss: 0.0310485 | val_loss: 0.0310409 | Time: 8569.44 ms [2022-02-28 01:03:51 main:574] : INFO : Epoch 2038 | loss: 0.0310343 | val_loss: 0.0310321 | Time: 9205.65 ms [2022-02-28 01:04:02 main:574] : INFO : Epoch 2039 | loss: 0.0310251 | val_loss: 0.0310295 | Time: 11056 ms [2022-02-28 01:04:15 main:574] : INFO : Epoch 2040 | loss: 0.0310278 | val_loss: 0.0310298 | Time: 12835 ms [2022-02-28 01:04:25 main:574] : INFO : Epoch 2041 | loss: 0.0310693 | val_loss: 0.0310933 | Time: 10642 ms [2022-02-28 01:04:37 main:574] : INFO : Epoch 2042 | loss: 0.0310842 | val_loss: 0.0310686 | Time: 11803.8 ms [2022-02-28 01:04:50 main:574] : INFO : Epoch 2043 | loss: 0.0310556 | val_loss: 0.0310521 | Time: 12980.5 ms [2022-02-28 01:05:02 main:574] : INFO : Epoch 2044 | loss: 0.0310537 | val_loss: 0.0310886 | Time: 11926.3 ms [2022-02-28 01:05:13 main:574] : INFO : Epoch 2045 | loss: 0.0310819 | val_loss: 0.0310789 | Time: 11118.5 ms [2022-02-28 01:05:24 main:574] : INFO : Epoch 2046 | loss: 0.0310645 | val_loss: 0.0310555 | Time: 10767.3 ms [2022-02-28 01:05:36 main:574] : INFO : Epoch 2047 | loss: 0.0310428 | val_loss: 0.0310399 | Time: 11806 ms [2022-02-28 01:05:50 main:574] : INFO : Epoch 2048 | loss: 0.0310281 | val_loss: 0.0310299 | Time: 14064.3 ms [2022-02-28 01:05:50 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0310299 [2022-02-28 01:05:50 main:603] : INFO : Saving end state to config to file [2022-02-28 01:05:50 main:608] : INFO : Success, exiting.. 01:05:50 (25448): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)