Task 14618018
| Name | ParityModified-1647045244-28190-3-0_0 |
| Workunit | 11638737 |
| Created | 12 Apr 2022, 6:07:14 UTC |
| Sent | 12 Apr 2022, 6:07:28 UTC |
| Report deadline | 20 Apr 2022, 6:07:28 UTC |
| Received | 20 Apr 2022, 6:09:09 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 13966 |
| Run time | 2 hours 36 min 8 sec |
| CPU time | 2 hours 31 min 13 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 10,956.98 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.37 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> val_loss: 0.0311711 | Time: 4463.59 ms [2022-04-20 14:24:55 main:574] : INFO : Epoch 1791 | loss: 0.0311504 | val_loss: 0.0311635 | Time: 4103.68 ms [2022-04-20 14:25:00 main:574] : INFO : Epoch 1792 | loss: 0.0311486 | val_loss: 0.0311699 | Time: 4212.4 ms [2022-04-20 14:25:04 main:574] : INFO : Epoch 1793 | loss: 0.0311465 | val_loss: 0.0311684 | Time: 4213.81 ms [2022-04-20 14:25:08 main:574] : INFO : Epoch 1794 | loss: 0.0311468 | val_loss: 0.0311666 | Time: 4220.79 ms [2022-04-20 14:25:12 main:574] : INFO : Epoch 1795 | loss: 0.0311458 | val_loss: 0.0311661 | Time: 4239.52 ms [2022-04-20 14:25:17 main:574] : INFO : Epoch 1796 | loss: 0.0311438 | val_loss: 0.0311718 | Time: 4222.76 ms [2022-04-20 14:25:21 main:574] : INFO : Epoch 1797 | loss: 0.0311442 | val_loss: 0.0311693 | Time: 4251.54 ms [2022-04-20 14:25:25 main:574] : INFO : Epoch 1798 | loss: 0.0311448 | val_loss: 0.0311693 | Time: 4156.19 ms [2022-04-20 14:25:29 main:574] : INFO : Epoch 1799 | loss: 0.0311446 | val_loss: 0.0311659 | Time: 4088.54 ms [2022-04-20 14:25:33 main:574] : INFO : Epoch 1800 | loss: 0.0311451 | val_loss: 0.0311709 | Time: 4236.77 ms [2022-04-20 14:25:37 main:574] : INFO : Epoch 1801 | loss: 0.0311464 | val_loss: 0.031164 | Time: 4093.75 ms [2022-04-20 14:25:42 main:574] : INFO : Epoch 1802 | loss: 0.0311459 | val_loss: 0.0311658 | Time: 4164.05 ms [2022-04-20 14:25:46 main:574] : INFO : Epoch 1803 | loss: 0.0311506 | val_loss: 0.0311655 | Time: 4201.42 ms [2022-04-20 14:25:50 main:574] : INFO : Epoch 1804 | loss: 0.0311454 | val_loss: 0.0311655 | Time: 4175.27 ms [2022-04-20 14:25:54 main:574] : INFO : Epoch 1805 | loss: 0.0311457 | val_loss: 0.0311625 | Time: 4154.65 ms [2022-04-20 14:25:58 main:574] : INFO : Epoch 1806 | loss: 0.0311459 | val_loss: 0.0311635 | Time: 4248.63 ms [2022-04-20 14:26:03 main:574] : INFO : Epoch 1807 | loss: 0.0311452 | val_loss: 0.0311709 | Time: 4170.49 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 14:31:51 main:435] : INFO : Set logging level to 1 [2022-04-20 14:31:51 main:441] : INFO : Running in BOINC Client mode [2022-04-20 14:31:51 main:444] : INFO : Resolving all filenames [2022-04-20 14:31:51 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 14:31:51 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 14:31:51 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 14:31:51 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 14:31:51 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 14:31:51 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 14:31:51 main:474] : INFO : Configuration: [2022-04-20 14:31:51 main:475] : INFO : Model type: GRU [2022-04-20 14:31:51 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 14:31:51 main:477] : INFO : Max Epochs: 2048 [2022-04-20 14:31:51 main:478] : INFO : Batch Size: 128 [2022-04-20 14:31:51 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 14:31:51 main:480] : INFO : Patience: 10 [2022-04-20 14:31:51 main:481] : INFO : Hidden Width: 12 [2022-04-20 14:31:51 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 14:31:51 main:483] : INFO : # Backend Layers: 4 [2022-04-20 14:31:51 main:484] : INFO : # Threads: 1 [2022-04-20 14:31:51 main:486] : INFO : Preparing Dataset [2022-04-20 14:31:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 14:31:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 14:31:53 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 14:31:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 14:31:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 14:31:53 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 14:31:53 main:494] : INFO : Creating Model [2022-04-20 14:31:53 main:507] : INFO : Preparing config file [2022-04-20 14:31:53 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 14:31:53 main:512] : INFO : Loading config [2022-04-20 14:31:53 main:514] : INFO : Loading state [2022-04-20 14:31:54 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 14:31:54 main:562] : INFO : Starting Training [2022-04-20 14:31:59 main:574] : INFO : Epoch 1805 | loss: 0.0311917 | val_loss: 0.0311732 | Time: 4784.57 ms [2022-04-20 14:32:03 main:574] : INFO : Epoch 1806 | loss: 0.0311564 | val_loss: 0.0311672 | Time: 4502.49 ms [2022-04-20 14:32:07 main:574] : INFO : Epoch 1807 | loss: 0.0311508 | val_loss: 0.0311689 | Time: 4194.9 ms [2022-04-20 14:32:12 main:574] : INFO : Epoch 1808 | loss: 0.0311503 | val_loss: 0.0311693 | Time: 4175.38 ms [2022-04-20 14:32:16 main:574] : INFO : Epoch 1809 | loss: 0.0311495 | val_loss: 0.0311663 | Time: 4133.1 ms [2022-04-20 14:32:20 main:574] : INFO : Epoch 1810 | loss: 0.0311473 | val_loss: 0.0311639 | Time: 4278.38 ms [2022-04-20 14:32:24 main:574] : INFO : Epoch 1811 | loss: 0.0311477 | val_loss: 0.0311646 | Time: 4160.94 ms [2022-04-20 14:32:28 main:574] : INFO : Epoch 1812 | loss: 0.0311458 | val_loss: 0.0311626 | Time: 4247.15 ms [2022-04-20 14:32:33 main:574] : INFO : Epoch 1813 | loss: 0.0311462 | val_loss: 0.0311665 | Time: 4261.71 ms [2022-04-20 14:32:37 main:574] : INFO : Epoch 1814 | loss: 0.0311455 | val_loss: 0.0311668 | Time: 4159.9 ms [2022-04-20 14:32:41 main:574] : INFO : Epoch 1815 | loss: 0.031147 | val_loss: 0.031164 | Time: 4137.86 ms [2022-04-20 14:32:45 main:574] : INFO : Epoch 1816 | loss: 0.0311452 | val_loss: 0.0311644 | Time: 4230.43 ms [2022-04-20 14:32:49 main:574] : INFO : Epoch 1817 | loss: 0.0311471 | val_loss: 0.0311658 | Time: 4269.3 ms [2022-04-20 14:32:54 main:574] : INFO : Epoch 1818 | loss: 0.0311483 | val_loss: 0.0311631 | Time: 4286.44 ms [2022-04-20 14:32:58 main:574] : INFO : Epoch 1819 | loss: 0.0311495 | val_loss: 0.0311653 | Time: 4242.99 ms [2022-04-20 14:33:02 main:574] : INFO : Epoch 1820 | loss: 0.0311486 | val_loss: 0.0311653 | Time: 4227.41 ms [2022-04-20 14:33:06 main:574] : INFO : Epoch 1821 | loss: 0.0311472 | val_loss: 0.0311633 | Time: 4140.74 ms [2022-04-20 14:33:11 main:574] : INFO : Epoch 1822 | loss: 0.0311445 | val_loss: 0.0311675 | Time: 4231.63 ms [2022-04-20 14:33:15 main:574] : INFO : Epoch 1823 | loss: 0.0311445 | val_loss: 0.031163 | Time: 4241.76 ms [2022-04-20 14:33:19 main:574] : INFO : Epoch 1824 | loss: 0.0311455 | val_loss: 0.0311598 | Time: 4278.59 ms [2022-04-20 14:33:23 main:574] : INFO : Epoch 1825 | loss: 0.0311457 | val_loss: 0.0311609 | Time: 4160.88 ms [2022-04-20 14:33:28 main:574] : INFO : Epoch 1826 | loss: 0.0311449 | val_loss: 0.0311634 | Time: 4214.03 ms [2022-04-20 14:33:32 main:574] : INFO : Epoch 1827 | loss: 0.0311545 | val_loss: 0.0311674 | Time: 4187.87 ms [2022-04-20 14:33:36 main:574] : INFO : Epoch 1828 | loss: 0.0311476 | val_loss: 0.0311626 | Time: 4229.51 ms [2022-04-20 14:33:40 main:574] : INFO : Epoch 1829 | loss: 0.0311421 | val_loss: 0.0311628 | Time: 4204.97 ms [2022-04-20 14:33:45 main:574] : INFO : Epoch 1830 | loss: 0.0311432 | val_loss: 0.0311627 | Time: 4263.94 ms [2022-04-20 14:33:49 main:574] : INFO : Epoch 1831 | loss: 0.0311405 | val_loss: 0.0311703 | Time: 4498.65 ms [2022-04-20 14:33:53 main:574] : INFO : Epoch 1832 | loss: 0.0311426 | val_loss: 0.0311651 | Time: 4279.27 ms [2022-04-20 14:33:58 main:574] : INFO : Epoch 1833 | loss: 0.0311418 | val_loss: 0.0311616 | Time: 4182.01 ms [2022-04-20 14:34:02 main:574] : INFO : Epoch 1834 | loss: 0.031141 | val_loss: 0.0311627 | Time: 4148.81 ms [2022-04-20 14:34:06 main:574] : INFO : Epoch 1835 | loss: 0.0311405 | val_loss: 0.0311644 | Time: 4236.59 ms [2022-04-20 14:34:10 main:574] : INFO : Epoch 1836 | loss: 0.0311425 | val_loss: 0.0311659 | Time: 4209.5 ms [2022-04-20 14:34:14 main:574] : INFO : Epoch 1837 | loss: 0.0311409 | val_loss: 0.0311643 | Time: 4182.19 ms [2022-04-20 14:34:19 main:574] : INFO : Epoch 1838 | loss: 0.0311403 | val_loss: 0.0311634 | Time: 4284.52 ms [2022-04-20 14:34:23 main:574] : INFO : Epoch 1839 | loss: 0.0311394 | val_loss: 0.0311616 | Time: 4206.98 ms [2022-04-20 14:34:27 main:574] : INFO : Epoch 1840 | loss: 0.0311403 | val_loss: 0.0311632 | Time: 4195.67 ms [2022-04-20 14:34:31 main:574] : INFO : Epoch 1841 | loss: 0.0311395 | val_loss: 0.0311661 | Time: 4207.19 ms [2022-04-20 14:34:35 main:574] : INFO : Epoch 1842 | loss: 0.0311392 | val_loss: 0.0311672 | Time: 4186.95 ms [2022-04-20 14:34:40 main:574] : INFO : Epoch 1843 | loss: 0.0311385 | val_loss: 0.0311672 | Time: 4219.94 ms [2022-04-20 14:34:44 main:574] : INFO : Epoch 1844 | loss: 0.0311367 | val_loss: 0.0311675 | Time: 4158.28 ms [2022-04-20 14:34:48 main:574] : INFO : Epoch 1845 | loss: 0.03114 | val_loss: 0.0311694 | Time: 4222.56 ms [2022-04-20 14:34:52 main:574] : INFO : Epoch 1846 | loss: 0.03114 | val_loss: 0.0311663 | Time: 4198.18 ms [2022-04-20 14:34:56 main:574] : INFO : Epoch 1847 | loss: 0.0311389 | val_loss: 0.031169 | Time: 4188.14 ms [2022-04-20 14:35:01 main:574] : INFO : Epoch 1848 | loss: 0.031138 | val_loss: 0.0311665 | Time: 4235.79 ms [2022-04-20 14:35:05 main:574] : INFO : Epoch 1849 | loss: 0.0311381 | val_loss: 0.0311667 | Time: 4208.87 ms [2022-04-20 14:35:09 main:574] : INFO : Epoch 1850 | loss: 0.0311401 | val_loss: 0.0311693 | Time: 4207.24 ms [2022-04-20 14:35:13 main:574] : INFO : Epoch 1851 | loss: 0.0311413 | val_loss: 0.0311644 | Time: 4184.51 ms [2022-04-20 14:35:18 main:574] : INFO : Epoch 1852 | loss: 0.0311378 | val_loss: 0.0311665 | Time: 4227.22 ms [2022-04-20 14:35:22 main:574] : INFO : Epoch 1853 | loss: 0.0311383 | val_loss: 0.0311648 | Time: 4221.68 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:03:19 main:435] : INFO : Set logging level to 1 [2022-04-20 15:03:19 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:03:19 main:444] : INFO : Resolving all filenames [2022-04-20 15:03:19 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:03:19 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:03:19 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:03:19 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:03:19 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:03:19 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:03:19 main:474] : INFO : Configuration: [2022-04-20 15:03:19 main:475] : INFO : Model type: GRU [2022-04-20 15:03:19 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:03:19 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:03:19 main:478] : INFO : Batch Size: 128 [2022-04-20 15:03:19 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:03:19 main:480] : INFO : Patience: 10 [2022-04-20 15:03:19 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:03:19 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:03:19 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:03:19 main:484] : INFO : # Threads: 1 [2022-04-20 15:03:19 main:486] : INFO : Preparing Dataset [2022-04-20 15:03:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:03:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:03:21 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:03:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:03:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:03:21 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:03:21 main:494] : INFO : Creating Model [2022-04-20 15:03:21 main:507] : INFO : Preparing config file [2022-04-20 15:03:21 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:03:21 main:512] : INFO : Loading config [2022-04-20 15:03:21 main:514] : INFO : Loading state [2022-04-20 15:03:22 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:03:22 main:562] : INFO : Starting Training [2022-04-20 15:03:27 main:574] : INFO : Epoch 1852 | loss: 0.0311825 | val_loss: 0.031171 | Time: 4652.51 ms [2022-04-20 15:03:31 main:574] : INFO : Epoch 1853 | loss: 0.0311444 | val_loss: 0.0311652 | Time: 4368.98 ms [2022-04-20 15:03:36 main:574] : INFO : Epoch 1854 | loss: 0.0311403 | val_loss: 0.0311588 | Time: 4376.41 ms [2022-04-20 15:03:40 main:574] : INFO : Epoch 1855 | loss: 0.0311407 | val_loss: 0.0311668 | Time: 4353.99 ms [2022-04-20 15:03:44 main:574] : INFO : Epoch 1856 | loss: 0.0311398 | val_loss: 0.031162 | Time: 4450.31 ms [2022-04-20 15:03:49 main:574] : INFO : Epoch 1857 | loss: 0.0311401 | val_loss: 0.0311649 | Time: 4384.06 ms [2022-04-20 15:03:53 main:574] : INFO : Epoch 1858 | loss: 0.0311391 | val_loss: 0.0311628 | Time: 4410.66 ms [2022-04-20 15:03:58 main:574] : INFO : Epoch 1859 | loss: 0.0311403 | val_loss: 0.0311654 | Time: 4364.34 ms [2022-04-20 15:04:02 main:574] : INFO : Epoch 1860 | loss: 0.0311392 | val_loss: 0.0311619 | Time: 4349.87 ms [2022-04-20 15:04:06 main:574] : INFO : Epoch 1861 | loss: 0.0311395 | val_loss: 0.0311616 | Time: 4450.92 ms [2022-04-20 15:04:11 main:574] : INFO : Epoch 1862 | loss: 0.0311405 | val_loss: 0.0311669 | Time: 4450.94 ms [2022-04-20 15:04:15 main:574] : INFO : Epoch 1863 | loss: 0.0311402 | val_loss: 0.0311615 | Time: 4408.42 ms [2022-04-20 15:04:20 main:574] : INFO : Epoch 1864 | loss: 0.0311405 | val_loss: 0.0311646 | Time: 4513.32 ms [2022-04-20 15:04:24 main:574] : INFO : Epoch 1865 | loss: 0.0311383 | val_loss: 0.0311595 | Time: 4359.55 ms [2022-04-20 15:04:28 main:574] : INFO : Epoch 1866 | loss: 0.03114 | val_loss: 0.0311597 | Time: 4342.03 ms [2022-04-20 15:04:33 main:574] : INFO : Epoch 1867 | loss: 0.0311401 | val_loss: 0.0311578 | Time: 4320.57 ms [2022-04-20 15:04:37 main:574] : INFO : Epoch 1868 | loss: 0.0311396 | val_loss: 0.031161 | Time: 4296.95 ms [2022-04-20 15:04:41 main:574] : INFO : Epoch 1869 | loss: 0.0311382 | val_loss: 0.0311658 | Time: 4337.39 ms [2022-04-20 15:04:46 main:574] : INFO : Epoch 1870 | loss: 0.0311387 | val_loss: 0.0311675 | Time: 4389.11 ms [2022-04-20 15:04:50 main:574] : INFO : Epoch 1871 | loss: 0.0311407 | val_loss: 0.0311599 | Time: 4376.88 ms [2022-04-20 15:04:54 main:574] : INFO : Epoch 1872 | loss: 0.0311427 | val_loss: 0.03116 | Time: 4301.7 ms [2022-04-20 15:04:59 main:574] : INFO : Epoch 1873 | loss: 0.0311405 | val_loss: 0.0311618 | Time: 4311.97 ms [2022-04-20 15:05:03 main:574] : INFO : Epoch 1874 | loss: 0.031141 | val_loss: 0.031163 | Time: 4379.43 ms [2022-04-20 15:05:08 main:574] : INFO : Epoch 1875 | loss: 0.0311407 | val_loss: 0.0311611 | Time: 4352.44 ms [2022-04-20 15:05:12 main:574] : INFO : Epoch 1876 | loss: 0.031146 | val_loss: 0.0311609 | Time: 4400.64 ms [2022-04-20 15:05:16 main:574] : INFO : Epoch 1877 | loss: 0.0311441 | val_loss: 0.0311637 | Time: 4356 ms [2022-04-20 15:05:21 main:574] : INFO : Epoch 1878 | loss: 0.031143 | val_loss: 0.031166 | Time: 4434.73 ms [2022-04-20 15:05:25 main:574] : INFO : Epoch 1879 | loss: 0.0311458 | val_loss: 0.0311622 | Time: 4365.55 ms [2022-04-20 15:05:29 main:574] : INFO : Epoch 1880 | loss: 0.0311414 | val_loss: 0.0311605 | Time: 4355.22 ms [2022-04-20 15:05:34 main:574] : INFO : Epoch 1881 | loss: 0.0311456 | val_loss: 0.0311617 | Time: 4312.2 ms [2022-04-20 15:05:38 main:574] : INFO : Epoch 1882 | loss: 0.0311455 | val_loss: 0.0311676 | Time: 4380.96 ms [2022-04-20 15:05:43 main:574] : INFO : Epoch 1883 | loss: 0.0311465 | val_loss: 0.03116 | Time: 4343 ms [2022-04-20 15:05:47 main:574] : INFO : Epoch 1884 | loss: 0.0311427 | val_loss: 0.0311622 | Time: 4269.55 ms [2022-04-20 15:05:51 main:574] : INFO : Epoch 1885 | loss: 0.0311438 | val_loss: 0.0311638 | Time: 4322.96 ms [2022-04-20 15:05:56 main:574] : INFO : Epoch 1886 | loss: 0.0311439 | val_loss: 0.031166 | Time: 4383.04 ms [2022-04-20 15:06:00 main:574] : INFO : Epoch 1887 | loss: 0.0311423 | val_loss: 0.0311618 | Time: 4303.83 ms [2022-04-20 15:06:04 main:574] : INFO : Epoch 1888 | loss: 0.0311411 | val_loss: 0.0311662 | Time: 4340.33 ms [2022-04-20 15:06:08 main:574] : INFO : Epoch 1889 | loss: 0.0311455 | val_loss: 0.0311674 | Time: 4313.19 ms [2022-04-20 15:06:13 main:574] : INFO : Epoch 1890 | loss: 0.0311392 | val_loss: 0.0311594 | Time: 4343.4 ms [2022-04-20 15:06:17 main:574] : INFO : Epoch 1891 | loss: 0.0311379 | val_loss: 0.0311627 | Time: 4431.74 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:09:20 main:435] : INFO : Set logging level to 1 [2022-04-20 15:09:20 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:09:20 main:444] : INFO : Resolving all filenames [2022-04-20 15:09:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:09:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:09:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:09:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:09:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:09:20 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:09:20 main:474] : INFO : Configuration: [2022-04-20 15:09:20 main:475] : INFO : Model type: GRU [2022-04-20 15:09:20 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:09:20 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:09:20 main:478] : INFO : Batch Size: 128 [2022-04-20 15:09:20 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:09:20 main:480] : INFO : Patience: 10 [2022-04-20 15:09:20 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:09:20 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:09:20 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:09:20 main:484] : INFO : # Threads: 1 [2022-04-20 15:09:20 main:486] : INFO : Preparing Dataset [2022-04-20 15:09:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:09:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:09:22 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:09:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:09:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:09:22 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:09:22 main:494] : INFO : Creating Model [2022-04-20 15:09:22 main:507] : INFO : Preparing config file [2022-04-20 15:09:22 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:09:22 main:512] : INFO : Loading config [2022-04-20 15:09:22 main:514] : INFO : Loading state [2022-04-20 15:09:23 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:09:23 main:562] : INFO : Starting Training [2022-04-20 15:09:28 main:574] : INFO : Epoch 1881 | loss: 0.0311901 | val_loss: 0.0311683 | Time: 4665.7 ms [2022-04-20 15:09:32 main:574] : INFO : Epoch 1882 | loss: 0.0311503 | val_loss: 0.0311678 | Time: 4349.24 ms [2022-04-20 15:09:36 main:574] : INFO : Epoch 1883 | loss: 0.0311448 | val_loss: 0.0311599 | Time: 4287.43 ms [2022-04-20 15:09:41 main:574] : INFO : Epoch 1884 | loss: 0.0311426 | val_loss: 0.0311611 | Time: 4329.36 ms [2022-04-20 15:09:45 main:574] : INFO : Epoch 1885 | loss: 0.031142 | val_loss: 0.0311607 | Time: 4373.9 ms [2022-04-20 15:09:49 main:574] : INFO : Epoch 1886 | loss: 0.0311413 | val_loss: 0.03116 | Time: 4386.54 ms [2022-04-20 15:09:54 main:574] : INFO : Epoch 1887 | loss: 0.0311393 | val_loss: 0.0311615 | Time: 4385.24 ms [2022-04-20 15:09:58 main:574] : INFO : Epoch 1888 | loss: 0.0311381 | val_loss: 0.0311633 | Time: 4341.74 ms [2022-04-20 15:10:03 main:574] : INFO : Epoch 1889 | loss: 0.0311374 | val_loss: 0.0311691 | Time: 4337.98 ms [2022-04-20 15:10:07 main:574] : INFO : Epoch 1890 | loss: 0.0311431 | val_loss: 0.0311621 | Time: 4343.62 ms [2022-04-20 15:10:11 main:574] : INFO : Epoch 1891 | loss: 0.0311469 | val_loss: 0.0311617 | Time: 4349.63 ms [2022-04-20 15:10:16 main:574] : INFO : Epoch 1892 | loss: 0.0311481 | val_loss: 0.0311618 | Time: 4312.59 ms [2022-04-20 15:10:20 main:574] : INFO : Epoch 1893 | loss: 0.0311472 | val_loss: 0.0311561 | Time: 4457.64 ms [2022-04-20 15:10:24 main:574] : INFO : Epoch 1894 | loss: 0.0311466 | val_loss: 0.0311555 | Time: 4439.96 ms [2022-04-20 15:10:29 main:574] : INFO : Epoch 1895 | loss: 0.0311444 | val_loss: 0.0311582 | Time: 4326.12 ms [2022-04-20 15:10:33 main:574] : INFO : Epoch 1896 | loss: 0.0311415 | val_loss: 0.0311573 | Time: 4306.39 ms [2022-04-20 15:10:37 main:574] : INFO : Epoch 1897 | loss: 0.0311412 | val_loss: 0.0311555 | Time: 4271.39 ms [2022-04-20 15:10:42 main:574] : INFO : Epoch 1898 | loss: 0.0311403 | val_loss: 0.0311569 | Time: 4312.84 ms [2022-04-20 15:10:46 main:574] : INFO : Epoch 1899 | loss: 0.0311395 | val_loss: 0.0311564 | Time: 4366.18 ms [2022-04-20 15:10:51 main:574] : INFO : Epoch 1900 | loss: 0.0311422 | val_loss: 0.0311706 | Time: 4391.66 ms [2022-04-20 15:10:55 main:574] : INFO : Epoch 1901 | loss: 0.0311444 | val_loss: 0.0311606 | Time: 4396.21 ms [2022-04-20 15:10:59 main:574] : INFO : Epoch 1902 | loss: 0.0311419 | val_loss: 0.0311647 | Time: 4385.19 ms [2022-04-20 15:11:04 main:574] : INFO : Epoch 1903 | loss: 0.031143 | val_loss: 0.0311658 | Time: 4308.03 ms [2022-04-20 15:11:08 main:574] : INFO : Epoch 1904 | loss: 0.0311398 | val_loss: 0.0311689 | Time: 4403.79 ms [2022-04-20 15:11:12 main:574] : INFO : Epoch 1905 | loss: 0.0311394 | val_loss: 0.0311618 | Time: 4337.07 ms [2022-04-20 15:11:17 main:574] : INFO : Epoch 1906 | loss: 0.0311449 | val_loss: 0.0311621 | Time: 4409.64 ms [2022-04-20 15:11:21 main:574] : INFO : Epoch 1907 | loss: 0.0311474 | val_loss: 0.0311624 | Time: 4381.43 ms [2022-04-20 15:11:25 main:574] : INFO : Epoch 1908 | loss: 0.0311438 | val_loss: 0.0311624 | Time: 4317.68 ms [2022-04-20 15:11:30 main:574] : INFO : Epoch 1909 | loss: 0.0311426 | val_loss: 0.0311617 | Time: 4348.48 ms [2022-04-20 15:11:34 main:574] : INFO : Epoch 1910 | loss: 0.0311426 | val_loss: 0.0311617 | Time: 4295.33 ms [2022-04-20 15:11:39 main:574] : INFO : Epoch 1911 | loss: 0.0311419 | val_loss: 0.0311663 | Time: 4394.08 ms [2022-04-20 15:11:43 main:574] : INFO : Epoch 1912 | loss: 0.0311441 | val_loss: 0.0311662 | Time: 4439.05 ms [2022-04-20 15:11:47 main:574] : INFO : Epoch 1913 | loss: 0.0311415 | val_loss: 0.0311691 | Time: 4372.33 ms [2022-04-20 15:11:52 main:574] : INFO : Epoch 1914 | loss: 0.0311447 | val_loss: 0.0311635 | Time: 4308.88 ms [2022-04-20 15:11:56 main:574] : INFO : Epoch 1915 | loss: 0.0311418 | val_loss: 0.0311667 | Time: 4323.04 ms [2022-04-20 15:12:00 main:574] : INFO : Epoch 1916 | loss: 0.0311411 | val_loss: 0.0311687 | Time: 4361.12 ms [2022-04-20 15:12:05 main:574] : INFO : Epoch 1917 | loss: 0.0311392 | val_loss: 0.0311649 | Time: 4425.91 ms [2022-04-20 15:12:09 main:574] : INFO : Epoch 1918 | loss: 0.0311387 | val_loss: 0.0311608 | Time: 4351.33 ms [2022-04-20 15:12:14 main:574] : INFO : Epoch 1919 | loss: 0.0311387 | val_loss: 0.0311663 | Time: 4409.75 ms [2022-04-20 15:12:18 main:574] : INFO : Epoch 1920 | loss: 0.0311373 | val_loss: 0.0311682 | Time: 4525 ms [2022-04-20 15:12:23 main:574] : INFO : Epoch 1921 | loss: 0.0311383 | val_loss: 0.0311692 | Time: 4546.36 ms [2022-04-20 15:12:27 main:574] : INFO : Epoch 1922 | loss: 0.0311376 | val_loss: 0.0311695 | Time: 4479.28 ms [2022-04-20 15:12:32 main:574] : INFO : Epoch 1923 | loss: 0.0311399 | val_loss: 0.0311677 | Time: 4394.06 ms [2022-04-20 15:12:36 main:574] : INFO : Epoch 1924 | loss: 0.0311355 | val_loss: 0.0311701 | Time: 4611.92 ms [2022-04-20 15:12:41 main:574] : INFO : Epoch 1925 | loss: 0.031135 | val_loss: 0.0311658 | Time: 4474.41 ms [2022-04-20 15:12:45 main:574] : INFO : Epoch 1926 | loss: 0.0311359 | val_loss: 0.031165 | Time: 4386.19 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:25:15 main:435] : INFO : Set logging level to 1 [2022-04-20 15:25:15 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:25:15 main:444] : INFO : Resolving all filenames [2022-04-20 15:25:15 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:25:15 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:25:15 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:25:15 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:25:15 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:25:15 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:25:15 main:474] : INFO : Configuration: [2022-04-20 15:25:15 main:475] : INFO : Model type: GRU [2022-04-20 15:25:15 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:25:15 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:25:15 main:478] : INFO : Batch Size: 128 [2022-04-20 15:25:15 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:25:15 main:480] : INFO : Patience: 10 [2022-04-20 15:25:15 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:25:15 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:25:15 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:25:15 main:484] : INFO : # Threads: 1 [2022-04-20 15:25:15 main:486] : INFO : Preparing Dataset [2022-04-20 15:25:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:25:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:25:17 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:25:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:25:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:25:17 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:25:17 main:494] : INFO : Creating Model [2022-04-20 15:25:17 main:507] : INFO : Preparing config file [2022-04-20 15:25:17 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:25:17 main:512] : INFO : Loading config [2022-04-20 15:25:17 main:514] : INFO : Loading state [2022-04-20 15:25:19 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:25:19 main:562] : INFO : Starting Training [2022-04-20 15:25:23 main:574] : INFO : Epoch 1926 | loss: 0.0311881 | val_loss: 0.0311751 | Time: 4687.28 ms [2022-04-20 15:25:28 main:574] : INFO : Epoch 1927 | loss: 0.0311432 | val_loss: 0.0311748 | Time: 4421.37 ms [2022-04-20 15:25:32 main:574] : INFO : Epoch 1928 | loss: 0.031141 | val_loss: 0.0311673 | Time: 4333.75 ms [2022-04-20 15:25:36 main:574] : INFO : Epoch 1929 | loss: 0.0311377 | val_loss: 0.0311621 | Time: 4309.64 ms [2022-04-20 15:25:41 main:574] : INFO : Epoch 1930 | loss: 0.0311372 | val_loss: 0.0311631 | Time: 4311.45 ms [2022-04-20 15:25:45 main:574] : INFO : Epoch 1931 | loss: 0.0311374 | val_loss: 0.0311644 | Time: 4328.75 ms [2022-04-20 15:25:49 main:574] : INFO : Epoch 1932 | loss: 0.0311381 | val_loss: 0.0311628 | Time: 4367.09 ms [2022-04-20 15:25:54 main:574] : INFO : Epoch 1933 | loss: 0.0311384 | val_loss: 0.031161 | Time: 4283.72 ms [2022-04-20 15:25:58 main:574] : INFO : Epoch 1934 | loss: 0.0311385 | val_loss: 0.0311637 | Time: 4346.73 ms [2022-04-20 15:26:02 main:574] : INFO : Epoch 1935 | loss: 0.0311367 | val_loss: 0.0311639 | Time: 4350.87 ms [2022-04-20 15:26:07 main:574] : INFO : Epoch 1936 | loss: 0.0311358 | val_loss: 0.0311672 | Time: 4291.52 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:45:11 main:435] : INFO : Set logging level to 1 [2022-04-20 15:45:12 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:45:12 main:444] : INFO : Resolving all filenames [2022-04-20 15:45:12 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:45:12 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:45:12 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:45:12 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:45:12 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:45:12 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:45:12 main:474] : INFO : Configuration: [2022-04-20 15:45:12 main:475] : INFO : Model type: GRU [2022-04-20 15:45:12 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:45:12 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:45:12 main:478] : INFO : Batch Size: 128 [2022-04-20 15:45:12 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:45:12 main:480] : INFO : Patience: 10 [2022-04-20 15:45:12 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:45:12 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:45:12 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:45:12 main:484] : INFO : # Threads: 1 [2022-04-20 15:45:12 main:486] : INFO : Preparing Dataset [2022-04-20 15:45:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:45:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:45:13 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:45:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:45:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:45:14 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:45:14 main:494] : INFO : Creating Model [2022-04-20 15:45:14 main:507] : INFO : Preparing config file [2022-04-20 15:45:14 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:45:14 main:512] : INFO : Loading config [2022-04-20 15:45:14 main:514] : INFO : Loading state [2022-04-20 15:45:15 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:45:15 main:562] : INFO : Starting Training [2022-04-20 15:45:19 main:574] : INFO : Epoch 1926 | loss: 0.0311859 | val_loss: 0.0311736 | Time: 4567.59 ms [2022-04-20 15:45:24 main:574] : INFO : Epoch 1927 | loss: 0.0311417 | val_loss: 0.0311628 | Time: 4277.63 ms [2022-04-20 15:45:28 main:574] : INFO : Epoch 1928 | loss: 0.0311382 | val_loss: 0.0311611 | Time: 4163.96 ms [2022-04-20 15:45:32 main:574] : INFO : Epoch 1929 | loss: 0.0311353 | val_loss: 0.031168 | Time: 4218.4 ms [2022-04-20 15:45:36 main:574] : INFO : Epoch 1930 | loss: 0.0311354 | val_loss: 0.0311619 | Time: 4287.1 ms [2022-04-20 15:45:41 main:574] : INFO : Epoch 1931 | loss: 0.0311354 | val_loss: 0.0311601 | Time: 4283.23 ms [2022-04-20 15:45:45 main:574] : INFO : Epoch 1932 | loss: 0.0311359 | val_loss: 0.0311594 | Time: 4278.6 ms [2022-04-20 15:45:49 main:574] : INFO : Epoch 1933 | loss: 0.0311379 | val_loss: 0.0311627 | Time: 4308.85 ms [2022-04-20 15:45:53 main:574] : INFO : Epoch 1934 | loss: 0.0311383 | val_loss: 0.03116 | Time: 4219.03 ms [2022-04-20 15:45:58 main:574] : INFO : Epoch 1935 | loss: 0.0311375 | val_loss: 0.0311648 | Time: 4227.34 ms [2022-04-20 15:46:02 main:574] : INFO : Epoch 1936 | loss: 0.0311381 | val_loss: 0.0311608 | Time: 4223.82 ms [2022-04-20 15:46:06 main:574] : INFO : Epoch 1937 | loss: 0.031136 | val_loss: 0.031164 | Time: 4207.2 ms [2022-04-20 15:46:10 main:574] : INFO : Epoch 1938 | loss: 0.0311379 | val_loss: 0.0311632 | Time: 4270.45 ms [2022-04-20 15:46:15 main:574] : INFO : Epoch 1939 | loss: 0.0311338 | val_loss: 0.0311597 | Time: 4296.5 ms [2022-04-20 15:46:19 main:574] : INFO : Epoch 1940 | loss: 0.0311335 | val_loss: 0.031159 | Time: 4220.96 ms [2022-04-20 15:46:23 main:574] : INFO : Epoch 1941 | loss: 0.0311312 | val_loss: 0.0311567 | Time: 4277.25 ms [2022-04-20 15:46:28 main:574] : INFO : Epoch 1942 | loss: 0.0311342 | val_loss: 0.0311608 | Time: 4481.68 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:49:30 main:435] : INFO : Set logging level to 1 [2022-04-20 15:49:30 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:49:30 main:444] : INFO : Resolving all filenames [2022-04-20 15:49:30 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:49:30 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:49:30 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:49:30 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:49:30 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:49:30 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:49:30 main:474] : INFO : Configuration: [2022-04-20 15:49:30 main:475] : INFO : Model type: GRU [2022-04-20 15:49:30 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:49:30 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:49:30 main:478] : INFO : Batch Size: 128 [2022-04-20 15:49:30 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:49:30 main:480] : INFO : Patience: 10 [2022-04-20 15:49:30 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:49:30 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:49:30 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:49:30 main:484] : INFO : # Threads: 1 [2022-04-20 15:49:30 main:486] : INFO : Preparing Dataset [2022-04-20 15:49:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:49:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:49:32 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:49:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:49:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:49:32 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:49:32 main:494] : INFO : Creating Model [2022-04-20 15:49:32 main:507] : INFO : Preparing config file [2022-04-20 15:49:32 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:49:32 main:512] : INFO : Loading config [2022-04-20 15:49:32 main:514] : INFO : Loading state [2022-04-20 15:49:33 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:49:33 main:562] : INFO : Starting Training [2022-04-20 15:49:38 main:574] : INFO : Epoch 1941 | loss: 0.0311831 | val_loss: 0.031168 | Time: 4538.02 ms [2022-04-20 15:49:42 main:574] : INFO : Epoch 1942 | loss: 0.031138 | val_loss: 0.0311688 | Time: 4228.27 ms [2022-04-20 15:49:46 main:574] : INFO : Epoch 1943 | loss: 0.0311371 | val_loss: 0.0311582 | Time: 4180.57 ms [2022-04-20 15:49:51 main:574] : INFO : Epoch 1944 | loss: 0.0311342 | val_loss: 0.031159 | Time: 4209.95 ms [2022-04-20 15:49:55 main:574] : INFO : Epoch 1945 | loss: 0.0311365 | val_loss: 0.0311614 | Time: 4204.57 ms [2022-04-20 15:49:59 main:574] : INFO : Epoch 1946 | loss: 0.0311351 | val_loss: 0.031161 | Time: 4200.62 ms [2022-04-20 15:50:03 main:574] : INFO : Epoch 1947 | loss: 0.0311324 | val_loss: 0.0311588 | Time: 4243.88 ms [2022-04-20 15:50:08 main:574] : INFO : Epoch 1948 | loss: 0.0311325 | val_loss: 0.031157 | Time: 4211.9 ms [2022-04-20 15:50:12 main:574] : INFO : Epoch 1949 | loss: 0.0311315 | val_loss: 0.0311618 | Time: 4239.8 ms [2022-04-20 15:50:16 main:574] : INFO : Epoch 1950 | loss: 0.0311334 | val_loss: 0.0311654 | Time: 4253.68 ms [2022-04-20 15:50:20 main:574] : INFO : Epoch 1951 | loss: 0.0311364 | val_loss: 0.0311585 | Time: 4259.54 ms [2022-04-20 15:50:25 main:574] : INFO : Epoch 1952 | loss: 0.0311391 | val_loss: 0.031161 | Time: 4275.36 ms [2022-04-20 15:50:29 main:574] : INFO : Epoch 1953 | loss: 0.0311354 | val_loss: 0.0311609 | Time: 4213.77 ms [2022-04-20 15:50:33 main:574] : INFO : Epoch 1954 | loss: 0.0311355 | val_loss: 0.0311588 | Time: 4320.67 ms [2022-04-20 15:50:37 main:574] : INFO : Epoch 1955 | loss: 0.0311356 | val_loss: 0.0311574 | Time: 4220.64 ms [2022-04-20 15:50:42 main:574] : INFO : Epoch 1956 | loss: 0.0311344 | val_loss: 0.031158 | Time: 4201.13 ms [2022-04-20 15:50:46 main:574] : INFO : Epoch 1957 | loss: 0.031133 | val_loss: 0.0311592 | Time: 4245.12 ms [2022-04-20 15:50:50 main:574] : INFO : Epoch 1958 | loss: 0.0311315 | val_loss: 0.0311632 | Time: 4275.61 ms [2022-04-20 15:50:54 main:574] : INFO : Epoch 1959 | loss: 0.0311341 | val_loss: 0.0311584 | Time: 4387.78 ms [2022-04-20 15:50:59 main:574] : INFO : Epoch 1960 | loss: 0.0311349 | val_loss: 0.0311592 | Time: 4189.96 ms [2022-04-20 15:51:03 main:574] : INFO : Epoch 1961 | loss: 0.0311334 | val_loss: 0.0311578 | Time: 4251.69 ms [2022-04-20 15:51:07 main:574] : INFO : Epoch 1962 | loss: 0.0311339 | val_loss: 0.0311616 | Time: 4273.73 ms [2022-04-20 15:51:11 main:574] : INFO : Epoch 1963 | loss: 0.031132 | val_loss: 0.0311644 | Time: 4206.46 ms [2022-04-20 15:51:16 main:574] : INFO : Epoch 1964 | loss: 0.0311316 | val_loss: 0.0311595 | Time: 4318.49 ms [2022-04-20 15:51:20 main:574] : INFO : Epoch 1965 | loss: 0.0311309 | val_loss: 0.0311631 | Time: 4209.73 ms [2022-04-20 15:51:24 main:574] : INFO : Epoch 1966 | loss: 0.0311323 | val_loss: 0.0311616 | Time: 4272.24 ms [2022-04-20 15:51:29 main:574] : INFO : Epoch 1967 | loss: 0.0311361 | val_loss: 0.0311624 | Time: 4534.91 ms [2022-04-20 15:51:33 main:574] : INFO : Epoch 1968 | loss: 0.0311354 | val_loss: 0.0311612 | Time: 4252.51 ms [2022-04-20 15:51:37 main:574] : INFO : Epoch 1969 | loss: 0.0311385 | val_loss: 0.0311609 | Time: 4148.2 ms [2022-04-20 15:51:41 main:574] : INFO : Epoch 1970 | loss: 0.0311419 | val_loss: 0.0311669 | Time: 4150.72 ms [2022-04-20 15:51:46 main:574] : INFO : Epoch 1971 | loss: 0.0311404 | val_loss: 0.0311621 | Time: 4160.88 ms [2022-04-20 15:51:50 main:574] : INFO : Epoch 1972 | loss: 0.0311396 | val_loss: 0.0311601 | Time: 4258.84 ms [2022-04-20 15:51:54 main:574] : INFO : Epoch 1973 | loss: 0.0311371 | val_loss: 0.0311594 | Time: 4153.94 ms [2022-04-20 15:51:58 main:574] : INFO : Epoch 1974 | loss: 0.0311672 | val_loss: 0.0312231 | Time: 4202.02 ms [2022-04-20 15:52:02 main:574] : INFO : Epoch 1975 | loss: 0.0312179 | val_loss: 0.0312014 | Time: 4191.1 ms [2022-04-20 15:52:07 main:574] : INFO : Epoch 1976 | loss: 0.0311793 | val_loss: 0.0311871 | Time: 4218.48 ms [2022-04-20 15:52:11 main:574] : INFO : Epoch 1977 | loss: 0.0311567 | val_loss: 0.0311643 | Time: 4283.06 ms [2022-04-20 15:52:15 main:574] : INFO : Epoch 1978 | loss: 0.0311477 | val_loss: 0.0311634 | Time: 4232.34 ms [2022-04-20 15:52:19 main:574] : INFO : Epoch 1979 | loss: 0.0311438 | val_loss: 0.0311637 | Time: 4237.36 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 15:57:44 main:435] : INFO : Set logging level to 1 [2022-04-20 15:57:44 main:441] : INFO : Running in BOINC Client mode [2022-04-20 15:57:44 main:444] : INFO : Resolving all filenames [2022-04-20 15:57:44 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 15:57:44 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 15:57:44 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 15:57:44 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 15:57:44 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 15:57:44 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 15:57:44 main:474] : INFO : Configuration: [2022-04-20 15:57:44 main:475] : INFO : Model type: GRU [2022-04-20 15:57:44 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 15:57:44 main:477] : INFO : Max Epochs: 2048 [2022-04-20 15:57:44 main:478] : INFO : Batch Size: 128 [2022-04-20 15:57:44 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 15:57:44 main:480] : INFO : Patience: 10 [2022-04-20 15:57:44 main:481] : INFO : Hidden Width: 12 [2022-04-20 15:57:44 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 15:57:44 main:483] : INFO : # Backend Layers: 4 [2022-04-20 15:57:44 main:484] : INFO : # Threads: 1 [2022-04-20 15:57:44 main:486] : INFO : Preparing Dataset [2022-04-20 15:57:44 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 15:57:44 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 15:57:46 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 15:57:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 15:57:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 15:57:46 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 15:57:46 main:494] : INFO : Creating Model [2022-04-20 15:57:46 main:507] : INFO : Preparing config file [2022-04-20 15:57:46 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 15:57:46 main:512] : INFO : Loading config [2022-04-20 15:57:46 main:514] : INFO : Loading state [2022-04-20 15:57:47 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 15:57:47 main:562] : INFO : Starting Training [2022-04-20 15:57:52 main:574] : INFO : Epoch 1972 | loss: 0.0311763 | val_loss: 0.0311653 | Time: 4612.25 ms [2022-04-20 15:57:56 main:574] : INFO : Epoch 1973 | loss: 0.0311429 | val_loss: 0.0311594 | Time: 4391.17 ms [2022-04-20 15:58:00 main:574] : INFO : Epoch 1974 | loss: 0.0311389 | val_loss: 0.0311659 | Time: 4274.82 ms [2022-04-20 15:58:04 main:574] : INFO : Epoch 1975 | loss: 0.0311373 | val_loss: 0.0311641 | Time: 4207.2 ms [2022-04-20 15:58:09 main:574] : INFO : Epoch 1976 | loss: 0.0311358 | val_loss: 0.0311632 | Time: 4236.27 ms [2022-04-20 15:58:13 main:574] : INFO : Epoch 1977 | loss: 0.0311357 | val_loss: 0.0311617 | Time: 4217.36 ms [2022-04-20 15:58:17 main:574] : INFO : Epoch 1978 | loss: 0.0311351 | val_loss: 0.0311598 | Time: 4212.62 ms [2022-04-20 15:58:21 main:574] : INFO : Epoch 1979 | loss: 0.0311384 | val_loss: 0.031162 | Time: 4233.69 ms [2022-04-20 15:58:26 main:574] : INFO : Epoch 1980 | loss: 0.0311379 | val_loss: 0.0311581 | Time: 4247.14 ms [2022-04-20 15:58:30 main:574] : INFO : Epoch 1981 | loss: 0.0311371 | val_loss: 0.0311659 | Time: 4202.31 ms [2022-04-20 15:58:34 main:574] : INFO : Epoch 1982 | loss: 0.0311378 | val_loss: 0.0311622 | Time: 4328.39 ms [2022-04-20 15:58:38 main:574] : INFO : Epoch 1983 | loss: 0.0311357 | val_loss: 0.0311639 | Time: 4217.48 ms [2022-04-20 15:58:43 main:574] : INFO : Epoch 1984 | loss: 0.0311342 | val_loss: 0.0311551 | Time: 4215.8 ms [2022-04-20 15:58:47 main:574] : INFO : Epoch 1985 | loss: 0.0311334 | val_loss: 0.0311584 | Time: 4363.06 ms [2022-04-20 15:58:51 main:574] : INFO : Epoch 1986 | loss: 0.0311364 | val_loss: 0.0311571 | Time: 4287.08 ms [2022-04-20 15:58:55 main:574] : INFO : Epoch 1987 | loss: 0.0311372 | val_loss: 0.0311595 | Time: 4210.6 ms [2022-04-20 15:59:00 main:574] : INFO : Epoch 1988 | loss: 0.0311372 | val_loss: 0.0311626 | Time: 4252.46 ms [2022-04-20 15:59:04 main:574] : INFO : Epoch 1989 | loss: 0.0311365 | val_loss: 0.0311611 | Time: 4271.45 ms [2022-04-20 15:59:08 main:574] : INFO : Epoch 1990 | loss: 0.031142 | val_loss: 0.0311896 | Time: 4259.78 ms [2022-04-20 15:59:13 main:574] : INFO : Epoch 1991 | loss: 0.0311511 | val_loss: 0.0311681 | Time: 4237.07 ms [2022-04-20 15:59:17 main:574] : INFO : Epoch 1992 | loss: 0.03115 | val_loss: 0.0311654 | Time: 4264.78 ms [2022-04-20 15:59:21 main:574] : INFO : Epoch 1993 | loss: 0.0311487 | val_loss: 0.031166 | Time: 4287.02 ms [2022-04-20 15:59:25 main:574] : INFO : Epoch 1994 | loss: 0.0311474 | val_loss: 0.0311671 | Time: 4210.48 ms [2022-04-20 15:59:29 main:574] : INFO : Epoch 1995 | loss: 0.0311485 | val_loss: 0.0311803 | Time: 4124.09 ms [2022-04-20 15:59:34 main:574] : INFO : Epoch 1996 | loss: 0.031147 | val_loss: 0.0311659 | Time: 4194.77 ms [2022-04-20 15:59:38 main:574] : INFO : Epoch 1997 | loss: 0.0311439 | val_loss: 0.0311687 | Time: 4238.88 ms [2022-04-20 15:59:43 main:574] : INFO : Epoch 1998 | loss: 0.0311439 | val_loss: 0.0311652 | Time: 4629.89 ms [2022-04-20 15:59:47 main:574] : INFO : Epoch 1999 | loss: 0.0311494 | val_loss: 0.0311728 | Time: 4250.78 ms [2022-04-20 15:59:51 main:574] : INFO : Epoch 2000 | loss: 0.031161 | val_loss: 0.0311728 | Time: 4219.82 ms [2022-04-20 15:59:55 main:574] : INFO : Epoch 2001 | loss: 0.031158 | val_loss: 0.0311727 | Time: 4171.89 ms [2022-04-20 15:59:59 main:574] : INFO : Epoch 2002 | loss: 0.0311529 | val_loss: 0.0311723 | Time: 4175.8 ms [2022-04-20 16:00:04 main:574] : INFO : Epoch 2003 | loss: 0.0311509 | val_loss: 0.0311739 | Time: 4212.84 ms [2022-04-20 16:00:08 main:574] : INFO : Epoch 2004 | loss: 0.0311491 | val_loss: 0.0311679 | Time: 4281.59 ms [2022-04-20 16:00:12 main:574] : INFO : Epoch 2005 | loss: 0.0311461 | val_loss: 0.0311679 | Time: 4340 ms [2022-04-20 16:00:16 main:574] : INFO : Epoch 2006 | loss: 0.0311453 | val_loss: 0.0311692 | Time: 4204.26 ms [2022-04-20 16:00:21 main:574] : INFO : Epoch 2007 | loss: 0.0311454 | val_loss: 0.0311694 | Time: 4313.1 ms [2022-04-20 16:00:25 main:574] : INFO : Epoch 2008 | loss: 0.0311443 | val_loss: 0.0311695 | Time: 4222.26 ms [2022-04-20 16:00:29 main:574] : INFO : Epoch 2009 | loss: 0.0311472 | val_loss: 0.0311766 | Time: 4212.78 ms [2022-04-20 16:00:33 main:574] : INFO : Epoch 2010 | loss: 0.031149 | val_loss: 0.0311658 | Time: 4141.58 ms [2022-04-20 16:00:37 main:574] : INFO : Epoch 2011 | loss: 0.0311452 | val_loss: 0.0311689 | Time: 4111.48 ms [2022-04-20 16:00:42 main:574] : INFO : Epoch 2012 | loss: 0.0311437 | val_loss: 0.0311703 | Time: 4146.43 ms [2022-04-20 16:00:46 main:574] : INFO : Epoch 2013 | loss: 0.0311437 | val_loss: 0.0311833 | Time: 4146.14 ms [2022-04-20 16:00:50 main:574] : INFO : Epoch 2014 | loss: 0.0311419 | val_loss: 0.0311685 | Time: 4224.32 ms [2022-04-20 16:00:54 main:574] : INFO : Epoch 2015 | loss: 0.0311431 | val_loss: 0.0311672 | Time: 4201.37 ms [2022-04-20 16:00:58 main:574] : INFO : Epoch 2016 | loss: 0.0311523 | val_loss: 0.0311725 | Time: 4162.27 ms [2022-04-20 16:01:03 main:574] : INFO : Epoch 2017 | loss: 0.0311591 | val_loss: 0.0311721 | Time: 4213.73 ms [2022-04-20 16:01:07 main:574] : INFO : Epoch 2018 | loss: 0.0311595 | val_loss: 0.0311764 | Time: 4201.98 ms [2022-04-20 16:01:11 main:574] : INFO : Epoch 2019 | loss: 0.031162 | val_loss: 0.0311763 | Time: 4153.56 ms [2022-04-20 16:01:15 main:574] : INFO : Epoch 2020 | loss: 0.0311574 | val_loss: 0.0311757 | Time: 4229.84 ms [2022-04-20 16:01:19 main:574] : INFO : Epoch 2021 | loss: 0.0311558 | val_loss: 0.0311767 | Time: 4224 ms [2022-04-20 16:01:24 main:574] : INFO : Epoch 2022 | loss: 0.0311526 | val_loss: 0.0311735 | Time: 4161.41 ms [2022-04-20 16:01:28 main:574] : INFO : Epoch 2023 | loss: 0.0311509 | val_loss: 0.0311725 | Time: 4237.22 ms [2022-04-20 16:01:32 main:574] : INFO : Epoch 2024 | loss: 0.0311505 | val_loss: 0.0311692 | Time: 4150.87 ms [2022-04-20 16:01:36 main:574] : INFO : Epoch 2025 | loss: 0.031152 | val_loss: 0.0311783 | Time: 4234.05 ms [2022-04-20 16:01:40 main:574] : INFO : Epoch 2026 | loss: 0.0311548 | val_loss: 0.0311723 | Time: 4207.01 ms [2022-04-20 16:01:45 main:574] : INFO : Epoch 2027 | loss: 0.0311502 | val_loss: 0.031174 | Time: 4213.99 ms [2022-04-20 16:01:49 main:574] : INFO : Epoch 2028 | loss: 0.031148 | val_loss: 0.0311692 | Time: 4234.23 ms [2022-04-20 16:01:53 main:574] : INFO : Epoch 2029 | loss: 0.0311461 | val_loss: 0.0311784 | Time: 4173.18 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-20 16:06:46 main:435] : INFO : Set logging level to 1 [2022-04-20 16:06:46 main:441] : INFO : Running in BOINC Client mode [2022-04-20 16:06:46 main:444] : INFO : Resolving all filenames [2022-04-20 16:06:46 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-20 16:06:46 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-20 16:06:46 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-20 16:06:46 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-20 16:06:46 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-20 16:06:46 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-20 16:06:46 main:474] : INFO : Configuration: [2022-04-20 16:06:46 main:475] : INFO : Model type: GRU [2022-04-20 16:06:46 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-20 16:06:46 main:477] : INFO : Max Epochs: 2048 [2022-04-20 16:06:46 main:478] : INFO : Batch Size: 128 [2022-04-20 16:06:46 main:479] : INFO : Learning Rate: 0.01 [2022-04-20 16:06:46 main:480] : INFO : Patience: 10 [2022-04-20 16:06:46 main:481] : INFO : Hidden Width: 12 [2022-04-20 16:06:46 main:482] : INFO : # Recurrent Layers: 4 [2022-04-20 16:06:46 main:483] : INFO : # Backend Layers: 4 [2022-04-20 16:06:46 main:484] : INFO : # Threads: 1 [2022-04-20 16:06:46 main:486] : INFO : Preparing Dataset [2022-04-20 16:06:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-20 16:06:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-20 16:06:48 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-20 16:06:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-20 16:06:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-20 16:06:48 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-20 16:06:48 main:494] : INFO : Creating Model [2022-04-20 16:06:48 main:507] : INFO : Preparing config file [2022-04-20 16:06:48 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-20 16:06:48 main:512] : INFO : Loading config [2022-04-20 16:06:48 main:514] : INFO : Loading state [2022-04-20 16:06:49 main:559] : INFO : Loading DataLoader into Memory [2022-04-20 16:06:49 main:562] : INFO : Starting Training [2022-04-20 16:06:53 main:574] : INFO : Epoch 2019 | loss: 0.0311936 | val_loss: 0.0311766 | Time: 4468.15 ms [2022-04-20 16:06:58 main:574] : INFO : Epoch 2020 | loss: 0.0311576 | val_loss: 0.031172 | Time: 4185.41 ms [2022-04-20 16:07:02 main:574] : INFO : Epoch 2021 | loss: 0.0311515 | val_loss: 0.0311706 | Time: 4260.01 ms [2022-04-20 16:07:06 main:574] : INFO : Epoch 2022 | loss: 0.0311492 | val_loss: 0.0311695 | Time: 4288.11 ms [2022-04-20 16:07:10 main:574] : INFO : Epoch 2023 | loss: 0.0311462 | val_loss: 0.0311708 | Time: 4196.79 ms [2022-04-20 16:07:15 main:574] : INFO : Epoch 2024 | loss: 0.0311449 | val_loss: 0.0311666 | Time: 4246.64 ms [2022-04-20 16:07:19 main:574] : INFO : Epoch 2025 | loss: 0.0311455 | val_loss: 0.0311646 | Time: 4217.1 ms [2022-04-20 16:07:23 main:574] : INFO : Epoch 2026 | loss: 0.0311462 | val_loss: 0.0311703 | Time: 4342.77 ms [2022-04-20 16:07:28 main:574] : INFO : Epoch 2027 | loss: 0.0311453 | val_loss: 0.0311641 | Time: 4319.56 ms [2022-04-20 16:07:32 main:574] : INFO : Epoch 2028 | loss: 0.0311441 | val_loss: 0.0311636 | Time: 4256.19 ms [2022-04-20 16:07:36 main:574] : INFO : Epoch 2029 | loss: 0.0311432 | val_loss: 0.0311652 | Time: 4191.05 ms [2022-04-20 16:07:40 main:574] : INFO : Epoch 2030 | loss: 0.0311439 | val_loss: 0.0311623 | Time: 4241.88 ms [2022-04-20 16:07:45 main:574] : INFO : Epoch 2031 | loss: 0.0311413 | val_loss: 0.0311665 | Time: 4276.1 ms [2022-04-20 16:07:49 main:574] : INFO : Epoch 2032 | loss: 0.0311406 | val_loss: 0.0311696 | Time: 4301.3 ms [2022-04-20 16:07:53 main:574] : INFO : Epoch 2033 | loss: 0.0311392 | val_loss: 0.0311633 | Time: 4298.39 ms [2022-04-20 16:07:57 main:574] : INFO : Epoch 2034 | loss: 0.0311395 | val_loss: 0.0311661 | Time: 4318.95 ms [2022-04-20 16:08:02 main:574] : INFO : Epoch 2035 | loss: 0.031137 | val_loss: 0.031165 | Time: 4236.47 ms [2022-04-20 16:08:06 main:574] : INFO : Epoch 2036 | loss: 0.0311377 | val_loss: 0.0311639 | Time: 4263.97 ms [2022-04-20 16:08:10 main:574] : INFO : Epoch 2037 | loss: 0.0311388 | val_loss: 0.0311682 | Time: 4234.35 ms [2022-04-20 16:08:14 main:574] : INFO : Epoch 2038 | loss: 0.031139 | val_loss: 0.0311578 | Time: 4161.26 ms [2022-04-20 16:08:19 main:574] : INFO : Epoch 2039 | loss: 0.031141 | val_loss: 0.0311642 | Time: 4276.88 ms [2022-04-20 16:08:23 main:574] : INFO : Epoch 2040 | loss: 0.0311474 | val_loss: 0.0311624 | Time: 4298.01 ms [2022-04-20 16:08:27 main:574] : INFO : Epoch 2041 | loss: 0.031146 | val_loss: 0.031162 | Time: 4202.28 ms [2022-04-20 16:08:31 main:574] : INFO : Epoch 2042 | loss: 0.0311449 | val_loss: 0.0311601 | Time: 4193.71 ms [2022-04-20 16:08:36 main:574] : INFO : Epoch 2043 | loss: 0.0311453 | val_loss: 0.0311619 | Time: 4278.77 ms [2022-04-20 16:08:40 main:574] : INFO : Epoch 2044 | loss: 0.031143 | val_loss: 0.0311611 | Time: 4516.59 ms [2022-04-20 16:08:45 main:574] : INFO : Epoch 2045 | loss: 0.0311453 | val_loss: 0.031159 | Time: 4620.49 ms [2022-04-20 16:08:49 main:574] : INFO : Epoch 2046 | loss: 0.0311408 | val_loss: 0.0311637 | Time: 4297.08 ms [2022-04-20 16:08:53 main:574] : INFO : Epoch 2047 | loss: 0.0311406 | val_loss: 0.0311606 | Time: 4210.03 ms [2022-04-20 16:08:58 main:574] : INFO : Epoch 2048 | loss: 0.0311409 | val_loss: 0.0311656 | Time: 4222.3 ms [2022-04-20 16:08:58 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311656 [2022-04-20 16:08:58 main:603] : INFO : Saving end state to config to file [2022-04-20 16:08:58 main:608] : INFO : Success, exiting.. 16:08:58 (66284): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)