Task 13810411
| Name | ParityModified-1647049340-2941-2-0_0 |
| Workunit | 10893067 |
| Created | 26 Mar 2022, 17:31:20 UTC |
| Sent | 1 Apr 2022, 22:18:02 UTC |
| Report deadline | 9 Apr 2022, 22:18:02 UTC |
| Received | 7 Apr 2022, 21:17:49 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 6170 |
| Run time | 6 hours 44 min 23 sec |
| CPU time | 5 hours 55 min 21 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 1,130.52 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.57 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 0.0311198 | val_loss: 0.0311579 | Time: 15523.1 ms [2022-04-07 00:11:19 main:574] : INFO : Epoch 1756 | loss: 0.0311226 | val_loss: 0.0311514 | Time: 13040.5 ms [2022-04-07 00:11:35 main:574] : INFO : Epoch 1757 | loss: 0.0311189 | val_loss: 0.0311537 | Time: 15715.5 ms [2022-04-07 00:11:47 main:574] : INFO : Epoch 1758 | loss: 0.0311152 | val_loss: 0.0311516 | Time: 11836.5 ms [2022-04-07 00:12:00 main:574] : INFO : Epoch 1759 | loss: 0.0311108 | val_loss: 0.0311519 | Time: 12610.7 ms [2022-04-07 00:12:10 main:574] : INFO : Epoch 1760 | loss: 0.0311088 | val_loss: 0.0311539 | Time: 10112.5 ms [2022-04-07 00:12:25 main:574] : INFO : Epoch 1761 | loss: 0.0311055 | val_loss: 0.0311489 | Time: 15311.7 ms [2022-04-07 00:12:39 main:574] : INFO : Epoch 1762 | loss: 0.0311059 | val_loss: 0.0311491 | Time: 13460.2 ms Machine Learning Dataset Generator vMachine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 12:48:07 main:435] : INFO : Set logging level to 1 [2022-04-07 12:48:07 main:441] : INFO : Running in BOINC Client mode [2022-04-07 12:48:07 main:444] : INFO : Resolving all filenames [2022-04-07 12:48:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 12:48:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 12:48:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 12:48:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 12:48:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 12:48:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 12:48:07 main:474] : INFO : Configuration: [2022-04-07 12:48:07 main:475] : INFO : Model type: GRU [2022-04-07 12:48:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 12:48:07 main:477] : INFO : Max Epochs: 2048 [2022-04-07 12:48:07 main:478] : INFO : Batch Size: 128 [2022-04-07 12:48:07 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 12:48:07 main:480] : INFO : Patience: 10 [2022-04-07 12:48:07 main:481] : INFO : Hidden Width: 12 [2022-04-07 12:48:07 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 12:48:07 main:483] : INFO : # Backend Layers: 4 [2022-04-07 12:48:07 main:484] : INFO : # Threads: 1 [2022-04-07 12:48:08 main:486] : INFO : Preparing Dataset [2022-04-07 12:48:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 12:48:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 12:48:16 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 12:48:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 12:48:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 12:48:17 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 12:48:17 main:494] : INFO : Creating Model [2022-04-07 12:48:17 main:507] : INFO : Preparing config file [2022-04-07 12:48:17 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 12:48:17 main:512] : INFO : Loading config [2022-04-07 12:48:17 main:514] : INFO : Loading state Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 12:50:20 main:435] : INFO : Set logging level to 1 [2022-04-07 12:50:20 main:441] : INFO : Running in BOINC Client mode [2022-04-07 12:50:20 main:444] : INFO : Resolving all filenames [2022-04-07 12:50:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 12:50:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 12:50:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 12:50:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 12:50:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 12:50:20 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 12:50:20 main:474] : INFO : Configuration: [2022-04-07 12:50:20 main:475] : INFO : Model type: GRU [2022-04-07 12:50:20 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 12:50:20 main:477] : INFO : Max Epochs: 2048 [2022-04-07 12:50:20 main:478] : INFO : Batch Size: 128 [2022-04-07 12:50:21 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 12:50:21 main:480] : INFO : Patience: 10 [2022-04-07 12:50:21 main:481] : INFO : Hidden Width: 12 [2022-04-07 12:50:21 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 12:50:21 main:483] : INFO : # Backend Layers: 4 [2022-04-07 12:50:21 main:484] : INFO : # Threads: 1 [2022-04-07 12:50:21 main:486] : INFO : Preparing Dataset [2022-04-07 12:50:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 12:50:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 12:50:27 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 12:50:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 12:50:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 12:50:28 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 12:50:28 main:494] : INFO : Creating Model [2022-04-07 12:50:28 main:507] : INFO : Preparing config file [2022-04-07 12:50:29 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 12:50:29 main:512] : INFO : Loading config [2022-04-07 12:50:29 main:514] : INFO : Loading state [2022-04-07 12:50:32 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 12:50:32 main:562] : INFO : Starting Training [2022-04-07 12:50:46 main:574] : INFO : Epoch 1762 | loss: 0.03114 | val_loss: 0.031156 | Time: 13711 ms [2022-04-07 12:50:59 main:574] : INFO : Epoch 1763 | loss: 0.0311061 | val_loss: 0.0311506 | Time: 12999.1 ms [2022-04-07 12:51:06 main:574] : INFO : Epoch 1764 | loss: 0.0311064 | val_loss: 0.0311514 | Time: 7893.64 ms [2022-04-07 12:51:15 main:574] : INFO : Epoch 1765 | loss: 0.0311047 | val_loss: 0.0311455 | Time: 8105.43 ms [2022-04-07 12:51:29 main:574] : INFO : Epoch 1766 | loss: 0.0311038 | val_loss: 0.0311592 | Time: 14080 ms [2022-04-07 12:51:44 main:574] : INFO : Epoch 1767 | loss: 0.0311051 | val_loss: 0.0311497 | Time: 15545.6 ms [2022-04-07 12:51:58 main:574] : INFO : Epoch 1768 | loss: 0.0311014 | val_loss: 0.0311493 | Time: 13178.4 ms [2022-04-07 12:52:09 main:574] : INFO : Epoch 1769 | loss: 0.0311003 | val_loss: 0.0311494 | Time: 11628.1 ms [2022-04-07 12:52:21 main:574] : INFO : Epoch 1770 | loss: 0.0311001 | val_loss: 0.0311508 | Time: 11599.8 ms [2022-04-07 12:52:34 main:574] : INFO : Epoch 1771 | loss: 0.0311035 | val_loss: 0.0311559 | Time: 12487.1 ms [2022-04-07 12:52:48 main:574] : INFO : Epoch 1772 | loss: 0.0311032 | val_loss: 0.0311552 | Time: 13840.3 ms [2022-04-07 12:53:01 main:574] : INFO : Epoch 1773 | loss: 0.0311032 | val_loss: 0.0311679 | Time: 13128.3 ms [2022-04-07 12:53:10 main:574] : INFO : Epoch 1774 | loss: 0.0311053 | val_loss: 0.0311497 | Time: 9607.43 ms [2022-04-07 12:53:18 main:574] : INFO : Epoch 1775 | loss: 0.0311058 | val_loss: 0.0311498 | Time: 7494.12 ms [2022-04-07 12:53:25 main:574] : INFO : Epoch 1776 | loss: 0.0311074 | val_loss: 0.0311493 | Time: 7336.45 ms [2022-04-07 12:53:40 main:574] : INFO : Epoch 1777 | loss: 0.0311058 | val_loss: 0.0311541 | Time: 14192.9 ms [2022-04-07 12:53:53 main:574] : INFO : Epoch 1778 | loss: 0.0311038 | val_loss: 0.0311507 | Time: 13395.3 ms [2022-04-07 12:54:08 main:574] : INFO : Epoch 1779 | loss: 0.0311058 | val_loss: 0.0311536 | Time: 14812.2 ms [2022-04-07 12:54:20 main:574] : INFO : Epoch 1780 | loss: 0.0311042 | val_loss: 0.0311529 | Time: 12212.3 ms [2022-04-07 12:54:29 main:574] : INFO : Epoch 1781 | loss: 0.0311032 | val_loss: 0.0311544 | Time: 8530.4 ms [2022-04-07 12:54:43 main:574] : INFO : Epoch 1782 | loss: 0.0311046 | val_loss: 0.0311567 | Time: 14037.8 ms [2022-04-07 12:54:58 main:574] : INFO : Epoch 1783 | loss: 0.0311037 | val_loss: 0.031158 | Time: 14686.2 ms [2022-04-07 12:55:12 main:574] : INFO : Epoch 1784 | loss: 0.031102 | val_loss: 0.0311494 | Time: 14321.3 ms [2022-04-07 12:55:26 main:574] : INFO : Epoch 1785 | loss: 0.0311015 | val_loss: 0.031158 | Time: 13445.4 ms [2022-04-07 12:55:39 main:574] : INFO : Epoch 1786 | loss: 0.0310992 | val_loss: 0.0311547 | Time: 13244.8 ms [2022-04-07 12:55:50 main:574] : INFO : Epoch 1787 | loss: 0.0310982 | val_loss: 0.0311494 | Time: 11257.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 13:01:47 main:435] : INFO : Set logging level to 1 [2022-04-07 13:01:47 main:441] : INFO : Running in BOINC Client mode [2022-04-07 13:01:47 main:444] : INFO : Resolving all filenames [2022-04-07 13:01:47 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 13:01:47 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 13:01:47 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 13:01:47 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 13:01:47 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 13:01:47 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 13:01:47 main:474] : INFO : Configuration: [2022-04-07 13:01:47 main:475] : INFO : Model type: GRU [2022-04-07 13:01:47 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 13:01:48 main:477] : INFO : Max Epochs: 2048 [2022-04-07 13:01:48 main:478] : INFO : Batch Size: 128 [2022-04-07 13:01:48 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 13:01:48 main:480] : INFO : Patience: 10 [2022-04-07 13:01:48 main:481] : INFO : Hidden Width: 12 [2022-04-07 13:01:48 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 13:01:48 main:483] : INFO : # Backend Layers: 4 [2022-04-07 13:01:48 main:484] : INFO : # Threads: 1 [2022-04-07 13:01:48 main:486] : INFO : Preparing Dataset [2022-04-07 13:01:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 13:01:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 13:01:57 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 13:01:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 13:01:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 13:01:58 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 13:01:58 main:494] : INFO : Creating Model [2022-04-07 13:01:58 main:507] : INFO : Preparing config file [2022-04-07 13:01:58 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 13:01:58 main:512] : INFO : Loading config [2022-04-07 13:01:58 main:514] : INFO : Loading state [2022-04-07 13:02:02 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 13:02:02 main:562] : INFO : Starting Training [2022-04-07 13:02:15 main:574] : INFO : Epoch 1788 | loss: 0.0311224 | val_loss: 0.0311603 | Time: 13210.1 ms [2022-04-07 13:02:26 main:574] : INFO : Epoch 1789 | loss: 0.0311034 | val_loss: 0.031155 | Time: 11355.1 ms [2022-04-07 13:02:38 main:574] : INFO : Epoch 1790 | loss: 0.0311029 | val_loss: 0.0311533 | Time: 11607.4 ms [2022-04-07 13:02:50 main:574] : INFO : Epoch 1791 | loss: 0.0311049 | val_loss: 0.0311519 | Time: 12079 ms [2022-04-07 13:02:58 main:574] : INFO : Epoch 1792 | loss: 0.0311004 | val_loss: 0.0311578 | Time: 7995.16 ms [2022-04-07 13:03:10 main:574] : INFO : Epoch 1793 | loss: 0.0310994 | val_loss: 0.0311536 | Time: 12089.3 ms [2022-04-07 13:03:19 main:574] : INFO : Epoch 1794 | loss: 0.0310976 | val_loss: 0.0311524 | Time: 8816.09 ms [2022-04-07 13:03:27 main:574] : INFO : Epoch 1795 | loss: 0.031097 | val_loss: 0.0311543 | Time: 7864.65 ms [2022-04-07 13:03:35 main:574] : INFO : Epoch 1796 | loss: 0.0310954 | val_loss: 0.0311518 | Time: 7356.07 ms [2022-04-07 13:03:46 main:574] : INFO : Epoch 1797 | loss: 0.0310986 | val_loss: 0.0311565 | Time: 11261.8 ms [2022-04-07 13:03:58 main:574] : INFO : Epoch 1798 | loss: 0.0311005 | val_loss: 0.0311574 | Time: 11803.1 ms [2022-04-07 13:04:10 main:574] : INFO : Epoch 1799 | loss: 0.0310983 | val_loss: 0.0311542 | Time: 12009.8 ms [2022-04-07 13:04:22 main:574] : INFO : Epoch 1800 | loss: 0.0310976 | val_loss: 0.0311574 | Time: 11377.9 ms [2022-04-07 13:04:29 main:574] : INFO : Epoch 1801 | loss: 0.0310953 | val_loss: 0.0311549 | Time: 7316.76 ms [2022-04-07 13:04:41 main:574] : INFO : Epoch 1802 | loss: 0.0310958 | val_loss: 0.0311501 | Time: 12339.9 ms [2022-04-07 13:04:50 main:574] : INFO : Epoch 1803 | loss: 0.0310945 | val_loss: 0.0311587 | Time: 8272.92 ms [2022-04-07 13:05:01 main:574] : INFO : Epoch 1804 | loss: 0.0310951 | val_loss: 0.0311535 | Time: 11326.4 ms [2022-04-07 13:05:13 main:574] : INFO : Epoch 1805 | loss: 0.0310942 | val_loss: 0.0311523 | Time: 11787.7 ms [2022-04-07 13:05:21 main:574] : INFO : Epoch 1806 | loss: 0.0310984 | val_loss: 0.0311556 | Time: 8047.62 ms [2022-04-07 13:05:28 main:574] : INFO : Epoch 1807 | loss: 0.0310994 | val_loss: 0.0311559 | Time: 7343.24 ms [2022-04-07 13:05:40 main:574] : INFO : Epoch 1808 | loss: 0.0310958 | val_loss: 0.0311566 | Time: 11483.9 ms [2022-04-07 13:05:48 main:574] : INFO : Epoch 1809 | loss: 0.0310974 | val_loss: 0.031156 | Time: 8129.91 ms [2022-04-07 13:06:01 main:574] : INFO : Epoch 1810 | loss: 0.0310975 | val_loss: 0.0311574 | Time: 12544.5 ms [2022-04-07 13:06:09 main:574] : INFO : Epoch 1811 | loss: 0.0311025 | val_loss: 0.0311545 | Time: 7998.17 ms [2022-04-07 13:06:20 main:574] : INFO : Epoch 1812 | loss: 0.0310984 | val_loss: 0.0311569 | Time: 11126.6 ms [2022-04-07 13:06:34 main:574] : INFO : Epoch 1813 | loss: 0.0310987 | val_loss: 0.031151 | Time: 13697.6 ms [2022-04-07 13:06:46 main:574] : INFO : Epoch 1814 | loss: 0.0310985 | val_loss: 0.0311596 | Time: 12153.9 ms [2022-04-07 13:06:58 main:574] : INFO : Epoch 1815 | loss: 0.0310979 | val_loss: 0.0311524 | Time: 12071.3 ms [2022-04-07 13:07:10 main:574] : INFO : Epoch 1816 | loss: 0.0311004 | val_loss: 0.0311519 | Time: 11443.6 ms [2022-04-07 13:07:22 main:574] : INFO : Epoch 1817 | loss: 0.0310983 | val_loss: 0.0311493 | Time: 12258.2 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 13:11:41 main:435] : INFO : Set logging level to 1 [2022-04-07 13:11:41 main:441] : INFO : Running in BOINC Client mode [2022-04-07 13:11:41 main:444] : INFO : Resolving all filenames [2022-04-07 13:11:41 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 13:11:41 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 13:11:41 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 13:11:41 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 13:11:41 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 13:11:41 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 13:11:41 main:474] : INFO : Configuration: [2022-04-07 13:11:41 main:475] : INFO : Model type: GRU [2022-04-07 13:11:41 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 13:11:41 main:477] : INFO : Max Epochs: 2048 [2022-04-07 13:11:41 main:478] : INFO : Batch Size: 128 [2022-04-07 13:11:41 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 13:11:42 main:480] : INFO : Patience: 10 [2022-04-07 13:11:42 main:481] : INFO : Hidden Width: 12 [2022-04-07 13:11:42 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 13:11:42 main:483] : INFO : # Backend Layers: 4 [2022-04-07 13:11:42 main:484] : INFO : # Threads: 1 [2022-04-07 13:11:42 main:486] : INFO : Preparing Dataset [2022-04-07 13:11:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 13:11:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 13:11:52 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 13:11:52 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 13:11:52 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 13:11:53 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 13:11:53 main:494] : INFO : Creating Model [2022-04-07 13:11:53 main:507] : INFO : Preparing config file [2022-04-07 13:11:53 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 13:11:53 main:512] : INFO : Loading config [2022-04-07 13:11:53 main:514] : INFO : Loading state [2022-04-07 13:11:56 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 13:11:57 main:562] : INFO : Starting Training [2022-04-07 13:12:09 main:574] : INFO : Epoch 1817 | loss: 0.0311282 | val_loss: 0.0311548 | Time: 12907.8 ms [2022-04-07 13:12:21 main:574] : INFO : Epoch 1818 | loss: 0.0311035 | val_loss: 0.031151 | Time: 11170.2 ms [2022-04-07 13:12:31 main:574] : INFO : Epoch 1819 | loss: 0.0311 | val_loss: 0.0311556 | Time: 10364.2 ms [2022-04-07 13:12:38 main:574] : INFO : Epoch 1820 | loss: 0.0310987 | val_loss: 0.0311537 | Time: 6672.31 ms [2022-04-07 13:12:45 main:574] : INFO : Epoch 1821 | loss: 0.0310973 | val_loss: 0.0311567 | Time: 6920.05 ms [2022-04-07 13:12:52 main:574] : INFO : Epoch 1822 | loss: 0.0310995 | val_loss: 0.0311563 | Time: 6941.36 ms [2022-04-07 13:12:59 main:574] : INFO : Epoch 1823 | loss: 0.0310971 | val_loss: 0.0311581 | Time: 6903.81 ms [2022-04-07 13:13:06 main:574] : INFO : Epoch 1824 | loss: 0.031096 | val_loss: 0.0311559 | Time: 6905.47 ms [2022-04-07 13:13:13 main:574] : INFO : Epoch 1825 | loss: 0.0310979 | val_loss: 0.0311569 | Time: 7012.05 ms [2022-04-07 13:13:20 main:574] : INFO : Epoch 1826 | loss: 0.0310958 | val_loss: 0.0311529 | Time: 6840.98 ms [2022-04-07 13:13:27 main:574] : INFO : Epoch 1827 | loss: 0.0310965 | val_loss: 0.0311555 | Time: 6907.59 ms [2022-04-07 13:13:38 main:574] : INFO : Epoch 1828 | loss: 0.0310958 | val_loss: 0.0311619 | Time: 11311.6 ms [2022-04-07 13:13:45 main:574] : INFO : Epoch 1829 | loss: 0.0310943 | val_loss: 0.0311544 | Time: 7187.32 ms [2022-04-07 13:13:52 main:574] : INFO : Epoch 1830 | loss: 0.0310938 | val_loss: 0.031159 | Time: 6880.2 ms [2022-04-07 13:14:03 main:574] : INFO : Epoch 1831 | loss: 0.0310963 | val_loss: 0.0311581 | Time: 10568 ms [2022-04-07 13:14:10 main:574] : INFO : Epoch 1832 | loss: 0.0310976 | val_loss: 0.0311562 | Time: 6807.64 ms [2022-04-07 13:14:20 main:574] : INFO : Epoch 1833 | loss: 0.0310947 | val_loss: 0.0311563 | Time: 10437.6 ms [2022-04-07 13:14:27 main:574] : INFO : Epoch 1834 | loss: 0.0310952 | val_loss: 0.0311655 | Time: 7017.13 ms [2022-04-07 13:14:38 main:574] : INFO : Epoch 1835 | loss: 0.031096 | val_loss: 0.0311544 | Time: 10655.9 ms [2022-04-07 13:14:45 main:574] : INFO : Epoch 1836 | loss: 0.0310975 | val_loss: 0.0311587 | Time: 7045.59 ms [2022-04-07 13:14:56 main:574] : INFO : Epoch 1837 | loss: 0.0311018 | val_loss: 0.0311597 | Time: 10578.5 ms [2022-04-07 13:15:07 main:574] : INFO : Epoch 1838 | loss: 0.0311002 | val_loss: 0.0311615 | Time: 11050.3 ms [2022-04-07 13:15:14 main:574] : INFO : Epoch 1839 | loss: 0.0310965 | val_loss: 0.0311589 | Time: 6869.08 ms [2022-04-07 13:15:21 main:574] : INFO : Epoch 1840 | loss: 0.0310982 | val_loss: 0.0311564 | Time: 6673.5 ms [2022-04-07 13:15:27 main:574] : INFO : Epoch 1841 | loss: 0.0310962 | val_loss: 0.0311569 | Time: 6593.02 ms [2022-04-07 13:15:34 main:574] : INFO : Epoch 1842 | loss: 0.0311003 | val_loss: 0.0311577 | Time: 6917.44 ms [2022-04-07 13:15:41 main:574] : INFO : Epoch 1843 | loss: 0.0310998 | val_loss: 0.0311526 | Time: 6739.7 ms [2022-04-07 13:15:52 main:574] : INFO : Epoch 1844 | loss: 0.0310988 | val_loss: 0.0311557 | Time: 10336.1 ms [2022-04-07 13:15:58 main:574] : INFO : Epoch 1845 | loss: 0.0311004 | val_loss: 0.0311542 | Time: 6749.01 ms [2022-04-07 13:16:05 main:574] : INFO : Epoch 1846 | loss: 0.031102 | val_loss: 0.0311584 | Time: 6743.78 ms [2022-04-07 13:16:16 main:574] : INFO : Epoch 1847 | loss: 0.0311003 | val_loss: 0.0311593 | Time: 10512.1 ms [2022-04-07 13:16:27 main:574] : INFO : Epoch 1848 | loss: 0.0310993 | val_loss: 0.0311656 | Time: 10900.3 ms [2022-04-07 13:16:37 main:574] : INFO : Epoch 1849 | loss: 0.0310977 | val_loss: 0.0311526 | Time: 10636 ms [2022-04-07 13:16:44 main:574] : INFO : Epoch 1850 | loss: 0.031102 | val_loss: 0.0311592 | Time: 6946.81 ms [2022-04-07 13:16:51 main:574] : INFO : Epoch 1851 | loss: 0.0311072 | val_loss: 0.0311553 | Time: 6718.61 ms [2022-04-07 13:16:58 main:574] : INFO : Epoch 1852 | loss: 0.0311089 | val_loss: 0.0311569 | Time: 6797.69 ms [2022-04-07 13:17:09 main:574] : INFO : Epoch 1853 | loss: 0.0311022 | val_loss: 0.0311579 | Time: 11182.7 ms [2022-04-07 13:17:20 main:574] : INFO : Epoch 1854 | loss: 0.0311059 | val_loss: 0.0311538 | Time: 10610.7 ms [2022-04-07 13:17:30 main:574] : INFO : Epoch 1855 | loss: 0.0311076 | val_loss: 0.0311523 | Time: 10266.5 ms [2022-04-07 13:17:37 main:574] : INFO : Epoch 1856 | loss: 0.0311044 | val_loss: 0.0311583 | Time: 6945.51 ms [2022-04-07 13:17:48 main:574] : INFO : Epoch 1857 | loss: 0.0311027 | val_loss: 0.0311575 | Time: 10442.9 ms [2022-04-07 13:17:59 main:574] : INFO : Epoch 1858 | loss: 0.0311014 | val_loss: 0.0311482 | Time: 10593.2 ms [2022-04-07 13:18:05 main:574] : INFO : Epoch 1859 | loss: 0.0311017 | val_loss: 0.031152 | Time: 6931.68 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 13:20:36 main:435] : INFO : Set logging level to 1 [2022-04-07 13:20:36 main:441] : INFO : Running in BOINC Client mode [2022-04-07 13:20:36 main:444] : INFO : Resolving all filenames [2022-04-07 13:20:36 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 13:20:37 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 13:20:37 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 13:20:37 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 13:20:37 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 13:20:37 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 13:20:37 main:474] : INFO : Configuration: [2022-04-07 13:20:37 main:475] : INFO : Model type: GRU [2022-04-07 13:20:37 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 13:20:37 main:477] : INFO : Max Epochs: 2048 [2022-04-07 13:20:37 main:478] : INFO : Batch Size: 128 [2022-04-07 13:20:37 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 13:20:37 main:480] : INFO : Patience: 10 [2022-04-07 13:20:37 main:481] : INFO : Hidden Width: 12 [2022-04-07 13:20:37 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 13:20:37 main:483] : INFO : # Backend Layers: 4 [2022-04-07 13:20:37 main:484] : INFO : # Threads: 1 [2022-04-07 13:20:37 main:486] : INFO : Preparing Dataset [2022-04-07 13:20:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 13:20:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 13:20:45 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 13:20:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 13:20:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 13:20:46 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 13:20:46 main:494] : INFO : Creating Model [2022-04-07 13:20:46 main:507] : INFO : Preparing config file [2022-04-07 13:20:46 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 13:20:46 main:512] : INFO : Loading config [2022-04-07 13:20:47 main:514] : INFO : Loading state [2022-04-07 13:20:50 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 13:20:50 main:562] : INFO : Starting Training [2022-04-07 13:21:06 main:574] : INFO : Epoch 1854 | loss: 0.0311299 | val_loss: 0.0311619 | Time: 16086.7 ms [2022-04-07 13:21:20 main:574] : INFO : Epoch 1855 | loss: 0.031104 | val_loss: 0.0311632 | Time: 14301.3 ms [2022-04-07 13:21:31 main:574] : INFO : Epoch 1856 | loss: 0.0311002 | val_loss: 0.0311526 | Time: 10643.6 ms [2022-04-07 13:21:45 main:574] : INFO : Epoch 1857 | loss: 0.0310998 | val_loss: 0.0311559 | Time: 13779.2 ms [2022-04-07 13:21:58 main:574] : INFO : Epoch 1858 | loss: 0.0311005 | val_loss: 0.0311579 | Time: 13427.1 ms [2022-04-07 13:22:08 main:574] : INFO : Epoch 1859 | loss: 0.0311017 | val_loss: 0.0311507 | Time: 9615.11 ms [2022-04-07 13:22:22 main:574] : INFO : Epoch 1860 | loss: 0.0311044 | val_loss: 0.0311584 | Time: 13712.1 ms [2022-04-07 13:22:36 main:574] : INFO : Epoch 1861 | loss: 0.031103 | val_loss: 0.0311555 | Time: 14593.6 ms [2022-04-07 13:22:47 main:574] : INFO : Epoch 1862 | loss: 0.0311009 | val_loss: 0.0311574 | Time: 10276.5 ms [2022-04-07 13:23:01 main:574] : INFO : Epoch 1863 | loss: 0.0310992 | val_loss: 0.0311552 | Time: 13978.2 ms [2022-04-07 13:23:12 main:574] : INFO : Epoch 1864 | loss: 0.0310985 | val_loss: 0.031155 | Time: 10984.9 ms [2022-04-07 13:23:21 main:574] : INFO : Epoch 1865 | loss: 0.0310978 | val_loss: 0.0311556 | Time: 9536.85 ms [2022-04-07 13:23:29 main:574] : INFO : Epoch 1866 | loss: 0.031097 | val_loss: 0.0311559 | Time: 7590.05 ms [2022-04-07 13:23:36 main:574] : INFO : Epoch 1867 | loss: 0.0310962 | val_loss: 0.0311526 | Time: 6982.08 ms [2022-04-07 13:23:43 main:574] : INFO : Epoch 1868 | loss: 0.0310954 | val_loss: 0.0311586 | Time: 7238.15 ms [2022-04-07 13:23:56 main:574] : INFO : Epoch 1869 | loss: 0.0310983 | val_loss: 0.0311583 | Time: 12238.3 ms [2022-04-07 13:24:08 main:574] : INFO : Epoch 1870 | loss: 0.0310957 | val_loss: 0.0311562 | Time: 12091.1 ms [2022-04-07 13:24:18 main:574] : INFO : Epoch 1871 | loss: 0.0310964 | val_loss: 0.0311647 | Time: 9751.48 ms [2022-04-07 13:24:27 main:574] : INFO : Epoch 1872 | loss: 0.0310971 | val_loss: 0.031161 | Time: 9080.4 ms [2022-04-07 13:24:35 main:574] : INFO : Epoch 1873 | loss: 0.031095 | val_loss: 0.0311545 | Time: 7887.58 ms [2022-04-07 13:24:49 main:574] : INFO : Epoch 1874 | loss: 0.0310973 | val_loss: 0.0311595 | Time: 14409.7 ms [2022-04-07 13:25:04 main:574] : INFO : Epoch 1875 | loss: 0.0310966 | val_loss: 0.0311584 | Time: 14934.7 ms [2022-04-07 13:25:20 main:574] : INFO : Epoch 1876 | loss: 0.0311032 | val_loss: 0.0311588 | Time: 15668.9 ms [2022-04-07 13:25:33 main:574] : INFO : Epoch 1877 | loss: 0.0311108 | val_loss: 0.0311683 | Time: 13448.5 ms [2022-04-07 13:25:48 main:574] : INFO : Epoch 1878 | loss: 0.0311093 | val_loss: 0.0311625 | Time: 14721.2 ms [2022-04-07 13:26:03 main:574] : INFO : Epoch 1879 | loss: 0.0311074 | val_loss: 0.0311573 | Time: 14447.2 ms [2022-04-07 13:26:15 main:574] : INFO : Epoch 1880 | loss: 0.0311035 | val_loss: 0.0311523 | Time: 12019.5 ms [2022-04-07 13:26:25 main:574] : INFO : Epoch 1881 | loss: 0.0310998 | val_loss: 0.0311559 | Time: 10452.4 ms [2022-04-07 13:26:40 main:574] : INFO : Epoch 1882 | loss: 0.0310976 | val_loss: 0.0311572 | Time: 14385.6 ms [2022-04-07 13:26:55 main:574] : INFO : Epoch 1883 | loss: 0.031101 | val_loss: 0.0311627 | Time: 14882.7 ms [2022-04-07 13:27:06 main:574] : INFO : Epoch 1884 | loss: 0.0311008 | val_loss: 0.0311566 | Time: 11585.5 ms [2022-04-07 13:27:20 main:574] : INFO : Epoch 1885 | loss: 0.0311023 | val_loss: 0.031155 | Time: 13638 ms [2022-04-07 13:27:31 main:574] : INFO : Epoch 1886 | loss: 0.0311067 | val_loss: 0.0311612 | Time: 10826.5 ms [2022-04-07 13:27:46 main:574] : INFO : Epoch 1887 | loss: 0.0311045 | val_loss: 0.0311624 | Time: 14799.6 ms [2022-04-07 13:27:59 main:574] : INFO : Epoch 1888 | loss: 0.031102 | val_loss: 0.0311546 | Time: 12932.1 ms [2022-04-07 13:28:12 main:574] : INFO : Epoch 1889 | loss: 0.0310997 | val_loss: 0.0311575 | Time: 13081 ms [2022-04-07 13:28:26 main:574] : INFO : Epoch 1890 | loss: 0.0310986 | val_loss: 0.0311586 | Time: 13607 ms [2022-04-07 13:28:36 main:574] : INFO : Epoch 1891 | loss: 0.031098 | val_loss: 0.0311492 | Time: 9897.18 ms [2022-04-07 13:28:50 main:574] : INFO : Epoch 1892 | loss: 0.0310957 | val_loss: 0.0311537 | Time: 14465.8 ms [2022-04-07 13:29:04 main:574] : INFO : Epoch 1893 | loss: 0.0310936 | val_loss: 0.0311521 | Time: 14290 ms [2022-04-07 13:29:19 main:574] : INFO : Epoch 1894 | loss: 0.0310944 | val_loss: 0.0311561 | Time: 14687 ms [2022-04-07 13:29:30 main:574] : INFO : Epoch 1895 | loss: 0.0310949 | val_loss: 0.0311559 | Time: 11158.3 ms [2022-04-07 13:29:38 main:574] : INFO : Epoch 1896 | loss: 0.0310971 | val_loss: 0.0311667 | Time: 7511.88 ms [2022-04-07 13:29:45 main:574] : INFO : Epoch 1897 | loss: 0.0310966 | val_loss: 0.0311609 | Time: 7226.89 ms [2022-04-07 13:29:52 main:574] : INFO : Epoch 1898 | loss: 0.031096 | val_loss: 0.0311661 | Time: 6691.62 ms [2022-04-07 13:30:06 main:574] : INFO : Epoch 1899 | loss: 0.031098 | val_loss: 0.0311572 | Time: 13912.3 ms [2022-04-07 13:30:17 main:574] : INFO : Epoch 1900 | loss: 0.0311004 | val_loss: 0.0311608 | Time: 11369.8 ms [2022-04-07 13:30:27 main:574] : INFO : Epoch 1901 | loss: 0.0310963 | val_loss: 0.0311677 | Time: 9277.7 ms [2022-04-07 13:30:41 main:574] : INFO : Epoch 1902 | loss: 0.0310943 | val_loss: 0.0311552 | Time: 13780.6 ms [2022-04-07 13:30:50 main:574] : INFO : Epoch 1903 | loss: 0.0311036 | val_loss: 0.031159 | Time: 9791.52 ms [2022-04-07 13:30:59 main:574] : INFO : Epoch 1904 | loss: 0.0311039 | val_loss: 0.0311616 | Time: 8888.41 ms [2022-04-07 13:31:07 main:574] : INFO : Epoch 1905 | loss: 0.0311024 | val_loss: 0.0311617 | Time: 7836.23 ms [2022-04-07 13:31:21 main:574] : INFO : Epoch 1906 | loss: 0.0311006 | val_loss: 0.0311538 | Time: 13924.1 ms [2022-04-07 13:31:36 main:574] : INFO : Epoch 1907 | loss: 0.0311039 | val_loss: 0.0311684 | Time: 14221.2 ms [2022-04-07 13:31:51 main:574] : INFO : Epoch 1908 | loss: 0.0311032 | val_loss: 0.0311568 | Time: 15215.2 ms [2022-04-07 13:32:03 main:574] : INFO : Epoch 1909 | loss: 0.0310976 | val_loss: 0.0311605 | Time: 12071.5 ms [2022-04-07 13:32:14 main:574] : INFO : Epoch 1910 | loss: 0.0310971 | val_loss: 0.0311642 | Time: 10781 ms [2022-04-07 13:32:26 main:574] : INFO : Epoch 1911 | loss: 0.0310978 | val_loss: 0.0311564 | Time: 12490.6 ms [2022-04-07 13:32:41 main:574] : INFO : Epoch 1912 | loss: 0.031094 | val_loss: 0.0311595 | Time: 14494.9 ms [2022-04-07 13:32:56 main:574] : INFO : Epoch 1913 | loss: 0.0310959 | val_loss: 0.0311539 | Time: 14923.5 ms [2022-04-07 13:33:09 main:574] : INFO : Epoch 1914 | loss: 0.0310954 | val_loss: 0.0311596 | Time: 13001.9 ms [2022-04-07 13:33:22 main:574] : INFO : Epoch 1915 | loss: 0.0310956 | val_loss: 0.031162 | Time: 13175.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 13:35:49 main:435] : INFO : Set logging level to 1 [2022-04-07 13:35:49 main:441] : INFO : Running in BOINC Client mode [2022-04-07 13:35:49 main:444] : INFO : Resolving all filenames [2022-04-07 13:35:49 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 13:35:49 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 13:35:49 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 13:35:49 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 13:35:49 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 13:35:49 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 13:35:49 main:474] : INFO : Configuration: [2022-04-07 13:35:49 main:475] : INFO : Model type: GRU [2022-04-07 13:35:49 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 13:35:49 main:477] : INFO : Max Epochs: 2048 [2022-04-07 13:35:49 main:478] : INFO : Batch Size: 128 [2022-04-07 13:35:49 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 13:35:49 main:480] : INFO : Patience: 10 [2022-04-07 13:35:49 main:481] : INFO : Hidden Width: 12 [2022-04-07 13:35:49 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 13:35:49 main:483] : INFO : # Backend Layers: 4 [2022-04-07 13:35:50 main:484] : INFO : # Threads: 1 [2022-04-07 13:35:50 main:486] : INFO : Preparing Dataset [2022-04-07 13:35:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 13:35:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 13:35:58 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 13:35:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 13:35:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 13:35:59 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 13:35:59 main:494] : INFO : Creating Model [2022-04-07 13:35:59 main:507] : INFO : Preparing config file [2022-04-07 13:35:59 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 13:35:59 main:512] : INFO : Loading config [2022-04-07 13:35:59 main:514] : INFO : Loading state [2022-04-07 13:36:02 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 13:36:03 main:562] : INFO : Starting Training [2022-04-07 13:36:20 main:574] : INFO : Epoch 1914 | loss: 0.0311305 | val_loss: 0.0311632 | Time: 17125.5 ms [2022-04-07 13:36:34 main:574] : INFO : Epoch 1915 | loss: 0.0311023 | val_loss: 0.0311741 | Time: 14388.8 ms [2022-04-07 13:36:47 main:574] : INFO : Epoch 1916 | loss: 0.0310976 | val_loss: 0.0311585 | Time: 12374.1 ms [2022-04-07 13:37:01 main:574] : INFO : Epoch 1917 | loss: 0.0310965 | val_loss: 0.0311556 | Time: 13925.4 ms [2022-04-07 13:37:16 main:574] : INFO : Epoch 1918 | loss: 0.0310999 | val_loss: 0.0311613 | Time: 15015.7 ms [2022-04-07 13:37:29 main:574] : INFO : Epoch 1919 | loss: 0.0310961 | val_loss: 0.0311599 | Time: 13824.6 ms [2022-04-07 13:37:44 main:574] : INFO : Epoch 1920 | loss: 0.0310989 | val_loss: 0.0311587 | Time: 14001.4 ms [2022-04-07 13:37:56 main:574] : INFO : Epoch 1921 | loss: 0.0310997 | val_loss: 0.0311614 | Time: 12424.4 ms [2022-04-07 13:38:04 main:574] : INFO : Epoch 1922 | loss: 0.0310994 | val_loss: 0.0311532 | Time: 7763.14 ms [2022-04-07 13:38:19 main:574] : INFO : Epoch 1923 | loss: 0.0310966 | val_loss: 0.0311609 | Time: 12224.3 ms [2022-04-07 13:38:32 main:574] : INFO : Epoch 1924 | loss: 0.0310942 | val_loss: 0.0311535 | Time: 13582.2 ms [2022-04-07 13:38:45 main:574] : INFO : Epoch 1925 | loss: 0.0310947 | val_loss: 0.0311559 | Time: 12814.7 ms [2022-04-07 13:38:57 main:574] : INFO : Epoch 1926 | loss: 0.0310934 | val_loss: 0.0311613 | Time: 11697.2 ms [2022-04-07 13:39:11 main:574] : INFO : Epoch 1927 | loss: 0.031093 | val_loss: 0.0311576 | Time: 14245.2 ms [2022-04-07 13:39:25 main:574] : INFO : Epoch 1928 | loss: 0.0310918 | val_loss: 0.0311639 | Time: 14040.9 ms [2022-04-07 13:39:36 main:574] : INFO : Epoch 1929 | loss: 0.0310932 | val_loss: 0.0311567 | Time: 10248.7 ms [2022-04-07 13:39:50 main:574] : INFO : Epoch 1930 | loss: 0.0310919 | val_loss: 0.0311605 | Time: 14534.4 ms [2022-04-07 13:40:04 main:574] : INFO : Epoch 1931 | loss: 0.031091 | val_loss: 0.0311635 | Time: 13616.6 ms [2022-04-07 13:40:18 main:574] : INFO : Epoch 1932 | loss: 0.0310918 | val_loss: 0.0311587 | Time: 14148.4 ms [2022-04-07 13:40:31 main:574] : INFO : Epoch 1933 | loss: 0.0310919 | val_loss: 0.0311595 | Time: 13325 ms [2022-04-07 13:40:44 main:574] : INFO : Epoch 1934 | loss: 0.0310892 | val_loss: 0.0311593 | Time: 12461.4 ms [2022-04-07 13:40:54 main:574] : INFO : Epoch 1935 | loss: 0.0310914 | val_loss: 0.0311723 | Time: 9841.21 ms [2022-04-07 13:41:07 main:574] : INFO : Epoch 1936 | loss: 0.0310914 | val_loss: 0.031158 | Time: 12663.6 ms [2022-04-07 13:41:19 main:574] : INFO : Epoch 1937 | loss: 0.031091 | val_loss: 0.0311678 | Time: 12688.1 ms [2022-04-07 13:41:29 main:574] : INFO : Epoch 1938 | loss: 0.0310893 | val_loss: 0.0311669 | Time: 9524.86 ms [2022-04-07 13:41:43 main:574] : INFO : Epoch 1939 | loss: 0.0310937 | val_loss: 0.0311643 | Time: 14074.5 ms [2022-04-07 13:41:57 main:574] : INFO : Epoch 1940 | loss: 0.0310932 | val_loss: 0.0311623 | Time: 13411 ms [2022-04-07 13:42:11 main:574] : INFO : Epoch 1941 | loss: 0.0310951 | val_loss: 0.0311529 | Time: 14009.9 ms [2022-04-07 13:42:21 main:574] : INFO : Epoch 1942 | loss: 0.0310973 | val_loss: 0.0311598 | Time: 9945.02 ms [2022-04-07 13:42:30 main:574] : INFO : Epoch 1943 | loss: 0.0311009 | val_loss: 0.0311595 | Time: 8889.1 ms [2022-04-07 13:42:38 main:574] : INFO : Epoch 1944 | loss: 0.0311002 | val_loss: 0.0311574 | Time: 8417.38 ms [2022-04-07 13:42:52 main:574] : INFO : Epoch 1945 | loss: 0.0311045 | val_loss: 0.0311599 | Time: 13422.6 ms [2022-04-07 13:43:01 main:574] : INFO : Epoch 1946 | loss: 0.0311041 | val_loss: 0.0311594 | Time: 9835.25 ms [2022-04-07 13:43:15 main:574] : INFO : Epoch 1947 | loss: 0.0310986 | val_loss: 0.0311519 | Time: 13631.5 ms [2022-04-07 13:43:29 main:574] : INFO : Epoch 1948 | loss: 0.0310963 | val_loss: 0.0311608 | Time: 13615.6 ms [2022-04-07 13:43:40 main:574] : INFO : Epoch 1949 | loss: 0.0310998 | val_loss: 0.0311527 | Time: 11586.6 ms [2022-04-07 13:43:55 main:574] : INFO : Epoch 1950 | loss: 0.0311012 | val_loss: 0.0311577 | Time: 14622.6 ms [2022-04-07 13:44:07 main:574] : INFO : Epoch 1951 | loss: 0.0311007 | val_loss: 0.0311564 | Time: 11853.4 ms [2022-04-07 13:44:22 main:574] : INFO : Epoch 1952 | loss: 0.0310958 | val_loss: 0.031154 | Time: 14446.3 ms [2022-04-07 13:44:35 main:574] : INFO : Epoch 1953 | loss: 0.0310974 | val_loss: 0.0311547 | Time: 13657 ms [2022-04-07 13:44:50 main:574] : INFO : Epoch 1954 | loss: 0.0310991 | val_loss: 0.031157 | Time: 14270.7 ms [2022-04-07 13:44:59 main:574] : INFO : Epoch 1955 | loss: 0.0310968 | val_loss: 0.0311498 | Time: 9748.28 ms [2022-04-07 13:45:09 main:574] : INFO : Epoch 1956 | loss: 0.0310972 | val_loss: 0.0311644 | Time: 9803.24 ms [2022-04-07 13:45:22 main:574] : INFO : Epoch 1957 | loss: 0.0310957 | val_loss: 0.0311657 | Time: 13205.7 ms [2022-04-07 13:45:36 main:574] : INFO : Epoch 1958 | loss: 0.0310954 | val_loss: 0.0311565 | Time: 13363 ms [2022-04-07 13:45:49 main:574] : INFO : Epoch 1959 | loss: 0.031097 | val_loss: 0.0311606 | Time: 12575.3 ms [2022-04-07 13:46:02 main:574] : INFO : Epoch 1960 | loss: 0.0310968 | val_loss: 0.0311584 | Time: 13559.3 ms [2022-04-07 13:46:16 main:574] : INFO : Epoch 1961 | loss: 0.0310998 | val_loss: 0.0311673 | Time: 14127.7 ms [2022-04-07 13:46:31 main:574] : INFO : Epoch 1962 | loss: 0.0311029 | val_loss: 0.0311639 | Time: 14673.1 ms [2022-04-07 13:46:47 main:574] : INFO : Epoch 1963 | loss: 0.0310998 | val_loss: 0.0311608 | Time: 15695.9 ms [2022-04-07 13:46:59 main:574] : INFO : Epoch 1964 | loss: 0.0310974 | val_loss: 0.0311646 | Time: 11608.2 ms [2022-04-07 13:47:13 main:574] : INFO : Epoch 1965 | loss: 0.0310989 | val_loss: 0.0311624 | Time: 14248.7 ms [2022-04-07 13:47:28 main:574] : INFO : Epoch 1966 | loss: 0.031097 | val_loss: 0.031153 | Time: 14888.3 ms [2022-04-07 13:47:40 main:574] : INFO : Epoch 1967 | loss: 0.0310966 | val_loss: 0.0311539 | Time: 12151.7 ms [2022-04-07 13:47:52 main:574] : INFO : Epoch 1968 | loss: 0.0310981 | val_loss: 0.0311584 | Time: 11873 ms [2022-04-07 13:48:07 main:574] : INFO : Epoch 1969 | loss: 0.0311 | val_loss: 0.0311576 | Time: 14834.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 13:53:03 main:435] : INFO : Set logging level to 1 [2022-04-07 13:53:03 main:441] : INFO : Running in BOINC Client mode [2022-04-07 13:53:03 main:444] : INFO : Resolving all filenames [2022-04-07 13:53:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 13:53:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 13:53:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 13:53:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 13:53:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 13:53:03 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 13:53:03 main:474] : INFO : Configuration: [2022-04-07 13:53:03 main:475] : INFO : Model type: GRU [2022-04-07 13:53:03 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 13:53:03 main:477] : INFO : Max Epochs: 2048 [2022-04-07 13:53:03 main:478] : INFO : Batch Size: 128 [2022-04-07 13:53:03 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 13:53:03 main:480] : INFO : Patience: 10 [2022-04-07 13:53:03 main:481] : INFO : Hidden Width: 12 [2022-04-07 13:53:03 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 13:53:03 main:483] : INFO : # Backend Layers: 4 [2022-04-07 13:53:03 main:484] : INFO : # Threads: 1 [2022-04-07 13:53:04 main:486] : INFO : Preparing Dataset [2022-04-07 13:53:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 13:53:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 13:53:11 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 13:53:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 13:53:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 13:53:12 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 13:53:12 main:494] : INFO : Creating Model [2022-04-07 13:53:12 main:507] : INFO : Preparing config file [2022-04-07 13:53:12 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 13:53:12 main:512] : INFO : Loading config [2022-04-07 13:53:12 main:514] : INFO : Loading state [2022-04-07 13:53:15 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 13:53:15 main:562] : INFO : Starting Training [2022-04-07 13:53:30 main:574] : INFO : Epoch 1969 | loss: 0.0311235 | val_loss: 0.0311598 | Time: 14800.4 ms [2022-04-07 13:53:44 main:574] : INFO : Epoch 1970 | loss: 0.0310999 | val_loss: 0.0311646 | Time: 13356.9 ms [2022-04-07 13:53:54 main:574] : INFO : Epoch 1971 | loss: 0.0310946 | val_loss: 0.0311616 | Time: 9754.16 ms [2022-04-07 13:54:07 main:574] : INFO : Epoch 1972 | loss: 0.0310946 | val_loss: 0.0311699 | Time: 13379.8 ms [2022-04-07 13:54:18 main:574] : INFO : Epoch 1973 | loss: 0.0311009 | val_loss: 0.0311758 | Time: 10591.1 ms [2022-04-07 13:54:32 main:574] : INFO : Epoch 1974 | loss: 0.0310998 | val_loss: 0.0311636 | Time: 14633.7 ms [2022-04-07 13:54:43 main:574] : INFO : Epoch 1975 | loss: 0.0310951 | val_loss: 0.0311663 | Time: 10523.8 ms [2022-04-07 13:54:56 main:574] : INFO : Epoch 1976 | loss: 0.0310936 | val_loss: 0.0311628 | Time: 12552.1 ms [2022-04-07 13:55:09 main:574] : INFO : Epoch 1977 | loss: 0.0310959 | val_loss: 0.0311565 | Time: 13875.5 ms [2022-04-07 13:55:23 main:574] : INFO : Epoch 1978 | loss: 0.0311015 | val_loss: 0.0311561 | Time: 13753.2 ms [2022-04-07 13:55:37 main:574] : INFO : Epoch 1979 | loss: 0.0311 | val_loss: 0.0311593 | Time: 13628.9 ms [2022-04-07 13:55:52 main:574] : INFO : Epoch 1980 | loss: 0.0310962 | val_loss: 0.0311598 | Time: 14565.9 ms [2022-04-07 13:56:04 main:574] : INFO : Epoch 1981 | loss: 0.0310973 | val_loss: 0.0311576 | Time: 12207.9 ms [2022-04-07 13:56:19 main:574] : INFO : Epoch 1982 | loss: 0.0310958 | val_loss: 0.031156 | Time: 14665.9 ms [2022-04-07 13:56:34 main:574] : INFO : Epoch 1983 | loss: 0.0310947 | val_loss: 0.0311634 | Time: 14903.2 ms [2022-04-07 13:56:47 main:574] : INFO : Epoch 1984 | loss: 0.0310948 | val_loss: 0.031157 | Time: 12978.4 ms [2022-04-07 13:56:58 main:574] : INFO : Epoch 1985 | loss: 0.031098 | val_loss: 0.0311595 | Time: 11177 ms [2022-04-07 13:57:12 main:574] : INFO : Epoch 1986 | loss: 0.0310927 | val_loss: 0.0311581 | Time: 13838.6 ms [2022-04-07 13:57:23 main:574] : INFO : Epoch 1987 | loss: 0.0310934 | val_loss: 0.0311603 | Time: 10775.1 ms [2022-04-07 13:57:32 main:574] : INFO : Epoch 1988 | loss: 0.0310987 | val_loss: 0.0311657 | Time: 9764.49 ms [2022-04-07 13:57:46 main:574] : INFO : Epoch 1989 | loss: 0.0310982 | val_loss: 0.0311603 | Time: 12988.6 ms [2022-04-07 13:57:58 main:574] : INFO : Epoch 1990 | loss: 0.0310972 | val_loss: 0.0311574 | Time: 12108.9 ms [2022-04-07 13:58:08 main:574] : INFO : Epoch 1991 | loss: 0.031098 | val_loss: 0.0311587 | Time: 10102.6 ms [2022-04-07 13:58:21 main:574] : INFO : Epoch 1992 | loss: 0.0310965 | val_loss: 0.0311646 | Time: 12571.9 ms [2022-04-07 13:58:33 main:574] : INFO : Epoch 1993 | loss: 0.0310986 | val_loss: 0.0311606 | Time: 11900.7 ms [2022-04-07 13:58:46 main:574] : INFO : Epoch 1994 | loss: 0.0310966 | val_loss: 0.0311526 | Time: 13111 ms [2022-04-07 13:59:00 main:574] : INFO : Epoch 1995 | loss: 0.0310939 | val_loss: 0.0311565 | Time: 13775.8 ms [2022-04-07 13:59:13 main:574] : INFO : Epoch 1996 | loss: 0.031093 | val_loss: 0.0311609 | Time: 13373.6 ms [2022-04-07 13:59:25 main:574] : INFO : Epoch 1997 | loss: 0.0310946 | val_loss: 0.0311566 | Time: 11518.2 ms [2022-04-07 13:59:33 main:574] : INFO : Epoch 1998 | loss: 0.031093 | val_loss: 0.0311592 | Time: 8017.34 ms [2022-04-07 13:59:47 main:574] : INFO : Epoch 1999 | loss: 0.0310919 | val_loss: 0.0311581 | Time: 14629.7 ms [2022-04-07 14:00:02 main:574] : INFO : Epoch 2000 | loss: 0.031091 | val_loss: 0.0311603 | Time: 14407.9 ms [2022-04-07 14:00:13 main:574] : INFO : Epoch 2001 | loss: 0.0310914 | val_loss: 0.0311633 | Time: 11486.5 ms [2022-04-07 14:00:28 main:574] : INFO : Epoch 2002 | loss: 0.0310945 | val_loss: 0.0311607 | Time: 14977.1 ms [2022-04-07 14:00:42 main:574] : INFO : Epoch 2003 | loss: 0.031095 | val_loss: 0.0311617 | Time: 14043.3 ms [2022-04-07 14:00:56 main:574] : INFO : Epoch 2004 | loss: 0.0310939 | val_loss: 0.0311572 | Time: 13884.3 ms [2022-04-07 14:01:11 main:574] : INFO : Epoch 2005 | loss: 0.0310945 | val_loss: 0.031161 | Time: 14141.3 ms [2022-04-07 14:01:24 main:574] : INFO : Epoch 2006 | loss: 0.0310911 | val_loss: 0.0311729 | Time: 13349.4 ms [2022-04-07 14:01:38 main:574] : INFO : Epoch 2007 | loss: 0.0310926 | val_loss: 0.0311611 | Time: 13476.2 ms [2022-04-07 14:01:50 main:574] : INFO : Epoch 2008 | loss: 0.0310911 | val_loss: 0.0311608 | Time: 12698.2 ms [2022-04-07 14:02:04 main:574] : INFO : Epoch 2009 | loss: 0.0310899 | val_loss: 0.0311628 | Time: 14080.9 ms [2022-04-07 14:02:15 main:574] : INFO : Epoch 2010 | loss: 0.0310896 | val_loss: 0.0311618 | Time: 10342.6 ms [2022-04-07 14:02:25 main:574] : INFO : Epoch 2011 | loss: 0.0310911 | val_loss: 0.0311786 | Time: 9861.91 ms [2022-04-07 14:02:39 main:574] : INFO : Epoch 2012 | loss: 0.0310937 | val_loss: 0.0311634 | Time: 14504.3 ms [2022-04-07 14:02:53 main:574] : INFO : Epoch 2013 | loss: 0.0310908 | val_loss: 0.0311598 | Time: 13183 ms [2022-04-07 14:03:07 main:574] : INFO : Epoch 2014 | loss: 0.0310936 | val_loss: 0.0311569 | Time: 14809.8 ms [2022-04-07 14:03:19 main:574] : INFO : Epoch 2015 | loss: 0.0311007 | val_loss: 0.0311647 | Time: 11731.5 ms [2022-04-07 14:03:34 main:574] : INFO : Epoch 2016 | loss: 0.0310993 | val_loss: 0.0311653 | Time: 14405.4 ms [2022-04-07 14:03:48 main:574] : INFO : Epoch 2017 | loss: 0.0310965 | val_loss: 0.0311699 | Time: 14333.9 ms [2022-04-07 14:04:02 main:574] : INFO : Epoch 2018 | loss: 0.0310991 | val_loss: 0.031167 | Time: 14067.2 ms [2022-04-07 14:04:16 main:574] : INFO : Epoch 2019 | loss: 0.0310986 | val_loss: 0.031164 | Time: 13607.6 ms [2022-04-07 14:04:30 main:574] : INFO : Epoch 2020 | loss: 0.0310984 | val_loss: 0.0311664 | Time: 13461.8 ms [2022-04-07 14:04:43 main:574] : INFO : Epoch 2021 | loss: 0.0310945 | val_loss: 0.0311707 | Time: 13507.1 ms [2022-04-07 14:04:55 main:574] : INFO : Epoch 2022 | loss: 0.0310977 | val_loss: 0.0311621 | Time: 11478 ms [2022-04-07 14:05:09 main:574] : INFO : Epoch 2023 | loss: 0.0311008 | val_loss: 0.031163 | Time: 14157.1 ms [2022-04-07 14:05:24 main:574] : INFO : Epoch 2024 | loss: 0.0311173 | val_loss: 0.0311624 | Time: 14850.9 ms [2022-04-07 14:05:37 main:574] : INFO : Epoch 2025 | loss: 0.0311167 | val_loss: 0.0311541 | Time: 13252.8 ms [2022-04-07 14:05:51 main:574] : INFO : Epoch 2026 | loss: 0.0311161 | val_loss: 0.0311594 | Time: 14240.5 ms [2022-04-07 14:06:03 main:574] : INFO : Epoch 2027 | loss: 0.0311136 | val_loss: 0.0311576 | Time: 11702.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-04-07 14:12:12 main:435] : INFO : Set logging level to 1 [2022-04-07 14:12:12 main:441] : INFO : Running in BOINC Client mode [2022-04-07 14:12:12 main:444] : INFO : Resolving all filenames [2022-04-07 14:12:12 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-07 14:12:12 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-07 14:12:12 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-07 14:12:12 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-07 14:12:12 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-07 14:12:13 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-07 14:12:13 main:474] : INFO : Configuration: [2022-04-07 14:12:13 main:475] : INFO : Model type: GRU [2022-04-07 14:12:13 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-07 14:12:13 main:477] : INFO : Max Epochs: 2048 [2022-04-07 14:12:13 main:478] : INFO : Batch Size: 128 [2022-04-07 14:12:13 main:479] : INFO : Learning Rate: 0.01 [2022-04-07 14:12:13 main:480] : INFO : Patience: 10 [2022-04-07 14:12:13 main:481] : INFO : Hidden Width: 12 [2022-04-07 14:12:13 main:482] : INFO : # Recurrent Layers: 4 [2022-04-07 14:12:13 main:483] : INFO : # Backend Layers: 4 [2022-04-07 14:12:13 main:484] : INFO : # Threads: 1 [2022-04-07 14:12:13 main:486] : INFO : Preparing Dataset [2022-04-07 14:12:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-07 14:12:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-07 14:12:21 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-07 14:12:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-07 14:12:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-07 14:12:22 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-07 14:12:22 main:494] : INFO : Creating Model [2022-04-07 14:12:22 main:507] : INFO : Preparing config file [2022-04-07 14:12:22 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-07 14:12:22 main:512] : INFO : Loading config [2022-04-07 14:12:22 main:514] : INFO : Loading state [2022-04-07 14:12:25 main:559] : INFO : Loading DataLoader into Memory [2022-04-07 14:12:25 main:562] : INFO : Starting Training [2022-04-07 14:12:39 main:574] : INFO : Epoch 2024 | loss: 0.0311244 | val_loss: 0.0311721 | Time: 14252.1 ms [2022-04-07 14:12:51 main:574] : INFO : Epoch 2025 | loss: 0.031103 | val_loss: 0.0311647 | Time: 11437.8 ms [2022-04-07 14:13:03 main:574] : INFO : Epoch 2026 | loss: 0.0310965 | val_loss: 0.0311579 | Time: 12305.9 ms [2022-04-07 14:13:16 main:574] : INFO : Epoch 2027 | loss: 0.0310984 | val_loss: 0.031164 | Time: 13129.5 ms [2022-04-07 14:13:30 main:574] : INFO : Epoch 2028 | loss: 0.0310966 | val_loss: 0.031166 | Time: 13958 ms [2022-04-07 14:13:44 main:574] : INFO : Epoch 2029 | loss: 0.0310972 | val_loss: 0.0311687 | Time: 14030.8 ms [2022-04-07 14:13:56 main:574] : INFO : Epoch 2030 | loss: 0.0310976 | val_loss: 0.0311683 | Time: 11116.1 ms [2022-04-07 14:14:08 main:574] : INFO : Epoch 2031 | loss: 0.0311013 | val_loss: 0.0311612 | Time: 12837.9 ms [2022-04-07 14:14:21 main:574] : INFO : Epoch 2032 | loss: 0.0311019 | val_loss: 0.0311636 | Time: 12344.3 ms [2022-04-07 14:14:34 main:574] : INFO : Epoch 2033 | loss: 0.0311021 | val_loss: 0.0311615 | Time: 12797.9 ms [2022-04-07 14:14:45 main:574] : INFO : Epoch 2034 | loss: 0.031101 | val_loss: 0.0311655 | Time: 11386.5 ms [2022-04-07 14:14:58 main:574] : INFO : Epoch 2035 | loss: 0.0310983 | val_loss: 0.0311651 | Time: 13260.8 ms [2022-04-07 14:15:08 main:574] : INFO : Epoch 2036 | loss: 0.0310962 | val_loss: 0.0311738 | Time: 9611.19 ms [2022-04-07 14:15:21 main:574] : INFO : Epoch 2037 | loss: 0.0310966 | val_loss: 0.031171 | Time: 12505.2 ms [2022-04-07 14:15:34 main:574] : INFO : Epoch 2038 | loss: 0.0310997 | val_loss: 0.0311756 | Time: 12754.2 ms [2022-04-07 14:15:45 main:574] : INFO : Epoch 2039 | loss: 0.0311011 | val_loss: 0.0311676 | Time: 10943.1 ms [2022-04-07 14:15:58 main:574] : INFO : Epoch 2040 | loss: 0.031106 | val_loss: 0.0311717 | Time: 13721 ms [2022-04-07 14:16:10 main:574] : INFO : Epoch 2041 | loss: 0.0311051 | val_loss: 0.0311637 | Time: 11560.2 ms [2022-04-07 14:16:22 main:574] : INFO : Epoch 2042 | loss: 0.0311017 | val_loss: 0.0311659 | Time: 12406.8 ms [2022-04-07 14:16:32 main:574] : INFO : Epoch 2043 | loss: 0.0310994 | val_loss: 0.0311682 | Time: 9563.25 ms [2022-04-07 14:16:44 main:574] : INFO : Epoch 2044 | loss: 0.0310993 | val_loss: 0.0311744 | Time: 11662.5 ms [2022-04-07 14:16:56 main:574] : INFO : Epoch 2045 | loss: 0.0310995 | val_loss: 0.0311728 | Time: 12148.8 ms [2022-04-07 14:17:10 main:574] : INFO : Epoch 2046 | loss: 0.0310966 | val_loss: 0.0311697 | Time: 13838.4 ms [2022-04-07 14:17:23 main:574] : INFO : Epoch 2047 | loss: 0.0310945 | val_loss: 0.0311715 | Time: 13139.5 ms [2022-04-07 14:17:35 main:574] : INFO : Epoch 2048 | loss: 0.0310936 | val_loss: 0.0311683 | Time: 12224.4 ms [2022-04-07 14:17:35 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311683 [2022-04-07 14:17:35 main:603] : INFO : Saving end state to config to file [2022-04-07 14:17:36 main:608] : INFO : Success, exiting.. 14:17:36 (11492): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)