Task 10836673
| Name | ParityModified-1639959623-28884-2-0_0 |
| Workunit | 8145259 |
| Created | 17 Jan 2022, 3:17:09 UTC |
| Sent | 4 Feb 2022, 10:29:27 UTC |
| Report deadline | 12 Feb 2022, 10:29:27 UTC |
| Received | 6 Sep 2022, 0:10:03 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -1073741523 (0xC000012D) Unknown error code |
| Computer ID | 6741 |
| Run time | 4 hours 48 min 29 sec |
| CPU time | 3 hours 13 min 23 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 786.79 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.54 GB |
| Peak swap size | 3.44 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.20.2</core_client_version> <![CDATA[ <message> (unknown error) - exit code 3221225773 (0xc000012d)</message> <stderr_txt> 8151.1 ms [2022-02-05 22:19:54 main:574] : INFO : Epoch 1249 | loss: 0.0311519 | val_loss: 0.0311687 | Time: 39817.8 ms [2022-02-05 22:20:34 main:574] : INFO : Epoch 1250 | loss: 0.0311544 | val_loss: 0.0311735 | Time: 40242.6 ms [2022-02-05 22:21:15 main:574] : INFO : Epoch 1251 | loss: 0.0311527 | val_loss: 0.0311669 | Time: 41346.3 ms [2022-02-05 22:21:54 main:574] : INFO : Epoch 1252 | loss: 0.031153 | val_loss: 0.0311734 | Time: 38455.4 ms [2022-02-05 22:22:34 main:574] : INFO : Epoch 1253 | loss: 0.0311561 | val_loss: 0.031173 | Time: 39559.3 ms [2022-02-05 22:23:14 main:574] : INFO : Epoch 1254 | loss: 0.031157 | val_loss: 0.0311696 | Time: 39872 ms [2022-02-05 22:24:04 main:574] : INFO : Epoch 1255 | loss: 0.031158 | val_loss: 0.03117 | Time: 50198.7 ms [2022-02-05 22:24:43 main:574] : INFO : Epoch 1256 | loss: 0.0311559 | val_loss: 0.0311706 | Time: 39035.4 ms [2022-02-05 22:25:24 main:574] : INFO : Epoch 1257 | loss: 0.0311584 | val_loss: 0.0311742 | Time: 40687.3 ms [2022-02-05 22:25:52 main:574] : INFO : Epoch 1258 | loss: 0.0311625 | val_loss: 0.0311717 | Time: 27822.9 ms [2022-02-05 22:26:31 main:574] : INFO : Epoch 1259 | loss: 0.0311615 | val_loss: 0.0311717 | Time: 38558.7 ms [2022-02-05 22:27:11 main:574] : INFO : Epoch 1260 | loss: 0.0311607 | val_loss: 0.0311707 | Time: 39791.3 ms [2022-02-05 22:27:50 main:574] : INFO : Epoch 1261 | loss: 0.0311573 | val_loss: 0.0311703 | Time: 38981.9 ms [2022-02-05 22:28:29 main:574] : INFO : Epoch 1262 | loss: 0.0311586 | val_loss: 0.0311685 | Time: 38121.2 ms [2022-02-05 22:29:19 main:574] : INFO : Epoch 1263 | loss: 0.0311644 | val_loss: 0.0311815 | Time: 50141.7 ms [2022-02-05 22:29:59 main:574] : INFO : Epoch 1264 | loss: 0.0311773 | val_loss: 0.0311795 | Time: 39650 ms [2022-02-05 22:31:21 main:574] : INFO : Epoch 1265 | loss: 0.0311725 | val_loss: 0.0311739 | Time: 81804.6 ms [2022-02-05 22:32:00 main:574] : INFO : Epoch 1266 | loss: 0.0311682 | val_loss: 0.0311684 | Time: 39263.1 ms [2022-02-05 22:32:39 main:574] : INFO : Epoch 1267 | loss: 0.0311606 | val_loss: 0.0311687 | Time: 38462.3 ms [2022-02-05 22:33:17 main:574] : INFO : Epoch 1268 | loss: 0.031159 | val_loss: 0.0311679 | Time: 38147 ms [2022-02-05 22:33:57 main:574] : INFO : Epoch 1269 | loss: 0.031157 | val_loss: 0.0311752 | Time: 40329.5 ms [2022-02-05 22:34:47 main:574] : INFO : Epoch 1270 | loss: 0.0311581 | val_loss: 0.0311711 | Time: 49129.8 ms [2022-02-05 22:35:26 main:574] : INFO : Epoch 1271 | loss: 0.0311597 | val_loss: 0.031169 | Time: 39792.5 ms [2022-02-05 22:36:08 main:574] : INFO : Epoch 1272 | loss: 0.0311577 | val_loss: 0.0311678 | Time: 41099 ms [2022-02-05 22:36:46 main:574] : INFO : Epoch 1273 | loss: 0.031159 | val_loss: 0.0311697 | Time: 37889.8 ms [2022-02-05 22:37:26 main:574] : INFO : Epoch 1274 | loss: 0.0311602 | val_loss: 0.031176 | Time: 39953.2 ms [2022-02-05 22:37:54 main:574] : INFO : Epoch 1275 | loss: 0.0311587 | val_loss: 0.0311683 | Time: 27875.5 ms [2022-02-05 22:38:33 main:574] : INFO : Epoch 1276 | loss: 0.0311583 | val_loss: 0.0311729 | Time: 38135.2 ms [2022-02-05 22:40:47 main:574] : INFO : Epoch 1277 | loss: 0.0311615 | val_loss: 0.0311657 | Time: 134201 ms [2022-02-05 22:41:35 main:574] : INFO : Epoch 1278 | loss: 0.0311641 | val_loss: 0.0311681 | Time: 48164.6 ms [2022-02-05 22:42:32 main:574] : INFO : Epoch 1279 | loss: 0.0311623 | val_loss: 0.0311685 | Time: 56397.8 ms [2022-02-05 22:43:12 main:574] : INFO : Epoch 1280 | loss: 0.0311599 | val_loss: 0.031166 | Time: 40077.6 ms [2022-02-05 22:43:51 main:574] : INFO : Epoch 1281 | loss: 0.03116 | val_loss: 0.0311678 | Time: 38801.9 ms [2022-02-05 22:44:49 main:574] : INFO : Epoch 1282 | loss: 0.0311582 | val_loss: 0.0311769 | Time: 58561.7 ms [2022-02-05 22:45:29 main:574] : INFO : Epoch 1283 | loss: 0.0311605 | val_loss: 0.0311727 | Time: 39406.7 ms [2022-02-05 22:46:08 main:574] : INFO : Epoch 1284 | loss: 0.0311671 | val_loss: 0.0311711 | Time: 39040.7 ms [2022-02-05 22:46:48 main:574] : INFO : Epoch 1285 | loss: 0.0311662 | val_loss: 0.0311695 | Time: 39329.3 ms [2022-02-05 22:47:16 main:574] : INFO : Epoch 1286 | loss: 0.0311665 | val_loss: 0.0311681 | Time: 28398.2 ms [2022-02-05 22:47:56 main:574] : INFO : Epoch 1287 | loss: 0.0311651 | val_loss: 0.0311693 | Time: 39425.1 ms [2022-02-05 22:48:32 main:574] : INFO : Epoch 1288 | loss: 0.0311618 | val_loss: 0.0311688 | Time: 36210.5 ms [2022-02-05 22:49:12 main:574] : INFO : Epoch 1289 | loss: 0.0311603 | val_loss: 0.0311685 | Time: 39777.2 ms [2022-02-05 22:50:04 main:574] : INFO : Epoch 1290 | loss: 0.0311597 | val_loss: 0.0311732 | Time: 51792.1 ms [2022-02-05 22:50:55 main:574] : INFO : Epoch 1291 | loss: 0.0311683 | val_loss: 0.0311761 | Time: 50833 ms [2022-02-05 22:51:34 main:574] : INFO : Epoch 1292 | loss: 0.0311749 | val_loss: 0.0311726 | Time: 38962.1 ms [2022-02-05 22:52:13 main:574] : INFO : Epoch 1293 | loss: 0.0311706 | val_loss: 0.0311719 | Time: 38744.8 ms [2022-02-05 22:52:53 main:574] : INFO : Epoch 1294 | loss: 0.0311703 | val_loss: 0.0311681 | Time: 39523.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 08:27:59 main:435] : INFO : Set logging level to 1 [2022-02-06 08:27:59 main:441] : INFO : Running in BOINC Client mode [2022-02-06 08:27:59 main:444] : INFO : Resolving all filenames [2022-02-06 08:27:59 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 08:27:59 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 08:27:59 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 08:27:59 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 08:27:59 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 08:27:59 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 08:27:59 main:474] : INFO : Configuration: [2022-02-06 08:27:59 main:475] : INFO : Model type: GRU [2022-02-06 08:27:59 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 08:27:59 main:477] : INFO : Max Epochs: 2048 [2022-02-06 08:27:59 main:478] : INFO : Batch Size: 128 [2022-02-06 08:27:59 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 08:27:59 main:480] : INFO : Patience: 10 [2022-02-06 08:27:59 main:481] : INFO : Hidden Width: 12 [2022-02-06 08:27:59 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 08:27:59 main:483] : INFO : # Backend Layers: 4 [2022-02-06 08:27:59 main:484] : INFO : # Threads: 1 [2022-02-06 08:27:59 main:486] : INFO : Preparing Dataset [2022-02-06 08:27:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 08:28:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 08:28:04 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 08:28:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 08:28:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 08:28:04 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 08:28:04 main:494] : INFO : Creating Model [2022-02-06 08:28:04 main:507] : INFO : Preparing config file [2022-02-06 08:28:04 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 08:28:04 main:512] : INFO : Loading config [2022-02-06 08:28:04 main:514] : INFO : Loading state [2022-02-06 08:28:06 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 08:28:06 main:562] : INFO : Starting Training [2022-02-06 08:28:13 main:574] : INFO : Epoch 1294 | loss: 0.0312903 | val_loss: 0.0312003 | Time: 7152.34 ms [2022-02-06 08:28:20 main:574] : INFO : Epoch 1295 | loss: 0.0311862 | val_loss: 0.0311881 | Time: 6731.4 ms [2022-02-06 08:28:26 main:574] : INFO : Epoch 1296 | loss: 0.0311714 | val_loss: 0.0311738 | Time: 6799.07 ms [2022-02-06 08:28:33 main:574] : INFO : Epoch 1297 | loss: 0.0311657 | val_loss: 0.0311749 | Time: 6846.48 ms [2022-02-06 08:28:40 main:574] : INFO : Epoch 1298 | loss: 0.0311654 | val_loss: 0.0311704 | Time: 6840.87 ms [2022-02-06 08:28:47 main:574] : INFO : Epoch 1299 | loss: 0.0311641 | val_loss: 0.031171 | Time: 6864.88 ms [2022-02-06 08:28:54 main:574] : INFO : Epoch 1300 | loss: 0.0311626 | val_loss: 0.0311731 | Time: 6833.97 ms [2022-02-06 08:29:04 main:574] : INFO : Epoch 1301 | loss: 0.0311644 | val_loss: 0.0311691 | Time: 9689.72 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 08:33:58 main:435] : INFO : Set logging level to 1 [2022-02-06 08:33:58 main:441] : INFO : Running in BOINC Client mode [2022-02-06 08:33:58 main:444] : INFO : Resolving all filenames [2022-02-06 08:33:58 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 08:33:58 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 08:33:58 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 08:33:58 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 08:33:58 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 08:33:58 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 08:33:58 main:474] : INFO : Configuration: [2022-02-06 08:33:58 main:475] : INFO : Model type: GRU [2022-02-06 08:33:58 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 08:33:58 main:477] : INFO : Max Epochs: 2048 [2022-02-06 08:33:58 main:478] : INFO : Batch Size: 128 [2022-02-06 08:33:58 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 08:33:58 main:480] : INFO : Patience: 10 [2022-02-06 08:33:58 main:481] : INFO : Hidden Width: 12 [2022-02-06 08:33:58 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 08:33:58 main:483] : INFO : # Backend Layers: 4 [2022-02-06 08:33:58 main:484] : INFO : # Threads: 1 [2022-02-06 08:33:58 main:486] : INFO : Preparing Dataset [2022-02-06 08:33:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 08:33:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 08:34:03 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 08:34:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 08:34:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 08:34:04 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 08:34:04 main:494] : INFO : Creating Model [2022-02-06 08:34:04 main:507] : INFO : Preparing config file [2022-02-06 08:34:04 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 08:34:04 main:512] : INFO : Loading config [2022-02-06 08:34:04 main:514] : INFO : Loading state [2022-02-06 08:34:06 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 08:34:06 main:562] : INFO : Starting Training [2022-02-06 08:34:24 main:574] : INFO : Epoch 1294 | loss: 0.0312919 | val_loss: 0.0311946 | Time: 17828.2 ms [2022-02-06 08:34:31 main:574] : INFO : Epoch 1295 | loss: 0.0311842 | val_loss: 0.0311752 | Time: 6736.13 ms [2022-02-06 08:34:38 main:574] : INFO : Epoch 1296 | loss: 0.0311737 | val_loss: 0.0311713 | Time: 6786.6 ms [2022-02-06 08:34:44 main:574] : INFO : Epoch 1297 | loss: 0.0311673 | val_loss: 0.0311705 | Time: 6851.07 ms [2022-02-06 08:34:51 main:574] : INFO : Epoch 1298 | loss: 0.0311691 | val_loss: 0.031173 | Time: 6825.22 ms [2022-02-06 08:34:58 main:574] : INFO : Epoch 1299 | loss: 0.0311651 | val_loss: 0.0311667 | Time: 6836.27 ms [2022-02-06 08:35:06 main:574] : INFO : Epoch 1300 | loss: 0.0311673 | val_loss: 0.0311711 | Time: 7955.95 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 08:39:15 main:435] : INFO : Set logging level to 1 [2022-02-06 08:39:15 main:441] : INFO : Running in BOINC Client mode [2022-02-06 08:39:15 main:444] : INFO : Resolving all filenames [2022-02-06 08:39:15 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 08:39:15 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 08:39:15 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 08:39:15 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 08:39:15 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 08:39:15 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 08:39:15 main:474] : INFO : Configuration: [2022-02-06 08:39:15 main:475] : INFO : Model type: GRU [2022-02-06 08:39:15 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 08:39:15 main:477] : INFO : Max Epochs: 2048 [2022-02-06 08:39:15 main:478] : INFO : Batch Size: 128 [2022-02-06 08:39:15 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 08:39:15 main:480] : INFO : Patience: 10 [2022-02-06 08:39:15 main:481] : INFO : Hidden Width: 12 [2022-02-06 08:39:15 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 08:39:15 main:483] : INFO : # Backend Layers: 4 [2022-02-06 08:39:16 main:484] : INFO : # Threads: 1 [2022-02-06 08:39:16 main:486] : INFO : Preparing Dataset [2022-02-06 08:39:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 08:39:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 08:39:19 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 08:39:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 08:39:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 08:39:19 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 08:39:19 main:494] : INFO : Creating Model [2022-02-06 08:39:19 main:507] : INFO : Preparing config file [2022-02-06 08:39:19 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 08:39:19 main:512] : INFO : Loading config [2022-02-06 08:39:19 main:514] : INFO : Loading state [2022-02-06 08:39:21 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 08:39:21 main:562] : INFO : Starting Training [2022-02-06 08:39:28 main:574] : INFO : Epoch 1294 | loss: 0.0312845 | val_loss: 0.031197 | Time: 7093.98 ms [2022-02-06 08:39:35 main:574] : INFO : Epoch 1295 | loss: 0.0311853 | val_loss: 0.0311842 | Time: 6655.7 ms [2022-02-06 08:39:42 main:574] : INFO : Epoch 1296 | loss: 0.0311722 | val_loss: 0.0311695 | Time: 6770.73 ms [2022-02-06 08:39:49 main:574] : INFO : Epoch 1297 | loss: 0.0311656 | val_loss: 0.0311674 | Time: 6786.23 ms [2022-02-06 08:39:55 main:574] : INFO : Epoch 1298 | loss: 0.031163 | val_loss: 0.0311661 | Time: 6807.32 ms [2022-02-06 08:40:03 main:574] : INFO : Epoch 1299 | loss: 0.0311616 | val_loss: 0.0311713 | Time: 7593.58 ms [2022-02-06 08:40:10 main:574] : INFO : Epoch 1300 | loss: 0.0311651 | val_loss: 0.0311662 | Time: 7092.64 ms [2022-02-06 08:40:17 main:574] : INFO : Epoch 1301 | loss: 0.0311639 | val_loss: 0.0311683 | Time: 6873.02 ms [2022-02-06 08:40:26 main:574] : INFO : Epoch 1302 | loss: 0.0311629 | val_loss: 0.0311659 | Time: 8694.46 ms [2022-02-06 08:41:23 main:574] : INFO : Epoch 1303 | loss: 0.0311656 | val_loss: 0.0311658 | Time: 57201.3 ms [2022-02-06 08:41:30 main:574] : INFO : Epoch 1304 | loss: 0.0311589 | val_loss: 0.0311656 | Time: 6749.32 ms [2022-02-06 08:41:36 main:574] : INFO : Epoch 1305 | loss: 0.0311586 | val_loss: 0.0311653 | Time: 6776.5 ms [2022-02-06 08:41:43 main:574] : INFO : Epoch 1306 | loss: 0.0311594 | val_loss: 0.0311662 | Time: 6791.99 ms [2022-02-06 08:41:50 main:574] : INFO : Epoch 1307 | loss: 0.0311581 | val_loss: 0.0311663 | Time: 6794.47 ms [2022-02-06 08:41:57 main:574] : INFO : Epoch 1308 | loss: 0.0311633 | val_loss: 0.0311705 | Time: 6801.11 ms [2022-02-06 08:42:04 main:574] : INFO : Epoch 1309 | loss: 0.0311641 | val_loss: 0.0311713 | Time: 6820.84 ms [2022-02-06 08:42:10 main:574] : INFO : Epoch 1310 | loss: 0.031163 | val_loss: 0.031167 | Time: 6789.55 ms [2022-02-06 08:42:17 main:574] : INFO : Epoch 1311 | loss: 0.031161 | val_loss: 0.0311651 | Time: 6808.04 ms [2022-02-06 08:42:24 main:574] : INFO : Epoch 1312 | loss: 0.03116 | val_loss: 0.0311638 | Time: 6843.7 ms [2022-02-06 08:42:31 main:574] : INFO : Epoch 1313 | loss: 0.0311643 | val_loss: 0.0311659 | Time: 6918.2 ms [2022-02-06 08:42:38 main:574] : INFO : Epoch 1314 | loss: 0.0311579 | val_loss: 0.0311807 | Time: 6808.67 ms [2022-02-06 08:42:45 main:574] : INFO : Epoch 1315 | loss: 0.0311669 | val_loss: 0.0311636 | Time: 6800.92 ms [2022-02-06 08:42:52 main:574] : INFO : Epoch 1316 | loss: 0.0311617 | val_loss: 0.0311645 | Time: 6810.05 ms [2022-02-06 08:42:59 main:574] : INFO : Epoch 1317 | loss: 0.0311588 | val_loss: 0.0311639 | Time: 6812.14 ms [2022-02-06 08:43:05 main:574] : INFO : Epoch 1318 | loss: 0.0311563 | val_loss: 0.0311642 | Time: 6856.5 ms [2022-02-06 08:43:12 main:574] : INFO : Epoch 1319 | loss: 0.0311579 | val_loss: 0.0311603 | Time: 6826.64 ms [2022-02-06 08:43:19 main:574] : INFO : Epoch 1320 | loss: 0.0311593 | val_loss: 0.0311696 | Time: 6811.17 ms [2022-02-06 08:43:26 main:574] : INFO : Epoch 1321 | loss: 0.0311568 | val_loss: 0.0311605 | Time: 7056.93 ms [2022-02-06 08:43:33 main:574] : INFO : Epoch 1322 | loss: 0.0311548 | val_loss: 0.0311638 | Time: 6791.14 ms [2022-02-06 08:43:40 main:574] : INFO : Epoch 1323 | loss: 0.031158 | val_loss: 0.0311658 | Time: 6803.77 ms [2022-02-06 08:43:47 main:574] : INFO : Epoch 1324 | loss: 0.0311623 | val_loss: 0.0311715 | Time: 6779.7 ms [2022-02-06 08:43:53 main:574] : INFO : Epoch 1325 | loss: 0.0311598 | val_loss: 0.0311637 | Time: 6875.67 ms [2022-02-06 08:44:00 main:574] : INFO : Epoch 1326 | loss: 0.0311561 | val_loss: 0.0311618 | Time: 6811.46 ms [2022-02-06 08:44:07 main:574] : INFO : Epoch 1327 | loss: 0.0311573 | val_loss: 0.0311629 | Time: 6808.91 ms [2022-02-06 08:44:14 main:574] : INFO : Epoch 1328 | loss: 0.0311582 | val_loss: 0.0311628 | Time: 6799.19 ms [2022-02-06 08:44:21 main:574] : INFO : Epoch 1329 | loss: 0.0311577 | val_loss: 0.0311652 | Time: 6820.6 ms [2022-02-06 08:44:28 main:574] : INFO : Epoch 1330 | loss: 0.0311546 | val_loss: 0.0311597 | Time: 6815.29 ms [2022-02-06 08:44:34 main:574] : INFO : Epoch 1331 | loss: 0.0311537 | val_loss: 0.0311628 | Time: 6793.37 ms [2022-02-06 08:44:41 main:574] : INFO : Epoch 1332 | loss: 0.031154 | val_loss: 0.0311657 | Time: 6808.64 ms [2022-02-06 08:44:48 main:574] : INFO : Epoch 1333 | loss: 0.0311559 | val_loss: 0.0311638 | Time: 6791.3 ms [2022-02-06 08:44:55 main:574] : INFO : Epoch 1334 | loss: 0.0311567 | val_loss: 0.0311604 | Time: 6804.25 ms [2022-02-06 08:45:02 main:574] : INFO : Epoch 1335 | loss: 0.0311568 | val_loss: 0.0311617 | Time: 6800.6 ms [2022-02-06 08:45:08 main:574] : INFO : Epoch 1336 | loss: 0.031159 | val_loss: 0.0311703 | Time: 6793.93 ms [2022-02-06 08:45:15 main:574] : INFO : Epoch 1337 | loss: 0.0311591 | val_loss: 0.0311666 | Time: 6817.57 ms [2022-02-06 08:45:22 main:574] : INFO : Epoch 1338 | loss: 0.0311533 | val_loss: 0.0311647 | Time: 6793.37 ms [2022-02-06 08:45:29 main:574] : INFO : Epoch 1339 | loss: 0.0311605 | val_loss: 0.0311687 | Time: 7160.86 ms [2022-02-06 08:45:37 main:574] : INFO : Epoch 1340 | loss: 0.0311593 | val_loss: 0.0311632 | Time: 7865.36 ms [2022-02-06 08:45:54 main:574] : INFO : Epoch 1341 | loss: 0.0311553 | val_loss: 0.0311639 | Time: 16906.4 ms [2022-02-06 08:46:01 main:574] : INFO : Epoch 1342 | loss: 0.031154 | val_loss: 0.0311621 | Time: 6793.54 ms [2022-02-06 08:46:08 main:574] : INFO : Epoch 1343 | loss: 0.0311572 | val_loss: 0.0311616 | Time: 6798.89 ms [2022-02-06 08:46:15 main:574] : INFO : Epoch 1344 | loss: 0.0311608 | val_loss: 0.0311669 | Time: 6796.02 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 09:25:42 main:435] : INFO : Set logging level to 1 [2022-02-06 09:25:42 main:441] : INFO : Running in BOINC Client mode [2022-02-06 09:25:42 main:444] : INFO : Resolving all filenames [2022-02-06 09:25:42 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 09:25:42 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 09:25:42 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 09:25:42 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 09:25:42 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 09:25:42 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 09:25:42 main:474] : INFO : Configuration: [2022-02-06 09:25:42 main:475] : INFO : Model type: GRU [2022-02-06 09:25:42 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 09:25:42 main:477] : INFO : Max Epochs: 2048 [2022-02-06 09:25:42 main:478] : INFO : Batch Size: 128 [2022-02-06 09:25:42 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 09:25:42 main:480] : INFO : Patience: 10 [2022-02-06 09:25:42 main:481] : INFO : Hidden Width: 12 [2022-02-06 09:25:42 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 09:25:42 main:483] : INFO : # Backend Layers: 4 [2022-02-06 09:25:42 main:484] : INFO : # Threads: 1 [2022-02-06 09:25:42 main:486] : INFO : Preparing Dataset [2022-02-06 09:25:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 09:25:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 09:25:46 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 09:25:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 09:25:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 09:25:47 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 09:25:47 main:494] : INFO : Creating Model [2022-02-06 09:25:47 main:507] : INFO : Preparing config file [2022-02-06 09:25:47 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 09:25:47 main:512] : INFO : Loading config [2022-02-06 09:25:47 main:514] : INFO : Loading state [2022-02-06 09:25:48 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 09:25:48 main:562] : INFO : Starting Training [2022-02-06 09:25:56 main:574] : INFO : Epoch 1340 | loss: 0.031324 | val_loss: 0.0311934 | Time: 7543.22 ms [2022-02-06 09:26:02 main:574] : INFO : Epoch 1341 | loss: 0.0311772 | val_loss: 0.0311747 | Time: 6780.38 ms [2022-02-06 09:26:09 main:574] : INFO : Epoch 1342 | loss: 0.0311625 | val_loss: 0.0311659 | Time: 6829.64 ms [2022-02-06 09:26:16 main:574] : INFO : Epoch 1343 | loss: 0.0311599 | val_loss: 0.0311857 | Time: 6780.87 ms [2022-02-06 09:26:23 main:574] : INFO : Epoch 1344 | loss: 0.0311637 | val_loss: 0.0311892 | Time: 6822.23 ms [2022-02-06 09:26:30 main:574] : INFO : Epoch 1345 | loss: 0.0311612 | val_loss: 0.0311666 | Time: 7014.39 ms [2022-02-06 09:26:37 main:574] : INFO : Epoch 1346 | loss: 0.0311558 | val_loss: 0.0311625 | Time: 6852.7 ms [2022-02-06 09:26:44 main:574] : INFO : Epoch 1347 | loss: 0.031154 | val_loss: 0.031166 | Time: 6830.44 ms [2022-02-06 09:27:12 main:574] : INFO : Epoch 1348 | loss: 0.0311591 | val_loss: 0.0311729 | Time: 28040.5 ms [2022-02-06 09:27:19 main:574] : INFO : Epoch 1349 | loss: 0.0311615 | val_loss: 0.0311665 | Time: 6808.83 ms [2022-02-06 09:27:25 main:574] : INFO : Epoch 1350 | loss: 0.0311541 | val_loss: 0.0311651 | Time: 6845.93 ms [2022-02-06 09:27:32 main:574] : INFO : Epoch 1351 | loss: 0.0311509 | val_loss: 0.0311611 | Time: 6800.43 ms [2022-02-06 09:27:39 main:574] : INFO : Epoch 1352 | loss: 0.0311492 | val_loss: 0.0311618 | Time: 6818.22 ms [2022-02-06 09:27:46 main:574] : INFO : Epoch 1353 | loss: 0.031148 | val_loss: 0.031165 | Time: 6826.94 ms [2022-02-06 09:27:53 main:574] : INFO : Epoch 1354 | loss: 0.0311483 | val_loss: 0.0311613 | Time: 6841.83 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 09:30:59 main:435] : INFO : Set logging level to 1 [2022-02-06 09:30:59 main:441] : INFO : Running in BOINC Client mode [2022-02-06 09:30:59 main:444] : INFO : Resolving all filenames [2022-02-06 09:30:59 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 09:30:59 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 09:30:59 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 09:30:59 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 09:30:59 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 09:30:59 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 09:30:59 main:474] : INFO : Configuration: [2022-02-06 09:30:59 main:475] : INFO : Model type: GRU [2022-02-06 09:30:59 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 09:31:00 main:477] : INFO : Max Epochs: 2048 [2022-02-06 09:31:00 main:478] : INFO : Batch Size: 128 [2022-02-06 09:31:00 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 09:31:00 main:480] : INFO : Patience: 10 [2022-02-06 09:31:00 main:481] : INFO : Hidden Width: 12 [2022-02-06 09:31:00 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 09:31:00 main:483] : INFO : # Backend Layers: 4 [2022-02-06 09:31:00 main:484] : INFO : # Threads: 1 [2022-02-06 09:31:00 main:486] : INFO : Preparing Dataset [2022-02-06 09:31:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 09:31:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 09:31:04 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 09:31:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 09:31:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 09:31:04 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 09:31:04 main:494] : INFO : Creating Model [2022-02-06 09:31:04 main:507] : INFO : Preparing config file [2022-02-06 09:31:04 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 09:31:04 main:512] : INFO : Loading config [2022-02-06 09:31:04 main:514] : INFO : Loading state [2022-02-06 09:31:06 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 09:31:06 main:562] : INFO : Starting Training [2022-02-06 09:31:13 main:574] : INFO : Epoch 1340 | loss: 0.0312658 | val_loss: 0.0311909 | Time: 7456.23 ms [2022-02-06 09:31:20 main:574] : INFO : Epoch 1341 | loss: 0.0311755 | val_loss: 0.0311656 | Time: 6794.98 ms [2022-02-06 09:31:27 main:574] : INFO : Epoch 1342 | loss: 0.0311584 | val_loss: 0.0311602 | Time: 6790.82 ms [2022-02-06 09:31:34 main:574] : INFO : Epoch 1343 | loss: 0.0311558 | val_loss: 0.0311612 | Time: 6820.41 ms [2022-02-06 09:31:40 main:574] : INFO : Epoch 1344 | loss: 0.0311559 | val_loss: 0.0311654 | Time: 6811.21 ms [2022-02-06 09:31:47 main:574] : INFO : Epoch 1345 | loss: 0.0311578 | val_loss: 0.0311604 | Time: 6843.55 ms [2022-02-06 09:31:54 main:574] : INFO : Epoch 1346 | loss: 0.0311556 | val_loss: 0.0311598 | Time: 6813.55 ms [2022-02-06 09:33:14 main:574] : INFO : Epoch 1347 | loss: 0.0311545 | val_loss: 0.0311608 | Time: 79759.5 ms [2022-02-06 09:33:21 main:574] : INFO : Epoch 1348 | loss: 0.0311563 | val_loss: 0.0311645 | Time: 7161.75 ms [2022-02-06 09:33:28 main:574] : INFO : Epoch 1349 | loss: 0.0311562 | val_loss: 0.0311682 | Time: 6781.18 ms [2022-02-06 09:33:35 main:574] : INFO : Epoch 1350 | loss: 0.0311601 | val_loss: 0.0311665 | Time: 6831.8 ms [2022-02-06 09:33:42 main:574] : INFO : Epoch 1351 | loss: 0.0311576 | val_loss: 0.0311611 | Time: 6830.91 ms [2022-02-06 09:33:49 main:574] : INFO : Epoch 1352 | loss: 0.0311597 | val_loss: 0.0311619 | Time: 6983.74 ms [2022-02-06 09:33:55 main:574] : INFO : Epoch 1353 | loss: 0.0311556 | val_loss: 0.031162 | Time: 6782.91 ms [2022-02-06 09:34:02 main:574] : INFO : Epoch 1354 | loss: 0.0311616 | val_loss: 0.0311665 | Time: 6901.38 ms [2022-02-06 09:34:09 main:574] : INFO : Epoch 1355 | loss: 0.0311609 | val_loss: 0.0311596 | Time: 6980.66 ms [2022-02-06 09:34:16 main:574] : INFO : Epoch 1356 | loss: 0.0311548 | val_loss: 0.0311594 | Time: 6997.29 ms [2022-02-06 09:34:24 main:574] : INFO : Epoch 1357 | loss: 0.0311557 | val_loss: 0.0311597 | Time: 7303.83 ms [2022-02-06 09:34:31 main:574] : INFO : Epoch 1358 | loss: 0.0311557 | val_loss: 0.0311599 | Time: 7027.36 ms [2022-02-06 09:34:38 main:574] : INFO : Epoch 1359 | loss: 0.0311554 | val_loss: 0.0311693 | Time: 6808.76 ms [2022-02-06 09:34:44 main:574] : INFO : Epoch 1360 | loss: 0.0311572 | val_loss: 0.031167 | Time: 6843.66 ms [2022-02-06 09:34:51 main:574] : INFO : Epoch 1361 | loss: 0.0311576 | val_loss: 0.0311655 | Time: 6868.12 ms [2022-02-06 09:34:58 main:574] : INFO : Epoch 1362 | loss: 0.0311573 | val_loss: 0.031162 | Time: 6819.26 ms [2022-02-06 09:35:05 main:574] : INFO : Epoch 1363 | loss: 0.0311559 | val_loss: 0.0311607 | Time: 7004.48 ms [2022-02-06 09:35:12 main:574] : INFO : Epoch 1364 | loss: 0.0311565 | val_loss: 0.0311606 | Time: 6843.73 ms [2022-02-06 09:35:19 main:574] : INFO : Epoch 1365 | loss: 0.031154 | val_loss: 0.031163 | Time: 6813.48 ms [2022-02-06 09:35:26 main:574] : INFO : Epoch 1366 | loss: 0.0311559 | val_loss: 0.0311674 | Time: 6863.37 ms [2022-02-06 09:35:33 main:574] : INFO : Epoch 1367 | loss: 0.0311594 | val_loss: 0.0311601 | Time: 6850.64 ms [2022-02-06 09:35:40 main:574] : INFO : Epoch 1368 | loss: 0.0311552 | val_loss: 0.0311584 | Time: 6987.35 ms [2022-02-06 09:35:47 main:574] : INFO : Epoch 1369 | loss: 0.0311525 | val_loss: 0.0311593 | Time: 6986.05 ms [2022-02-06 09:35:54 main:574] : INFO : Epoch 1370 | loss: 0.0311613 | val_loss: 0.0311727 | Time: 7382.36 ms [2022-02-06 09:36:01 main:574] : INFO : Epoch 1371 | loss: 0.0311687 | val_loss: 0.0311743 | Time: 6817.25 ms [2022-02-06 09:36:08 main:574] : INFO : Epoch 1372 | loss: 0.0311699 | val_loss: 0.0311754 | Time: 6810.85 ms [2022-02-06 09:36:15 main:574] : INFO : Epoch 1373 | loss: 0.0311699 | val_loss: 0.0311728 | Time: 6821.83 ms [2022-02-06 09:36:21 main:574] : INFO : Epoch 1374 | loss: 0.0311656 | val_loss: 0.031165 | Time: 6810.17 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 09:39:31 main:435] : INFO : Set logging level to 1 [2022-02-06 09:39:31 main:441] : INFO : Running in BOINC Client mode [2022-02-06 09:39:31 main:444] : INFO : Resolving all filenames [2022-02-06 09:39:31 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 09:39:31 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 09:39:31 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 09:39:31 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 09:39:31 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 09:39:31 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 09:39:31 main:474] : INFO : Configuration: [2022-02-06 09:39:31 main:475] : INFO : Model type: GRU [2022-02-06 09:39:31 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 09:39:31 main:477] : INFO : Max Epochs: 2048 [2022-02-06 09:39:31 main:478] : INFO : Batch Size: 128 [2022-02-06 09:39:31 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 09:39:31 main:480] : INFO : Patience: 10 [2022-02-06 09:39:31 main:481] : INFO : Hidden Width: 12 [2022-02-06 09:39:31 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 09:39:31 main:483] : INFO : # Backend Layers: 4 [2022-02-06 09:39:31 main:484] : INFO : # Threads: 1 [2022-02-06 09:39:31 main:486] : INFO : Preparing Dataset [2022-02-06 09:39:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 09:39:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 09:39:35 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 09:39:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 09:39:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 09:39:35 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 09:39:35 main:494] : INFO : Creating Model [2022-02-06 09:39:35 main:507] : INFO : Preparing config file [2022-02-06 09:39:35 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 09:39:35 main:512] : INFO : Loading config [2022-02-06 09:39:35 main:514] : INFO : Loading state [2022-02-06 09:39:37 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 09:39:37 main:562] : INFO : Starting Training [2022-02-06 09:39:44 main:574] : INFO : Epoch 1356 | loss: 0.0314984 | val_loss: 0.0312352 | Time: 7149.39 ms [2022-02-06 09:39:51 main:574] : INFO : Epoch 1357 | loss: 0.031205 | val_loss: 0.0311756 | Time: 6780.27 ms [2022-02-06 09:39:57 main:574] : INFO : Epoch 1358 | loss: 0.0311679 | val_loss: 0.0311656 | Time: 6779.66 ms [2022-02-06 09:40:04 main:574] : INFO : Epoch 1359 | loss: 0.0311585 | val_loss: 0.0311636 | Time: 6840.45 ms [2022-02-06 09:40:11 main:574] : INFO : Epoch 1360 | loss: 0.0311562 | val_loss: 0.031163 | Time: 6814.98 ms [2022-02-06 09:40:18 main:574] : INFO : Epoch 1361 | loss: 0.0311579 | val_loss: 0.0311746 | Time: 6831.64 ms [2022-02-06 09:40:25 main:574] : INFO : Epoch 1362 | loss: 0.0311596 | val_loss: 0.0311647 | Time: 6816.34 ms [2022-02-06 09:40:32 main:574] : INFO : Epoch 1363 | loss: 0.0311567 | val_loss: 0.0311604 | Time: 6966.51 ms [2022-02-06 09:40:39 main:574] : INFO : Epoch 1364 | loss: 0.0311532 | val_loss: 0.0311586 | Time: 7036.8 ms [2022-02-06 09:41:26 main:574] : INFO : Epoch 1365 | loss: 0.0311526 | val_loss: 0.0311605 | Time: 47090.2 ms [2022-02-06 09:41:33 main:574] : INFO : Epoch 1366 | loss: 0.0311567 | val_loss: 0.0311704 | Time: 6808 ms [2022-02-06 09:41:40 main:574] : INFO : Epoch 1367 | loss: 0.0311546 | val_loss: 0.0311654 | Time: 6890.3 ms [2022-02-06 09:41:46 main:574] : INFO : Epoch 1368 | loss: 0.0311511 | val_loss: 0.0311627 | Time: 6899.57 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 09:44:49 main:435] : INFO : Set logging level to 1 [2022-02-06 09:44:49 main:441] : INFO : Running in BOINC Client mode [2022-02-06 09:44:49 main:444] : INFO : Resolving all filenames [2022-02-06 09:44:49 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 09:44:49 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 09:44:49 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 09:44:49 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 09:44:49 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 09:44:49 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 09:44:49 main:474] : INFO : Configuration: [2022-02-06 09:44:49 main:475] : INFO : Model type: GRU [2022-02-06 09:44:49 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 09:44:49 main:477] : INFO : Max Epochs: 2048 [2022-02-06 09:44:49 main:478] : INFO : Batch Size: 128 [2022-02-06 09:44:49 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 09:44:49 main:480] : INFO : Patience: 10 [2022-02-06 09:44:49 main:481] : INFO : Hidden Width: 12 [2022-02-06 09:44:49 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 09:44:49 main:483] : INFO : # Backend Layers: 4 [2022-02-06 09:44:49 main:484] : INFO : # Threads: 1 [2022-02-06 09:44:49 main:486] : INFO : Preparing Dataset [2022-02-06 09:44:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 09:44:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 09:44:52 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 09:44:52 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 09:44:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 09:44:53 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 09:44:53 main:494] : INFO : Creating Model [2022-02-06 09:44:53 main:507] : INFO : Preparing config file [2022-02-06 09:44:53 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 09:44:53 main:512] : INFO : Loading config [2022-02-06 09:44:53 main:514] : INFO : Loading state [2022-02-06 09:44:54 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 09:44:54 main:562] : INFO : Starting Training [2022-02-06 09:45:01 main:574] : INFO : Epoch 1356 | loss: 0.0315154 | val_loss: 0.0312134 | Time: 7297.33 ms [2022-02-06 09:45:08 main:574] : INFO : Epoch 1357 | loss: 0.0311978 | val_loss: 0.0311721 | Time: 6733.94 ms [2022-02-06 09:45:15 main:574] : INFO : Epoch 1358 | loss: 0.0311642 | val_loss: 0.0311639 | Time: 6781.88 ms [2022-02-06 09:45:22 main:574] : INFO : Epoch 1359 | loss: 0.0311574 | val_loss: 0.0311615 | Time: 6854 ms [2022-02-06 09:45:29 main:574] : INFO : Epoch 1360 | loss: 0.0311568 | val_loss: 0.0311606 | Time: 6812.18 ms [2022-02-06 09:45:35 main:574] : INFO : Epoch 1361 | loss: 0.031159 | val_loss: 0.0311643 | Time: 6806.5 ms [2022-02-06 09:45:42 main:574] : INFO : Epoch 1362 | loss: 0.0311579 | val_loss: 0.0311611 | Time: 6813.94 ms [2022-02-06 09:45:49 main:574] : INFO : Epoch 1363 | loss: 0.0311566 | val_loss: 0.0311635 | Time: 6829.81 ms [2022-02-06 09:45:56 main:574] : INFO : Epoch 1364 | loss: 0.0311569 | val_loss: 0.0311644 | Time: 6960.8 ms [2022-02-06 09:46:03 main:574] : INFO : Epoch 1365 | loss: 0.031158 | val_loss: 0.0311687 | Time: 6838.91 ms [2022-02-06 09:46:10 main:574] : INFO : Epoch 1366 | loss: 0.0311585 | val_loss: 0.0311632 | Time: 6838.08 ms [2022-02-06 09:46:17 main:574] : INFO : Epoch 1367 | loss: 0.0311549 | val_loss: 0.0311657 | Time: 6814.82 ms [2022-02-06 09:46:23 main:574] : INFO : Epoch 1368 | loss: 0.0311577 | val_loss: 0.0311648 | Time: 6820.47 ms [2022-02-06 09:46:30 main:574] : INFO : Epoch 1369 | loss: 0.0311574 | val_loss: 0.0311687 | Time: 6820.24 ms [2022-02-06 09:46:37 main:574] : INFO : Epoch 1370 | loss: 0.0311563 | val_loss: 0.0311621 | Time: 6818.58 ms [2022-02-06 09:46:44 main:574] : INFO : Epoch 1371 | loss: 0.0311546 | val_loss: 0.0311634 | Time: 6817.54 ms [2022-02-06 09:46:51 main:574] : INFO : Epoch 1372 | loss: 0.0311528 | val_loss: 0.0311621 | Time: 6839.25 ms [2022-02-06 09:46:58 main:574] : INFO : Epoch 1373 | loss: 0.031154 | val_loss: 0.0311638 | Time: 7002.29 ms [2022-02-06 09:47:05 main:574] : INFO : Epoch 1374 | loss: 0.0311616 | val_loss: 0.0311652 | Time: 6900.18 ms [2022-02-06 09:47:12 main:574] : INFO : Epoch 1375 | loss: 0.0311612 | val_loss: 0.0311655 | Time: 6875.87 ms [2022-02-06 09:47:19 main:574] : INFO : Epoch 1376 | loss: 0.0311594 | val_loss: 0.0311651 | Time: 6896.34 ms [2022-02-06 09:47:26 main:574] : INFO : Epoch 1377 | loss: 0.0311584 | val_loss: 0.031165 | Time: 6953.36 ms [2022-02-06 09:47:54 main:574] : INFO : Epoch 1378 | loss: 0.0311609 | val_loss: 0.0311647 | Time: 28516.8 ms [2022-02-06 09:48:01 main:574] : INFO : Epoch 1379 | loss: 0.031164 | val_loss: 0.0311699 | Time: 6888.8 ms [2022-02-06 09:48:08 main:574] : INFO : Epoch 1380 | loss: 0.0311677 | val_loss: 0.0311681 | Time: 6868.1 ms [2022-02-06 09:48:15 main:574] : INFO : Epoch 1381 | loss: 0.0311642 | val_loss: 0.0311688 | Time: 6893.91 ms [2022-02-06 09:48:22 main:574] : INFO : Epoch 1382 | loss: 0.0311631 | val_loss: 0.0311652 | Time: 6900.16 ms [2022-02-06 09:48:29 main:574] : INFO : Epoch 1383 | loss: 0.0311614 | val_loss: 0.0311703 | Time: 6891.58 ms [2022-02-06 09:48:36 main:574] : INFO : Epoch 1384 | loss: 0.0311605 | val_loss: 0.0311693 | Time: 6911.87 ms [2022-02-06 09:48:42 main:574] : INFO : Epoch 1385 | loss: 0.0311609 | val_loss: 0.0311673 | Time: 6894.62 ms [2022-02-06 09:48:49 main:574] : INFO : Epoch 1386 | loss: 0.0311574 | val_loss: 0.0311612 | Time: 6885.65 ms [2022-02-06 09:48:56 main:574] : INFO : Epoch 1387 | loss: 0.0311551 | val_loss: 0.0311621 | Time: 6888.12 ms [2022-02-06 09:49:03 main:574] : INFO : Epoch 1388 | loss: 0.0311554 | val_loss: 0.0311666 | Time: 6929.41 ms [2022-02-06 09:49:10 main:574] : INFO : Epoch 1389 | loss: 0.0311568 | val_loss: 0.0311671 | Time: 6889.43 ms [2022-02-06 09:49:17 main:574] : INFO : Epoch 1390 | loss: 0.0311617 | val_loss: 0.0311692 | Time: 6887.58 ms [2022-02-06 09:49:24 main:574] : INFO : Epoch 1391 | loss: 0.0311642 | val_loss: 0.0311713 | Time: 7092.84 ms [2022-02-06 09:49:31 main:574] : INFO : Epoch 1392 | loss: 0.0311621 | val_loss: 0.0311663 | Time: 7094.84 ms [2022-02-06 09:49:38 main:574] : INFO : Epoch 1393 | loss: 0.0311581 | val_loss: 0.0311678 | Time: 6908 ms [2022-02-06 09:49:45 main:574] : INFO : Epoch 1394 | loss: 0.0311561 | val_loss: 0.0311654 | Time: 6892.31 ms [2022-02-06 09:49:52 main:574] : INFO : Epoch 1395 | loss: 0.0311585 | val_loss: 0.0311666 | Time: 6888.84 ms [2022-02-06 09:50:00 main:574] : INFO : Epoch 1396 | loss: 0.0311553 | val_loss: 0.0311666 | Time: 7527.38 ms [2022-02-06 09:50:17 main:574] : INFO : Epoch 1397 | loss: 0.0311544 | val_loss: 0.0311618 | Time: 17165.3 ms [2022-02-06 09:50:24 main:574] : INFO : Epoch 1398 | loss: 0.0311536 | val_loss: 0.0311637 | Time: 6928.05 ms [2022-02-06 09:50:31 main:574] : INFO : Epoch 1399 | loss: 0.0311569 | val_loss: 0.0311716 | Time: 6889.04 ms [2022-02-06 09:50:38 main:574] : INFO : Epoch 1400 | loss: 0.0311673 | val_loss: 0.0311727 | Time: 6937.27 ms [2022-02-06 09:50:45 main:574] : INFO : Epoch 1401 | loss: 0.0311646 | val_loss: 0.0311688 | Time: 7286.36 ms [2022-02-06 09:50:52 main:574] : INFO : Epoch 1402 | loss: 0.0311628 | val_loss: 0.0311683 | Time: 6892.59 ms [2022-02-06 09:50:59 main:574] : INFO : Epoch 1403 | loss: 0.0311638 | val_loss: 0.0311707 | Time: 6877.84 ms [2022-02-06 09:51:06 main:574] : INFO : Epoch 1404 | loss: 0.0311643 | val_loss: 0.0311708 | Time: 6912.38 ms [2022-02-06 09:51:13 main:574] : INFO : Epoch 1405 | loss: 0.0311618 | val_loss: 0.0311687 | Time: 6893.91 ms [2022-02-06 09:51:20 main:574] : INFO : Epoch 1406 | loss: 0.0311594 | val_loss: 0.0311675 | Time: 6942.06 ms [2022-02-06 09:51:26 main:574] : INFO : Epoch 1407 | loss: 0.0311569 | val_loss: 0.0311706 | Time: 6890.83 ms [2022-02-06 09:51:33 main:574] : INFO : Epoch 1408 | loss: 0.031161 | val_loss: 0.0311715 | Time: 7002.25 ms [2022-02-06 09:51:41 main:574] : INFO : Epoch 1409 | loss: 0.0311627 | val_loss: 0.0311706 | Time: 7464.92 ms [2022-02-06 09:51:48 main:574] : INFO : Epoch 1410 | loss: 0.0311618 | val_loss: 0.0311703 | Time: 6894.63 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 11:03:13 main:435] : INFO : Set logging level to 1 [2022-02-06 11:03:13 main:441] : INFO : Running in BOINC Client mode [2022-02-06 11:03:13 main:444] : INFO : Resolving all filenames [2022-02-06 11:03:13 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 11:03:13 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 11:03:13 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 11:03:13 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 11:03:13 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 11:03:13 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 11:03:13 main:474] : INFO : Configuration: [2022-02-06 11:03:13 main:475] : INFO : Model type: GRU [2022-02-06 11:03:13 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 11:03:13 main:477] : INFO : Max Epochs: 2048 [2022-02-06 11:03:13 main:478] : INFO : Batch Size: 128 [2022-02-06 11:03:13 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 11:03:13 main:480] : INFO : Patience: 10 [2022-02-06 11:03:13 main:481] : INFO : Hidden Width: 12 [2022-02-06 11:03:13 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 11:03:13 main:483] : INFO : # Backend Layers: 4 [2022-02-06 11:03:13 main:484] : INFO : # Threads: 1 [2022-02-06 11:03:13 main:486] : INFO : Preparing Dataset [2022-02-06 11:03:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 11:03:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 11:03:19 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 11:03:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 11:03:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 11:03:19 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 11:03:19 main:494] : INFO : Creating Model [2022-02-06 11:03:19 main:507] : INFO : Preparing config file [2022-02-06 11:03:19 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 11:03:19 main:512] : INFO : Loading config [2022-02-06 11:03:19 main:514] : INFO : Loading state [2022-02-06 11:03:23 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 11:03:23 main:562] : INFO : Starting Training [2022-02-06 11:03:43 main:574] : INFO : Epoch 1404 | loss: 0.0312549 | val_loss: 0.0311836 | Time: 19951.1 ms [2022-02-06 11:03:50 main:574] : INFO : Epoch 1405 | loss: 0.0311711 | val_loss: 0.0311758 | Time: 6799.57 ms [2022-02-06 11:03:57 main:574] : INFO : Epoch 1406 | loss: 0.0311628 | val_loss: 0.0311726 | Time: 6791.31 ms [2022-02-06 11:04:04 main:574] : INFO : Epoch 1407 | loss: 0.031163 | val_loss: 0.031176 | Time: 6848.24 ms [2022-02-06 11:04:10 main:574] : INFO : Epoch 1408 | loss: 0.0311649 | val_loss: 0.0311652 | Time: 6807.15 ms [2022-02-06 11:04:17 main:574] : INFO : Epoch 1409 | loss: 0.0311611 | val_loss: 0.0311727 | Time: 6872.53 ms [2022-02-06 11:04:24 main:574] : INFO : Epoch 1410 | loss: 0.0311646 | val_loss: 0.0311608 | Time: 6847.26 ms [2022-02-06 11:04:31 main:574] : INFO : Epoch 1411 | loss: 0.0311597 | val_loss: 0.0311618 | Time: 6847.48 ms [2022-02-06 11:04:38 main:574] : INFO : Epoch 1412 | loss: 0.0311576 | val_loss: 0.0311717 | Time: 6828.48 ms [2022-02-06 11:04:45 main:574] : INFO : Epoch 1413 | loss: 0.0311575 | val_loss: 0.031164 | Time: 6830.67 ms [2022-02-06 11:04:52 main:574] : INFO : Epoch 1414 | loss: 0.0311549 | val_loss: 0.0311631 | Time: 6832.37 ms [2022-02-06 11:04:59 main:574] : INFO : Epoch 1415 | loss: 0.0311548 | val_loss: 0.0311645 | Time: 7240.73 ms [2022-02-06 11:05:08 main:574] : INFO : Epoch 1416 | loss: 0.0311542 | val_loss: 0.0311614 | Time: 8701.2 ms [2022-02-06 11:05:25 main:574] : INFO : Epoch 1417 | loss: 0.0311556 | val_loss: 0.0311585 | Time: 16926.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 930MX) [2022-02-06 11:08:30 main:435] : INFO : Set logging level to 1 [2022-02-06 11:08:30 main:441] : INFO : Running in BOINC Client mode [2022-02-06 11:08:30 main:444] : INFO : Resolving all filenames [2022-02-06 11:08:30 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-06 11:08:30 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-06 11:08:30 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-06 11:08:30 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-06 11:08:30 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-06 11:08:30 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-06 11:08:30 main:474] : INFO : Configuration: [2022-02-06 11:08:30 main:475] : INFO : Model type: GRU [2022-02-06 11:08:30 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-06 11:08:30 main:477] : INFO : Max Epochs: 2048 [2022-02-06 11:08:30 main:478] : INFO : Batch Size: 128 [2022-02-06 11:08:30 main:479] : INFO : Learning Rate: 0.01 [2022-02-06 11:08:31 main:480] : INFO : Patience: 10 [2022-02-06 11:08:31 main:481] : INFO : Hidden Width: 12 [2022-02-06 11:08:31 main:482] : INFO : # Recurrent Layers: 4 [2022-02-06 11:08:31 main:483] : INFO : # Backend Layers: 4 [2022-02-06 11:08:31 main:484] : INFO : # Threads: 1 [2022-02-06 11:08:31 main:486] : INFO : Preparing Dataset [2022-02-06 11:08:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-06 11:08:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-06 11:08:34 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-06 11:08:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-06 11:08:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-06 11:08:34 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-06 11:08:34 main:494] : INFO : Creating Model [2022-02-06 11:08:34 main:507] : INFO : Preparing config file [2022-02-06 11:08:34 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-06 11:08:34 main:512] : INFO : Loading config [2022-02-06 11:08:34 main:514] : INFO : Loading state [2022-02-06 11:08:35 main:559] : INFO : Loading DataLoader into Memory [2022-02-06 11:08:35 main:562] : INFO : Starting Training [2022-02-06 11:08:42 main:574] : INFO : Epoch 1404 | loss: 0.0314167 | val_loss: 0.0312892 | Time: 7025.01 ms [2022-02-06 11:08:49 main:574] : INFO : Epoch 1405 | loss: 0.0312525 | val_loss: 0.0312205 | Time: 6741.97 ms [2022-02-06 11:08:56 main:574] : INFO : Epoch 1406 | loss: 0.031195 | val_loss: 0.0311742 | Time: 6783.72 ms [2022-02-06 11:09:03 main:574] : INFO : Epoch 1407 | loss: 0.0311653 | val_loss: 0.0311674 | Time: 6855.96 ms [2022-02-06 11:09:10 main:574] : INFO : Epoch 1408 | loss: 0.0311592 | val_loss: 0.0311603 | Time: 6814.93 ms [2022-02-06 11:09:17 main:574] : INFO : Epoch 1409 | loss: 0.031156 | val_loss: 0.0311603 | Time: 6819.41 ms [2022-02-06 11:09:24 main:574] : INFO : Epoch 1410 | loss: 0.0311564 | val_loss: 0.0311609 | Time: 6968.46 ms [2022-02-06 11:09:31 main:574] : INFO : Epoch 1411 | loss: 0.0311547 | val_loss: 0.0311617 | Time: 7146.42 ms [2022-02-06 11:09:38 main:574] : INFO : Epoch 1412 | loss: 0.0311543 | val_loss: 0.0311641 | Time: 6808.37 ms [2022-02-06 11:09:44 main:574] : INFO : Epoch 1413 | loss: 0.0311585 | val_loss: 0.0311637 | Time: 6827.79 ms [2022-02-06 11:09:51 main:574] : INFO : Epoch 1414 | loss: 0.031155 | val_loss: 0.0311672 | Time: 6826.94 ms [2022-02-06 11:09:58 main:574] : INFO : Epoch 1415 | loss: 0.0311533 | val_loss: 0.0311607 | Time: 6850.08 ms [2022-02-06 11:10:05 main:574] : INFO : Epoch 1416 | loss: 0.0311517 | val_loss: 0.0311664 | Time: 6945.22 ms [2022-02-06 11:10:33 main:574] : INFO : Epoch 1417 | loss: 0.0311543 | val_loss: 0.0311587 | Time: 27583.3 ms [2022-02-06 11:10:40 main:574] : INFO : Epoch 1418 | loss: 0.0311536 | val_loss: 0.0311601 | Time: 6871.58 ms [2022-02-06 11:10:46 main:574] : INFO : Epoch 1419 | loss: 0.0311499 | val_loss: 0.0311623 | Time: 6872.08 ms [2022-02-06 11:10:53 main:574] : INFO : Epoch 1420 | loss: 0.0311503 | val_loss: 0.0311625 | Time: 6933.68 ms [2022-02-06 11:11:00 main:574] : INFO : Epoch 1421 | loss: 0.0311562 | val_loss: 0.0311596 | Time: 6952.99 ms [2022-02-06 11:11:07 main:574] : INFO : Epoch 1422 | loss: 0.0311566 | val_loss: 0.0311581 | Time: 6885.62 ms [2022-02-06 11:11:14 main:574] : INFO : Epoch 1423 | loss: 0.0311606 | val_loss: 0.0311668 | Time: 7223.37 ms [2022-02-06 11:11:21 main:574] : INFO : Epoch 1424 | loss: 0.0311578 | val_loss: 0.0311628 | Time: 6877.62 ms [2022-02-06 11:11:28 main:574] : INFO : Epoch 1425 | loss: 0.0311564 | val_loss: 0.0311649 | Time: 6930.46 ms [2022-02-06 11:11:35 main:574] : INFO : Epoch 1426 | loss: 0.0311588 | val_loss: 0.0311639 | Time: 6895.38 ms [2022-02-06 11:13:04 main:574] : INFO : Epoch 1427 | loss: 0.0311586 | val_loss: 0.0311603 | Time: 88202.8 ms [2022-02-06 11:13:10 main:574] : INFO : Epoch 1428 | loss: 0.0311564 | val_loss: 0.0311653 | Time: 6833 ms [2022-02-06 11:13:17 main:574] : INFO : Epoch 1429 | loss: 0.031156 | val_loss: 0.0311599 | Time: 6796.83 ms [2022-02-06 11:13:24 main:574] : INFO : Epoch 1430 | loss: 0.0311544 | val_loss: 0.0311591 | Time: 6945.18 ms [2022-02-06 11:13:31 main:574] : INFO : Epoch 1431 | loss: 0.0311541 | val_loss: 0.0311707 | Time: 7106.74 ms [2022-02-06 11:13:38 main:574] : INFO : Epoch 1432 | loss: 0.031158 | val_loss: 0.0311608 | Time: 6849.08 ms [2022-02-06 11:13:45 main:574] : INFO : Epoch 1433 | loss: 0.0311565 | val_loss: 0.0311684 | Time: 7074.92 ms [2022-02-06 11:13:52 main:574] : INFO : Epoch 1434 | loss: 0.0311571 | val_loss: 0.0311671 | Time: 6987.7 ms [2022-02-06 11:13:59 main:574] : INFO : Epoch 1435 | loss: 0.0311565 | val_loss: 0.0311675 | Time: 6890.58 ms [2022-02-06 11:14:07 main:574] : INFO : Epoch 1436 | loss: 0.0311575 | val_loss: 0.0311712 | Time: 7330.23 ms [2022-02-06 11:14:14 main:574] : INFO : Epoch 1437 | loss: 0.0311598 | val_loss: 0.0311728 | Time: 7180.69 ms [2022-02-06 11:14:21 main:574] : INFO : Epoch 1438 | loss: 0.0311564 | val_loss: 0.0311634 | Time: 7044.78 ms [2022-02-06 11:14:28 main:574] : INFO : Epoch 1439 | loss: 0.0311519 | val_loss: 0.0311625 | Time: 7566.94 ms [2022-02-06 11:14:45 main:574] : INFO : Epoch 1440 | loss: 0.0311531 | val_loss: 0.0311614 | Time: 16962.8 ms [2022-02-06 11:14:53 main:574] : INFO : Epoch 1441 | loss: 0.0311541 | val_loss: 0.0311611 | Time: 7074.62 ms [2022-02-06 11:14:59 main:574] : INFO : Epoch 1442 | loss: 0.0311503 | val_loss: 0.0311615 | Time: 6892.87 ms [2022-02-06 11:15:07 main:574] : INFO : Epoch 1443 | loss: 0.0311495 | val_loss: 0.0311681 | Time: 7193.46 ms [2022-02-06 11:15:14 main:574] : INFO : Epoch 1444 | loss: 0.0311535 | val_loss: 0.0311681 | Time: 6941.31 ms </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)