Task 14549995
| Name | ParityModified-1647048575-19463-3-0_0 |
| Workunit | 11575450 |
| Created | 8 Apr 2022, 12:01:56 UTC |
| Sent | 10 Apr 2022, 17:29:59 UTC |
| Report deadline | 18 Apr 2022, 17:29:59 UTC |
| Received | 18 Apr 2022, 18:10:56 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 18579 |
| Run time | 2 hours 12 min 8 sec |
| CPU time | 2 hours 4 min 54 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 14,366.03 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.42 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 8:31 main:574] : INFO : Epoch 1667 | loss: 0.0307697 | val_loss: 0.0309015 | Time: 3494.03 ms [2022-04-12 22:48:35 main:574] : INFO : Epoch 1668 | loss: 0.0307658 | val_loss: 0.0309129 | Time: 3491.87 ms [2022-04-12 22:48:38 main:574] : INFO : Epoch 1669 | loss: 0.0307667 | val_loss: 0.0309205 | Time: 3499.7 ms [2022-04-12 22:48:42 main:574] : INFO : Epoch 1670 | loss: 0.0307944 | val_loss: 0.0309033 | Time: 3533.07 ms [2022-04-12 22:48:45 main:574] : INFO : Epoch 1671 | loss: 0.0307986 | val_loss: 0.0309054 | Time: 3535.56 ms [2022-04-12 22:48:49 main:574] : INFO : Epoch 1672 | loss: 0.0307875 | val_loss: 0.0309033 | Time: 3491.41 ms [2022-04-12 22:48:52 main:574] : INFO : Epoch 1673 | loss: 0.030781 | val_loss: 0.0308955 | Time: 3462.34 ms [2022-04-12 22:48:56 main:574] : INFO : Epoch 1674 | loss: 0.0307738 | val_loss: 0.030896 | Time: 3511.17 ms [2022-04-12 22:48:59 main:574] : INFO : Epoch 1675 | loss: 0.0307797 | val_loss: 0.0309004 | Time: 3530.16 ms [2022-04-12 22:49:03 main:574] : INFO : Epoch 1676 | loss: 0.0307925 | val_loss: 0.030906 | Time: 3479.95 ms [2022-04-12 22:49:06 main:574] : INFO : Epoch 1677 | loss: 0.0307927 | val_loss: 0.0308807 | Time: 3484.71 ms [2022-04-12 22:49:10 main:574] : INFO : Epoch 1678 | loss: 0.030775 | val_loss: 0.0309163 | Time: 3482.76 ms [2022-04-12 22:49:13 main:574] : INFO : Epoch 1679 | loss: 0.0307722 | val_loss: 0.030894 | Time: 3500.03 ms [2022-04-12 22:49:17 main:574] : INFO : Epoch 1680 | loss: 0.0307701 | val_loss: 0.0309006 | Time: 3519.14 ms [2022-04-12 22:49:20 main:574] : INFO : Epoch 1681 | loss: 0.0307681 | val_loss: 0.0308897 | Time: 3500.63 ms [2022-04-12 22:49:24 main:574] : INFO : Epoch 1682 | loss: 0.030768 | val_loss: 0.0308942 | Time: 3493.54 ms [2022-04-12 22:49:28 main:574] : INFO : Epoch 1683 | loss: 0.0307707 | val_loss: 0.030882 | Time: 3551.38 ms [2022-04-12 22:49:31 main:574] : INFO : Epoch 1684 | loss: 0.0307637 | val_loss: 0.0308883 | Time: 3487.76 ms [2022-04-12 22:49:35 main:574] : INFO : Epoch 1685 | loss: 0.0307625 | val_loss: 0.0309174 | Time: 3468.63 ms [2022-04-12 22:49:38 main:574] : INFO : Epoch 1686 | loss: 0.0307648 | val_loss: 0.0308963 | Time: 3486.62 ms [2022-04-12 22:49:42 main:574] : INFO : Epoch 1687 | loss: 0.0307542 | val_loss: 0.0308988 | Time: 3511.85 ms [2022-04-12 22:49:45 main:574] : INFO : Epoch 1688 | loss: 0.0307552 | val_loss: 0.0308989 | Time: 3517.36 ms [2022-04-12 22:49:49 main:574] : INFO : Epoch 1689 | loss: 0.0307571 | val_loss: 0.0308932 | Time: 3505.59 ms [2022-04-12 22:49:52 main:574] : INFO : Epoch 1690 | loss: 0.0307528 | val_loss: 0.0308773 | Time: 3497.83 ms [2022-04-12 22:49:56 main:574] : INFO : Epoch 1691 | loss: 0.0307514 | val_loss: 0.0308934 | Time: 3480.93 ms [2022-04-12 22:49:59 main:574] : INFO : Epoch 1692 | loss: 0.03075 | val_loss: 0.0309069 | Time: 3470.41 ms [2022-04-12 22:50:03 main:574] : INFO : Epoch 1693 | loss: 0.0307471 | val_loss: 0.0309202 | Time: 3489.99 ms [2022-04-12 22:50:06 main:574] : INFO : Epoch 1694 | loss: 0.0307513 | val_loss: 0.0309231 | Time: 3485.57 ms [2022-04-12 22:50:10 main:574] : INFO : Epoch 1695 | loss: 0.0307568 | val_loss: 0.0309189 | Time: 3489.37 ms [2022-04-12 22:50:13 main:574] : INFO : Epoch 1696 | loss: 0.0307472 | val_loss: 0.0308833 | Time: 3480.94 ms [2022-04-12 22:50:17 main:574] : INFO : Epoch 1697 | loss: 0.0307395 | val_loss: 0.030943 | Time: 3475.64 ms [2022-04-12 22:50:20 main:574] : INFO : Epoch 1698 | loss: 0.0307578 | val_loss: 0.0308984 | Time: 3539.36 ms [2022-04-12 22:50:24 main:574] : INFO : Epoch 1699 | loss: 0.0307717 | val_loss: 0.0308792 | Time: 3470.19 ms [2022-04-12 22:50:27 main:574] : INFO : Epoch 1700 | loss: 0.0307583 | val_loss: 0.0309036 | Time: 3479.74 ms [2022-04-12 22:50:31 main:574] : INFO : Epoch 1701 | loss: 0.0307512 | val_loss: 0.0308816 | Time: 3478.81 ms [2022-04-12 22:50:34 main:574] : INFO : Epoch 1702 | loss: 0.030759 | val_loss: 0.0308717 | Time: 3500.1 ms [2022-04-12 22:50:38 main:574] : INFO : Epoch 1703 | loss: 0.0307675 | val_loss: 0.0309412 | Time: 3494.45 ms [2022-04-12 22:50:41 main:574] : INFO : Epoch 1704 | loss: 0.030759 | val_loss: 0.0309245 | Time: 3535.07 ms [2022-04-12 22:50:45 main:574] : INFO : Epoch 1705 | loss: 0.0307488 | val_loss: 0.0309003 | Time: 3503.68 ms [2022-04-12 22:50:48 main:574] : INFO : Epoch 1706 | loss: 0.0307677 | val_loss: 0.0309114 | Time: 3491.74 ms [2022-04-12 22:50:52 main:574] : INFO : Epoch 1707 | loss: 0.0307639 | val_loss: 0.0308823 | Time: 3489.65 ms [2022-04-12 22:50:55 main:574] : INFO : Epoch 1708 | loss: 0.0307526 | val_loss: 0.0308818 | Time: 3476.88 ms [2022-04-12 22:50:59 main:574] : INFO : Epoch 1709 | loss: 0.0307443 | val_loss: 0.030904 | Time: 3499.06 ms [2022-04-12 22:51:02 main:574] : INFO : Epoch 1710 | loss: 0.0307312 | val_loss: 0.0309096 | Time: 3487.58 ms [2022-04-12 22:51:06 main:574] : INFO : Epoch 1711 | loss: 0.0307398 | val_loss: 0.0309104 | Time: 3521.82 ms [2022-04-12 22:51:09 main:574] : INFO : Epoch 1712 | loss: 0.0307423 | val_loss: 0.0309259 | Time: 3506.12 ms [2022-04-12 22:51:13 main:574] : INFO : Epoch 1713 | loss: 0.030753 | val_loss: 0.0309057 | Time: 3513.68 ms [2022-04-12 22:51:16 main:574] : INFO : Epoch 1714 | loss: 0.0308091 | val_loss: 0.0309346 | Time: 3491.87 ms [2022-04-12 22:51:20 main:574] : INFO : Epoch 1715 | loss: 0.0307797 | val_loss: 0.030889 | Time: 3520.94 ms [2022-04-12 22:51:23 main:574] : INFO : Epoch 1716 | loss: 0.0307964 | val_loss: 0.0309182 | Time: 3511.3 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-04-13 23:19:06 main:435] : INFO : Set logging level to 1 [2022-04-13 23:19:06 main:441] : INFO : Running in BOINC Client mode [2022-04-13 23:19:06 main:444] : INFO : Resolving all filenames [2022-04-13 23:19:06 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 23:19:06 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 23:19:06 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 23:19:06 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 23:19:06 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 23:19:06 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 23:19:06 main:474] : INFO : Configuration: [2022-04-13 23:19:06 main:475] : INFO : Model type: GRU [2022-04-13 23:19:06 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 23:19:06 main:477] : INFO : Max Epochs: 2048 [2022-04-13 23:19:06 main:478] : INFO : Batch Size: 128 [2022-04-13 23:19:06 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 23:19:06 main:480] : INFO : Patience: 10 [2022-04-13 23:19:06 main:481] : INFO : Hidden Width: 12 [2022-04-13 23:19:06 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 23:19:06 main:483] : INFO : # Backend Layers: 4 [2022-04-13 23:19:06 main:484] : INFO : # Threads: 1 [2022-04-13 23:19:06 main:486] : INFO : Preparing Dataset [2022-04-13 23:19:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 23:19:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 23:19:08 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 23:19:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 23:19:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 23:19:09 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 23:19:09 main:494] : INFO : Creating Model [2022-04-13 23:19:09 main:507] : INFO : Preparing config file [2022-04-13 23:19:09 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 23:19:09 main:512] : INFO : Loading config [2022-04-13 23:19:09 main:514] : INFO : Loading state [2022-04-13 23:19:11 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 23:19:11 main:562] : INFO : Starting Training [2022-04-13 23:19:15 main:574] : INFO : Epoch 1705 | loss: 0.0309189 | val_loss: 0.030934 | Time: 4080.2 ms [2022-04-13 23:19:23 main:574] : INFO : Epoch 1706 | loss: 0.0308051 | val_loss: 0.030896 | Time: 8024.9 ms [2022-04-13 23:19:36 main:574] : INFO : Epoch 1707 | loss: 0.0307661 | val_loss: 0.0308805 | Time: 13821.6 ms [2022-04-13 23:19:48 main:574] : INFO : Epoch 1708 | loss: 0.0307514 | val_loss: 0.0308819 | Time: 11104.8 ms [2022-04-13 23:20:02 main:574] : INFO : Epoch 1709 | loss: 0.0307438 | val_loss: 0.0308886 | Time: 14096.7 ms [2022-04-13 23:20:17 main:574] : INFO : Epoch 1710 | loss: 0.0307419 | val_loss: 0.0308912 | Time: 15083.9 ms [2022-04-13 23:20:28 main:574] : INFO : Epoch 1711 | loss: 0.0307405 | val_loss: 0.0309072 | Time: 11065.3 ms [2022-04-13 23:20:37 main:574] : INFO : Epoch 1712 | loss: 0.0307395 | val_loss: 0.0309063 | Time: 9319.77 ms [2022-04-13 23:20:49 main:574] : INFO : Epoch 1713 | loss: 0.0307362 | val_loss: 0.0308881 | Time: 11947.6 ms [2022-04-13 23:20:59 main:574] : INFO : Epoch 1714 | loss: 0.0307344 | val_loss: 0.0308925 | Time: 9911.17 ms [2022-04-13 23:21:06 main:574] : INFO : Epoch 1715 | loss: 0.0307561 | val_loss: 0.0309375 | Time: 7249.7 ms [2022-04-13 23:21:14 main:574] : INFO : Epoch 1716 | loss: 0.0307337 | val_loss: 0.030888 | Time: 7280.72 ms [2022-04-13 23:21:21 main:574] : INFO : Epoch 1717 | loss: 0.0307488 | val_loss: 0.0309266 | Time: 7299.17 ms [2022-04-13 23:21:28 main:574] : INFO : Epoch 1718 | loss: 0.0307543 | val_loss: 0.0308905 | Time: 7132.01 ms [2022-04-13 23:21:36 main:574] : INFO : Epoch 1719 | loss: 0.0307512 | val_loss: 0.0309104 | Time: 7870.92 ms [2022-04-13 23:21:43 main:574] : INFO : Epoch 1720 | loss: 0.0307503 | val_loss: 0.0309103 | Time: 7067.85 ms [2022-04-13 23:21:50 main:574] : INFO : Epoch 1721 | loss: 0.030767 | val_loss: 0.0308879 | Time: 6935.44 ms [2022-04-13 23:21:57 main:574] : INFO : Epoch 1722 | loss: 0.0307542 | val_loss: 0.0309194 | Time: 6766.54 ms [2022-04-13 23:22:04 main:574] : INFO : Epoch 1723 | loss: 0.0307612 | val_loss: 0.0309108 | Time: 6758.41 ms [2022-04-13 23:22:11 main:574] : INFO : Epoch 1724 | loss: 0.0307464 | val_loss: 0.0308747 | Time: 6805.68 ms [2022-04-13 23:22:17 main:574] : INFO : Epoch 1725 | loss: 0.0307425 | val_loss: 0.0309095 | Time: 6641.42 ms [2022-04-13 23:22:24 main:574] : INFO : Epoch 1726 | loss: 0.0307445 | val_loss: 0.0309319 | Time: 6640.66 ms [2022-04-13 23:22:31 main:574] : INFO : Epoch 1727 | loss: 0.0307438 | val_loss: 0.0308852 | Time: 6801.49 ms [2022-04-13 23:22:38 main:574] : INFO : Epoch 1728 | loss: 0.0307448 | val_loss: 0.0308997 | Time: 6921.84 ms [2022-04-13 23:22:44 main:574] : INFO : Epoch 1729 | loss: 0.030738 | val_loss: 0.0309077 | Time: 6729.84 ms [2022-04-13 23:22:51 main:574] : INFO : Epoch 1730 | loss: 0.0307421 | val_loss: 0.0308894 | Time: 6621.85 ms [2022-04-13 23:22:58 main:574] : INFO : Epoch 1731 | loss: 0.0307858 | val_loss: 0.0309054 | Time: 6696.39 ms [2022-04-13 23:23:05 main:574] : INFO : Epoch 1732 | loss: 0.0308207 | val_loss: 0.030903 | Time: 6933.78 ms [2022-04-13 23:23:12 main:574] : INFO : Epoch 1733 | loss: 0.0307743 | val_loss: 0.0308725 | Time: 7100.41 ms [2022-04-13 23:23:18 main:574] : INFO : Epoch 1734 | loss: 0.0307588 | val_loss: 0.030872 | Time: 6731.36 ms [2022-04-13 23:23:26 main:574] : INFO : Epoch 1735 | loss: 0.0307424 | val_loss: 0.0309119 | Time: 7123.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-04-13 23:32:12 main:435] : INFO : Set logging level to 1 [2022-04-13 23:32:12 main:441] : INFO : Running in BOINC Client mode [2022-04-13 23:32:12 main:444] : INFO : Resolving all filenames [2022-04-13 23:32:12 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 23:32:12 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 23:32:12 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 23:32:12 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 23:32:12 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 23:32:12 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 23:32:12 main:474] : INFO : Configuration: [2022-04-13 23:32:12 main:475] : INFO : Model type: GRU [2022-04-13 23:32:12 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 23:32:12 main:477] : INFO : Max Epochs: 2048 [2022-04-13 23:32:12 main:478] : INFO : Batch Size: 128 [2022-04-13 23:32:12 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 23:32:12 main:480] : INFO : Patience: 10 [2022-04-13 23:32:12 main:481] : INFO : Hidden Width: 12 [2022-04-13 23:32:12 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 23:32:12 main:483] : INFO : # Backend Layers: 4 [2022-04-13 23:32:12 main:484] : INFO : # Threads: 1 [2022-04-13 23:32:12 main:486] : INFO : Preparing Dataset [2022-04-13 23:32:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 23:32:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 23:32:14 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 23:32:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 23:32:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 23:32:14 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 23:32:14 main:494] : INFO : Creating Model [2022-04-13 23:32:14 main:507] : INFO : Preparing config file [2022-04-13 23:32:14 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 23:32:14 main:512] : INFO : Loading config [2022-04-13 23:32:14 main:514] : INFO : Loading state [2022-04-13 23:32:15 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 23:32:15 main:562] : INFO : Starting Training [2022-04-13 23:32:20 main:574] : INFO : Epoch 1729 | loss: 0.0309066 | val_loss: 0.0309314 | Time: 4122.71 ms [2022-04-13 23:32:23 main:574] : INFO : Epoch 1730 | loss: 0.0307884 | val_loss: 0.0309257 | Time: 3576.09 ms [2022-04-13 23:32:30 main:574] : INFO : Epoch 1731 | loss: 0.0307461 | val_loss: 0.0308852 | Time: 6697.34 ms [2022-04-13 23:32:37 main:574] : INFO : Epoch 1732 | loss: 0.0307412 | val_loss: 0.0309123 | Time: 7224.2 ms [2022-04-13 23:32:45 main:574] : INFO : Epoch 1733 | loss: 0.0307674 | val_loss: 0.0308905 | Time: 7534.36 ms [2022-04-13 23:32:52 main:574] : INFO : Epoch 1734 | loss: 0.0307467 | val_loss: 0.0309107 | Time: 7476.51 ms [2022-04-13 23:33:00 main:574] : INFO : Epoch 1735 | loss: 0.0307298 | val_loss: 0.0308851 | Time: 7643.29 ms [2022-04-13 23:33:07 main:574] : INFO : Epoch 1736 | loss: 0.030727 | val_loss: 0.0309082 | Time: 7479.66 ms [2022-04-13 23:33:15 main:574] : INFO : Epoch 1737 | loss: 0.0307277 | val_loss: 0.0309234 | Time: 7664.85 ms [2022-04-13 23:33:23 main:574] : INFO : Epoch 1738 | loss: 0.0307237 | val_loss: 0.030926 | Time: 7563.02 ms [2022-04-13 23:33:30 main:574] : INFO : Epoch 1739 | loss: 0.030731 | val_loss: 0.0308825 | Time: 7637.49 ms [2022-04-13 23:33:38 main:574] : INFO : Epoch 1740 | loss: 0.0307427 | val_loss: 0.0309429 | Time: 7526.56 ms [2022-04-13 23:33:45 main:574] : INFO : Epoch 1741 | loss: 0.0307487 | val_loss: 0.0308963 | Time: 7536.31 ms [2022-04-13 23:33:53 main:574] : INFO : Epoch 1742 | loss: 0.0307459 | val_loss: 0.0308989 | Time: 7507.12 ms [2022-04-13 23:34:00 main:574] : INFO : Epoch 1743 | loss: 0.0307547 | val_loss: 0.0309143 | Time: 7551.35 ms [2022-04-13 23:34:08 main:574] : INFO : Epoch 1744 | loss: 0.0307417 | val_loss: 0.0309126 | Time: 7193.5 ms [2022-04-13 23:34:15 main:574] : INFO : Epoch 1745 | loss: 0.0307489 | val_loss: 0.0309379 | Time: 7548.61 ms [2022-04-13 23:34:23 main:574] : INFO : Epoch 1746 | loss: 0.0307376 | val_loss: 0.0309018 | Time: 7663.55 ms [2022-04-13 23:34:30 main:574] : INFO : Epoch 1747 | loss: 0.0307367 | val_loss: 0.0308879 | Time: 7519.21 ms [2022-04-13 23:34:40 main:574] : INFO : Epoch 1748 | loss: 0.0307353 | val_loss: 0.0308938 | Time: 9177.99 ms [2022-04-13 23:34:49 main:574] : INFO : Epoch 1749 | loss: 0.0307667 | val_loss: 0.030903 | Time: 9008.15 ms [2022-04-13 23:34:56 main:574] : INFO : Epoch 1750 | loss: 0.0307421 | val_loss: 0.0308967 | Time: 7815.26 ms [2022-04-13 23:35:04 main:574] : INFO : Epoch 1751 | loss: 0.0307283 | val_loss: 0.0309075 | Time: 7813.97 ms [2022-04-13 23:35:12 main:574] : INFO : Epoch 1752 | loss: 0.0307439 | val_loss: 0.0308877 | Time: 7715.44 ms [2022-04-13 23:35:19 main:574] : INFO : Epoch 1753 | loss: 0.0307321 | val_loss: 0.0309116 | Time: 7350.04 ms [2022-04-13 23:35:27 main:574] : INFO : Epoch 1754 | loss: 0.0307261 | val_loss: 0.0309065 | Time: 7518.85 ms [2022-04-13 23:35:35 main:574] : INFO : Epoch 1755 | loss: 0.0307272 | val_loss: 0.0309053 | Time: 7604.26 ms [2022-04-13 23:35:42 main:574] : INFO : Epoch 1756 | loss: 0.0307286 | val_loss: 0.0308956 | Time: 7425.23 ms [2022-04-13 23:35:50 main:574] : INFO : Epoch 1757 | loss: 0.0307168 | val_loss: 0.0309169 | Time: 7534.33 ms [2022-04-13 23:35:57 main:574] : INFO : Epoch 1758 | loss: 0.0307091 | val_loss: 0.0309247 | Time: 7524.83 ms [2022-04-13 23:36:05 main:574] : INFO : Epoch 1759 | loss: 0.0307216 | val_loss: 0.0309223 | Time: 7552.45 ms [2022-04-13 23:36:12 main:574] : INFO : Epoch 1760 | loss: 0.0307366 | val_loss: 0.0308859 | Time: 7117.11 ms [2022-04-13 23:36:19 main:574] : INFO : Epoch 1761 | loss: 0.0307337 | val_loss: 0.0308914 | Time: 6988.27 ms [2022-04-13 23:36:26 main:574] : INFO : Epoch 1762 | loss: 0.0307517 | val_loss: 0.0309058 | Time: 6881.56 ms [2022-04-13 23:36:33 main:574] : INFO : Epoch 1763 | loss: 0.0307511 | val_loss: 0.0308929 | Time: 6960.15 ms [2022-04-13 23:36:40 main:574] : INFO : Epoch 1764 | loss: 0.0307692 | val_loss: 0.030916 | Time: 7141.95 ms [2022-04-13 23:36:48 main:574] : INFO : Epoch 1765 | loss: 0.0307513 | val_loss: 0.0308837 | Time: 8324.38 ms [2022-04-13 23:36:55 main:574] : INFO : Epoch 1766 | loss: 0.0307357 | val_loss: 0.0309067 | Time: 6710.12 ms [2022-04-13 23:37:02 main:574] : INFO : Epoch 1767 | loss: 0.0307237 | val_loss: 0.0308812 | Time: 6948.08 ms [2022-04-13 23:37:09 main:574] : INFO : Epoch 1768 | loss: 0.0307302 | val_loss: 0.0309178 | Time: 6994.08 ms [2022-04-13 23:37:16 main:574] : INFO : Epoch 1769 | loss: 0.030736 | val_loss: 0.0308987 | Time: 7043.35 ms [2022-04-13 23:37:23 main:574] : INFO : Epoch 1770 | loss: 0.0307243 | val_loss: 0.0308708 | Time: 6978.5 ms [2022-04-13 23:37:30 main:574] : INFO : Epoch 1771 | loss: 0.0307204 | val_loss: 0.0309143 | Time: 6999.44 ms [2022-04-13 23:37:37 main:574] : INFO : Epoch 1772 | loss: 0.0307163 | val_loss: 0.0309254 | Time: 7008.26 ms [2022-04-13 23:37:45 main:574] : INFO : Epoch 1773 | loss: 0.0307157 | val_loss: 0.0309058 | Time: 7443.81 ms [2022-04-13 23:37:52 main:574] : INFO : Epoch 1774 | loss: 0.030736 | val_loss: 0.0309032 | Time: 7892.56 ms [2022-04-13 23:38:00 main:574] : INFO : Epoch 1775 | loss: 0.0307298 | val_loss: 0.0309112 | Time: 7646.2 ms [2022-04-13 23:38:09 main:574] : INFO : Epoch 1776 | loss: 0.0307241 | val_loss: 0.0309147 | Time: 8726.59 ms [2022-04-13 23:38:18 main:574] : INFO : Epoch 1777 | loss: 0.0307318 | val_loss: 0.0309057 | Time: 8950.6 ms [2022-04-13 23:38:25 main:574] : INFO : Epoch 1778 | loss: 0.0307342 | val_loss: 0.030894 | Time: 7427.6 ms [2022-04-13 23:38:34 main:574] : INFO : Epoch 1779 | loss: 0.0307235 | val_loss: 0.03089 | Time: 8989.73 ms [2022-04-13 23:38:42 main:574] : INFO : Epoch 1780 | loss: 0.0307076 | val_loss: 0.0309113 | Time: 7469.12 ms [2022-04-13 23:38:51 main:574] : INFO : Epoch 1781 | loss: 0.0307102 | val_loss: 0.0309182 | Time: 9536.41 ms [2022-04-13 23:39:00 main:574] : INFO : Epoch 1782 | loss: 0.0307117 | val_loss: 0.0309013 | Time: 8342.4 ms [2022-04-13 23:39:07 main:574] : INFO : Epoch 1783 | loss: 0.0307199 | val_loss: 0.0309485 | Time: 7426.88 ms [2022-04-13 23:39:17 main:574] : INFO : Epoch 1784 | loss: 0.0307194 | val_loss: 0.0309027 | Time: 9613.54 ms [2022-04-13 23:39:24 main:574] : INFO : Epoch 1785 | loss: 0.0307249 | val_loss: 0.0309112 | Time: 7254.44 ms [2022-04-13 23:39:32 main:574] : INFO : Epoch 1786 | loss: 0.0307229 | val_loss: 0.0308796 | Time: 7466.23 ms [2022-04-13 23:39:39 main:574] : INFO : Epoch 1787 | loss: 0.0307097 | val_loss: 0.030954 | Time: 7027.13 ms [2022-04-13 23:39:46 main:574] : INFO : Epoch 1788 | loss: 0.0307084 | val_loss: 0.0308951 | Time: 7113.84 ms [2022-04-13 23:39:53 main:574] : INFO : Epoch 1789 | loss: 0.0307103 | val_loss: 0.0309676 | Time: 7065.27 ms [2022-04-13 23:40:00 main:574] : INFO : Epoch 1790 | loss: 0.030712 | val_loss: 0.0309335 | Time: 7062.54 ms [2022-04-13 23:40:07 main:574] : INFO : Epoch 1791 | loss: 0.0307105 | val_loss: 0.030909 | Time: 6840.38 ms [2022-04-13 23:40:14 main:574] : INFO : Epoch 1792 | loss: 0.0307128 | val_loss: 0.0309182 | Time: 6965.36 ms [2022-04-13 23:40:21 main:574] : INFO : Epoch 1793 | loss: 0.0307059 | val_loss: 0.030938 | Time: 6972.66 ms [2022-04-13 23:40:28 main:574] : INFO : Epoch 1794 | loss: 0.0307065 | val_loss: 0.0309313 | Time: 7002.97 ms [2022-04-13 23:40:35 main:574] : INFO : Epoch 1795 | loss: 0.0307157 | val_loss: 0.0309268 | Time: 7081.45 ms [2022-04-13 23:40:42 main:574] : INFO : Epoch 1796 | loss: 0.0307624 | val_loss: 0.0309054 | Time: 6750.95 ms [2022-04-13 23:40:49 main:574] : INFO : Epoch 1797 | loss: 0.0307553 | val_loss: 0.0309225 | Time: 6945.25 ms [2022-04-13 23:40:56 main:574] : INFO : Epoch 1798 | loss: 0.0307412 | val_loss: 0.0309021 | Time: 6925.28 ms [2022-04-13 23:41:02 main:574] : INFO : Epoch 1799 | loss: 0.0307322 | val_loss: 0.0309057 | Time: 6914.23 ms [2022-04-13 23:41:09 main:574] : INFO : Epoch 1800 | loss: 0.0307182 | val_loss: 0.0309058 | Time: 6955.71 ms [2022-04-13 23:41:16 main:574] : INFO : Epoch 1801 | loss: 0.030706 | val_loss: 0.0309261 | Time: 6671.67 ms [2022-04-13 23:41:23 main:574] : INFO : Epoch 1802 | loss: 0.030709 | val_loss: 0.0309324 | Time: 6963.44 ms [2022-04-13 23:41:30 main:574] : INFO : Epoch 1803 | loss: 0.0307218 | val_loss: 0.0309264 | Time: 7099.54 ms [2022-04-13 23:41:37 main:574] : INFO : Epoch 1804 | loss: 0.0307251 | val_loss: 0.0308976 | Time: 7044.13 ms [2022-04-13 23:41:44 main:574] : INFO : Epoch 1805 | loss: 0.0307174 | val_loss: 0.030899 | Time: 6955.47 ms [2022-04-13 23:41:51 main:574] : INFO : Epoch 1806 | loss: 0.0307113 | val_loss: 0.030904 | Time: 6957.95 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-04-18 19:15:20 main:435] : INFO : Set logging level to 1 [2022-04-18 19:15:20 main:441] : INFO : Running in BOINC Client mode [2022-04-18 19:15:20 main:444] : INFO : Resolving all filenames [2022-04-18 19:15:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-18 19:15:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-18 19:15:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-18 19:15:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-18 19:15:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-18 19:15:20 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-18 19:15:20 main:474] : INFO : Configuration: [2022-04-18 19:15:20 main:475] : INFO : Model type: GRU [2022-04-18 19:15:20 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-18 19:15:20 main:477] : INFO : Max Epochs: 2048 [2022-04-18 19:15:20 main:478] : INFO : Batch Size: 128 [2022-04-18 19:15:20 main:479] : INFO : Learning Rate: 0.01 [2022-04-18 19:15:20 main:480] : INFO : Patience: 10 [2022-04-18 19:15:20 main:481] : INFO : Hidden Width: 12 [2022-04-18 19:15:20 main:482] : INFO : # Recurrent Layers: 4 [2022-04-18 19:15:20 main:483] : INFO : # Backend Layers: 4 [2022-04-18 19:15:20 main:484] : INFO : # Threads: 1 [2022-04-18 19:15:20 main:486] : INFO : Preparing Dataset [2022-04-18 19:15:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-18 19:15:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-18 19:15:23 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-18 19:15:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-18 19:15:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-18 19:15:23 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-18 19:15:23 main:494] : INFO : Creating Model [2022-04-18 19:15:23 main:507] : INFO : Preparing config file [2022-04-18 19:15:23 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-18 19:15:23 main:512] : INFO : Loading config [2022-04-18 19:15:23 main:514] : INFO : Loading state [2022-04-18 19:15:25 main:559] : INFO : Loading DataLoader into Memory [2022-04-18 19:15:25 main:562] : INFO : Starting Training [2022-04-18 19:15:30 main:574] : INFO : Epoch 1803 | loss: 0.0308295 | val_loss: 0.0309272 | Time: 4650.43 ms [2022-04-18 19:15:34 main:574] : INFO : Epoch 1804 | loss: 0.0307257 | val_loss: 0.0308963 | Time: 4000.44 ms [2022-04-18 19:15:38 main:574] : INFO : Epoch 1805 | loss: 0.0307137 | val_loss: 0.0309241 | Time: 3826.94 ms [2022-04-18 19:15:42 main:574] : INFO : Epoch 1806 | loss: 0.0307246 | val_loss: 0.0309532 | Time: 3907.6 ms [2022-04-18 19:15:46 main:574] : INFO : Epoch 1807 | loss: 0.0307215 | val_loss: 0.0309195 | Time: 3966 ms [2022-04-18 19:15:49 main:574] : INFO : Epoch 1808 | loss: 0.0307056 | val_loss: 0.0309562 | Time: 3900.55 ms [2022-04-18 19:15:53 main:574] : INFO : Epoch 1809 | loss: 0.0307377 | val_loss: 0.0309023 | Time: 3875.59 ms [2022-04-18 19:15:57 main:574] : INFO : Epoch 1810 | loss: 0.0307284 | val_loss: 0.0309089 | Time: 3862.23 ms [2022-04-18 19:16:01 main:574] : INFO : Epoch 1811 | loss: 0.0307774 | val_loss: 0.0308938 | Time: 3883.88 ms [2022-04-18 19:16:05 main:574] : INFO : Epoch 1812 | loss: 0.0307496 | val_loss: 0.0308776 | Time: 3907.55 ms [2022-04-18 19:16:09 main:574] : INFO : Epoch 1813 | loss: 0.0307238 | val_loss: 0.0308869 | Time: 3907.79 ms [2022-04-18 19:16:13 main:574] : INFO : Epoch 1814 | loss: 0.0307272 | val_loss: 0.0309 | Time: 3918.28 ms [2022-04-18 19:16:17 main:574] : INFO : Epoch 1815 | loss: 0.0307122 | val_loss: 0.0308867 | Time: 3926.36 ms [2022-04-18 19:16:21 main:574] : INFO : Epoch 1816 | loss: 0.0307034 | val_loss: 0.0308944 | Time: 3892.48 ms [2022-04-18 19:16:25 main:574] : INFO : Epoch 1817 | loss: 0.0306931 | val_loss: 0.0308918 | Time: 3901.08 ms [2022-04-18 19:16:29 main:574] : INFO : Epoch 1818 | loss: 0.0306889 | val_loss: 0.030886 | Time: 3906.54 ms [2022-04-18 19:16:32 main:574] : INFO : Epoch 1819 | loss: 0.0306991 | val_loss: 0.0308931 | Time: 3925.03 ms [2022-04-18 19:16:36 main:574] : INFO : Epoch 1820 | loss: 0.0306873 | val_loss: 0.0309303 | Time: 3910.74 ms [2022-04-18 19:16:40 main:574] : INFO : Epoch 1821 | loss: 0.0307277 | val_loss: 0.0308858 | Time: 3915.9 ms [2022-04-18 19:16:44 main:574] : INFO : Epoch 1822 | loss: 0.0307319 | val_loss: 0.0309391 | Time: 3982.39 ms [2022-04-18 19:16:48 main:574] : INFO : Epoch 1823 | loss: 0.030712 | val_loss: 0.0309228 | Time: 3954.22 ms [2022-04-18 19:16:52 main:574] : INFO : Epoch 1824 | loss: 0.0306978 | val_loss: 0.0308898 | Time: 3957.31 ms [2022-04-18 19:16:56 main:574] : INFO : Epoch 1825 | loss: 0.0306896 | val_loss: 0.0309073 | Time: 4047.79 ms [2022-04-18 19:17:00 main:574] : INFO : Epoch 1826 | loss: 0.0306878 | val_loss: 0.0309142 | Time: 3960.23 ms [2022-04-18 19:17:05 main:574] : INFO : Epoch 1827 | loss: 0.0306915 | val_loss: 0.0308825 | Time: 4442.05 ms [2022-04-18 19:17:09 main:574] : INFO : Epoch 1828 | loss: 0.0306934 | val_loss: 0.0308918 | Time: 4417.63 ms [2022-04-18 19:17:13 main:574] : INFO : Epoch 1829 | loss: 0.0307042 | val_loss: 0.0309241 | Time: 4160.64 ms [2022-04-18 19:17:17 main:574] : INFO : Epoch 1830 | loss: 0.0307484 | val_loss: 0.0309388 | Time: 4123.81 ms [2022-04-18 19:17:22 main:574] : INFO : Epoch 1831 | loss: 0.0307469 | val_loss: 0.0308752 | Time: 4158.5 ms [2022-04-18 19:17:26 main:574] : INFO : Epoch 1832 | loss: 0.0307143 | val_loss: 0.0309015 | Time: 4223.61 ms [2022-04-18 19:17:30 main:574] : INFO : Epoch 1833 | loss: 0.030711 | val_loss: 0.0309007 | Time: 4029.81 ms [2022-04-18 19:17:34 main:574] : INFO : Epoch 1834 | loss: 0.0307285 | val_loss: 0.0309267 | Time: 4060.33 ms [2022-04-18 19:17:38 main:574] : INFO : Epoch 1835 | loss: 0.0307098 | val_loss: 0.0309185 | Time: 4094.24 ms [2022-04-18 19:17:42 main:574] : INFO : Epoch 1836 | loss: 0.0307093 | val_loss: 0.0309146 | Time: 3990.75 ms [2022-04-18 19:17:46 main:574] : INFO : Epoch 1837 | loss: 0.0307163 | val_loss: 0.0308812 | Time: 4020.06 ms [2022-04-18 19:17:50 main:574] : INFO : Epoch 1838 | loss: 0.030712 | val_loss: 0.030948 | Time: 4159.6 ms [2022-04-18 19:17:55 main:574] : INFO : Epoch 1839 | loss: 0.0306983 | val_loss: 0.0309174 | Time: 4507.3 ms [2022-04-18 19:17:59 main:574] : INFO : Epoch 1840 | loss: 0.0307098 | val_loss: 0.0308895 | Time: 4206.86 ms [2022-04-18 19:18:03 main:574] : INFO : Epoch 1841 | loss: 0.0307475 | val_loss: 0.0308996 | Time: 4072.99 ms [2022-04-18 19:18:08 main:574] : INFO : Epoch 1842 | loss: 0.0307186 | val_loss: 0.0308944 | Time: 4583.73 ms [2022-04-18 19:18:12 main:574] : INFO : Epoch 1843 | loss: 0.0307252 | val_loss: 0.0309379 | Time: 4730.05 ms [2022-04-18 19:18:16 main:574] : INFO : Epoch 1844 | loss: 0.0307412 | val_loss: 0.0309054 | Time: 3957.36 ms [2022-04-18 19:18:20 main:574] : INFO : Epoch 1845 | loss: 0.0307279 | val_loss: 0.0309298 | Time: 3939.87 ms [2022-04-18 19:18:24 main:574] : INFO : Epoch 1846 | loss: 0.0307176 | val_loss: 0.0308947 | Time: 4004.88 ms [2022-04-18 19:18:28 main:574] : INFO : Epoch 1847 | loss: 0.0307148 | val_loss: 0.0309301 | Time: 3945.04 ms [2022-04-18 19:18:32 main:574] : INFO : Epoch 1848 | loss: 0.0307161 | val_loss: 0.0308777 | Time: 3982.77 ms [2022-04-18 19:18:36 main:574] : INFO : Epoch 1849 | loss: 0.0307012 | val_loss: 0.0308704 | Time: 3959.84 ms [2022-04-18 19:18:40 main:574] : INFO : Epoch 1850 | loss: 0.0306905 | val_loss: 0.0308999 | Time: 3953.71 ms [2022-04-18 19:18:44 main:574] : INFO : Epoch 1851 | loss: 0.0307085 | val_loss: 0.0309216 | Time: 3929.61 ms [2022-04-18 19:18:48 main:574] : INFO : Epoch 1852 | loss: 0.0306943 | val_loss: 0.0309152 | Time: 4182.9 ms [2022-04-18 19:18:52 main:574] : INFO : Epoch 1853 | loss: 0.0307138 | val_loss: 0.0309111 | Time: 3986.84 ms [2022-04-18 19:18:56 main:574] : INFO : Epoch 1854 | loss: 0.0307098 | val_loss: 0.0308829 | Time: 4025.34 ms [2022-04-18 19:19:04 main:574] : INFO : Epoch 1855 | loss: 0.0307075 | val_loss: 0.0308939 | Time: 7867.45 ms [2022-04-18 19:19:15 main:574] : INFO : Epoch 1856 | loss: 0.0306913 | val_loss: 0.0309381 | Time: 10564.3 ms [2022-04-18 19:19:19 main:574] : INFO : Epoch 1857 | loss: 0.030681 | val_loss: 0.0309107 | Time: 3964.91 ms [2022-04-18 19:19:23 main:574] : INFO : Epoch 1858 | loss: 0.0306892 | val_loss: 0.0309081 | Time: 3992.26 ms [2022-04-18 19:19:27 main:574] : INFO : Epoch 1859 | loss: 0.0306869 | val_loss: 0.0309159 | Time: 3896.07 ms [2022-04-18 19:19:31 main:574] : INFO : Epoch 1860 | loss: 0.0307191 | val_loss: 0.0308937 | Time: 3913.8 ms [2022-04-18 19:19:35 main:574] : INFO : Epoch 1861 | loss: 0.0307293 | val_loss: 0.0308807 | Time: 3945.62 ms [2022-04-18 19:19:39 main:574] : INFO : Epoch 1862 | loss: 0.0307365 | val_loss: 0.0309161 | Time: 3954.55 ms [2022-04-18 19:19:43 main:574] : INFO : Epoch 1863 | loss: 0.0307203 | val_loss: 0.0308626 | Time: 3911.61 ms [2022-04-18 19:19:47 main:574] : INFO : Epoch 1864 | loss: 0.0307063 | val_loss: 0.0309639 | Time: 3950.15 ms [2022-04-18 19:19:50 main:574] : INFO : Epoch 1865 | loss: 0.0307155 | val_loss: 0.0308842 | Time: 3841.47 ms [2022-04-18 19:19:54 main:574] : INFO : Epoch 1866 | loss: 0.0306894 | val_loss: 0.0309076 | Time: 3910.74 ms [2022-04-18 19:19:58 main:574] : INFO : Epoch 1867 | loss: 0.0308513 | val_loss: 0.0309171 | Time: 3940.7 ms [2022-04-18 19:20:02 main:574] : INFO : Epoch 1868 | loss: 0.0307685 | val_loss: 0.0308726 | Time: 3983.3 ms [2022-04-18 19:20:06 main:574] : INFO : Epoch 1869 | loss: 0.030713 | val_loss: 0.0308873 | Time: 3881.71 ms [2022-04-18 19:20:10 main:574] : INFO : Epoch 1870 | loss: 0.0306873 | val_loss: 0.030913 | Time: 3917.29 ms [2022-04-18 19:20:14 main:574] : INFO : Epoch 1871 | loss: 0.0307017 | val_loss: 0.0308957 | Time: 3932.56 ms [2022-04-18 19:20:18 main:574] : INFO : Epoch 1872 | loss: 0.0307177 | val_loss: 0.0309331 | Time: 3897.1 ms [2022-04-18 19:20:22 main:574] : INFO : Epoch 1873 | loss: 0.0307215 | val_loss: 0.0309277 | Time: 4029.05 ms [2022-04-18 19:20:26 main:574] : INFO : Epoch 1874 | loss: 0.0307129 | val_loss: 0.0309193 | Time: 3956.61 ms [2022-04-18 19:20:30 main:574] : INFO : Epoch 1875 | loss: 0.0307068 | val_loss: 0.0308867 | Time: 4011.56 ms [2022-04-18 19:20:34 main:574] : INFO : Epoch 1876 | loss: 0.0307037 | val_loss: 0.0309268 | Time: 3974.24 ms [2022-04-18 19:20:38 main:574] : INFO : Epoch 1877 | loss: 0.0306946 | val_loss: 0.0309124 | Time: 3961.35 ms [2022-04-18 19:20:42 main:574] : INFO : Epoch 1878 | loss: 0.0307499 | val_loss: 0.0309058 | Time: 3895.41 ms [2022-04-18 19:20:46 main:574] : INFO : Epoch 1879 | loss: 0.030725 | val_loss: 0.0308987 | Time: 3850.3 ms [2022-04-18 19:20:50 main:574] : INFO : Epoch 1880 | loss: 0.0307025 | val_loss: 0.0308675 | Time: 3936.93 ms [2022-04-18 19:20:54 main:574] : INFO : Epoch 1881 | loss: 0.0306806 | val_loss: 0.0308989 | Time: 3943.45 ms [2022-04-18 19:20:58 main:574] : INFO : Epoch 1882 | loss: 0.0306816 | val_loss: 0.0309057 | Time: 3950.09 ms [2022-04-18 19:21:02 main:574] : INFO : Epoch 1883 | loss: 0.030689 | val_loss: 0.0309337 | Time: 3991.19 ms [2022-04-18 19:21:06 main:574] : INFO : Epoch 1884 | loss: 0.0306762 | val_loss: 0.0308975 | Time: 4005.21 ms [2022-04-18 19:21:10 main:574] : INFO : Epoch 1885 | loss: 0.0306869 | val_loss: 0.0309201 | Time: 3929.21 ms [2022-04-18 19:21:13 main:574] : INFO : Epoch 1886 | loss: 0.0306856 | val_loss: 0.0308898 | Time: 3936.62 ms [2022-04-18 19:21:17 main:574] : INFO : Epoch 1887 | loss: 0.0306921 | val_loss: 0.0309037 | Time: 3914.18 ms [2022-04-18 19:21:21 main:574] : INFO : Epoch 1888 | loss: 0.0306905 | val_loss: 0.0308735 | Time: 3924.71 ms [2022-04-18 19:21:25 main:574] : INFO : Epoch 1889 | loss: 0.0307109 | val_loss: 0.0309078 | Time: 3928.02 ms [2022-04-18 19:21:29 main:574] : INFO : Epoch 1890 | loss: 0.0306889 | val_loss: 0.0308868 | Time: 3952.93 ms [2022-04-18 19:21:33 main:574] : INFO : Epoch 1891 | loss: 0.0306861 | val_loss: 0.030934 | Time: 3965.47 ms [2022-04-18 19:21:37 main:574] : INFO : Epoch 1892 | loss: 0.0306886 | val_loss: 0.030892 | Time: 3863.2 ms [2022-04-18 19:21:41 main:574] : INFO : Epoch 1893 | loss: 0.0307163 | val_loss: 0.0309226 | Time: 3918.97 ms [2022-04-18 19:21:45 main:574] : INFO : Epoch 1894 | loss: 0.0306945 | val_loss: 0.0308637 | Time: 3944.32 ms [2022-04-18 19:21:49 main:574] : INFO : Epoch 1895 | loss: 0.0306926 | val_loss: 0.0309474 | Time: 3899.33 ms [2022-04-18 19:21:53 main:574] : INFO : Epoch 1896 | loss: 0.030674 | val_loss: 0.0309035 | Time: 3924.86 ms [2022-04-18 19:21:57 main:574] : INFO : Epoch 1897 | loss: 0.030668 | val_loss: 0.0309245 | Time: 3890.75 ms [2022-04-18 19:22:01 main:574] : INFO : Epoch 1898 | loss: 0.0306731 | val_loss: 0.0309292 | Time: 3967.88 ms [2022-04-18 19:22:05 main:574] : INFO : Epoch 1899 | loss: 0.0306639 | val_loss: 0.030926 | Time: 3982.22 ms [2022-04-18 19:22:09 main:574] : INFO : Epoch 1900 | loss: 0.0306624 | val_loss: 0.0308929 | Time: 3972.74 ms [2022-04-18 19:22:13 main:574] : INFO : Epoch 1901 | loss: 0.030678 | val_loss: 0.0308842 | Time: 3946.06 ms [2022-04-18 19:22:17 main:574] : INFO : Epoch 1902 | loss: 0.030685 | val_loss: 0.0309084 | Time: 4151.34 ms [2022-04-18 19:22:21 main:574] : INFO : Epoch 1903 | loss: 0.0306738 | val_loss: 0.0308982 | Time: 4020.54 ms [2022-04-18 19:22:25 main:574] : INFO : Epoch 1904 | loss: 0.0306674 | val_loss: 0.030925 | Time: 4023.97 ms [2022-04-18 19:22:29 main:574] : INFO : Epoch 1905 | loss: 0.030662 | val_loss: 0.0309043 | Time: 3974.78 ms [2022-04-18 19:22:33 main:574] : INFO : Epoch 1906 | loss: 0.0306682 | val_loss: 0.0309184 | Time: 3953.63 ms [2022-04-18 19:22:37 main:574] : INFO : Epoch 1907 | loss: 0.0306682 | val_loss: 0.0309197 | Time: 3955.15 ms [2022-04-18 19:22:41 main:574] : INFO : Epoch 1908 | loss: 0.0306644 | val_loss: 0.0309314 | Time: 3992.2 ms [2022-04-18 19:22:45 main:574] : INFO : Epoch 1909 | loss: 0.0306722 | val_loss: 0.0308975 | Time: 3961.62 ms [2022-04-18 19:22:49 main:574] : INFO : Epoch 1910 | loss: 0.0306816 | val_loss: 0.0309133 | Time: 3964.78 ms [2022-04-18 19:22:53 main:574] : INFO : Epoch 1911 | loss: 0.0306978 | val_loss: 0.0308974 | Time: 4047.29 ms [2022-04-18 19:22:57 main:574] : INFO : Epoch 1912 | loss: 0.0306874 | val_loss: 0.0309332 | Time: 3926.21 ms [2022-04-18 19:23:01 main:574] : INFO : Epoch 1913 | loss: 0.0306703 | val_loss: 0.0309019 | Time: 3985.95 ms [2022-04-18 19:23:05 main:574] : INFO : Epoch 1914 | loss: 0.0306689 | val_loss: 0.0309241 | Time: 3942.31 ms [2022-04-18 19:23:09 main:574] : INFO : Epoch 1915 | loss: 0.0306907 | val_loss: 0.0308998 | Time: 3982.91 ms [2022-04-18 19:23:13 main:574] : INFO : Epoch 1916 | loss: 0.0307165 | val_loss: 0.0308743 | Time: 3924.15 ms [2022-04-18 19:23:17 main:574] : INFO : Epoch 1917 | loss: 0.0307563 | val_loss: 0.0308972 | Time: 3987.65 ms [2022-04-18 19:23:21 main:574] : INFO : Epoch 1918 | loss: 0.030695 | val_loss: 0.0308685 | Time: 3991.55 ms [2022-04-18 19:23:25 main:574] : INFO : Epoch 1919 | loss: 0.0306896 | val_loss: 0.0308686 | Time: 3927.05 ms [2022-04-18 19:23:29 main:574] : INFO : Epoch 1920 | loss: 0.0306921 | val_loss: 0.0308914 | Time: 4020.4 ms [2022-04-18 19:23:33 main:574] : INFO : Epoch 1921 | loss: 0.0307031 | val_loss: 0.0308518 | Time: 4018.32 ms [2022-04-18 19:23:36 main:574] : INFO : Epoch 1922 | loss: 0.0307501 | val_loss: 0.0309339 | Time: 3903.07 ms [2022-04-18 19:23:40 main:574] : INFO : Epoch 1923 | loss: 0.0307161 | val_loss: 0.0308994 | Time: 3973.12 ms [2022-04-18 19:23:44 main:574] : INFO : Epoch 1924 | loss: 0.0306865 | val_loss: 0.0308902 | Time: 3947.56 ms [2022-04-18 19:23:48 main:574] : INFO : Epoch 1925 | loss: 0.0306875 | val_loss: 0.0309046 | Time: 3933.61 ms [2022-04-18 19:23:52 main:574] : INFO : Epoch 1926 | loss: 0.0306959 | val_loss: 0.0309114 | Time: 3977.95 ms [2022-04-18 19:23:56 main:574] : INFO : Epoch 1927 | loss: 0.0307298 | val_loss: 0.030905 | Time: 3978.4 ms [2022-04-18 19:24:00 main:574] : INFO : Epoch 1928 | loss: 0.0306881 | val_loss: 0.0308669 | Time: 3989.16 ms [2022-04-18 19:24:04 main:574] : INFO : Epoch 1929 | loss: 0.0306904 | val_loss: 0.0308934 | Time: 3956.86 ms [2022-04-18 19:24:08 main:574] : INFO : Epoch 1930 | loss: 0.0306906 | val_loss: 0.0308966 | Time: 3957.01 ms [2022-04-18 19:24:12 main:574] : INFO : Epoch 1931 | loss: 0.0306913 | val_loss: 0.0308863 | Time: 3954.81 ms [2022-04-18 19:24:16 main:574] : INFO : Epoch 1932 | loss: 0.0306816 | val_loss: 0.0309093 | Time: 3960.52 ms [2022-04-18 19:24:20 main:574] : INFO : Epoch 1933 | loss: 0.0306787 | val_loss: 0.0308782 | Time: 3995.39 ms [2022-04-18 19:24:24 main:574] : INFO : Epoch 1934 | loss: 0.0307095 | val_loss: 0.0309223 | Time: 4008.48 ms [2022-04-18 19:24:28 main:574] : INFO : Epoch 1935 | loss: 0.0307199 | val_loss: 0.0309419 | Time: 4004.31 ms [2022-04-18 19:24:32 main:574] : INFO : Epoch 1936 | loss: 0.0307098 | val_loss: 0.030858 | Time: 4012.33 ms [2022-04-18 19:24:36 main:574] : INFO : Epoch 1937 | loss: 0.030719 | val_loss: 0.0308633 | Time: 4065.83 ms [2022-04-18 19:24:40 main:574] : INFO : Epoch 1938 | loss: 0.0306935 | val_loss: 0.0308761 | Time: 4020.77 ms [2022-04-18 19:24:44 main:574] : INFO : Epoch 1939 | loss: 0.0306728 | val_loss: 0.0308868 | Time: 4019.11 ms [2022-04-18 19:24:48 main:574] : INFO : Epoch 1940 | loss: 0.0306778 | val_loss: 0.0308957 | Time: 3893.83 ms [2022-04-18 19:24:52 main:574] : INFO : Epoch 1941 | loss: 0.0306646 | val_loss: 0.030935 | Time: 3972.56 ms [2022-04-18 19:24:56 main:574] : INFO : Epoch 1942 | loss: 0.0306742 | val_loss: 0.0309124 | Time: 4032.85 ms [2022-04-18 19:25:01 main:574] : INFO : Epoch 1943 | loss: 0.0306934 | val_loss: 0.0309024 | Time: 4157.49 ms [2022-04-18 19:25:05 main:574] : INFO : Epoch 1944 | loss: 0.0306805 | val_loss: 0.0308907 | Time: 4184.36 ms [2022-04-18 19:25:09 main:574] : INFO : Epoch 1945 | loss: 0.0306818 | val_loss: 0.0309108 | Time: 4240.98 ms [2022-04-18 19:25:14 main:574] : INFO : Epoch 1946 | loss: 0.0306753 | val_loss: 0.0309186 | Time: 4810.71 ms [2022-04-18 19:25:19 main:574] : INFO : Epoch 1947 | loss: 0.0306921 | val_loss: 0.0308866 | Time: 4755.18 ms [2022-04-18 19:25:23 main:574] : INFO : Epoch 1948 | loss: 0.0307445 | val_loss: 0.0308994 | Time: 4688 ms [2022-04-18 19:25:28 main:574] : INFO : Epoch 1949 | loss: 0.0307495 | val_loss: 0.0309024 | Time: 4442.71 ms [2022-04-18 19:25:32 main:574] : INFO : Epoch 1950 | loss: 0.0307018 | val_loss: 0.0308634 | Time: 4165.37 ms [2022-04-18 19:25:36 main:574] : INFO : Epoch 1951 | loss: 0.0307998 | val_loss: 0.031059 | Time: 4169.38 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-04-18 20:03:22 main:435] : INFO : Set logging level to 1 [2022-04-18 20:03:22 main:441] : INFO : Running in BOINC Client mode [2022-04-18 20:03:22 main:444] : INFO : Resolving all filenames [2022-04-18 20:03:22 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-18 20:03:22 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-18 20:03:22 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-18 20:03:22 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-18 20:03:22 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-18 20:03:22 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-18 20:03:22 main:474] : INFO : Configuration: [2022-04-18 20:03:22 main:475] : INFO : Model type: GRU [2022-04-18 20:03:22 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-18 20:03:22 main:477] : INFO : Max Epochs: 2048 [2022-04-18 20:03:22 main:478] : INFO : Batch Size: 128 [2022-04-18 20:03:22 main:479] : INFO : Learning Rate: 0.01 [2022-04-18 20:03:22 main:480] : INFO : Patience: 10 [2022-04-18 20:03:22 main:481] : INFO : Hidden Width: 12 [2022-04-18 20:03:22 main:482] : INFO : # Recurrent Layers: 4 [2022-04-18 20:03:22 main:483] : INFO : # Backend Layers: 4 [2022-04-18 20:03:22 main:484] : INFO : # Threads: 1 [2022-04-18 20:03:22 main:486] : INFO : Preparing Dataset [2022-04-18 20:03:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-18 20:03:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-18 20:03:24 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-18 20:03:24 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-18 20:03:24 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-18 20:03:24 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-18 20:03:24 main:494] : INFO : Creating Model [2022-04-18 20:03:24 main:507] : INFO : Preparing config file [2022-04-18 20:03:24 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-18 20:03:24 main:512] : INFO : Loading config [2022-04-18 20:03:24 main:514] : INFO : Loading state [2022-04-18 20:03:26 main:559] : INFO : Loading DataLoader into Memory [2022-04-18 20:03:26 main:562] : INFO : Starting Training [2022-04-18 20:03:30 main:574] : INFO : Epoch 1950 | loss: 0.0307989 | val_loss: 0.0309678 | Time: 4429.24 ms [2022-04-18 20:03:34 main:574] : INFO : Epoch 1951 | loss: 0.0307239 | val_loss: 0.0309045 | Time: 4060.12 ms [2022-04-18 20:03:38 main:574] : INFO : Epoch 1952 | loss: 0.0307147 | val_loss: 0.0308855 | Time: 3965.78 ms [2022-04-18 20:03:42 main:574] : INFO : Epoch 1953 | loss: 0.030689 | val_loss: 0.0309128 | Time: 4084.55 ms [2022-04-18 20:03:46 main:574] : INFO : Epoch 1954 | loss: 0.0306734 | val_loss: 0.030926 | Time: 4143.79 ms [2022-04-18 20:03:50 main:574] : INFO : Epoch 1955 | loss: 0.0306775 | val_loss: 0.0309078 | Time: 4098.82 ms [2022-04-18 20:03:55 main:574] : INFO : Epoch 1956 | loss: 0.0306706 | val_loss: 0.0309081 | Time: 4436.76 ms [2022-04-18 20:03:59 main:574] : INFO : Epoch 1957 | loss: 0.0306708 | val_loss: 0.0308991 | Time: 4187.48 ms [2022-04-18 20:04:03 main:574] : INFO : Epoch 1958 | loss: 0.0306664 | val_loss: 0.0309232 | Time: 4346.11 ms [2022-04-18 20:04:07 main:574] : INFO : Epoch 1959 | loss: 0.0306707 | val_loss: 0.03093 | Time: 3942.91 ms [2022-04-18 20:04:11 main:574] : INFO : Epoch 1960 | loss: 0.030728 | val_loss: 0.0308947 | Time: 4003.73 ms [2022-04-18 20:04:16 main:574] : INFO : Epoch 1961 | loss: 0.0307132 | val_loss: 0.0308921 | Time: 4195.39 ms [2022-04-18 20:04:20 main:574] : INFO : Epoch 1962 | loss: 0.0307092 | val_loss: 0.0308902 | Time: 4024.64 ms [2022-04-18 20:04:24 main:574] : INFO : Epoch 1963 | loss: 0.0307013 | val_loss: 0.0308984 | Time: 4004.22 ms [2022-04-18 20:04:28 main:574] : INFO : Epoch 1964 | loss: 0.0306762 | val_loss: 0.0309078 | Time: 4082.1 ms [2022-04-18 20:04:32 main:574] : INFO : Epoch 1965 | loss: 0.0306832 | val_loss: 0.0309002 | Time: 3942.08 ms [2022-04-18 20:04:36 main:574] : INFO : Epoch 1966 | loss: 0.0306641 | val_loss: 0.0308954 | Time: 3974.02 ms [2022-04-18 20:04:40 main:574] : INFO : Epoch 1967 | loss: 0.030662 | val_loss: 0.0308888 | Time: 4177.65 ms [2022-04-18 20:04:44 main:574] : INFO : Epoch 1968 | loss: 0.0306651 | val_loss: 0.0308802 | Time: 4152.3 ms [2022-04-18 20:04:48 main:574] : INFO : Epoch 1969 | loss: 0.0306705 | val_loss: 0.0308787 | Time: 4097.91 ms [2022-04-18 20:04:52 main:574] : INFO : Epoch 1970 | loss: 0.0307001 | val_loss: 0.0310246 | Time: 4099.57 ms [2022-04-18 20:04:56 main:574] : INFO : Epoch 1971 | loss: 0.0307527 | val_loss: 0.0308768 | Time: 4174.51 ms [2022-04-18 20:05:01 main:574] : INFO : Epoch 1972 | loss: 0.03072 | val_loss: 0.0309056 | Time: 4178.77 ms [2022-04-18 20:05:05 main:574] : INFO : Epoch 1973 | loss: 0.0307422 | val_loss: 0.0309443 | Time: 4072.69 ms [2022-04-18 20:05:09 main:574] : INFO : Epoch 1974 | loss: 0.0307778 | val_loss: 0.0308986 | Time: 3889.65 ms [2022-04-18 20:05:13 main:574] : INFO : Epoch 1975 | loss: 0.0307273 | val_loss: 0.0309202 | Time: 3959.89 ms [2022-04-18 20:05:17 main:574] : INFO : Epoch 1976 | loss: 0.030717 | val_loss: 0.0309106 | Time: 4347.93 ms [2022-04-18 20:05:21 main:574] : INFO : Epoch 1977 | loss: 0.0306931 | val_loss: 0.0308853 | Time: 3975.03 ms [2022-04-18 20:05:25 main:574] : INFO : Epoch 1978 | loss: 0.0306866 | val_loss: 0.030902 | Time: 3952.18 ms [2022-04-18 20:05:29 main:574] : INFO : Epoch 1979 | loss: 0.0306645 | val_loss: 0.0308914 | Time: 4248.86 ms [2022-04-18 20:05:33 main:574] : INFO : Epoch 1980 | loss: 0.0306579 | val_loss: 0.0308883 | Time: 3945.74 ms [2022-04-18 20:05:37 main:574] : INFO : Epoch 1981 | loss: 0.0306667 | val_loss: 0.0308893 | Time: 4151.69 ms [2022-04-18 20:05:41 main:574] : INFO : Epoch 1982 | loss: 0.0306581 | val_loss: 0.0309182 | Time: 4035.98 ms [2022-04-18 20:05:45 main:574] : INFO : Epoch 1983 | loss: 0.0306691 | val_loss: 0.0308963 | Time: 3933.72 ms [2022-04-18 20:05:49 main:574] : INFO : Epoch 1984 | loss: 0.0306693 | val_loss: 0.0308792 | Time: 3937.58 ms [2022-04-18 20:05:54 main:574] : INFO : Epoch 1985 | loss: 0.0306702 | val_loss: 0.0308781 | Time: 4187.73 ms [2022-04-18 20:05:58 main:574] : INFO : Epoch 1986 | loss: 0.0306769 | val_loss: 0.0308993 | Time: 3960.67 ms [2022-04-18 20:06:01 main:574] : INFO : Epoch 1987 | loss: 0.0306695 | val_loss: 0.0308877 | Time: 3926.85 ms [2022-04-18 20:06:06 main:574] : INFO : Epoch 1988 | loss: 0.0306723 | val_loss: 0.0308758 | Time: 4161.12 ms [2022-04-18 20:06:10 main:574] : INFO : Epoch 1989 | loss: 0.0306901 | val_loss: 0.0309048 | Time: 3996.88 ms [2022-04-18 20:06:14 main:574] : INFO : Epoch 1990 | loss: 0.0307038 | val_loss: 0.0308764 | Time: 4051.29 ms [2022-04-18 20:06:18 main:574] : INFO : Epoch 1991 | loss: 0.0306821 | val_loss: 0.0309047 | Time: 4155.37 ms [2022-04-18 20:06:22 main:574] : INFO : Epoch 1992 | loss: 0.0306697 | val_loss: 0.0309016 | Time: 3912.4 ms [2022-04-18 20:06:26 main:574] : INFO : Epoch 1993 | loss: 0.0306883 | val_loss: 0.0309122 | Time: 3884.67 ms [2022-04-18 20:06:30 main:574] : INFO : Epoch 1994 | loss: 0.0306899 | val_loss: 0.0309112 | Time: 3914.67 ms [2022-04-18 20:06:34 main:574] : INFO : Epoch 1995 | loss: 0.0306971 | val_loss: 0.0308731 | Time: 3942.5 ms [2022-04-18 20:06:38 main:574] : INFO : Epoch 1996 | loss: 0.0307338 | val_loss: 0.0309212 | Time: 3945.05 ms [2022-04-18 20:06:42 main:574] : INFO : Epoch 1997 | loss: 0.030776 | val_loss: 0.0308686 | Time: 3940.86 ms [2022-04-18 20:06:46 main:574] : INFO : Epoch 1998 | loss: 0.030757 | val_loss: 0.0308857 | Time: 3979.24 ms [2022-04-18 20:06:50 main:574] : INFO : Epoch 1999 | loss: 0.030718 | val_loss: 0.0308409 | Time: 3955.6 ms [2022-04-18 20:06:54 main:574] : INFO : Epoch 2000 | loss: 0.0306877 | val_loss: 0.0308975 | Time: 3994.94 ms [2022-04-18 20:06:58 main:574] : INFO : Epoch 2001 | loss: 0.0306691 | val_loss: 0.0308856 | Time: 3960.95 ms [2022-04-18 20:07:02 main:574] : INFO : Epoch 2002 | loss: 0.0307033 | val_loss: 0.0308527 | Time: 3943.45 ms [2022-04-18 20:07:06 main:574] : INFO : Epoch 2003 | loss: 0.0306903 | val_loss: 0.0309232 | Time: 3989.2 ms [2022-04-18 20:07:10 main:574] : INFO : Epoch 2004 | loss: 0.0306813 | val_loss: 0.0308681 | Time: 4214.56 ms [2022-04-18 20:07:14 main:574] : INFO : Epoch 2005 | loss: 0.0306893 | val_loss: 0.0309032 | Time: 3972.97 ms [2022-04-18 20:07:18 main:574] : INFO : Epoch 2006 | loss: 0.0306998 | val_loss: 0.0309043 | Time: 3947.77 ms [2022-04-18 20:07:22 main:574] : INFO : Epoch 2007 | loss: 0.0306793 | val_loss: 0.0308545 | Time: 3963.19 ms [2022-04-18 20:07:26 main:574] : INFO : Epoch 2008 | loss: 0.0306758 | val_loss: 0.0308915 | Time: 3882.99 ms [2022-04-18 20:07:30 main:574] : INFO : Epoch 2009 | loss: 0.0306774 | val_loss: 0.0308832 | Time: 3928.55 ms [2022-04-18 20:07:34 main:574] : INFO : Epoch 2010 | loss: 0.0306635 | val_loss: 0.0308781 | Time: 3922.71 ms [2022-04-18 20:07:38 main:574] : INFO : Epoch 2011 | loss: 0.030665 | val_loss: 0.030916 | Time: 3953.59 ms [2022-04-18 20:07:42 main:574] : INFO : Epoch 2012 | loss: 0.0306667 | val_loss: 0.030915 | Time: 4033.15 ms [2022-04-18 20:07:46 main:574] : INFO : Epoch 2013 | loss: 0.0306797 | val_loss: 0.0308844 | Time: 3906.86 ms [2022-04-18 20:07:49 main:574] : INFO : Epoch 2014 | loss: 0.0306759 | val_loss: 0.0309117 | Time: 3893.21 ms [2022-04-18 20:07:54 main:574] : INFO : Epoch 2015 | loss: 0.0306508 | val_loss: 0.0309066 | Time: 4290.47 ms [2022-04-18 20:07:58 main:574] : INFO : Epoch 2016 | loss: 0.0306492 | val_loss: 0.0308975 | Time: 4030.08 ms [2022-04-18 20:08:02 main:574] : INFO : Epoch 2017 | loss: 0.0306438 | val_loss: 0.0308673 | Time: 3902.52 ms [2022-04-18 20:08:06 main:574] : INFO : Epoch 2018 | loss: 0.0306561 | val_loss: 0.0309542 | Time: 4162.16 ms [2022-04-18 20:08:10 main:574] : INFO : Epoch 2019 | loss: 0.0306619 | val_loss: 0.0308751 | Time: 3950.88 ms [2022-04-18 20:08:14 main:574] : INFO : Epoch 2020 | loss: 0.0307077 | val_loss: 0.0308513 | Time: 4000.42 ms [2022-04-18 20:08:18 main:574] : INFO : Epoch 2021 | loss: 0.0306876 | val_loss: 0.0308775 | Time: 3920.95 ms [2022-04-18 20:08:22 main:574] : INFO : Epoch 2022 | loss: 0.0307354 | val_loss: 0.0309568 | Time: 4183.5 ms [2022-04-18 20:08:26 main:574] : INFO : Epoch 2023 | loss: 0.0307114 | val_loss: 0.0308684 | Time: 3943.62 ms [2022-04-18 20:08:30 main:574] : INFO : Epoch 2024 | loss: 0.0307155 | val_loss: 0.0308727 | Time: 3870.23 ms [2022-04-18 20:08:34 main:574] : INFO : Epoch 2025 | loss: 0.0306788 | val_loss: 0.0308623 | Time: 3936.76 ms [2022-04-18 20:08:38 main:574] : INFO : Epoch 2026 | loss: 0.0306727 | val_loss: 0.0309417 | Time: 4107.48 ms [2022-04-18 20:08:42 main:574] : INFO : Epoch 2027 | loss: 0.030746 | val_loss: 0.0308726 | Time: 4112.84 ms [2022-04-18 20:08:46 main:574] : INFO : Epoch 2028 | loss: 0.0307389 | val_loss: 0.0308716 | Time: 3915.47 ms [2022-04-18 20:08:50 main:574] : INFO : Epoch 2029 | loss: 0.0307133 | val_loss: 0.0308826 | Time: 3938.45 ms [2022-04-18 20:08:54 main:574] : INFO : Epoch 2030 | loss: 0.0306871 | val_loss: 0.030903 | Time: 3999.84 ms [2022-04-18 20:08:58 main:574] : INFO : Epoch 2031 | loss: 0.0306825 | val_loss: 0.0308705 | Time: 4209.7 ms [2022-04-18 20:09:02 main:574] : INFO : Epoch 2032 | loss: 0.0306772 | val_loss: 0.0309217 | Time: 4065.91 ms [2022-04-18 20:09:06 main:574] : INFO : Epoch 2033 | loss: 0.0307022 | val_loss: 0.0309113 | Time: 3914.37 ms [2022-04-18 20:09:10 main:574] : INFO : Epoch 2034 | loss: 0.0306938 | val_loss: 0.0308537 | Time: 3948.49 ms [2022-04-18 20:09:14 main:574] : INFO : Epoch 2035 | loss: 0.0306692 | val_loss: 0.030884 | Time: 3993.1 ms [2022-04-18 20:09:18 main:574] : INFO : Epoch 2036 | loss: 0.0306705 | val_loss: 0.0308567 | Time: 3884.2 ms [2022-04-18 20:09:22 main:574] : INFO : Epoch 2037 | loss: 0.0306948 | val_loss: 0.030917 | Time: 4151.65 ms [2022-04-18 20:09:26 main:574] : INFO : Epoch 2038 | loss: 0.0306872 | val_loss: 0.0308718 | Time: 3955.68 ms [2022-04-18 20:09:30 main:574] : INFO : Epoch 2039 | loss: 0.0306661 | val_loss: 0.0308818 | Time: 3921.74 ms [2022-04-18 20:09:34 main:574] : INFO : Epoch 2040 | loss: 0.0306811 | val_loss: 0.0309065 | Time: 3886.25 ms [2022-04-18 20:09:38 main:574] : INFO : Epoch 2041 | loss: 0.0306824 | val_loss: 0.0308756 | Time: 3897.04 ms [2022-04-18 20:09:43 main:574] : INFO : Epoch 2042 | loss: 0.0306797 | val_loss: 0.0308862 | Time: 4289.73 ms [2022-04-18 20:09:46 main:574] : INFO : Epoch 2043 | loss: 0.0306769 | val_loss: 0.0308653 | Time: 3937.83 ms [2022-04-18 20:09:50 main:574] : INFO : Epoch 2044 | loss: 0.0306681 | val_loss: 0.0309144 | Time: 3969.83 ms [2022-04-18 20:09:54 main:574] : INFO : Epoch 2045 | loss: 0.0306573 | val_loss: 0.0309083 | Time: 3963.07 ms [2022-04-18 20:09:59 main:574] : INFO : Epoch 2046 | loss: 0.030644 | val_loss: 0.0308896 | Time: 4171.24 ms [2022-04-18 20:10:03 main:574] : INFO : Epoch 2047 | loss: 0.0306686 | val_loss: 0.030915 | Time: 4252.56 ms [2022-04-18 20:10:07 main:574] : INFO : Epoch 2048 | loss: 0.0306899 | val_loss: 0.0308851 | Time: 4086.69 ms [2022-04-18 20:10:07 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0308851 [2022-04-18 20:10:07 main:603] : INFO : Saving end state to config to file [2022-04-18 20:10:07 main:608] : INFO : Success, exiting.. 20:10:07 (26180): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)