Task 13738469
| Name | ParityModified-1647046462-1547-2-0_0 |
| Workunit | 10826366 |
| Created | 25 Mar 2022, 5:03:23 UTC |
| Sent | 31 Mar 2022, 16:41:46 UTC |
| Report deadline | 8 Apr 2022, 16:41:46 UTC |
| Received | 15 Apr 2022, 16:26:07 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 10282 |
| Run time | 4 hours 39 min 26 sec |
| CPU time | 4 hours 37 min 10 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 5,441.11 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.10 GB |
| Peak swap size | 4.34 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> loss: 0.0312059 | Time: 9222.2 ms [2022-04-12 12:32:48 main:574] : INFO : Epoch 1704 | loss: 0.0311929 | val_loss: 0.0312038 | Time: 9254.56 ms [2022-04-12 12:32:57 main:574] : INFO : Epoch 1705 | loss: 0.0311934 | val_loss: 0.0312041 | Time: 9424.75 ms [2022-04-12 12:33:07 main:574] : INFO : Epoch 1706 | loss: 0.0311939 | val_loss: 0.0312067 | Time: 9369 ms [2022-04-12 12:33:16 main:574] : INFO : Epoch 1707 | loss: 0.0311934 | val_loss: 0.0312041 | Time: 8933.2 ms [2022-04-12 12:33:25 main:574] : INFO : Epoch 1708 | loss: 0.0311933 | val_loss: 0.031204 | Time: 8997.78 ms [2022-04-12 12:33:34 main:574] : INFO : Epoch 1709 | loss: 0.0311935 | val_loss: 0.0312034 | Time: 9241.58 ms [2022-04-12 12:33:43 main:574] : INFO : Epoch 1710 | loss: 0.0311935 | val_loss: 0.0312038 | Time: 9412.55 ms [2022-04-12 12:33:52 main:574] : INFO : Epoch 1711 | loss: 0.031193 | val_loss: 0.0312058 | Time: 8487.55 ms [2022-04-12 12:34:01 main:574] : INFO : Epoch 1712 | loss: 0.0311945 | val_loss: 0.0312088 | Time: 9403.38 ms [2022-04-12 12:34:10 main:574] : INFO : Epoch 1713 | loss: 0.031195 | val_loss: 0.0312049 | Time: 9225.57 ms [2022-04-12 12:34:19 main:574] : INFO : Epoch 1714 | loss: 0.0311929 | val_loss: 0.0312066 | Time: 8913.45 ms [2022-04-12 12:34:29 main:574] : INFO : Epoch 1715 | loss: 0.0311961 | val_loss: 0.0312071 | Time: 9413.1 ms [2022-04-12 12:34:37 main:574] : INFO : Epoch 1716 | loss: 0.0311942 | val_loss: 0.0312049 | Time: 8692.48 ms [2022-04-12 12:34:47 main:574] : INFO : Epoch 1717 | loss: 0.0311937 | val_loss: 0.031204 | Time: 9030.25 ms [2022-04-12 12:34:55 main:574] : INFO : Epoch 1718 | loss: 0.0311933 | val_loss: 0.0312038 | Time: 8301.79 ms [2022-04-12 12:35:05 main:574] : INFO : Epoch 1719 | loss: 0.031193 | val_loss: 0.0312033 | Time: 9818.6 ms [2022-04-12 12:35:13 main:574] : INFO : Epoch 1720 | loss: 0.0311929 | val_loss: 0.0312046 | Time: 8615.97 ms [2022-04-12 12:35:22 main:574] : INFO : Epoch 1721 | loss: 0.0311933 | val_loss: 0.0312034 | Time: 8434.97 ms [2022-04-12 12:35:30 main:574] : INFO : Epoch 1722 | loss: 0.0311923 | val_loss: 0.0312035 | Time: 8307.18 ms [2022-04-12 12:35:39 main:574] : INFO : Epoch 1723 | loss: 0.0311927 | val_loss: 0.0312033 | Time: 8910.59 ms [2022-04-12 12:35:48 main:574] : INFO : Epoch 1724 | loss: 0.0311928 | val_loss: 0.0312054 | Time: 9170.53 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1660 Ti) [2022-04-13 12:20:08 main:435] : INFO : Set logging level to 1 [2022-04-13 12:20:08 main:441] : INFO : Running in BOINC Client mode [2022-04-13 12:20:08 main:444] : INFO : Resolving all filenames [2022-04-13 12:20:08 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 12:20:08 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 12:20:08 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 12:20:08 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 12:20:08 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 12:20:08 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 12:20:08 main:474] : INFO : Configuration: [2022-04-13 12:20:08 main:475] : INFO : Model type: GRU [2022-04-13 12:20:08 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 12:20:08 main:477] : INFO : Max Epochs: 2048 [2022-04-13 12:20:08 main:478] : INFO : Batch Size: 128 [2022-04-13 12:20:08 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 12:20:08 main:480] : INFO : Patience: 10 [2022-04-13 12:20:08 main:481] : INFO : Hidden Width: 12 [2022-04-13 12:20:08 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 12:20:08 main:483] : INFO : # Backend Layers: 4 [2022-04-13 12:20:08 main:484] : INFO : # Threads: 1 [2022-04-13 12:20:08 main:486] : INFO : Preparing Dataset [2022-04-13 12:20:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 12:20:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 12:20:13 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 12:20:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 12:20:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 12:20:13 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 12:20:13 main:494] : INFO : Creating Model [2022-04-13 12:20:13 main:507] : INFO : Preparing config file [2022-04-13 12:20:13 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 12:20:13 main:512] : INFO : Loading config [2022-04-13 12:20:13 main:514] : INFO : Loading state [2022-04-13 12:20:17 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 12:20:17 main:562] : INFO : Starting Training [2022-04-13 12:20:26 main:574] : INFO : Epoch 1721 | loss: 0.0312303 | val_loss: 0.0312061 | Time: 8682.56 ms [2022-04-13 12:20:34 main:574] : INFO : Epoch 1722 | loss: 0.0312038 | val_loss: 0.031204 | Time: 8058.12 ms [2022-04-13 12:20:42 main:574] : INFO : Epoch 1723 | loss: 0.0311984 | val_loss: 0.0312017 | Time: 7792.92 ms [2022-04-13 12:20:49 main:574] : INFO : Epoch 1724 | loss: 0.0311946 | val_loss: 0.0312056 | Time: 7577.43 ms [2022-04-13 12:20:57 main:574] : INFO : Epoch 1725 | loss: 0.0311949 | val_loss: 0.0311978 | Time: 7580.62 ms [2022-04-13 12:21:05 main:574] : INFO : Epoch 1726 | loss: 0.0311925 | val_loss: 0.0311982 | Time: 7749.93 ms [2022-04-13 12:21:13 main:574] : INFO : Epoch 1727 | loss: 0.0311957 | val_loss: 0.0312026 | Time: 7985.31 ms [2022-04-13 12:21:20 main:574] : INFO : Epoch 1728 | loss: 0.0311981 | val_loss: 0.031203 | Time: 7784.53 ms [2022-04-13 12:21:28 main:574] : INFO : Epoch 1729 | loss: 0.031197 | val_loss: 0.0312 | Time: 7982.3 ms [2022-04-13 12:21:36 main:574] : INFO : Epoch 1730 | loss: 0.0311952 | val_loss: 0.0312042 | Time: 7975.68 ms [2022-04-13 12:21:44 main:574] : INFO : Epoch 1731 | loss: 0.0311973 | val_loss: 0.0312058 | Time: 7736.32 ms [2022-04-13 12:21:52 main:574] : INFO : Epoch 1732 | loss: 0.0311961 | val_loss: 0.0312028 | Time: 7783.53 ms [2022-04-13 12:22:00 main:574] : INFO : Epoch 1733 | loss: 0.0311944 | val_loss: 0.0312021 | Time: 8179.51 ms [2022-04-13 12:22:08 main:574] : INFO : Epoch 1734 | loss: 0.0311941 | val_loss: 0.0312019 | Time: 7901.12 ms [2022-04-13 12:22:16 main:574] : INFO : Epoch 1735 | loss: 0.0311948 | val_loss: 0.0312016 | Time: 7751.15 ms [2022-04-13 12:22:24 main:574] : INFO : Epoch 1736 | loss: 0.0311959 | val_loss: 0.0312053 | Time: 7935.26 ms [2022-04-13 12:22:31 main:574] : INFO : Epoch 1737 | loss: 0.0311967 | val_loss: 0.0312077 | Time: 7600.16 ms [2022-04-13 12:22:40 main:574] : INFO : Epoch 1738 | loss: 0.0311947 | val_loss: 0.0312028 | Time: 8247.07 ms [2022-04-13 12:22:48 main:574] : INFO : Epoch 1739 | loss: 0.0311935 | val_loss: 0.0312031 | Time: 8098.72 ms [2022-04-13 12:22:56 main:574] : INFO : Epoch 1740 | loss: 0.0311947 | val_loss: 0.0312063 | Time: 7919.54 ms [2022-04-13 12:23:04 main:574] : INFO : Epoch 1741 | loss: 0.0311959 | val_loss: 0.031204 | Time: 8004.65 ms [2022-04-13 12:23:11 main:574] : INFO : Epoch 1742 | loss: 0.0311972 | val_loss: 0.0312035 | Time: 7830.33 ms [2022-04-13 12:23:19 main:574] : INFO : Epoch 1743 | loss: 0.0312033 | val_loss: 0.0312072 | Time: 7585.69 ms [2022-04-13 12:23:27 main:574] : INFO : Epoch 1744 | loss: 0.0312124 | val_loss: 0.0312136 | Time: 7560.71 ms [2022-04-13 12:23:34 main:574] : INFO : Epoch 1745 | loss: 0.031214 | val_loss: 0.0312136 | Time: 7578.87 ms [2022-04-13 12:23:42 main:574] : INFO : Epoch 1746 | loss: 0.031212 | val_loss: 0.0312163 | Time: 7600.1 ms [2022-04-13 12:23:50 main:574] : INFO : Epoch 1747 | loss: 0.0312117 | val_loss: 0.0312141 | Time: 7899.21 ms [2022-04-13 12:23:57 main:574] : INFO : Epoch 1748 | loss: 0.0312108 | val_loss: 0.031212 | Time: 7678.39 ms [2022-04-13 12:24:06 main:574] : INFO : Epoch 1749 | loss: 0.0312097 | val_loss: 0.0312116 | Time: 8175.14 ms [2022-04-13 12:24:14 main:574] : INFO : Epoch 1750 | loss: 0.0312099 | val_loss: 0.0312114 | Time: 8293.23 ms [2022-04-13 12:24:22 main:574] : INFO : Epoch 1751 | loss: 0.0312101 | val_loss: 0.031213 | Time: 8242.17 ms [2022-04-13 12:24:30 main:574] : INFO : Epoch 1752 | loss: 0.0312082 | val_loss: 0.0312117 | Time: 8151.79 ms [2022-04-13 12:24:38 main:574] : INFO : Epoch 1753 | loss: 0.0312084 | val_loss: 0.0312118 | Time: 7567.78 ms [2022-04-13 12:24:45 main:574] : INFO : Epoch 1754 | loss: 0.0312085 | val_loss: 0.0312147 | Time: 7562.76 ms [2022-04-13 12:24:53 main:574] : INFO : Epoch 1755 | loss: 0.0312094 | val_loss: 0.0312108 | Time: 7992.58 ms [2022-04-13 12:25:01 main:574] : INFO : Epoch 1756 | loss: 0.031211 | val_loss: 0.0312119 | Time: 7549.85 ms [2022-04-13 12:25:09 main:574] : INFO : Epoch 1757 | loss: 0.0312088 | val_loss: 0.0312119 | Time: 7800.67 ms [2022-04-13 12:25:17 main:574] : INFO : Epoch 1758 | loss: 0.031208 | val_loss: 0.0312125 | Time: 7998.04 ms [2022-04-13 12:25:25 main:574] : INFO : Epoch 1759 | loss: 0.0312092 | val_loss: 0.0312114 | Time: 8081.51 ms [2022-04-13 12:25:33 main:574] : INFO : Epoch 1760 | loss: 0.0312091 | val_loss: 0.0312121 | Time: 8035.79 ms [2022-04-13 12:25:41 main:574] : INFO : Epoch 1761 | loss: 0.0312071 | val_loss: 0.0312087 | Time: 8010.45 ms [2022-04-13 12:25:49 main:574] : INFO : Epoch 1762 | loss: 0.0312062 | val_loss: 0.0312095 | Time: 8032.5 ms [2022-04-13 12:25:57 main:574] : INFO : Epoch 1763 | loss: 0.0312055 | val_loss: 0.0312082 | Time: 7783.64 ms [2022-04-13 12:26:05 main:574] : INFO : Epoch 1764 | loss: 0.0312048 | val_loss: 0.0312084 | Time: 8242.39 ms [2022-04-13 12:26:13 main:574] : INFO : Epoch 1765 | loss: 0.0312044 | val_loss: 0.0312094 | Time: 8137.13 ms [2022-04-13 12:26:21 main:574] : INFO : Epoch 1766 | loss: 0.0312049 | val_loss: 0.0312105 | Time: 7934.43 ms [2022-04-13 12:26:29 main:574] : INFO : Epoch 1767 | loss: 0.0312044 | val_loss: 0.0312081 | Time: 8040.44 ms [2022-04-13 12:26:37 main:574] : INFO : Epoch 1768 | loss: 0.0312041 | val_loss: 0.0312084 | Time: 8112.44 ms [2022-04-13 12:26:45 main:574] : INFO : Epoch 1769 | loss: 0.0312037 | val_loss: 0.0312079 | Time: 7743.95 ms [2022-04-13 12:26:53 main:574] : INFO : Epoch 1770 | loss: 0.031204 | val_loss: 0.0312084 | Time: 7599.04 ms [2022-04-13 12:27:01 main:574] : INFO : Epoch 1771 | loss: 0.0312037 | val_loss: 0.0312075 | Time: 8058.39 ms [2022-04-13 12:27:09 main:574] : INFO : Epoch 1772 | loss: 0.0312037 | val_loss: 0.0312081 | Time: 8075.8 ms [2022-04-13 12:27:17 main:574] : INFO : Epoch 1773 | loss: 0.0312032 | val_loss: 0.0312084 | Time: 8254.26 ms [2022-04-13 12:27:25 main:574] : INFO : Epoch 1774 | loss: 0.0312034 | val_loss: 0.0312101 | Time: 7518.8 ms [2022-04-13 12:27:32 main:574] : INFO : Epoch 1775 | loss: 0.0312048 | val_loss: 0.0312092 | Time: 7740.12 ms [2022-04-13 12:27:40 main:574] : INFO : Epoch 1776 | loss: 0.0312036 | val_loss: 0.0312093 | Time: 7813.38 ms [2022-04-13 12:27:48 main:574] : INFO : Epoch 1777 | loss: 0.0312053 | val_loss: 0.0312098 | Time: 7793.03 ms [2022-04-13 12:27:56 main:574] : INFO : Epoch 1778 | loss: 0.0312038 | val_loss: 0.0312088 | Time: 8123.43 ms [2022-04-13 12:28:04 main:574] : INFO : Epoch 1779 | loss: 0.0312092 | val_loss: 0.0312142 | Time: 8040.63 ms [2022-04-13 12:28:12 main:574] : INFO : Epoch 1780 | loss: 0.0312047 | val_loss: 0.0312088 | Time: 7940.46 ms [2022-04-13 12:28:20 main:574] : INFO : Epoch 1781 | loss: 0.0312032 | val_loss: 0.0312084 | Time: 7653.53 ms [2022-04-13 12:28:28 main:574] : INFO : Epoch 1782 | loss: 0.0312033 | val_loss: 0.0312128 | Time: 7870.45 ms [2022-04-13 12:28:35 main:574] : INFO : Epoch 1783 | loss: 0.0312049 | val_loss: 0.0312104 | Time: 7895.87 ms [2022-04-13 12:28:43 main:574] : INFO : Epoch 1784 | loss: 0.0312042 | val_loss: 0.031208 | Time: 7981.69 ms [2022-04-13 12:28:51 main:574] : INFO : Epoch 1785 | loss: 0.031203 | val_loss: 0.0312106 | Time: 7756.48 ms [2022-04-13 12:28:59 main:574] : INFO : Epoch 1786 | loss: 0.0312033 | val_loss: 0.0312106 | Time: 7894.42 ms [2022-04-13 12:29:07 main:574] : INFO : Epoch 1787 | loss: 0.0312036 | val_loss: 0.0312084 | Time: 7799.04 ms [2022-04-13 12:29:15 main:574] : INFO : Epoch 1788 | loss: 0.0312027 | val_loss: 0.0312092 | Time: 7931.99 ms [2022-04-13 12:29:23 main:574] : INFO : Epoch 1789 | loss: 0.031205 | val_loss: 0.0312138 | Time: 7819.87 ms [2022-04-13 12:29:30 main:574] : INFO : Epoch 1790 | loss: 0.0312032 | val_loss: 0.0312082 | Time: 7679.64 ms [2022-04-13 12:29:38 main:574] : INFO : Epoch 1791 | loss: 0.0312024 | val_loss: 0.0312101 | Time: 7795.71 ms [2022-04-13 12:29:46 main:574] : INFO : Epoch 1792 | loss: 0.0312049 | val_loss: 0.0312098 | Time: 7672.34 ms [2022-04-13 12:29:53 main:574] : INFO : Epoch 1793 | loss: 0.0312024 | val_loss: 0.0312082 | Time: 7610.51 ms [2022-04-13 12:30:02 main:574] : INFO : Epoch 1794 | loss: 0.0312021 | val_loss: 0.0312077 | Time: 8119.47 ms [2022-04-13 12:30:09 main:574] : INFO : Epoch 1795 | loss: 0.031202 | val_loss: 0.0312084 | Time: 7501.08 ms [2022-04-13 12:30:17 main:574] : INFO : Epoch 1796 | loss: 0.0312023 | val_loss: 0.0312078 | Time: 7551.18 ms [2022-04-13 12:30:25 main:574] : INFO : Epoch 1797 | loss: 0.0312013 | val_loss: 0.0312089 | Time: 8003.44 ms [2022-04-13 12:30:32 main:574] : INFO : Epoch 1798 | loss: 0.0312022 | val_loss: 0.0312075 | Time: 7588.48 ms [2022-04-13 12:30:40 main:574] : INFO : Epoch 1799 | loss: 0.0312034 | val_loss: 0.0312142 | Time: 7661.95 ms [2022-04-13 12:30:48 main:574] : INFO : Epoch 1800 | loss: 0.031203 | val_loss: 0.031208 | Time: 7879.36 ms [2022-04-13 12:30:56 main:574] : INFO : Epoch 1801 | loss: 0.0312016 | val_loss: 0.031208 | Time: 7809.05 ms [2022-04-13 12:31:03 main:574] : INFO : Epoch 1802 | loss: 0.0312025 | val_loss: 0.0312168 | Time: 7736.47 ms [2022-04-13 12:31:11 main:574] : INFO : Epoch 1803 | loss: 0.0312042 | val_loss: 0.0312098 | Time: 7911.35 ms [2022-04-13 12:31:19 main:574] : INFO : Epoch 1804 | loss: 0.0312046 | val_loss: 0.0312099 | Time: 7654.76 ms [2022-04-13 12:31:27 main:574] : INFO : Epoch 1805 | loss: 0.0312019 | val_loss: 0.031211 | Time: 7909.79 ms [2022-04-13 12:31:35 main:574] : INFO : Epoch 1806 | loss: 0.0312028 | val_loss: 0.0312074 | Time: 8043.67 ms [2022-04-13 12:31:42 main:574] : INFO : Epoch 1807 | loss: 0.0312027 | val_loss: 0.0312079 | Time: 7495.12 ms [2022-04-13 12:31:50 main:574] : INFO : Epoch 1808 | loss: 0.031202 | val_loss: 0.0312082 | Time: 7773.5 ms [2022-04-13 12:31:58 main:574] : INFO : Epoch 1809 | loss: 0.0312014 | val_loss: 0.0312104 | Time: 8013.06 ms [2022-04-13 12:32:06 main:574] : INFO : Epoch 1810 | loss: 0.0312012 | val_loss: 0.0312089 | Time: 7893.15 ms [2022-04-13 12:32:14 main:574] : INFO : Epoch 1811 | loss: 0.0312007 | val_loss: 0.0312084 | Time: 7908.25 ms [2022-04-13 12:32:22 main:574] : INFO : Epoch 1812 | loss: 0.0312023 | val_loss: 0.0312088 | Time: 7913.08 ms [2022-04-13 12:32:30 main:574] : INFO : Epoch 1813 | loss: 0.0312028 | val_loss: 0.0312132 | Time: 7816.66 ms [2022-04-13 12:32:38 main:574] : INFO : Epoch 1814 | loss: 0.0312026 | val_loss: 0.0312113 | Time: 7881.01 ms [2022-04-13 12:32:45 main:574] : INFO : Epoch 1815 | loss: 0.0312019 | val_loss: 0.0312086 | Time: 7830.76 ms [2022-04-13 12:32:53 main:574] : INFO : Epoch 1816 | loss: 0.0311998 | val_loss: 0.0312089 | Time: 7478.87 ms [2022-04-13 12:33:01 main:574] : INFO : Epoch 1817 | loss: 0.0312002 | val_loss: 0.03121 | Time: 8059.01 ms [2022-04-13 12:33:09 main:574] : INFO : Epoch 1818 | loss: 0.0312001 | val_loss: 0.0312101 | Time: 7890.29 ms [2022-04-13 12:33:17 main:574] : INFO : Epoch 1819 | loss: 0.0312006 | val_loss: 0.0312087 | Time: 8008.12 ms [2022-04-13 12:33:25 main:574] : INFO : Epoch 1820 | loss: 0.0312006 | val_loss: 0.0312077 | Time: 7941.12 ms [2022-04-13 12:33:33 main:574] : INFO : Epoch 1821 | loss: 0.0312001 | val_loss: 0.0312085 | Time: 7973.3 ms [2022-04-13 12:33:41 main:574] : INFO : Epoch 1822 | loss: 0.0312 | val_loss: 0.0312099 | Time: 7752.02 ms [2022-04-13 12:33:48 main:574] : INFO : Epoch 1823 | loss: 0.0312002 | val_loss: 0.0312071 | Time: 7704.86 ms [2022-04-13 12:33:56 main:574] : INFO : Epoch 1824 | loss: 0.0312003 | val_loss: 0.0312102 | Time: 8009.63 ms [2022-04-13 12:34:04 main:574] : INFO : Epoch 1825 | loss: 0.031206 | val_loss: 0.0312112 | Time: 8035.45 ms [2022-04-13 12:34:22 main:574] : INFO : Epoch 1826 | loss: 0.0312052 | val_loss: 0.0312075 | Time: 17865.2 ms [2022-04-13 12:34:30 main:574] : INFO : Epoch 1827 | loss: 0.0312047 | val_loss: 0.0312086 | Time: 7416.36 ms [2022-04-13 12:34:37 main:574] : INFO : Epoch 1828 | loss: 0.0312015 | val_loss: 0.0312093 | Time: 7471.54 ms [2022-04-13 12:34:45 main:574] : INFO : Epoch 1829 | loss: 0.0312011 | val_loss: 0.0312067 | Time: 7961.43 ms [2022-04-13 12:34:53 main:574] : INFO : Epoch 1830 | loss: 0.0312035 | val_loss: 0.0312079 | Time: 7534.43 ms [2022-04-13 12:35:00 main:574] : INFO : Epoch 1831 | loss: 0.0312034 | val_loss: 0.03121 | Time: 7569 ms [2022-04-13 12:35:07 main:574] : INFO : Epoch 1832 | loss: 0.031202 | val_loss: 0.0312103 | Time: 7255.27 ms [2022-04-13 12:35:15 main:574] : INFO : Epoch 1833 | loss: 0.0312012 | val_loss: 0.0312078 | Time: 7222.97 ms [2022-04-13 12:35:22 main:574] : INFO : Epoch 1834 | loss: 0.0312005 | val_loss: 0.0312079 | Time: 7813.61 ms [2022-04-13 12:35:30 main:574] : INFO : Epoch 1835 | loss: 0.0311993 | val_loss: 0.0312107 | Time: 7302.47 ms [2022-04-13 12:35:38 main:574] : INFO : Epoch 1836 | loss: 0.0311998 | val_loss: 0.0312071 | Time: 7744.99 ms [2022-04-13 12:35:45 main:574] : INFO : Epoch 1837 | loss: 0.0312034 | val_loss: 0.0312119 | Time: 7835.89 ms [2022-04-13 12:35:53 main:574] : INFO : Epoch 1838 | loss: 0.0312029 | val_loss: 0.0312081 | Time: 7447.9 ms [2022-04-13 12:36:00 main:574] : INFO : Epoch 1839 | loss: 0.0312016 | val_loss: 0.0312094 | Time: 7633.93 ms [2022-04-13 12:36:08 main:574] : INFO : Epoch 1840 | loss: 0.0312007 | val_loss: 0.0312107 | Time: 7383.9 ms [2022-04-13 12:36:16 main:574] : INFO : Epoch 1841 | loss: 0.0312007 | val_loss: 0.0312088 | Time: 7730.1 ms [2022-04-13 12:36:23 main:574] : INFO : Epoch 1842 | loss: 0.0312 | val_loss: 0.0312086 | Time: 7262.99 ms [2022-04-13 12:36:31 main:574] : INFO : Epoch 1843 | loss: 0.0311997 | val_loss: 0.0312085 | Time: 7676.29 ms [2022-04-13 12:36:38 main:574] : INFO : Epoch 1844 | loss: 0.0312005 | val_loss: 0.0312084 | Time: 7217.79 ms [2022-04-13 12:36:45 main:574] : INFO : Epoch 1845 | loss: 0.0312002 | val_loss: 0.0312092 | Time: 7237.59 ms [2022-04-13 12:36:53 main:574] : INFO : Epoch 1846 | loss: 0.0312003 | val_loss: 0.0312095 | Time: 7851.97 ms [2022-04-13 12:37:00 main:574] : INFO : Epoch 1847 | loss: 0.0311999 | val_loss: 0.0312115 | Time: 7237.04 ms [2022-04-13 12:37:08 main:574] : INFO : Epoch 1848 | loss: 0.031201 | val_loss: 0.0312087 | Time: 7729.65 ms [2022-04-13 12:37:15 main:574] : INFO : Epoch 1849 | loss: 0.0311999 | val_loss: 0.0312092 | Time: 7198.32 ms [2022-04-13 12:37:23 main:574] : INFO : Epoch 1850 | loss: 0.0311999 | val_loss: 0.0312083 | Time: 7757.43 ms [2022-04-13 12:37:30 main:574] : INFO : Epoch 1851 | loss: 0.0311994 | val_loss: 0.0312096 | Time: 7486.9 ms [2022-04-13 12:37:38 main:574] : INFO : Epoch 1852 | loss: 0.0312006 | val_loss: 0.0312081 | Time: 7252.95 ms [2022-04-13 12:37:45 main:574] : INFO : Epoch 1853 | loss: 0.0311998 | val_loss: 0.0312093 | Time: 7734.55 ms [2022-04-13 12:37:53 main:574] : INFO : Epoch 1854 | loss: 0.0311993 | val_loss: 0.0312097 | Time: 7610.92 ms [2022-04-13 12:38:00 main:574] : INFO : Epoch 1855 | loss: 0.0312007 | val_loss: 0.031209 | Time: 7247.46 ms [2022-04-13 12:38:08 main:574] : INFO : Epoch 1856 | loss: 0.0312012 | val_loss: 0.0312101 | Time: 7568.81 ms [2022-04-13 12:38:15 main:574] : INFO : Epoch 1857 | loss: 0.0312016 | val_loss: 0.0312091 | Time: 7650.01 ms [2022-04-13 12:38:23 main:574] : INFO : Epoch 1858 | loss: 0.0312008 | val_loss: 0.0312115 | Time: 7218.79 ms [2022-04-13 12:38:30 main:574] : INFO : Epoch 1859 | loss: 0.0312026 | val_loss: 0.0312087 | Time: 7285.58 ms [2022-04-13 12:38:37 main:574] : INFO : Epoch 1860 | loss: 0.0312003 | val_loss: 0.0312084 | Time: 7513.68 ms [2022-04-13 12:38:45 main:574] : INFO : Epoch 1861 | loss: 0.0311993 | val_loss: 0.0312107 | Time: 7892.4 ms [2022-04-13 12:38:53 main:574] : INFO : Epoch 1862 | loss: 0.0311992 | val_loss: 0.0312094 | Time: 7550.98 ms [2022-04-13 12:39:00 main:574] : INFO : Epoch 1863 | loss: 0.0311995 | val_loss: 0.0312091 | Time: 7446.37 ms [2022-04-13 12:39:08 main:574] : INFO : Epoch 1864 | loss: 0.0311992 | val_loss: 0.0312096 | Time: 7438.05 ms [2022-04-13 12:39:15 main:574] : INFO : Epoch 1865 | loss: 0.0311996 | val_loss: 0.031209 | Time: 7303.86 ms [2022-04-13 12:39:23 main:574] : INFO : Epoch 1866 | loss: 0.031199 | val_loss: 0.0312079 | Time: 7807.28 ms [2022-04-13 12:39:31 main:574] : INFO : Epoch 1867 | loss: 0.0311994 | val_loss: 0.0312101 | Time: 7791.75 ms [2022-04-13 12:39:38 main:574] : INFO : Epoch 1868 | loss: 0.0311998 | val_loss: 0.0312088 | Time: 7336.04 ms [2022-04-13 12:39:46 main:574] : INFO : Epoch 1869 | loss: 0.0311988 | val_loss: 0.03121 | Time: 7578.01 ms [2022-04-13 12:39:54 main:574] : INFO : Epoch 1870 | loss: 0.0311998 | val_loss: 0.0312114 | Time: 8406.82 ms [2022-04-13 12:40:03 main:574] : INFO : Epoch 1871 | loss: 0.0311998 | val_loss: 0.0312099 | Time: 8565 ms [2022-04-13 12:40:11 main:574] : INFO : Epoch 1872 | loss: 0.0311992 | val_loss: 0.031209 | Time: 8652.51 ms [2022-04-13 12:40:19 main:574] : INFO : Epoch 1873 | loss: 0.0311994 | val_loss: 0.0312106 | Time: 8071.64 ms [2022-04-13 12:40:28 main:574] : INFO : Epoch 1874 | loss: 0.0311997 | val_loss: 0.0312113 | Time: 8221 ms [2022-04-13 12:40:36 main:574] : INFO : Epoch 1875 | loss: 0.0311997 | val_loss: 0.0312094 | Time: 8056.72 ms [2022-04-13 12:40:44 main:574] : INFO : Epoch 1876 | loss: 0.0311993 | val_loss: 0.0312094 | Time: 7977.84 ms [2022-04-13 12:40:51 main:574] : INFO : Epoch 1877 | loss: 0.0311988 | val_loss: 0.0312092 | Time: 7297.38 ms [2022-04-13 12:40:58 main:574] : INFO : Epoch 1878 | loss: 0.0311988 | val_loss: 0.0312133 | Time: 7276.76 ms [2022-04-13 12:41:07 main:574] : INFO : Epoch 1879 | loss: 0.0312013 | val_loss: 0.0312103 | Time: 8475.52 ms [2022-04-13 12:41:15 main:574] : INFO : Epoch 1880 | loss: 0.0311997 | val_loss: 0.0312107 | Time: 8053.44 ms [2022-04-13 12:41:23 main:574] : INFO : Epoch 1881 | loss: 0.0312 | val_loss: 0.0312088 | Time: 8301.93 ms [2022-04-13 12:41:32 main:574] : INFO : Epoch 1882 | loss: 0.0311987 | val_loss: 0.0312094 | Time: 8566.12 ms [2022-04-13 12:41:40 main:574] : INFO : Epoch 1883 | loss: 0.0311995 | val_loss: 0.0312098 | Time: 8592.81 ms [2022-04-13 12:41:48 main:574] : INFO : Epoch 1884 | loss: 0.0311981 | val_loss: 0.0312092 | Time: 8281.41 ms [2022-04-13 12:41:56 main:574] : INFO : Epoch 1885 | loss: 0.0311991 | val_loss: 0.0312098 | Time: 7985.44 ms [2022-04-13 12:42:04 main:574] : INFO : Epoch 1886 | loss: 0.0311985 | val_loss: 0.0312097 | Time: 8036.9 ms [2022-04-13 12:42:12 main:574] : INFO : Epoch 1887 | loss: 0.0311994 | val_loss: 0.0312085 | Time: 7861.22 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1660 Ti) [2022-04-13 22:44:25 main:435] : INFO : Set logging level to 1 [2022-04-13 22:44:25 main:441] : INFO : Running in BOINC Client mode [2022-04-13 22:44:25 main:444] : INFO : Resolving all filenames [2022-04-13 22:44:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-13 22:44:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-13 22:44:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-13 22:44:25 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-13 22:44:25 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-13 22:44:25 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-13 22:44:26 main:474] : INFO : Configuration: [2022-04-13 22:44:26 main:475] : INFO : Model type: GRU [2022-04-13 22:44:26 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-13 22:44:26 main:477] : INFO : Max Epochs: 2048 [2022-04-13 22:44:26 main:478] : INFO : Batch Size: 128 [2022-04-13 22:44:26 main:479] : INFO : Learning Rate: 0.01 [2022-04-13 22:44:26 main:480] : INFO : Patience: 10 [2022-04-13 22:44:26 main:481] : INFO : Hidden Width: 12 [2022-04-13 22:44:26 main:482] : INFO : # Recurrent Layers: 4 [2022-04-13 22:44:26 main:483] : INFO : # Backend Layers: 4 [2022-04-13 22:44:26 main:484] : INFO : # Threads: 1 [2022-04-13 22:44:26 main:486] : INFO : Preparing Dataset [2022-04-13 22:44:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-13 22:44:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-13 22:44:32 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-13 22:44:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-13 22:44:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-13 22:44:32 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-13 22:44:32 main:494] : INFO : Creating Model [2022-04-13 22:44:32 main:507] : INFO : Preparing config file [2022-04-13 22:44:32 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-13 22:44:32 main:512] : INFO : Loading config [2022-04-13 22:44:32 main:514] : INFO : Loading state [2022-04-13 22:44:38 main:559] : INFO : Loading DataLoader into Memory [2022-04-13 22:44:38 main:562] : INFO : Starting Training [2022-04-13 22:44:49 main:574] : INFO : Epoch 1864 | loss: 0.0312293 | val_loss: 0.0312095 | Time: 10514.7 ms [2022-04-13 22:44:58 main:574] : INFO : Epoch 1865 | loss: 0.0312061 | val_loss: 0.0312094 | Time: 9027.9 ms [2022-04-13 22:45:07 main:574] : INFO : Epoch 1866 | loss: 0.0312054 | val_loss: 0.0312097 | Time: 8971.13 ms [2022-04-13 22:45:16 main:574] : INFO : Epoch 1867 | loss: 0.0312038 | val_loss: 0.0312068 | Time: 9163.94 ms [2022-04-13 22:45:25 main:574] : INFO : Epoch 1868 | loss: 0.0312014 | val_loss: 0.0312088 | Time: 9159.58 ms [2022-04-13 22:45:34 main:574] : INFO : Epoch 1869 | loss: 0.031201 | val_loss: 0.0312064 | Time: 9247.54 ms [2022-04-13 22:45:43 main:574] : INFO : Epoch 1870 | loss: 0.0312001 | val_loss: 0.0312051 | Time: 8974.2 ms [2022-04-13 22:45:52 main:574] : INFO : Epoch 1871 | loss: 0.0311987 | val_loss: 0.0312059 | Time: 8695.9 ms [2022-04-13 22:46:01 main:574] : INFO : Epoch 1872 | loss: 0.0311995 | val_loss: 0.0312062 | Time: 9097.8 ms [2022-04-13 22:46:10 main:574] : INFO : Epoch 1873 | loss: 0.0311983 | val_loss: 0.0312064 | Time: 8790.53 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1660 Ti) [2022-04-14 12:15:27 main:435] : INFO : Set logging level to 1 [2022-04-14 12:15:27 main:441] : INFO : Running in BOINC Client mode [2022-04-14 12:15:27 main:444] : INFO : Resolving all filenames [2022-04-14 12:15:27 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-14 12:15:27 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-14 12:15:27 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-14 12:15:27 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-14 12:15:27 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-14 12:15:27 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-14 12:15:27 main:474] : INFO : Configuration: [2022-04-14 12:15:27 main:475] : INFO : Model type: GRU [2022-04-14 12:15:27 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-14 12:15:27 main:477] : INFO : Max Epochs: 2048 [2022-04-14 12:15:27 main:478] : INFO : Batch Size: 128 [2022-04-14 12:15:27 main:479] : INFO : Learning Rate: 0.01 [2022-04-14 12:15:27 main:480] : INFO : Patience: 10 [2022-04-14 12:15:27 main:481] : INFO : Hidden Width: 12 [2022-04-14 12:15:27 main:482] : INFO : # Recurrent Layers: 4 [2022-04-14 12:15:27 main:483] : INFO : # Backend Layers: 4 [2022-04-14 12:15:27 main:484] : INFO : # Threads: 1 [2022-04-14 12:15:27 main:486] : INFO : Preparing Dataset [2022-04-14 12:15:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-14 12:15:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-14 12:15:32 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-14 12:15:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-14 12:15:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-14 12:15:32 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-14 12:15:32 main:494] : INFO : Creating Model [2022-04-14 12:15:32 main:507] : INFO : Preparing config file [2022-04-14 12:15:32 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-14 12:15:32 main:512] : INFO : Loading config [2022-04-14 12:15:32 main:514] : INFO : Loading state [2022-04-14 12:15:36 main:559] : INFO : Loading DataLoader into Memory [2022-04-14 12:15:36 main:562] : INFO : Starting Training [2022-04-14 12:15:45 main:574] : INFO : Epoch 1864 | loss: 0.0312333 | val_loss: 0.0312125 | Time: 9161.13 ms [2022-04-14 12:15:53 main:574] : INFO : Epoch 1865 | loss: 0.0312058 | val_loss: 0.0312104 | Time: 8236.46 ms [2022-04-14 12:16:01 main:574] : INFO : Epoch 1866 | loss: 0.0312041 | val_loss: 0.0312123 | Time: 7965.49 ms [2022-04-14 12:16:09 main:574] : INFO : Epoch 1867 | loss: 0.0312031 | val_loss: 0.0312116 | Time: 8025.01 ms [2022-04-14 12:16:17 main:574] : INFO : Epoch 1868 | loss: 0.0312043 | val_loss: 0.0312165 | Time: 7892.92 ms [2022-04-14 12:16:25 main:574] : INFO : Epoch 1869 | loss: 0.0312069 | val_loss: 0.0312147 | Time: 7903.22 ms [2022-04-14 12:16:33 main:574] : INFO : Epoch 1870 | loss: 0.0312041 | val_loss: 0.0312106 | Time: 7971.22 ms [2022-04-14 12:16:41 main:574] : INFO : Epoch 1871 | loss: 0.0312023 | val_loss: 0.0312099 | Time: 8068.3 ms [2022-04-14 12:16:49 main:574] : INFO : Epoch 1872 | loss: 0.0312029 | val_loss: 0.031212 | Time: 8181.94 ms [2022-04-14 12:16:57 main:574] : INFO : Epoch 1873 | loss: 0.0312032 | val_loss: 0.0312082 | Time: 7953.28 ms [2022-04-14 12:17:05 main:574] : INFO : Epoch 1874 | loss: 0.0312026 | val_loss: 0.0312098 | Time: 7930.44 ms [2022-04-14 12:17:13 main:574] : INFO : Epoch 1875 | loss: 0.0312018 | val_loss: 0.0312139 | Time: 7841.2 ms [2022-04-14 12:17:21 main:574] : INFO : Epoch 1876 | loss: 0.0312031 | val_loss: 0.031208 | Time: 8091.25 ms [2022-04-14 12:17:29 main:574] : INFO : Epoch 1877 | loss: 0.0312011 | val_loss: 0.0312079 | Time: 8023.47 ms [2022-04-14 12:17:37 main:574] : INFO : Epoch 1878 | loss: 0.0312003 | val_loss: 0.0312085 | Time: 8478.99 ms [2022-04-14 12:17:45 main:574] : INFO : Epoch 1879 | loss: 0.0312003 | val_loss: 0.0312071 | Time: 7932.56 ms [2022-04-14 12:17:53 main:574] : INFO : Epoch 1880 | loss: 0.0312005 | val_loss: 0.0312065 | Time: 8047.99 ms [2022-04-14 12:18:01 main:574] : INFO : Epoch 1881 | loss: 0.0312005 | val_loss: 0.0312075 | Time: 7984.61 ms [2022-04-14 12:18:09 main:574] : INFO : Epoch 1882 | loss: 0.0312001 | val_loss: 0.0312079 | Time: 8003.34 ms [2022-04-14 12:18:17 main:574] : INFO : Epoch 1883 | loss: 0.0312006 | val_loss: 0.0312083 | Time: 7995.34 ms [2022-04-14 12:18:25 main:574] : INFO : Epoch 1884 | loss: 0.031204 | val_loss: 0.0312124 | Time: 7943.73 ms [2022-04-14 12:18:33 main:574] : INFO : Epoch 1885 | loss: 0.0312026 | val_loss: 0.0312096 | Time: 7915.18 ms [2022-04-14 12:18:41 main:574] : INFO : Epoch 1886 | loss: 0.0312014 | val_loss: 0.0312085 | Time: 7864.12 ms [2022-04-14 12:18:49 main:574] : INFO : Epoch 1887 | loss: 0.0312016 | val_loss: 0.0312133 | Time: 7978.17 ms [2022-04-14 12:18:58 main:574] : INFO : Epoch 1888 | loss: 0.0312028 | val_loss: 0.0312087 | Time: 8416.03 ms [2022-04-14 12:19:06 main:574] : INFO : Epoch 1889 | loss: 0.0312016 | val_loss: 0.0312109 | Time: 7962.69 ms [2022-04-14 12:19:14 main:574] : INFO : Epoch 1890 | loss: 0.0311997 | val_loss: 0.0312084 | Time: 7997.25 ms [2022-04-14 12:19:21 main:574] : INFO : Epoch 1891 | loss: 0.0312003 | val_loss: 0.031209 | Time: 7906.04 ms [2022-04-14 12:19:29 main:574] : INFO : Epoch 1892 | loss: 0.0312002 | val_loss: 0.0312088 | Time: 7919.48 ms [2022-04-14 12:19:37 main:574] : INFO : Epoch 1893 | loss: 0.0312 | val_loss: 0.0312093 | Time: 8016.4 ms [2022-04-14 12:19:46 main:574] : INFO : Epoch 1894 | loss: 0.0312014 | val_loss: 0.0312115 | Time: 8158.56 ms [2022-04-14 12:19:54 main:574] : INFO : Epoch 1895 | loss: 0.0312003 | val_loss: 0.0312081 | Time: 8408.23 ms [2022-04-14 12:20:02 main:574] : INFO : Epoch 1896 | loss: 0.0311983 | val_loss: 0.0312089 | Time: 8020.65 ms [2022-04-14 12:20:10 main:574] : INFO : Epoch 1897 | loss: 0.0311986 | val_loss: 0.0312092 | Time: 8033.28 ms [2022-04-14 12:20:18 main:574] : INFO : Epoch 1898 | loss: 0.0311998 | val_loss: 0.0312077 | Time: 8077.94 ms [2022-04-14 12:20:26 main:574] : INFO : Epoch 1899 | loss: 0.031201 | val_loss: 0.0312073 | Time: 7914.14 ms [2022-04-14 12:20:34 main:574] : INFO : Epoch 1900 | loss: 0.0311998 | val_loss: 0.0312124 | Time: 7890.82 ms [2022-04-14 12:20:42 main:574] : INFO : Epoch 1901 | loss: 0.0312011 | val_loss: 0.0312072 | Time: 7960.83 ms [2022-04-14 12:20:50 main:574] : INFO : Epoch 1902 | loss: 0.0311984 | val_loss: 0.031209 | Time: 7927.48 ms [2022-04-14 12:20:58 main:574] : INFO : Epoch 1903 | loss: 0.0311989 | val_loss: 0.0312077 | Time: 8382.17 ms [2022-04-14 12:21:06 main:574] : INFO : Epoch 1904 | loss: 0.0311981 | val_loss: 0.0312073 | Time: 8104.75 ms [2022-04-14 12:21:14 main:574] : INFO : Epoch 1905 | loss: 0.0312013 | val_loss: 0.0312097 | Time: 8056.28 ms [2022-04-14 12:21:22 main:574] : INFO : Epoch 1906 | loss: 0.0312013 | val_loss: 0.0312089 | Time: 7958.73 ms [2022-04-14 12:21:30 main:574] : INFO : Epoch 1907 | loss: 0.0312004 | val_loss: 0.031208 | Time: 7953.99 ms [2022-04-14 12:21:38 main:574] : INFO : Epoch 1908 | loss: 0.0312003 | val_loss: 0.0312082 | Time: 7932.5 ms [2022-04-14 12:21:46 main:574] : INFO : Epoch 1909 | loss: 0.0312017 | val_loss: 0.0312157 | Time: 7877.84 ms [2022-04-14 12:21:54 main:574] : INFO : Epoch 1910 | loss: 0.0312034 | val_loss: 0.0312098 | Time: 8260.41 ms [2022-04-14 12:22:03 main:574] : INFO : Epoch 1911 | loss: 0.0312005 | val_loss: 0.031209 | Time: 8298.04 ms [2022-04-14 12:22:11 main:574] : INFO : Epoch 1912 | loss: 0.031201 | val_loss: 0.0312101 | Time: 7980.76 ms [2022-04-14 12:22:19 main:574] : INFO : Epoch 1913 | loss: 0.031202 | val_loss: 0.0312077 | Time: 8426.9 ms [2022-04-14 12:22:27 main:574] : INFO : Epoch 1914 | loss: 0.0312003 | val_loss: 0.0312104 | Time: 7962.3 ms [2022-04-14 12:22:36 main:574] : INFO : Epoch 1915 | loss: 0.0312021 | val_loss: 0.03121 | Time: 8466.04 ms [2022-04-14 12:22:43 main:574] : INFO : Epoch 1916 | loss: 0.0312014 | val_loss: 0.0312102 | Time: 7705.05 ms [2022-04-14 12:22:51 main:574] : INFO : Epoch 1917 | loss: 0.0312014 | val_loss: 0.0312114 | Time: 7420.89 ms [2022-04-14 12:22:59 main:574] : INFO : Epoch 1918 | loss: 0.0312004 | val_loss: 0.0312072 | Time: 8005.68 ms [2022-04-14 12:23:07 main:574] : INFO : Epoch 1919 | loss: 0.0311989 | val_loss: 0.0312086 | Time: 8081.95 ms [2022-04-14 12:23:15 main:574] : INFO : Epoch 1920 | loss: 0.0311987 | val_loss: 0.0312078 | Time: 7759.95 ms [2022-04-14 12:23:22 main:574] : INFO : Epoch 1921 | loss: 0.0311986 | val_loss: 0.031208 | Time: 7770.63 ms [2022-04-14 12:23:30 main:574] : INFO : Epoch 1922 | loss: 0.0311996 | val_loss: 0.0312083 | Time: 7753.1 ms [2022-04-14 12:23:38 main:574] : INFO : Epoch 1923 | loss: 0.0311989 | val_loss: 0.0312099 | Time: 7901.93 ms [2022-04-14 12:23:45 main:574] : INFO : Epoch 1924 | loss: 0.031201 | val_loss: 0.0312101 | Time: 7316.8 ms [2022-04-14 12:23:53 main:574] : INFO : Epoch 1925 | loss: 0.0311995 | val_loss: 0.0312085 | Time: 7667.56 ms [2022-04-14 12:24:00 main:574] : INFO : Epoch 1926 | loss: 0.0311981 | val_loss: 0.0312061 | Time: 7500.64 ms [2022-04-14 12:24:08 main:574] : INFO : Epoch 1927 | loss: 0.031198 | val_loss: 0.0312084 | Time: 7726.87 ms [2022-04-14 12:24:16 main:574] : INFO : Epoch 1928 | loss: 0.0311996 | val_loss: 0.0312091 | Time: 7376.56 ms [2022-04-14 12:24:23 main:574] : INFO : Epoch 1929 | loss: 0.0311999 | val_loss: 0.031208 | Time: 7537.58 ms [2022-04-14 12:24:30 main:574] : INFO : Epoch 1930 | loss: 0.0311985 | val_loss: 0.0312105 | Time: 7138.96 ms [2022-04-14 12:24:38 main:574] : INFO : Epoch 1931 | loss: 0.0312001 | val_loss: 0.0312098 | Time: 7581.09 ms [2022-04-14 12:24:45 main:574] : INFO : Epoch 1932 | loss: 0.0311992 | val_loss: 0.0312067 | Time: 7258.89 ms [2022-04-14 12:24:53 main:574] : INFO : Epoch 1933 | loss: 0.0311979 | val_loss: 0.0312074 | Time: 7592.32 ms [2022-04-14 12:25:00 main:574] : INFO : Epoch 1934 | loss: 0.0311977 | val_loss: 0.031206 | Time: 7674.07 ms [2022-04-14 12:25:08 main:574] : INFO : Epoch 1935 | loss: 0.0311991 | val_loss: 0.0312088 | Time: 7482.43 ms [2022-04-14 12:25:16 main:574] : INFO : Epoch 1936 | loss: 0.0311992 | val_loss: 0.031206 | Time: 7817.89 ms [2022-04-14 12:25:23 main:574] : INFO : Epoch 1937 | loss: 0.0311998 | val_loss: 0.0312083 | Time: 7552.1 ms [2022-04-14 12:25:31 main:574] : INFO : Epoch 1938 | loss: 0.0311978 | val_loss: 0.0312097 | Time: 7639.22 ms [2022-04-14 12:25:39 main:574] : INFO : Epoch 1939 | loss: 0.0311992 | val_loss: 0.0312068 | Time: 7701.46 ms [2022-04-14 12:25:46 main:574] : INFO : Epoch 1940 | loss: 0.0311983 | val_loss: 0.0312066 | Time: 7806.49 ms [2022-04-14 12:25:54 main:574] : INFO : Epoch 1941 | loss: 0.0312005 | val_loss: 0.0312079 | Time: 7205.52 ms [2022-04-14 12:26:01 main:574] : INFO : Epoch 1942 | loss: 0.0311987 | val_loss: 0.0312072 | Time: 7656.3 ms [2022-04-14 12:26:09 main:574] : INFO : Epoch 1943 | loss: 0.0311991 | val_loss: 0.0312065 | Time: 7488.69 ms [2022-04-14 12:26:16 main:574] : INFO : Epoch 1944 | loss: 0.0311992 | val_loss: 0.0312097 | Time: 7551.53 ms [2022-04-14 12:26:24 main:574] : INFO : Epoch 1945 | loss: 0.0311996 | val_loss: 0.031206 | Time: 7547.55 ms [2022-04-14 12:26:32 main:574] : INFO : Epoch 1946 | loss: 0.0311986 | val_loss: 0.0312074 | Time: 8130.27 ms [2022-04-14 12:26:39 main:574] : INFO : Epoch 1947 | loss: 0.0311987 | val_loss: 0.0312051 | Time: 7287.01 ms [2022-04-14 12:26:47 main:574] : INFO : Epoch 1948 | loss: 0.0311992 | val_loss: 0.031208 | Time: 7759.72 ms [2022-04-14 12:26:55 main:574] : INFO : Epoch 1949 | loss: 0.0311986 | val_loss: 0.0312073 | Time: 7697.47 ms [2022-04-14 12:27:02 main:574] : INFO : Epoch 1950 | loss: 0.0311981 | val_loss: 0.0312072 | Time: 7488.02 ms [2022-04-14 12:27:10 main:574] : INFO : Epoch 1951 | loss: 0.0311984 | val_loss: 0.0312083 | Time: 7787.47 ms [2022-04-14 12:27:18 main:574] : INFO : Epoch 1952 | loss: 0.0311977 | val_loss: 0.0312087 | Time: 7478.25 ms [2022-04-14 12:27:25 main:574] : INFO : Epoch 1953 | loss: 0.0311976 | val_loss: 0.0312073 | Time: 7708.85 ms [2022-04-14 12:27:33 main:574] : INFO : Epoch 1954 | loss: 0.0311971 | val_loss: 0.0312072 | Time: 7617.62 ms [2022-04-14 12:27:41 main:574] : INFO : Epoch 1955 | loss: 0.0311981 | val_loss: 0.0312075 | Time: 7734.51 ms [2022-04-14 12:27:48 main:574] : INFO : Epoch 1956 | loss: 0.0311972 | val_loss: 0.0312087 | Time: 7784.31 ms [2022-04-14 12:27:56 main:574] : INFO : Epoch 1957 | loss: 0.0311986 | val_loss: 0.0312084 | Time: 7904.97 ms [2022-04-14 12:28:04 main:574] : INFO : Epoch 1958 | loss: 0.031198 | val_loss: 0.0312077 | Time: 7336.87 ms [2022-04-14 12:28:11 main:574] : INFO : Epoch 1959 | loss: 0.0311982 | val_loss: 0.0312086 | Time: 7524.02 ms [2022-04-14 12:28:19 main:574] : INFO : Epoch 1960 | loss: 0.0311982 | val_loss: 0.0312083 | Time: 7421.56 ms [2022-04-14 12:28:26 main:574] : INFO : Epoch 1961 | loss: 0.0311983 | val_loss: 0.0312088 | Time: 7763.46 ms [2022-04-14 12:28:34 main:574] : INFO : Epoch 1962 | loss: 0.0311986 | val_loss: 0.0312075 | Time: 7452.24 ms [2022-04-14 12:28:41 main:574] : INFO : Epoch 1963 | loss: 0.031198 | val_loss: 0.0312076 | Time: 7397.4 ms [2022-04-14 12:28:49 main:574] : INFO : Epoch 1964 | loss: 0.0312003 | val_loss: 0.0312111 | Time: 7682.76 ms [2022-04-14 12:28:57 main:574] : INFO : Epoch 1965 | loss: 0.0311999 | val_loss: 0.0312122 | Time: 7618.73 ms [2022-04-14 12:29:04 main:574] : INFO : Epoch 1966 | loss: 0.0311992 | val_loss: 0.0312074 | Time: 7543.99 ms [2022-04-14 12:29:12 main:574] : INFO : Epoch 1967 | loss: 0.0311979 | val_loss: 0.0312105 | Time: 7432.69 ms [2022-04-14 12:29:19 main:574] : INFO : Epoch 1968 | loss: 0.0311986 | val_loss: 0.0312073 | Time: 7419.14 ms [2022-04-14 12:29:27 main:574] : INFO : Epoch 1969 | loss: 0.0311988 | val_loss: 0.0312083 | Time: 7687.97 ms [2022-04-14 12:29:34 main:574] : INFO : Epoch 1970 | loss: 0.0311995 | val_loss: 0.0312096 | Time: 7610.73 ms [2022-04-14 12:29:42 main:574] : INFO : Epoch 1971 | loss: 0.0312005 | val_loss: 0.0312075 | Time: 7527.62 ms [2022-04-14 12:29:49 main:574] : INFO : Epoch 1972 | loss: 0.0311992 | val_loss: 0.0312081 | Time: 7609.2 ms [2022-04-14 12:29:58 main:574] : INFO : Epoch 1973 | loss: 0.0311988 | val_loss: 0.0312092 | Time: 8054.51 ms [2022-04-14 12:30:05 main:574] : INFO : Epoch 1974 | loss: 0.0311982 | val_loss: 0.031211 | Time: 7965.61 ms [2022-04-14 12:30:13 main:574] : INFO : Epoch 1975 | loss: 0.0311979 | val_loss: 0.0312083 | Time: 7870.21 ms [2022-04-14 12:30:21 main:574] : INFO : Epoch 1976 | loss: 0.0311991 | val_loss: 0.0312074 | Time: 7519.76 ms [2022-04-14 12:30:29 main:574] : INFO : Epoch 1977 | loss: 0.0311997 | val_loss: 0.0312087 | Time: 7979.46 ms [2022-04-14 12:30:37 main:574] : INFO : Epoch 1978 | loss: 0.0312014 | val_loss: 0.0312069 | Time: 7796.61 ms [2022-04-14 12:30:44 main:574] : INFO : Epoch 1979 | loss: 0.0311997 | val_loss: 0.0312081 | Time: 7601.26 ms [2022-04-14 12:30:52 main:574] : INFO : Epoch 1980 | loss: 0.0311993 | val_loss: 0.0312093 | Time: 7657.84 ms [2022-04-14 12:30:59 main:574] : INFO : Epoch 1981 | loss: 0.0312002 | val_loss: 0.0312078 | Time: 7381.24 ms [2022-04-14 12:31:07 main:574] : INFO : Epoch 1982 | loss: 0.0312001 | val_loss: 0.0312131 | Time: 7963.58 ms [2022-04-14 12:31:15 main:574] : INFO : Epoch 1983 | loss: 0.0312015 | val_loss: 0.0312072 | Time: 7880.34 ms [2022-04-14 12:31:23 main:574] : INFO : Epoch 1984 | loss: 0.0311987 | val_loss: 0.0312068 | Time: 7814.77 ms [2022-04-14 12:31:30 main:574] : INFO : Epoch 1985 | loss: 0.0311989 | val_loss: 0.0312088 | Time: 7208.62 ms [2022-04-14 12:31:38 main:574] : INFO : Epoch 1986 | loss: 0.0311987 | val_loss: 0.0312082 | Time: 7505.4 ms [2022-04-14 12:31:45 main:574] : INFO : Epoch 1987 | loss: 0.0312004 | val_loss: 0.0312105 | Time: 7612.6 ms [2022-04-14 12:31:53 main:574] : INFO : Epoch 1988 | loss: 0.0312011 | val_loss: 0.0312091 | Time: 7814.7 ms [2022-04-14 12:32:01 main:574] : INFO : Epoch 1989 | loss: 0.0312009 | val_loss: 0.0312081 | Time: 7625.96 ms [2022-04-14 12:32:08 main:574] : INFO : Epoch 1990 | loss: 0.0311994 | val_loss: 0.0312087 | Time: 7559.69 ms [2022-04-14 12:32:16 main:574] : INFO : Epoch 1991 | loss: 0.0312001 | val_loss: 0.0312087 | Time: 7814.68 ms [2022-04-14 12:32:24 main:574] : INFO : Epoch 1992 | loss: 0.0311999 | val_loss: 0.0312106 | Time: 7858.59 ms [2022-04-14 12:32:32 main:574] : INFO : Epoch 1993 | loss: 0.0311994 | val_loss: 0.0312091 | Time: 7783.31 ms [2022-04-14 12:32:40 main:574] : INFO : Epoch 1994 | loss: 0.0311991 | val_loss: 0.0312102 | Time: 7684.8 ms [2022-04-14 12:32:47 main:574] : INFO : Epoch 1995 | loss: 0.0311987 | val_loss: 0.0312093 | Time: 7408.51 ms [2022-04-14 12:32:54 main:574] : INFO : Epoch 1996 | loss: 0.0311993 | val_loss: 0.0312118 | Time: 7425.69 ms [2022-04-14 12:33:02 main:574] : INFO : Epoch 1997 | loss: 0.0312015 | val_loss: 0.0312153 | Time: 7649.96 ms [2022-04-14 12:33:10 main:574] : INFO : Epoch 1998 | loss: 0.0312013 | val_loss: 0.0312121 | Time: 7507.75 ms [2022-04-14 12:33:17 main:574] : INFO : Epoch 1999 | loss: 0.0312002 | val_loss: 0.0312136 | Time: 7274.1 ms [2022-04-14 12:33:24 main:574] : INFO : Epoch 2000 | loss: 0.0312003 | val_loss: 0.031209 | Time: 7544.78 ms [2022-04-14 12:33:32 main:574] : INFO : Epoch 2001 | loss: 0.0311988 | val_loss: 0.0312104 | Time: 7363.35 ms [2022-04-14 12:33:39 main:574] : INFO : Epoch 2002 | loss: 0.0311994 | val_loss: 0.0312089 | Time: 7230.82 ms [2022-04-14 12:33:47 main:574] : INFO : Epoch 2003 | loss: 0.0312011 | val_loss: 0.0312078 | Time: 7568.12 ms [2022-04-14 12:33:54 main:574] : INFO : Epoch 2004 | loss: 0.0311982 | val_loss: 0.0312072 | Time: 7861.9 ms [2022-04-14 12:34:02 main:574] : INFO : Epoch 2005 | loss: 0.0311988 | val_loss: 0.0312087 | Time: 7758.69 ms [2022-04-14 12:34:10 main:574] : INFO : Epoch 2006 | loss: 0.0312004 | val_loss: 0.0312076 | Time: 7811.65 ms [2022-04-14 12:34:18 main:574] : INFO : Epoch 2007 | loss: 0.0311993 | val_loss: 0.0312112 | Time: 7632.76 ms [2022-04-14 12:34:25 main:574] : INFO : Epoch 2008 | loss: 0.031198 | val_loss: 0.0312081 | Time: 7777.41 ms [2022-04-14 12:34:33 main:574] : INFO : Epoch 2009 | loss: 0.0311986 | val_loss: 0.0312073 | Time: 7424.35 ms [2022-04-14 12:34:41 main:574] : INFO : Epoch 2010 | loss: 0.0311982 | val_loss: 0.031208 | Time: 7736.59 ms [2022-04-14 12:34:48 main:574] : INFO : Epoch 2011 | loss: 0.0311987 | val_loss: 0.0312068 | Time: 7601.76 ms [2022-04-14 12:34:56 main:574] : INFO : Epoch 2012 | loss: 0.0312014 | val_loss: 0.0312063 | Time: 7399.57 ms [2022-04-14 12:35:03 main:574] : INFO : Epoch 2013 | loss: 0.0311975 | val_loss: 0.0312086 | Time: 7680.65 ms [2022-04-14 12:35:11 main:574] : INFO : Epoch 2014 | loss: 0.0311973 | val_loss: 0.0312074 | Time: 7501.56 ms [2022-04-14 12:35:18 main:574] : INFO : Epoch 2015 | loss: 0.0312016 | val_loss: 0.0312101 | Time: 7605.44 ms [2022-04-14 12:35:26 main:574] : INFO : Epoch 2016 | loss: 0.0312045 | val_loss: 0.03121 | Time: 7284.66 ms [2022-04-14 12:35:33 main:574] : INFO : Epoch 2017 | loss: 0.0312047 | val_loss: 0.0312143 | Time: 7560.77 ms [2022-04-14 12:35:41 main:574] : INFO : Epoch 2018 | loss: 0.0312045 | val_loss: 0.03121 | Time: 7467.16 ms [2022-04-14 12:35:48 main:574] : INFO : Epoch 2019 | loss: 0.0312041 | val_loss: 0.0312112 | Time: 7636.96 ms [2022-04-14 12:35:56 main:574] : INFO : Epoch 2020 | loss: 0.0312102 | val_loss: 0.031218 | Time: 7463.93 ms [2022-04-14 12:36:03 main:574] : INFO : Epoch 2021 | loss: 0.0312169 | val_loss: 0.0312183 | Time: 7519.82 ms [2022-04-14 12:36:11 main:574] : INFO : Epoch 2022 | loss: 0.0312141 | val_loss: 0.0312183 | Time: 8140.13 ms [2022-04-14 12:36:19 main:574] : INFO : Epoch 2023 | loss: 0.0312154 | val_loss: 0.0312174 | Time: 7355.3 ms [2022-04-14 12:36:26 main:574] : INFO : Epoch 2024 | loss: 0.031215 | val_loss: 0.0312165 | Time: 7459.13 ms [2022-04-14 12:36:34 main:574] : INFO : Epoch 2025 | loss: 0.031215 | val_loss: 0.0312204 | Time: 7618.58 ms [2022-04-14 12:36:41 main:574] : INFO : Epoch 2026 | loss: 0.0312157 | val_loss: 0.0312155 | Time: 7398.89 ms [2022-04-14 12:36:49 main:574] : INFO : Epoch 2027 | loss: 0.0312135 | val_loss: 0.0312148 | Time: 7571.81 ms [2022-04-14 12:36:57 main:574] : INFO : Epoch 2028 | loss: 0.031214 | val_loss: 0.0312149 | Time: 7886.91 ms [2022-04-14 12:37:04 main:574] : INFO : Epoch 2029 | loss: 0.0312138 | val_loss: 0.0312187 | Time: 7651.7 ms [2022-04-14 12:37:12 main:574] : INFO : Epoch 2030 | loss: 0.0312129 | val_loss: 0.031215 | Time: 7926.97 ms [2022-04-14 12:37:20 main:574] : INFO : Epoch 2031 | loss: 0.0312113 | val_loss: 0.0312146 | Time: 7876.48 ms [2022-04-14 12:37:28 main:574] : INFO : Epoch 2032 | loss: 0.0312108 | val_loss: 0.0312137 | Time: 7784.77 ms [2022-04-14 12:37:36 main:574] : INFO : Epoch 2033 | loss: 0.0312107 | val_loss: 0.0312139 | Time: 7920.97 ms [2022-04-14 12:37:43 main:574] : INFO : Epoch 2034 | loss: 0.0312104 | val_loss: 0.0312144 | Time: 6993.48 ms [2022-04-14 12:37:51 main:574] : INFO : Epoch 2035 | loss: 0.03121 | val_loss: 0.0312146 | Time: 7687.01 ms [2022-04-14 12:37:58 main:574] : INFO : Epoch 2036 | loss: 0.0312104 | val_loss: 0.0312163 | Time: 7712.33 ms [2022-04-14 12:38:06 main:574] : INFO : Epoch 2037 | loss: 0.0312098 | val_loss: 0.031215 | Time: 7696.79 ms [2022-04-14 12:38:14 main:574] : INFO : Epoch 2038 | loss: 0.0312098 | val_loss: 0.0312167 | Time: 7844.97 ms [2022-04-14 12:38:22 main:574] : INFO : Epoch 2039 | loss: 0.0312098 | val_loss: 0.0312129 | Time: 7616.49 ms [2022-04-14 12:38:29 main:574] : INFO : Epoch 2040 | loss: 0.0312082 | val_loss: 0.0312128 | Time: 7592.45 ms [2022-04-14 12:38:36 main:574] : INFO : Epoch 2041 | loss: 0.0312099 | val_loss: 0.0312123 | Time: 7339.26 ms [2022-04-14 12:38:44 main:574] : INFO : Epoch 2042 | loss: 0.0312089 | val_loss: 0.031215 | Time: 7262.12 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1660 Ti) [2022-04-15 12:21:54 main:435] : INFO : Set logging level to 1 [2022-04-15 12:21:54 main:441] : INFO : Running in BOINC Client mode [2022-04-15 12:21:54 main:444] : INFO : Resolving all filenames [2022-04-15 12:21:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-15 12:21:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-15 12:21:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-15 12:21:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-15 12:21:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-15 12:21:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-15 12:21:54 main:474] : INFO : Configuration: [2022-04-15 12:21:54 main:475] : INFO : Model type: GRU [2022-04-15 12:21:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-15 12:21:54 main:477] : INFO : Max Epochs: 2048 [2022-04-15 12:21:54 main:478] : INFO : Batch Size: 128 [2022-04-15 12:21:54 main:479] : INFO : Learning Rate: 0.01 [2022-04-15 12:21:54 main:480] : INFO : Patience: 10 [2022-04-15 12:21:54 main:481] : INFO : Hidden Width: 12 [2022-04-15 12:21:54 main:482] : INFO : # Recurrent Layers: 4 [2022-04-15 12:21:54 main:483] : INFO : # Backend Layers: 4 [2022-04-15 12:21:54 main:484] : INFO : # Threads: 1 [2022-04-15 12:21:54 main:486] : INFO : Preparing Dataset [2022-04-15 12:21:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-15 12:21:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-15 12:21:58 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-15 12:21:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-15 12:21:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-15 12:21:59 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-15 12:21:59 main:494] : INFO : Creating Model [2022-04-15 12:21:59 main:507] : INFO : Preparing config file [2022-04-15 12:21:59 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-15 12:21:59 main:512] : INFO : Loading config [2022-04-15 12:21:59 main:514] : INFO : Loading state [2022-04-15 12:22:01 main:559] : INFO : Loading DataLoader into Memory [2022-04-15 12:22:01 main:562] : INFO : Starting Training [2022-04-15 12:22:09 main:574] : INFO : Epoch 2019 | loss: 0.0312289 | val_loss: 0.0312174 | Time: 8312.22 ms [2022-04-15 12:22:17 main:574] : INFO : Epoch 2020 | loss: 0.0312069 | val_loss: 0.0312093 | Time: 7969.1 ms [2022-04-15 12:22:25 main:574] : INFO : Epoch 2021 | loss: 0.0312041 | val_loss: 0.0312084 | Time: 7774.37 ms [2022-04-15 12:22:33 main:574] : INFO : Epoch 2022 | loss: 0.0312031 | val_loss: 0.0312078 | Time: 7836.09 ms [2022-04-15 12:22:40 main:574] : INFO : Epoch 2023 | loss: 0.0312014 | val_loss: 0.0312052 | Time: 7803.22 ms [2022-04-15 12:22:48 main:574] : INFO : Epoch 2024 | loss: 0.0311994 | val_loss: 0.0312044 | Time: 7936.6 ms [2022-04-15 12:22:56 main:574] : INFO : Epoch 2025 | loss: 0.0312021 | val_loss: 0.0312067 | Time: 7496.47 ms [2022-04-15 12:23:04 main:574] : INFO : Epoch 2026 | loss: 0.0312043 | val_loss: 0.0312105 | Time: 7649.07 ms [2022-04-15 12:23:11 main:574] : INFO : Epoch 2027 | loss: 0.031202 | val_loss: 0.0312108 | Time: 7357.84 ms [2022-04-15 12:23:19 main:574] : INFO : Epoch 2028 | loss: 0.0312026 | val_loss: 0.0312084 | Time: 7812.96 ms [2022-04-15 12:23:27 main:574] : INFO : Epoch 2029 | loss: 0.0312007 | val_loss: 0.0312083 | Time: 8062.4 ms [2022-04-15 12:23:35 main:574] : INFO : Epoch 2030 | loss: 0.0312002 | val_loss: 0.031208 | Time: 8064.56 ms [2022-04-15 12:23:43 main:574] : INFO : Epoch 2031 | loss: 0.0311999 | val_loss: 0.031208 | Time: 7798.52 ms [2022-04-15 12:23:51 main:574] : INFO : Epoch 2032 | loss: 0.0311999 | val_loss: 0.0312069 | Time: 8083.61 ms [2022-04-15 12:23:59 main:574] : INFO : Epoch 2033 | loss: 0.0312015 | val_loss: 0.0312065 | Time: 7898.8 ms [2022-04-15 12:24:07 main:574] : INFO : Epoch 2034 | loss: 0.031201 | val_loss: 0.0312081 | Time: 7834.76 ms [2022-04-15 12:24:14 main:574] : INFO : Epoch 2035 | loss: 0.0312005 | val_loss: 0.0312038 | Time: 7767.3 ms [2022-04-15 12:24:22 main:574] : INFO : Epoch 2036 | loss: 0.0311999 | val_loss: 0.0312036 | Time: 7501.78 ms [2022-04-15 12:24:30 main:574] : INFO : Epoch 2037 | loss: 0.0312009 | val_loss: 0.0312048 | Time: 8073.26 ms [2022-04-15 12:24:38 main:574] : INFO : Epoch 2038 | loss: 0.0312022 | val_loss: 0.0312103 | Time: 7670.67 ms [2022-04-15 12:24:46 main:574] : INFO : Epoch 2039 | loss: 0.0311997 | val_loss: 0.0312032 | Time: 7943.53 ms [2022-04-15 12:24:54 main:574] : INFO : Epoch 2040 | loss: 0.0311988 | val_loss: 0.0312027 | Time: 8118.69 ms [2022-04-15 12:25:02 main:574] : INFO : Epoch 2041 | loss: 0.0311985 | val_loss: 0.0312037 | Time: 7896.03 ms [2022-04-15 12:25:10 main:574] : INFO : Epoch 2042 | loss: 0.0311998 | val_loss: 0.031207 | Time: 8083.05 ms [2022-04-15 12:25:18 main:574] : INFO : Epoch 2043 | loss: 0.0311993 | val_loss: 0.0312055 | Time: 7868.57 ms [2022-04-15 12:25:25 main:574] : INFO : Epoch 2044 | loss: 0.0311978 | val_loss: 0.031201 | Time: 7801.5 ms [2022-04-15 12:25:33 main:574] : INFO : Epoch 2045 | loss: 0.0311975 | val_loss: 0.0312024 | Time: 7950.26 ms [2022-04-15 12:25:41 main:574] : INFO : Epoch 2046 | loss: 0.0311974 | val_loss: 0.0312025 | Time: 7943.99 ms [2022-04-15 12:25:49 main:574] : INFO : Epoch 2047 | loss: 0.0311968 | val_loss: 0.0312025 | Time: 7623.4 ms [2022-04-15 12:25:57 main:574] : INFO : Epoch 2048 | loss: 0.0311961 | val_loss: 0.0312035 | Time: 7634.13 ms [2022-04-15 12:25:57 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0312035 [2022-04-15 12:25:57 main:603] : INFO : Saving end state to config to file [2022-04-15 12:25:57 main:608] : INFO : Success, exiting.. 12:25:57 (31944): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)