Task 14621850
| Name | ParityModified-1647047026-642-2-0_0 |
| Workunit | 11639362 |
| Created | 15 Apr 2022, 14:57:20 UTC |
| Sent | 15 Apr 2022, 14:57:39 UTC |
| Report deadline | 23 Apr 2022, 14:57:39 UTC |
| Received | 24 Apr 2022, 10:48:10 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 13966 |
| Run time | 2 hours 41 min 58 sec |
| CPU time | 2 hours 35 min 46 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 10,956.98 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.37 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 48 | Time: 4981.98 ms [2022-04-24 19:39:24 main:574] : INFO : Epoch 1735 | loss: 0.031116 | val_loss: 0.0311779 | Time: 4846.29 ms [2022-04-24 19:39:29 main:574] : INFO : Epoch 1736 | loss: 0.0311122 | val_loss: 0.031192 | Time: 5125.31 ms [2022-04-24 19:39:34 main:574] : INFO : Epoch 1737 | loss: 0.031112 | val_loss: 0.0311888 | Time: 5064.03 ms [2022-04-24 19:39:39 main:574] : INFO : Epoch 1738 | loss: 0.0311189 | val_loss: 0.0311786 | Time: 4997.78 ms [2022-04-24 19:39:44 main:574] : INFO : Epoch 1739 | loss: 0.0311205 | val_loss: 0.0311851 | Time: 4851.98 ms [2022-04-24 19:39:49 main:574] : INFO : Epoch 1740 | loss: 0.0311146 | val_loss: 0.0311819 | Time: 4935.78 ms [2022-04-24 19:39:54 main:574] : INFO : Epoch 1741 | loss: 0.031117 | val_loss: 0.0311793 | Time: 4905.91 ms [2022-04-24 19:39:59 main:574] : INFO : Epoch 1742 | loss: 0.0311137 | val_loss: 0.0311828 | Time: 4937.7 ms [2022-04-24 19:40:04 main:574] : INFO : Epoch 1743 | loss: 0.0311098 | val_loss: 0.0311849 | Time: 4894.14 ms [2022-04-24 19:40:09 main:574] : INFO : Epoch 1744 | loss: 0.0311089 | val_loss: 0.0311833 | Time: 4880.11 ms [2022-04-24 19:40:13 main:574] : INFO : Epoch 1745 | loss: 0.0311069 | val_loss: 0.031189 | Time: 4889.14 ms [2022-04-24 19:40:18 main:574] : INFO : Epoch 1746 | loss: 0.0311068 | val_loss: 0.0311928 | Time: 5004.83 ms [2022-04-24 19:40:24 main:574] : INFO : Epoch 1747 | loss: 0.0311074 | val_loss: 0.0311888 | Time: 5050.96 ms [2022-04-24 19:40:28 main:574] : INFO : Epoch 1748 | loss: 0.0311083 | val_loss: 0.0311941 | Time: 4798.37 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 19:43:33 main:435] : INFO : Set logging level to 1 [2022-04-24 19:43:33 main:441] : INFO : Running in BOINC Client mode [2022-04-24 19:43:33 main:444] : INFO : Resolving all filenames [2022-04-24 19:43:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 19:43:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 19:43:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 19:43:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 19:43:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 19:43:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 19:43:33 main:474] : INFO : Configuration: [2022-04-24 19:43:33 main:475] : INFO : Model type: GRU [2022-04-24 19:43:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 19:43:33 main:477] : INFO : Max Epochs: 2048 [2022-04-24 19:43:33 main:478] : INFO : Batch Size: 128 [2022-04-24 19:43:34 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 19:43:34 main:480] : INFO : Patience: 10 [2022-04-24 19:43:34 main:481] : INFO : Hidden Width: 12 [2022-04-24 19:43:34 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 19:43:34 main:483] : INFO : # Backend Layers: 4 [2022-04-24 19:43:34 main:484] : INFO : # Threads: 1 [2022-04-24 19:43:34 main:486] : INFO : Preparing Dataset [2022-04-24 19:43:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 19:43:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 19:43:36 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 19:43:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 19:43:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 19:43:36 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 19:43:36 main:494] : INFO : Creating Model [2022-04-24 19:43:36 main:507] : INFO : Preparing config file [2022-04-24 19:43:36 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 19:43:36 main:512] : INFO : Loading config [2022-04-24 19:43:36 main:514] : INFO : Loading state [2022-04-24 19:43:37 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 19:43:37 main:562] : INFO : Starting Training [2022-04-24 19:43:43 main:574] : INFO : Epoch 1746 | loss: 0.0311531 | val_loss: 0.0311897 | Time: 5600.8 ms [2022-04-24 19:43:48 main:574] : INFO : Epoch 1747 | loss: 0.0311212 | val_loss: 0.0311814 | Time: 4970.13 ms [2022-04-24 19:43:53 main:574] : INFO : Epoch 1748 | loss: 0.0311144 | val_loss: 0.0311943 | Time: 4978.71 ms [2022-04-24 19:43:58 main:574] : INFO : Epoch 1749 | loss: 0.0311126 | val_loss: 0.0311818 | Time: 4919 ms [2022-04-24 19:44:03 main:574] : INFO : Epoch 1750 | loss: 0.0311123 | val_loss: 0.0311886 | Time: 4984.93 ms [2022-04-24 19:44:08 main:574] : INFO : Epoch 1751 | loss: 0.0311128 | val_loss: 0.0311787 | Time: 4962.15 ms [2022-04-24 19:44:12 main:574] : INFO : Epoch 1752 | loss: 0.0311175 | val_loss: 0.03118 | Time: 4940.86 ms [2022-04-24 19:44:17 main:574] : INFO : Epoch 1753 | loss: 0.0311174 | val_loss: 0.0311842 | Time: 4855.71 ms [2022-04-24 19:44:22 main:574] : INFO : Epoch 1754 | loss: 0.0311153 | val_loss: 0.0311897 | Time: 4935.24 ms [2022-04-24 19:44:27 main:574] : INFO : Epoch 1755 | loss: 0.0311115 | val_loss: 0.031188 | Time: 4899.38 ms [2022-04-24 19:44:32 main:574] : INFO : Epoch 1756 | loss: 0.0311101 | val_loss: 0.03119 | Time: 4956.28 ms [2022-04-24 19:44:37 main:574] : INFO : Epoch 1757 | loss: 0.0311103 | val_loss: 0.0311972 | Time: 4996.56 ms [2022-04-24 19:44:42 main:574] : INFO : Epoch 1758 | loss: 0.0311118 | val_loss: 0.0311861 | Time: 4982.8 ms [2022-04-24 19:44:47 main:574] : INFO : Epoch 1759 | loss: 0.0311066 | val_loss: 0.0311918 | Time: 4967.05 ms [2022-04-24 19:44:52 main:574] : INFO : Epoch 1760 | loss: 0.0311059 | val_loss: 0.031197 | Time: 4925.61 ms [2022-04-24 19:44:57 main:574] : INFO : Epoch 1761 | loss: 0.0311117 | val_loss: 0.0311987 | Time: 4918.8 ms [2022-04-24 19:45:02 main:574] : INFO : Epoch 1762 | loss: 0.0311098 | val_loss: 0.0311824 | Time: 4944.86 ms [2022-04-24 19:45:07 main:574] : INFO : Epoch 1763 | loss: 0.0311121 | val_loss: 0.0311995 | Time: 4925 ms [2022-04-24 19:45:12 main:574] : INFO : Epoch 1764 | loss: 0.0311111 | val_loss: 0.0311968 | Time: 4882.89 ms [2022-04-24 19:45:17 main:574] : INFO : Epoch 1765 | loss: 0.0311072 | val_loss: 0.0311872 | Time: 4856.33 ms [2022-04-24 19:45:22 main:574] : INFO : Epoch 1766 | loss: 0.031107 | val_loss: 0.0311891 | Time: 4914.84 ms [2022-04-24 19:45:26 main:574] : INFO : Epoch 1767 | loss: 0.0311072 | val_loss: 0.0311868 | Time: 4874.14 ms [2022-04-24 19:45:31 main:574] : INFO : Epoch 1768 | loss: 0.0311034 | val_loss: 0.0311988 | Time: 4921.95 ms [2022-04-24 19:45:36 main:574] : INFO : Epoch 1769 | loss: 0.031102 | val_loss: 0.0311884 | Time: 4947.52 ms [2022-04-24 19:45:42 main:574] : INFO : Epoch 1770 | loss: 0.0311044 | val_loss: 0.031196 | Time: 5244.25 ms [2022-04-24 19:45:47 main:574] : INFO : Epoch 1771 | loss: 0.0311052 | val_loss: 0.0312008 | Time: 4954.86 ms [2022-04-24 19:45:52 main:574] : INFO : Epoch 1772 | loss: 0.0311025 | val_loss: 0.0311943 | Time: 5006.28 ms [2022-04-24 19:45:56 main:574] : INFO : Epoch 1773 | loss: 0.0310993 | val_loss: 0.0311918 | Time: 4952.29 ms [2022-04-24 19:46:02 main:574] : INFO : Epoch 1774 | loss: 0.0311004 | val_loss: 0.0311957 | Time: 5017.96 ms [2022-04-24 19:46:06 main:574] : INFO : Epoch 1775 | loss: 0.0311005 | val_loss: 0.0311929 | Time: 4933.82 ms [2022-04-24 19:46:11 main:574] : INFO : Epoch 1776 | loss: 0.031102 | val_loss: 0.0311847 | Time: 4970.62 ms [2022-04-24 19:46:16 main:574] : INFO : Epoch 1777 | loss: 0.0311041 | val_loss: 0.031204 | Time: 4866.31 ms [2022-04-24 19:46:21 main:574] : INFO : Epoch 1778 | loss: 0.0311062 | val_loss: 0.0311912 | Time: 4861.36 ms [2022-04-24 19:46:26 main:574] : INFO : Epoch 1779 | loss: 0.0311073 | val_loss: 0.0311919 | Time: 4908.84 ms [2022-04-24 19:46:31 main:574] : INFO : Epoch 1780 | loss: 0.0311083 | val_loss: 0.0311914 | Time: 4851.74 ms [2022-04-24 19:46:36 main:574] : INFO : Epoch 1781 | loss: 0.0311033 | val_loss: 0.0311942 | Time: 4901.95 ms [2022-04-24 19:46:41 main:574] : INFO : Epoch 1782 | loss: 0.0311021 | val_loss: 0.0311898 | Time: 4945.73 ms [2022-04-24 19:46:46 main:574] : INFO : Epoch 1783 | loss: 0.031105 | val_loss: 0.0311867 | Time: 4967.81 ms [2022-04-24 19:46:51 main:574] : INFO : Epoch 1784 | loss: 0.0311121 | val_loss: 0.031186 | Time: 4896.44 ms [2022-04-24 19:46:56 main:574] : INFO : Epoch 1785 | loss: 0.0311396 | val_loss: 0.0311746 | Time: 4919.99 ms [2022-04-24 19:47:00 main:574] : INFO : Epoch 1786 | loss: 0.0311464 | val_loss: 0.0311745 | Time: 4862.21 ms [2022-04-24 19:47:05 main:574] : INFO : Epoch 1787 | loss: 0.031142 | val_loss: 0.0311732 | Time: 4952.61 ms [2022-04-24 19:47:10 main:574] : INFO : Epoch 1788 | loss: 0.0311342 | val_loss: 0.0311735 | Time: 4869.38 ms [2022-04-24 19:47:15 main:574] : INFO : Epoch 1789 | loss: 0.03113 | val_loss: 0.0311775 | Time: 4922.63 ms [2022-04-24 19:47:20 main:574] : INFO : Epoch 1790 | loss: 0.0311252 | val_loss: 0.0311848 | Time: 4913.64 ms [2022-04-24 19:47:25 main:574] : INFO : Epoch 1791 | loss: 0.0311237 | val_loss: 0.0311853 | Time: 4852.76 ms [2022-04-24 19:47:30 main:574] : INFO : Epoch 1792 | loss: 0.0311178 | val_loss: 0.0311851 | Time: 4924.24 ms [2022-04-24 19:47:35 main:574] : INFO : Epoch 1793 | loss: 0.0311159 | val_loss: 0.0311899 | Time: 4947.63 ms [2022-04-24 19:47:40 main:574] : INFO : Epoch 1794 | loss: 0.0311137 | val_loss: 0.0311848 | Time: 5024.71 ms [2022-04-24 19:47:45 main:574] : INFO : Epoch 1795 | loss: 0.0311144 | val_loss: 0.0311807 | Time: 4953.17 ms [2022-04-24 19:47:50 main:574] : INFO : Epoch 1796 | loss: 0.0311139 | val_loss: 0.0311829 | Time: 4967.86 ms [2022-04-24 19:47:55 main:574] : INFO : Epoch 1797 | loss: 0.0311122 | val_loss: 0.0311933 | Time: 4972.32 ms [2022-04-24 19:48:00 main:574] : INFO : Epoch 1798 | loss: 0.0311078 | val_loss: 0.0311853 | Time: 4928.18 ms [2022-04-24 19:48:05 main:574] : INFO : Epoch 1799 | loss: 0.0311106 | val_loss: 0.0311933 | Time: 5020.59 ms [2022-04-24 19:48:10 main:574] : INFO : Epoch 1800 | loss: 0.0311093 | val_loss: 0.031193 | Time: 5051.17 ms [2022-04-24 19:48:15 main:574] : INFO : Epoch 1801 | loss: 0.0311085 | val_loss: 0.0311865 | Time: 5024.23 ms [2022-04-24 19:48:20 main:574] : INFO : Epoch 1802 | loss: 0.0311089 | val_loss: 0.0311902 | Time: 4907.87 ms [2022-04-24 19:48:25 main:574] : INFO : Epoch 1803 | loss: 0.0311082 | val_loss: 0.0311871 | Time: 4900.22 ms [2022-04-24 19:48:30 main:574] : INFO : Epoch 1804 | loss: 0.031106 | val_loss: 0.0311907 | Time: 4877.62 ms [2022-04-24 19:48:35 main:574] : INFO : Epoch 1805 | loss: 0.0311077 | val_loss: 0.0311966 | Time: 4919.01 ms [2022-04-24 19:48:39 main:574] : INFO : Epoch 1806 | loss: 0.0311089 | val_loss: 0.031188 | Time: 4959.84 ms [2022-04-24 19:48:44 main:574] : INFO : Epoch 1807 | loss: 0.0311035 | val_loss: 0.0311921 | Time: 4920.74 ms [2022-04-24 19:48:49 main:574] : INFO : Epoch 1808 | loss: 0.0311064 | val_loss: 0.0311992 | Time: 4895.35 ms [2022-04-24 19:48:54 main:574] : INFO : Epoch 1809 | loss: 0.0311103 | val_loss: 0.0311852 | Time: 4939.17 ms [2022-04-24 19:48:59 main:574] : INFO : Epoch 1810 | loss: 0.0311126 | val_loss: 0.0311858 | Time: 4976.43 ms [2022-04-24 19:49:04 main:574] : INFO : Epoch 1811 | loss: 0.0311096 | val_loss: 0.0312004 | Time: 4989.84 ms [2022-04-24 19:49:09 main:574] : INFO : Epoch 1812 | loss: 0.0311171 | val_loss: 0.0311803 | Time: 4932.69 ms [2022-04-24 19:49:14 main:574] : INFO : Epoch 1813 | loss: 0.0311165 | val_loss: 0.0311968 | Time: 4973.47 ms [2022-04-24 19:49:19 main:574] : INFO : Epoch 1814 | loss: 0.0311193 | val_loss: 0.0311935 | Time: 4911.54 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 19:52:21 main:435] : INFO : Set logging level to 1 [2022-04-24 19:52:21 main:441] : INFO : Running in BOINC Client mode [2022-04-24 19:52:21 main:444] : INFO : Resolving all filenames [2022-04-24 19:52:21 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 19:52:21 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 19:52:21 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 19:52:21 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 19:52:21 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 19:52:21 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 19:52:21 main:474] : INFO : Configuration: [2022-04-24 19:52:21 main:475] : INFO : Model type: GRU [2022-04-24 19:52:21 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 19:52:21 main:477] : INFO : Max Epochs: 2048 [2022-04-24 19:52:21 main:478] : INFO : Batch Size: 128 [2022-04-24 19:52:21 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 19:52:21 main:480] : INFO : Patience: 10 [2022-04-24 19:52:21 main:481] : INFO : Hidden Width: 12 [2022-04-24 19:52:21 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 19:52:21 main:483] : INFO : # Backend Layers: 4 [2022-04-24 19:52:21 main:484] : INFO : # Threads: 1 [2022-04-24 19:52:21 main:486] : INFO : Preparing Dataset [2022-04-24 19:52:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 19:52:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 19:52:23 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 19:52:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 19:52:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 19:52:23 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 19:52:23 main:494] : INFO : Creating Model [2022-04-24 19:52:23 main:507] : INFO : Preparing config file [2022-04-24 19:52:23 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 19:52:23 main:512] : INFO : Loading config [2022-04-24 19:52:23 main:514] : INFO : Loading state [2022-04-24 19:52:24 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 19:52:24 main:562] : INFO : Starting Training [2022-04-24 19:52:29 main:574] : INFO : Epoch 1814 | loss: 0.0311441 | val_loss: 0.0311854 | Time: 5125.29 ms [2022-04-24 19:52:34 main:574] : INFO : Epoch 1815 | loss: 0.0311186 | val_loss: 0.0311958 | Time: 4930.73 ms [2022-04-24 19:52:39 main:574] : INFO : Epoch 1816 | loss: 0.0311132 | val_loss: 0.0311906 | Time: 4983.44 ms [2022-04-24 19:52:44 main:574] : INFO : Epoch 1817 | loss: 0.031109 | val_loss: 0.0311925 | Time: 4955.32 ms [2022-04-24 19:52:49 main:574] : INFO : Epoch 1818 | loss: 0.0311091 | val_loss: 0.0311942 | Time: 4894.13 ms [2022-04-24 19:52:54 main:574] : INFO : Epoch 1819 | loss: 0.0311132 | val_loss: 0.0311956 | Time: 4889.03 ms [2022-04-24 19:52:59 main:574] : INFO : Epoch 1820 | loss: 0.0311117 | val_loss: 0.0311859 | Time: 4826 ms [2022-04-24 19:53:04 main:574] : INFO : Epoch 1821 | loss: 0.0311091 | val_loss: 0.031185 | Time: 4884.86 ms [2022-04-24 19:53:09 main:574] : INFO : Epoch 1822 | loss: 0.0311105 | val_loss: 0.0311959 | Time: 4873.2 ms [2022-04-24 19:53:14 main:574] : INFO : Epoch 1823 | loss: 0.0311194 | val_loss: 0.031189 | Time: 4935 ms [2022-04-24 19:53:19 main:574] : INFO : Epoch 1824 | loss: 0.031125 | val_loss: 0.0311844 | Time: 4864.71 ms [2022-04-24 19:53:23 main:574] : INFO : Epoch 1825 | loss: 0.031117 | val_loss: 0.0311914 | Time: 4894.38 ms [2022-04-24 19:53:28 main:574] : INFO : Epoch 1826 | loss: 0.0311139 | val_loss: 0.031192 | Time: 4871.54 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 20:06:05 main:435] : INFO : Set logging level to 1 [2022-04-24 20:06:05 main:441] : INFO : Running in BOINC Client mode [2022-04-24 20:06:05 main:444] : INFO : Resolving all filenames [2022-04-24 20:06:05 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 20:06:05 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 20:06:05 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 20:06:05 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 20:06:05 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 20:06:05 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 20:06:05 main:474] : INFO : Configuration: [2022-04-24 20:06:05 main:475] : INFO : Model type: GRU [2022-04-24 20:06:05 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 20:06:05 main:477] : INFO : Max Epochs: 2048 [2022-04-24 20:06:05 main:478] : INFO : Batch Size: 128 [2022-04-24 20:06:05 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 20:06:05 main:480] : INFO : Patience: 10 [2022-04-24 20:06:05 main:481] : INFO : Hidden Width: 12 [2022-04-24 20:06:05 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 20:06:05 main:483] : INFO : # Backend Layers: 4 [2022-04-24 20:06:05 main:484] : INFO : # Threads: 1 [2022-04-24 20:06:05 main:486] : INFO : Preparing Dataset [2022-04-24 20:06:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 20:06:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 20:06:07 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 20:06:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 20:06:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 20:06:08 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 20:06:08 main:494] : INFO : Creating Model [2022-04-24 20:06:08 main:507] : INFO : Preparing config file [2022-04-24 20:06:08 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 20:06:08 main:512] : INFO : Loading config [2022-04-24 20:06:08 main:514] : INFO : Loading state [2022-04-24 20:06:09 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 20:06:09 main:562] : INFO : Starting Training [2022-04-24 20:06:15 main:574] : INFO : Epoch 1827 | loss: 0.0311509 | val_loss: 0.031208 | Time: 5676.57 ms [2022-04-24 20:06:20 main:574] : INFO : Epoch 1828 | loss: 0.0311169 | val_loss: 0.0311908 | Time: 4929.06 ms [2022-04-24 20:06:24 main:574] : INFO : Epoch 1829 | loss: 0.0311134 | val_loss: 0.0311894 | Time: 4883.13 ms [2022-04-24 20:06:30 main:574] : INFO : Epoch 1830 | loss: 0.031115 | val_loss: 0.03119 | Time: 5467.41 ms [2022-04-24 20:06:35 main:574] : INFO : Epoch 1831 | loss: 0.0311103 | val_loss: 0.0311825 | Time: 5064.74 ms [2022-04-24 20:06:40 main:574] : INFO : Epoch 1832 | loss: 0.0311093 | val_loss: 0.0311888 | Time: 5027.56 ms [2022-04-24 20:06:45 main:574] : INFO : Epoch 1833 | loss: 0.0311094 | val_loss: 0.0311851 | Time: 4890.35 ms [2022-04-24 20:06:50 main:574] : INFO : Epoch 1834 | loss: 0.0311128 | val_loss: 0.0311925 | Time: 4941.52 ms [2022-04-24 20:06:55 main:574] : INFO : Epoch 1835 | loss: 0.0311249 | val_loss: 0.0311857 | Time: 4912.34 ms [2022-04-24 20:07:00 main:574] : INFO : Epoch 1836 | loss: 0.0311267 | val_loss: 0.0311864 | Time: 4939.78 ms [2022-04-24 20:07:05 main:574] : INFO : Epoch 1837 | loss: 0.0311237 | val_loss: 0.0311933 | Time: 5021.34 ms [2022-04-24 20:07:10 main:574] : INFO : Epoch 1838 | loss: 0.0311158 | val_loss: 0.0311913 | Time: 5216.23 ms [2022-04-24 20:07:15 main:574] : INFO : Epoch 1839 | loss: 0.0311142 | val_loss: 0.0311935 | Time: 4969.81 ms [2022-04-24 20:07:20 main:574] : INFO : Epoch 1840 | loss: 0.0311147 | val_loss: 0.0311914 | Time: 4885.8 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 20:11:18 main:435] : INFO : Set logging level to 1 [2022-04-24 20:11:18 main:441] : INFO : Running in BOINC Client mode [2022-04-24 20:11:18 main:444] : INFO : Resolving all filenames [2022-04-24 20:11:18 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 20:11:18 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 20:11:18 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 20:11:18 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 20:11:18 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 20:11:18 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 20:11:18 main:474] : INFO : Configuration: [2022-04-24 20:11:18 main:475] : INFO : Model type: GRU [2022-04-24 20:11:18 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 20:11:18 main:477] : INFO : Max Epochs: 2048 [2022-04-24 20:11:18 main:478] : INFO : Batch Size: 128 [2022-04-24 20:11:18 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 20:11:18 main:480] : INFO : Patience: 10 [2022-04-24 20:11:18 main:481] : INFO : Hidden Width: 12 [2022-04-24 20:11:18 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 20:11:18 main:483] : INFO : # Backend Layers: 4 [2022-04-24 20:11:18 main:484] : INFO : # Threads: 1 [2022-04-24 20:11:18 main:486] : INFO : Preparing Dataset [2022-04-24 20:11:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 20:11:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 20:11:21 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 20:11:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 20:11:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 20:11:21 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 20:11:21 main:494] : INFO : Creating Model [2022-04-24 20:11:21 main:507] : INFO : Preparing config file [2022-04-24 20:11:21 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 20:11:21 main:512] : INFO : Loading config [2022-04-24 20:11:21 main:514] : INFO : Loading state [2022-04-24 20:11:22 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 20:11:22 main:562] : INFO : Starting Training [2022-04-24 20:11:27 main:574] : INFO : Epoch 1840 | loss: 0.0311527 | val_loss: 0.0311943 | Time: 5330.55 ms [2022-04-24 20:11:32 main:574] : INFO : Epoch 1841 | loss: 0.0311198 | val_loss: 0.0311864 | Time: 4973.82 ms [2022-04-24 20:11:37 main:574] : INFO : Epoch 1842 | loss: 0.0311122 | val_loss: 0.0311893 | Time: 4950.08 ms [2022-04-24 20:11:42 main:574] : INFO : Epoch 1843 | loss: 0.0311124 | val_loss: 0.0311884 | Time: 5008.63 ms [2022-04-24 20:11:47 main:574] : INFO : Epoch 1844 | loss: 0.0311161 | val_loss: 0.0311865 | Time: 5007.66 ms [2022-04-24 20:11:52 main:574] : INFO : Epoch 1845 | loss: 0.0311112 | val_loss: 0.0311866 | Time: 4993.61 ms [2022-04-24 20:11:57 main:574] : INFO : Epoch 1846 | loss: 0.0311112 | val_loss: 0.0311986 | Time: 4979.32 ms [2022-04-24 20:12:02 main:574] : INFO : Epoch 1847 | loss: 0.0311148 | val_loss: 0.0311892 | Time: 4984.28 ms [2022-04-24 20:12:07 main:574] : INFO : Epoch 1848 | loss: 0.0311218 | val_loss: 0.0311886 | Time: 4932.11 ms [2022-04-24 20:12:12 main:574] : INFO : Epoch 1849 | loss: 0.0311211 | val_loss: 0.0311934 | Time: 4935.45 ms [2022-04-24 20:12:17 main:574] : INFO : Epoch 1850 | loss: 0.0311138 | val_loss: 0.0311984 | Time: 5086.38 ms [2022-04-24 20:12:22 main:574] : INFO : Epoch 1851 | loss: 0.0311123 | val_loss: 0.0311868 | Time: 5170.53 ms [2022-04-24 20:12:28 main:574] : INFO : Epoch 1852 | loss: 0.0311104 | val_loss: 0.0311962 | Time: 5071.01 ms [2022-04-24 20:12:33 main:574] : INFO : Epoch 1853 | loss: 0.0311052 | val_loss: 0.0311879 | Time: 5061.99 ms [2022-04-24 20:12:38 main:574] : INFO : Epoch 1854 | loss: 0.0311097 | val_loss: 0.0311878 | Time: 5152.33 ms [2022-04-24 20:12:43 main:574] : INFO : Epoch 1855 | loss: 0.031109 | val_loss: 0.0311938 | Time: 5137.43 ms [2022-04-24 20:12:48 main:574] : INFO : Epoch 1856 | loss: 0.0311105 | val_loss: 0.0311926 | Time: 4986.33 ms [2022-04-24 20:12:53 main:574] : INFO : Epoch 1857 | loss: 0.0311198 | val_loss: 0.0311845 | Time: 4949.88 ms [2022-04-24 20:12:58 main:574] : INFO : Epoch 1858 | loss: 0.0311206 | val_loss: 0.0311888 | Time: 4947.82 ms [2022-04-24 20:13:03 main:574] : INFO : Epoch 1859 | loss: 0.0311125 | val_loss: 0.0311873 | Time: 4907.06 ms [2022-04-24 20:13:08 main:574] : INFO : Epoch 1860 | loss: 0.0311154 | val_loss: 0.0311829 | Time: 4937.77 ms [2022-04-24 20:13:13 main:574] : INFO : Epoch 1861 | loss: 0.0311249 | val_loss: 0.0311822 | Time: 4888.47 ms [2022-04-24 20:13:18 main:574] : INFO : Epoch 1862 | loss: 0.0311184 | val_loss: 0.0311944 | Time: 4939.43 ms [2022-04-24 20:13:22 main:574] : INFO : Epoch 1863 | loss: 0.0311168 | val_loss: 0.0311811 | Time: 4938.27 ms [2022-04-24 20:13:27 main:574] : INFO : Epoch 1864 | loss: 0.0311133 | val_loss: 0.0311847 | Time: 4966.82 ms [2022-04-24 20:13:32 main:574] : INFO : Epoch 1865 | loss: 0.0311235 | val_loss: 0.0311785 | Time: 4911.74 ms [2022-04-24 20:13:37 main:574] : INFO : Epoch 1866 | loss: 0.0311207 | val_loss: 0.0311827 | Time: 4922.58 ms [2022-04-24 20:13:42 main:574] : INFO : Epoch 1867 | loss: 0.0311196 | val_loss: 0.0311829 | Time: 5070.74 ms [2022-04-24 20:13:47 main:574] : INFO : Epoch 1868 | loss: 0.0311158 | val_loss: 0.0311843 | Time: 4917.96 ms [2022-04-24 20:13:52 main:574] : INFO : Epoch 1869 | loss: 0.0311103 | val_loss: 0.0311861 | Time: 4946.49 ms [2022-04-24 20:13:57 main:574] : INFO : Epoch 1870 | loss: 0.0311117 | val_loss: 0.0311908 | Time: 4927.63 ms [2022-04-24 20:14:02 main:574] : INFO : Epoch 1871 | loss: 0.0311067 | val_loss: 0.0311864 | Time: 4975.55 ms [2022-04-24 20:14:07 main:574] : INFO : Epoch 1872 | loss: 0.0311047 | val_loss: 0.0311926 | Time: 4985.24 ms [2022-04-24 20:14:12 main:574] : INFO : Epoch 1873 | loss: 0.031107 | val_loss: 0.0311899 | Time: 4923.83 ms [2022-04-24 20:14:17 main:574] : INFO : Epoch 1874 | loss: 0.0311033 | val_loss: 0.0311878 | Time: 4944.37 ms [2022-04-24 20:14:22 main:574] : INFO : Epoch 1875 | loss: 0.0311014 | val_loss: 0.0311926 | Time: 4935.29 ms [2022-04-24 20:14:27 main:574] : INFO : Epoch 1876 | loss: 0.0311001 | val_loss: 0.0311951 | Time: 4926.86 ms [2022-04-24 20:14:32 main:574] : INFO : Epoch 1877 | loss: 0.0311013 | val_loss: 0.0311845 | Time: 4879.61 ms [2022-04-24 20:14:37 main:574] : INFO : Epoch 1878 | loss: 0.0311063 | val_loss: 0.0312023 | Time: 4967.03 ms [2022-04-24 20:14:42 main:574] : INFO : Epoch 1879 | loss: 0.0311058 | val_loss: 0.0311917 | Time: 5032.2 ms [2022-04-24 20:14:47 main:574] : INFO : Epoch 1880 | loss: 0.0311021 | val_loss: 0.031193 | Time: 5003.63 ms [2022-04-24 20:14:52 main:574] : INFO : Epoch 1881 | loss: 0.0311016 | val_loss: 0.0311972 | Time: 4951.55 ms [2022-04-24 20:14:57 main:574] : INFO : Epoch 1882 | loss: 0.0310989 | val_loss: 0.0312049 | Time: 4958.51 ms [2022-04-24 20:15:02 main:574] : INFO : Epoch 1883 | loss: 0.0311056 | val_loss: 0.0311848 | Time: 4981.95 ms [2022-04-24 20:15:07 main:574] : INFO : Epoch 1884 | loss: 0.0311111 | val_loss: 0.0311903 | Time: 4954.37 ms [2022-04-24 20:15:12 main:574] : INFO : Epoch 1885 | loss: 0.0311101 | val_loss: 0.0311902 | Time: 4915.94 ms [2022-04-24 20:15:16 main:574] : INFO : Epoch 1886 | loss: 0.0311065 | val_loss: 0.0311933 | Time: 4853.23 ms [2022-04-24 20:15:21 main:574] : INFO : Epoch 1887 | loss: 0.0311033 | val_loss: 0.0311918 | Time: 4893.54 ms [2022-04-24 20:15:26 main:574] : INFO : Epoch 1888 | loss: 0.0311008 | val_loss: 0.0311894 | Time: 4855.41 ms [2022-04-24 20:15:31 main:574] : INFO : Epoch 1889 | loss: 0.0311047 | val_loss: 0.0312005 | Time: 4834.67 ms [2022-04-24 20:15:36 main:574] : INFO : Epoch 1890 | loss: 0.031109 | val_loss: 0.0311876 | Time: 4888.89 ms [2022-04-24 20:15:41 main:574] : INFO : Epoch 1891 | loss: 0.031105 | val_loss: 0.0311966 | Time: 4974.78 ms [2022-04-24 20:15:46 main:574] : INFO : Epoch 1892 | loss: 0.0311053 | val_loss: 0.0311891 | Time: 4932.5 ms [2022-04-24 20:15:51 main:574] : INFO : Epoch 1893 | loss: 0.0311029 | val_loss: 0.0311887 | Time: 4859.68 ms [2022-04-24 20:15:56 main:574] : INFO : Epoch 1894 | loss: 0.0311185 | val_loss: 0.0311851 | Time: 4960.62 ms [2022-04-24 20:16:01 main:574] : INFO : Epoch 1895 | loss: 0.0311201 | val_loss: 0.0311832 | Time: 4870.63 ms [2022-04-24 20:16:06 main:574] : INFO : Epoch 1896 | loss: 0.0311121 | val_loss: 0.0311922 | Time: 4958.83 ms [2022-04-24 20:16:10 main:574] : INFO : Epoch 1897 | loss: 0.0311099 | val_loss: 0.0311897 | Time: 4900.36 ms [2022-04-24 20:16:15 main:574] : INFO : Epoch 1898 | loss: 0.031107 | val_loss: 0.0311871 | Time: 4930.6 ms [2022-04-24 20:16:20 main:574] : INFO : Epoch 1899 | loss: 0.0311025 | val_loss: 0.0312013 | Time: 4998.5 ms [2022-04-24 20:16:25 main:574] : INFO : Epoch 1900 | loss: 0.0311016 | val_loss: 0.0311919 | Time: 4938.07 ms [2022-04-24 20:16:30 main:574] : INFO : Epoch 1901 | loss: 0.0311004 | val_loss: 0.0311943 | Time: 4936.91 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 20:19:33 main:435] : INFO : Set logging level to 1 [2022-04-24 20:19:33 main:441] : INFO : Running in BOINC Client mode [2022-04-24 20:19:33 main:444] : INFO : Resolving all filenames [2022-04-24 20:19:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 20:19:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 20:19:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 20:19:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 20:19:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 20:19:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 20:19:33 main:474] : INFO : Configuration: [2022-04-24 20:19:33 main:475] : INFO : Model type: GRU [2022-04-24 20:19:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 20:19:33 main:477] : INFO : Max Epochs: 2048 [2022-04-24 20:19:33 main:478] : INFO : Batch Size: 128 [2022-04-24 20:19:33 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 20:19:33 main:480] : INFO : Patience: 10 [2022-04-24 20:19:33 main:481] : INFO : Hidden Width: 12 [2022-04-24 20:19:33 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 20:19:33 main:483] : INFO : # Backend Layers: 4 [2022-04-24 20:19:33 main:484] : INFO : # Threads: 1 [2022-04-24 20:19:33 main:486] : INFO : Preparing Dataset [2022-04-24 20:19:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 20:19:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 20:19:35 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 20:19:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 20:19:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 20:19:35 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 20:19:35 main:494] : INFO : Creating Model [2022-04-24 20:19:35 main:507] : INFO : Preparing config file [2022-04-24 20:19:35 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 20:19:35 main:512] : INFO : Loading config [2022-04-24 20:19:35 main:514] : INFO : Loading state [2022-04-24 20:19:36 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 20:19:36 main:562] : INFO : Starting Training [2022-04-24 20:19:42 main:574] : INFO : Epoch 1894 | loss: 0.0311573 | val_loss: 0.031211 | Time: 5390.66 ms [2022-04-24 20:19:47 main:574] : INFO : Epoch 1895 | loss: 0.0311174 | val_loss: 0.0311845 | Time: 4948.98 ms [2022-04-24 20:19:52 main:574] : INFO : Epoch 1896 | loss: 0.0311124 | val_loss: 0.0311885 | Time: 4969.5 ms [2022-04-24 20:19:57 main:574] : INFO : Epoch 1897 | loss: 0.0311245 | val_loss: 0.031187 | Time: 4931.67 ms [2022-04-24 20:20:02 main:574] : INFO : Epoch 1898 | loss: 0.0311241 | val_loss: 0.0311909 | Time: 4869.18 ms [2022-04-24 20:20:07 main:574] : INFO : Epoch 1899 | loss: 0.0311138 | val_loss: 0.0311891 | Time: 4967.04 ms [2022-04-24 20:20:12 main:574] : INFO : Epoch 1900 | loss: 0.0311072 | val_loss: 0.031189 | Time: 4933.12 ms [2022-04-24 20:20:16 main:574] : INFO : Epoch 1901 | loss: 0.0311011 | val_loss: 0.0311932 | Time: 4957.11 ms [2022-04-24 20:20:21 main:574] : INFO : Epoch 1902 | loss: 0.0311025 | val_loss: 0.0311991 | Time: 4919.19 ms [2022-04-24 20:20:26 main:574] : INFO : Epoch 1903 | loss: 0.0311064 | val_loss: 0.0311808 | Time: 4933.45 ms [2022-04-24 20:20:31 main:574] : INFO : Epoch 1904 | loss: 0.03112 | val_loss: 0.0311773 | Time: 4912.12 ms [2022-04-24 20:20:36 main:574] : INFO : Epoch 1905 | loss: 0.0311171 | val_loss: 0.0311837 | Time: 4921.6 ms [2022-04-24 20:20:41 main:574] : INFO : Epoch 1906 | loss: 0.0311149 | val_loss: 0.0311762 | Time: 4938.2 ms [2022-04-24 20:20:46 main:574] : INFO : Epoch 1907 | loss: 0.0311115 | val_loss: 0.0311799 | Time: 5179.41 ms [2022-04-24 20:20:51 main:574] : INFO : Epoch 1908 | loss: 0.0311088 | val_loss: 0.0311806 | Time: 4948.89 ms [2022-04-24 20:20:56 main:574] : INFO : Epoch 1909 | loss: 0.0311394 | val_loss: 0.0312224 | Time: 4913.21 ms [2022-04-24 20:21:01 main:574] : INFO : Epoch 1910 | loss: 0.031188 | val_loss: 0.031198 | Time: 4959.92 ms [2022-04-24 20:21:06 main:574] : INFO : Epoch 1911 | loss: 0.0311715 | val_loss: 0.0311965 | Time: 4916.16 ms [2022-04-24 20:21:11 main:574] : INFO : Epoch 1912 | loss: 0.0311579 | val_loss: 0.0311853 | Time: 4937.42 ms [2022-04-24 20:21:16 main:574] : INFO : Epoch 1913 | loss: 0.0311488 | val_loss: 0.0311858 | Time: 4922.41 ms [2022-04-24 20:21:21 main:574] : INFO : Epoch 1914 | loss: 0.0311462 | val_loss: 0.0311821 | Time: 4953.98 ms [2022-04-24 20:21:26 main:574] : INFO : Epoch 1915 | loss: 0.0311417 | val_loss: 0.0311808 | Time: 4924.98 ms [2022-04-24 20:21:31 main:574] : INFO : Epoch 1916 | loss: 0.0311374 | val_loss: 0.0311812 | Time: 5032.15 ms [2022-04-24 20:21:36 main:574] : INFO : Epoch 1917 | loss: 0.0311338 | val_loss: 0.0311835 | Time: 4956.24 ms [2022-04-24 20:21:41 main:574] : INFO : Epoch 1918 | loss: 0.0311322 | val_loss: 0.0311819 | Time: 5108.96 ms [2022-04-24 20:21:46 main:574] : INFO : Epoch 1919 | loss: 0.0311325 | val_loss: 0.0311846 | Time: 4960.25 ms [2022-04-24 20:21:51 main:574] : INFO : Epoch 1920 | loss: 0.0311301 | val_loss: 0.0311843 | Time: 5090.21 ms [2022-04-24 20:21:56 main:574] : INFO : Epoch 1921 | loss: 0.0311277 | val_loss: 0.0311869 | Time: 5074.17 ms [2022-04-24 20:22:01 main:574] : INFO : Epoch 1922 | loss: 0.0311305 | val_loss: 0.0311831 | Time: 4854.83 ms [2022-04-24 20:22:06 main:574] : INFO : Epoch 1923 | loss: 0.0311286 | val_loss: 0.0311827 | Time: 4924.19 ms [2022-04-24 20:22:11 main:574] : INFO : Epoch 1924 | loss: 0.0311268 | val_loss: 0.0311868 | Time: 4894.52 ms [2022-04-24 20:22:16 main:574] : INFO : Epoch 1925 | loss: 0.0311239 | val_loss: 0.0311841 | Time: 4948.47 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 20:30:27 main:435] : INFO : Set logging level to 1 [2022-04-24 20:30:27 main:441] : INFO : Running in BOINC Client mode [2022-04-24 20:30:27 main:444] : INFO : Resolving all filenames [2022-04-24 20:30:27 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 20:30:27 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 20:30:27 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 20:30:27 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 20:30:27 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 20:30:27 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 20:30:27 main:474] : INFO : Configuration: [2022-04-24 20:30:27 main:475] : INFO : Model type: GRU [2022-04-24 20:30:27 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 20:30:27 main:477] : INFO : Max Epochs: 2048 [2022-04-24 20:30:27 main:478] : INFO : Batch Size: 128 [2022-04-24 20:30:27 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 20:30:27 main:480] : INFO : Patience: 10 [2022-04-24 20:30:27 main:481] : INFO : Hidden Width: 12 [2022-04-24 20:30:27 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 20:30:27 main:483] : INFO : # Backend Layers: 4 [2022-04-24 20:30:27 main:484] : INFO : # Threads: 1 [2022-04-24 20:30:27 main:486] : INFO : Preparing Dataset [2022-04-24 20:30:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 20:30:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 20:30:29 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 20:30:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 20:30:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 20:30:29 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 20:30:29 main:494] : INFO : Creating Model [2022-04-24 20:30:29 main:507] : INFO : Preparing config file [2022-04-24 20:30:29 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 20:30:29 main:512] : INFO : Loading config [2022-04-24 20:30:29 main:514] : INFO : Loading state [2022-04-24 20:30:31 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 20:30:31 main:562] : INFO : Starting Training [2022-04-24 20:30:36 main:574] : INFO : Epoch 1921 | loss: 0.0311728 | val_loss: 0.0311804 | Time: 4887.39 ms [2022-04-24 20:30:40 main:574] : INFO : Epoch 1922 | loss: 0.0311331 | val_loss: 0.0311881 | Time: 4722.21 ms [2022-04-24 20:30:45 main:574] : INFO : Epoch 1923 | loss: 0.031126 | val_loss: 0.0311918 | Time: 4692.13 ms [2022-04-24 20:30:50 main:574] : INFO : Epoch 1924 | loss: 0.0311225 | val_loss: 0.0311862 | Time: 4798.44 ms [2022-04-24 20:30:54 main:574] : INFO : Epoch 1925 | loss: 0.0311197 | val_loss: 0.0311966 | Time: 4613.33 ms [2022-04-24 20:30:59 main:574] : INFO : Epoch 1926 | loss: 0.0311205 | val_loss: 0.0311827 | Time: 4637.77 ms [2022-04-24 20:31:04 main:574] : INFO : Epoch 1927 | loss: 0.0311172 | val_loss: 0.0311886 | Time: 4658.15 ms [2022-04-24 20:31:08 main:574] : INFO : Epoch 1928 | loss: 0.0311128 | val_loss: 0.0311894 | Time: 4584.49 ms [2022-04-24 20:31:13 main:574] : INFO : Epoch 1929 | loss: 0.0311137 | val_loss: 0.0311917 | Time: 4576.66 ms [2022-04-24 20:31:18 main:574] : INFO : Epoch 1930 | loss: 0.0311179 | val_loss: 0.0311843 | Time: 4704.69 ms [2022-04-24 20:31:22 main:574] : INFO : Epoch 1931 | loss: 0.0311154 | val_loss: 0.031186 | Time: 4658.98 ms [2022-04-24 20:31:27 main:574] : INFO : Epoch 1932 | loss: 0.0311125 | val_loss: 0.0311873 | Time: 4626.53 ms [2022-04-24 20:31:33 main:574] : INFO : Epoch 1933 | loss: 0.0311134 | val_loss: 0.0311936 | Time: 6068.75 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080) [2022-04-24 20:38:46 main:435] : INFO : Set logging level to 1 [2022-04-24 20:38:46 main:441] : INFO : Running in BOINC Client mode [2022-04-24 20:38:46 main:444] : INFO : Resolving all filenames [2022-04-24 20:38:46 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-24 20:38:46 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-24 20:38:46 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-24 20:38:46 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-24 20:38:46 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-24 20:38:46 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-24 20:38:46 main:474] : INFO : Configuration: [2022-04-24 20:38:46 main:475] : INFO : Model type: GRU [2022-04-24 20:38:46 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-24 20:38:46 main:477] : INFO : Max Epochs: 2048 [2022-04-24 20:38:46 main:478] : INFO : Batch Size: 128 [2022-04-24 20:38:46 main:479] : INFO : Learning Rate: 0.01 [2022-04-24 20:38:46 main:480] : INFO : Patience: 10 [2022-04-24 20:38:46 main:481] : INFO : Hidden Width: 12 [2022-04-24 20:38:46 main:482] : INFO : # Recurrent Layers: 4 [2022-04-24 20:38:46 main:483] : INFO : # Backend Layers: 4 [2022-04-24 20:38:46 main:484] : INFO : # Threads: 1 [2022-04-24 20:38:46 main:486] : INFO : Preparing Dataset [2022-04-24 20:38:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-24 20:38:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-24 20:38:48 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-24 20:38:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-24 20:38:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-24 20:38:48 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-24 20:38:48 main:494] : INFO : Creating Model [2022-04-24 20:38:48 main:507] : INFO : Preparing config file [2022-04-24 20:38:48 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-24 20:38:48 main:512] : INFO : Loading config [2022-04-24 20:38:48 main:514] : INFO : Loading state [2022-04-24 20:38:49 main:559] : INFO : Loading DataLoader into Memory [2022-04-24 20:38:50 main:562] : INFO : Starting Training [2022-04-24 20:38:55 main:574] : INFO : Epoch 1934 | loss: 0.031153 | val_loss: 0.0311831 | Time: 4988.42 ms [2022-04-24 20:38:59 main:574] : INFO : Epoch 1935 | loss: 0.0311213 | val_loss: 0.0311949 | Time: 4715.07 ms [2022-04-24 20:39:04 main:574] : INFO : Epoch 1936 | loss: 0.031111 | val_loss: 0.0311877 | Time: 4695.95 ms [2022-04-24 20:39:09 main:574] : INFO : Epoch 1937 | loss: 0.031108 | val_loss: 0.0311914 | Time: 4771.57 ms [2022-04-24 20:39:13 main:574] : INFO : Epoch 1938 | loss: 0.031108 | val_loss: 0.0311949 | Time: 4762.77 ms [2022-04-24 20:39:18 main:574] : INFO : Epoch 1939 | loss: 0.0311118 | val_loss: 0.0311909 | Time: 4751.74 ms [2022-04-24 20:39:23 main:574] : INFO : Epoch 1940 | loss: 0.0311151 | val_loss: 0.0311936 | Time: 4701.77 ms [2022-04-24 20:39:28 main:574] : INFO : Epoch 1941 | loss: 0.0311114 | val_loss: 0.0311897 | Time: 4676.92 ms [2022-04-24 20:39:32 main:574] : INFO : Epoch 1942 | loss: 0.031113 | val_loss: 0.0311948 | Time: 4724.98 ms [2022-04-24 20:39:37 main:574] : INFO : Epoch 1943 | loss: 0.0311136 | val_loss: 0.0312021 | Time: 4772.02 ms [2022-04-24 20:39:42 main:574] : INFO : Epoch 1944 | loss: 0.0311286 | val_loss: 0.0311885 | Time: 4838.6 ms [2022-04-24 20:39:47 main:574] : INFO : Epoch 1945 | loss: 0.0311413 | val_loss: 0.0311828 | Time: 5208.96 ms [2022-04-24 20:39:52 main:574] : INFO : Epoch 1946 | loss: 0.0311411 | val_loss: 0.031181 | Time: 4763.34 ms [2022-04-24 20:39:57 main:574] : INFO : Epoch 1947 | loss: 0.0311319 | val_loss: 0.0311845 | Time: 4729.97 ms [2022-04-24 20:40:02 main:574] : INFO : Epoch 1948 | loss: 0.031129 | val_loss: 0.0311826 | Time: 4699.15 ms [2022-04-24 20:40:06 main:574] : INFO : Epoch 1949 | loss: 0.0311259 | val_loss: 0.0311781 | Time: 4739.19 ms [2022-04-24 20:40:11 main:574] : INFO : Epoch 1950 | loss: 0.0311276 | val_loss: 0.0311897 | Time: 4693.17 ms [2022-04-24 20:40:16 main:574] : INFO : Epoch 1951 | loss: 0.031127 | val_loss: 0.031178 | Time: 4762.81 ms [2022-04-24 20:40:20 main:574] : INFO : Epoch 1952 | loss: 0.0311257 | val_loss: 0.0311835 | Time: 4719.94 ms [2022-04-24 20:40:25 main:574] : INFO : Epoch 1953 | loss: 0.0311239 | val_loss: 0.0311842 | Time: 4673.08 ms [2022-04-24 20:40:30 main:574] : INFO : Epoch 1954 | loss: 0.0311211 | val_loss: 0.031189 | Time: 4708.79 ms [2022-04-24 20:40:35 main:574] : INFO : Epoch 1955 | loss: 0.0311182 | val_loss: 0.0311908 | Time: 4676.43 ms [2022-04-24 20:40:39 main:574] : INFO : Epoch 1956 | loss: 0.0311204 | val_loss: 0.031188 | Time: 4960.52 ms [2022-04-24 20:40:45 main:574] : INFO : Epoch 1957 | loss: 0.0311134 | val_loss: 0.0311811 | Time: 5001.86 ms [2022-04-24 20:40:50 main:574] : INFO : Epoch 1958 | loss: 0.0311134 | val_loss: 0.0311907 | Time: 5017.67 ms [2022-04-24 20:40:54 main:574] : INFO : Epoch 1959 | loss: 0.0311105 | val_loss: 0.0311883 | Time: 4895.89 ms [2022-04-24 20:40:59 main:574] : INFO : Epoch 1960 | loss: 0.031107 | val_loss: 0.0311933 | Time: 4912.81 ms [2022-04-24 20:41:04 main:574] : INFO : Epoch 1961 | loss: 0.0311109 | val_loss: 0.0311867 | Time: 4771.82 ms [2022-04-24 20:41:09 main:574] : INFO : Epoch 1962 | loss: 0.0311105 | val_loss: 0.031184 | Time: 4785.19 ms [2022-04-24 20:41:14 main:574] : INFO : Epoch 1963 | loss: 0.031106 | val_loss: 0.0311868 | Time: 4718.98 ms [2022-04-24 20:41:18 main:574] : INFO : Epoch 1964 | loss: 0.0311053 | val_loss: 0.0311931 | Time: 4738.41 ms [2022-04-24 20:41:23 main:574] : INFO : Epoch 1965 | loss: 0.0311041 | val_loss: 0.0311925 | Time: 4766.46 ms [2022-04-24 20:41:28 main:574] : INFO : Epoch 1966 | loss: 0.0311054 | val_loss: 0.0311856 | Time: 4741.93 ms [2022-04-24 20:41:33 main:574] : INFO : Epoch 1967 | loss: 0.0311087 | val_loss: 0.0311905 | Time: 4680.17 ms [2022-04-24 20:41:37 main:574] : INFO : Epoch 1968 | loss: 0.0311045 | val_loss: 0.0311928 | Time: 4798.85 ms [2022-04-24 20:41:42 main:574] : INFO : Epoch 1969 | loss: 0.0311031 | val_loss: 0.0311907 | Time: 4806.9 ms [2022-04-24 20:41:47 main:574] : INFO : Epoch 1970 | loss: 0.0311054 | val_loss: 0.0311982 | Time: 4856.86 ms [2022-04-24 20:41:52 main:574] : INFO : Epoch 1971 | loss: 0.0311065 | val_loss: 0.0311904 | Time: 4771.9 ms [2022-04-24 20:41:57 main:574] : INFO : Epoch 1972 | loss: 0.0311043 | val_loss: 0.0312008 | Time: 4857.99 ms [2022-04-24 20:42:01 main:574] : INFO : Epoch 1973 | loss: 0.0311113 | val_loss: 0.0311895 | Time: 4733.29 ms [2022-04-24 20:42:06 main:574] : INFO : Epoch 1974 | loss: 0.0311108 | val_loss: 0.0311867 | Time: 4749.91 ms [2022-04-24 20:42:11 main:574] : INFO : Epoch 1975 | loss: 0.0311095 | val_loss: 0.0311895 | Time: 4694.1 ms [2022-04-24 20:42:16 main:574] : INFO : Epoch 1976 | loss: 0.0311076 | val_loss: 0.0311927 | Time: 4691.59 ms [2022-04-24 20:42:20 main:574] : INFO : Epoch 1977 | loss: 0.0311068 | val_loss: 0.0311918 | Time: 4740.11 ms [2022-04-24 20:42:25 main:574] : INFO : Epoch 1978 | loss: 0.0311058 | val_loss: 0.0311906 | Time: 4668.51 ms [2022-04-24 20:42:30 main:574] : INFO : Epoch 1979 | loss: 0.031105 | val_loss: 0.0311935 | Time: 4762.25 ms [2022-04-24 20:42:35 main:574] : INFO : Epoch 1980 | loss: 0.0311037 | val_loss: 0.0311951 | Time: 4673.31 ms [2022-04-24 20:42:39 main:574] : INFO : Epoch 1981 | loss: 0.0311016 | val_loss: 0.031188 | Time: 4705.09 ms [2022-04-24 20:42:44 main:574] : INFO : Epoch 1982 | loss: 0.031104 | val_loss: 0.0311956 | Time: 4676.61 ms [2022-04-24 20:42:49 main:574] : INFO : Epoch 1983 | loss: 0.0311068 | val_loss: 0.0311864 | Time: 4708.42 ms [2022-04-24 20:42:53 main:574] : INFO : Epoch 1984 | loss: 0.0311056 | val_loss: 0.0311938 | Time: 4702.5 ms [2022-04-24 20:42:58 main:574] : INFO : Epoch 1985 | loss: 0.0311104 | val_loss: 0.0311983 | Time: 4752.07 ms [2022-04-24 20:43:03 main:574] : INFO : Epoch 1986 | loss: 0.0311179 | val_loss: 0.0311883 | Time: 4720.56 ms [2022-04-24 20:43:08 main:574] : INFO : Epoch 1987 | loss: 0.0311134 | val_loss: 0.0311878 | Time: 4711.82 ms [2022-04-24 20:43:12 main:574] : INFO : Epoch 1988 | loss: 0.0311183 | val_loss: 0.0311838 | Time: 4734.14 ms [2022-04-24 20:43:17 main:574] : INFO : Epoch 1989 | loss: 0.0311189 | val_loss: 0.0311961 | Time: 4661.07 ms [2022-04-24 20:43:22 main:574] : INFO : Epoch 1990 | loss: 0.0311217 | val_loss: 0.031195 | Time: 4745.39 ms [2022-04-24 20:43:26 main:574] : INFO : Epoch 1991 | loss: 0.0311167 | val_loss: 0.031189 | Time: 4688.15 ms [2022-04-24 20:43:31 main:574] : INFO : Epoch 1992 | loss: 0.0311109 | val_loss: 0.0311874 | Time: 4655.87 ms [2022-04-24 20:43:36 main:574] : INFO : Epoch 1993 | loss: 0.0311128 | val_loss: 0.0311861 | Time: 4687.13 ms [2022-04-24 20:43:40 main:574] : INFO : Epoch 1994 | loss: 0.0311169 | val_loss: 0.0311899 | Time: 4742.99 ms [2022-04-24 20:43:45 main:574] : INFO : Epoch 1995 | loss: 0.0311158 | val_loss: 0.0311905 | Time: 4710.53 ms [2022-04-24 20:43:50 main:574] : INFO : Epoch 1996 | loss: 0.031112 | val_loss: 0.0311933 | Time: 4605.43 ms [2022-04-24 20:43:55 main:574] : INFO : Epoch 1997 | loss: 0.0311212 | val_loss: 0.031195 | Time: 4812.77 ms [2022-04-24 20:43:59 main:574] : INFO : Epoch 1998 | loss: 0.0311319 | val_loss: 0.0311861 | Time: 4842.84 ms [2022-04-24 20:44:04 main:574] : INFO : Epoch 1999 | loss: 0.031128 | val_loss: 0.0311923 | Time: 4801.75 ms [2022-04-24 20:44:09 main:574] : INFO : Epoch 2000 | loss: 0.0311213 | val_loss: 0.0311876 | Time: 4860.43 ms [2022-04-24 20:44:14 main:574] : INFO : Epoch 2001 | loss: 0.0311188 | val_loss: 0.031197 | Time: 4837.55 ms [2022-04-24 20:44:19 main:574] : INFO : Epoch 2002 | loss: 0.031114 | val_loss: 0.0311932 | Time: 4787.98 ms [2022-04-24 20:44:24 main:574] : INFO : Epoch 2003 | loss: 0.0311141 | val_loss: 0.0311916 | Time: 4734.66 ms [2022-04-24 20:44:28 main:574] : INFO : Epoch 2004 | loss: 0.0311119 | val_loss: 0.0311899 | Time: 4781.44 ms [2022-04-24 20:44:33 main:574] : INFO : Epoch 2005 | loss: 0.0311098 | val_loss: 0.0311947 | Time: 4670.15 ms [2022-04-24 20:44:38 main:574] : INFO : Epoch 2006 | loss: 0.0311103 | val_loss: 0.0311949 | Time: 4741.71 ms [2022-04-24 20:44:43 main:574] : INFO : Epoch 2007 | loss: 0.0311069 | val_loss: 0.0311956 | Time: 4747 ms [2022-04-24 20:44:47 main:574] : INFO : Epoch 2008 | loss: 0.0311174 | val_loss: 0.0311824 | Time: 4785.37 ms [2022-04-24 20:44:52 main:574] : INFO : Epoch 2009 | loss: 0.031126 | val_loss: 0.0311891 | Time: 4690.08 ms [2022-04-24 20:44:57 main:574] : INFO : Epoch 2010 | loss: 0.0311329 | val_loss: 0.0311871 | Time: 4757.4 ms [2022-04-24 20:45:02 main:574] : INFO : Epoch 2011 | loss: 0.0311277 | val_loss: 0.0311831 | Time: 4749.68 ms [2022-04-24 20:45:06 main:574] : INFO : Epoch 2012 | loss: 0.0311181 | val_loss: 0.0311928 | Time: 4650.1 ms [2022-04-24 20:45:11 main:574] : INFO : Epoch 2013 | loss: 0.0311147 | val_loss: 0.0311866 | Time: 4723.35 ms [2022-04-24 20:45:16 main:574] : INFO : Epoch 2014 | loss: 0.0311138 | val_loss: 0.0311938 | Time: 4695.71 ms [2022-04-24 20:45:20 main:574] : INFO : Epoch 2015 | loss: 0.0311151 | val_loss: 0.031195 | Time: 4673.75 ms [2022-04-24 20:45:25 main:574] : INFO : Epoch 2016 | loss: 0.0311096 | val_loss: 0.0311878 | Time: 4753.48 ms [2022-04-24 20:45:30 main:574] : INFO : Epoch 2017 | loss: 0.0311107 | val_loss: 0.0311955 | Time: 4667.11 ms [2022-04-24 20:45:34 main:574] : INFO : Epoch 2018 | loss: 0.0311103 | val_loss: 0.0311942 | Time: 4663.17 ms [2022-04-24 20:45:39 main:574] : INFO : Epoch 2019 | loss: 0.0311184 | val_loss: 0.0311847 | Time: 4713.96 ms [2022-04-24 20:45:44 main:574] : INFO : Epoch 2020 | loss: 0.0311357 | val_loss: 0.0311762 | Time: 4708.99 ms [2022-04-24 20:45:49 main:574] : INFO : Epoch 2021 | loss: 0.0311304 | val_loss: 0.0311775 | Time: 4710.95 ms [2022-04-24 20:45:53 main:574] : INFO : Epoch 2022 | loss: 0.0311272 | val_loss: 0.0311874 | Time: 4709.71 ms [2022-04-24 20:45:58 main:574] : INFO : Epoch 2023 | loss: 0.0311375 | val_loss: 0.0311803 | Time: 4753.55 ms [2022-04-24 20:46:03 main:574] : INFO : Epoch 2024 | loss: 0.0311365 | val_loss: 0.0311779 | Time: 4700.72 ms [2022-04-24 20:46:07 main:574] : INFO : Epoch 2025 | loss: 0.0311287 | val_loss: 0.0311831 | Time: 4724.26 ms [2022-04-24 20:46:12 main:574] : INFO : Epoch 2026 | loss: 0.0311276 | val_loss: 0.0311758 | Time: 4714.51 ms [2022-04-24 20:46:17 main:574] : INFO : Epoch 2027 | loss: 0.031123 | val_loss: 0.0311778 | Time: 4768.18 ms [2022-04-24 20:46:22 main:574] : INFO : Epoch 2028 | loss: 0.0311189 | val_loss: 0.0311814 | Time: 4763.56 ms [2022-04-24 20:46:26 main:574] : INFO : Epoch 2029 | loss: 0.0311186 | val_loss: 0.0311837 | Time: 4768.62 ms [2022-04-24 20:46:31 main:574] : INFO : Epoch 2030 | loss: 0.0311171 | val_loss: 0.0311883 | Time: 4765.78 ms [2022-04-24 20:46:36 main:574] : INFO : Epoch 2031 | loss: 0.0311286 | val_loss: 0.0311803 | Time: 4699.32 ms [2022-04-24 20:46:41 main:574] : INFO : Epoch 2032 | loss: 0.0311386 | val_loss: 0.0311794 | Time: 4802.37 ms [2022-04-24 20:46:45 main:574] : INFO : Epoch 2033 | loss: 0.0311348 | val_loss: 0.0311791 | Time: 4651.46 ms [2022-04-24 20:46:50 main:574] : INFO : Epoch 2034 | loss: 0.031131 | val_loss: 0.0311785 | Time: 4736.43 ms [2022-04-24 20:46:55 main:574] : INFO : Epoch 2035 | loss: 0.0311281 | val_loss: 0.031182 | Time: 4702.22 ms [2022-04-24 20:47:00 main:574] : INFO : Epoch 2036 | loss: 0.0311406 | val_loss: 0.0311727 | Time: 4697.84 ms [2022-04-24 20:47:04 main:574] : INFO : Epoch 2037 | loss: 0.0311466 | val_loss: 0.0311747 | Time: 4766.17 ms [2022-04-24 20:47:09 main:574] : INFO : Epoch 2038 | loss: 0.0311417 | val_loss: 0.0311766 | Time: 4676.36 ms [2022-04-24 20:47:14 main:574] : INFO : Epoch 2039 | loss: 0.0311388 | val_loss: 0.0311753 | Time: 4813.93 ms [2022-04-24 20:47:19 main:574] : INFO : Epoch 2040 | loss: 0.0311343 | val_loss: 0.0311697 | Time: 4728.38 ms [2022-04-24 20:47:23 main:574] : INFO : Epoch 2041 | loss: 0.0311301 | val_loss: 0.0311817 | Time: 4718.53 ms [2022-04-24 20:47:28 main:574] : INFO : Epoch 2042 | loss: 0.0311304 | val_loss: 0.0311732 | Time: 4780.21 ms [2022-04-24 20:47:33 main:574] : INFO : Epoch 2043 | loss: 0.0311285 | val_loss: 0.0311792 | Time: 4723.32 ms [2022-04-24 20:47:38 main:574] : INFO : Epoch 2044 | loss: 0.0311273 | val_loss: 0.0311748 | Time: 4726.86 ms [2022-04-24 20:47:42 main:574] : INFO : Epoch 2045 | loss: 0.0311244 | val_loss: 0.031179 | Time: 4776.74 ms [2022-04-24 20:47:47 main:574] : INFO : Epoch 2046 | loss: 0.0311214 | val_loss: 0.0311868 | Time: 4680.15 ms [2022-04-24 20:47:52 main:574] : INFO : Epoch 2047 | loss: 0.0311217 | val_loss: 0.0311819 | Time: 4737.05 ms [2022-04-24 20:47:57 main:574] : INFO : Epoch 2048 | loss: 0.0311191 | val_loss: 0.0311949 | Time: 4739.2 ms [2022-04-24 20:47:57 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311949 [2022-04-24 20:47:57 main:603] : INFO : Saving end state to config to file [2022-04-24 20:47:57 main:608] : INFO : Success, exiting.. 20:47:57 (131252): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)