Task 12425201
| Name | ParityModified-1645998465-10096-1-0_0 |
| Workunit | 9594839 |
| Created | 2 Mar 2022, 23:27:42 UTC |
| Sent | 10 Mar 2022, 18:25:07 UTC |
| Report deadline | 18 Mar 2022, 18:25:07 UTC |
| Received | 23 Mar 2022, 17:57:55 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 20730 |
| Run time | 56 min 52 sec |
| CPU time | 55 min 31 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 2,197.48 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.63 GB |
| Peak swap size | 3.59 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> -03-22 16:12:30 main:574] : INFO : Epoch 1734 | loss: 0.0312167 | val_loss: 0.0312104 | Time: 1676.2 ms [2022-03-22 16:12:32 main:574] : INFO : Epoch 1735 | loss: 0.0312144 | val_loss: 0.0312068 | Time: 1675.48 ms [2022-03-22 16:12:34 main:574] : INFO : Epoch 1736 | loss: 0.0312111 | val_loss: 0.0312046 | Time: 1672.05 ms [2022-03-22 16:12:35 main:574] : INFO : Epoch 1737 | loss: 0.0312085 | val_loss: 0.0312037 | Time: 1700.31 ms [2022-03-22 16:12:37 main:574] : INFO : Epoch 1738 | loss: 0.0312081 | val_loss: 0.0312024 | Time: 1674.02 ms [2022-03-22 16:12:39 main:574] : INFO : Epoch 1739 | loss: 0.0312053 | val_loss: 0.0312031 | Time: 1673.3 ms [2022-03-22 16:12:40 main:574] : INFO : Epoch 1740 | loss: 0.0312049 | val_loss: 0.0312033 | Time: 1676.66 ms [2022-03-22 16:12:42 main:574] : INFO : Epoch 1741 | loss: 0.0312044 | val_loss: 0.0312033 | Time: 1674.26 ms [2022-03-22 16:12:44 main:574] : INFO : Epoch 1742 | loss: 0.0312028 | val_loss: 0.0312023 | Time: 1675.5 ms [2022-03-22 16:12:45 main:574] : INFO : Epoch 1743 | loss: 0.0312027 | val_loss: 0.0312014 | Time: 1667.78 ms [2022-03-22 16:12:47 main:574] : INFO : Epoch 1744 | loss: 0.0312009 | val_loss: 0.0312003 | Time: 1673.37 ms [2022-03-22 16:12:49 main:574] : INFO : Epoch 1745 | loss: 0.0312003 | val_loss: 0.0311983 | Time: 1673.66 ms [2022-03-22 16:12:50 main:574] : INFO : Epoch 1746 | loss: 0.0311997 | val_loss: 0.031201 | Time: 1668.82 ms [2022-03-22 16:12:52 main:574] : INFO : Epoch 1747 | loss: 0.0312002 | val_loss: 0.031199 | Time: 1669.97 ms [2022-03-22 16:12:54 main:574] : INFO : Epoch 1748 | loss: 0.0311983 | val_loss: 0.0311989 | Time: 1670.33 ms [2022-03-22 16:12:55 main:574] : INFO : Epoch 1749 | loss: 0.0311993 | val_loss: 0.0312003 | Time: 1675.13 ms [2022-03-22 16:12:57 main:574] : INFO : Epoch 1750 | loss: 0.0312038 | val_loss: 0.0312018 | Time: 1678.19 ms [2022-03-22 16:12:59 main:574] : INFO : Epoch 1751 | loss: 0.0312045 | val_loss: 0.0312099 | Time: 1674.92 ms [2022-03-22 16:13:00 main:574] : INFO : Epoch 1752 | loss: 0.0312027 | val_loss: 0.0311987 | Time: 1680.34 ms [2022-03-22 16:13:02 main:574] : INFO : Epoch 1753 | loss: 0.0311977 | val_loss: 0.0311973 | Time: 1675.29 ms [2022-03-22 16:13:04 main:574] : INFO : Epoch 1754 | loss: 0.0311978 | val_loss: 0.0311968 | Time: 1706.32 ms [2022-03-22 16:13:05 main:574] : INFO : Epoch 1755 | loss: 0.0311964 | val_loss: 0.0311973 | Time: 1671.93 ms [2022-03-22 16:13:07 main:574] : INFO : Epoch 1756 | loss: 0.0311969 | val_loss: 0.0312001 | Time: 1679.74 ms [2022-03-22 16:13:09 main:574] : INFO : Epoch 1757 | loss: 0.0311967 | val_loss: 0.0311973 | Time: 1677.46 ms [2022-03-22 16:13:10 main:574] : INFO : Epoch 1758 | loss: 0.0311948 | val_loss: 0.031196 | Time: 1673.4 ms [2022-03-22 16:13:12 main:574] : INFO : Epoch 1759 | loss: 0.0311948 | val_loss: 0.0311943 | Time: 1676.05 ms [2022-03-22 16:13:14 main:574] : INFO : Epoch 1760 | loss: 0.0311934 | val_loss: 0.0311939 | Time: 1669.01 ms [2022-03-22 16:13:16 main:574] : INFO : Epoch 1761 | loss: 0.031193 | val_loss: 0.0311933 | Time: 1676.49 ms [2022-03-22 16:13:17 main:574] : INFO : Epoch 1762 | loss: 0.0311927 | val_loss: 0.0311939 | Time: 1676.91 ms [2022-03-22 16:13:19 main:574] : INFO : Epoch 1763 | loss: 0.0311939 | val_loss: 0.0311937 | Time: 1668.89 ms [2022-03-22 16:13:21 main:574] : INFO : Epoch 1764 | loss: 0.0311944 | val_loss: 0.0311932 | Time: 1673.1 ms [2022-03-22 16:13:22 main:574] : INFO : Epoch 1765 | loss: 0.0311936 | val_loss: 0.0311946 | Time: 1673.12 ms [2022-03-22 16:13:24 main:574] : INFO : Epoch 1766 | loss: 0.0311938 | val_loss: 0.0311939 | Time: 1677.16 ms [2022-03-22 16:13:26 main:574] : INFO : Epoch 1767 | loss: 0.0311937 | val_loss: 0.0311929 | Time: 1677.97 ms [2022-03-22 16:13:27 main:574] : INFO : Epoch 1768 | loss: 0.0311935 | val_loss: 0.0311966 | Time: 1675.91 ms [2022-03-22 16:13:29 main:574] : INFO : Epoch 1769 | loss: 0.0311949 | val_loss: 0.0311909 | Time: 1670.81 ms [2022-03-22 16:13:31 main:574] : INFO : Epoch 1770 | loss: 0.0311909 | val_loss: 0.03119 | Time: 1675.23 ms [2022-03-22 16:13:32 main:574] : INFO : Epoch 1771 | loss: 0.0311898 | val_loss: 0.0311903 | Time: 1704.08 ms [2022-03-22 16:13:34 main:574] : INFO : Epoch 1772 | loss: 0.0311892 | val_loss: 0.0311896 | Time: 1681.31 ms [2022-03-22 16:13:36 main:574] : INFO : Epoch 1773 | loss: 0.0311905 | val_loss: 0.031192 | Time: 1675.77 ms [2022-03-22 16:13:37 main:574] : INFO : Epoch 1774 | loss: 0.031191 | val_loss: 0.0311915 | Time: 1675.67 ms [2022-03-22 16:13:39 main:574] : INFO : Epoch 1775 | loss: 0.0311878 | val_loss: 0.0311893 | Time: 1675.6 ms [2022-03-22 16:13:41 main:574] : INFO : Epoch 1776 | loss: 0.0311878 | val_loss: 0.0311893 | Time: 1673.04 ms [2022-03-22 16:13:42 main:574] : INFO : Epoch 1777 | loss: 0.0311892 | val_loss: 0.0311933 | Time: 1670.05 ms [2022-03-22 16:13:44 main:574] : INFO : Epoch 1778 | loss: 0.0311933 | val_loss: 0.0311944 | Time: 1676.25 ms [2022-03-22 16:13:46 main:574] : INFO : Epoch 1779 | loss: 0.0311929 | val_loss: 0.0311954 | Time: 1675.22 ms [2022-03-22 16:13:47 main:574] : INFO : Epoch 1780 | loss: 0.0311921 | val_loss: 0.0311978 | Time: 1671.62 ms [2022-03-22 16:13:49 main:574] : INFO : Epoch 1781 | loss: 0.0311918 | val_loss: 0.031194 | Time: 1677.66 ms [2022-03-22 16:13:51 main:574] : INFO : Epoch 1782 | loss: 0.0311916 | val_loss: 0.0311969 | Time: 1673.52 ms [2022-03-22 16:13:52 main:574] : INFO : Epoch 1783 | loss: 0.0311932 | val_loss: 0.0311951 | Time: 1674.9 ms [2022-03-22 16:13:54 main:574] : INFO : Epoch 1784 | loss: 0.031193 | val_loss: 0.0311946 | Time: 1674.52 ms [2022-03-22 16:13:56 main:574] : INFO : Epoch 1785 | loss: 0.0311921 | val_loss: 0.0311931 | Time: 1670.59 ms [2022-03-22 16:13:57 main:574] : INFO : Epoch 1786 | loss: 0.0311923 | val_loss: 0.0311954 | Time: 1675.67 ms [2022-03-22 16:13:59 main:574] : INFO : Epoch 1787 | loss: 0.0311922 | val_loss: 0.0311951 | Time: 1676.77 ms [2022-03-22 16:14:01 main:574] : INFO : Epoch 1788 | loss: 0.0311907 | val_loss: 0.0311925 | Time: 1669.9 ms [2022-03-22 16:14:03 main:574] : INFO : Epoch 1789 | loss: 0.0311912 | val_loss: 0.0311928 | Time: 1702.95 ms [2022-03-22 16:14:04 main:574] : INFO : Epoch 1790 | loss: 0.0311909 | val_loss: 0.0311911 | Time: 1671.66 ms [2022-03-22 16:14:06 main:574] : INFO : Epoch 1791 | loss: 0.0311905 | val_loss: 0.0311932 | Time: 1675.48 ms [2022-03-22 16:14:08 main:574] : INFO : Epoch 1792 | loss: 0.0311973 | val_loss: 0.031197 | Time: 1672.81 ms [2022-03-22 16:14:09 main:574] : INFO : Epoch 1793 | loss: 0.0311967 | val_loss: 0.0311972 | Time: 1674.73 ms [2022-03-22 16:14:11 main:574] : INFO : Epoch 1794 | loss: 0.0311951 | val_loss: 0.0311979 | Time: 1667.26 ms [2022-03-22 16:14:13 main:574] : INFO : Epoch 1795 | loss: 0.0311963 | val_loss: 0.0311948 | Time: 1695.89 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-03-23 11:56:17 main:435] : INFO : Set logging level to 1 [2022-03-23 11:56:17 main:441] : INFO : Running in BOINC Client mode [2022-03-23 11:56:17 main:444] : INFO : Resolving all filenames [2022-03-23 11:56:17 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 11:56:17 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 11:56:17 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 11:56:17 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 11:56:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 11:56:17 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 11:56:17 main:474] : INFO : Configuration: [2022-03-23 11:56:17 main:475] : INFO : Model type: GRU [2022-03-23 11:56:17 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 11:56:17 main:477] : INFO : Max Epochs: 2048 [2022-03-23 11:56:17 main:478] : INFO : Batch Size: 128 [2022-03-23 11:56:17 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 11:56:17 main:480] : INFO : Patience: 10 [2022-03-23 11:56:17 main:481] : INFO : Hidden Width: 12 [2022-03-23 11:56:17 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 11:56:17 main:483] : INFO : # Backend Layers: 4 [2022-03-23 11:56:17 main:484] : INFO : # Threads: 1 [2022-03-23 11:56:17 main:486] : INFO : Preparing Dataset [2022-03-23 11:56:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 11:56:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-23 11:56:20 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-23 11:56:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-23 11:56:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-23 11:56:20 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-23 11:56:20 main:494] : INFO : Creating Model [2022-03-23 11:56:20 main:507] : INFO : Preparing config file [2022-03-23 11:56:20 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-23 11:56:20 main:512] : INFO : Loading config [2022-03-23 11:56:20 main:514] : INFO : Loading state [2022-03-23 11:56:22 main:559] : INFO : Loading DataLoader into Memory [2022-03-23 11:56:22 main:562] : INFO : Starting Training [2022-03-23 11:56:24 main:574] : INFO : Epoch 1751 | loss: 0.031231 | val_loss: 0.0312082 | Time: 2350.35 ms [2022-03-23 11:56:26 main:574] : INFO : Epoch 1752 | loss: 0.0312006 | val_loss: 0.0311959 | Time: 1633.84 ms [2022-03-23 11:56:27 main:574] : INFO : Epoch 1753 | loss: 0.0311947 | val_loss: 0.0311926 | Time: 1640.3 ms [2022-03-23 11:56:29 main:574] : INFO : Epoch 1754 | loss: 0.03119 | val_loss: 0.0311882 | Time: 1632.33 ms [2022-03-23 11:56:31 main:574] : INFO : Epoch 1755 | loss: 0.0311883 | val_loss: 0.0311904 | Time: 1637.21 ms [2022-03-23 11:56:32 main:574] : INFO : Epoch 1756 | loss: 0.0311887 | val_loss: 0.0311913 | Time: 1643.92 ms [2022-03-23 11:56:34 main:574] : INFO : Epoch 1757 | loss: 0.0311863 | val_loss: 0.03119 | Time: 1640.7 ms [2022-03-23 11:56:36 main:574] : INFO : Epoch 1758 | loss: 0.0311859 | val_loss: 0.0311889 | Time: 1632.86 ms [2022-03-23 11:56:37 main:574] : INFO : Epoch 1759 | loss: 0.0311856 | val_loss: 0.0311881 | Time: 1666.96 ms [2022-03-23 11:56:39 main:574] : INFO : Epoch 1760 | loss: 0.0311848 | val_loss: 0.0311859 | Time: 1643.09 ms [2022-03-23 11:56:41 main:574] : INFO : Epoch 1761 | loss: 0.031183 | val_loss: 0.031185 | Time: 1637.61 ms [2022-03-23 11:56:42 main:574] : INFO : Epoch 1762 | loss: 0.0311835 | val_loss: 0.0311883 | Time: 1642.13 ms [2022-03-23 11:56:44 main:574] : INFO : Epoch 1763 | loss: 0.0311884 | val_loss: 0.0311877 | Time: 1641.92 ms [2022-03-23 11:56:45 main:574] : INFO : Epoch 1764 | loss: 0.0311892 | val_loss: 0.0311869 | Time: 1641.4 ms [2022-03-23 11:56:47 main:574] : INFO : Epoch 1765 | loss: 0.0311913 | val_loss: 0.0311907 | Time: 1645.17 ms [2022-03-23 11:56:49 main:574] : INFO : Epoch 1766 | loss: 0.0311964 | val_loss: 0.0311949 | Time: 1642.86 ms [2022-03-23 11:56:50 main:574] : INFO : Epoch 1767 | loss: 0.0311968 | val_loss: 0.0311943 | Time: 1645.34 ms [2022-03-23 11:56:52 main:574] : INFO : Epoch 1768 | loss: 0.031194 | val_loss: 0.0311959 | Time: 1643.02 ms [2022-03-23 11:56:54 main:574] : INFO : Epoch 1769 | loss: 0.0311918 | val_loss: 0.0311932 | Time: 1639.89 ms [2022-03-23 11:56:55 main:574] : INFO : Epoch 1770 | loss: 0.0311917 | val_loss: 0.0311913 | Time: 1672.73 ms [2022-03-23 11:56:57 main:574] : INFO : Epoch 1771 | loss: 0.0311899 | val_loss: 0.0311921 | Time: 1647 ms [2022-03-23 11:56:59 main:574] : INFO : Epoch 1772 | loss: 0.0311895 | val_loss: 0.0311906 | Time: 1643.71 ms [2022-03-23 11:57:00 main:574] : INFO : Epoch 1773 | loss: 0.0311887 | val_loss: 0.0311917 | Time: 1648.69 ms [2022-03-23 11:57:02 main:574] : INFO : Epoch 1774 | loss: 0.0311887 | val_loss: 0.03119 | Time: 1642.25 ms [2022-03-23 11:57:04 main:574] : INFO : Epoch 1775 | loss: 0.0311874 | val_loss: 0.0311883 | Time: 1644.41 ms [2022-03-23 11:57:05 main:574] : INFO : Epoch 1776 | loss: 0.0311876 | val_loss: 0.0311923 | Time: 1661.68 ms [2022-03-23 11:57:07 main:574] : INFO : Epoch 1777 | loss: 0.0311894 | val_loss: 0.0311869 | Time: 1654.42 ms [2022-03-23 11:57:09 main:574] : INFO : Epoch 1778 | loss: 0.0311858 | val_loss: 0.0311865 | Time: 1641.93 ms [2022-03-23 11:57:10 main:574] : INFO : Epoch 1779 | loss: 0.0311863 | val_loss: 0.0311905 | Time: 1641.85 ms [2022-03-23 11:57:12 main:574] : INFO : Epoch 1780 | loss: 0.0311852 | val_loss: 0.0311886 | Time: 1647.81 ms [2022-03-23 11:57:14 main:574] : INFO : Epoch 1781 | loss: 0.0311846 | val_loss: 0.0311891 | Time: 1647.42 ms [2022-03-23 11:57:15 main:574] : INFO : Epoch 1782 | loss: 0.031183 | val_loss: 0.0311863 | Time: 1647.16 ms [2022-03-23 11:57:17 main:574] : INFO : Epoch 1783 | loss: 0.0311825 | val_loss: 0.0311868 | Time: 1700.85 ms [2022-03-23 11:57:19 main:574] : INFO : Epoch 1784 | loss: 0.0311827 | val_loss: 0.0311858 | Time: 1666.31 ms [2022-03-23 11:57:20 main:574] : INFO : Epoch 1785 | loss: 0.0311817 | val_loss: 0.0311861 | Time: 1661.46 ms [2022-03-23 11:57:22 main:574] : INFO : Epoch 1786 | loss: 0.0311825 | val_loss: 0.0311855 | Time: 1662.64 ms [2022-03-23 11:57:24 main:574] : INFO : Epoch 1787 | loss: 0.0311817 | val_loss: 0.0311855 | Time: 1659.82 ms [2022-03-23 11:57:25 main:574] : INFO : Epoch 1788 | loss: 0.0311833 | val_loss: 0.031189 | Time: 1652.67 ms [2022-03-23 11:57:27 main:574] : INFO : Epoch 1789 | loss: 0.0311818 | val_loss: 0.0311849 | Time: 1648.67 ms [2022-03-23 11:57:28 main:574] : INFO : Epoch 1790 | loss: 0.0311811 | val_loss: 0.0311865 | Time: 1644.43 ms [2022-03-23 11:57:30 main:574] : INFO : Epoch 1791 | loss: 0.0311817 | val_loss: 0.0311855 | Time: 1643.92 ms [2022-03-23 11:57:32 main:574] : INFO : Epoch 1792 | loss: 0.0311818 | val_loss: 0.0311871 | Time: 1650.15 ms [2022-03-23 11:57:33 main:574] : INFO : Epoch 1793 | loss: 0.0311812 | val_loss: 0.0311857 | Time: 1650.82 ms [2022-03-23 11:57:35 main:574] : INFO : Epoch 1794 | loss: 0.031181 | val_loss: 0.0311883 | Time: 1678.52 ms [2022-03-23 11:57:37 main:574] : INFO : Epoch 1795 | loss: 0.0311816 | val_loss: 0.0311854 | Time: 1649.22 ms [2022-03-23 11:57:38 main:574] : INFO : Epoch 1796 | loss: 0.0311807 | val_loss: 0.0311854 | Time: 1644.98 ms [2022-03-23 11:57:40 main:574] : INFO : Epoch 1797 | loss: 0.0311799 | val_loss: 0.031185 | Time: 1642.64 ms [2022-03-23 11:57:42 main:574] : INFO : Epoch 1798 | loss: 0.0311808 | val_loss: 0.0311859 | Time: 1645.86 ms [2022-03-23 11:57:43 main:574] : INFO : Epoch 1799 | loss: 0.0311797 | val_loss: 0.031185 | Time: 1645.68 ms [2022-03-23 11:57:45 main:574] : INFO : Epoch 1800 | loss: 0.0311791 | val_loss: 0.0311852 | Time: 1650.15 ms [2022-03-23 11:57:47 main:574] : INFO : Epoch 1801 | loss: 0.0311795 | val_loss: 0.0311863 | Time: 1642.36 ms [2022-03-23 11:57:48 main:574] : INFO : Epoch 1802 | loss: 0.0311784 | val_loss: 0.0311847 | Time: 1646.93 ms [2022-03-23 11:57:50 main:574] : INFO : Epoch 1803 | loss: 0.0311785 | val_loss: 0.0311856 | Time: 1690.96 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-03-23 12:10:31 main:435] : INFO : Set logging level to 1 [2022-03-23 12:10:31 main:441] : INFO : Running in BOINC Client mode [2022-03-23 12:10:31 main:444] : INFO : Resolving all filenames [2022-03-23 12:10:31 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 12:10:31 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 12:10:31 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 12:10:31 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 12:10:31 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 12:10:31 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 12:10:31 main:474] : INFO : Configuration: [2022-03-23 12:10:31 main:475] : INFO : Model type: GRU [2022-03-23 12:10:31 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 12:10:31 main:477] : INFO : Max Epochs: 2048 [2022-03-23 12:10:31 main:478] : INFO : Batch Size: 128 [2022-03-23 12:10:31 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 12:10:31 main:480] : INFO : Patience: 10 [2022-03-23 12:10:31 main:481] : INFO : Hidden Width: 12 [2022-03-23 12:10:31 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 12:10:31 main:483] : INFO : # Backend Layers: 4 [2022-03-23 12:10:31 main:484] : INFO : # Threads: 1 [2022-03-23 12:10:31 main:486] : INFO : Preparing Dataset [2022-03-23 12:10:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 12:10:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-23 12:10:33 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-23 12:10:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-23 12:10:33 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-23 12:10:33 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-23 12:10:33 main:494] : INFO : Creating Model [2022-03-23 12:10:33 main:507] : INFO : Preparing config file [2022-03-23 12:10:33 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-23 12:10:33 main:512] : INFO : Loading config [2022-03-23 12:10:33 main:514] : INFO : Loading state [2022-03-23 12:10:34 main:559] : INFO : Loading DataLoader into Memory [2022-03-23 12:10:34 main:562] : INFO : Starting Training [2022-03-23 12:10:35 main:574] : INFO : Epoch 1751 | loss: 0.0312202 | val_loss: 0.0311972 | Time: 1856.8 ms [2022-03-23 12:10:37 main:574] : INFO : Epoch 1752 | loss: 0.0311928 | val_loss: 0.0311899 | Time: 1630.8 ms [2022-03-23 12:10:39 main:574] : INFO : Epoch 1753 | loss: 0.0311881 | val_loss: 0.0311877 | Time: 1636.63 ms [2022-03-23 12:10:40 main:574] : INFO : Epoch 1754 | loss: 0.0311869 | val_loss: 0.0311886 | Time: 1637.5 ms [2022-03-23 12:10:42 main:574] : INFO : Epoch 1755 | loss: 0.0311856 | val_loss: 0.0311899 | Time: 1640.61 ms [2022-03-23 12:10:44 main:574] : INFO : Epoch 1756 | loss: 0.0311871 | val_loss: 0.031195 | Time: 1658.52 ms [2022-03-23 12:10:45 main:574] : INFO : Epoch 1757 | loss: 0.0311866 | val_loss: 0.0311888 | Time: 1632.82 ms [2022-03-23 12:10:47 main:574] : INFO : Epoch 1758 | loss: 0.031185 | val_loss: 0.0311894 | Time: 1634.4 ms [2022-03-23 12:10:49 main:574] : INFO : Epoch 1759 | loss: 0.0311865 | val_loss: 0.0311916 | Time: 1636.13 ms [2022-03-23 12:10:50 main:574] : INFO : Epoch 1760 | loss: 0.0311837 | val_loss: 0.031187 | Time: 1639.81 ms [2022-03-23 12:10:52 main:574] : INFO : Epoch 1761 | loss: 0.0311815 | val_loss: 0.0311864 | Time: 1644.45 ms [2022-03-23 12:10:54 main:574] : INFO : Epoch 1762 | loss: 0.0311817 | val_loss: 0.0311856 | Time: 1639.55 ms [2022-03-23 12:10:55 main:574] : INFO : Epoch 1763 | loss: 0.0311805 | val_loss: 0.0311845 | Time: 1634.34 ms [2022-03-23 12:10:57 main:574] : INFO : Epoch 1764 | loss: 0.0311811 | val_loss: 0.0311834 | Time: 1651.86 ms [2022-03-23 12:10:58 main:574] : INFO : Epoch 1765 | loss: 0.0311795 | val_loss: 0.0311828 | Time: 1636.13 ms [2022-03-23 12:11:00 main:574] : INFO : Epoch 1766 | loss: 0.0311786 | val_loss: 0.031185 | Time: 1645.22 ms [2022-03-23 12:11:02 main:574] : INFO : Epoch 1767 | loss: 0.0311786 | val_loss: 0.0311821 | Time: 1647.09 ms [2022-03-23 12:11:03 main:574] : INFO : Epoch 1768 | loss: 0.0311772 | val_loss: 0.0311812 | Time: 1646.48 ms [2022-03-23 12:11:05 main:574] : INFO : Epoch 1769 | loss: 0.0311782 | val_loss: 0.0311814 | Time: 1643.15 ms [2022-03-23 12:11:07 main:574] : INFO : Epoch 1770 | loss: 0.0311767 | val_loss: 0.0311813 | Time: 1641.17 ms [2022-03-23 12:11:08 main:574] : INFO : Epoch 1771 | loss: 0.0311766 | val_loss: 0.031185 | Time: 1644.41 ms [2022-03-23 12:11:10 main:574] : INFO : Epoch 1772 | loss: 0.0311771 | val_loss: 0.0311813 | Time: 1643.3 ms [2022-03-23 12:11:12 main:574] : INFO : Epoch 1773 | loss: 0.0311766 | val_loss: 0.0311809 | Time: 1652.28 ms [2022-03-23 12:11:13 main:574] : INFO : Epoch 1774 | loss: 0.031176 | val_loss: 0.0311793 | Time: 1658.56 ms [2022-03-23 12:11:15 main:574] : INFO : Epoch 1775 | loss: 0.0311778 | val_loss: 0.0311785 | Time: 1643.01 ms [2022-03-23 12:11:17 main:574] : INFO : Epoch 1776 | loss: 0.0311778 | val_loss: 0.0311784 | Time: 1649.43 ms [2022-03-23 12:11:18 main:574] : INFO : Epoch 1777 | loss: 0.0311754 | val_loss: 0.031178 | Time: 1644.11 ms [2022-03-23 12:11:20 main:574] : INFO : Epoch 1778 | loss: 0.0311752 | val_loss: 0.031179 | Time: 1638.62 ms [2022-03-23 12:11:22 main:574] : INFO : Epoch 1779 | loss: 0.0311746 | val_loss: 0.0311795 | Time: 1645.84 ms [2022-03-23 12:11:23 main:574] : INFO : Epoch 1780 | loss: 0.0311746 | val_loss: 0.0311809 | Time: 1647.06 ms [2022-03-23 12:11:25 main:574] : INFO : Epoch 1781 | loss: 0.0311749 | val_loss: 0.0311778 | Time: 1643.54 ms [2022-03-23 12:11:27 main:574] : INFO : Epoch 1782 | loss: 0.0311761 | val_loss: 0.0311813 | Time: 1643.76 ms [2022-03-23 12:11:28 main:574] : INFO : Epoch 1783 | loss: 0.0311743 | val_loss: 0.0311796 | Time: 1642.73 ms [2022-03-23 12:11:30 main:574] : INFO : Epoch 1784 | loss: 0.0311726 | val_loss: 0.0311778 | Time: 1644.93 ms [2022-03-23 12:11:31 main:574] : INFO : Epoch 1785 | loss: 0.0311731 | val_loss: 0.0311796 | Time: 1634.44 ms [2022-03-23 12:11:33 main:574] : INFO : Epoch 1786 | loss: 0.0311753 | val_loss: 0.0311776 | Time: 1675.31 ms [2022-03-23 12:11:35 main:574] : INFO : Epoch 1787 | loss: 0.0311732 | val_loss: 0.0311787 | Time: 1652.28 ms [2022-03-23 12:11:36 main:574] : INFO : Epoch 1788 | loss: 0.0311716 | val_loss: 0.0311775 | Time: 1647.77 ms [2022-03-23 12:11:38 main:574] : INFO : Epoch 1789 | loss: 0.031171 | val_loss: 0.0311801 | Time: 1644.39 ms [2022-03-23 12:11:40 main:574] : INFO : Epoch 1790 | loss: 0.0311723 | val_loss: 0.0311776 | Time: 1647.43 ms [2022-03-23 12:11:41 main:574] : INFO : Epoch 1791 | loss: 0.0311717 | val_loss: 0.0311816 | Time: 1668.97 ms [2022-03-23 12:11:43 main:574] : INFO : Epoch 1792 | loss: 0.0311724 | val_loss: 0.0311774 | Time: 1642.43 ms [2022-03-23 12:11:45 main:574] : INFO : Epoch 1793 | loss: 0.0311742 | val_loss: 0.031178 | Time: 1638.02 ms [2022-03-23 12:11:46 main:574] : INFO : Epoch 1794 | loss: 0.0311736 | val_loss: 0.0311765 | Time: 1641.28 ms [2022-03-23 12:11:48 main:574] : INFO : Epoch 1795 | loss: 0.0311745 | val_loss: 0.0311776 | Time: 1636.78 ms [2022-03-23 12:11:50 main:574] : INFO : Epoch 1796 | loss: 0.0311733 | val_loss: 0.0311779 | Time: 1643.82 ms [2022-03-23 12:11:51 main:574] : INFO : Epoch 1797 | loss: 0.0311744 | val_loss: 0.0311783 | Time: 1646.13 ms [2022-03-23 12:11:53 main:574] : INFO : Epoch 1798 | loss: 0.0311756 | val_loss: 0.0311775 | Time: 1641.47 ms [2022-03-23 12:11:55 main:574] : INFO : Epoch 1799 | loss: 0.0311737 | val_loss: 0.0311754 | Time: 1656.11 ms [2022-03-23 12:11:56 main:574] : INFO : Epoch 1800 | loss: 0.0311734 | val_loss: 0.0311757 | Time: 1643.97 ms [2022-03-23 12:11:58 main:574] : INFO : Epoch 1801 | loss: 0.0311723 | val_loss: 0.0311781 | Time: 1648.17 ms [2022-03-23 12:12:00 main:574] : INFO : Epoch 1802 | loss: 0.0311718 | val_loss: 0.0311776 | Time: 1643.91 ms [2022-03-23 12:12:01 main:574] : INFO : Epoch 1803 | loss: 0.0311711 | val_loss: 0.0311763 | Time: 1636.72 ms [2022-03-23 12:12:03 main:574] : INFO : Epoch 1804 | loss: 0.0311715 | val_loss: 0.031178 | Time: 1650.18 ms [2022-03-23 12:12:04 main:574] : INFO : Epoch 1805 | loss: 0.0311727 | val_loss: 0.0311784 | Time: 1641.61 ms [2022-03-23 12:12:06 main:574] : INFO : Epoch 1806 | loss: 0.0311763 | val_loss: 0.0311887 | Time: 1651.19 ms [2022-03-23 12:12:08 main:574] : INFO : Epoch 1807 | loss: 0.0311751 | val_loss: 0.0311762 | Time: 1638.66 ms [2022-03-23 12:12:09 main:574] : INFO : Epoch 1808 | loss: 0.0311749 | val_loss: 0.0311766 | Time: 1653.98 ms [2022-03-23 12:12:11 main:574] : INFO : Epoch 1809 | loss: 0.0311772 | val_loss: 0.0311776 | Time: 1671 ms [2022-03-23 12:12:13 main:574] : INFO : Epoch 1810 | loss: 0.0311776 | val_loss: 0.0311797 | Time: 1643.09 ms [2022-03-23 12:12:14 main:574] : INFO : Epoch 1811 | loss: 0.0311778 | val_loss: 0.0311823 | Time: 1649.34 ms [2022-03-23 12:12:16 main:574] : INFO : Epoch 1812 | loss: 0.0311774 | val_loss: 0.031182 | Time: 1644.98 ms [2022-03-23 12:12:18 main:574] : INFO : Epoch 1813 | loss: 0.031177 | val_loss: 0.0311827 | Time: 1649.48 ms [2022-03-23 12:12:19 main:574] : INFO : Epoch 1814 | loss: 0.0311749 | val_loss: 0.0311817 | Time: 1643.55 ms [2022-03-23 12:12:21 main:574] : INFO : Epoch 1815 | loss: 0.0311749 | val_loss: 0.0311823 | Time: 1648.18 ms [2022-03-23 12:12:23 main:574] : INFO : Epoch 1816 | loss: 0.0311749 | val_loss: 0.031184 | Time: 1643.57 ms [2022-03-23 12:12:24 main:574] : INFO : Epoch 1817 | loss: 0.0311752 | val_loss: 0.0311796 | Time: 1650.58 ms [2022-03-23 12:12:26 main:574] : INFO : Epoch 1818 | loss: 0.0311732 | val_loss: 0.0311817 | Time: 1644.19 ms [2022-03-23 12:12:28 main:574] : INFO : Epoch 1819 | loss: 0.0311739 | val_loss: 0.0311798 | Time: 1653.1 ms [2022-03-23 12:12:29 main:574] : INFO : Epoch 1820 | loss: 0.0311735 | val_loss: 0.0311811 | Time: 1676.92 ms [2022-03-23 12:12:31 main:574] : INFO : Epoch 1821 | loss: 0.0311724 | val_loss: 0.0311792 | Time: 1680.94 ms [2022-03-23 12:12:33 main:574] : INFO : Epoch 1822 | loss: 0.0311708 | val_loss: 0.0311791 | Time: 1649.69 ms [2022-03-23 12:12:34 main:574] : INFO : Epoch 1823 | loss: 0.0311711 | val_loss: 0.0311781 | Time: 1645.49 ms [2022-03-23 12:12:36 main:574] : INFO : Epoch 1824 | loss: 0.0311704 | val_loss: 0.0311761 | Time: 1640.59 ms [2022-03-23 12:12:38 main:574] : INFO : Epoch 1825 | loss: 0.0311708 | val_loss: 0.0311753 | Time: 1643.89 ms [2022-03-23 12:12:39 main:574] : INFO : Epoch 1826 | loss: 0.03117 | val_loss: 0.0311775 | Time: 1657.54 ms [2022-03-23 12:12:41 main:574] : INFO : Epoch 1827 | loss: 0.0311702 | val_loss: 0.031177 | Time: 1663.28 ms [2022-03-23 12:12:43 main:574] : INFO : Epoch 1828 | loss: 0.0311694 | val_loss: 0.0311763 | Time: 1638.91 ms [2022-03-23 12:12:44 main:574] : INFO : Epoch 1829 | loss: 0.031169 | val_loss: 0.0311755 | Time: 1646.8 ms [2022-03-23 12:12:46 main:574] : INFO : Epoch 1830 | loss: 0.0311694 | val_loss: 0.031178 | Time: 1653.02 ms [2022-03-23 12:12:47 main:574] : INFO : Epoch 1831 | loss: 0.0311717 | val_loss: 0.0311777 | Time: 1651.31 ms [2022-03-23 12:12:49 main:574] : INFO : Epoch 1832 | loss: 0.0311726 | val_loss: 0.0311756 | Time: 1651.59 ms [2022-03-23 12:12:51 main:574] : INFO : Epoch 1833 | loss: 0.0311707 | val_loss: 0.0311762 | Time: 1650.02 ms [2022-03-23 12:12:52 main:574] : INFO : Epoch 1834 | loss: 0.0311708 | val_loss: 0.0311775 | Time: 1650.03 ms [2022-03-23 12:12:54 main:574] : INFO : Epoch 1835 | loss: 0.0311723 | val_loss: 0.0311754 | Time: 1647.61 ms [2022-03-23 12:12:56 main:574] : INFO : Epoch 1836 | loss: 0.0311692 | val_loss: 0.0311756 | Time: 1654.83 ms [2022-03-23 12:12:57 main:574] : INFO : Epoch 1837 | loss: 0.0311676 | val_loss: 0.0311755 | Time: 1653.79 ms [2022-03-23 12:12:59 main:574] : INFO : Epoch 1838 | loss: 0.0311676 | val_loss: 0.0311748 | Time: 1649.96 ms [2022-03-23 12:13:01 main:574] : INFO : Epoch 1839 | loss: 0.0311677 | val_loss: 0.0311763 | Time: 1654.8 ms [2022-03-23 12:13:02 main:574] : INFO : Epoch 1840 | loss: 0.0311675 | val_loss: 0.0311783 | Time: 1651.25 ms [2022-03-23 12:13:04 main:574] : INFO : Epoch 1841 | loss: 0.0311682 | val_loss: 0.0311762 | Time: 1644.14 ms [2022-03-23 12:13:06 main:574] : INFO : Epoch 1842 | loss: 0.0311685 | val_loss: 0.0311796 | Time: 1649.36 ms [2022-03-23 12:13:07 main:574] : INFO : Epoch 1843 | loss: 0.0311704 | val_loss: 0.0311769 | Time: 1650.82 ms [2022-03-23 12:13:09 main:574] : INFO : Epoch 1844 | loss: 0.0311705 | val_loss: 0.0311792 | Time: 1679.14 ms [2022-03-23 12:13:11 main:574] : INFO : Epoch 1845 | loss: 0.0311696 | val_loss: 0.0311773 | Time: 1647.58 ms [2022-03-23 12:13:12 main:574] : INFO : Epoch 1846 | loss: 0.0311679 | val_loss: 0.0311782 | Time: 1652.16 ms [2022-03-23 12:13:14 main:574] : INFO : Epoch 1847 | loss: 0.0311689 | val_loss: 0.0311815 | Time: 1649.98 ms [2022-03-23 12:13:16 main:574] : INFO : Epoch 1848 | loss: 0.0311715 | val_loss: 0.0311805 | Time: 1658.36 ms [2022-03-23 12:13:17 main:574] : INFO : Epoch 1849 | loss: 0.0311707 | val_loss: 0.0311814 | Time: 1648.05 ms [2022-03-23 12:13:19 main:574] : INFO : Epoch 1850 | loss: 0.0311715 | val_loss: 0.0311811 | Time: 1656.65 ms [2022-03-23 12:13:21 main:574] : INFO : Epoch 1851 | loss: 0.0311731 | val_loss: 0.0311809 | Time: 1657.44 ms [2022-03-23 12:13:22 main:574] : INFO : Epoch 1852 | loss: 0.0311745 | val_loss: 0.0311819 | Time: 1648.68 ms [2022-03-23 12:13:24 main:574] : INFO : Epoch 1853 | loss: 0.0311767 | val_loss: 0.0311852 | Time: 1653.23 ms [2022-03-23 12:13:26 main:574] : INFO : Epoch 1854 | loss: 0.0311761 | val_loss: 0.0311881 | Time: 1651.2 ms [2022-03-23 12:13:27 main:574] : INFO : Epoch 1855 | loss: 0.0311749 | val_loss: 0.031189 | Time: 1639.52 ms [2022-03-23 12:13:29 main:574] : INFO : Epoch 1856 | loss: 0.0311754 | val_loss: 0.0311866 | Time: 1650.9 ms [2022-03-23 12:13:31 main:574] : INFO : Epoch 1857 | loss: 0.031175 | val_loss: 0.0311865 | Time: 1647.5 ms [2022-03-23 12:13:32 main:574] : INFO : Epoch 1858 | loss: 0.031175 | val_loss: 0.0311861 | Time: 1652.6 ms [2022-03-23 12:13:34 main:574] : INFO : Epoch 1859 | loss: 0.031176 | val_loss: 0.0311853 | Time: 1650.21 ms [2022-03-23 12:13:35 main:574] : INFO : Epoch 1860 | loss: 0.0311751 | val_loss: 0.0311883 | Time: 1652.39 ms [2022-03-23 12:13:37 main:574] : INFO : Epoch 1861 | loss: 0.0311788 | val_loss: 0.0311897 | Time: 1662.71 ms [2022-03-23 12:13:39 main:574] : INFO : Epoch 1862 | loss: 0.0311783 | val_loss: 0.0311907 | Time: 1663.77 ms [2022-03-23 12:13:40 main:574] : INFO : Epoch 1863 | loss: 0.0311782 | val_loss: 0.0311907 | Time: 1654.09 ms [2022-03-23 12:13:42 main:574] : INFO : Epoch 1864 | loss: 0.031178 | val_loss: 0.0311914 | Time: 1653.51 ms [2022-03-23 12:13:44 main:574] : INFO : Epoch 1865 | loss: 0.031177 | val_loss: 0.0311923 | Time: 1643.44 ms [2022-03-23 12:13:45 main:574] : INFO : Epoch 1866 | loss: 0.0311762 | val_loss: 0.0311905 | Time: 1649.87 ms [2022-03-23 12:13:47 main:574] : INFO : Epoch 1867 | loss: 0.0311778 | val_loss: 0.0311895 | Time: 1653.63 ms [2022-03-23 12:13:49 main:574] : INFO : Epoch 1868 | loss: 0.0311797 | val_loss: 0.0311897 | Time: 1647.22 ms [2022-03-23 12:13:50 main:574] : INFO : Epoch 1869 | loss: 0.0311811 | val_loss: 0.0311916 | Time: 1646.56 ms [2022-03-23 12:13:52 main:574] : INFO : Epoch 1870 | loss: 0.0311819 | val_loss: 0.0311897 | Time: 1654.68 ms [2022-03-23 12:13:54 main:574] : INFO : Epoch 1871 | loss: 0.0311798 | val_loss: 0.0311903 | Time: 1660.25 ms [2022-03-23 12:13:55 main:574] : INFO : Epoch 1872 | loss: 0.0311798 | val_loss: 0.0311881 | Time: 1652.37 ms [2022-03-23 12:13:57 main:574] : INFO : Epoch 1873 | loss: 0.031179 | val_loss: 0.0311906 | Time: 1652.32 ms [2022-03-23 12:13:59 main:574] : INFO : Epoch 1874 | loss: 0.031178 | val_loss: 0.031191 | Time: 1647.81 ms [2022-03-23 12:14:00 main:574] : INFO : Epoch 1875 | loss: 0.0311803 | val_loss: 0.0311881 | Time: 1652.57 ms [2022-03-23 12:14:02 main:574] : INFO : Epoch 1876 | loss: 0.0311802 | val_loss: 0.031189 | Time: 1648.69 ms [2022-03-23 12:14:04 main:574] : INFO : Epoch 1877 | loss: 0.0311799 | val_loss: 0.031189 | Time: 1655.93 ms [2022-03-23 12:14:05 main:574] : INFO : Epoch 1878 | loss: 0.0311782 | val_loss: 0.0311888 | Time: 1650.39 ms [2022-03-23 12:14:07 main:574] : INFO : Epoch 1879 | loss: 0.0311782 | val_loss: 0.0311876 | Time: 1679.47 ms [2022-03-23 12:14:09 main:574] : INFO : Epoch 1880 | loss: 0.031179 | val_loss: 0.0311918 | Time: 1648.29 ms [2022-03-23 12:14:10 main:574] : INFO : Epoch 1881 | loss: 0.0311791 | val_loss: 0.031189 | Time: 1653.52 ms [2022-03-23 12:14:12 main:574] : INFO : Epoch 1882 | loss: 0.0311779 | val_loss: 0.0311863 | Time: 1648.94 ms [2022-03-23 12:14:14 main:574] : INFO : Epoch 1883 | loss: 0.0311763 | val_loss: 0.0311888 | Time: 1647.06 ms [2022-03-23 12:14:15 main:574] : INFO : Epoch 1884 | loss: 0.031177 | val_loss: 0.0311881 | Time: 1650.8 ms [2022-03-23 12:14:17 main:574] : INFO : Epoch 1885 | loss: 0.0311764 | val_loss: 0.0311838 | Time: 1649.34 ms [2022-03-23 12:14:19 main:574] : INFO : Epoch 1886 | loss: 0.0311751 | val_loss: 0.0311877 | Time: 1652.44 ms [2022-03-23 12:14:20 main:574] : INFO : Epoch 1887 | loss: 0.0311738 | val_loss: 0.0311846 | Time: 1655.52 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1050 Ti) [2022-03-23 13:52:34 main:435] : INFO : Set logging level to 1 [2022-03-23 13:52:34 main:441] : INFO : Running in BOINC Client mode [2022-03-23 13:52:34 main:444] : INFO : Resolving all filenames [2022-03-23 13:52:34 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 13:52:34 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 13:52:34 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 13:52:34 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 13:52:34 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 13:52:34 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 13:52:34 main:474] : INFO : Configuration: [2022-03-23 13:52:34 main:475] : INFO : Model type: GRU [2022-03-23 13:52:34 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 13:52:34 main:477] : INFO : Max Epochs: 2048 [2022-03-23 13:52:34 main:478] : INFO : Batch Size: 128 [2022-03-23 13:52:34 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 13:52:34 main:480] : INFO : Patience: 10 [2022-03-23 13:52:34 main:481] : INFO : Hidden Width: 12 [2022-03-23 13:52:34 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 13:52:34 main:483] : INFO : # Backend Layers: 4 [2022-03-23 13:52:34 main:484] : INFO : # Threads: 1 [2022-03-23 13:52:34 main:486] : INFO : Preparing Dataset [2022-03-23 13:52:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 13:52:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-23 13:52:37 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-23 13:52:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-23 13:52:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-23 13:52:37 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-23 13:52:37 main:494] : INFO : Creating Model [2022-03-23 13:52:37 main:507] : INFO : Preparing config file [2022-03-23 13:52:37 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-23 13:52:37 main:512] : INFO : Loading config [2022-03-23 13:52:37 main:514] : INFO : Loading state [2022-03-23 13:52:38 main:559] : INFO : Loading DataLoader into Memory [2022-03-23 13:52:38 main:562] : INFO : Starting Training [2022-03-23 13:52:41 main:574] : INFO : Epoch 1869 | loss: 0.0312037 | val_loss: 0.0311951 | Time: 2330.75 ms [2022-03-23 13:52:42 main:574] : INFO : Epoch 1870 | loss: 0.0311781 | val_loss: 0.0311872 | Time: 1641.55 ms [2022-03-23 13:52:44 main:574] : INFO : Epoch 1871 | loss: 0.0311774 | val_loss: 0.0311871 | Time: 1642.23 ms [2022-03-23 13:52:46 main:574] : INFO : Epoch 1872 | loss: 0.0311748 | val_loss: 0.0311846 | Time: 1672.26 ms [2022-03-23 13:52:47 main:574] : INFO : Epoch 1873 | loss: 0.0311726 | val_loss: 0.0311829 | Time: 1648.97 ms [2022-03-23 13:52:49 main:574] : INFO : Epoch 1874 | loss: 0.0311723 | val_loss: 0.0311845 | Time: 1654.15 ms [2022-03-23 13:52:51 main:574] : INFO : Epoch 1875 | loss: 0.0311723 | val_loss: 0.0311837 | Time: 1647.71 ms [2022-03-23 13:52:52 main:574] : INFO : Epoch 1876 | loss: 0.0311733 | val_loss: 0.031188 | Time: 1650.01 ms [2022-03-23 13:52:54 main:574] : INFO : Epoch 1877 | loss: 0.0311741 | val_loss: 0.0311859 | Time: 1653.01 ms [2022-03-23 13:52:56 main:574] : INFO : Epoch 1878 | loss: 0.0311725 | val_loss: 0.0311822 | Time: 1650.67 ms [2022-03-23 13:52:57 main:574] : INFO : Epoch 1879 | loss: 0.0311713 | val_loss: 0.0311828 | Time: 1654.27 ms [2022-03-23 13:52:59 main:574] : INFO : Epoch 1880 | loss: 0.0311707 | val_loss: 0.0311843 | Time: 1653.02 ms [2022-03-23 13:53:01 main:574] : INFO : Epoch 1881 | loss: 0.0311691 | val_loss: 0.0311822 | Time: 1658.67 ms [2022-03-23 13:53:02 main:574] : INFO : Epoch 1882 | loss: 0.0311695 | val_loss: 0.0311866 | Time: 1644.82 ms [2022-03-23 13:53:04 main:574] : INFO : Epoch 1883 | loss: 0.0311698 | val_loss: 0.0311835 | Time: 1645.43 ms [2022-03-23 13:53:06 main:574] : INFO : Epoch 1884 | loss: 0.0311692 | val_loss: 0.0311817 | Time: 1650.33 ms [2022-03-23 13:53:07 main:574] : INFO : Epoch 1885 | loss: 0.031168 | val_loss: 0.0311851 | Time: 1650.6 ms [2022-03-23 13:53:09 main:574] : INFO : Epoch 1886 | loss: 0.0311676 | val_loss: 0.0311814 | Time: 1656.49 ms [2022-03-23 13:53:11 main:574] : INFO : Epoch 1887 | loss: 0.0311669 | val_loss: 0.0311828 | Time: 1657.64 ms [2022-03-23 13:53:12 main:574] : INFO : Epoch 1888 | loss: 0.0311663 | val_loss: 0.0311817 | Time: 1650.02 ms [2022-03-23 13:53:14 main:574] : INFO : Epoch 1889 | loss: 0.0311665 | val_loss: 0.0311816 | Time: 1683.01 ms [2022-03-23 13:53:16 main:574] : INFO : Epoch 1890 | loss: 0.0311658 | val_loss: 0.0311804 | Time: 1649.27 ms [2022-03-23 13:53:17 main:574] : INFO : Epoch 1891 | loss: 0.031166 | val_loss: 0.0311829 | Time: 1674.73 ms [2022-03-23 13:53:19 main:574] : INFO : Epoch 1892 | loss: 0.0311674 | val_loss: 0.0311826 | Time: 1655.56 ms [2022-03-23 13:53:21 main:574] : INFO : Epoch 1893 | loss: 0.031166 | val_loss: 0.0311913 | Time: 1653.38 ms [2022-03-23 13:53:22 main:574] : INFO : Epoch 1894 | loss: 0.0311657 | val_loss: 0.0311792 | Time: 1656.36 ms [2022-03-23 13:53:24 main:574] : INFO : Epoch 1895 | loss: 0.0311652 | val_loss: 0.0311809 | Time: 1656.01 ms [2022-03-23 13:53:26 main:574] : INFO : Epoch 1896 | loss: 0.0311643 | val_loss: 0.0311842 | Time: 1660.3 ms [2022-03-23 13:53:27 main:574] : INFO : Epoch 1897 | loss: 0.0311668 | val_loss: 0.0311833 | Time: 1665.27 ms [2022-03-23 13:53:29 main:574] : INFO : Epoch 1898 | loss: 0.0311716 | val_loss: 0.0311829 | Time: 1657.27 ms [2022-03-23 13:53:31 main:574] : INFO : Epoch 1899 | loss: 0.0311699 | val_loss: 0.03118 | Time: 1653.26 ms [2022-03-23 13:53:32 main:574] : INFO : Epoch 1900 | loss: 0.0311702 | val_loss: 0.0311833 | Time: 1656.83 ms [2022-03-23 13:53:34 main:574] : INFO : Epoch 1901 | loss: 0.0311708 | val_loss: 0.0311817 | Time: 1722.57 ms [2022-03-23 13:53:36 main:574] : INFO : Epoch 1902 | loss: 0.031169 | val_loss: 0.0311814 | Time: 1665.4 ms [2022-03-23 13:53:37 main:574] : INFO : Epoch 1903 | loss: 0.0311675 | val_loss: 0.0311817 | Time: 1663.08 ms [2022-03-23 13:53:39 main:574] : INFO : Epoch 1904 | loss: 0.0311671 | val_loss: 0.0311799 | Time: 1824.51 ms [2022-03-23 13:53:41 main:574] : INFO : Epoch 1905 | loss: 0.0311667 | val_loss: 0.0311805 | Time: 1693.01 ms [2022-03-23 13:53:42 main:574] : INFO : Epoch 1906 | loss: 0.0311664 | val_loss: 0.0311811 | Time: 1676.59 ms [2022-03-23 13:53:44 main:574] : INFO : Epoch 1907 | loss: 0.0311655 | val_loss: 0.0311809 | Time: 1719.75 ms [2022-03-23 13:53:46 main:574] : INFO : Epoch 1908 | loss: 0.0311656 | val_loss: 0.0311791 | Time: 1677.17 ms [2022-03-23 13:53:48 main:574] : INFO : Epoch 1909 | loss: 0.0311644 | val_loss: 0.031179 | Time: 1679.45 ms [2022-03-23 13:53:49 main:574] : INFO : Epoch 1910 | loss: 0.0311656 | val_loss: 0.0311783 | Time: 1680.43 ms [2022-03-23 13:53:51 main:574] : INFO : Epoch 1911 | loss: 0.0311643 | val_loss: 0.0311819 | Time: 1679.45 ms [2022-03-23 13:53:53 main:574] : INFO : Epoch 1912 | loss: 0.0311673 | val_loss: 0.0311802 | Time: 1678.5 ms [2022-03-23 13:53:54 main:574] : INFO : Epoch 1913 | loss: 0.0311673 | val_loss: 0.0311808 | Time: 1683.47 ms [2022-03-23 13:53:56 main:574] : INFO : Epoch 1914 | loss: 0.031168 | val_loss: 0.0311796 | Time: 1686.07 ms [2022-03-23 13:53:58 main:574] : INFO : Epoch 1915 | loss: 0.0311673 | val_loss: 0.0311843 | Time: 1681.84 ms [2022-03-23 13:53:59 main:574] : INFO : Epoch 1916 | loss: 0.0311655 | val_loss: 0.031177 | Time: 1676.4 ms [2022-03-23 13:54:01 main:574] : INFO : Epoch 1917 | loss: 0.0311632 | val_loss: 0.0311788 | Time: 1677.22 ms [2022-03-23 13:54:03 main:574] : INFO : Epoch 1918 | loss: 0.0311641 | val_loss: 0.0311769 | Time: 1679.1 ms [2022-03-23 13:54:04 main:574] : INFO : Epoch 1919 | loss: 0.031167 | val_loss: 0.0311789 | Time: 1689.04 ms [2022-03-23 13:54:06 main:574] : INFO : Epoch 1920 | loss: 0.0311668 | val_loss: 0.0311835 | Time: 1681.81 ms [2022-03-23 13:54:08 main:574] : INFO : Epoch 1921 | loss: 0.0311682 | val_loss: 0.0311778 | Time: 1679.31 ms [2022-03-23 13:54:09 main:574] : INFO : Epoch 1922 | loss: 0.0311648 | val_loss: 0.0311805 | Time: 1681.94 ms [2022-03-23 13:54:11 main:574] : INFO : Epoch 1923 | loss: 0.0311637 | val_loss: 0.0311812 | Time: 1680.75 ms [2022-03-23 13:54:13 main:574] : INFO : Epoch 1924 | loss: 0.0311723 | val_loss: 0.031183 | Time: 1675.98 ms [2022-03-23 13:54:15 main:574] : INFO : Epoch 1925 | loss: 0.03117 | val_loss: 0.031177 | Time: 1730.02 ms [2022-03-23 13:54:16 main:574] : INFO : Epoch 1926 | loss: 0.0311696 | val_loss: 0.0311763 | Time: 1682.59 ms [2022-03-23 13:54:18 main:574] : INFO : Epoch 1927 | loss: 0.0311695 | val_loss: 0.0311764 | Time: 1679.43 ms [2022-03-23 13:54:20 main:574] : INFO : Epoch 1928 | loss: 0.0311675 | val_loss: 0.0311798 | Time: 1684.12 ms [2022-03-23 13:54:21 main:574] : INFO : Epoch 1929 | loss: 0.0311668 | val_loss: 0.0311772 | Time: 1679.24 ms [2022-03-23 13:54:23 main:574] : INFO : Epoch 1930 | loss: 0.0311657 | val_loss: 0.0311804 | Time: 1734.47 ms [2022-03-23 13:54:25 main:574] : INFO : Epoch 1931 | loss: 0.0311685 | val_loss: 0.0311758 | Time: 1688.74 ms [2022-03-23 13:54:26 main:574] : INFO : Epoch 1932 | loss: 0.0311644 | val_loss: 0.031176 | Time: 1688.37 ms [2022-03-23 13:54:28 main:574] : INFO : Epoch 1933 | loss: 0.0311648 | val_loss: 0.0311814 | Time: 1680.08 ms [2022-03-23 13:54:30 main:574] : INFO : Epoch 1934 | loss: 0.0311661 | val_loss: 0.0311739 | Time: 1678.05 ms [2022-03-23 13:54:31 main:574] : INFO : Epoch 1935 | loss: 0.0311639 | val_loss: 0.0311746 | Time: 1677.45 ms [2022-03-23 13:54:33 main:574] : INFO : Epoch 1936 | loss: 0.0311635 | val_loss: 0.031171 | Time: 1676.83 ms [2022-03-23 13:54:35 main:574] : INFO : Epoch 1937 | loss: 0.0311617 | val_loss: 0.0311709 | Time: 1750.1 ms [2022-03-23 13:54:37 main:574] : INFO : Epoch 1938 | loss: 0.0311617 | val_loss: 0.0311703 | Time: 1696 ms [2022-03-23 13:54:38 main:574] : INFO : Epoch 1939 | loss: 0.0311632 | val_loss: 0.0311734 | Time: 1697.07 ms [2022-03-23 13:54:40 main:574] : INFO : Epoch 1940 | loss: 0.0311659 | val_loss: 0.0311806 | Time: 1696.98 ms [2022-03-23 13:54:42 main:574] : INFO : Epoch 1941 | loss: 0.031166 | val_loss: 0.0311724 | Time: 1697.92 ms [2022-03-23 13:54:43 main:574] : INFO : Epoch 1942 | loss: 0.0311643 | val_loss: 0.0311757 | Time: 1691.12 ms [2022-03-23 13:54:45 main:574] : INFO : Epoch 1943 | loss: 0.0311644 | val_loss: 0.0311735 | Time: 1734.21 ms [2022-03-23 13:54:47 main:574] : INFO : Epoch 1944 | loss: 0.0311662 | val_loss: 0.0311751 | Time: 1696.74 ms [2022-03-23 13:54:48 main:574] : INFO : Epoch 1945 | loss: 0.031171 | val_loss: 0.031184 | Time: 1695.21 ms [2022-03-23 13:54:50 main:574] : INFO : Epoch 1946 | loss: 0.0311913 | val_loss: 0.0312125 | Time: 1692.58 ms [2022-03-23 13:54:52 main:574] : INFO : Epoch 1947 | loss: 0.0312188 | val_loss: 0.0312216 | Time: 1695.58 ms [2022-03-23 13:54:54 main:574] : INFO : Epoch 1948 | loss: 0.0312189 | val_loss: 0.0312211 | Time: 1694.53 ms [2022-03-23 13:54:55 main:574] : INFO : Epoch 1949 | loss: 0.0312199 | val_loss: 0.0312208 | Time: 1692.27 ms [2022-03-23 13:54:57 main:574] : INFO : Epoch 1950 | loss: 0.03122 | val_loss: 0.0312221 | Time: 1692.39 ms [2022-03-23 13:54:59 main:574] : INFO : Epoch 1951 | loss: 0.0312176 | val_loss: 0.0312195 | Time: 1694.01 ms [2022-03-23 13:55:00 main:574] : INFO : Epoch 1952 | loss: 0.0312159 | val_loss: 0.0312166 | Time: 1696.32 ms [2022-03-23 13:55:02 main:574] : INFO : Epoch 1953 | loss: 0.0312155 | val_loss: 0.0312193 | Time: 1696.75 ms [2022-03-23 13:55:04 main:574] : INFO : Epoch 1954 | loss: 0.0312145 | val_loss: 0.0312175 | Time: 1697.26 ms [2022-03-23 13:55:06 main:574] : INFO : Epoch 1955 | loss: 0.0312128 | val_loss: 0.0312129 | Time: 1697.29 ms [2022-03-23 13:55:07 main:574] : INFO : Epoch 1956 | loss: 0.0312113 | val_loss: 0.0312122 | Time: 1697.52 ms [2022-03-23 13:55:09 main:574] : INFO : Epoch 1957 | loss: 0.0312104 | val_loss: 0.0312116 | Time: 1698.89 ms [2022-03-23 13:55:11 main:574] : INFO : Epoch 1958 | loss: 0.0312101 | val_loss: 0.0312102 | Time: 1705.28 ms [2022-03-23 13:55:12 main:574] : INFO : Epoch 1959 | loss: 0.0312092 | val_loss: 0.0312103 | Time: 1701.12 ms [2022-03-23 13:55:14 main:574] : INFO : Epoch 1960 | loss: 0.03121 | val_loss: 0.0312106 | Time: 1744.96 ms [2022-03-23 13:55:16 main:574] : INFO : Epoch 1961 | loss: 0.0312123 | val_loss: 0.0312267 | Time: 1697.05 ms [2022-03-23 13:55:17 main:574] : INFO : Epoch 1962 | loss: 0.0312129 | val_loss: 0.0312105 | Time: 1703.05 ms [2022-03-23 13:55:19 main:574] : INFO : Epoch 1963 | loss: 0.0312099 | val_loss: 0.0312095 | Time: 1696.02 ms [2022-03-23 13:55:21 main:574] : INFO : Epoch 1964 | loss: 0.03121 | val_loss: 0.0312119 | Time: 1704.58 ms [2022-03-23 13:55:23 main:574] : INFO : Epoch 1965 | loss: 0.0312084 | val_loss: 0.0312099 | Time: 1699.85 ms [2022-03-23 13:55:24 main:574] : INFO : Epoch 1966 | loss: 0.0312085 | val_loss: 0.0312085 | Time: 1701.7 ms [2022-03-23 13:55:26 main:574] : INFO : Epoch 1967 | loss: 0.0312057 | val_loss: 0.0312085 | Time: 1707.33 ms [2022-03-23 13:55:28 main:574] : INFO : Epoch 1968 | loss: 0.0312054 | val_loss: 0.031208 | Time: 1702.96 ms [2022-03-23 13:55:29 main:574] : INFO : Epoch 1969 | loss: 0.0312054 | val_loss: 0.0312093 | Time: 1696.71 ms [2022-03-23 13:55:31 main:574] : INFO : Epoch 1970 | loss: 0.0312047 | val_loss: 0.031208 | Time: 1702.89 ms [2022-03-23 13:55:33 main:574] : INFO : Epoch 1971 | loss: 0.031204 | val_loss: 0.0312107 | Time: 1699.9 ms [2022-03-23 13:55:35 main:574] : INFO : Epoch 1972 | loss: 0.0312054 | val_loss: 0.0312066 | Time: 1703.52 ms [2022-03-23 13:55:36 main:574] : INFO : Epoch 1973 | loss: 0.0312027 | val_loss: 0.0312058 | Time: 1704.66 ms [2022-03-23 13:55:38 main:574] : INFO : Epoch 1974 | loss: 0.0312028 | val_loss: 0.031206 | Time: 1700.45 ms [2022-03-23 13:55:40 main:574] : INFO : Epoch 1975 | loss: 0.0312027 | val_loss: 0.0312122 | Time: 1704.61 ms [2022-03-23 13:55:41 main:574] : INFO : Epoch 1976 | loss: 0.0312058 | val_loss: 0.0312044 | Time: 1701.91 ms [2022-03-23 13:55:43 main:574] : INFO : Epoch 1977 | loss: 0.0312011 | val_loss: 0.031207 | Time: 1704.79 ms [2022-03-23 13:55:45 main:574] : INFO : Epoch 1978 | loss: 0.0312008 | val_loss: 0.0312067 | Time: 1737.94 ms [2022-03-23 13:55:47 main:574] : INFO : Epoch 1979 | loss: 0.0312002 | val_loss: 0.0312026 | Time: 1697.7 ms [2022-03-23 13:55:48 main:574] : INFO : Epoch 1980 | loss: 0.031199 | val_loss: 0.0312057 | Time: 1714.1 ms [2022-03-23 13:55:50 main:574] : INFO : Epoch 1981 | loss: 0.031199 | val_loss: 0.0312025 | Time: 1702.81 ms [2022-03-23 13:55:52 main:574] : INFO : Epoch 1982 | loss: 0.0312004 | val_loss: 0.0312088 | Time: 1698.64 ms [2022-03-23 13:55:53 main:574] : INFO : Epoch 1983 | loss: 0.0312023 | val_loss: 0.0312044 | Time: 1702.97 ms [2022-03-23 13:55:55 main:574] : INFO : Epoch 1984 | loss: 0.0312012 | val_loss: 0.0312039 | Time: 1698.35 ms [2022-03-23 13:55:57 main:574] : INFO : Epoch 1985 | loss: 0.0312005 | val_loss: 0.0312034 | Time: 1700.83 ms [2022-03-23 13:55:58 main:574] : INFO : Epoch 1986 | loss: 0.0312008 | val_loss: 0.0312031 | Time: 1703.65 ms [2022-03-23 13:56:00 main:574] : INFO : Epoch 1987 | loss: 0.0311992 | val_loss: 0.0312016 | Time: 1702.67 ms [2022-03-23 13:56:02 main:574] : INFO : Epoch 1988 | loss: 0.0311984 | val_loss: 0.0312012 | Time: 1697.56 ms [2022-03-23 13:56:04 main:574] : INFO : Epoch 1989 | loss: 0.0311979 | val_loss: 0.0312024 | Time: 1699.85 ms [2022-03-23 13:56:05 main:574] : INFO : Epoch 1990 | loss: 0.0311981 | val_loss: 0.0312004 | Time: 1702.06 ms [2022-03-23 13:56:07 main:574] : INFO : Epoch 1991 | loss: 0.0311974 | val_loss: 0.0312002 | Time: 1700.28 ms [2022-03-23 13:56:09 main:574] : INFO : Epoch 1992 | loss: 0.0311967 | val_loss: 0.0311997 | Time: 1696.61 ms [2022-03-23 13:56:10 main:574] : INFO : Epoch 1993 | loss: 0.0311959 | val_loss: 0.0311991 | Time: 1702.07 ms [2022-03-23 13:56:12 main:574] : INFO : Epoch 1994 | loss: 0.0311956 | val_loss: 0.0311991 | Time: 1703.04 ms [2022-03-23 13:56:14 main:574] : INFO : Epoch 1995 | loss: 0.0311956 | val_loss: 0.0311983 | Time: 1731.89 ms [2022-03-23 13:56:16 main:574] : INFO : Epoch 1996 | loss: 0.0311952 | val_loss: 0.031199 | Time: 1705.24 ms [2022-03-23 13:56:17 main:574] : INFO : Epoch 1997 | loss: 0.0311974 | val_loss: 0.0312064 | Time: 1697.12 ms [2022-03-23 13:56:19 main:574] : INFO : Epoch 1998 | loss: 0.0311954 | val_loss: 0.0311973 | Time: 1696.64 ms [2022-03-23 13:56:21 main:574] : INFO : Epoch 1999 | loss: 0.0311934 | val_loss: 0.031198 | Time: 1703.7 ms [2022-03-23 13:56:22 main:574] : INFO : Epoch 2000 | loss: 0.0311931 | val_loss: 0.031197 | Time: 1699.08 ms [2022-03-23 13:56:24 main:574] : INFO : Epoch 2001 | loss: 0.031193 | val_loss: 0.0311972 | Time: 1702.08 ms [2022-03-23 13:56:26 main:574] : INFO : Epoch 2002 | loss: 0.0311927 | val_loss: 0.0311974 | Time: 1704.37 ms [2022-03-23 13:56:27 main:574] : INFO : Epoch 2003 | loss: 0.0311962 | val_loss: 0.031198 | Time: 1703.77 ms [2022-03-23 13:56:29 main:574] : INFO : Epoch 2004 | loss: 0.0311924 | val_loss: 0.0311978 | Time: 1702.07 ms [2022-03-23 13:56:31 main:574] : INFO : Epoch 2005 | loss: 0.0311926 | val_loss: 0.0311971 | Time: 1702.21 ms [2022-03-23 13:56:33 main:574] : INFO : Epoch 2006 | loss: 0.0311918 | val_loss: 0.031198 | Time: 1695.82 ms [2022-03-23 13:56:34 main:574] : INFO : Epoch 2007 | loss: 0.0311925 | val_loss: 0.0311975 | Time: 1703.73 ms [2022-03-23 13:56:36 main:574] : INFO : Epoch 2008 | loss: 0.0311922 | val_loss: 0.0311973 | Time: 1698.16 ms [2022-03-23 13:56:38 main:574] : INFO : Epoch 2009 | loss: 0.0311928 | val_loss: 0.0311956 | Time: 1699.82 ms [2022-03-23 13:56:39 main:574] : INFO : Epoch 2010 | loss: 0.0311926 | val_loss: 0.0311948 | Time: 1717.42 ms [2022-03-23 13:56:41 main:574] : INFO : Epoch 2011 | loss: 0.0311933 | val_loss: 0.0311962 | Time: 1712.58 ms [2022-03-23 13:56:43 main:574] : INFO : Epoch 2012 | loss: 0.0311932 | val_loss: 0.0311956 | Time: 1698.43 ms [2022-03-23 13:56:45 main:574] : INFO : Epoch 2013 | loss: 0.0311923 | val_loss: 0.0311972 | Time: 1709.06 ms [2022-03-23 13:56:46 main:574] : INFO : Epoch 2014 | loss: 0.0311913 | val_loss: 0.0311951 | Time: 1682.61 ms [2022-03-23 13:56:48 main:574] : INFO : Epoch 2015 | loss: 0.0311912 | val_loss: 0.0311946 | Time: 1681.39 ms [2022-03-23 13:56:50 main:574] : INFO : Epoch 2016 | loss: 0.0311948 | val_loss: 0.031194 | Time: 1683.34 ms [2022-03-23 13:56:51 main:574] : INFO : Epoch 2017 | loss: 0.0311928 | val_loss: 0.0311957 | Time: 1683.1 ms [2022-03-23 13:56:53 main:574] : INFO : Epoch 2018 | loss: 0.0311903 | val_loss: 0.0311939 | Time: 1682.79 ms [2022-03-23 13:56:55 main:574] : INFO : Epoch 2019 | loss: 0.0311907 | val_loss: 0.0311938 | Time: 1684.36 ms [2022-03-23 13:56:56 main:574] : INFO : Epoch 2020 | loss: 0.031191 | val_loss: 0.0311965 | Time: 1686.41 ms [2022-03-23 13:56:58 main:574] : INFO : Epoch 2021 | loss: 0.0311903 | val_loss: 0.0311941 | Time: 1683.21 ms [2022-03-23 13:57:00 main:574] : INFO : Epoch 2022 | loss: 0.0311898 | val_loss: 0.0311953 | Time: 1687.82 ms [2022-03-23 13:57:01 main:574] : INFO : Epoch 2023 | loss: 0.0311901 | val_loss: 0.0311951 | Time: 1687.96 ms [2022-03-23 13:57:03 main:574] : INFO : Epoch 2024 | loss: 0.0311891 | val_loss: 0.0311928 | Time: 1690.48 ms [2022-03-23 13:57:05 main:574] : INFO : Epoch 2025 | loss: 0.0311896 | val_loss: 0.0311927 | Time: 1690.87 ms [2022-03-23 13:57:06 main:574] : INFO : Epoch 2026 | loss: 0.0311889 | val_loss: 0.0311936 | Time: 1695.91 ms [2022-03-23 13:57:08 main:574] : INFO : Epoch 2027 | loss: 0.0311893 | val_loss: 0.0311936 | Time: 1682.42 ms [2022-03-23 13:57:10 main:574] : INFO : Epoch 2028 | loss: 0.0311893 | val_loss: 0.0311928 | Time: 1681.4 ms [2022-03-23 13:57:12 main:574] : INFO : Epoch 2029 | loss: 0.0311893 | val_loss: 0.0311929 | Time: 1686.71 ms [2022-03-23 13:57:13 main:574] : INFO : Epoch 2030 | loss: 0.031191 | val_loss: 0.0311992 | Time: 1686.59 ms [2022-03-23 13:57:15 main:574] : INFO : Epoch 2031 | loss: 0.0311903 | val_loss: 0.0311929 | Time: 1712.65 ms [2022-03-23 13:57:17 main:574] : INFO : Epoch 2032 | loss: 0.0311888 | val_loss: 0.0311925 | Time: 1685.83 ms [2022-03-23 13:57:18 main:574] : INFO : Epoch 2033 | loss: 0.0311891 | val_loss: 0.0311955 | Time: 1681.7 ms [2022-03-23 13:57:20 main:574] : INFO : Epoch 2034 | loss: 0.0311885 | val_loss: 0.0311926 | Time: 1695.41 ms [2022-03-23 13:57:22 main:574] : INFO : Epoch 2035 | loss: 0.0311877 | val_loss: 0.031194 | Time: 1704.11 ms [2022-03-23 13:57:23 main:574] : INFO : Epoch 2036 | loss: 0.0311933 | val_loss: 0.0311943 | Time: 1690.66 ms [2022-03-23 13:57:25 main:574] : INFO : Epoch 2037 | loss: 0.0311922 | val_loss: 0.0311956 | Time: 1697.75 ms [2022-03-23 13:57:27 main:574] : INFO : Epoch 2038 | loss: 0.0311904 | val_loss: 0.0311931 | Time: 1705.53 ms [2022-03-23 13:57:29 main:574] : INFO : Epoch 2039 | loss: 0.03119 | val_loss: 0.0311957 | Time: 1697.07 ms [2022-03-23 13:57:30 main:574] : INFO : Epoch 2040 | loss: 0.0311922 | val_loss: 0.0311944 | Time: 1690.32 ms [2022-03-23 13:57:32 main:574] : INFO : Epoch 2041 | loss: 0.03119 | val_loss: 0.0311952 | Time: 1700.78 ms [2022-03-23 13:57:34 main:574] : INFO : Epoch 2042 | loss: 0.03119 | val_loss: 0.0311956 | Time: 1703.57 ms [2022-03-23 13:57:35 main:574] : INFO : Epoch 2043 | loss: 0.0311885 | val_loss: 0.0311919 | Time: 1701.9 ms [2022-03-23 13:57:37 main:574] : INFO : Epoch 2044 | loss: 0.0311883 | val_loss: 0.0311925 | Time: 1694.6 ms [2022-03-23 13:57:39 main:574] : INFO : Epoch 2045 | loss: 0.0311862 | val_loss: 0.0311912 | Time: 1703.54 ms [2022-03-23 13:57:40 main:574] : INFO : Epoch 2046 | loss: 0.0311868 | val_loss: 0.0311915 | Time: 1701.98 ms [2022-03-23 13:57:42 main:574] : INFO : Epoch 2047 | loss: 0.0311872 | val_loss: 0.0311922 | Time: 1701.99 ms [2022-03-23 13:57:44 main:574] : INFO : Epoch 2048 | loss: 0.0311866 | val_loss: 0.0311913 | Time: 1728.53 ms [2022-03-23 13:57:44 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311913 [2022-03-23 13:57:44 main:603] : INFO : Saving end state to config to file [2022-03-23 13:57:44 main:608] : INFO : Success, exiting.. 13:57:44 (8528): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)