Task 11218388
| Name | ParityModified-1639950398-30656-3-0_0 |
| Workunit | 8490105 |
| Created | 1 Feb 2022, 9:53:07 UTC |
| Sent | 15 Feb 2022, 21:28:19 UTC |
| Report deadline | 23 Feb 2022, 21:28:19 UTC |
| Received | 22 Feb 2022, 20:20:46 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 17116 |
| Run time | 47 min 2 sec |
| CPU time | 45 min 42 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 9,205.27 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.63 GB |
| Peak swap size | 3.66 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 1424.86 ms [2022-02-21 13:51:33 main:574] : INFO : Epoch 1779 | loss: 0.0311567 | val_loss: 0.0311705 | Time: 1389.17 ms [2022-02-21 13:51:44 main:574] : INFO : Epoch 1780 | loss: 0.0311578 | val_loss: 0.0311707 | Time: 11435.2 ms [2022-02-21 13:51:45 main:574] : INFO : Epoch 1781 | loss: 0.0311576 | val_loss: 0.0311712 | Time: 1373.12 ms [2022-02-21 13:51:47 main:574] : INFO : Epoch 1782 | loss: 0.0311602 | val_loss: 0.0311702 | Time: 1373.8 ms [2022-02-21 13:51:48 main:574] : INFO : Epoch 1783 | loss: 0.0311582 | val_loss: 0.0311693 | Time: 1376.6 ms [2022-02-21 13:51:50 main:574] : INFO : Epoch 1784 | loss: 0.0311564 | val_loss: 0.0311704 | Time: 1378.91 ms [2022-02-21 13:51:51 main:574] : INFO : Epoch 1785 | loss: 0.0311559 | val_loss: 0.0311689 | Time: 1379.11 ms [2022-02-21 13:51:52 main:574] : INFO : Epoch 1786 | loss: 0.0311559 | val_loss: 0.0311681 | Time: 1371.2 ms [2022-02-21 13:51:54 main:574] : INFO : Epoch 1787 | loss: 0.0311565 | val_loss: 0.0311693 | Time: 1376.15 ms [2022-02-21 13:51:55 main:574] : INFO : Epoch 1788 | loss: 0.0311596 | val_loss: 0.0311684 | Time: 1379.63 ms [2022-02-21 13:51:56 main:574] : INFO : Epoch 1789 | loss: 0.0311568 | val_loss: 0.0311728 | Time: 1377.44 ms [2022-02-21 13:51:58 main:574] : INFO : Epoch 1790 | loss: 0.0311573 | val_loss: 0.0311667 | Time: 1375.61 ms [2022-02-21 13:51:59 main:574] : INFO : Epoch 1791 | loss: 0.0311583 | val_loss: 0.0311664 | Time: 1380.24 ms [2022-02-21 13:52:01 main:574] : INFO : Epoch 1792 | loss: 0.0311606 | val_loss: 0.0311664 | Time: 1378.32 ms [2022-02-21 13:52:02 main:574] : INFO : Epoch 1793 | loss: 0.0311638 | val_loss: 0.0311674 | Time: 1391.75 ms [2022-02-21 13:52:03 main:574] : INFO : Epoch 1794 | loss: 0.0311651 | val_loss: 0.0311698 | Time: 1377.15 ms [2022-02-21 13:52:05 main:574] : INFO : Epoch 1795 | loss: 0.0311644 | val_loss: 0.0311739 | Time: 1369.33 ms [2022-02-21 13:52:06 main:574] : INFO : Epoch 1796 | loss: 0.0311641 | val_loss: 0.0311696 | Time: 1371.56 ms [2022-02-21 13:52:07 main:574] : INFO : Epoch 1797 | loss: 0.0311626 | val_loss: 0.031168 | Time: 1373.82 ms [2022-02-21 13:52:09 main:574] : INFO : Epoch 1798 | loss: 0.0311611 | val_loss: 0.0311714 | Time: 1374.73 ms [2022-02-21 13:52:10 main:574] : INFO : Epoch 1799 | loss: 0.0311614 | val_loss: 0.0311702 | Time: 1386.6 ms [2022-02-21 13:52:12 main:574] : INFO : Epoch 1800 | loss: 0.0311611 | val_loss: 0.0311694 | Time: 1409.03 ms [2022-02-21 13:52:13 main:574] : INFO : Epoch 1801 | loss: 0.0311601 | val_loss: 0.0311693 | Time: 1380.49 ms [2022-02-21 13:52:14 main:574] : INFO : Epoch 1802 | loss: 0.0311596 | val_loss: 0.0311725 | Time: 1370.53 ms [2022-02-21 13:52:16 main:574] : INFO : Epoch 1803 | loss: 0.0311592 | val_loss: 0.0311794 | Time: 1391.78 ms [2022-02-21 13:52:17 main:574] : INFO : Epoch 1804 | loss: 0.0311616 | val_loss: 0.0311702 | Time: 1382.78 ms [2022-02-21 13:52:19 main:574] : INFO : Epoch 1805 | loss: 0.0311588 | val_loss: 0.0311711 | Time: 1377.66 ms [2022-02-21 13:52:20 main:574] : INFO : Epoch 1806 | loss: 0.0311594 | val_loss: 0.0311746 | Time: 1369.71 ms [2022-02-21 13:52:21 main:574] : INFO : Epoch 1807 | loss: 0.0311596 | val_loss: 0.0311709 | Time: 1372.68 ms [2022-02-21 13:52:23 main:574] : INFO : Epoch 1808 | loss: 0.031158 | val_loss: 0.0311724 | Time: 1386.67 ms [2022-02-21 13:52:24 main:574] : INFO : Epoch 1809 | loss: 0.0311573 | val_loss: 0.0311696 | Time: 1376.51 ms [2022-02-21 13:52:25 main:574] : INFO : Epoch 1810 | loss: 0.0311562 | val_loss: 0.031171 | Time: 1404.69 ms [2022-02-21 13:52:27 main:574] : INFO : Epoch 1811 | loss: 0.031157 | val_loss: 0.0311705 | Time: 1398.06 ms [2022-02-21 13:52:28 main:574] : INFO : Epoch 1812 | loss: 0.0311556 | val_loss: 0.0311692 | Time: 1387.92 ms [2022-02-21 13:52:30 main:574] : INFO : Epoch 1813 | loss: 0.0311567 | val_loss: 0.0311699 | Time: 1386.2 ms [2022-02-21 13:52:31 main:574] : INFO : Epoch 1814 | loss: 0.0311558 | val_loss: 0.0311734 | Time: 1379.8 ms [2022-02-21 13:52:32 main:574] : INFO : Epoch 1815 | loss: 0.0311558 | val_loss: 0.0311687 | Time: 1389.37 ms [2022-02-21 13:52:34 main:574] : INFO : Epoch 1816 | loss: 0.0311551 | val_loss: 0.0311698 | Time: 1384.07 ms [2022-02-21 13:52:35 main:574] : INFO : Epoch 1817 | loss: 0.0311546 | val_loss: 0.03117 | Time: 1381.96 ms [2022-02-21 13:52:37 main:574] : INFO : Epoch 1818 | loss: 0.0311568 | val_loss: 0.0311811 | Time: 1383.16 ms [2022-02-21 13:52:38 main:574] : INFO : Epoch 1819 | loss: 0.0311581 | val_loss: 0.0311792 | Time: 1387.21 ms [2022-02-21 13:52:39 main:574] : INFO : Epoch 1820 | loss: 0.0311574 | val_loss: 0.0311725 | Time: 1366.43 ms [2022-02-21 13:52:41 main:574] : INFO : Epoch 1821 | loss: 0.0311552 | val_loss: 0.0311736 | Time: 1383.83 ms [2022-02-21 13:52:42 main:574] : INFO : Epoch 1822 | loss: 0.0311575 | val_loss: 0.0311738 | Time: 1394.72 ms [2022-02-21 13:52:44 main:574] : INFO : Epoch 1823 | loss: 0.0311616 | val_loss: 0.0311722 | Time: 1399.1 ms [2022-02-21 13:52:45 main:574] : INFO : Epoch 1824 | loss: 0.0311608 | val_loss: 0.0311729 | Time: 1408.8 ms [2022-02-21 13:52:46 main:574] : INFO : Epoch 1825 | loss: 0.0311603 | val_loss: 0.031172 | Time: 1385.69 ms [2022-02-21 13:52:48 main:574] : INFO : Epoch 1826 | loss: 0.0311622 | val_loss: 0.0311735 | Time: 1379.15 ms [2022-02-21 13:52:49 main:574] : INFO : Epoch 1827 | loss: 0.0311608 | val_loss: 0.0311722 | Time: 1393.27 ms [2022-02-21 13:52:50 main:574] : INFO : Epoch 1828 | loss: 0.0311599 | val_loss: 0.031172 | Time: 1382.46 ms [2022-02-21 13:52:52 main:574] : INFO : Epoch 1829 | loss: 0.031159 | val_loss: 0.0311739 | Time: 1384.27 ms [2022-02-21 13:52:53 main:574] : INFO : Epoch 1830 | loss: 0.0311587 | val_loss: 0.0311713 | Time: 1383.98 ms [2022-02-21 13:52:55 main:574] : INFO : Epoch 1831 | loss: 0.0311604 | val_loss: 0.0311751 | Time: 1388.1 ms [2022-02-21 13:52:56 main:574] : INFO : Epoch 1832 | loss: 0.0311602 | val_loss: 0.0311788 | Time: 1376.58 ms [2022-02-21 13:52:57 main:574] : INFO : Epoch 1833 | loss: 0.0311598 | val_loss: 0.031171 | Time: 1395.37 ms [2022-02-21 13:52:59 main:574] : INFO : Epoch 1834 | loss: 0.031159 | val_loss: 0.0311732 | Time: 1386.02 ms [2022-02-21 13:53:00 main:574] : INFO : Epoch 1835 | loss: 0.031161 | val_loss: 0.0311752 | Time: 1397.43 ms [2022-02-21 13:53:02 main:574] : INFO : Epoch 1836 | loss: 0.031162 | val_loss: 0.0311766 | Time: 1383.51 ms [2022-02-21 13:53:03 main:574] : INFO : Epoch 1837 | loss: 0.0311641 | val_loss: 0.0311737 | Time: 1402.2 ms [2022-02-21 13:53:04 main:574] : INFO : Epoch 1838 | loss: 0.0311651 | val_loss: 0.031175 | Time: 1385.11 ms [2022-02-21 13:53:06 main:574] : INFO : Epoch 1839 | loss: 0.0311722 | val_loss: 0.0311775 | Time: 1376.91 ms [2022-02-21 13:53:07 main:574] : INFO : Epoch 1840 | loss: 0.031171 | val_loss: 0.0311775 | Time: 1382.99 ms [2022-02-21 13:53:09 main:574] : INFO : Epoch 1841 | loss: 0.0311695 | val_loss: 0.0311779 | Time: 1386.19 ms [2022-02-21 13:53:10 main:574] : INFO : Epoch 1842 | loss: 0.0311705 | val_loss: 0.0311795 | Time: 1378.81 ms [2022-02-21 13:53:11 main:574] : INFO : Epoch 1843 | loss: 0.0311679 | val_loss: 0.0311773 | Time: 1377.13 ms [2022-02-21 13:53:13 main:574] : INFO : Epoch 1844 | loss: 0.0311672 | val_loss: 0.0311774 | Time: 1390.7 ms [2022-02-21 13:53:14 main:574] : INFO : Epoch 1845 | loss: 0.0311653 | val_loss: 0.0311805 | Time: 1373.3 ms [2022-02-21 13:53:15 main:574] : INFO : Epoch 1846 | loss: 0.0311673 | val_loss: 0.0311751 | Time: 1389.15 ms [2022-02-21 13:53:17 main:574] : INFO : Epoch 1847 | loss: 0.0311652 | val_loss: 0.0311763 | Time: 1410.08 ms [2022-02-21 13:53:18 main:574] : INFO : Epoch 1848 | loss: 0.0311647 | val_loss: 0.0311797 | Time: 1370.77 ms [2022-02-21 13:53:20 main:574] : INFO : Epoch 1849 | loss: 0.0311752 | val_loss: 0.0311852 | Time: 1383.94 ms [2022-02-21 13:53:21 main:574] : INFO : Epoch 1850 | loss: 0.0311824 | val_loss: 0.031184 | Time: 1391.02 ms [2022-02-21 13:53:22 main:574] : INFO : Epoch 1851 | loss: 0.031182 | val_loss: 0.0311838 | Time: 1378.83 ms [2022-02-21 13:53:24 main:574] : INFO : Epoch 1852 | loss: 0.0311789 | val_loss: 0.0311856 | Time: 1393.16 ms [2022-02-21 13:53:25 main:574] : INFO : Epoch 1853 | loss: 0.0311774 | val_loss: 0.0311851 | Time: 1392.8 ms [2022-02-21 13:53:27 main:574] : INFO : Epoch 1854 | loss: 0.0311739 | val_loss: 0.0311811 | Time: 1380.76 ms [2022-02-21 13:53:28 main:574] : INFO : Epoch 1855 | loss: 0.0311721 | val_loss: 0.0311814 | Time: 1396.7 ms [2022-02-21 13:53:29 main:574] : INFO : Epoch 1856 | loss: 0.0311716 | val_loss: 0.0311814 | Time: 1388.41 ms [2022-02-21 13:53:31 main:574] : INFO : Epoch 1857 | loss: 0.0311705 | val_loss: 0.031182 | Time: 1374.75 ms [2022-02-21 13:53:32 main:574] : INFO : Epoch 1858 | loss: 0.0311704 | val_loss: 0.0311821 | Time: 1410.67 ms [2022-02-21 13:53:34 main:574] : INFO : Epoch 1859 | loss: 0.0311691 | val_loss: 0.0311806 | Time: 1389.11 ms [2022-02-21 13:53:35 main:574] : INFO : Epoch 1860 | loss: 0.0311692 | val_loss: 0.0311801 | Time: 1377.5 ms [2022-02-21 13:53:36 main:574] : INFO : Epoch 1861 | loss: 0.0311699 | val_loss: 0.031181 | Time: 1385.55 ms [2022-02-21 13:53:38 main:574] : INFO : Epoch 1862 | loss: 0.0311697 | val_loss: 0.0311854 | Time: 1405.6 ms [2022-02-21 13:53:39 main:574] : INFO : Epoch 1863 | loss: 0.0311692 | val_loss: 0.0311784 | Time: 1382.53 ms [2022-02-21 13:53:41 main:574] : INFO : Epoch 1864 | loss: 0.0311674 | val_loss: 0.0311795 | Time: 1427.32 ms [2022-02-21 13:53:42 main:574] : INFO : Epoch 1865 | loss: 0.0311678 | val_loss: 0.0311816 | Time: 1391.83 ms [2022-02-21 13:53:43 main:574] : INFO : Epoch 1866 | loss: 0.0311689 | val_loss: 0.0311774 | Time: 1385.51 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-21 14:07:29 main:435] : INFO : Set logging level to 1 [2022-02-21 14:07:29 main:441] : INFO : Running in BOINC Client mode [2022-02-21 14:07:29 main:444] : INFO : Resolving all filenames [2022-02-21 14:07:30 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-21 14:07:30 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-21 14:07:30 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-21 14:07:30 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-21 14:07:30 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-21 14:07:30 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-21 14:07:30 main:474] : INFO : Configuration: [2022-02-21 14:07:30 main:475] : INFO : Model type: GRU [2022-02-21 14:07:30 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-21 14:07:30 main:477] : INFO : Max Epochs: 2048 [2022-02-21 14:07:30 main:478] : INFO : Batch Size: 128 [2022-02-21 14:07:30 main:479] : INFO : Learning Rate: 0.01 [2022-02-21 14:07:30 main:480] : INFO : Patience: 10 [2022-02-21 14:07:30 main:481] : INFO : Hidden Width: 12 [2022-02-21 14:07:30 main:482] : INFO : # Recurrent Layers: 4 [2022-02-21 14:07:30 main:483] : INFO : # Backend Layers: 4 [2022-02-21 14:07:30 main:484] : INFO : # Threads: 1 [2022-02-21 14:07:30 main:486] : INFO : Preparing Dataset [2022-02-21 14:07:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-21 14:07:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-21 14:07:31 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-21 14:07:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-21 14:07:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-21 14:07:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-21 14:07:31 main:494] : INFO : Creating Model [2022-02-21 14:07:31 main:507] : INFO : Preparing config file [2022-02-21 14:07:31 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-21 14:07:31 main:512] : INFO : Loading config [2022-02-21 14:07:31 main:514] : INFO : Loading state [2022-02-21 14:07:32 main:559] : INFO : Loading DataLoader into Memory [2022-02-21 14:07:32 main:562] : INFO : Starting Training [2022-02-21 14:07:33 main:574] : INFO : Epoch 1866 | loss: 0.0311983 | val_loss: 0.0311813 | Time: 1566.71 ms [2022-02-21 14:07:35 main:574] : INFO : Epoch 1867 | loss: 0.0311725 | val_loss: 0.031179 | Time: 1388.85 ms [2022-02-21 14:07:36 main:574] : INFO : Epoch 1868 | loss: 0.0311689 | val_loss: 0.0311773 | Time: 1393.99 ms [2022-02-21 14:07:37 main:574] : INFO : Epoch 1869 | loss: 0.0311672 | val_loss: 0.031176 | Time: 1400.38 ms [2022-02-21 14:07:39 main:574] : INFO : Epoch 1870 | loss: 0.031166 | val_loss: 0.0311798 | Time: 1406.8 ms [2022-02-21 14:07:40 main:574] : INFO : Epoch 1871 | loss: 0.0311655 | val_loss: 0.0311766 | Time: 1408.11 ms [2022-02-21 14:07:42 main:574] : INFO : Epoch 1872 | loss: 0.0311657 | val_loss: 0.0311761 | Time: 1391.25 ms [2022-02-21 14:07:43 main:574] : INFO : Epoch 1873 | loss: 0.0311644 | val_loss: 0.0311772 | Time: 1402.97 ms [2022-02-21 14:07:44 main:574] : INFO : Epoch 1874 | loss: 0.0311646 | val_loss: 0.0311819 | Time: 1392.69 ms [2022-02-21 14:07:46 main:574] : INFO : Epoch 1875 | loss: 0.0311711 | val_loss: 0.031179 | Time: 1395.03 ms [2022-02-21 14:07:47 main:574] : INFO : Epoch 1876 | loss: 0.031169 | val_loss: 0.0311812 | Time: 1393.26 ms [2022-02-21 14:07:49 main:574] : INFO : Epoch 1877 | loss: 0.0311658 | val_loss: 0.0311789 | Time: 1397.23 ms [2022-02-21 14:07:50 main:574] : INFO : Epoch 1878 | loss: 0.0311652 | val_loss: 0.0311845 | Time: 1404.21 ms [2022-02-21 14:07:51 main:574] : INFO : Epoch 1879 | loss: 0.0311671 | val_loss: 0.0311806 | Time: 1408.65 ms [2022-02-21 14:07:53 main:574] : INFO : Epoch 1880 | loss: 0.0311664 | val_loss: 0.0311833 | Time: 1397.5 ms [2022-02-21 14:07:54 main:574] : INFO : Epoch 1881 | loss: 0.0311671 | val_loss: 0.0311802 | Time: 1397.31 ms [2022-02-21 14:07:56 main:574] : INFO : Epoch 1882 | loss: 0.0311648 | val_loss: 0.0311766 | Time: 1396.69 ms [2022-02-21 14:07:57 main:574] : INFO : Epoch 1883 | loss: 0.0311651 | val_loss: 0.0311845 | Time: 1409.13 ms [2022-02-21 14:07:59 main:574] : INFO : Epoch 1884 | loss: 0.0311644 | val_loss: 0.0311751 | Time: 1400.1 ms [2022-02-21 14:08:00 main:574] : INFO : Epoch 1885 | loss: 0.0311636 | val_loss: 0.0311819 | Time: 1400.69 ms [2022-02-21 14:08:01 main:574] : INFO : Epoch 1886 | loss: 0.0311625 | val_loss: 0.0311775 | Time: 1406.9 ms [2022-02-21 14:08:03 main:574] : INFO : Epoch 1887 | loss: 0.0311628 | val_loss: 0.0311801 | Time: 1393.81 ms [2022-02-21 14:08:04 main:574] : INFO : Epoch 1888 | loss: 0.0311613 | val_loss: 0.0311759 | Time: 1407.15 ms [2022-02-21 14:08:06 main:574] : INFO : Epoch 1889 | loss: 0.0311626 | val_loss: 0.0311768 | Time: 1392.75 ms [2022-02-21 14:08:07 main:574] : INFO : Epoch 1890 | loss: 0.0311641 | val_loss: 0.0311866 | Time: 1402.24 ms [2022-02-21 14:08:08 main:574] : INFO : Epoch 1891 | loss: 0.0311627 | val_loss: 0.0311813 | Time: 1399.64 ms [2022-02-21 14:08:10 main:574] : INFO : Epoch 1892 | loss: 0.031162 | val_loss: 0.0311777 | Time: 1398.53 ms [2022-02-21 14:08:11 main:574] : INFO : Epoch 1893 | loss: 0.0311606 | val_loss: 0.0311778 | Time: 1406.16 ms [2022-02-21 14:08:13 main:574] : INFO : Epoch 1894 | loss: 0.0311599 | val_loss: 0.0311804 | Time: 1393.74 ms [2022-02-21 14:08:14 main:574] : INFO : Epoch 1895 | loss: 0.0311581 | val_loss: 0.0311816 | Time: 1419.35 ms [2022-02-21 14:08:15 main:574] : INFO : Epoch 1896 | loss: 0.0311588 | val_loss: 0.03118 | Time: 1413.4 ms [2022-02-21 14:08:17 main:574] : INFO : Epoch 1897 | loss: 0.0311605 | val_loss: 0.031179 | Time: 1400.48 ms [2022-02-21 14:08:18 main:574] : INFO : Epoch 1898 | loss: 0.0311598 | val_loss: 0.0311788 | Time: 1395.59 ms [2022-02-21 14:08:20 main:574] : INFO : Epoch 1899 | loss: 0.0311622 | val_loss: 0.0311848 | Time: 1401.6 ms [2022-02-21 14:08:21 main:574] : INFO : Epoch 1900 | loss: 0.0311656 | val_loss: 0.0311794 | Time: 1407.79 ms [2022-02-21 14:08:22 main:574] : INFO : Epoch 1901 | loss: 0.031165 | val_loss: 0.0311818 | Time: 1394.33 ms [2022-02-21 14:08:24 main:574] : INFO : Epoch 1902 | loss: 0.031164 | val_loss: 0.0311775 | Time: 1393.45 ms [2022-02-21 14:08:25 main:574] : INFO : Epoch 1903 | loss: 0.0311658 | val_loss: 0.0311785 | Time: 1400.29 ms [2022-02-21 14:08:27 main:574] : INFO : Epoch 1904 | loss: 0.031163 | val_loss: 0.0311759 | Time: 1395.33 ms [2022-02-21 14:08:28 main:574] : INFO : Epoch 1905 | loss: 0.0311631 | val_loss: 0.0311825 | Time: 1404.73 ms [2022-02-21 14:08:29 main:574] : INFO : Epoch 1906 | loss: 0.0311618 | val_loss: 0.031178 | Time: 1401.36 ms [2022-02-21 14:08:31 main:574] : INFO : Epoch 1907 | loss: 0.0311618 | val_loss: 0.0311806 | Time: 1408.08 ms [2022-02-21 14:08:32 main:574] : INFO : Epoch 1908 | loss: 0.0311609 | val_loss: 0.0311776 | Time: 1400.94 ms [2022-02-21 14:08:34 main:574] : INFO : Epoch 1909 | loss: 0.0311604 | val_loss: 0.0311748 | Time: 1399.26 ms [2022-02-21 14:08:35 main:574] : INFO : Epoch 1910 | loss: 0.0311602 | val_loss: 0.0311738 | Time: 1407.74 ms [2022-02-21 14:08:36 main:574] : INFO : Epoch 1911 | loss: 0.0311592 | val_loss: 0.0311754 | Time: 1396.4 ms [2022-02-21 14:08:38 main:574] : INFO : Epoch 1912 | loss: 0.031159 | val_loss: 0.0311732 | Time: 1401.03 ms [2022-02-21 14:08:39 main:574] : INFO : Epoch 1913 | loss: 0.031159 | val_loss: 0.0311732 | Time: 1403.2 ms [2022-02-21 14:08:41 main:574] : INFO : Epoch 1914 | loss: 0.031158 | val_loss: 0.0311725 | Time: 1537.02 ms [2022-02-21 14:08:42 main:574] : INFO : Epoch 1915 | loss: 0.0311575 | val_loss: 0.0311754 | Time: 1411.44 ms [2022-02-21 14:08:44 main:574] : INFO : Epoch 1916 | loss: 0.0311578 | val_loss: 0.0311743 | Time: 1383.15 ms [2022-02-21 14:08:45 main:574] : INFO : Epoch 1917 | loss: 0.0311595 | val_loss: 0.0311717 | Time: 1402.11 ms [2022-02-21 14:08:46 main:574] : INFO : Epoch 1918 | loss: 0.0311581 | val_loss: 0.031172 | Time: 1403.84 ms [2022-02-21 14:08:48 main:574] : INFO : Epoch 1919 | loss: 0.031158 | val_loss: 0.0311762 | Time: 1420.93 ms [2022-02-21 14:08:49 main:574] : INFO : Epoch 1920 | loss: 0.0311608 | val_loss: 0.0311729 | Time: 1400.38 ms [2022-02-21 14:08:51 main:574] : INFO : Epoch 1921 | loss: 0.0311598 | val_loss: 0.0311748 | Time: 1390.01 ms [2022-02-21 14:08:52 main:574] : INFO : Epoch 1922 | loss: 0.0311596 | val_loss: 0.0311719 | Time: 1392.09 ms [2022-02-21 14:08:53 main:574] : INFO : Epoch 1923 | loss: 0.0311594 | val_loss: 0.0311732 | Time: 1395.71 ms [2022-02-21 14:08:55 main:574] : INFO : Epoch 1924 | loss: 0.0311604 | val_loss: 0.0311751 | Time: 1391.15 ms [2022-02-21 14:08:56 main:574] : INFO : Epoch 1925 | loss: 0.0311601 | val_loss: 0.0311746 | Time: 1388.83 ms [2022-02-21 14:08:58 main:574] : INFO : Epoch 1926 | loss: 0.0311593 | val_loss: 0.0311722 | Time: 1390.4 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-21 14:13:28 main:435] : INFO : Set logging level to 1 [2022-02-21 14:13:28 main:441] : INFO : Running in BOINC Client mode [2022-02-21 14:13:28 main:444] : INFO : Resolving all filenames [2022-02-21 14:13:28 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-21 14:13:28 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-21 14:13:28 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-21 14:13:28 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-21 14:13:28 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-21 14:13:28 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-21 14:13:28 main:474] : INFO : Configuration: [2022-02-21 14:13:28 main:475] : INFO : Model type: GRU [2022-02-21 14:13:28 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-21 14:13:28 main:477] : INFO : Max Epochs: 2048 [2022-02-21 14:13:28 main:478] : INFO : Batch Size: 128 [2022-02-21 14:13:28 main:479] : INFO : Learning Rate: 0.01 [2022-02-21 14:13:28 main:480] : INFO : Patience: 10 [2022-02-21 14:13:28 main:481] : INFO : Hidden Width: 12 [2022-02-21 14:13:28 main:482] : INFO : # Recurrent Layers: 4 [2022-02-21 14:13:28 main:483] : INFO : # Backend Layers: 4 [2022-02-21 14:13:28 main:484] : INFO : # Threads: 1 [2022-02-21 14:13:28 main:486] : INFO : Preparing Dataset [2022-02-21 14:13:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-21 14:13:28 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-21 14:13:29 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-21 14:13:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-21 14:13:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-21 14:13:29 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-21 14:13:29 main:494] : INFO : Creating Model [2022-02-21 14:13:29 main:507] : INFO : Preparing config file [2022-02-21 14:13:29 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-21 14:13:29 main:512] : INFO : Loading config [2022-02-21 14:13:29 main:514] : INFO : Loading state [2022-02-21 14:13:30 main:559] : INFO : Loading DataLoader into Memory [2022-02-21 14:13:30 main:562] : INFO : Starting Training [2022-02-21 14:13:32 main:574] : INFO : Epoch 1866 | loss: 0.0311963 | val_loss: 0.0311834 | Time: 1583.17 ms [2022-02-21 14:13:33 main:574] : INFO : Epoch 1867 | loss: 0.0311699 | val_loss: 0.0311813 | Time: 1367.07 ms [2022-02-21 14:13:34 main:574] : INFO : Epoch 1868 | loss: 0.0311677 | val_loss: 0.0311783 | Time: 1358.99 ms [2022-02-21 14:13:36 main:574] : INFO : Epoch 1869 | loss: 0.0311668 | val_loss: 0.0311774 | Time: 1364.51 ms [2022-02-21 14:13:37 main:574] : INFO : Epoch 1870 | loss: 0.0311661 | val_loss: 0.0311778 | Time: 1373.08 ms [2022-02-21 14:13:38 main:574] : INFO : Epoch 1871 | loss: 0.031166 | val_loss: 0.0311757 | Time: 1359.94 ms [2022-02-21 14:13:40 main:574] : INFO : Epoch 1872 | loss: 0.0311681 | val_loss: 0.0311775 | Time: 1369.36 ms [2022-02-21 14:13:41 main:574] : INFO : Epoch 1873 | loss: 0.0311659 | val_loss: 0.031175 | Time: 1365.21 ms [2022-02-21 14:13:42 main:574] : INFO : Epoch 1874 | loss: 0.0311631 | val_loss: 0.0311745 | Time: 1362.13 ms [2022-02-21 14:13:44 main:574] : INFO : Epoch 1875 | loss: 0.031163 | val_loss: 0.0311765 | Time: 1367.38 ms [2022-02-21 14:13:45 main:574] : INFO : Epoch 1876 | loss: 0.0311616 | val_loss: 0.0311728 | Time: 1366.76 ms [2022-02-21 14:13:47 main:574] : INFO : Epoch 1877 | loss: 0.0311614 | val_loss: 0.0311745 | Time: 1362.85 ms [2022-02-21 14:13:48 main:574] : INFO : Epoch 1878 | loss: 0.0311605 | val_loss: 0.0311739 | Time: 1364.12 ms [2022-02-21 14:13:49 main:574] : INFO : Epoch 1879 | loss: 0.0311616 | val_loss: 0.0311779 | Time: 1362.83 ms [2022-02-21 14:13:51 main:574] : INFO : Epoch 1880 | loss: 0.0311625 | val_loss: 0.0311759 | Time: 1372.18 ms [2022-02-21 14:13:52 main:574] : INFO : Epoch 1881 | loss: 0.0311617 | val_loss: 0.031177 | Time: 1363.05 ms [2022-02-21 14:13:53 main:574] : INFO : Epoch 1882 | loss: 0.0311616 | val_loss: 0.0311797 | Time: 1373.92 ms [2022-02-21 14:13:55 main:574] : INFO : Epoch 1883 | loss: 0.031162 | val_loss: 0.0311798 | Time: 1381.14 ms [2022-02-21 14:13:56 main:574] : INFO : Epoch 1884 | loss: 0.0311627 | val_loss: 0.0311811 | Time: 1364.43 ms [2022-02-21 14:13:58 main:574] : INFO : Epoch 1885 | loss: 0.0311654 | val_loss: 0.0311748 | Time: 1377.16 ms [2022-02-21 14:13:59 main:574] : INFO : Epoch 1886 | loss: 0.031162 | val_loss: 0.0311785 | Time: 1368.77 ms [2022-02-21 14:14:00 main:574] : INFO : Epoch 1887 | loss: 0.0311629 | val_loss: 0.0311773 | Time: 1366.22 ms [2022-02-21 14:14:02 main:574] : INFO : Epoch 1888 | loss: 0.0311616 | val_loss: 0.0311778 | Time: 1382.34 ms [2022-02-21 14:14:03 main:574] : INFO : Epoch 1889 | loss: 0.0311595 | val_loss: 0.0311791 | Time: 1369.59 ms [2022-02-21 14:14:04 main:574] : INFO : Epoch 1890 | loss: 0.0311599 | val_loss: 0.031179 | Time: 1372.81 ms [2022-02-21 14:14:06 main:574] : INFO : Epoch 1891 | loss: 0.0311602 | val_loss: 0.0311845 | Time: 1372.02 ms [2022-02-21 14:14:07 main:574] : INFO : Epoch 1892 | loss: 0.03116 | val_loss: 0.0311787 | Time: 1362.7 ms [2022-02-21 14:14:09 main:574] : INFO : Epoch 1893 | loss: 0.0311618 | val_loss: 0.031182 | Time: 1370.87 ms [2022-02-21 14:14:10 main:574] : INFO : Epoch 1894 | loss: 0.0311643 | val_loss: 0.0311763 | Time: 1370.49 ms [2022-02-21 14:14:11 main:574] : INFO : Epoch 1895 | loss: 0.0311648 | val_loss: 0.0311729 | Time: 1372.61 ms [2022-02-21 14:14:13 main:574] : INFO : Epoch 1896 | loss: 0.0311617 | val_loss: 0.0311761 | Time: 1380.32 ms [2022-02-21 14:14:14 main:574] : INFO : Epoch 1897 | loss: 0.0311606 | val_loss: 0.031175 | Time: 1379.37 ms [2022-02-21 14:14:15 main:574] : INFO : Epoch 1898 | loss: 0.0311585 | val_loss: 0.0311737 | Time: 1366.97 ms [2022-02-21 14:14:17 main:574] : INFO : Epoch 1899 | loss: 0.0311588 | val_loss: 0.0311755 | Time: 1370.44 ms [2022-02-21 14:14:18 main:574] : INFO : Epoch 1900 | loss: 0.0311586 | val_loss: 0.0311727 | Time: 1365.63 ms [2022-02-21 14:14:20 main:574] : INFO : Epoch 1901 | loss: 0.0311578 | val_loss: 0.0311742 | Time: 1366.71 ms [2022-02-21 14:14:21 main:574] : INFO : Epoch 1902 | loss: 0.0311597 | val_loss: 0.0311718 | Time: 1364.43 ms [2022-02-21 14:14:22 main:574] : INFO : Epoch 1903 | loss: 0.0311595 | val_loss: 0.0311753 | Time: 1371.13 ms [2022-02-21 14:14:24 main:574] : INFO : Epoch 1904 | loss: 0.0311582 | val_loss: 0.0311748 | Time: 1383.39 ms [2022-02-21 14:14:25 main:574] : INFO : Epoch 1905 | loss: 0.0311648 | val_loss: 0.03118 | Time: 1370.74 ms [2022-02-21 14:14:26 main:574] : INFO : Epoch 1906 | loss: 0.0311685 | val_loss: 0.0311777 | Time: 1374.96 ms [2022-02-21 14:14:28 main:574] : INFO : Epoch 1907 | loss: 0.0311659 | val_loss: 0.0311724 | Time: 1391.75 ms [2022-02-21 14:14:29 main:574] : INFO : Epoch 1908 | loss: 0.0311644 | val_loss: 0.0311832 | Time: 1376.3 ms [2022-02-21 14:14:31 main:574] : INFO : Epoch 1909 | loss: 0.0311686 | val_loss: 0.0311765 | Time: 1371.83 ms [2022-02-21 14:14:32 main:574] : INFO : Epoch 1910 | loss: 0.0311671 | val_loss: 0.0311824 | Time: 1374.99 ms [2022-02-21 14:14:33 main:574] : INFO : Epoch 1911 | loss: 0.0311676 | val_loss: 0.0311783 | Time: 1370.39 ms [2022-02-21 14:14:35 main:574] : INFO : Epoch 1912 | loss: 0.0311659 | val_loss: 0.0311763 | Time: 1376.23 ms [2022-02-21 14:14:36 main:574] : INFO : Epoch 1913 | loss: 0.0311642 | val_loss: 0.0311733 | Time: 1373.56 ms [2022-02-21 14:14:37 main:574] : INFO : Epoch 1914 | loss: 0.0311631 | val_loss: 0.0311803 | Time: 1370.35 ms [2022-02-21 14:14:39 main:574] : INFO : Epoch 1915 | loss: 0.0311625 | val_loss: 0.0311753 | Time: 1364.05 ms [2022-02-21 14:14:40 main:574] : INFO : Epoch 1916 | loss: 0.0311612 | val_loss: 0.0311778 | Time: 1367.13 ms [2022-02-21 14:14:42 main:574] : INFO : Epoch 1917 | loss: 0.0311611 | val_loss: 0.0311761 | Time: 1372.37 ms [2022-02-21 14:14:43 main:574] : INFO : Epoch 1918 | loss: 0.0311604 | val_loss: 0.0311746 | Time: 1363.9 ms [2022-02-21 14:14:44 main:574] : INFO : Epoch 1919 | loss: 0.0311616 | val_loss: 0.0311724 | Time: 1373.72 ms [2022-02-21 14:14:46 main:574] : INFO : Epoch 1920 | loss: 0.0311608 | val_loss: 0.0311747 | Time: 1366.63 ms [2022-02-21 14:14:47 main:574] : INFO : Epoch 1921 | loss: 0.0311626 | val_loss: 0.0311723 | Time: 1375.29 ms [2022-02-21 14:14:48 main:574] : INFO : Epoch 1922 | loss: 0.0311625 | val_loss: 0.0311742 | Time: 1373.97 ms [2022-02-21 14:14:50 main:574] : INFO : Epoch 1923 | loss: 0.0311609 | val_loss: 0.0311726 | Time: 1370.12 ms [2022-02-21 14:14:51 main:574] : INFO : Epoch 1924 | loss: 0.031161 | val_loss: 0.0311711 | Time: 1364.73 ms [2022-02-21 14:14:52 main:574] : INFO : Epoch 1925 | loss: 0.0311626 | val_loss: 0.0311747 | Time: 1378.41 ms [2022-02-21 14:14:54 main:574] : INFO : Epoch 1926 | loss: 0.0311656 | val_loss: 0.0311807 | Time: 1370.17 ms [2022-02-21 14:14:55 main:574] : INFO : Epoch 1927 | loss: 0.0311619 | val_loss: 0.0311739 | Time: 1383.23 ms [2022-02-21 14:14:57 main:574] : INFO : Epoch 1928 | loss: 0.0311602 | val_loss: 0.0311708 | Time: 1372.79 ms [2022-02-21 14:14:58 main:574] : INFO : Epoch 1929 | loss: 0.0311597 | val_loss: 0.0311735 | Time: 1368.62 ms [2022-02-21 14:14:59 main:574] : INFO : Epoch 1930 | loss: 0.031159 | val_loss: 0.0311734 | Time: 1381.37 ms [2022-02-21 14:15:01 main:574] : INFO : Epoch 1931 | loss: 0.0311591 | val_loss: 0.0311724 | Time: 1372 ms [2022-02-21 14:15:02 main:574] : INFO : Epoch 1932 | loss: 0.031159 | val_loss: 0.0311735 | Time: 1367.89 ms [2022-02-21 14:15:03 main:574] : INFO : Epoch 1933 | loss: 0.0311603 | val_loss: 0.0311755 | Time: 1368.33 ms [2022-02-21 14:15:05 main:574] : INFO : Epoch 1934 | loss: 0.0311605 | val_loss: 0.031172 | Time: 1369.23 ms [2022-02-21 14:15:06 main:574] : INFO : Epoch 1935 | loss: 0.0311594 | val_loss: 0.0311723 | Time: 1363.59 ms [2022-02-21 14:15:08 main:574] : INFO : Epoch 1936 | loss: 0.0311621 | val_loss: 0.031176 | Time: 1371.89 ms [2022-02-21 14:15:09 main:574] : INFO : Epoch 1937 | loss: 0.0311623 | val_loss: 0.031175 | Time: 1366.56 ms [2022-02-21 14:15:10 main:574] : INFO : Epoch 1938 | loss: 0.0311613 | val_loss: 0.0311743 | Time: 1380.84 ms [2022-02-21 14:15:12 main:574] : INFO : Epoch 1939 | loss: 0.0311624 | val_loss: 0.0311727 | Time: 1380.28 ms [2022-02-21 14:15:13 main:574] : INFO : Epoch 1940 | loss: 0.031163 | val_loss: 0.031172 | Time: 1369.85 ms [2022-02-21 14:15:15 main:574] : INFO : Epoch 1941 | loss: 0.0311616 | val_loss: 0.03117 | Time: 1393.35 ms [2022-02-21 14:15:16 main:574] : INFO : Epoch 1942 | loss: 0.031161 | val_loss: 0.0311713 | Time: 1370.62 ms [2022-02-21 14:15:17 main:574] : INFO : Epoch 1943 | loss: 0.0311628 | val_loss: 0.0311709 | Time: 1373.23 ms [2022-02-21 14:15:19 main:574] : INFO : Epoch 1944 | loss: 0.0311626 | val_loss: 0.031173 | Time: 1373.98 ms [2022-02-21 14:15:20 main:574] : INFO : Epoch 1945 | loss: 0.0311601 | val_loss: 0.0311701 | Time: 1363.61 ms [2022-02-21 14:15:21 main:574] : INFO : Epoch 1946 | loss: 0.0311597 | val_loss: 0.0311714 | Time: 1372.9 ms [2022-02-21 14:15:23 main:574] : INFO : Epoch 1947 | loss: 0.0311609 | val_loss: 0.0311677 | Time: 1372.02 ms [2022-02-21 14:15:24 main:574] : INFO : Epoch 1948 | loss: 0.0311591 | val_loss: 0.0311699 | Time: 1367.08 ms [2022-02-21 14:15:25 main:574] : INFO : Epoch 1949 | loss: 0.0311584 | val_loss: 0.0311699 | Time: 1374.14 ms [2022-02-21 14:15:27 main:574] : INFO : Epoch 1950 | loss: 0.0311585 | val_loss: 0.031169 | Time: 1371.14 ms [2022-02-21 14:15:28 main:574] : INFO : Epoch 1951 | loss: 0.0311618 | val_loss: 0.0311757 | Time: 1403.98 ms [2022-02-21 14:15:30 main:574] : INFO : Epoch 1952 | loss: 0.0311592 | val_loss: 0.0311706 | Time: 1479.65 ms [2022-02-21 14:15:31 main:574] : INFO : Epoch 1953 | loss: 0.0311573 | val_loss: 0.0311695 | Time: 1377.46 ms [2022-02-21 14:15:33 main:574] : INFO : Epoch 1954 | loss: 0.0311589 | val_loss: 0.0311789 | Time: 1388.12 ms [2022-02-21 14:15:34 main:574] : INFO : Epoch 1955 | loss: 0.0311633 | val_loss: 0.0311699 | Time: 1412.7 ms [2022-02-21 14:15:35 main:574] : INFO : Epoch 1956 | loss: 0.0311587 | val_loss: 0.0311711 | Time: 1387.34 ms [2022-02-21 14:15:37 main:574] : INFO : Epoch 1957 | loss: 0.0311588 | val_loss: 0.0311705 | Time: 1379.65 ms [2022-02-21 14:15:38 main:574] : INFO : Epoch 1958 | loss: 0.0311582 | val_loss: 0.0311726 | Time: 1383.56 ms [2022-02-21 14:15:39 main:574] : INFO : Epoch 1959 | loss: 0.0311562 | val_loss: 0.0311706 | Time: 1381.57 ms [2022-02-21 14:15:41 main:574] : INFO : Epoch 1960 | loss: 0.0311575 | val_loss: 0.0311714 | Time: 1380.45 ms [2022-02-21 14:15:42 main:574] : INFO : Epoch 1961 | loss: 0.0311642 | val_loss: 0.0311716 | Time: 1370.46 ms [2022-02-21 14:15:44 main:574] : INFO : Epoch 1962 | loss: 0.0311614 | val_loss: 0.0311707 | Time: 1378.33 ms [2022-02-21 14:15:45 main:574] : INFO : Epoch 1963 | loss: 0.0311587 | val_loss: 0.0311713 | Time: 1382.9 ms [2022-02-21 14:15:46 main:574] : INFO : Epoch 1964 | loss: 0.0311582 | val_loss: 0.0311726 | Time: 1378.55 ms [2022-02-21 14:15:48 main:574] : INFO : Epoch 1965 | loss: 0.0311587 | val_loss: 0.0311744 | Time: 1421.47 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-22 14:15:14 main:435] : INFO : Set logging level to 1 [2022-02-22 14:15:14 main:441] : INFO : Running in BOINC Client mode [2022-02-22 14:15:14 main:444] : INFO : Resolving all filenames [2022-02-22 14:15:14 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-22 14:15:14 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-22 14:15:14 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-22 14:15:14 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-22 14:15:14 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-22 14:15:14 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-22 14:15:14 main:474] : INFO : Configuration: [2022-02-22 14:15:14 main:475] : INFO : Model type: GRU [2022-02-22 14:15:14 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-22 14:15:14 main:477] : INFO : Max Epochs: 2048 [2022-02-22 14:15:14 main:478] : INFO : Batch Size: 128 [2022-02-22 14:15:14 main:479] : INFO : Learning Rate: 0.01 [2022-02-22 14:15:14 main:480] : INFO : Patience: 10 [2022-02-22 14:15:14 main:481] : INFO : Hidden Width: 12 [2022-02-22 14:15:14 main:482] : INFO : # Recurrent Layers: 4 [2022-02-22 14:15:14 main:483] : INFO : # Backend Layers: 4 [2022-02-22 14:15:14 main:484] : INFO : # Threads: 1 [2022-02-22 14:15:14 main:486] : INFO : Preparing Dataset [2022-02-22 14:15:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-22 14:15:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-22 14:15:17 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-22 14:15:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-22 14:15:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-22 14:15:17 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-22 14:15:17 main:494] : INFO : Creating Model [2022-02-22 14:15:17 main:507] : INFO : Preparing config file [2022-02-22 14:15:17 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-22 14:15:17 main:512] : INFO : Loading config [2022-02-22 14:15:17 main:514] : INFO : Loading state [2022-02-22 14:15:19 main:559] : INFO : Loading DataLoader into Memory [2022-02-22 14:15:19 main:562] : INFO : Starting Training [2022-02-22 14:15:21 main:574] : INFO : Epoch 1866 | loss: 0.0311995 | val_loss: 0.0311832 | Time: 2318.19 ms [2022-02-22 14:15:23 main:574] : INFO : Epoch 1867 | loss: 0.0311713 | val_loss: 0.031178 | Time: 1510.95 ms [2022-02-22 14:15:24 main:574] : INFO : Epoch 1868 | loss: 0.031168 | val_loss: 0.0311781 | Time: 1540.47 ms [2022-02-22 14:15:26 main:574] : INFO : Epoch 1869 | loss: 0.0311677 | val_loss: 0.031179 | Time: 1522.57 ms [2022-02-22 14:15:27 main:574] : INFO : Epoch 1870 | loss: 0.0311688 | val_loss: 0.0311774 | Time: 1532.28 ms [2022-02-22 14:15:29 main:574] : INFO : Epoch 1871 | loss: 0.0311673 | val_loss: 0.0311774 | Time: 1529.03 ms [2022-02-22 14:15:30 main:574] : INFO : Epoch 1872 | loss: 0.0311648 | val_loss: 0.0311782 | Time: 1543.22 ms [2022-02-22 14:15:32 main:574] : INFO : Epoch 1873 | loss: 0.0311647 | val_loss: 0.0311752 | Time: 1553.5 ms [2022-02-22 14:15:33 main:574] : INFO : Epoch 1874 | loss: 0.0311636 | val_loss: 0.0311772 | Time: 1553.39 ms [2022-02-22 14:15:35 main:574] : INFO : Epoch 1875 | loss: 0.0311645 | val_loss: 0.0311771 | Time: 1535.04 ms [2022-02-22 14:15:36 main:574] : INFO : Epoch 1876 | loss: 0.0311636 | val_loss: 0.0311773 | Time: 1539.62 ms [2022-02-22 14:15:38 main:574] : INFO : Epoch 1877 | loss: 0.0311634 | val_loss: 0.0311761 | Time: 1540.69 ms [2022-02-22 14:15:40 main:574] : INFO : Epoch 1878 | loss: 0.0311633 | val_loss: 0.0311782 | Time: 1542.83 ms [2022-02-22 14:15:41 main:574] : INFO : Epoch 1879 | loss: 0.0311635 | val_loss: 0.0311711 | Time: 1567.19 ms [2022-02-22 14:15:43 main:574] : INFO : Epoch 1880 | loss: 0.0311625 | val_loss: 0.0311731 | Time: 1547.04 ms [2022-02-22 14:15:44 main:574] : INFO : Epoch 1881 | loss: 0.0311668 | val_loss: 0.0311747 | Time: 1548.19 ms [2022-02-22 14:15:46 main:574] : INFO : Epoch 1882 | loss: 0.031168 | val_loss: 0.031177 | Time: 1545.79 ms [2022-02-22 14:15:47 main:574] : INFO : Epoch 1883 | loss: 0.0311671 | val_loss: 0.0311763 | Time: 1547 ms [2022-02-22 14:15:49 main:574] : INFO : Epoch 1884 | loss: 0.0311713 | val_loss: 0.0311753 | Time: 1551.49 ms [2022-02-22 14:15:50 main:574] : INFO : Epoch 1885 | loss: 0.0311735 | val_loss: 0.0311789 | Time: 1539.31 ms [2022-02-22 14:15:52 main:574] : INFO : Epoch 1886 | loss: 0.0311714 | val_loss: 0.0311795 | Time: 1540.03 ms [2022-02-22 14:15:54 main:574] : INFO : Epoch 1887 | loss: 0.0311692 | val_loss: 0.0311809 | Time: 1546.02 ms [2022-02-22 14:16:15 main:574] : INFO : Epoch 1888 | loss: 0.0311669 | val_loss: 0.0311773 | Time: 21887.9 ms [2022-02-22 14:16:17 main:574] : INFO : Epoch 1889 | loss: 0.0311669 | val_loss: 0.0311789 | Time: 1548.27 ms [2022-02-22 14:16:18 main:574] : INFO : Epoch 1890 | loss: 0.0311655 | val_loss: 0.0311745 | Time: 1515.51 ms [2022-02-22 14:16:20 main:574] : INFO : Epoch 1891 | loss: 0.0311651 | val_loss: 0.0311735 | Time: 1525.39 ms [2022-02-22 14:16:22 main:574] : INFO : Epoch 1892 | loss: 0.0311645 | val_loss: 0.0311739 | Time: 1533.34 ms [2022-02-22 14:16:23 main:574] : INFO : Epoch 1893 | loss: 0.0311655 | val_loss: 0.0311747 | Time: 1527.82 ms [2022-02-22 14:16:25 main:574] : INFO : Epoch 1894 | loss: 0.0311624 | val_loss: 0.0311727 | Time: 1540.75 ms [2022-02-22 14:16:26 main:574] : INFO : Epoch 1895 | loss: 0.0311616 | val_loss: 0.0311744 | Time: 1512.38 ms [2022-02-22 14:16:28 main:574] : INFO : Epoch 1896 | loss: 0.0311624 | val_loss: 0.0311762 | Time: 1530.06 ms [2022-02-22 14:16:29 main:574] : INFO : Epoch 1897 | loss: 0.0311633 | val_loss: 0.0311745 | Time: 1528.89 ms [2022-02-22 14:16:31 main:574] : INFO : Epoch 1898 | loss: 0.0311628 | val_loss: 0.0311785 | Time: 1520.75 ms [2022-02-22 14:16:32 main:574] : INFO : Epoch 1899 | loss: 0.0311631 | val_loss: 0.0311737 | Time: 1539.72 ms [2022-02-22 14:16:34 main:574] : INFO : Epoch 1900 | loss: 0.0311652 | val_loss: 0.031178 | Time: 1612.03 ms [2022-02-22 14:16:35 main:574] : INFO : Epoch 1901 | loss: 0.0311642 | val_loss: 0.0311737 | Time: 1555.65 ms [2022-02-22 14:16:37 main:574] : INFO : Epoch 1902 | loss: 0.0311619 | val_loss: 0.031177 | Time: 1546.23 ms [2022-02-22 14:16:39 main:574] : INFO : Epoch 1903 | loss: 0.0311616 | val_loss: 0.0311769 | Time: 1549.18 ms [2022-02-22 14:16:40 main:574] : INFO : Epoch 1904 | loss: 0.031163 | val_loss: 0.0311727 | Time: 1553.83 ms [2022-02-22 14:16:42 main:574] : INFO : Epoch 1905 | loss: 0.0311653 | val_loss: 0.031174 | Time: 1539.82 ms [2022-02-22 14:16:43 main:574] : INFO : Epoch 1906 | loss: 0.0311658 | val_loss: 0.0311752 | Time: 1517.76 ms [2022-02-22 14:16:45 main:574] : INFO : Epoch 1907 | loss: 0.0311629 | val_loss: 0.0311722 | Time: 1527.52 ms [2022-02-22 14:16:46 main:574] : INFO : Epoch 1908 | loss: 0.0311625 | val_loss: 0.031172 | Time: 1526.4 ms [2022-02-22 14:16:48 main:574] : INFO : Epoch 1909 | loss: 0.0311622 | val_loss: 0.0311728 | Time: 1525.55 ms [2022-02-22 14:16:49 main:574] : INFO : Epoch 1910 | loss: 0.0311629 | val_loss: 0.0311739 | Time: 1522.09 ms [2022-02-22 14:16:51 main:574] : INFO : Epoch 1911 | loss: 0.0311628 | val_loss: 0.0311741 | Time: 1518.54 ms [2022-02-22 14:16:52 main:574] : INFO : Epoch 1912 | loss: 0.0311657 | val_loss: 0.0311786 | Time: 1526.18 ms [2022-02-22 14:16:54 main:574] : INFO : Epoch 1913 | loss: 0.0311628 | val_loss: 0.0311766 | Time: 1525.24 ms [2022-02-22 14:16:55 main:574] : INFO : Epoch 1914 | loss: 0.0311616 | val_loss: 0.0311745 | Time: 1520.97 ms [2022-02-22 14:16:57 main:574] : INFO : Epoch 1915 | loss: 0.03116 | val_loss: 0.0311772 | Time: 1525.08 ms [2022-02-22 14:16:58 main:574] : INFO : Epoch 1916 | loss: 0.031159 | val_loss: 0.0311748 | Time: 1522.66 ms [2022-02-22 14:17:00 main:574] : INFO : Epoch 1917 | loss: 0.0311607 | val_loss: 0.0311748 | Time: 1525.35 ms [2022-02-22 14:17:01 main:574] : INFO : Epoch 1918 | loss: 0.0311595 | val_loss: 0.0311729 | Time: 1528.57 ms [2022-02-22 14:17:03 main:574] : INFO : Epoch 1919 | loss: 0.0311587 | val_loss: 0.0311736 | Time: 1536.4 ms [2022-02-22 14:17:05 main:574] : INFO : Epoch 1920 | loss: 0.0311602 | val_loss: 0.031175 | Time: 1522.76 ms [2022-02-22 14:17:16 main:574] : INFO : Epoch 1921 | loss: 0.0311591 | val_loss: 0.0311729 | Time: 11684 ms [2022-02-22 14:17:18 main:574] : INFO : Epoch 1922 | loss: 0.0311595 | val_loss: 0.0311754 | Time: 1532.32 ms [2022-02-22 14:17:19 main:574] : INFO : Epoch 1923 | loss: 0.0311581 | val_loss: 0.0311735 | Time: 1524.5 ms [2022-02-22 14:17:21 main:574] : INFO : Epoch 1924 | loss: 0.0311577 | val_loss: 0.0311744 | Time: 1536.61 ms [2022-02-22 14:17:22 main:574] : INFO : Epoch 1925 | loss: 0.0311597 | val_loss: 0.0311745 | Time: 1543.47 ms [2022-02-22 14:17:24 main:574] : INFO : Epoch 1926 | loss: 0.0311612 | val_loss: 0.0311781 | Time: 1534 ms [2022-02-22 14:17:25 main:574] : INFO : Epoch 1927 | loss: 0.0311588 | val_loss: 0.0311768 | Time: 1539.48 ms [2022-02-22 14:17:27 main:574] : INFO : Epoch 1928 | loss: 0.031159 | val_loss: 0.0311753 | Time: 1528.88 ms [2022-02-22 14:17:28 main:574] : INFO : Epoch 1929 | loss: 0.031157 | val_loss: 0.0311728 | Time: 1529.96 ms [2022-02-22 14:17:30 main:574] : INFO : Epoch 1930 | loss: 0.0311571 | val_loss: 0.0311761 | Time: 1538.46 ms [2022-02-22 14:17:32 main:574] : INFO : Epoch 1931 | loss: 0.0311594 | val_loss: 0.0311753 | Time: 1529.41 ms [2022-02-22 14:17:33 main:574] : INFO : Epoch 1932 | loss: 0.0311613 | val_loss: 0.0311794 | Time: 1480.49 ms [2022-02-22 14:17:35 main:574] : INFO : Epoch 1933 | loss: 0.0311621 | val_loss: 0.0311764 | Time: 1535.08 ms [2022-02-22 14:17:36 main:574] : INFO : Epoch 1934 | loss: 0.0311607 | val_loss: 0.0311753 | Time: 1429.81 ms [2022-02-22 14:17:37 main:574] : INFO : Epoch 1935 | loss: 0.03116 | val_loss: 0.0311746 | Time: 1435.12 ms [2022-02-22 14:17:39 main:574] : INFO : Epoch 1936 | loss: 0.0311722 | val_loss: 0.0311804 | Time: 1424.94 ms [2022-02-22 14:17:40 main:574] : INFO : Epoch 1937 | loss: 0.0311792 | val_loss: 0.0311837 | Time: 1456.83 ms [2022-02-22 14:17:42 main:574] : INFO : Epoch 1938 | loss: 0.0311764 | val_loss: 0.031178 | Time: 1443.06 ms [2022-02-22 14:17:43 main:574] : INFO : Epoch 1939 | loss: 0.0311769 | val_loss: 0.0311771 | Time: 1425.74 ms [2022-02-22 14:17:45 main:574] : INFO : Epoch 1940 | loss: 0.0311744 | val_loss: 0.0311765 | Time: 1419.32 ms [2022-02-22 14:17:46 main:574] : INFO : Epoch 1941 | loss: 0.0311726 | val_loss: 0.0311765 | Time: 1427.75 ms [2022-02-22 14:17:47 main:574] : INFO : Epoch 1942 | loss: 0.0311711 | val_loss: 0.0311756 | Time: 1416.83 ms [2022-02-22 14:17:49 main:574] : INFO : Epoch 1943 | loss: 0.0311711 | val_loss: 0.0311762 | Time: 1414.04 ms [2022-02-22 14:17:50 main:574] : INFO : Epoch 1944 | loss: 0.031171 | val_loss: 0.0311758 | Time: 1420.69 ms [2022-02-22 14:17:52 main:574] : INFO : Epoch 1945 | loss: 0.0311709 | val_loss: 0.0311737 | Time: 1425.48 ms [2022-02-22 14:17:53 main:574] : INFO : Epoch 1946 | loss: 0.0311694 | val_loss: 0.0311772 | Time: 1423.45 ms [2022-02-22 14:17:55 main:574] : INFO : Epoch 1947 | loss: 0.0311683 | val_loss: 0.0311771 | Time: 1430.17 ms [2022-02-22 14:17:56 main:574] : INFO : Epoch 1948 | loss: 0.0311693 | val_loss: 0.0311755 | Time: 1429.48 ms [2022-02-22 14:17:57 main:574] : INFO : Epoch 1949 | loss: 0.031169 | val_loss: 0.0311765 | Time: 1430.37 ms [2022-02-22 14:17:59 main:574] : INFO : Epoch 1950 | loss: 0.0311682 | val_loss: 0.031178 | Time: 1419.06 ms [2022-02-22 14:18:00 main:574] : INFO : Epoch 1951 | loss: 0.0311687 | val_loss: 0.0311749 | Time: 1439.31 ms [2022-02-22 14:18:02 main:574] : INFO : Epoch 1952 | loss: 0.0311685 | val_loss: 0.0311757 | Time: 1430.78 ms [2022-02-22 14:18:03 main:574] : INFO : Epoch 1953 | loss: 0.0311693 | val_loss: 0.0311774 | Time: 1426.43 ms [2022-02-22 14:18:05 main:574] : INFO : Epoch 1954 | loss: 0.0311692 | val_loss: 0.0311765 | Time: 1424.85 ms [2022-02-22 14:18:06 main:574] : INFO : Epoch 1955 | loss: 0.0311698 | val_loss: 0.0311763 | Time: 1431.04 ms [2022-02-22 14:18:07 main:574] : INFO : Epoch 1956 | loss: 0.0311689 | val_loss: 0.0311763 | Time: 1449.98 ms [2022-02-22 14:18:09 main:574] : INFO : Epoch 1957 | loss: 0.0311686 | val_loss: 0.0311782 | Time: 1408.31 ms [2022-02-22 14:18:10 main:574] : INFO : Epoch 1958 | loss: 0.0311676 | val_loss: 0.0311764 | Time: 1427.95 ms [2022-02-22 14:18:12 main:574] : INFO : Epoch 1959 | loss: 0.0311686 | val_loss: 0.0311775 | Time: 1428.05 ms [2022-02-22 14:18:13 main:574] : INFO : Epoch 1960 | loss: 0.0311662 | val_loss: 0.0311773 | Time: 1425 ms [2022-02-22 14:18:15 main:574] : INFO : Epoch 1961 | loss: 0.0311673 | val_loss: 0.0311763 | Time: 1426.34 ms [2022-02-22 14:18:16 main:574] : INFO : Epoch 1962 | loss: 0.0311665 | val_loss: 0.0311747 | Time: 1425.52 ms [2022-02-22 14:18:17 main:574] : INFO : Epoch 1963 | loss: 0.0311665 | val_loss: 0.0311758 | Time: 1445.15 ms [2022-02-22 14:18:19 main:574] : INFO : Epoch 1964 | loss: 0.0311687 | val_loss: 0.0311795 | Time: 1415.65 ms [2022-02-22 14:18:20 main:574] : INFO : Epoch 1965 | loss: 0.0311681 | val_loss: 0.0311777 | Time: 1444.1 ms [2022-02-22 14:18:22 main:574] : INFO : Epoch 1966 | loss: 0.0311646 | val_loss: 0.0311762 | Time: 1441.7 ms [2022-02-22 14:18:23 main:574] : INFO : Epoch 1967 | loss: 0.0311665 | val_loss: 0.0311781 | Time: 1418.15 ms [2022-02-22 14:18:25 main:574] : INFO : Epoch 1968 | loss: 0.0311666 | val_loss: 0.03118 | Time: 1423.57 ms [2022-02-22 14:18:26 main:574] : INFO : Epoch 1969 | loss: 0.0311676 | val_loss: 0.0311849 | Time: 1426 ms [2022-02-22 14:18:28 main:574] : INFO : Epoch 1970 | loss: 0.0311663 | val_loss: 0.0311769 | Time: 1426.32 ms [2022-02-22 14:18:29 main:574] : INFO : Epoch 1971 | loss: 0.0311666 | val_loss: 0.0311772 | Time: 1416.23 ms [2022-02-22 14:18:30 main:574] : INFO : Epoch 1972 | loss: 0.0311672 | val_loss: 0.0311771 | Time: 1426.14 ms [2022-02-22 14:18:32 main:574] : INFO : Epoch 1973 | loss: 0.0311673 | val_loss: 0.0311784 | Time: 1435.29 ms [2022-02-22 14:18:33 main:574] : INFO : Epoch 1974 | loss: 0.0311663 | val_loss: 0.0311784 | Time: 1424.86 ms [2022-02-22 14:18:35 main:574] : INFO : Epoch 1975 | loss: 0.0311654 | val_loss: 0.0311775 | Time: 1426.9 ms [2022-02-22 14:18:36 main:574] : INFO : Epoch 1976 | loss: 0.0311645 | val_loss: 0.0311756 | Time: 1426.5 ms [2022-02-22 14:18:48 main:574] : INFO : Epoch 1977 | loss: 0.031165 | val_loss: 0.0311772 | Time: 11571.5 ms [2022-02-22 14:18:49 main:574] : INFO : Epoch 1978 | loss: 0.0311646 | val_loss: 0.0311781 | Time: 1414.93 ms [2022-02-22 14:18:51 main:574] : INFO : Epoch 1979 | loss: 0.0311676 | val_loss: 0.0311763 | Time: 1478.22 ms [2022-02-22 14:18:52 main:574] : INFO : Epoch 1980 | loss: 0.0311665 | val_loss: 0.03118 | Time: 1428.66 ms [2022-02-22 14:18:53 main:574] : INFO : Epoch 1981 | loss: 0.0311662 | val_loss: 0.0311749 | Time: 1416.78 ms [2022-02-22 14:18:55 main:574] : INFO : Epoch 1982 | loss: 0.0311661 | val_loss: 0.0311765 | Time: 1425.54 ms [2022-02-22 14:18:56 main:574] : INFO : Epoch 1983 | loss: 0.0311655 | val_loss: 0.0311763 | Time: 1433.89 ms [2022-02-22 14:18:58 main:574] : INFO : Epoch 1984 | loss: 0.0311677 | val_loss: 0.0311807 | Time: 1430.32 ms [2022-02-22 14:18:59 main:574] : INFO : Epoch 1985 | loss: 0.0311652 | val_loss: 0.0311766 | Time: 1439.49 ms [2022-02-22 14:19:01 main:574] : INFO : Epoch 1986 | loss: 0.0311644 | val_loss: 0.0311803 | Time: 1432.77 ms [2022-02-22 14:19:02 main:574] : INFO : Epoch 1987 | loss: 0.0311659 | val_loss: 0.0311768 | Time: 1431.55 ms [2022-02-22 14:19:03 main:574] : INFO : Epoch 1988 | loss: 0.0311654 | val_loss: 0.0311775 | Time: 1413.58 ms [2022-02-22 14:19:05 main:574] : INFO : Epoch 1989 | loss: 0.0311654 | val_loss: 0.0311771 | Time: 1437.41 ms [2022-02-22 14:19:06 main:574] : INFO : Epoch 1990 | loss: 0.0311634 | val_loss: 0.0311787 | Time: 1436.93 ms [2022-02-22 14:19:08 main:574] : INFO : Epoch 1991 | loss: 0.0311636 | val_loss: 0.0311818 | Time: 1417.18 ms [2022-02-22 14:19:09 main:574] : INFO : Epoch 1992 | loss: 0.0311641 | val_loss: 0.031178 | Time: 1421.7 ms [2022-02-22 14:19:11 main:574] : INFO : Epoch 1993 | loss: 0.0311632 | val_loss: 0.0311775 | Time: 1426.7 ms [2022-02-22 14:19:12 main:574] : INFO : Epoch 1994 | loss: 0.0311623 | val_loss: 0.0311792 | Time: 1426.79 ms [2022-02-22 14:19:13 main:574] : INFO : Epoch 1995 | loss: 0.0311635 | val_loss: 0.0311839 | Time: 1425.73 ms [2022-02-22 14:19:15 main:574] : INFO : Epoch 1996 | loss: 0.0311644 | val_loss: 0.031178 | Time: 1426.79 ms [2022-02-22 14:19:16 main:574] : INFO : Epoch 1997 | loss: 0.0311628 | val_loss: 0.0311775 | Time: 1423.92 ms [2022-02-22 14:19:18 main:574] : INFO : Epoch 1998 | loss: 0.0311625 | val_loss: 0.0311769 | Time: 1427.73 ms [2022-02-22 14:19:19 main:574] : INFO : Epoch 1999 | loss: 0.0311619 | val_loss: 0.0311801 | Time: 1424.91 ms [2022-02-22 14:19:21 main:574] : INFO : Epoch 2000 | loss: 0.0311623 | val_loss: 0.031178 | Time: 1424.36 ms [2022-02-22 14:19:22 main:574] : INFO : Epoch 2001 | loss: 0.0311616 | val_loss: 0.031177 | Time: 1426.14 ms [2022-02-22 14:19:23 main:574] : INFO : Epoch 2002 | loss: 0.031162 | val_loss: 0.0311777 | Time: 1417.79 ms [2022-02-22 14:19:25 main:574] : INFO : Epoch 2003 | loss: 0.0311647 | val_loss: 0.0311769 | Time: 1426.24 ms [2022-02-22 14:19:26 main:574] : INFO : Epoch 2004 | loss: 0.0311653 | val_loss: 0.0311765 | Time: 1434.08 ms [2022-02-22 14:19:28 main:574] : INFO : Epoch 2005 | loss: 0.0311651 | val_loss: 0.0311774 | Time: 1419.09 ms [2022-02-22 14:19:29 main:574] : INFO : Epoch 2006 | loss: 0.0311632 | val_loss: 0.0311781 | Time: 1422.56 ms [2022-02-22 14:19:31 main:574] : INFO : Epoch 2007 | loss: 0.0311632 | val_loss: 0.0311802 | Time: 1423.47 ms [2022-02-22 14:19:32 main:574] : INFO : Epoch 2008 | loss: 0.0311651 | val_loss: 0.0311767 | Time: 1417.86 ms [2022-02-22 14:19:33 main:574] : INFO : Epoch 2009 | loss: 0.0311626 | val_loss: 0.03118 | Time: 1423.58 ms [2022-02-22 14:19:35 main:574] : INFO : Epoch 2010 | loss: 0.0311638 | val_loss: 0.0311784 | Time: 1434.59 ms [2022-02-22 14:19:36 main:574] : INFO : Epoch 2011 | loss: 0.0311631 | val_loss: 0.0311772 | Time: 1417.01 ms [2022-02-22 14:19:38 main:574] : INFO : Epoch 2012 | loss: 0.0311635 | val_loss: 0.0311769 | Time: 1424.49 ms [2022-02-22 14:19:39 main:574] : INFO : Epoch 2013 | loss: 0.0311615 | val_loss: 0.0311764 | Time: 1439.23 ms [2022-02-22 14:19:41 main:574] : INFO : Epoch 2014 | loss: 0.0311615 | val_loss: 0.031177 | Time: 1418.59 ms [2022-02-22 14:19:42 main:574] : INFO : Epoch 2015 | loss: 0.0311614 | val_loss: 0.0311806 | Time: 1427.57 ms [2022-02-22 14:19:43 main:574] : INFO : Epoch 2016 | loss: 0.0311616 | val_loss: 0.0311784 | Time: 1423.75 ms [2022-02-22 14:19:45 main:574] : INFO : Epoch 2017 | loss: 0.031163 | val_loss: 0.0311799 | Time: 1426.71 ms [2022-02-22 14:19:46 main:574] : INFO : Epoch 2018 | loss: 0.0311621 | val_loss: 0.0311785 | Time: 1437.33 ms [2022-02-22 14:19:48 main:574] : INFO : Epoch 2019 | loss: 0.0311615 | val_loss: 0.0311778 | Time: 1424.85 ms [2022-02-22 14:19:49 main:574] : INFO : Epoch 2020 | loss: 0.0311612 | val_loss: 0.031187 | Time: 1419.51 ms [2022-02-22 14:19:51 main:574] : INFO : Epoch 2021 | loss: 0.0311637 | val_loss: 0.0311755 | Time: 1437.86 ms [2022-02-22 14:19:52 main:574] : INFO : Epoch 2022 | loss: 0.0311623 | val_loss: 0.0311826 | Time: 1432.59 ms [2022-02-22 14:19:53 main:574] : INFO : Epoch 2023 | loss: 0.031164 | val_loss: 0.0311776 | Time: 1424.51 ms [2022-02-22 14:19:55 main:574] : INFO : Epoch 2024 | loss: 0.0311647 | val_loss: 0.0311765 | Time: 1432.84 ms [2022-02-22 14:19:56 main:574] : INFO : Epoch 2025 | loss: 0.0311685 | val_loss: 0.0311819 | Time: 1418.86 ms [2022-02-22 14:20:08 main:574] : INFO : Epoch 2026 | loss: 0.0311697 | val_loss: 0.0311819 | Time: 11605.2 ms [2022-02-22 14:20:09 main:574] : INFO : Epoch 2027 | loss: 0.0311682 | val_loss: 0.0311798 | Time: 1429.06 ms [2022-02-22 14:20:11 main:574] : INFO : Epoch 2028 | loss: 0.0311657 | val_loss: 0.0311789 | Time: 1434.07 ms [2022-02-22 14:20:12 main:574] : INFO : Epoch 2029 | loss: 0.0311649 | val_loss: 0.0311781 | Time: 1421.82 ms [2022-02-22 14:20:14 main:574] : INFO : Epoch 2030 | loss: 0.0311652 | val_loss: 0.0311795 | Time: 1418.57 ms [2022-02-22 14:20:15 main:574] : INFO : Epoch 2031 | loss: 0.0311654 | val_loss: 0.0311857 | Time: 1431.5 ms [2022-02-22 14:20:16 main:574] : INFO : Epoch 2032 | loss: 0.0311685 | val_loss: 0.0311792 | Time: 1421.11 ms [2022-02-22 14:20:18 main:574] : INFO : Epoch 2033 | loss: 0.0311652 | val_loss: 0.0311804 | Time: 1417 ms [2022-02-22 14:20:19 main:574] : INFO : Epoch 2034 | loss: 0.0311666 | val_loss: 0.0311797 | Time: 1433.6 ms [2022-02-22 14:20:21 main:574] : INFO : Epoch 2035 | loss: 0.0311624 | val_loss: 0.0311803 | Time: 1422.83 ms [2022-02-22 14:20:22 main:574] : INFO : Epoch 2036 | loss: 0.0311648 | val_loss: 0.0311776 | Time: 1421.75 ms [2022-02-22 14:20:24 main:574] : INFO : Epoch 2037 | loss: 0.0311667 | val_loss: 0.0311793 | Time: 1431.84 ms [2022-02-22 14:20:25 main:574] : INFO : Epoch 2038 | loss: 0.031163 | val_loss: 0.0311776 | Time: 1419.56 ms [2022-02-22 14:20:26 main:574] : INFO : Epoch 2039 | loss: 0.0311633 | val_loss: 0.0311748 | Time: 1419.96 ms [2022-02-22 14:20:28 main:574] : INFO : Epoch 2040 | loss: 0.0311631 | val_loss: 0.0311769 | Time: 1422.6 ms [2022-02-22 14:20:29 main:574] : INFO : Epoch 2041 | loss: 0.0311629 | val_loss: 0.0311782 | Time: 1418.05 ms [2022-02-22 14:20:31 main:574] : INFO : Epoch 2042 | loss: 0.0311642 | val_loss: 0.0311771 | Time: 1428.64 ms [2022-02-22 14:20:32 main:574] : INFO : Epoch 2043 | loss: 0.0311636 | val_loss: 0.0311777 | Time: 1428.01 ms [2022-02-22 14:20:34 main:574] : INFO : Epoch 2044 | loss: 0.0311639 | val_loss: 0.0311804 | Time: 1432.56 ms [2022-02-22 14:20:35 main:574] : INFO : Epoch 2045 | loss: 0.0311654 | val_loss: 0.0311813 | Time: 1431.5 ms [2022-02-22 14:20:36 main:574] : INFO : Epoch 2046 | loss: 0.0311652 | val_loss: 0.0311773 | Time: 1416.11 ms [2022-02-22 14:20:38 main:574] : INFO : Epoch 2047 | loss: 0.0311642 | val_loss: 0.0311809 | Time: 1423.84 ms [2022-02-22 14:20:39 main:574] : INFO : Epoch 2048 | loss: 0.0311695 | val_loss: 0.0311821 | Time: 1439.72 ms [2022-02-22 14:20:39 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311821 [2022-02-22 14:20:39 main:603] : INFO : Saving end state to config to file [2022-02-22 14:20:39 main:608] : INFO : Success, exiting.. 14:20:39 (21452): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)