Task 10947512
| Name | ParityModified-1639963433-28551-2-0_0 |
| Workunit | 8241812 |
| Created | 22 Jan 2022, 0:39:44 UTC |
| Sent | 6 Feb 2022, 21:05:30 UTC |
| Report deadline | 14 Feb 2022, 21:05:30 UTC |
| Received | 9 Feb 2022, 6:37:04 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 17116 |
| Run time | 49 min 49 sec |
| CPU time | 48 min 55 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 9,205.27 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.63 GB |
| Peak swap size | 3.66 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> :28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-08 20:20:51 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-08 20:20:51 main:494] : INFO : Creating Model [2022-02-08 20:20:51 main:507] : INFO : Preparing config file [2022-02-08 20:20:51 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-08 20:20:51 main:512] : INFO : Loading config [2022-02-08 20:20:51 main:514] : INFO : Loading state [2022-02-08 20:20:52 main:559] : INFO : Loading DataLoader into Memory [2022-02-08 20:20:52 main:562] : INFO : Starting Training [2022-02-08 20:20:53 main:574] : INFO : Epoch 1774 | loss: 0.0319968 | val_loss: 0.031389 | Time: 1653.9 ms [2022-02-08 20:20:55 main:574] : INFO : Epoch 1775 | loss: 0.0312668 | val_loss: 0.0312337 | Time: 1457.19 ms [2022-02-08 20:20:56 main:574] : INFO : Epoch 1776 | loss: 0.0312129 | val_loss: 0.0311952 | Time: 1467.9 ms [2022-02-08 20:20:58 main:574] : INFO : Epoch 1777 | loss: 0.0311926 | val_loss: 0.0311842 | Time: 1475.01 ms [2022-02-08 20:20:59 main:574] : INFO : Epoch 1778 | loss: 0.031182 | val_loss: 0.0311774 | Time: 1478.56 ms [2022-02-08 20:21:01 main:574] : INFO : Epoch 1779 | loss: 0.0311779 | val_loss: 0.031178 | Time: 1481.38 ms [2022-02-08 20:21:02 main:574] : INFO : Epoch 1780 | loss: 0.0311756 | val_loss: 0.0311717 | Time: 1486.12 ms [2022-02-08 20:21:14 main:574] : INFO : Epoch 1781 | loss: 0.0311702 | val_loss: 0.0311668 | Time: 11565.9 ms [2022-02-08 20:21:15 main:574] : INFO : Epoch 1782 | loss: 0.0311665 | val_loss: 0.0311644 | Time: 1471.77 ms [2022-02-08 20:21:17 main:574] : INFO : Epoch 1783 | loss: 0.0311656 | val_loss: 0.0311634 | Time: 1478.53 ms [2022-02-08 20:21:18 main:574] : INFO : Epoch 1784 | loss: 0.0311616 | val_loss: 0.0311633 | Time: 1475.81 ms [2022-02-08 20:21:20 main:574] : INFO : Epoch 1785 | loss: 0.0311614 | val_loss: 0.0311615 | Time: 1478.93 ms [2022-02-08 20:21:21 main:574] : INFO : Epoch 1786 | loss: 0.0311625 | val_loss: 0.0311627 | Time: 1477.85 ms [2022-02-08 20:21:23 main:574] : INFO : Epoch 1787 | loss: 0.0311615 | val_loss: 0.0311679 | Time: 1483.98 ms [2022-02-08 20:21:34 main:574] : INFO : Epoch 1788 | loss: 0.0311637 | val_loss: 0.0311655 | Time: 11589 ms [2022-02-08 20:21:36 main:574] : INFO : Epoch 1789 | loss: 0.0311622 | val_loss: 0.0311615 | Time: 1482.76 ms [2022-02-08 20:21:37 main:574] : INFO : Epoch 1790 | loss: 0.0311591 | val_loss: 0.0311621 | Time: 1482.68 ms [2022-02-08 20:21:39 main:574] : INFO : Epoch 1791 | loss: 0.0311561 | val_loss: 0.0311604 | Time: 1480.23 ms [2022-02-08 20:21:40 main:574] : INFO : Epoch 1792 | loss: 0.0311564 | val_loss: 0.031161 | Time: 1481.47 ms [2022-02-08 20:21:42 main:574] : INFO : Epoch 1793 | loss: 0.0311554 | val_loss: 0.0311608 | Time: 1481.25 ms [2022-02-08 20:21:43 main:574] : INFO : Epoch 1794 | loss: 0.0311547 | val_loss: 0.0311594 | Time: 1488.37 ms [2022-02-08 20:21:45 main:574] : INFO : Epoch 1795 | loss: 0.0311538 | val_loss: 0.0311584 | Time: 1487.76 ms [2022-02-08 20:21:46 main:574] : INFO : Epoch 1796 | loss: 0.0311528 | val_loss: 0.0311584 | Time: 1483.35 ms [2022-02-08 20:21:48 main:574] : INFO : Epoch 1797 | loss: 0.0311522 | val_loss: 0.0311545 | Time: 1480.63 ms [2022-02-08 20:21:49 main:574] : INFO : Epoch 1798 | loss: 0.03115 | val_loss: 0.0311538 | Time: 1480.39 ms [2022-02-08 20:21:51 main:574] : INFO : Epoch 1799 | loss: 0.0311499 | val_loss: 0.0311583 | Time: 1535.51 ms [2022-02-08 20:21:52 main:574] : INFO : Epoch 1800 | loss: 0.0311504 | val_loss: 0.0311549 | Time: 1507.73 ms [2022-02-08 20:21:54 main:574] : INFO : Epoch 1801 | loss: 0.0311503 | val_loss: 0.0311577 | Time: 1513.32 ms [2022-02-08 20:21:55 main:574] : INFO : Epoch 1802 | loss: 0.0311482 | val_loss: 0.0311571 | Time: 1497.82 ms [2022-02-08 20:21:57 main:574] : INFO : Epoch 1803 | loss: 0.0311486 | val_loss: 0.0311588 | Time: 1512.05 ms [2022-02-08 20:21:58 main:574] : INFO : Epoch 1804 | loss: 0.0311472 | val_loss: 0.0311539 | Time: 1485.46 ms [2022-02-08 20:22:00 main:574] : INFO : Epoch 1805 | loss: 0.0311492 | val_loss: 0.0311608 | Time: 1486.38 ms [2022-02-08 20:22:01 main:574] : INFO : Epoch 1806 | loss: 0.0311536 | val_loss: 0.0311639 | Time: 1487.69 ms [2022-02-08 20:22:03 main:574] : INFO : Epoch 1807 | loss: 0.031152 | val_loss: 0.0311618 | Time: 1481.41 ms [2022-02-08 20:22:14 main:574] : INFO : Epoch 1808 | loss: 0.0311518 | val_loss: 0.031157 | Time: 11644.8 ms [2022-02-08 20:22:16 main:574] : INFO : Epoch 1809 | loss: 0.0311511 | val_loss: 0.0311585 | Time: 1473.52 ms [2022-02-08 20:22:17 main:574] : INFO : Epoch 1810 | loss: 0.0311481 | val_loss: 0.0311596 | Time: 1476.12 ms [2022-02-08 20:22:19 main:574] : INFO : Epoch 1811 | loss: 0.0311473 | val_loss: 0.0311538 | Time: 1470.33 ms [2022-02-08 20:22:20 main:574] : INFO : Epoch 1812 | loss: 0.031146 | val_loss: 0.0311538 | Time: 1472.71 ms [2022-02-08 20:22:22 main:574] : INFO : Epoch 1813 | loss: 0.0311453 | val_loss: 0.0311545 | Time: 1465.74 ms [2022-02-08 20:22:23 main:574] : INFO : Epoch 1814 | loss: 0.0311442 | val_loss: 0.0311543 | Time: 1475.05 ms [2022-02-08 20:22:25 main:574] : INFO : Epoch 1815 | loss: 0.0311454 | val_loss: 0.0311599 | Time: 1474.05 ms [2022-02-08 20:22:26 main:574] : INFO : Epoch 1816 | loss: 0.0311445 | val_loss: 0.0311529 | Time: 1483.47 ms [2022-02-08 20:22:28 main:574] : INFO : Epoch 1817 | loss: 0.0311421 | val_loss: 0.0311554 | Time: 1487.78 ms [2022-02-08 20:22:29 main:574] : INFO : Epoch 1818 | loss: 0.0311442 | val_loss: 0.0311537 | Time: 1481.41 ms [2022-02-08 20:22:31 main:574] : INFO : Epoch 1819 | loss: 0.0311415 | val_loss: 0.0311535 | Time: 1475.05 ms [2022-02-08 20:22:32 main:574] : INFO : Epoch 1820 | loss: 0.0311438 | val_loss: 0.0311539 | Time: 1477.31 ms [2022-02-08 20:22:33 main:574] : INFO : Epoch 1821 | loss: 0.0311439 | val_loss: 0.0311575 | Time: 1473.63 ms [2022-02-08 20:22:35 main:574] : INFO : Epoch 1822 | loss: 0.031145 | val_loss: 0.0311611 | Time: 1473.76 ms [2022-02-08 20:22:36 main:574] : INFO : Epoch 1823 | loss: 0.0311422 | val_loss: 0.0311577 | Time: 1473.16 ms [2022-02-08 20:22:38 main:574] : INFO : Epoch 1824 | loss: 0.0311419 | val_loss: 0.0311564 | Time: 1472.16 ms [2022-02-08 20:22:39 main:574] : INFO : Epoch 1825 | loss: 0.0311434 | val_loss: 0.0311556 | Time: 1490.19 ms [2022-02-08 20:22:41 main:574] : INFO : Epoch 1826 | loss: 0.0311436 | val_loss: 0.0311496 | Time: 1494.65 ms [2022-02-08 20:22:42 main:574] : INFO : Epoch 1827 | loss: 0.0311407 | val_loss: 0.0311525 | Time: 1539.67 ms [2022-02-08 20:22:44 main:574] : INFO : Epoch 1828 | loss: 0.0311403 | val_loss: 0.0311506 | Time: 1475.64 ms [2022-02-08 20:22:45 main:574] : INFO : Epoch 1829 | loss: 0.0311413 | val_loss: 0.0311544 | Time: 1474.2 ms [2022-02-08 20:22:47 main:574] : INFO : Epoch 1830 | loss: 0.0311411 | val_loss: 0.0311525 | Time: 1477.07 ms [2022-02-08 20:22:48 main:574] : INFO : Epoch 1831 | loss: 0.0311411 | val_loss: 0.0311545 | Time: 1475.74 ms [2022-02-08 20:22:50 main:574] : INFO : Epoch 1832 | loss: 0.0311398 | val_loss: 0.0311539 | Time: 1458.85 ms [2022-02-08 20:22:51 main:574] : INFO : Epoch 1833 | loss: 0.0311406 | val_loss: 0.0311548 | Time: 1493.13 ms [2022-02-08 20:22:53 main:574] : INFO : Epoch 1834 | loss: 0.031141 | val_loss: 0.0311547 | Time: 1548.19 ms [2022-02-08 20:22:54 main:574] : INFO : Epoch 1835 | loss: 0.03114 | val_loss: 0.0311548 | Time: 1486.75 ms [2022-02-08 20:22:56 main:574] : INFO : Epoch 1836 | loss: 0.0311391 | val_loss: 0.031158 | Time: 1491.96 ms [2022-02-08 20:22:57 main:574] : INFO : Epoch 1837 | loss: 0.0311414 | val_loss: 0.0311535 | Time: 1470.28 ms [2022-02-08 20:22:59 main:574] : INFO : Epoch 1838 | loss: 0.0311384 | val_loss: 0.0311522 | Time: 1475.98 ms [2022-02-08 20:23:00 main:574] : INFO : Epoch 1839 | loss: 0.0311393 | val_loss: 0.031158 | Time: 1464.9 ms [2022-02-08 20:23:02 main:574] : INFO : Epoch 1840 | loss: 0.0311388 | val_loss: 0.0311522 | Time: 1469.73 ms [2022-02-08 20:23:03 main:574] : INFO : Epoch 1841 | loss: 0.0311373 | val_loss: 0.031155 | Time: 1487.67 ms [2022-02-08 20:23:15 main:574] : INFO : Epoch 1842 | loss: 0.0311366 | val_loss: 0.031151 | Time: 11606.2 ms [2022-02-08 20:23:16 main:574] : INFO : Epoch 1843 | loss: 0.0311363 | val_loss: 0.0311557 | Time: 1468.77 ms [2022-02-08 20:23:18 main:574] : INFO : Epoch 1844 | loss: 0.0311399 | val_loss: 0.0311568 | Time: 1473.07 ms [2022-02-08 20:23:19 main:574] : INFO : Epoch 1845 | loss: 0.031139 | val_loss: 0.031154 | Time: 1473.52 ms [2022-02-08 20:23:21 main:574] : INFO : Epoch 1846 | loss: 0.0311385 | val_loss: 0.0311553 | Time: 1477.79 ms [2022-02-08 20:23:22 main:574] : INFO : Epoch 1847 | loss: 0.0311387 | val_loss: 0.0311631 | Time: 1484.63 ms [2022-02-08 20:23:24 main:574] : INFO : Epoch 1848 | loss: 0.0311406 | val_loss: 0.0311546 | Time: 1472.34 ms [2022-02-08 20:23:25 main:574] : INFO : Epoch 1849 | loss: 0.0311378 | val_loss: 0.0311554 | Time: 1475.89 ms [2022-02-08 20:23:27 main:574] : INFO : Epoch 1850 | loss: 0.0311384 | val_loss: 0.0311552 | Time: 1470.06 ms [2022-02-08 20:23:28 main:574] : INFO : Epoch 1851 | loss: 0.0311391 | val_loss: 0.0311617 | Time: 1469.26 ms [2022-02-08 20:23:30 main:574] : INFO : Epoch 1852 | loss: 0.0311402 | val_loss: 0.0311586 | Time: 1480.45 ms [2022-02-08 20:23:31 main:574] : INFO : Epoch 1853 | loss: 0.0311388 | val_loss: 0.0311535 | Time: 1471.46 ms [2022-02-08 20:23:33 main:574] : INFO : Epoch 1854 | loss: 0.031138 | val_loss: 0.031155 | Time: 1478.5 ms [2022-02-08 20:23:34 main:574] : INFO : Epoch 1855 | loss: 0.0311391 | val_loss: 0.0311516 | Time: 1468.89 ms [2022-02-08 20:23:35 main:574] : INFO : Epoch 1856 | loss: 0.0311404 | val_loss: 0.0311538 | Time: 1463.57 ms [2022-02-08 20:23:37 main:574] : INFO : Epoch 1857 | loss: 0.0311385 | val_loss: 0.0311577 | Time: 1476.54 ms [2022-02-08 20:23:38 main:574] : INFO : Epoch 1858 | loss: 0.0311373 | val_loss: 0.0311516 | Time: 1473.26 ms [2022-02-08 20:23:40 main:574] : INFO : Epoch 1859 | loss: 0.0311401 | val_loss: 0.03116 | Time: 1490.85 ms [2022-02-08 20:23:41 main:574] : INFO : Epoch 1860 | loss: 0.0311414 | val_loss: 0.0311585 | Time: 1467.85 ms [2022-02-08 20:23:43 main:574] : INFO : Epoch 1861 | loss: 0.0311383 | val_loss: 0.0311539 | Time: 1473.86 ms [2022-02-08 20:23:44 main:574] : INFO : Epoch 1862 | loss: 0.0311369 | val_loss: 0.031153 | Time: 1478.43 ms [2022-02-08 20:23:46 main:574] : INFO : Epoch 1863 | loss: 0.0311364 | val_loss: 0.0311558 | Time: 1466.86 ms [2022-02-08 20:23:47 main:574] : INFO : Epoch 1864 | loss: 0.0311397 | val_loss: 0.0311535 | Time: 1468.75 ms [2022-02-08 20:23:49 main:574] : INFO : Epoch 1865 | loss: 0.0311387 | val_loss: 0.0311631 | Time: 1484.37 ms [2022-02-08 20:23:50 main:574] : INFO : Epoch 1866 | loss: 0.0311387 | val_loss: 0.0311527 | Time: 1467.02 ms [2022-02-08 20:23:52 main:574] : INFO : Epoch 1867 | loss: 0.0311367 | val_loss: 0.0311524 | Time: 1474.64 ms [2022-02-08 20:23:53 main:574] : INFO : Epoch 1868 | loss: 0.0311366 | val_loss: 0.031152 | Time: 1468.73 ms [2022-02-08 20:23:55 main:574] : INFO : Epoch 1869 | loss: 0.0311397 | val_loss: 0.0311548 | Time: 1467.53 ms [2022-02-08 20:23:56 main:574] : INFO : Epoch 1870 | loss: 0.0311419 | val_loss: 0.0311506 | Time: 1503.06 ms [2022-02-08 20:23:58 main:574] : INFO : Epoch 1871 | loss: 0.0311414 | val_loss: 0.0311534 | Time: 1473.37 ms [2022-02-08 20:23:59 main:574] : INFO : Epoch 1872 | loss: 0.0311404 | val_loss: 0.0311547 | Time: 1479.09 ms [2022-02-08 20:24:01 main:574] : INFO : Epoch 1873 | loss: 0.0311375 | val_loss: 0.0311496 | Time: 1476.48 ms [2022-02-08 20:24:02 main:574] : INFO : Epoch 1874 | loss: 0.0311375 | val_loss: 0.031152 | Time: 1465.19 ms [2022-02-08 20:24:04 main:574] : INFO : Epoch 1875 | loss: 0.0311398 | val_loss: 0.0311516 | Time: 1479.78 ms [2022-02-08 20:24:15 main:574] : INFO : Epoch 1876 | loss: 0.0311368 | val_loss: 0.0311494 | Time: 11659.3 ms [2022-02-08 20:24:17 main:574] : INFO : Epoch 1877 | loss: 0.0311352 | val_loss: 0.0311532 | Time: 1470.65 ms [2022-02-08 20:24:18 main:574] : INFO : Epoch 1878 | loss: 0.0311359 | val_loss: 0.031157 | Time: 1482.13 ms [2022-02-08 20:24:20 main:574] : INFO : Epoch 1879 | loss: 0.0311349 | val_loss: 0.0311505 | Time: 1468.77 ms [2022-02-08 20:24:21 main:574] : INFO : Epoch 1880 | loss: 0.0311359 | val_loss: 0.0311479 | Time: 1467.78 ms [2022-02-08 20:24:23 main:574] : INFO : Epoch 1881 | loss: 0.0311369 | val_loss: 0.0311495 | Time: 1487.12 ms [2022-02-08 20:24:24 main:574] : INFO : Epoch 1882 | loss: 0.0311344 | val_loss: 0.03115 | Time: 1469.74 ms [2022-02-08 20:24:26 main:574] : INFO : Epoch 1883 | loss: 0.0311345 | val_loss: 0.0311499 | Time: 1464.8 ms [2022-02-08 20:24:27 main:574] : INFO : Epoch 1884 | loss: 0.0311352 | val_loss: 0.031148 | Time: 1473.39 ms [2022-02-08 20:24:28 main:574] : INFO : Epoch 1885 | loss: 0.0311366 | val_loss: 0.0311492 | Time: 1462.01 ms [2022-02-08 20:24:30 main:574] : INFO : Epoch 1886 | loss: 0.0311344 | val_loss: 0.0311467 | Time: 1474.21 ms [2022-02-08 20:24:31 main:574] : INFO : Epoch 1887 | loss: 0.0311339 | val_loss: 0.0311521 | Time: 1468.21 ms [2022-02-08 20:24:33 main:574] : INFO : Epoch 1888 | loss: 0.0311366 | val_loss: 0.0311513 | Time: 1471.21 ms [2022-02-08 20:24:34 main:574] : INFO : Epoch 1889 | loss: 0.0311339 | val_loss: 0.0311492 | Time: 1481.24 ms [2022-02-08 20:24:36 main:574] : INFO : Epoch 1890 | loss: 0.0311348 | val_loss: 0.0311501 | Time: 1467.46 ms [2022-02-08 20:24:37 main:574] : INFO : Epoch 1891 | loss: 0.0311321 | val_loss: 0.0311513 | Time: 1478 ms [2022-02-08 20:24:39 main:574] : INFO : Epoch 1892 | loss: 0.0311311 | val_loss: 0.0311522 | Time: 1473.22 ms [2022-02-08 20:24:40 main:574] : INFO : Epoch 1893 | loss: 0.0311316 | val_loss: 0.0311505 | Time: 1460.27 ms [2022-02-08 20:24:42 main:574] : INFO : Epoch 1894 | loss: 0.0311313 | val_loss: 0.0311522 | Time: 1523.12 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-09 00:21:13 main:435] : INFO : Set logging level to 1 [2022-02-09 00:21:13 main:441] : INFO : Running in BOINC Client mode [2022-02-09 00:21:13 main:444] : INFO : Resolving all filenames [2022-02-09 00:21:13 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-09 00:21:13 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-09 00:21:13 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-09 00:21:13 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-09 00:21:13 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-09 00:21:13 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-09 00:21:13 main:474] : INFO : Configuration: [2022-02-09 00:21:14 main:475] : INFO : Model type: GRU [2022-02-09 00:21:14 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-09 00:21:14 main:477] : INFO : Max Epochs: 2048 [2022-02-09 00:21:14 main:478] : INFO : Batch Size: 128 [2022-02-09 00:21:14 main:479] : INFO : Learning Rate: 0.01 [2022-02-09 00:21:14 main:480] : INFO : Patience: 10 [2022-02-09 00:21:14 main:481] : INFO : Hidden Width: 12 [2022-02-09 00:21:14 main:482] : INFO : # Recurrent Layers: 4 [2022-02-09 00:21:14 main:483] : INFO : # Backend Layers: 4 [2022-02-09 00:21:14 main:484] : INFO : # Threads: 1 [2022-02-09 00:21:14 main:486] : INFO : Preparing Dataset [2022-02-09 00:21:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-09 00:21:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-09 00:21:15 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-09 00:21:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-09 00:21:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-09 00:21:15 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-09 00:21:15 main:494] : INFO : Creating Model [2022-02-09 00:21:15 main:507] : INFO : Preparing config file [2022-02-09 00:21:15 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-09 00:21:15 main:512] : INFO : Loading config [2022-02-09 00:21:15 main:514] : INFO : Loading state [2022-02-09 00:21:16 main:559] : INFO : Loading DataLoader into Memory [2022-02-09 00:21:16 main:562] : INFO : Starting Training [2022-02-09 00:21:18 main:574] : INFO : Epoch 1877 | loss: 0.0313975 | val_loss: 0.031217 | Time: 1744.13 ms [2022-02-09 00:21:19 main:574] : INFO : Epoch 1878 | loss: 0.0311835 | val_loss: 0.0311639 | Time: 1542.07 ms [2022-02-09 00:21:21 main:574] : INFO : Epoch 1879 | loss: 0.0311478 | val_loss: 0.031154 | Time: 1540.22 ms [2022-02-09 00:21:22 main:574] : INFO : Epoch 1880 | loss: 0.0311394 | val_loss: 0.0311501 | Time: 1580.1 ms [2022-02-09 00:21:24 main:574] : INFO : Epoch 1881 | loss: 0.0311386 | val_loss: 0.0311478 | Time: 1548.42 ms [2022-02-09 00:21:25 main:574] : INFO : Epoch 1882 | loss: 0.0311392 | val_loss: 0.0311491 | Time: 1555.6 ms [2022-02-09 00:21:27 main:574] : INFO : Epoch 1883 | loss: 0.0311393 | val_loss: 0.0311521 | Time: 1564.8 ms [2022-02-09 00:21:29 main:574] : INFO : Epoch 1884 | loss: 0.0311369 | val_loss: 0.0311534 | Time: 1567.12 ms [2022-02-09 00:21:30 main:574] : INFO : Epoch 1885 | loss: 0.0311369 | val_loss: 0.0311468 | Time: 1586.79 ms [2022-02-09 00:21:32 main:574] : INFO : Epoch 1886 | loss: 0.0311353 | val_loss: 0.0311486 | Time: 1565.31 ms [2022-02-09 00:21:33 main:574] : INFO : Epoch 1887 | loss: 0.0311355 | val_loss: 0.0311501 | Time: 1565.77 ms [2022-02-09 00:21:35 main:574] : INFO : Epoch 1888 | loss: 0.0311346 | val_loss: 0.0311484 | Time: 1571.38 ms [2022-02-09 00:21:36 main:574] : INFO : Epoch 1889 | loss: 0.0311343 | val_loss: 0.0311463 | Time: 1571.08 ms [2022-02-09 00:21:38 main:574] : INFO : Epoch 1890 | loss: 0.0311331 | val_loss: 0.031145 | Time: 1565.35 ms [2022-02-09 00:21:40 main:574] : INFO : Epoch 1891 | loss: 0.0311328 | val_loss: 0.0311442 | Time: 1570.59 ms [2022-02-09 00:21:41 main:574] : INFO : Epoch 1892 | loss: 0.0311335 | val_loss: 0.0311471 | Time: 1568.68 ms [2022-02-09 00:21:43 main:574] : INFO : Epoch 1893 | loss: 0.0311338 | val_loss: 0.031145 | Time: 1575.91 ms [2022-02-09 00:21:44 main:574] : INFO : Epoch 1894 | loss: 0.0311331 | val_loss: 0.0311458 | Time: 1593.35 ms [2022-02-09 00:21:46 main:574] : INFO : Epoch 1895 | loss: 0.0311368 | val_loss: 0.0311454 | Time: 1564.48 ms [2022-02-09 00:21:47 main:574] : INFO : Epoch 1896 | loss: 0.0311343 | val_loss: 0.0311458 | Time: 1566.25 ms [2022-02-09 00:21:49 main:574] : INFO : Epoch 1897 | loss: 0.0311361 | val_loss: 0.0311488 | Time: 1568.38 ms [2022-02-09 00:21:51 main:574] : INFO : Epoch 1898 | loss: 0.0311362 | val_loss: 0.0311481 | Time: 1568.22 ms [2022-02-09 00:21:52 main:574] : INFO : Epoch 1899 | loss: 0.0311335 | val_loss: 0.0311506 | Time: 1570 ms [2022-02-09 00:21:54 main:574] : INFO : Epoch 1900 | loss: 0.0311361 | val_loss: 0.0311517 | Time: 1583.05 ms [2022-02-09 00:21:55 main:574] : INFO : Epoch 1901 | loss: 0.0311371 | val_loss: 0.0311531 | Time: 1568.53 ms [2022-02-09 00:21:57 main:574] : INFO : Epoch 1902 | loss: 0.0311354 | val_loss: 0.031147 | Time: 1564.68 ms [2022-02-09 00:21:59 main:574] : INFO : Epoch 1903 | loss: 0.0311353 | val_loss: 0.0311514 | Time: 1574.41 ms [2022-02-09 00:22:00 main:574] : INFO : Epoch 1904 | loss: 0.0311344 | val_loss: 0.0311527 | Time: 1566.94 ms [2022-02-09 00:22:12 main:574] : INFO : Epoch 1905 | loss: 0.0311318 | val_loss: 0.0311481 | Time: 11757.2 ms [2022-02-09 00:22:13 main:574] : INFO : Epoch 1906 | loss: 0.0311335 | val_loss: 0.0311452 | Time: 1549.81 ms [2022-02-09 00:22:15 main:574] : INFO : Epoch 1907 | loss: 0.0311305 | val_loss: 0.031145 | Time: 1577.42 ms [2022-02-09 00:22:17 main:574] : INFO : Epoch 1908 | loss: 0.0311336 | val_loss: 0.0311481 | Time: 1579.33 ms [2022-02-09 00:22:18 main:574] : INFO : Epoch 1909 | loss: 0.0311349 | val_loss: 0.0311479 | Time: 1569.56 ms [2022-02-09 00:22:20 main:574] : INFO : Epoch 1910 | loss: 0.0311337 | val_loss: 0.0311502 | Time: 1569.24 ms [2022-02-09 00:22:21 main:574] : INFO : Epoch 1911 | loss: 0.0311371 | val_loss: 0.0311477 | Time: 1555.47 ms [2022-02-09 00:22:23 main:574] : INFO : Epoch 1912 | loss: 0.0311364 | val_loss: 0.0311481 | Time: 1593.91 ms [2022-02-09 00:22:24 main:574] : INFO : Epoch 1913 | loss: 0.0311334 | val_loss: 0.0311495 | Time: 1576.27 ms [2022-02-09 00:22:26 main:574] : INFO : Epoch 1914 | loss: 0.0311322 | val_loss: 0.0311492 | Time: 1563.02 ms [2022-02-09 00:22:28 main:574] : INFO : Epoch 1915 | loss: 0.0311331 | val_loss: 0.0311471 | Time: 1554.34 ms [2022-02-09 00:22:29 main:574] : INFO : Epoch 1916 | loss: 0.0311318 | val_loss: 0.0311444 | Time: 1545 ms [2022-02-09 00:22:31 main:574] : INFO : Epoch 1917 | loss: 0.0311306 | val_loss: 0.0311454 | Time: 1551.99 ms [2022-02-09 00:22:32 main:574] : INFO : Epoch 1918 | loss: 0.0311309 | val_loss: 0.0311488 | Time: 1567.19 ms [2022-02-09 00:22:34 main:574] : INFO : Epoch 1919 | loss: 0.0311314 | val_loss: 0.0311499 | Time: 1558.77 ms [2022-02-09 00:22:35 main:574] : INFO : Epoch 1920 | loss: 0.0311349 | val_loss: 0.0311488 | Time: 1606.83 ms [2022-02-09 00:22:37 main:574] : INFO : Epoch 1921 | loss: 0.0311347 | val_loss: 0.0311474 | Time: 1592.13 ms [2022-02-09 00:22:39 main:574] : INFO : Epoch 1922 | loss: 0.0311305 | val_loss: 0.0311479 | Time: 1569.77 ms [2022-02-09 00:22:40 main:574] : INFO : Epoch 1923 | loss: 0.031129 | val_loss: 0.0311469 | Time: 1609.81 ms [2022-02-09 00:22:42 main:574] : INFO : Epoch 1924 | loss: 0.0311317 | val_loss: 0.0311495 | Time: 1600.08 ms [2022-02-09 00:22:43 main:574] : INFO : Epoch 1925 | loss: 0.0311394 | val_loss: 0.0311585 | Time: 1547.26 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-09 00:25:51 main:435] : INFO : Set logging level to 1 [2022-02-09 00:25:51 main:441] : INFO : Running in BOINC Client mode [2022-02-09 00:25:51 main:444] : INFO : Resolving all filenames [2022-02-09 00:25:51 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-09 00:25:51 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-09 00:25:51 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-09 00:25:51 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-09 00:25:51 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-09 00:25:51 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-09 00:25:51 main:474] : INFO : Configuration: [2022-02-09 00:25:51 main:475] : INFO : Model type: GRU [2022-02-09 00:25:51 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-09 00:25:51 main:477] : INFO : Max Epochs: 2048 [2022-02-09 00:25:51 main:478] : INFO : Batch Size: 128 [2022-02-09 00:25:51 main:479] : INFO : Learning Rate: 0.01 [2022-02-09 00:25:51 main:480] : INFO : Patience: 10 [2022-02-09 00:25:51 main:481] : INFO : Hidden Width: 12 [2022-02-09 00:25:51 main:482] : INFO : # Recurrent Layers: 4 [2022-02-09 00:25:51 main:483] : INFO : # Backend Layers: 4 [2022-02-09 00:25:51 main:484] : INFO : # Threads: 1 [2022-02-09 00:25:51 main:486] : INFO : Preparing Dataset [2022-02-09 00:25:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-09 00:25:52 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-09 00:25:53 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-09 00:25:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-09 00:25:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-09 00:25:53 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-09 00:25:53 main:494] : INFO : Creating Model [2022-02-09 00:25:53 main:507] : INFO : Preparing config file [2022-02-09 00:25:53 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-09 00:25:53 main:512] : INFO : Loading config [2022-02-09 00:25:53 main:514] : INFO : Loading state [2022-02-09 00:25:54 main:559] : INFO : Loading DataLoader into Memory [2022-02-09 00:25:54 main:562] : INFO : Starting Training [2022-02-09 00:25:55 main:574] : INFO : Epoch 1877 | loss: 0.0314099 | val_loss: 0.0312239 | Time: 1747.5 ms [2022-02-09 00:25:57 main:574] : INFO : Epoch 1878 | loss: 0.0311908 | val_loss: 0.0311628 | Time: 1528.54 ms [2022-02-09 00:25:59 main:574] : INFO : Epoch 1879 | loss: 0.0311437 | val_loss: 0.0311536 | Time: 1526.48 ms [2022-02-09 00:26:00 main:574] : INFO : Epoch 1880 | loss: 0.0311387 | val_loss: 0.0311521 | Time: 1522.48 ms [2022-02-09 00:26:02 main:574] : INFO : Epoch 1881 | loss: 0.031135 | val_loss: 0.0311493 | Time: 1533.06 ms [2022-02-09 00:26:03 main:574] : INFO : Epoch 1882 | loss: 0.0311353 | val_loss: 0.0311487 | Time: 1552.27 ms [2022-02-09 00:26:15 main:574] : INFO : Epoch 1883 | loss: 0.031136 | val_loss: 0.0311557 | Time: 11837.5 ms [2022-02-09 00:26:16 main:574] : INFO : Epoch 1884 | loss: 0.0311374 | val_loss: 0.0311514 | Time: 1519.64 ms [2022-02-09 00:26:18 main:574] : INFO : Epoch 1885 | loss: 0.0311364 | val_loss: 0.0311508 | Time: 1546.14 ms [2022-02-09 00:26:20 main:574] : INFO : Epoch 1886 | loss: 0.0311358 | val_loss: 0.0311525 | Time: 1530.22 ms [2022-02-09 00:26:21 main:574] : INFO : Epoch 1887 | loss: 0.0311363 | val_loss: 0.031152 | Time: 1521.01 ms [2022-02-09 00:26:23 main:574] : INFO : Epoch 1888 | loss: 0.0311343 | val_loss: 0.0311511 | Time: 1523.28 ms [2022-02-09 00:26:24 main:574] : INFO : Epoch 1889 | loss: 0.031136 | val_loss: 0.0311523 | Time: 1525.53 ms [2022-02-09 00:26:26 main:574] : INFO : Epoch 1890 | loss: 0.0311331 | val_loss: 0.0311508 | Time: 1535.3 ms [2022-02-09 00:26:27 main:574] : INFO : Epoch 1891 | loss: 0.0311335 | val_loss: 0.0311546 | Time: 1535.43 ms [2022-02-09 00:26:29 main:574] : INFO : Epoch 1892 | loss: 0.0311369 | val_loss: 0.0311544 | Time: 1530.71 ms [2022-02-09 00:26:30 main:574] : INFO : Epoch 1893 | loss: 0.0311355 | val_loss: 0.0311512 | Time: 1535.45 ms [2022-02-09 00:26:32 main:574] : INFO : Epoch 1894 | loss: 0.031134 | val_loss: 0.0311504 | Time: 1527.92 ms [2022-02-09 00:26:33 main:574] : INFO : Epoch 1895 | loss: 0.0311354 | val_loss: 0.0311523 | Time: 1539.2 ms [2022-02-09 00:26:35 main:574] : INFO : Epoch 1896 | loss: 0.0311327 | val_loss: 0.0311502 | Time: 1535.57 ms [2022-02-09 00:26:36 main:574] : INFO : Epoch 1897 | loss: 0.0311321 | val_loss: 0.0311499 | Time: 1533.6 ms [2022-02-09 00:26:38 main:574] : INFO : Epoch 1898 | loss: 0.0311333 | val_loss: 0.0311526 | Time: 1525.11 ms [2022-02-09 00:26:40 main:574] : INFO : Epoch 1899 | loss: 0.0311314 | val_loss: 0.0311533 | Time: 1539.03 ms [2022-02-09 00:26:41 main:574] : INFO : Epoch 1900 | loss: 0.0311337 | val_loss: 0.0311535 | Time: 1530.02 ms [2022-02-09 00:26:43 main:574] : INFO : Epoch 1901 | loss: 0.0311318 | val_loss: 0.0311568 | Time: 1532.54 ms [2022-02-09 00:26:44 main:574] : INFO : Epoch 1902 | loss: 0.0311342 | val_loss: 0.0311495 | Time: 1541.4 ms [2022-02-09 00:26:46 main:574] : INFO : Epoch 1903 | loss: 0.0311334 | val_loss: 0.0311492 | Time: 1534.42 ms [2022-02-09 00:26:47 main:574] : INFO : Epoch 1904 | loss: 0.0311317 | val_loss: 0.0311508 | Time: 1543.68 ms [2022-02-09 00:26:49 main:574] : INFO : Epoch 1905 | loss: 0.0311322 | val_loss: 0.0311527 | Time: 1538.47 ms [2022-02-09 00:26:50 main:574] : INFO : Epoch 1906 | loss: 0.0311352 | val_loss: 0.031149 | Time: 1536.29 ms [2022-02-09 00:26:52 main:574] : INFO : Epoch 1907 | loss: 0.0311356 | val_loss: 0.0311519 | Time: 1555.07 ms [2022-02-09 00:26:53 main:574] : INFO : Epoch 1908 | loss: 0.0311332 | val_loss: 0.031152 | Time: 1542.26 ms [2022-02-09 00:26:55 main:574] : INFO : Epoch 1909 | loss: 0.0311337 | val_loss: 0.0311471 | Time: 1541.35 ms [2022-02-09 00:26:56 main:574] : INFO : Epoch 1910 | loss: 0.0311325 | val_loss: 0.0311468 | Time: 1559.31 ms [2022-02-09 00:26:58 main:574] : INFO : Epoch 1911 | loss: 0.0311304 | val_loss: 0.0311457 | Time: 1533.55 ms [2022-02-09 00:27:00 main:574] : INFO : Epoch 1912 | loss: 0.0311296 | val_loss: 0.0311476 | Time: 1537.89 ms [2022-02-09 00:27:01 main:574] : INFO : Epoch 1913 | loss: 0.0311287 | val_loss: 0.0311492 | Time: 1537.55 ms [2022-02-09 00:27:03 main:574] : INFO : Epoch 1914 | loss: 0.0311303 | val_loss: 0.0311475 | Time: 1540.86 ms [2022-02-09 00:27:14 main:574] : INFO : Epoch 1915 | loss: 0.0311301 | val_loss: 0.0311542 | Time: 11854.2 ms [2022-02-09 00:27:16 main:574] : INFO : Epoch 1916 | loss: 0.0311306 | val_loss: 0.0311475 | Time: 1528.17 ms [2022-02-09 00:27:18 main:574] : INFO : Epoch 1917 | loss: 0.0311323 | val_loss: 0.0311505 | Time: 1533.97 ms [2022-02-09 00:27:19 main:574] : INFO : Epoch 1918 | loss: 0.0311301 | val_loss: 0.0311494 | Time: 1533.25 ms [2022-02-09 00:27:21 main:574] : INFO : Epoch 1919 | loss: 0.0311314 | val_loss: 0.0311497 | Time: 1533.61 ms [2022-02-09 00:27:22 main:574] : INFO : Epoch 1920 | loss: 0.0311314 | val_loss: 0.0311515 | Time: 1546.26 ms [2022-02-09 00:27:24 main:574] : INFO : Epoch 1921 | loss: 0.0311333 | val_loss: 0.0311504 | Time: 1538.19 ms [2022-02-09 00:27:25 main:574] : INFO : Epoch 1922 | loss: 0.0311329 | val_loss: 0.0311478 | Time: 1535.74 ms [2022-02-09 00:27:27 main:574] : INFO : Epoch 1923 | loss: 0.0311311 | val_loss: 0.031147 | Time: 1540.77 ms [2022-02-09 00:27:28 main:574] : INFO : Epoch 1924 | loss: 0.0311306 | val_loss: 0.0311501 | Time: 1532.95 ms [2022-02-09 00:27:30 main:574] : INFO : Epoch 1925 | loss: 0.0311367 | val_loss: 0.031158 | Time: 1551.47 ms [2022-02-09 00:27:31 main:574] : INFO : Epoch 1926 | loss: 0.031139 | val_loss: 0.0311528 | Time: 1527.89 ms [2022-02-09 00:27:33 main:574] : INFO : Epoch 1927 | loss: 0.0311339 | val_loss: 0.0311444 | Time: 1530.4 ms [2022-02-09 00:27:35 main:574] : INFO : Epoch 1928 | loss: 0.0311335 | val_loss: 0.0311458 | Time: 1554.71 ms [2022-02-09 00:27:36 main:574] : INFO : Epoch 1929 | loss: 0.0311315 | val_loss: 0.0311462 | Time: 1534.93 ms [2022-02-09 00:27:38 main:574] : INFO : Epoch 1930 | loss: 0.0311296 | val_loss: 0.0311478 | Time: 1532.66 ms [2022-02-09 00:27:39 main:574] : INFO : Epoch 1931 | loss: 0.031128 | val_loss: 0.0311445 | Time: 1536.58 ms [2022-02-09 00:27:41 main:574] : INFO : Epoch 1932 | loss: 0.0311301 | val_loss: 0.0311471 | Time: 1542.39 ms [2022-02-09 00:27:42 main:574] : INFO : Epoch 1933 | loss: 0.0311286 | val_loss: 0.0311444 | Time: 1548.29 ms [2022-02-09 00:27:44 main:574] : INFO : Epoch 1934 | loss: 0.0311285 | val_loss: 0.0311506 | Time: 1524.57 ms [2022-02-09 00:27:45 main:574] : INFO : Epoch 1935 | loss: 0.0311325 | val_loss: 0.0311556 | Time: 1531.2 ms [2022-02-09 00:27:47 main:574] : INFO : Epoch 1936 | loss: 0.0311369 | val_loss: 0.0311571 | Time: 1544.03 ms [2022-02-09 00:27:48 main:574] : INFO : Epoch 1937 | loss: 0.0311347 | val_loss: 0.0311504 | Time: 1547.49 ms [2022-02-09 00:27:50 main:574] : INFO : Epoch 1938 | loss: 0.0311314 | val_loss: 0.0311513 | Time: 1534.63 ms [2022-02-09 00:27:51 main:574] : INFO : Epoch 1939 | loss: 0.0311304 | val_loss: 0.0311498 | Time: 1543.54 ms [2022-02-09 00:27:53 main:574] : INFO : Epoch 1940 | loss: 0.0311304 | val_loss: 0.0311489 | Time: 1520.12 ms [2022-02-09 00:27:55 main:574] : INFO : Epoch 1941 | loss: 0.0311294 | val_loss: 0.0311489 | Time: 1534.11 ms [2022-02-09 00:27:56 main:574] : INFO : Epoch 1942 | loss: 0.0311311 | val_loss: 0.0311501 | Time: 1565.89 ms [2022-02-09 00:27:58 main:574] : INFO : Epoch 1943 | loss: 0.0311323 | val_loss: 0.0311527 | Time: 1533.37 ms [2022-02-09 00:27:59 main:574] : INFO : Epoch 1944 | loss: 0.0311304 | val_loss: 0.0311506 | Time: 1526.34 ms [2022-02-09 00:28:01 main:574] : INFO : Epoch 1945 | loss: 0.0311285 | val_loss: 0.0311571 | Time: 1545.26 ms [2022-02-09 00:28:02 main:574] : INFO : Epoch 1946 | loss: 0.0311299 | val_loss: 0.0311524 | Time: 1528.72 ms [2022-02-09 00:28:04 main:574] : INFO : Epoch 1947 | loss: 0.0311295 | val_loss: 0.0311492 | Time: 1559.82 ms [2022-02-09 00:28:16 main:574] : INFO : Epoch 1948 | loss: 0.0311302 | val_loss: 0.0311482 | Time: 11738.6 ms [2022-02-09 00:28:17 main:574] : INFO : Epoch 1949 | loss: 0.0311285 | val_loss: 0.031148 | Time: 1526.35 ms [2022-02-09 00:28:19 main:574] : INFO : Epoch 1950 | loss: 0.0311285 | val_loss: 0.0311484 | Time: 1525.54 ms [2022-02-09 00:28:20 main:574] : INFO : Epoch 1951 | loss: 0.0311282 | val_loss: 0.0311487 | Time: 1539.86 ms [2022-02-09 00:28:22 main:574] : INFO : Epoch 1952 | loss: 0.0311304 | val_loss: 0.0311483 | Time: 1537.84 ms [2022-02-09 00:28:23 main:574] : INFO : Epoch 1953 | loss: 0.0311302 | val_loss: 0.0311502 | Time: 1529.41 ms [2022-02-09 00:28:25 main:574] : INFO : Epoch 1954 | loss: 0.0311275 | val_loss: 0.0311478 | Time: 1533.13 ms [2022-02-09 00:28:36 main:574] : INFO : Epoch 1955 | loss: 0.0311295 | val_loss: 0.0311545 | Time: 11649.9 ms [2022-02-09 00:28:38 main:574] : INFO : Epoch 1956 | loss: 0.0311283 | val_loss: 0.0311503 | Time: 1528.81 ms [2022-02-09 00:28:39 main:574] : INFO : Epoch 1957 | loss: 0.0311308 | val_loss: 0.0311501 | Time: 1533.77 ms [2022-02-09 00:28:41 main:574] : INFO : Epoch 1958 | loss: 0.031131 | val_loss: 0.0311506 | Time: 1526.2 ms [2022-02-09 00:28:43 main:574] : INFO : Epoch 1959 | loss: 0.0311283 | val_loss: 0.0311471 | Time: 1534.55 ms [2022-02-09 00:28:44 main:574] : INFO : Epoch 1960 | loss: 0.0311303 | val_loss: 0.0311515 | Time: 1534.4 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1080) [2022-02-09 00:31:46 main:435] : INFO : Set logging level to 1 [2022-02-09 00:31:46 main:441] : INFO : Running in BOINC Client mode [2022-02-09 00:31:46 main:444] : INFO : Resolving all filenames [2022-02-09 00:31:46 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-09 00:31:46 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-09 00:31:46 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-09 00:31:46 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-09 00:31:46 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-09 00:31:46 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-09 00:31:46 main:474] : INFO : Configuration: [2022-02-09 00:31:46 main:475] : INFO : Model type: GRU [2022-02-09 00:31:46 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-09 00:31:46 main:477] : INFO : Max Epochs: 2048 [2022-02-09 00:31:46 main:478] : INFO : Batch Size: 128 [2022-02-09 00:31:46 main:479] : INFO : Learning Rate: 0.01 [2022-02-09 00:31:46 main:480] : INFO : Patience: 10 [2022-02-09 00:31:46 main:481] : INFO : Hidden Width: 12 [2022-02-09 00:31:46 main:482] : INFO : # Recurrent Layers: 4 [2022-02-09 00:31:46 main:483] : INFO : # Backend Layers: 4 [2022-02-09 00:31:46 main:484] : INFO : # Threads: 1 [2022-02-09 00:31:46 main:486] : INFO : Preparing Dataset [2022-02-09 00:31:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-09 00:31:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-09 00:31:47 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-09 00:31:47 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-09 00:31:47 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-09 00:31:47 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-09 00:31:47 main:494] : INFO : Creating Model [2022-02-09 00:31:47 main:507] : INFO : Preparing config file [2022-02-09 00:31:47 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-09 00:31:47 main:512] : INFO : Loading config [2022-02-09 00:31:47 main:514] : INFO : Loading state [2022-02-09 00:31:48 main:559] : INFO : Loading DataLoader into Memory [2022-02-09 00:31:48 main:562] : INFO : Starting Training [2022-02-09 00:31:50 main:574] : INFO : Epoch 1877 | loss: 0.0314135 | val_loss: 0.0312398 | Time: 1664.77 ms [2022-02-09 00:31:51 main:574] : INFO : Epoch 1878 | loss: 0.0311974 | val_loss: 0.0311693 | Time: 1462.36 ms [2022-02-09 00:31:53 main:574] : INFO : Epoch 1879 | loss: 0.031147 | val_loss: 0.0311564 | Time: 1453.78 ms [2022-02-09 00:31:54 main:574] : INFO : Epoch 1880 | loss: 0.0311392 | val_loss: 0.0311543 | Time: 1470.7 ms [2022-02-09 00:31:55 main:574] : INFO : Epoch 1881 | loss: 0.0311385 | val_loss: 0.0311524 | Time: 1465.59 ms [2022-02-09 00:31:57 main:574] : INFO : Epoch 1882 | loss: 0.031137 | val_loss: 0.0311531 | Time: 1484.25 ms [2022-02-09 00:31:58 main:574] : INFO : Epoch 1883 | loss: 0.031136 | val_loss: 0.0311537 | Time: 1474.26 ms [2022-02-09 00:32:00 main:574] : INFO : Epoch 1884 | loss: 0.0311351 | val_loss: 0.0311516 | Time: 1474.3 ms [2022-02-09 00:32:01 main:574] : INFO : Epoch 1885 | loss: 0.0311358 | val_loss: 0.0311499 | Time: 1470.41 ms [2022-02-09 00:32:03 main:574] : INFO : Epoch 1886 | loss: 0.0311343 | val_loss: 0.0311495 | Time: 1493.47 ms [2022-02-09 00:32:04 main:574] : INFO : Epoch 1887 | loss: 0.031136 | val_loss: 0.0311504 | Time: 1471.39 ms [2022-02-09 00:32:06 main:574] : INFO : Epoch 1888 | loss: 0.0311357 | val_loss: 0.0311525 | Time: 1472.24 ms [2022-02-09 00:32:07 main:574] : INFO : Epoch 1889 | loss: 0.0311345 | val_loss: 0.0311503 | Time: 1466.93 ms [2022-02-09 00:32:09 main:574] : INFO : Epoch 1890 | loss: 0.0311346 | val_loss: 0.0311508 | Time: 1473.83 ms [2022-02-09 00:32:10 main:574] : INFO : Epoch 1891 | loss: 0.0311339 | val_loss: 0.0311503 | Time: 1466.2 ms [2022-02-09 00:32:12 main:574] : INFO : Epoch 1892 | loss: 0.0311328 | val_loss: 0.0311504 | Time: 1470.25 ms [2022-02-09 00:32:13 main:574] : INFO : Epoch 1893 | loss: 0.0311337 | val_loss: 0.0311528 | Time: 1481.37 ms [2022-02-09 00:32:15 main:574] : INFO : Epoch 1894 | loss: 0.0311337 | val_loss: 0.0311495 | Time: 1472.08 ms [2022-02-09 00:32:16 main:574] : INFO : Epoch 1895 | loss: 0.0311357 | val_loss: 0.0311531 | Time: 1479.69 ms [2022-02-09 00:32:18 main:574] : INFO : Epoch 1896 | loss: 0.0311398 | val_loss: 0.0311564 | Time: 1476.73 ms [2022-02-09 00:32:19 main:574] : INFO : Epoch 1897 | loss: 0.0311374 | val_loss: 0.0311542 | Time: 1489.61 ms [2022-02-09 00:32:21 main:574] : INFO : Epoch 1898 | loss: 0.0311363 | val_loss: 0.0311499 | Time: 1484.08 ms [2022-02-09 00:32:22 main:574] : INFO : Epoch 1899 | loss: 0.0311345 | val_loss: 0.0311503 | Time: 1477.82 ms [2022-02-09 00:32:24 main:574] : INFO : Epoch 1900 | loss: 0.0311346 | val_loss: 0.0311489 | Time: 1479.81 ms [2022-02-09 00:32:25 main:574] : INFO : Epoch 1901 | loss: 0.0311328 | val_loss: 0.0311486 | Time: 1486.49 ms [2022-02-09 00:32:27 main:574] : INFO : Epoch 1902 | loss: 0.0311323 | val_loss: 0.0311511 | Time: 1474.81 ms [2022-02-09 00:32:28 main:574] : INFO : Epoch 1903 | loss: 0.0311334 | val_loss: 0.0311541 | Time: 1503.64 ms [2022-02-09 00:32:30 main:574] : INFO : Epoch 1904 | loss: 0.0311301 | val_loss: 0.0311473 | Time: 1483.98 ms [2022-02-09 00:32:31 main:574] : INFO : Epoch 1905 | loss: 0.0311309 | val_loss: 0.0311483 | Time: 1475.14 ms [2022-02-09 00:32:32 main:574] : INFO : Epoch 1906 | loss: 0.0311326 | val_loss: 0.0311505 | Time: 1482.09 ms [2022-02-09 00:32:34 main:574] : INFO : Epoch 1907 | loss: 0.0311305 | val_loss: 0.0311542 | Time: 1475.83 ms [2022-02-09 00:32:35 main:574] : INFO : Epoch 1908 | loss: 0.0311311 | val_loss: 0.0311499 | Time: 1484.34 ms [2022-02-09 00:32:37 main:574] : INFO : Epoch 1909 | loss: 0.0311308 | val_loss: 0.0311501 | Time: 1490.83 ms [2022-02-09 00:32:38 main:574] : INFO : Epoch 1910 | loss: 0.0311288 | val_loss: 0.0311493 | Time: 1477.73 ms [2022-02-09 00:32:40 main:574] : INFO : Epoch 1911 | loss: 0.0311287 | val_loss: 0.031149 | Time: 1476.26 ms [2022-02-09 00:32:41 main:574] : INFO : Epoch 1912 | loss: 0.031128 | val_loss: 0.0311493 | Time: 1488.14 ms [2022-02-09 00:32:43 main:574] : INFO : Epoch 1913 | loss: 0.0311284 | val_loss: 0.0311498 | Time: 1488.27 ms [2022-02-09 00:32:44 main:574] : INFO : Epoch 1914 | loss: 0.0311296 | val_loss: 0.0311491 | Time: 1483.34 ms [2022-02-09 00:32:46 main:574] : INFO : Epoch 1915 | loss: 0.0311293 | val_loss: 0.0311531 | Time: 1472.71 ms [2022-02-09 00:32:47 main:574] : INFO : Epoch 1916 | loss: 0.0311281 | val_loss: 0.0311509 | Time: 1496.64 ms [2022-02-09 00:32:49 main:574] : INFO : Epoch 1917 | loss: 0.031129 | val_loss: 0.0311538 | Time: 1478.49 ms [2022-02-09 00:32:50 main:574] : INFO : Epoch 1918 | loss: 0.0311311 | val_loss: 0.0311578 | Time: 1478.66 ms [2022-02-09 00:32:52 main:574] : INFO : Epoch 1919 | loss: 0.0311363 | val_loss: 0.0311548 | Time: 1486.06 ms [2022-02-09 00:32:53 main:574] : INFO : Epoch 1920 | loss: 0.031134 | val_loss: 0.0311506 | Time: 1477.82 ms [2022-02-09 00:32:55 main:574] : INFO : Epoch 1921 | loss: 0.031135 | val_loss: 0.0311523 | Time: 1472.63 ms [2022-02-09 00:32:56 main:574] : INFO : Epoch 1922 | loss: 0.0311372 | val_loss: 0.0311515 | Time: 1490.9 ms [2022-02-09 00:33:08 main:574] : INFO : Epoch 1923 | loss: 0.0311309 | val_loss: 0.0311555 | Time: 11751.6 ms [2022-02-09 00:33:09 main:574] : INFO : Epoch 1924 | loss: 0.0311316 | val_loss: 0.0311563 | Time: 1471.35 ms [2022-02-09 00:33:11 main:574] : INFO : Epoch 1925 | loss: 0.0311327 | val_loss: 0.0311513 | Time: 1474.18 ms [2022-02-09 00:33:12 main:574] : INFO : Epoch 1926 | loss: 0.0311297 | val_loss: 0.0311486 | Time: 1471.66 ms [2022-02-09 00:33:14 main:574] : INFO : Epoch 1927 | loss: 0.0311274 | val_loss: 0.0311498 | Time: 1485.82 ms [2022-02-09 00:33:15 main:574] : INFO : Epoch 1928 | loss: 0.0311285 | val_loss: 0.0311505 | Time: 1478.33 ms [2022-02-09 00:33:17 main:574] : INFO : Epoch 1929 | loss: 0.0311285 | val_loss: 0.0311481 | Time: 1469.62 ms [2022-02-09 00:33:18 main:574] : INFO : Epoch 1930 | loss: 0.0311297 | val_loss: 0.0311487 | Time: 1473.26 ms [2022-02-09 00:33:20 main:574] : INFO : Epoch 1931 | loss: 0.0311309 | val_loss: 0.0311482 | Time: 1473.87 ms [2022-02-09 00:33:21 main:574] : INFO : Epoch 1932 | loss: 0.0311332 | val_loss: 0.0311516 | Time: 1486.33 ms [2022-02-09 00:33:23 main:574] : INFO : Epoch 1933 | loss: 0.031134 | val_loss: 0.0311601 | Time: 1471.69 ms [2022-02-09 00:33:24 main:574] : INFO : Epoch 1934 | loss: 0.0311355 | val_loss: 0.0311479 | Time: 1470.88 ms [2022-02-09 00:33:26 main:574] : INFO : Epoch 1935 | loss: 0.0311325 | val_loss: 0.0311513 | Time: 1476.2 ms [2022-02-09 00:33:27 main:574] : INFO : Epoch 1936 | loss: 0.0311332 | val_loss: 0.0311492 | Time: 1478.57 ms [2022-02-09 00:33:29 main:574] : INFO : Epoch 1937 | loss: 0.0311315 | val_loss: 0.0311544 | Time: 1480.84 ms [2022-02-09 00:33:30 main:574] : INFO : Epoch 1938 | loss: 0.0311327 | val_loss: 0.0311588 | Time: 1481.88 ms [2022-02-09 00:33:32 main:574] : INFO : Epoch 1939 | loss: 0.0311363 | val_loss: 0.0311507 | Time: 1469.49 ms [2022-02-09 00:33:33 main:574] : INFO : Epoch 1940 | loss: 0.0311309 | val_loss: 0.0311561 | Time: 1474.35 ms [2022-02-09 00:33:35 main:574] : INFO : Epoch 1941 | loss: 0.0311326 | val_loss: 0.0311519 | Time: 1475.53 ms [2022-02-09 00:33:36 main:574] : INFO : Epoch 1942 | loss: 0.0311313 | val_loss: 0.0311523 | Time: 1470.22 ms [2022-02-09 00:33:38 main:574] : INFO : Epoch 1943 | loss: 0.031129 | val_loss: 0.0311473 | Time: 1478.52 ms [2022-02-09 00:33:39 main:574] : INFO : Epoch 1944 | loss: 0.031128 | val_loss: 0.0311468 | Time: 1479.78 ms [2022-02-09 00:33:41 main:574] : INFO : Epoch 1945 | loss: 0.0311285 | val_loss: 0.0311481 | Time: 1469.25 ms [2022-02-09 00:33:42 main:574] : INFO : Epoch 1946 | loss: 0.0311273 | val_loss: 0.0311541 | Time: 1483.59 ms [2022-02-09 00:33:43 main:574] : INFO : Epoch 1947 | loss: 0.0311296 | val_loss: 0.0311542 | Time: 1469.93 ms [2022-02-09 00:33:45 main:574] : INFO : Epoch 1948 | loss: 0.0311289 | val_loss: 0.0311526 | Time: 1475.49 ms [2022-02-09 00:33:46 main:574] : INFO : Epoch 1949 | loss: 0.0311267 | val_loss: 0.0311467 | Time: 1469.19 ms [2022-02-09 00:33:48 main:574] : INFO : Epoch 1950 | loss: 0.031126 | val_loss: 0.0311478 | Time: 1480.32 ms [2022-02-09 00:33:49 main:574] : INFO : Epoch 1951 | loss: 0.0311286 | val_loss: 0.0311498 | Time: 1459.75 ms [2022-02-09 00:33:51 main:574] : INFO : Epoch 1952 | loss: 0.0311279 | val_loss: 0.0311503 | Time: 1454.48 ms [2022-02-09 00:33:52 main:574] : INFO : Epoch 1953 | loss: 0.0311284 | val_loss: 0.0311447 | Time: 1461.75 ms [2022-02-09 00:33:54 main:574] : INFO : Epoch 1954 | loss: 0.0311275 | val_loss: 0.0311514 | Time: 1464.01 ms [2022-02-09 00:33:55 main:574] : INFO : Epoch 1955 | loss: 0.0311276 | val_loss: 0.0311496 | Time: 1454.11 ms [2022-02-09 00:33:57 main:574] : INFO : Epoch 1956 | loss: 0.0311282 | val_loss: 0.0311499 | Time: 1465.67 ms [2022-02-09 00:33:58 main:574] : INFO : Epoch 1957 | loss: 0.0311262 | val_loss: 0.0311478 | Time: 1454.59 ms [2022-02-09 00:34:00 main:574] : INFO : Epoch 1958 | loss: 0.0311266 | val_loss: 0.0311463 | Time: 1491.93 ms [2022-02-09 00:34:01 main:574] : INFO : Epoch 1959 | loss: 0.0311289 | val_loss: 0.0311521 | Time: 1467.48 ms [2022-02-09 00:34:03 main:574] : INFO : Epoch 1960 | loss: 0.0311312 | val_loss: 0.0311542 | Time: 1449.48 ms [2022-02-09 00:34:04 main:574] : INFO : Epoch 1961 | loss: 0.0311299 | val_loss: 0.0311455 | Time: 1457 ms [2022-02-09 00:34:05 main:574] : INFO : Epoch 1962 | loss: 0.0311268 | val_loss: 0.031152 | Time: 1461.12 ms [2022-02-09 00:34:07 main:574] : INFO : Epoch 1963 | loss: 0.0311296 | val_loss: 0.0311484 | Time: 1460.33 ms [2022-02-09 00:34:08 main:574] : INFO : Epoch 1964 | loss: 0.0311307 | val_loss: 0.0311453 | Time: 1469.56 ms [2022-02-09 00:34:10 main:574] : INFO : Epoch 1965 | loss: 0.0311269 | val_loss: 0.0311494 | Time: 1449.77 ms [2022-02-09 00:34:11 main:574] : INFO : Epoch 1966 | loss: 0.0311305 | val_loss: 0.0311482 | Time: 1450.4 ms [2022-02-09 00:34:13 main:574] : INFO : Epoch 1967 | loss: 0.0311317 | val_loss: 0.0311465 | Time: 1461.24 ms [2022-02-09 00:34:14 main:574] : INFO : Epoch 1968 | loss: 0.0311327 | val_loss: 0.0311491 | Time: 1451.06 ms [2022-02-09 00:34:16 main:574] : INFO : Epoch 1969 | loss: 0.0311321 | val_loss: 0.0311538 | Time: 1471.61 ms [2022-02-09 00:34:17 main:574] : INFO : Epoch 1970 | loss: 0.0311361 | val_loss: 0.0311477 | Time: 1446.72 ms [2022-02-09 00:34:19 main:574] : INFO : Epoch 1971 | loss: 0.0311335 | val_loss: 0.0311499 | Time: 1450.04 ms [2022-02-09 00:34:20 main:574] : INFO : Epoch 1972 | loss: 0.031133 | val_loss: 0.0311506 | Time: 1469.68 ms [2022-02-09 00:34:22 main:574] : INFO : Epoch 1973 | loss: 0.031131 | val_loss: 0.0311525 | Time: 1445.93 ms [2022-02-09 00:34:23 main:574] : INFO : Epoch 1974 | loss: 0.03113 | val_loss: 0.031146 | Time: 1459.17 ms [2022-02-09 00:34:24 main:574] : INFO : Epoch 1975 | loss: 0.0311292 | val_loss: 0.0311459 | Time: 1444.9 ms [2022-02-09 00:34:26 main:574] : INFO : Epoch 1976 | loss: 0.0311262 | val_loss: 0.0311458 | Time: 1451.6 ms [2022-02-09 00:34:27 main:574] : INFO : Epoch 1977 | loss: 0.0311254 | val_loss: 0.0311485 | Time: 1457.34 ms [2022-02-09 00:34:29 main:574] : INFO : Epoch 1978 | loss: 0.0311258 | val_loss: 0.0311494 | Time: 1460.99 ms [2022-02-09 00:34:30 main:574] : INFO : Epoch 1979 | loss: 0.0311275 | val_loss: 0.0311505 | Time: 1485.16 ms [2022-02-09 00:34:32 main:574] : INFO : Epoch 1980 | loss: 0.0311268 | val_loss: 0.0311504 | Time: 1452.36 ms [2022-02-09 00:34:33 main:574] : INFO : Epoch 1981 | loss: 0.0311254 | val_loss: 0.0311463 | Time: 1446.54 ms [2022-02-09 00:34:35 main:574] : INFO : Epoch 1982 | loss: 0.0311245 | val_loss: 0.0311507 | Time: 1457.21 ms [2022-02-09 00:34:36 main:574] : INFO : Epoch 1983 | loss: 0.031125 | val_loss: 0.0311578 | Time: 1450.7 ms [2022-02-09 00:34:38 main:574] : INFO : Epoch 1984 | loss: 0.0311276 | val_loss: 0.0311528 | Time: 1455.29 ms [2022-02-09 00:34:39 main:574] : INFO : Epoch 1985 | loss: 0.0311293 | val_loss: 0.0311534 | Time: 1456.98 ms [2022-02-09 00:34:40 main:574] : INFO : Epoch 1986 | loss: 0.031126 | val_loss: 0.0311465 | Time: 1448.7 ms [2022-02-09 00:34:42 main:574] : INFO : Epoch 1987 | loss: 0.0311262 | val_loss: 0.0311455 | Time: 1463.49 ms [2022-02-09 00:34:43 main:574] : INFO : Epoch 1988 | loss: 0.0311273 | val_loss: 0.0311511 | Time: 1447.76 ms [2022-02-09 00:34:45 main:574] : INFO : Epoch 1989 | loss: 0.0311264 | val_loss: 0.0311476 | Time: 1453.71 ms [2022-02-09 00:34:46 main:574] : INFO : Epoch 1990 | loss: 0.0311258 | val_loss: 0.0311457 | Time: 1479.22 ms [2022-02-09 00:34:48 main:574] : INFO : Epoch 1991 | loss: 0.0311267 | val_loss: 0.0311497 | Time: 1442.18 ms [2022-02-09 00:34:49 main:574] : INFO : Epoch 1992 | loss: 0.0311255 | val_loss: 0.0311457 | Time: 1458.39 ms [2022-02-09 00:34:51 main:574] : INFO : Epoch 1993 | loss: 0.0311272 | val_loss: 0.0311529 | Time: 1454.23 ms [2022-02-09 00:34:52 main:574] : INFO : Epoch 1994 | loss: 0.0311271 | val_loss: 0.0311492 | Time: 1450.55 ms [2022-02-09 00:34:54 main:574] : INFO : Epoch 1995 | loss: 0.0311263 | val_loss: 0.031153 | Time: 1457.18 ms [2022-02-09 00:34:55 main:574] : INFO : Epoch 1996 | loss: 0.0311277 | val_loss: 0.0311466 | Time: 1445.33 ms [2022-02-09 00:34:56 main:574] : INFO : Epoch 1997 | loss: 0.0311243 | val_loss: 0.0311499 | Time: 1465.77 ms [2022-02-09 00:34:58 main:574] : INFO : Epoch 1998 | loss: 0.0311273 | val_loss: 0.0311492 | Time: 1451.86 ms [2022-02-09 00:35:10 main:574] : INFO : Epoch 1999 | loss: 0.0311266 | val_loss: 0.0311472 | Time: 11683.7 ms [2022-02-09 00:35:11 main:574] : INFO : Epoch 2000 | loss: 0.0311283 | val_loss: 0.0311453 | Time: 1452.67 ms [2022-02-09 00:35:13 main:574] : INFO : Epoch 2001 | loss: 0.0311258 | val_loss: 0.0311446 | Time: 1459.53 ms [2022-02-09 00:35:14 main:574] : INFO : Epoch 2002 | loss: 0.0311262 | val_loss: 0.0311485 | Time: 1444.1 ms [2022-02-09 00:35:15 main:574] : INFO : Epoch 2003 | loss: 0.0311283 | val_loss: 0.0311481 | Time: 1460.02 ms [2022-02-09 00:35:17 main:574] : INFO : Epoch 2004 | loss: 0.0311266 | val_loss: 0.0311441 | Time: 1451.97 ms [2022-02-09 00:35:18 main:574] : INFO : Epoch 2005 | loss: 0.0311257 | val_loss: 0.0311475 | Time: 1461.94 ms [2022-02-09 00:35:20 main:574] : INFO : Epoch 2006 | loss: 0.0311265 | val_loss: 0.0311476 | Time: 1458.1 ms [2022-02-09 00:35:21 main:574] : INFO : Epoch 2007 | loss: 0.0311245 | val_loss: 0.0311469 | Time: 1445.55 ms [2022-02-09 00:35:23 main:574] : INFO : Epoch 2008 | loss: 0.0311247 | val_loss: 0.031145 | Time: 1458.54 ms [2022-02-09 00:35:24 main:574] : INFO : Epoch 2009 | loss: 0.0311236 | val_loss: 0.0311493 | Time: 1448.78 ms [2022-02-09 00:35:26 main:574] : INFO : Epoch 2010 | loss: 0.0311263 | val_loss: 0.0311496 | Time: 1446.84 ms [2022-02-09 00:35:27 main:574] : INFO : Epoch 2011 | loss: 0.0311307 | val_loss: 0.0311549 | Time: 1460.46 ms [2022-02-09 00:35:29 main:574] : INFO : Epoch 2012 | loss: 0.031131 | val_loss: 0.031155 | Time: 1453.48 ms [2022-02-09 00:35:30 main:574] : INFO : Epoch 2013 | loss: 0.0311506 | val_loss: 0.0311589 | Time: 1459.51 ms [2022-02-09 00:35:31 main:574] : INFO : Epoch 2014 | loss: 0.0311472 | val_loss: 0.0311503 | Time: 1452.32 ms [2022-02-09 00:35:33 main:574] : INFO : Epoch 2015 | loss: 0.0311429 | val_loss: 0.0311524 | Time: 1463.07 ms [2022-02-09 00:35:34 main:574] : INFO : Epoch 2016 | loss: 0.0311381 | val_loss: 0.0311525 | Time: 1470.59 ms [2022-02-09 00:35:36 main:574] : INFO : Epoch 2017 | loss: 0.0311346 | val_loss: 0.0311488 | Time: 1439.59 ms [2022-02-09 00:35:37 main:574] : INFO : Epoch 2018 | loss: 0.0311334 | val_loss: 0.0311488 | Time: 1457.12 ms [2022-02-09 00:35:39 main:574] : INFO : Epoch 2019 | loss: 0.031137 | val_loss: 0.0311515 | Time: 1451.84 ms [2022-02-09 00:35:40 main:574] : INFO : Epoch 2020 | loss: 0.0311328 | val_loss: 0.0311497 | Time: 1472.39 ms [2022-02-09 00:35:42 main:574] : INFO : Epoch 2021 | loss: 0.0311331 | val_loss: 0.0311536 | Time: 1459.37 ms [2022-02-09 00:35:43 main:574] : INFO : Epoch 2022 | loss: 0.0311303 | val_loss: 0.0311503 | Time: 1449.91 ms [2022-02-09 00:35:45 main:574] : INFO : Epoch 2023 | loss: 0.0311314 | val_loss: 0.0311526 | Time: 1454.23 ms [2022-02-09 00:35:46 main:574] : INFO : Epoch 2024 | loss: 0.0311349 | val_loss: 0.031153 | Time: 1461.28 ms [2022-02-09 00:35:48 main:574] : INFO : Epoch 2025 | loss: 0.0311412 | val_loss: 0.031154 | Time: 1451.05 ms [2022-02-09 00:35:49 main:574] : INFO : Epoch 2026 | loss: 0.031144 | val_loss: 0.031148 | Time: 1468.73 ms [2022-02-09 00:35:50 main:574] : INFO : Epoch 2027 | loss: 0.0311407 | val_loss: 0.0311496 | Time: 1452.04 ms [2022-02-09 00:35:52 main:574] : INFO : Epoch 2028 | loss: 0.0311353 | val_loss: 0.0311525 | Time: 1456.72 ms [2022-02-09 00:35:53 main:574] : INFO : Epoch 2029 | loss: 0.0311345 | val_loss: 0.0311512 | Time: 1450.09 ms [2022-02-09 00:35:55 main:574] : INFO : Epoch 2030 | loss: 0.0311335 | val_loss: 0.0311489 | Time: 1448.57 ms [2022-02-09 00:35:56 main:574] : INFO : Epoch 2031 | loss: 0.0311329 | val_loss: 0.0311469 | Time: 1466.5 ms [2022-02-09 00:35:58 main:574] : INFO : Epoch 2032 | loss: 0.0311333 | val_loss: 0.0311461 | Time: 1449.74 ms [2022-02-09 00:36:20 main:574] : INFO : Epoch 2033 | loss: 0.0311329 | val_loss: 0.0311501 | Time: 21834.1 ms [2022-02-09 00:36:21 main:574] : INFO : Epoch 2034 | loss: 0.0311316 | val_loss: 0.0311467 | Time: 1441.09 ms [2022-02-09 00:36:22 main:574] : INFO : Epoch 2035 | loss: 0.0311311 | val_loss: 0.0311456 | Time: 1447.92 ms [2022-02-09 00:36:24 main:574] : INFO : Epoch 2036 | loss: 0.0311305 | val_loss: 0.0311461 | Time: 1433.83 ms [2022-02-09 00:36:25 main:574] : INFO : Epoch 2037 | loss: 0.0311319 | val_loss: 0.0311468 | Time: 1458.14 ms [2022-02-09 00:36:27 main:574] : INFO : Epoch 2038 | loss: 0.0311308 | val_loss: 0.0311462 | Time: 1457.08 ms [2022-02-09 00:36:28 main:574] : INFO : Epoch 2039 | loss: 0.0311311 | val_loss: 0.0311527 | Time: 1437.14 ms [2022-02-09 00:36:40 main:574] : INFO : Epoch 2040 | loss: 0.0311329 | val_loss: 0.0311484 | Time: 11594.9 ms [2022-02-09 00:36:41 main:574] : INFO : Epoch 2041 | loss: 0.0311338 | val_loss: 0.031152 | Time: 1448.96 ms [2022-02-09 00:36:43 main:574] : INFO : Epoch 2042 | loss: 0.0311327 | val_loss: 0.031149 | Time: 1444.86 ms [2022-02-09 00:36:44 main:574] : INFO : Epoch 2043 | loss: 0.0311292 | val_loss: 0.0311474 | Time: 1450.61 ms [2022-02-09 00:36:46 main:574] : INFO : Epoch 2044 | loss: 0.0311282 | val_loss: 0.0311519 | Time: 1436.54 ms [2022-02-09 00:36:47 main:574] : INFO : Epoch 2045 | loss: 0.0311301 | val_loss: 0.0311535 | Time: 1451.03 ms [2022-02-09 00:36:49 main:574] : INFO : Epoch 2046 | loss: 0.0311304 | val_loss: 0.0311466 | Time: 1451.78 ms [2022-02-09 00:36:50 main:574] : INFO : Epoch 2047 | loss: 0.0311367 | val_loss: 0.0311651 | Time: 1441.27 ms [2022-02-09 00:36:51 main:574] : INFO : Epoch 2048 | loss: 0.0311387 | val_loss: 0.0311556 | Time: 1452.71 ms [2022-02-09 00:36:51 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311556 [2022-02-09 00:36:51 main:603] : INFO : Saving end state to config to file [2022-02-09 00:36:51 main:608] : INFO : Success, exiting.. 00:36:51 (20056): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)