Task 11621872
| Name | ParityModified-1639959933-7739-3-0_0 |
| Workunit | 8838343 |
| Created | 16 Feb 2022, 19:57:28 UTC |
| Sent | 26 Feb 2022, 14:48:19 UTC |
| Report deadline | 6 Mar 2022, 14:48:19 UTC |
| Received | 27 Feb 2022, 14:29:12 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 8158 |
| Run time | 4 hours 54 min 6 sec |
| CPU time | 4 hours 21 min 34 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 13,453.09 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.41 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 11:10:58 main:574] : INFO : Epoch 1755 | loss: 0.0311401 | val_loss: 0.0311504 | Time: 6924.3 ms [2022-02-27 11:11:05 main:574] : INFO : Epoch 1756 | loss: 0.0311407 | val_loss: 0.0311497 | Time: 6930.74 ms [2022-02-27 11:11:12 main:574] : INFO : Epoch 1757 | loss: 0.0311405 | val_loss: 0.0311528 | Time: 6826.44 ms [2022-02-27 11:11:18 main:574] : INFO : Epoch 1758 | loss: 0.0311413 | val_loss: 0.0311476 | Time: 6856.08 ms [2022-02-27 11:11:25 main:574] : INFO : Epoch 1759 | loss: 0.0311417 | val_loss: 0.0311489 | Time: 6906.13 ms [2022-02-27 11:11:32 main:574] : INFO : Epoch 1760 | loss: 0.0311444 | val_loss: 0.0311496 | Time: 6906.34 ms [2022-02-27 11:11:39 main:574] : INFO : Epoch 1761 | loss: 0.0311399 | val_loss: 0.0311495 | Time: 6828.16 ms [2022-02-27 11:11:46 main:574] : INFO : Epoch 1762 | loss: 0.0311406 | val_loss: 0.0311511 | Time: 7042.16 ms [2022-02-27 11:11:53 main:574] : INFO : Epoch 1763 | loss: 0.0311429 | val_loss: 0.0311483 | Time: 6940.84 ms [2022-02-27 11:12:00 main:574] : INFO : Epoch 1764 | loss: 0.0311402 | val_loss: 0.0311488 | Time: 6899.18 ms [2022-02-27 11:12:07 main:574] : INFO : Epoch 1765 | loss: 0.0311415 | val_loss: 0.0311517 | Time: 6844.4 ms [2022-02-27 11:12:14 main:574] : INFO : Epoch 1766 | loss: 0.0311404 | val_loss: 0.0311498 | Time: 6899.53 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-02-27 12:15:07 main:435] : INFO : Set logging level to 1 [2022-02-27 12:15:07 main:441] : INFO : Running in BOINC Client mode [2022-02-27 12:15:07 main:444] : INFO : Resolving all filenames [2022-02-27 12:15:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 12:15:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 12:15:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 12:15:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 12:15:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 12:15:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 12:15:07 main:474] : INFO : Configuration: [2022-02-27 12:15:07 main:475] : INFO : Model type: GRU [2022-02-27 12:15:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 12:15:07 main:477] : INFO : Max Epochs: 2048 [2022-02-27 12:15:07 main:478] : INFO : Batch Size: 128 [2022-02-27 12:15:07 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 12:15:07 main:480] : INFO : Patience: 10 [2022-02-27 12:15:07 main:481] : INFO : Hidden Width: 12 [2022-02-27 12:15:08 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 12:15:08 main:483] : INFO : # Backend Layers: 4 [2022-02-27 12:15:08 main:484] : INFO : # Threads: 1 [2022-02-27 12:15:08 main:486] : INFO : Preparing Dataset [2022-02-27 12:15:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 12:15:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 12:15:09 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 12:15:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 12:15:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 12:15:09 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 12:15:09 main:494] : INFO : Creating Model [2022-02-27 12:15:09 main:507] : INFO : Preparing config file [2022-02-27 12:15:09 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 12:15:09 main:512] : INFO : Loading config [2022-02-27 12:15:09 main:514] : INFO : Loading state [2022-02-27 12:15:10 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 12:15:10 main:562] : INFO : Starting Training [2022-02-27 12:15:14 main:574] : INFO : Epoch 1742 | loss: 0.031187 | val_loss: 0.031159 | Time: 3621.26 ms [2022-02-27 12:15:17 main:574] : INFO : Epoch 1743 | loss: 0.0311522 | val_loss: 0.0311543 | Time: 3238.91 ms [2022-02-27 12:15:20 main:574] : INFO : Epoch 1744 | loss: 0.0311466 | val_loss: 0.0311501 | Time: 3283.93 ms [2022-02-27 12:15:24 main:574] : INFO : Epoch 1745 | loss: 0.0311441 | val_loss: 0.0311508 | Time: 3360.43 ms [2022-02-27 12:15:27 main:574] : INFO : Epoch 1746 | loss: 0.0311423 | val_loss: 0.0311491 | Time: 3270.13 ms [2022-02-27 12:15:31 main:574] : INFO : Epoch 1747 | loss: 0.0311419 | val_loss: 0.031148 | Time: 3343.61 ms [2022-02-27 12:15:34 main:574] : INFO : Epoch 1748 | loss: 0.0311422 | val_loss: 0.0311502 | Time: 3365.06 ms [2022-02-27 12:15:37 main:574] : INFO : Epoch 1749 | loss: 0.0311423 | val_loss: 0.0311492 | Time: 3288.99 ms [2022-02-27 12:15:41 main:574] : INFO : Epoch 1750 | loss: 0.0311435 | val_loss: 0.0311489 | Time: 3287.59 ms [2022-02-27 12:15:44 main:574] : INFO : Epoch 1751 | loss: 0.0311436 | val_loss: 0.0311495 | Time: 3345.64 ms [2022-02-27 12:15:47 main:574] : INFO : Epoch 1752 | loss: 0.0311454 | val_loss: 0.0311478 | Time: 3289.12 ms [2022-02-27 12:15:51 main:574] : INFO : Epoch 1753 | loss: 0.0311453 | val_loss: 0.031149 | Time: 3320.66 ms [2022-02-27 12:15:54 main:574] : INFO : Epoch 1754 | loss: 0.0311452 | val_loss: 0.031149 | Time: 3350.53 ms [2022-02-27 12:15:57 main:574] : INFO : Epoch 1755 | loss: 0.0311437 | val_loss: 0.0311489 | Time: 3334.27 ms [2022-02-27 12:16:01 main:574] : INFO : Epoch 1756 | loss: 0.0311444 | val_loss: 0.0311495 | Time: 3267.62 ms [2022-02-27 12:16:04 main:574] : INFO : Epoch 1757 | loss: 0.0311478 | val_loss: 0.0311522 | Time: 3313.64 ms [2022-02-27 12:16:07 main:574] : INFO : Epoch 1758 | loss: 0.0311478 | val_loss: 0.0311544 | Time: 3284.94 ms [2022-02-27 12:16:11 main:574] : INFO : Epoch 1759 | loss: 0.0311466 | val_loss: 0.0311526 | Time: 3323.06 ms [2022-02-27 12:16:14 main:574] : INFO : Epoch 1760 | loss: 0.0311463 | val_loss: 0.0311497 | Time: 3405.94 ms [2022-02-27 12:16:18 main:574] : INFO : Epoch 1761 | loss: 0.0311437 | val_loss: 0.031151 | Time: 3265.49 ms [2022-02-27 12:16:21 main:574] : INFO : Epoch 1762 | loss: 0.031143 | val_loss: 0.0311483 | Time: 3282.43 ms [2022-02-27 12:16:24 main:574] : INFO : Epoch 1763 | loss: 0.0311427 | val_loss: 0.0311516 | Time: 3328.26 ms [2022-02-27 12:16:28 main:574] : INFO : Epoch 1764 | loss: 0.0311445 | val_loss: 0.0311521 | Time: 3285.07 ms [2022-02-27 12:16:31 main:574] : INFO : Epoch 1765 | loss: 0.0311459 | val_loss: 0.0311535 | Time: 3295.57 ms [2022-02-27 12:16:34 main:574] : INFO : Epoch 1766 | loss: 0.0311442 | val_loss: 0.0311525 | Time: 3365.72 ms [2022-02-27 12:16:38 main:574] : INFO : Epoch 1767 | loss: 0.0311428 | val_loss: 0.0311502 | Time: 3394.98 ms [2022-02-27 12:16:41 main:574] : INFO : Epoch 1768 | loss: 0.0311417 | val_loss: 0.031152 | Time: 3243.33 ms [2022-02-27 12:16:44 main:574] : INFO : Epoch 1769 | loss: 0.0311462 | val_loss: 0.031158 | Time: 3327.89 ms [2022-02-27 12:16:48 main:574] : INFO : Epoch 1770 | loss: 0.0311492 | val_loss: 0.0311528 | Time: 3244.56 ms [2022-02-27 12:16:51 main:574] : INFO : Epoch 1771 | loss: 0.0311451 | val_loss: 0.0311513 | Time: 3317.66 ms [2022-02-27 12:16:54 main:574] : INFO : Epoch 1772 | loss: 0.031145 | val_loss: 0.0311509 | Time: 3337.81 ms [2022-02-27 12:16:58 main:574] : INFO : Epoch 1773 | loss: 0.0311447 | val_loss: 0.0311499 | Time: 3299.17 ms [2022-02-27 12:17:01 main:574] : INFO : Epoch 1774 | loss: 0.0311435 | val_loss: 0.0311491 | Time: 3263.71 ms [2022-02-27 12:17:04 main:574] : INFO : Epoch 1775 | loss: 0.0311431 | val_loss: 0.0311504 | Time: 3344.26 ms [2022-02-27 12:17:08 main:574] : INFO : Epoch 1776 | loss: 0.0311432 | val_loss: 0.0311513 | Time: 3171.08 ms [2022-02-27 12:17:11 main:574] : INFO : Epoch 1777 | loss: 0.0311426 | val_loss: 0.0311503 | Time: 3377.61 ms [2022-02-27 12:17:14 main:574] : INFO : Epoch 1778 | loss: 0.031142 | val_loss: 0.0311513 | Time: 3323.3 ms [2022-02-27 12:17:18 main:574] : INFO : Epoch 1779 | loss: 0.0311412 | val_loss: 0.0311486 | Time: 3300.62 ms [2022-02-27 12:17:21 main:574] : INFO : Epoch 1780 | loss: 0.0311402 | val_loss: 0.0311482 | Time: 3231.35 ms [2022-02-27 12:17:24 main:574] : INFO : Epoch 1781 | loss: 0.031141 | val_loss: 0.0311474 | Time: 3345.72 ms [2022-02-27 12:17:28 main:574] : INFO : Epoch 1782 | loss: 0.0311412 | val_loss: 0.0311467 | Time: 3284.17 ms [2022-02-27 12:17:31 main:574] : INFO : Epoch 1783 | loss: 0.0311408 | val_loss: 0.0311526 | Time: 3336.06 ms [2022-02-27 12:17:34 main:574] : INFO : Epoch 1784 | loss: 0.0311505 | val_loss: 0.031158 | Time: 3356.07 ms [2022-02-27 12:17:38 main:574] : INFO : Epoch 1785 | loss: 0.0311517 | val_loss: 0.0311523 | Time: 3301.28 ms [2022-02-27 12:17:41 main:574] : INFO : Epoch 1786 | loss: 0.0311484 | val_loss: 0.0311514 | Time: 3299.89 ms [2022-02-27 12:17:45 main:574] : INFO : Epoch 1787 | loss: 0.0311455 | val_loss: 0.0311515 | Time: 3473.86 ms [2022-02-27 12:17:48 main:574] : INFO : Epoch 1788 | loss: 0.0311458 | val_loss: 0.0311546 | Time: 3324.11 ms [2022-02-27 12:17:51 main:574] : INFO : Epoch 1789 | loss: 0.0311453 | val_loss: 0.0311519 | Time: 3293.95 ms [2022-02-27 12:17:55 main:574] : INFO : Epoch 1790 | loss: 0.0311451 | val_loss: 0.0311513 | Time: 3386.24 ms [2022-02-27 12:17:58 main:574] : INFO : Epoch 1791 | loss: 0.0311475 | val_loss: 0.0311535 | Time: 3357.01 ms [2022-02-27 12:18:01 main:574] : INFO : Epoch 1792 | loss: 0.0311447 | val_loss: 0.0311529 | Time: 3300.36 ms [2022-02-27 12:18:05 main:574] : INFO : Epoch 1793 | loss: 0.0311447 | val_loss: 0.0311514 | Time: 3425.49 ms [2022-02-27 12:18:08 main:574] : INFO : Epoch 1794 | loss: 0.0311491 | val_loss: 0.0311602 | Time: 3355.57 ms [2022-02-27 12:18:12 main:574] : INFO : Epoch 1795 | loss: 0.031161 | val_loss: 0.0311634 | Time: 3319.91 ms [2022-02-27 12:18:15 main:574] : INFO : Epoch 1796 | loss: 0.0311565 | val_loss: 0.0311543 | Time: 3383.99 ms [2022-02-27 12:18:19 main:574] : INFO : Epoch 1797 | loss: 0.0311535 | val_loss: 0.0311524 | Time: 3339.61 ms [2022-02-27 12:18:22 main:574] : INFO : Epoch 1798 | loss: 0.0311502 | val_loss: 0.0311537 | Time: 3355.42 ms [2022-02-27 12:18:25 main:574] : INFO : Epoch 1799 | loss: 0.0311495 | val_loss: 0.0311509 | Time: 3387.49 ms [2022-02-27 12:18:29 main:574] : INFO : Epoch 1800 | loss: 0.0311487 | val_loss: 0.0311547 | Time: 3412.6 ms [2022-02-27 12:18:32 main:574] : INFO : Epoch 1801 | loss: 0.0311498 | val_loss: 0.0311532 | Time: 3430.7 ms [2022-02-27 12:18:36 main:574] : INFO : Epoch 1802 | loss: 0.0311485 | val_loss: 0.0311528 | Time: 3316.18 ms [2022-02-27 12:18:39 main:574] : INFO : Epoch 1803 | loss: 0.0311488 | val_loss: 0.0311495 | Time: 3366.63 ms [2022-02-27 12:18:43 main:574] : INFO : Epoch 1804 | loss: 0.0311461 | val_loss: 0.0311498 | Time: 3441.32 ms [2022-02-27 12:18:46 main:574] : INFO : Epoch 1805 | loss: 0.0311453 | val_loss: 0.03115 | Time: 3312.16 ms [2022-02-27 12:18:49 main:574] : INFO : Epoch 1806 | loss: 0.0311469 | val_loss: 0.03115 | Time: 3340.52 ms [2022-02-27 12:18:53 main:574] : INFO : Epoch 1807 | loss: 0.0311489 | val_loss: 0.0311485 | Time: 3407.77 ms [2022-02-27 12:18:56 main:574] : INFO : Epoch 1808 | loss: 0.031147 | val_loss: 0.031149 | Time: 3383.39 ms [2022-02-27 12:19:00 main:574] : INFO : Epoch 1809 | loss: 0.0311462 | val_loss: 0.0311493 | Time: 3492.25 ms [2022-02-27 12:19:03 main:574] : INFO : Epoch 1810 | loss: 0.0311455 | val_loss: 0.0311491 | Time: 3440.65 ms [2022-02-27 12:19:07 main:574] : INFO : Epoch 1811 | loss: 0.0311453 | val_loss: 0.0311523 | Time: 3452.42 ms [2022-02-27 12:19:10 main:574] : INFO : Epoch 1812 | loss: 0.0311472 | val_loss: 0.031153 | Time: 3403.43 ms [2022-02-27 12:19:13 main:574] : INFO : Epoch 1813 | loss: 0.0311485 | val_loss: 0.031156 | Time: 3334.9 ms [2022-02-27 12:19:17 main:574] : INFO : Epoch 1814 | loss: 0.0311481 | val_loss: 0.0311525 | Time: 3420.93 ms [2022-02-27 12:19:20 main:574] : INFO : Epoch 1815 | loss: 0.0311501 | val_loss: 0.031156 | Time: 3494.24 ms [2022-02-27 12:19:24 main:574] : INFO : Epoch 1816 | loss: 0.0311505 | val_loss: 0.0311522 | Time: 3394.36 ms [2022-02-27 12:19:27 main:574] : INFO : Epoch 1817 | loss: 0.0311515 | val_loss: 0.0311629 | Time: 3361.05 ms [2022-02-27 12:19:31 main:574] : INFO : Epoch 1818 | loss: 0.031152 | val_loss: 0.0311542 | Time: 3449.16 ms [2022-02-27 12:19:34 main:574] : INFO : Epoch 1819 | loss: 0.031151 | val_loss: 0.031152 | Time: 3408.94 ms [2022-02-27 12:19:38 main:574] : INFO : Epoch 1820 | loss: 0.0311489 | val_loss: 0.0311546 | Time: 3341.61 ms [2022-02-27 12:19:41 main:574] : INFO : Epoch 1821 | loss: 0.0311486 | val_loss: 0.0311529 | Time: 3444.42 ms [2022-02-27 12:19:44 main:574] : INFO : Epoch 1822 | loss: 0.0311478 | val_loss: 0.0311532 | Time: 3387.22 ms [2022-02-27 12:19:48 main:574] : INFO : Epoch 1823 | loss: 0.0311467 | val_loss: 0.0311506 | Time: 3399.79 ms [2022-02-27 12:19:51 main:574] : INFO : Epoch 1824 | loss: 0.0311474 | val_loss: 0.0311517 | Time: 3483.5 ms [2022-02-27 12:19:55 main:574] : INFO : Epoch 1825 | loss: 0.0311477 | val_loss: 0.0311511 | Time: 3362.29 ms [2022-02-27 12:19:58 main:574] : INFO : Epoch 1826 | loss: 0.0311478 | val_loss: 0.0311523 | Time: 3402.42 ms [2022-02-27 12:20:02 main:574] : INFO : Epoch 1827 | loss: 0.0311464 | val_loss: 0.0311529 | Time: 3490.06 ms [2022-02-27 12:20:05 main:574] : INFO : Epoch 1828 | loss: 0.031146 | val_loss: 0.0311524 | Time: 3458.27 ms [2022-02-27 12:20:09 main:574] : INFO : Epoch 1829 | loss: 0.031148 | val_loss: 0.0311544 | Time: 3477.32 ms [2022-02-27 12:20:12 main:574] : INFO : Epoch 1830 | loss: 0.031154 | val_loss: 0.0311643 | Time: 3395.51 ms [2022-02-27 12:20:16 main:574] : INFO : Epoch 1831 | loss: 0.0311519 | val_loss: 0.0311539 | Time: 3382.36 ms [2022-02-27 12:20:19 main:574] : INFO : Epoch 1832 | loss: 0.0311497 | val_loss: 0.031154 | Time: 3503.99 ms [2022-02-27 12:20:23 main:574] : INFO : Epoch 1833 | loss: 0.0311484 | val_loss: 0.0311554 | Time: 3392.79 ms [2022-02-27 12:20:26 main:574] : INFO : Epoch 1834 | loss: 0.0311481 | val_loss: 0.0311558 | Time: 3309.25 ms [2022-02-27 12:20:29 main:574] : INFO : Epoch 1835 | loss: 0.0311496 | val_loss: 0.0311523 | Time: 3442.1 ms [2022-02-27 12:20:33 main:574] : INFO : Epoch 1836 | loss: 0.031149 | val_loss: 0.0311523 | Time: 3347.36 ms [2022-02-27 12:20:36 main:574] : INFO : Epoch 1837 | loss: 0.0311485 | val_loss: 0.0311577 | Time: 3361.9 ms [2022-02-27 12:20:40 main:574] : INFO : Epoch 1838 | loss: 0.0311507 | val_loss: 0.0311568 | Time: 3508.02 ms [2022-02-27 12:20:43 main:574] : INFO : Epoch 1839 | loss: 0.0311474 | val_loss: 0.0311517 | Time: 3394.62 ms [2022-02-27 12:20:47 main:574] : INFO : Epoch 1840 | loss: 0.0311448 | val_loss: 0.0311508 | Time: 3376.06 ms [2022-02-27 12:20:50 main:574] : INFO : Epoch 1841 | loss: 0.0311465 | val_loss: 0.0311598 | Time: 3282.66 ms [2022-02-27 12:20:53 main:574] : INFO : Epoch 1842 | loss: 0.0311476 | val_loss: 0.0311505 | Time: 3442.58 ms [2022-02-27 12:20:57 main:574] : INFO : Epoch 1843 | loss: 0.0311457 | val_loss: 0.0311509 | Time: 3491.81 ms [2022-02-27 12:21:00 main:574] : INFO : Epoch 1844 | loss: 0.0311437 | val_loss: 0.0311494 | Time: 3225.07 ms [2022-02-27 12:21:04 main:574] : INFO : Epoch 1845 | loss: 0.0311432 | val_loss: 0.0311495 | Time: 3335.02 ms [2022-02-27 12:21:07 main:574] : INFO : Epoch 1846 | loss: 0.0311447 | val_loss: 0.0311515 | Time: 3369.79 ms [2022-02-27 12:21:10 main:574] : INFO : Epoch 1847 | loss: 0.0311443 | val_loss: 0.0311519 | Time: 3311.34 ms [2022-02-27 12:21:14 main:574] : INFO : Epoch 1848 | loss: 0.0311455 | val_loss: 0.0311568 | Time: 3411.29 ms [2022-02-27 12:21:17 main:574] : INFO : Epoch 1849 | loss: 0.0311431 | val_loss: 0.0311499 | Time: 3373.43 ms [2022-02-27 12:21:21 main:574] : INFO : Epoch 1850 | loss: 0.0311431 | val_loss: 0.0311514 | Time: 3403.99 ms [2022-02-27 12:21:24 main:574] : INFO : Epoch 1851 | loss: 0.0311425 | val_loss: 0.0311502 | Time: 3285.02 ms [2022-02-27 12:21:27 main:574] : INFO : Epoch 1852 | loss: 0.0311428 | val_loss: 0.0311515 | Time: 3417.81 ms [2022-02-27 12:21:31 main:574] : INFO : Epoch 1853 | loss: 0.0311421 | val_loss: 0.0311514 | Time: 3416.48 ms [2022-02-27 12:21:34 main:574] : INFO : Epoch 1854 | loss: 0.0311418 | val_loss: 0.0311508 | Time: 3237.81 ms [2022-02-27 12:21:37 main:574] : INFO : Epoch 1855 | loss: 0.0311453 | val_loss: 0.0311494 | Time: 3321.3 ms [2022-02-27 12:21:41 main:574] : INFO : Epoch 1856 | loss: 0.0311424 | val_loss: 0.0311519 | Time: 3359.19 ms [2022-02-27 12:21:44 main:574] : INFO : Epoch 1857 | loss: 0.031143 | val_loss: 0.03115 | Time: 3390.33 ms [2022-02-27 12:21:48 main:574] : INFO : Epoch 1858 | loss: 0.0311434 | val_loss: 0.0311512 | Time: 3374.57 ms [2022-02-27 12:21:51 main:574] : INFO : Epoch 1859 | loss: 0.0311414 | val_loss: 0.0311525 | Time: 3365.77 ms [2022-02-27 12:21:54 main:574] : INFO : Epoch 1860 | loss: 0.0311424 | val_loss: 0.0311518 | Time: 3253.74 ms [2022-02-27 12:21:58 main:574] : INFO : Epoch 1861 | loss: 0.0311419 | val_loss: 0.0311547 | Time: 3303.87 ms [2022-02-27 12:22:01 main:574] : INFO : Epoch 1862 | loss: 0.0311412 | val_loss: 0.0311513 | Time: 3443.9 ms [2022-02-27 12:22:05 main:574] : INFO : Epoch 1863 | loss: 0.0311409 | val_loss: 0.0311501 | Time: 3450.55 ms [2022-02-27 12:22:08 main:574] : INFO : Epoch 1864 | loss: 0.0311398 | val_loss: 0.0311502 | Time: 3305.21 ms [2022-02-27 12:22:11 main:574] : INFO : Epoch 1865 | loss: 0.0311416 | val_loss: 0.0311517 | Time: 3449.96 ms [2022-02-27 12:22:15 main:574] : INFO : Epoch 1866 | loss: 0.0311443 | val_loss: 0.0311489 | Time: 3423.69 ms [2022-02-27 12:22:18 main:574] : INFO : Epoch 1867 | loss: 0.0311409 | val_loss: 0.031149 | Time: 3336.83 ms [2022-02-27 12:22:22 main:574] : INFO : Epoch 1868 | loss: 0.0311409 | val_loss: 0.031149 | Time: 3527.38 ms [2022-02-27 12:22:25 main:574] : INFO : Epoch 1869 | loss: 0.0311479 | val_loss: 0.0311586 | Time: 3384.94 ms [2022-02-27 12:22:29 main:574] : INFO : Epoch 1870 | loss: 0.031154 | val_loss: 0.0311612 | Time: 3228.57 ms [2022-02-27 12:22:32 main:574] : INFO : Epoch 1871 | loss: 0.031153 | val_loss: 0.0311604 | Time: 3345.95 ms [2022-02-27 12:22:35 main:574] : INFO : Epoch 1872 | loss: 0.0311566 | val_loss: 0.0311573 | Time: 3415.18 ms [2022-02-27 12:22:39 main:574] : INFO : Epoch 1873 | loss: 0.0311505 | val_loss: 0.0311516 | Time: 3305.02 ms [2022-02-27 12:22:42 main:574] : INFO : Epoch 1874 | loss: 0.0311452 | val_loss: 0.0311496 | Time: 3366.11 ms [2022-02-27 12:22:46 main:574] : INFO : Epoch 1875 | loss: 0.0311436 | val_loss: 0.0311506 | Time: 3425.51 ms [2022-02-27 12:22:49 main:574] : INFO : Epoch 1876 | loss: 0.0311426 | val_loss: 0.0311494 | Time: 3339.41 ms [2022-02-27 12:22:52 main:574] : INFO : Epoch 1877 | loss: 0.0311423 | val_loss: 0.0311501 | Time: 3305.46 ms [2022-02-27 12:22:56 main:574] : INFO : Epoch 1878 | loss: 0.0311432 | val_loss: 0.0311531 | Time: 3381.4 ms [2022-02-27 12:22:59 main:574] : INFO : Epoch 1879 | loss: 0.0311456 | val_loss: 0.0311537 | Time: 3361.48 ms [2022-02-27 12:23:02 main:574] : INFO : Epoch 1880 | loss: 0.0311434 | val_loss: 0.0311513 | Time: 3304.21 ms [2022-02-27 12:23:06 main:574] : INFO : Epoch 1881 | loss: 0.0311424 | val_loss: 0.0311506 | Time: 3449.5 ms [2022-02-27 12:23:09 main:574] : INFO : Epoch 1882 | loss: 0.0311426 | val_loss: 0.0311528 | Time: 3441.7 ms [2022-02-27 12:23:13 main:574] : INFO : Epoch 1883 | loss: 0.0311421 | val_loss: 0.031148 | Time: 3392.59 ms [2022-02-27 12:23:16 main:574] : INFO : Epoch 1884 | loss: 0.0311424 | val_loss: 0.0311502 | Time: 3439.53 ms [2022-02-27 12:23:20 main:574] : INFO : Epoch 1885 | loss: 0.0311433 | val_loss: 0.0311487 | Time: 3426.97 ms [2022-02-27 12:23:23 main:574] : INFO : Epoch 1886 | loss: 0.0311421 | val_loss: 0.0311475 | Time: 3398.08 ms [2022-02-27 12:23:27 main:574] : INFO : Epoch 1887 | loss: 0.0311406 | val_loss: 0.0311467 | Time: 3292.03 ms [2022-02-27 12:23:30 main:574] : INFO : Epoch 1888 | loss: 0.0311427 | val_loss: 0.0311477 | Time: 3416.65 ms [2022-02-27 12:23:33 main:574] : INFO : Epoch 1889 | loss: 0.0311414 | val_loss: 0.0311503 | Time: 3426.52 ms [2022-02-27 12:23:37 main:574] : INFO : Epoch 1890 | loss: 0.0311439 | val_loss: 0.0311513 | Time: 3325.34 ms [2022-02-27 12:23:40 main:574] : INFO : Epoch 1891 | loss: 0.0311423 | val_loss: 0.0311506 | Time: 3459.08 ms [2022-02-27 12:23:44 main:574] : INFO : Epoch 1892 | loss: 0.0311425 | val_loss: 0.0311528 | Time: 3328.52 ms [2022-02-27 12:23:47 main:574] : INFO : Epoch 1893 | loss: 0.0311441 | val_loss: 0.0311529 | Time: 3437.02 ms [2022-02-27 12:23:51 main:574] : INFO : Epoch 1894 | loss: 0.031142 | val_loss: 0.0311511 | Time: 3336.53 ms [2022-02-27 12:23:54 main:574] : INFO : Epoch 1895 | loss: 0.0311422 | val_loss: 0.0311503 | Time: 3436.97 ms [2022-02-27 12:23:57 main:574] : INFO : Epoch 1896 | loss: 0.0311424 | val_loss: 0.0311502 | Time: 3281.43 ms [2022-02-27 12:24:01 main:574] : INFO : Epoch 1897 | loss: 0.0311456 | val_loss: 0.0311488 | Time: 3348.65 ms [2022-02-27 12:24:04 main:574] : INFO : Epoch 1898 | loss: 0.0311449 | val_loss: 0.0311485 | Time: 3410.59 ms [2022-02-27 12:24:08 main:574] : INFO : Epoch 1899 | loss: 0.0311437 | val_loss: 0.0311507 | Time: 3416 ms [2022-02-27 12:24:11 main:574] : INFO : Epoch 1900 | loss: 0.0311433 | val_loss: 0.0311522 | Time: 3274.88 ms [2022-02-27 12:24:14 main:574] : INFO : Epoch 1901 | loss: 0.0311419 | val_loss: 0.0311507 | Time: 3422.37 ms [2022-02-27 12:24:18 main:574] : INFO : Epoch 1902 | loss: 0.0311416 | val_loss: 0.0311487 | Time: 3395.05 ms [2022-02-27 12:24:21 main:574] : INFO : Epoch 1903 | loss: 0.03114 | val_loss: 0.031149 | Time: 3300.07 ms [2022-02-27 12:24:25 main:574] : INFO : Epoch 1904 | loss: 0.0311411 | val_loss: 0.0311498 | Time: 3433.64 ms [2022-02-27 12:24:28 main:574] : INFO : Epoch 1905 | loss: 0.0311408 | val_loss: 0.0311493 | Time: 3376.62 ms [2022-02-27 12:24:31 main:574] : INFO : Epoch 1906 | loss: 0.0311406 | val_loss: 0.0311494 | Time: 3309.65 ms [2022-02-27 12:24:35 main:574] : INFO : Epoch 1907 | loss: 0.0311399 | val_loss: 0.0311487 | Time: 3429.53 ms [2022-02-27 12:24:38 main:574] : INFO : Epoch 1908 | loss: 0.0311409 | val_loss: 0.0311527 | Time: 3391.59 ms [2022-02-27 12:24:42 main:574] : INFO : Epoch 1909 | loss: 0.0311427 | val_loss: 0.031153 | Time: 3336.1 ms [2022-02-27 12:24:45 main:574] : INFO : Epoch 1910 | loss: 0.0311415 | val_loss: 0.0311529 | Time: 3352.98 ms [2022-02-27 12:24:48 main:574] : INFO : Epoch 1911 | loss: 0.0311448 | val_loss: 0.0311508 | Time: 3401.23 ms [2022-02-27 12:24:52 main:574] : INFO : Epoch 1912 | loss: 0.0311435 | val_loss: 0.031152 | Time: 3397.06 ms [2022-02-27 12:24:55 main:574] : INFO : Epoch 1913 | loss: 0.0311414 | val_loss: 0.0311511 | Time: 3305.57 ms [2022-02-27 12:24:59 main:574] : INFO : Epoch 1914 | loss: 0.0311412 | val_loss: 0.0311515 | Time: 3413.52 ms [2022-02-27 12:25:02 main:574] : INFO : Epoch 1915 | loss: 0.0311402 | val_loss: 0.0311597 | Time: 3509.72 ms [2022-02-27 12:25:06 main:574] : INFO : Epoch 1916 | loss: 0.031144 | val_loss: 0.0311539 | Time: 3338.71 ms [2022-02-27 12:25:09 main:574] : INFO : Epoch 1917 | loss: 0.031142 | val_loss: 0.0311548 | Time: 3355.23 ms [2022-02-27 12:25:13 main:574] : INFO : Epoch 1918 | loss: 0.0311427 | val_loss: 0.0311502 | Time: 3460.93 ms [2022-02-27 12:25:16 main:574] : INFO : Epoch 1919 | loss: 0.0311408 | val_loss: 0.0311494 | Time: 3461.99 ms [2022-02-27 12:25:19 main:574] : INFO : Epoch 1920 | loss: 0.0311395 | val_loss: 0.0311495 | Time: 3290.28 ms [2022-02-27 12:25:23 main:574] : INFO : Epoch 1921 | loss: 0.0311404 | val_loss: 0.0311535 | Time: 3388.22 ms [2022-02-27 12:25:26 main:574] : INFO : Epoch 1922 | loss: 0.031141 | val_loss: 0.031152 | Time: 3520.99 ms [2022-02-27 12:25:30 main:574] : INFO : Epoch 1923 | loss: 0.0311398 | val_loss: 0.0311508 | Time: 3336.21 ms [2022-02-27 12:25:33 main:574] : INFO : Epoch 1924 | loss: 0.0311394 | val_loss: 0.0311496 | Time: 3292.81 ms [2022-02-27 12:25:36 main:574] : INFO : Epoch 1925 | loss: 0.0311421 | val_loss: 0.0311526 | Time: 3408.81 ms [2022-02-27 12:25:40 main:574] : INFO : Epoch 1926 | loss: 0.0311433 | val_loss: 0.0311536 | Time: 3396.17 ms [2022-02-27 12:25:43 main:574] : INFO : Epoch 1927 | loss: 0.0311436 | val_loss: 0.0311518 | Time: 3267.48 ms [2022-02-27 12:25:47 main:574] : INFO : Epoch 1928 | loss: 0.0311425 | val_loss: 0.0311512 | Time: 3320.71 ms [2022-02-27 12:25:50 main:574] : INFO : Epoch 1929 | loss: 0.0311404 | val_loss: 0.0311554 | Time: 3380.4 ms [2022-02-27 12:25:53 main:574] : INFO : Epoch 1930 | loss: 0.0311403 | val_loss: 0.0311588 | Time: 3308.02 ms [2022-02-27 12:25:57 main:574] : INFO : Epoch 1931 | loss: 0.0311424 | val_loss: 0.0311526 | Time: 3431.22 ms [2022-02-27 12:26:00 main:574] : INFO : Epoch 1932 | loss: 0.0311411 | val_loss: 0.0311521 | Time: 3402.91 ms [2022-02-27 12:26:04 main:574] : INFO : Epoch 1933 | loss: 0.031142 | val_loss: 0.0311502 | Time: 3422.17 ms [2022-02-27 12:26:07 main:574] : INFO : Epoch 1934 | loss: 0.031143 | val_loss: 0.0311525 | Time: 3375.44 ms [2022-02-27 12:26:10 main:574] : INFO : Epoch 1935 | loss: 0.0311418 | val_loss: 0.0311538 | Time: 3379.61 ms [2022-02-27 12:26:14 main:574] : INFO : Epoch 1936 | loss: 0.0311437 | val_loss: 0.0311547 | Time: 3298.01 ms [2022-02-27 12:26:17 main:574] : INFO : Epoch 1937 | loss: 0.0311457 | val_loss: 0.0311523 | Time: 3461.5 ms [2022-02-27 12:26:21 main:574] : INFO : Epoch 1938 | loss: 0.0311447 | val_loss: 0.0311526 | Time: 3440.05 ms [2022-02-27 12:26:24 main:574] : INFO : Epoch 1939 | loss: 0.0311435 | val_loss: 0.0311485 | Time: 3430.78 ms [2022-02-27 12:26:28 main:574] : INFO : Epoch 1940 | loss: 0.0311425 | val_loss: 0.0311483 | Time: 3315.32 ms [2022-02-27 12:26:31 main:574] : INFO : Epoch 1941 | loss: 0.0311455 | val_loss: 0.0311598 | Time: 3469.97 ms [2022-02-27 12:26:35 main:574] : INFO : Epoch 1942 | loss: 0.0311445 | val_loss: 0.031152 | Time: 3471.5 ms [2022-02-27 12:26:38 main:574] : INFO : Epoch 1943 | loss: 0.0311403 | val_loss: 0.0311519 | Time: 3245.73 ms [2022-02-27 12:26:41 main:574] : INFO : Epoch 1944 | loss: 0.0311422 | val_loss: 0.0311523 | Time: 3331.84 ms [2022-02-27 12:26:45 main:574] : INFO : Epoch 1945 | loss: 0.0311387 | val_loss: 0.0311494 | Time: 3434.56 ms [2022-02-27 12:26:48 main:574] : INFO : Epoch 1946 | loss: 0.0311419 | val_loss: 0.0311516 | Time: 3341.14 ms [2022-02-27 12:26:51 main:574] : INFO : Epoch 1947 | loss: 0.031146 | val_loss: 0.0311568 | Time: 3264.65 ms [2022-02-27 12:26:55 main:574] : INFO : Epoch 1948 | loss: 0.0311577 | val_loss: 0.0311943 | Time: 3404.34 ms [2022-02-27 12:26:58 main:574] : INFO : Epoch 1949 | loss: 0.0318791 | val_loss: 0.0318146 | Time: 3405.14 ms [2022-02-27 12:27:02 main:574] : INFO : Epoch 1950 | loss: 0.0321382 | val_loss: 0.0314086 | Time: 3331.24 ms [2022-02-27 12:27:05 main:574] : INFO : Epoch 1951 | loss: 0.0312776 | val_loss: 0.0312028 | Time: 3324.23 ms [2022-02-27 12:27:08 main:574] : INFO : Epoch 1952 | loss: 0.031165 | val_loss: 0.0311563 | Time: 3337.68 ms [2022-02-27 12:27:12 main:574] : INFO : Epoch 1953 | loss: 0.03114 | val_loss: 0.031148 | Time: 3426.57 ms [2022-02-27 12:27:15 main:574] : INFO : Epoch 1954 | loss: 0.0311355 | val_loss: 0.0311476 | Time: 3332.77 ms [2022-02-27 12:27:19 main:574] : INFO : Epoch 1955 | loss: 0.0311329 | val_loss: 0.0311463 | Time: 3326.89 ms [2022-02-27 12:27:22 main:574] : INFO : Epoch 1956 | loss: 0.0311311 | val_loss: 0.0311507 | Time: 3314.84 ms [2022-02-27 12:27:25 main:574] : INFO : Epoch 1957 | loss: 0.0311305 | val_loss: 0.0311495 | Time: 3349.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-02-27 13:04:20 main:435] : INFO : Set logging level to 1 [2022-02-27 13:04:20 main:441] : INFO : Running in BOINC Client mode [2022-02-27 13:04:20 main:444] : INFO : Resolving all filenames [2022-02-27 13:04:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 13:04:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 13:04:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 13:04:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 13:04:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 13:04:20 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 13:04:20 main:474] : INFO : Configuration: [2022-02-27 13:04:20 main:475] : INFO : Model type: GRU [2022-02-27 13:04:20 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 13:04:20 main:477] : INFO : Max Epochs: 2048 [2022-02-27 13:04:21 main:478] : INFO : Batch Size: 128 [2022-02-27 13:04:21 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 13:04:21 main:480] : INFO : Patience: 10 [2022-02-27 13:04:21 main:481] : INFO : Hidden Width: 12 [2022-02-27 13:04:21 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 13:04:21 main:483] : INFO : # Backend Layers: 4 [2022-02-27 13:04:21 main:484] : INFO : # Threads: 1 [2022-02-27 13:04:21 main:486] : INFO : Preparing Dataset [2022-02-27 13:04:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 13:04:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 13:04:23 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 13:04:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 13:04:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 13:04:23 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 13:04:23 main:494] : INFO : Creating Model [2022-02-27 13:04:23 main:507] : INFO : Preparing config file [2022-02-27 13:04:23 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 13:04:23 main:512] : INFO : Loading config [2022-02-27 13:04:23 main:514] : INFO : Loading state [2022-02-27 13:04:24 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 13:04:24 main:562] : INFO : Starting Training [2022-02-27 13:04:28 main:574] : INFO : Epoch 1916 | loss: 0.0311641 | val_loss: 0.0311568 | Time: 3613.59 ms [2022-02-27 13:04:31 main:574] : INFO : Epoch 1917 | loss: 0.0311442 | val_loss: 0.0311542 | Time: 3431.05 ms [2022-02-27 13:04:35 main:574] : INFO : Epoch 1918 | loss: 0.0311419 | val_loss: 0.0311514 | Time: 3271.03 ms [2022-02-27 13:04:38 main:574] : INFO : Epoch 1919 | loss: 0.0311442 | val_loss: 0.0311534 | Time: 3311.88 ms [2022-02-27 13:04:42 main:574] : INFO : Epoch 1920 | loss: 0.0311426 | val_loss: 0.0311517 | Time: 3338.46 ms [2022-02-27 13:04:45 main:574] : INFO : Epoch 1921 | loss: 0.031142 | val_loss: 0.0311513 | Time: 3393.63 ms [2022-02-27 13:04:48 main:574] : INFO : Epoch 1922 | loss: 0.0311414 | val_loss: 0.0311491 | Time: 3302.15 ms [2022-02-27 13:04:52 main:574] : INFO : Epoch 1923 | loss: 0.0311392 | val_loss: 0.0311509 | Time: 3308.21 ms [2022-02-27 13:04:55 main:574] : INFO : Epoch 1924 | loss: 0.0311401 | val_loss: 0.0311511 | Time: 3435.05 ms [2022-02-27 13:04:59 main:574] : INFO : Epoch 1925 | loss: 0.0311416 | val_loss: 0.0311522 | Time: 3286.19 ms [2022-02-27 13:05:02 main:574] : INFO : Epoch 1926 | loss: 0.0311421 | val_loss: 0.0311497 | Time: 3335.71 ms [2022-02-27 13:05:05 main:574] : INFO : Epoch 1927 | loss: 0.0311427 | val_loss: 0.0311525 | Time: 3357.32 ms [2022-02-27 13:05:09 main:574] : INFO : Epoch 1928 | loss: 0.0311442 | val_loss: 0.0311518 | Time: 3384.57 ms [2022-02-27 13:05:13 main:574] : INFO : Epoch 1929 | loss: 0.0311445 | val_loss: 0.0311514 | Time: 3518.38 ms [2022-02-27 13:05:16 main:574] : INFO : Epoch 1930 | loss: 0.0311422 | val_loss: 0.0311518 | Time: 3430 ms [2022-02-27 13:05:19 main:574] : INFO : Epoch 1931 | loss: 0.031141 | val_loss: 0.0311516 | Time: 3416.17 ms [2022-02-27 13:05:23 main:574] : INFO : Epoch 1932 | loss: 0.0311411 | val_loss: 0.0311545 | Time: 3442.93 ms [2022-02-27 13:05:26 main:574] : INFO : Epoch 1933 | loss: 0.0311454 | val_loss: 0.0311529 | Time: 3253.07 ms [2022-02-27 13:05:30 main:574] : INFO : Epoch 1934 | loss: 0.0311448 | val_loss: 0.0311533 | Time: 3408.17 ms [2022-02-27 13:05:33 main:574] : INFO : Epoch 1935 | loss: 0.0311457 | val_loss: 0.0311528 | Time: 3394.26 ms [2022-02-27 13:05:37 main:574] : INFO : Epoch 1936 | loss: 0.0311472 | val_loss: 0.0311496 | Time: 3416.92 ms [2022-02-27 13:05:40 main:574] : INFO : Epoch 1937 | loss: 0.0311456 | val_loss: 0.0311491 | Time: 3329.85 ms [2022-02-27 13:05:44 main:574] : INFO : Epoch 1938 | loss: 0.0311453 | val_loss: 0.031157 | Time: 3340.2 ms [2022-02-27 13:05:47 main:574] : INFO : Epoch 1939 | loss: 0.031148 | val_loss: 0.0311535 | Time: 3342.49 ms [2022-02-27 13:05:50 main:574] : INFO : Epoch 1940 | loss: 0.0311456 | val_loss: 0.031151 | Time: 3447.44 ms [2022-02-27 13:05:54 main:574] : INFO : Epoch 1941 | loss: 0.0311437 | val_loss: 0.0311487 | Time: 3432.25 ms [2022-02-27 13:05:57 main:574] : INFO : Epoch 1942 | loss: 0.0311436 | val_loss: 0.0311549 | Time: 3457.13 ms [2022-02-27 13:06:01 main:574] : INFO : Epoch 1943 | loss: 0.031143 | val_loss: 0.031153 | Time: 3311.04 ms [2022-02-27 13:06:04 main:574] : INFO : Epoch 1944 | loss: 0.0311415 | val_loss: 0.0311524 | Time: 3295.56 ms [2022-02-27 13:06:08 main:574] : INFO : Epoch 1945 | loss: 0.0311415 | val_loss: 0.0311509 | Time: 3519.62 ms [2022-02-27 13:06:11 main:574] : INFO : Epoch 1946 | loss: 0.0311428 | val_loss: 0.0311516 | Time: 3520.74 ms [2022-02-27 13:06:15 main:574] : INFO : Epoch 1947 | loss: 0.0311419 | val_loss: 0.0311517 | Time: 3394.43 ms [2022-02-27 13:06:18 main:574] : INFO : Epoch 1948 | loss: 0.0311416 | val_loss: 0.031151 | Time: 3375.55 ms [2022-02-27 13:06:22 main:574] : INFO : Epoch 1949 | loss: 0.0311417 | val_loss: 0.0311497 | Time: 3452.82 ms [2022-02-27 13:06:25 main:574] : INFO : Epoch 1950 | loss: 0.0311396 | val_loss: 0.0311486 | Time: 3396.53 ms [2022-02-27 13:06:29 main:574] : INFO : Epoch 1951 | loss: 0.031139 | val_loss: 0.0311491 | Time: 3328.16 ms [2022-02-27 13:06:32 main:574] : INFO : Epoch 1952 | loss: 0.0311388 | val_loss: 0.031149 | Time: 3468.35 ms [2022-02-27 13:06:36 main:574] : INFO : Epoch 1953 | loss: 0.0311446 | val_loss: 0.0311487 | Time: 3389.81 ms [2022-02-27 13:06:39 main:574] : INFO : Epoch 1954 | loss: 0.0311429 | val_loss: 0.0311502 | Time: 3407.89 ms [2022-02-27 13:06:42 main:574] : INFO : Epoch 1955 | loss: 0.0311405 | val_loss: 0.0311475 | Time: 3388.28 ms [2022-02-27 13:06:46 main:574] : INFO : Epoch 1956 | loss: 0.0311424 | val_loss: 0.0311526 | Time: 3491.97 ms [2022-02-27 13:06:49 main:574] : INFO : Epoch 1957 | loss: 0.0311415 | val_loss: 0.031152 | Time: 3295.87 ms [2022-02-27 13:06:53 main:574] : INFO : Epoch 1958 | loss: 0.0311396 | val_loss: 0.0311495 | Time: 3351.12 ms [2022-02-27 13:06:56 main:574] : INFO : Epoch 1959 | loss: 0.0311401 | val_loss: 0.0311487 | Time: 3380.66 ms [2022-02-27 13:07:00 main:574] : INFO : Epoch 1960 | loss: 0.0311393 | val_loss: 0.0311578 | Time: 3454.92 ms [2022-02-27 13:07:03 main:574] : INFO : Epoch 1961 | loss: 0.031141 | val_loss: 0.0311487 | Time: 3346.33 ms [2022-02-27 13:07:06 main:574] : INFO : Epoch 1962 | loss: 0.0311392 | val_loss: 0.0311492 | Time: 3370.27 ms [2022-02-27 13:07:10 main:574] : INFO : Epoch 1963 | loss: 0.0311425 | val_loss: 0.0311542 | Time: 3455.36 ms [2022-02-27 13:07:13 main:574] : INFO : Epoch 1964 | loss: 0.0311412 | val_loss: 0.0311521 | Time: 3379.01 ms [2022-02-27 13:07:17 main:574] : INFO : Epoch 1965 | loss: 0.0311416 | val_loss: 0.0311509 | Time: 3440.69 ms [2022-02-27 13:07:20 main:574] : INFO : Epoch 1966 | loss: 0.0311419 | val_loss: 0.0311499 | Time: 3351.54 ms [2022-02-27 13:07:23 main:574] : INFO : Epoch 1967 | loss: 0.0311397 | val_loss: 0.0311552 | Time: 3292.14 ms [2022-02-27 13:07:27 main:574] : INFO : Epoch 1968 | loss: 0.031142 | val_loss: 0.0311493 | Time: 3424.34 ms [2022-02-27 13:07:30 main:574] : INFO : Epoch 1969 | loss: 0.0311394 | val_loss: 0.0311502 | Time: 3370.64 ms [2022-02-27 13:07:34 main:574] : INFO : Epoch 1970 | loss: 0.0311396 | val_loss: 0.0311512 | Time: 3497.68 ms [2022-02-27 13:07:37 main:574] : INFO : Epoch 1971 | loss: 0.0311392 | val_loss: 0.031149 | Time: 3258.86 ms [2022-02-27 13:07:41 main:574] : INFO : Epoch 1972 | loss: 0.0311404 | val_loss: 0.0311479 | Time: 3404.03 ms [2022-02-27 13:07:44 main:574] : INFO : Epoch 1973 | loss: 0.0311413 | val_loss: 0.0311507 | Time: 3446.31 ms [2022-02-27 13:07:47 main:574] : INFO : Epoch 1974 | loss: 0.0311402 | val_loss: 0.0311494 | Time: 3349.91 ms [2022-02-27 13:07:51 main:574] : INFO : Epoch 1975 | loss: 0.0311391 | val_loss: 0.0311464 | Time: 3331.77 ms [2022-02-27 13:07:54 main:574] : INFO : Epoch 1976 | loss: 0.0311378 | val_loss: 0.0311505 | Time: 3462.62 ms [2022-02-27 13:07:58 main:574] : INFO : Epoch 1977 | loss: 0.0311387 | val_loss: 0.0311474 | Time: 3332.13 ms [2022-02-27 13:08:01 main:574] : INFO : Epoch 1978 | loss: 0.0311378 | val_loss: 0.0311475 | Time: 3346.77 ms [2022-02-27 13:08:05 main:574] : INFO : Epoch 1979 | loss: 0.0311377 | val_loss: 0.0311481 | Time: 3428.74 ms [2022-02-27 13:08:08 main:574] : INFO : Epoch 1980 | loss: 0.0311372 | val_loss: 0.0311484 | Time: 3407.96 ms [2022-02-27 13:08:11 main:574] : INFO : Epoch 1981 | loss: 0.031138 | val_loss: 0.0311473 | Time: 3467.82 ms [2022-02-27 13:08:15 main:574] : INFO : Epoch 1982 | loss: 0.0311376 | val_loss: 0.0311495 | Time: 3340.62 ms [2022-02-27 13:08:18 main:574] : INFO : Epoch 1983 | loss: 0.0311381 | val_loss: 0.0311499 | Time: 3404.34 ms [2022-02-27 13:08:22 main:574] : INFO : Epoch 1984 | loss: 0.0311403 | val_loss: 0.031148 | Time: 3334.48 ms [2022-02-27 13:08:25 main:574] : INFO : Epoch 1985 | loss: 0.0311399 | val_loss: 0.0311461 | Time: 3365.01 ms [2022-02-27 13:08:29 main:574] : INFO : Epoch 1986 | loss: 0.031144 | val_loss: 0.0311485 | Time: 3423.94 ms [2022-02-27 13:08:32 main:574] : INFO : Epoch 1987 | loss: 0.0311457 | val_loss: 0.0311462 | Time: 3393.4 ms [2022-02-27 13:08:35 main:574] : INFO : Epoch 1988 | loss: 0.0311451 | val_loss: 0.0311474 | Time: 3361.05 ms [2022-02-27 13:08:39 main:574] : INFO : Epoch 1989 | loss: 0.0311423 | val_loss: 0.0311439 | Time: 3501.28 ms [2022-02-27 13:08:42 main:574] : INFO : Epoch 1990 | loss: 0.0311408 | val_loss: 0.0311433 | Time: 3411 ms [2022-02-27 13:08:46 main:574] : INFO : Epoch 1991 | loss: 0.0311416 | val_loss: 0.0311441 | Time: 3336.54 ms [2022-02-27 13:08:49 main:574] : INFO : Epoch 1992 | loss: 0.0311432 | val_loss: 0.0311569 | Time: 3313.94 ms [2022-02-27 13:08:53 main:574] : INFO : Epoch 1993 | loss: 0.0311438 | val_loss: 0.0311449 | Time: 3510.21 ms [2022-02-27 13:08:56 main:574] : INFO : Epoch 1994 | loss: 0.0311396 | val_loss: 0.0311463 | Time: 3401.01 ms [2022-02-27 13:08:59 main:574] : INFO : Epoch 1995 | loss: 0.0311391 | val_loss: 0.0311438 | Time: 3320.48 ms [2022-02-27 13:09:03 main:574] : INFO : Epoch 1996 | loss: 0.0311382 | val_loss: 0.0311435 | Time: 3370.11 ms [2022-02-27 13:09:06 main:574] : INFO : Epoch 1997 | loss: 0.0311388 | val_loss: 0.0311457 | Time: 3465.52 ms [2022-02-27 13:09:10 main:574] : INFO : Epoch 1998 | loss: 0.0311413 | val_loss: 0.0311487 | Time: 3360.68 ms [2022-02-27 13:09:13 main:574] : INFO : Epoch 1999 | loss: 0.0311422 | val_loss: 0.0311453 | Time: 3370.59 ms [2022-02-27 13:09:17 main:574] : INFO : Epoch 2000 | loss: 0.0311373 | val_loss: 0.0311448 | Time: 3406 ms [2022-02-27 13:09:20 main:574] : INFO : Epoch 2001 | loss: 0.0311383 | val_loss: 0.0311454 | Time: 3353.1 ms [2022-02-27 13:09:23 main:574] : INFO : Epoch 2002 | loss: 0.0311389 | val_loss: 0.031144 | Time: 3272.52 ms [2022-02-27 13:09:27 main:574] : INFO : Epoch 2003 | loss: 0.0311393 | val_loss: 0.0311457 | Time: 3437.16 ms [2022-02-27 13:09:30 main:574] : INFO : Epoch 2004 | loss: 0.031139 | val_loss: 0.0311467 | Time: 3342.51 ms [2022-02-27 13:09:34 main:574] : INFO : Epoch 2005 | loss: 0.0311395 | val_loss: 0.031148 | Time: 3388.4 ms [2022-02-27 13:09:37 main:574] : INFO : Epoch 2006 | loss: 0.0311407 | val_loss: 0.0311454 | Time: 3322.36 ms [2022-02-27 13:09:40 main:574] : INFO : Epoch 2007 | loss: 0.0311395 | val_loss: 0.0311448 | Time: 3456.88 ms [2022-02-27 13:09:44 main:574] : INFO : Epoch 2008 | loss: 0.0311396 | val_loss: 0.0311442 | Time: 3446.31 ms [2022-02-27 13:09:47 main:574] : INFO : Epoch 2009 | loss: 0.0311487 | val_loss: 0.0311642 | Time: 3411.52 ms [2022-02-27 13:09:51 main:574] : INFO : Epoch 2010 | loss: 0.0311631 | val_loss: 0.0311624 | Time: 3489.45 ms [2022-02-27 13:09:54 main:574] : INFO : Epoch 2011 | loss: 0.0311648 | val_loss: 0.0311605 | Time: 3330.05 ms [2022-02-27 13:09:58 main:574] : INFO : Epoch 2012 | loss: 0.0311608 | val_loss: 0.0311608 | Time: 3337.67 ms [2022-02-27 13:10:01 main:574] : INFO : Epoch 2013 | loss: 0.0311579 | val_loss: 0.0311586 | Time: 3344.11 ms [2022-02-27 13:10:05 main:574] : INFO : Epoch 2014 | loss: 0.0311567 | val_loss: 0.0311585 | Time: 3457.25 ms [2022-02-27 13:10:08 main:574] : INFO : Epoch 2015 | loss: 0.0311549 | val_loss: 0.0311563 | Time: 3524.83 ms [2022-02-27 13:10:12 main:574] : INFO : Epoch 2016 | loss: 0.031155 | val_loss: 0.0311563 | Time: 3406.25 ms [2022-02-27 13:10:15 main:574] : INFO : Epoch 2017 | loss: 0.0311528 | val_loss: 0.031156 | Time: 3416.79 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-02-27 13:13:33 main:435] : INFO : Set logging level to 1 [2022-02-27 13:13:33 main:441] : INFO : Running in BOINC Client mode [2022-02-27 13:13:33 main:444] : INFO : Resolving all filenames [2022-02-27 13:13:33 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 13:13:33 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 13:13:33 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 13:13:33 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 13:13:33 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 13:13:33 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 13:13:33 main:474] : INFO : Configuration: [2022-02-27 13:13:33 main:475] : INFO : Model type: GRU [2022-02-27 13:13:33 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 13:13:33 main:477] : INFO : Max Epochs: 2048 [2022-02-27 13:13:33 main:478] : INFO : Batch Size: 128 [2022-02-27 13:13:33 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 13:13:33 main:480] : INFO : Patience: 10 [2022-02-27 13:13:33 main:481] : INFO : Hidden Width: 12 [2022-02-27 13:13:33 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 13:13:33 main:483] : INFO : # Backend Layers: 4 [2022-02-27 13:13:33 main:484] : INFO : # Threads: 1 [2022-02-27 13:13:34 main:486] : INFO : Preparing Dataset [2022-02-27 13:13:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 13:13:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 13:13:35 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 13:13:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 13:13:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 13:13:36 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 13:13:36 main:494] : INFO : Creating Model [2022-02-27 13:13:36 main:507] : INFO : Preparing config file [2022-02-27 13:13:36 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 13:13:36 main:512] : INFO : Loading config [2022-02-27 13:13:36 main:514] : INFO : Loading state [2022-02-27 13:13:37 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 13:13:37 main:562] : INFO : Starting Training Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-02-27 14:22:48 main:435] : INFO : Set logging level to 1 [2022-02-27 14:22:48 main:441] : INFO : Running in BOINC Client mode [2022-02-27 14:22:48 main:444] : INFO : Resolving all filenames [2022-02-27 14:22:48 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-02-27 14:22:49 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-02-27 14:22:49 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-02-27 14:22:49 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-02-27 14:22:49 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-02-27 14:22:49 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-02-27 14:22:49 main:474] : INFO : Configuration: [2022-02-27 14:22:49 main:475] : INFO : Model type: GRU [2022-02-27 14:22:49 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-02-27 14:22:49 main:477] : INFO : Max Epochs: 2048 [2022-02-27 14:22:49 main:478] : INFO : Batch Size: 128 [2022-02-27 14:22:49 main:479] : INFO : Learning Rate: 0.01 [2022-02-27 14:22:49 main:480] : INFO : Patience: 10 [2022-02-27 14:22:49 main:481] : INFO : Hidden Width: 12 [2022-02-27 14:22:49 main:482] : INFO : # Recurrent Layers: 4 [2022-02-27 14:22:49 main:483] : INFO : # Backend Layers: 4 [2022-02-27 14:22:49 main:484] : INFO : # Threads: 1 [2022-02-27 14:22:49 main:486] : INFO : Preparing Dataset [2022-02-27 14:22:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-02-27 14:22:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-02-27 14:22:51 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-02-27 14:22:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-02-27 14:22:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-02-27 14:22:51 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-02-27 14:22:51 main:494] : INFO : Creating Model [2022-02-27 14:22:51 main:507] : INFO : Preparing config file [2022-02-27 14:22:51 main:511] : INFO : Found checkpoint, attempting to load... [2022-02-27 14:22:51 main:512] : INFO : Loading config [2022-02-27 14:22:51 main:514] : INFO : Loading state [2022-02-27 14:22:52 main:559] : INFO : Loading DataLoader into Memory [2022-02-27 14:22:52 main:562] : INFO : Starting Training [2022-02-27 14:22:57 main:574] : INFO : Epoch 1972 | loss: 0.0311733 | val_loss: 0.0311559 | Time: 4168.86 ms [2022-02-27 14:23:01 main:574] : INFO : Epoch 1973 | loss: 0.0311429 | val_loss: 0.0311489 | Time: 3906.14 ms [2022-02-27 14:23:05 main:574] : INFO : Epoch 1974 | loss: 0.0311398 | val_loss: 0.031148 | Time: 3888.62 ms [2022-02-27 14:23:09 main:574] : INFO : Epoch 1975 | loss: 0.0311386 | val_loss: 0.031148 | Time: 3935.85 ms [2022-02-27 14:23:13 main:574] : INFO : Epoch 1976 | loss: 0.0311397 | val_loss: 0.0311476 | Time: 3937.99 ms [2022-02-27 14:23:16 main:574] : INFO : Epoch 1977 | loss: 0.0311388 | val_loss: 0.031149 | Time: 3948.67 ms [2022-02-27 14:23:21 main:574] : INFO : Epoch 1978 | loss: 0.0311426 | val_loss: 0.0311587 | Time: 3989.01 ms [2022-02-27 14:23:24 main:574] : INFO : Epoch 1979 | loss: 0.0311404 | val_loss: 0.031149 | Time: 3835.42 ms [2022-02-27 14:23:28 main:574] : INFO : Epoch 1980 | loss: 0.031141 | val_loss: 0.0311515 | Time: 3920.89 ms [2022-02-27 14:23:32 main:574] : INFO : Epoch 1981 | loss: 0.0311474 | val_loss: 0.0311545 | Time: 3891.89 ms [2022-02-27 14:23:36 main:574] : INFO : Epoch 1982 | loss: 0.0311475 | val_loss: 0.0311504 | Time: 3956.83 ms [2022-02-27 14:23:40 main:574] : INFO : Epoch 1983 | loss: 0.0311472 | val_loss: 0.0311522 | Time: 3977.68 ms [2022-02-27 14:23:44 main:574] : INFO : Epoch 1984 | loss: 0.0311469 | val_loss: 0.0311496 | Time: 3903.53 ms [2022-02-27 14:23:48 main:574] : INFO : Epoch 1985 | loss: 0.0311454 | val_loss: 0.0311501 | Time: 4057.27 ms [2022-02-27 14:23:53 main:574] : INFO : Epoch 1986 | loss: 0.0311445 | val_loss: 0.0311529 | Time: 5007.86 ms [2022-02-27 14:23:58 main:574] : INFO : Epoch 1987 | loss: 0.03115 | val_loss: 0.0311563 | Time: 4065.93 ms [2022-02-27 14:24:02 main:574] : INFO : Epoch 1988 | loss: 0.0311515 | val_loss: 0.0311577 | Time: 4045.17 ms [2022-02-27 14:24:06 main:574] : INFO : Epoch 1989 | loss: 0.0311505 | val_loss: 0.0311608 | Time: 4259.55 ms [2022-02-27 14:24:10 main:574] : INFO : Epoch 1990 | loss: 0.0311502 | val_loss: 0.0311552 | Time: 4143.27 ms [2022-02-27 14:24:15 main:574] : INFO : Epoch 1991 | loss: 0.0311485 | val_loss: 0.0311539 | Time: 4844.21 ms [2022-02-27 14:24:20 main:574] : INFO : Epoch 1992 | loss: 0.0311467 | val_loss: 0.0311513 | Time: 4232.77 ms [2022-02-27 14:24:24 main:574] : INFO : Epoch 1993 | loss: 0.0311461 | val_loss: 0.0311522 | Time: 3856.05 ms [2022-02-27 14:24:29 main:574] : INFO : Epoch 1994 | loss: 0.0311457 | val_loss: 0.0311532 | Time: 3828.73 ms [2022-02-27 14:24:33 main:574] : INFO : Epoch 1995 | loss: 0.0311461 | val_loss: 0.031152 | Time: 3939.69 ms [2022-02-27 14:24:37 main:574] : INFO : Epoch 1996 | loss: 0.0311462 | val_loss: 0.0311577 | Time: 3947.15 ms [2022-02-27 14:24:42 main:574] : INFO : Epoch 1997 | loss: 0.031149 | val_loss: 0.0311564 | Time: 3902.91 ms [2022-02-27 14:24:46 main:574] : INFO : Epoch 1998 | loss: 0.0311486 | val_loss: 0.0311598 | Time: 3873.98 ms [2022-02-27 14:24:51 main:574] : INFO : Epoch 1999 | loss: 0.0311514 | val_loss: 0.0311581 | Time: 3954.62 ms [2022-02-27 14:24:55 main:574] : INFO : Epoch 2000 | loss: 0.0311468 | val_loss: 0.0311517 | Time: 4051.25 ms [2022-02-27 14:25:00 main:574] : INFO : Epoch 2001 | loss: 0.0311442 | val_loss: 0.0311513 | Time: 3843.38 ms [2022-02-27 14:25:04 main:574] : INFO : Epoch 2002 | loss: 0.031145 | val_loss: 0.0311519 | Time: 3937.93 ms [2022-02-27 14:25:09 main:574] : INFO : Epoch 2003 | loss: 0.0311442 | val_loss: 0.0311533 | Time: 3861.18 ms [2022-02-27 14:25:13 main:574] : INFO : Epoch 2004 | loss: 0.0311454 | val_loss: 0.031153 | Time: 3939.99 ms [2022-02-27 14:25:17 main:574] : INFO : Epoch 2005 | loss: 0.0311447 | val_loss: 0.0311525 | Time: 3893.84 ms [2022-02-27 14:25:22 main:574] : INFO : Epoch 2006 | loss: 0.0311449 | val_loss: 0.0311528 | Time: 3906.54 ms [2022-02-27 14:25:26 main:574] : INFO : Epoch 2007 | loss: 0.0311455 | val_loss: 0.0311519 | Time: 3851.27 ms [2022-02-27 14:25:30 main:574] : INFO : Epoch 2008 | loss: 0.0311451 | val_loss: 0.0311536 | Time: 3980.13 ms [2022-02-27 14:25:34 main:574] : INFO : Epoch 2009 | loss: 0.0311475 | val_loss: 0.0311573 | Time: 4021.9 ms [2022-02-27 14:25:38 main:574] : INFO : Epoch 2010 | loss: 0.0311477 | val_loss: 0.0311541 | Time: 3861.84 ms [2022-02-27 14:25:42 main:574] : INFO : Epoch 2011 | loss: 0.0311447 | val_loss: 0.0311569 | Time: 4147.39 ms [2022-02-27 14:25:46 main:574] : INFO : Epoch 2012 | loss: 0.0311456 | val_loss: 0.0311552 | Time: 3923.82 ms [2022-02-27 14:25:50 main:574] : INFO : Epoch 2013 | loss: 0.0311447 | val_loss: 0.031154 | Time: 3921.51 ms [2022-02-27 14:25:55 main:574] : INFO : Epoch 2014 | loss: 0.0311439 | val_loss: 0.0311564 | Time: 4070.95 ms [2022-02-27 14:25:59 main:574] : INFO : Epoch 2015 | loss: 0.0311439 | val_loss: 0.0311562 | Time: 3946.19 ms [2022-02-27 14:26:03 main:574] : INFO : Epoch 2016 | loss: 0.0311451 | val_loss: 0.0311567 | Time: 3938.32 ms [2022-02-27 14:26:06 main:574] : INFO : Epoch 2017 | loss: 0.0311438 | val_loss: 0.0311528 | Time: 3888.38 ms [2022-02-27 14:26:11 main:574] : INFO : Epoch 2018 | loss: 0.0311428 | val_loss: 0.0311522 | Time: 3917.74 ms [2022-02-27 14:26:14 main:574] : INFO : Epoch 2019 | loss: 0.0311423 | val_loss: 0.0311524 | Time: 3861.64 ms [2022-02-27 14:26:18 main:574] : INFO : Epoch 2020 | loss: 0.0311425 | val_loss: 0.0311533 | Time: 3907.14 ms [2022-02-27 14:26:22 main:574] : INFO : Epoch 2021 | loss: 0.0311448 | val_loss: 0.0311553 | Time: 3987.5 ms [2022-02-27 14:26:26 main:574] : INFO : Epoch 2022 | loss: 0.031144 | val_loss: 0.031152 | Time: 3936.68 ms [2022-02-27 14:26:30 main:574] : INFO : Epoch 2023 | loss: 0.0311421 | val_loss: 0.031154 | Time: 3931.04 ms [2022-02-27 14:26:34 main:574] : INFO : Epoch 2024 | loss: 0.0311429 | val_loss: 0.0311533 | Time: 3808.73 ms [2022-02-27 14:26:38 main:574] : INFO : Epoch 2025 | loss: 0.0311428 | val_loss: 0.0311569 | Time: 3898.95 ms [2022-02-27 14:26:42 main:574] : INFO : Epoch 2026 | loss: 0.0311444 | val_loss: 0.0311565 | Time: 3993.96 ms [2022-02-27 14:26:46 main:574] : INFO : Epoch 2027 | loss: 0.0311431 | val_loss: 0.0311536 | Time: 3974.09 ms [2022-02-27 14:26:50 main:574] : INFO : Epoch 2028 | loss: 0.0311421 | val_loss: 0.0311537 | Time: 3922.74 ms [2022-02-27 14:26:54 main:574] : INFO : Epoch 2029 | loss: 0.0311407 | val_loss: 0.0311526 | Time: 4186.31 ms [2022-02-27 14:26:58 main:574] : INFO : Epoch 2030 | loss: 0.0311396 | val_loss: 0.0311527 | Time: 3890.2 ms [2022-02-27 14:27:02 main:574] : INFO : Epoch 2031 | loss: 0.0311398 | val_loss: 0.0311533 | Time: 3953.75 ms [2022-02-27 14:27:06 main:574] : INFO : Epoch 2032 | loss: 0.0311402 | val_loss: 0.0311515 | Time: 3873.51 ms [2022-02-27 14:27:10 main:574] : INFO : Epoch 2033 | loss: 0.0311385 | val_loss: 0.0311519 | Time: 3814.24 ms [2022-02-27 14:27:14 main:574] : INFO : Epoch 2034 | loss: 0.0311386 | val_loss: 0.0311507 | Time: 3932.49 ms [2022-02-27 14:27:18 main:574] : INFO : Epoch 2035 | loss: 0.0311398 | val_loss: 0.0311522 | Time: 3984.2 ms [2022-02-27 14:27:22 main:574] : INFO : Epoch 2036 | loss: 0.0311423 | val_loss: 0.0311508 | Time: 3936.46 ms [2022-02-27 14:27:26 main:574] : INFO : Epoch 2037 | loss: 0.0311402 | val_loss: 0.0311505 | Time: 3952.97 ms [2022-02-27 14:27:30 main:574] : INFO : Epoch 2038 | loss: 0.031143 | val_loss: 0.0311494 | Time: 3927.96 ms [2022-02-27 14:27:34 main:574] : INFO : Epoch 2039 | loss: 0.0311393 | val_loss: 0.0311521 | Time: 3951.81 ms [2022-02-27 14:27:38 main:574] : INFO : Epoch 2040 | loss: 0.0311394 | val_loss: 0.0311517 | Time: 3896.69 ms [2022-02-27 14:27:42 main:574] : INFO : Epoch 2041 | loss: 0.0311391 | val_loss: 0.0311505 | Time: 3982.41 ms [2022-02-27 14:27:46 main:574] : INFO : Epoch 2042 | loss: 0.031141 | val_loss: 0.0311501 | Time: 3807.35 ms [2022-02-27 14:27:50 main:574] : INFO : Epoch 2043 | loss: 0.0311375 | val_loss: 0.0311513 | Time: 3987.32 ms [2022-02-27 14:27:54 main:574] : INFO : Epoch 2044 | loss: 0.0311372 | val_loss: 0.0311504 | Time: 4226.76 ms [2022-02-27 14:27:58 main:574] : INFO : Epoch 2045 | loss: 0.031137 | val_loss: 0.0311524 | Time: 3892.29 ms [2022-02-27 14:28:02 main:574] : INFO : Epoch 2046 | loss: 0.0311371 | val_loss: 0.031152 | Time: 3914.16 ms [2022-02-27 14:28:06 main:574] : INFO : Epoch 2047 | loss: 0.0311396 | val_loss: 0.031151 | Time: 3905.75 ms [2022-02-27 14:28:10 main:574] : INFO : Epoch 2048 | loss: 0.0311393 | val_loss: 0.0311508 | Time: 3899.28 ms [2022-02-27 14:28:10 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311508 [2022-02-27 14:28:10 main:603] : INFO : Saving end state to config to file [2022-02-27 14:28:10 main:608] : INFO : Success, exiting.. 14:28:10 (9064): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)