Task 12987442
| Name | ParityModified-1645998043-13458-3-0_0 |
| Workunit | 10118343 |
| Created | 12 Mar 2022, 7:40:12 UTC |
| Sent | 20 Mar 2022, 17:41:21 UTC |
| Report deadline | 28 Mar 2022, 17:41:21 UTC |
| Received | 24 Mar 2022, 18:03:47 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 8158 |
| Run time | 3 hours 11 min 6 sec |
| CPU time | 3 hours 0 min 48 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 13,453.15 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.42 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 3 22:22:25 main:574] : INFO : Epoch 1664 | loss: 0.0311325 | val_loss: 0.0311589 | Time: 3118.59 ms [2022-03-23 22:22:28 main:574] : INFO : Epoch 1665 | loss: 0.0311309 | val_loss: 0.031162 | Time: 3102.42 ms [2022-03-23 22:22:31 main:574] : INFO : Epoch 1666 | loss: 0.0311289 | val_loss: 0.0311571 | Time: 3101.7 ms [2022-03-23 22:22:34 main:574] : INFO : Epoch 1667 | loss: 0.0311286 | val_loss: 0.0311553 | Time: 2985.44 ms [2022-03-23 22:22:38 main:574] : INFO : Epoch 1668 | loss: 0.0311291 | val_loss: 0.031164 | Time: 3317.5 ms [2022-03-23 22:22:41 main:574] : INFO : Epoch 1669 | loss: 0.031132 | val_loss: 0.0311593 | Time: 3215.45 ms [2022-03-23 22:22:44 main:574] : INFO : Epoch 1670 | loss: 0.0311312 | val_loss: 0.0311574 | Time: 3108.04 ms [2022-03-23 22:22:47 main:574] : INFO : Epoch 1671 | loss: 0.0311291 | val_loss: 0.031164 | Time: 3169.43 ms [2022-03-23 22:22:51 main:574] : INFO : Epoch 1672 | loss: 0.0311279 | val_loss: 0.0311612 | Time: 3382.62 ms [2022-03-23 22:22:54 main:574] : INFO : Epoch 1673 | loss: 0.0311286 | val_loss: 0.031163 | Time: 3290.1 ms [2022-03-23 22:22:57 main:574] : INFO : Epoch 1674 | loss: 0.0311297 | val_loss: 0.0311592 | Time: 3136.16 ms [2022-03-23 22:23:00 main:574] : INFO : Epoch 1675 | loss: 0.0311317 | val_loss: 0.0311567 | Time: 3077.08 ms [2022-03-23 22:23:03 main:574] : INFO : Epoch 1676 | loss: 0.0311312 | val_loss: 0.0311632 | Time: 3100.17 ms [2022-03-23 22:23:07 main:574] : INFO : Epoch 1677 | loss: 0.0311307 | val_loss: 0.0311603 | Time: 3087.04 ms [2022-03-23 22:23:10 main:574] : INFO : Epoch 1678 | loss: 0.0311305 | val_loss: 0.0311599 | Time: 3162.23 ms [2022-03-23 22:23:13 main:574] : INFO : Epoch 1679 | loss: 0.0311297 | val_loss: 0.0311616 | Time: 3141.27 ms [2022-03-23 22:23:16 main:574] : INFO : Epoch 1680 | loss: 0.0311316 | val_loss: 0.0311695 | Time: 3111.77 ms [2022-03-23 22:23:19 main:574] : INFO : Epoch 1681 | loss: 0.0311319 | val_loss: 0.0311585 | Time: 3115.45 ms [2022-03-23 22:23:22 main:574] : INFO : Epoch 1682 | loss: 0.0311297 | val_loss: 0.0311603 | Time: 3029.32 ms [2022-03-23 22:23:26 main:574] : INFO : Epoch 1683 | loss: 0.0311358 | val_loss: 0.0311653 | Time: 3293.79 ms [2022-03-23 22:23:29 main:574] : INFO : Epoch 1684 | loss: 0.0311496 | val_loss: 0.0311752 | Time: 2984.73 ms [2022-03-23 22:23:32 main:574] : INFO : Epoch 1685 | loss: 0.0311449 | val_loss: 0.0311641 | Time: 3138.98 ms [2022-03-23 22:23:35 main:574] : INFO : Epoch 1686 | loss: 0.0311397 | val_loss: 0.0311627 | Time: 3013.55 ms [2022-03-23 22:23:38 main:574] : INFO : Epoch 1687 | loss: 0.031136 | val_loss: 0.0311682 | Time: 3209.76 ms [2022-03-23 22:23:41 main:574] : INFO : Epoch 1688 | loss: 0.0311383 | val_loss: 0.0311603 | Time: 3133.25 ms [2022-03-23 22:23:44 main:574] : INFO : Epoch 1689 | loss: 0.0311434 | val_loss: 0.0311584 | Time: 3118.11 ms [2022-03-23 22:23:48 main:574] : INFO : Epoch 1690 | loss: 0.0311448 | val_loss: 0.0311592 | Time: 3226.83 ms [2022-03-23 22:23:51 main:574] : INFO : Epoch 1691 | loss: 0.031144 | val_loss: 0.0311568 | Time: 3141.32 ms [2022-03-23 22:23:54 main:574] : INFO : Epoch 1692 | loss: 0.0311422 | val_loss: 0.031159 | Time: 3231.8 ms [2022-03-23 22:23:57 main:574] : INFO : Epoch 1693 | loss: 0.0311407 | val_loss: 0.0311612 | Time: 3146.16 ms [2022-03-23 22:24:01 main:574] : INFO : Epoch 1694 | loss: 0.0311392 | val_loss: 0.0311601 | Time: 3206.58 ms [2022-03-23 22:24:04 main:574] : INFO : Epoch 1695 | loss: 0.0311382 | val_loss: 0.0311604 | Time: 3097.93 ms [2022-03-23 22:24:07 main:574] : INFO : Epoch 1696 | loss: 0.0311383 | val_loss: 0.0311586 | Time: 3107.48 ms [2022-03-23 22:24:10 main:574] : INFO : Epoch 1697 | loss: 0.0311365 | val_loss: 0.0311568 | Time: 3109.47 ms [2022-03-23 22:24:13 main:574] : INFO : Epoch 1698 | loss: 0.0311346 | val_loss: 0.0311594 | Time: 3156.5 ms [2022-03-23 22:24:16 main:574] : INFO : Epoch 1699 | loss: 0.0311348 | val_loss: 0.0311592 | Time: 3183.77 ms [2022-03-23 22:24:20 main:574] : INFO : Epoch 1700 | loss: 0.0311349 | val_loss: 0.0311571 | Time: 3180.23 ms [2022-03-23 22:24:23 main:574] : INFO : Epoch 1701 | loss: 0.0311367 | val_loss: 0.0311631 | Time: 3071.85 ms [2022-03-23 22:24:26 main:574] : INFO : Epoch 1702 | loss: 0.0311347 | val_loss: 0.0311593 | Time: 3104.98 ms [2022-03-23 22:24:29 main:574] : INFO : Epoch 1703 | loss: 0.0311336 | val_loss: 0.0311606 | Time: 3259.54 ms [2022-03-23 22:24:33 main:574] : INFO : Epoch 1704 | loss: 0.0311344 | val_loss: 0.0311575 | Time: 3235.68 ms [2022-03-23 22:24:36 main:574] : INFO : Epoch 1705 | loss: 0.0311342 | val_loss: 0.0311548 | Time: 3127.13 ms [2022-03-23 22:24:39 main:574] : INFO : Epoch 1706 | loss: 0.0311335 | val_loss: 0.0311603 | Time: 3325.08 ms [2022-03-23 22:24:42 main:574] : INFO : Epoch 1707 | loss: 0.0311338 | val_loss: 0.0311654 | Time: 3109.08 ms [2022-03-23 22:24:45 main:574] : INFO : Epoch 1708 | loss: 0.031134 | val_loss: 0.0311663 | Time: 3058.27 ms [2022-03-23 22:24:48 main:574] : INFO : Epoch 1709 | loss: 0.0311336 | val_loss: 0.0311602 | Time: 3047.18 ms [2022-03-23 22:24:51 main:574] : INFO : Epoch 1710 | loss: 0.0311315 | val_loss: 0.0311592 | Time: 3052.11 ms [2022-03-23 22:24:55 main:574] : INFO : Epoch 1711 | loss: 0.0311321 | val_loss: 0.0311611 | Time: 3143.49 ms [2022-03-23 22:24:58 main:574] : INFO : Epoch 1712 | loss: 0.0311313 | val_loss: 0.0311593 | Time: 3162.91 ms [2022-03-23 22:25:01 main:574] : INFO : Epoch 1713 | loss: 0.0311306 | val_loss: 0.0311636 | Time: 3090.4 ms [2022-03-23 22:25:04 main:574] : INFO : Epoch 1714 | loss: 0.0311292 | val_loss: 0.0311616 | Time: 3044.5 ms [2022-03-23 22:25:07 main:574] : INFO : Epoch 1715 | loss: 0.03113 | val_loss: 0.0311628 | Time: 3089.36 ms [2022-03-23 22:25:10 main:574] : INFO : Epoch 1716 | loss: 0.0311316 | val_loss: 0.0311596 | Time: 3307.45 ms [2022-03-23 22:25:14 main:574] : INFO : Epoch 1717 | loss: 0.0311313 | val_loss: 0.0311614 | Time: 3236.09 ms [2022-03-23 22:25:17 main:574] : INFO : Epoch 1718 | loss: 0.0311293 | val_loss: 0.0311609 | Time: 3243.91 ms [2022-03-23 22:25:20 main:574] : INFO : Epoch 1719 | loss: 0.0311306 | val_loss: 0.0311585 | Time: 2962.8 ms [2022-03-23 22:25:23 main:574] : INFO : Epoch 1720 | loss: 0.0311303 | val_loss: 0.0311604 | Time: 3124.06 ms [2022-03-23 22:25:27 main:574] : INFO : Epoch 1721 | loss: 0.0311283 | val_loss: 0.0311596 | Time: 3266.06 ms [2022-03-23 22:25:30 main:574] : INFO : Epoch 1722 | loss: 0.0311386 | val_loss: 0.0311723 | Time: 3000.4 ms [2022-03-23 22:25:33 main:574] : INFO : Epoch 1723 | loss: 0.0311518 | val_loss: 0.0311639 | Time: 3028.49 ms [2022-03-23 22:25:36 main:574] : INFO : Epoch 1724 | loss: 0.0311469 | val_loss: 0.0311608 | Time: 3012.01 ms [2022-03-23 22:25:39 main:574] : INFO : Epoch 1725 | loss: 0.0311465 | val_loss: 0.0311646 | Time: 3037.89 ms [2022-03-23 22:25:42 main:574] : INFO : Epoch 1726 | loss: 0.0311424 | val_loss: 0.031163 | Time: 3114.14 ms [2022-03-23 22:25:45 main:574] : INFO : Epoch 1727 | loss: 0.0311406 | val_loss: 0.031161 | Time: 3508.36 ms [2022-03-23 22:25:49 main:574] : INFO : Epoch 1728 | loss: 0.0311384 | val_loss: 0.0311642 | Time: 3153.72 ms [2022-03-23 22:25:52 main:574] : INFO : Epoch 1729 | loss: 0.0311375 | val_loss: 0.031163 | Time: 3014.2 ms [2022-03-23 22:25:55 main:574] : INFO : Epoch 1730 | loss: 0.0311343 | val_loss: 0.031159 | Time: 3026.3 ms [2022-03-23 22:25:58 main:574] : INFO : Epoch 1731 | loss: 0.0311366 | val_loss: 0.0311637 | Time: 3145.81 ms [2022-03-23 22:26:01 main:574] : INFO : Epoch 1732 | loss: 0.0311414 | val_loss: 0.0311608 | Time: 3071.73 ms [2022-03-23 22:26:04 main:574] : INFO : Epoch 1733 | loss: 0.0320325 | val_loss: 0.0327777 | Time: 3294.91 ms [2022-03-23 22:26:08 main:574] : INFO : Epoch 1734 | loss: 0.0347149 | val_loss: 0.148061 | Time: 3302.94 ms [2022-03-23 22:26:11 main:574] : INFO : Epoch 1735 | loss: 0.0434134 | val_loss: 0.0321407 | Time: 3194.04 ms [2022-03-23 22:26:14 main:574] : INFO : Epoch 1736 | loss: 0.0319907 | val_loss: 0.0315567 | Time: 3138.06 ms [2022-03-23 22:26:17 main:574] : INFO : Epoch 1737 | loss: 0.0313759 | val_loss: 0.0312822 | Time: 3161.29 ms [2022-03-23 22:26:21 main:574] : INFO : Epoch 1738 | loss: 0.0312482 | val_loss: 0.0312205 | Time: 3248.14 ms [2022-03-23 22:26:24 main:574] : INFO : Epoch 1739 | loss: 0.031218 | val_loss: 0.0312126 | Time: 3071.31 ms [2022-03-23 22:26:27 main:574] : INFO : Epoch 1740 | loss: 0.0312131 | val_loss: 0.031211 | Time: 2969.49 ms [2022-03-23 22:26:30 main:574] : INFO : Epoch 1741 | loss: 0.031209 | val_loss: 0.0312057 | Time: 3109.41 ms [2022-03-23 22:26:33 main:574] : INFO : Epoch 1742 | loss: 0.0312051 | val_loss: 0.0312047 | Time: 3190.07 ms [2022-03-23 22:26:36 main:574] : INFO : Epoch 1743 | loss: 0.0312038 | val_loss: 0.0311912 | Time: 3088.2 ms [2022-03-23 22:26:39 main:574] : INFO : Epoch 1744 | loss: 0.0311912 | val_loss: 0.031186 | Time: 3096.56 ms [2022-03-23 22:26:43 main:574] : INFO : Epoch 1745 | loss: 0.0311855 | val_loss: 0.0311836 | Time: 3135.86 ms [2022-03-23 22:26:46 main:574] : INFO : Epoch 1746 | loss: 0.0311838 | val_loss: 0.0311823 | Time: 3277.99 ms [2022-03-23 22:26:49 main:574] : INFO : Epoch 1747 | loss: 0.0311823 | val_loss: 0.0311807 | Time: 3274.06 ms [2022-03-23 22:26:52 main:574] : INFO : Epoch 1748 | loss: 0.0311812 | val_loss: 0.0311797 | Time: 3120.06 ms [2022-03-23 22:26:56 main:574] : INFO : Epoch 1749 | loss: 0.0311789 | val_loss: 0.0311771 | Time: 3164.29 ms [2022-03-23 22:26:59 main:574] : INFO : Epoch 1750 | loss: 0.0311769 | val_loss: 0.0311761 | Time: 3016.47 ms [2022-03-23 22:27:02 main:574] : INFO : Epoch 1751 | loss: 0.0311762 | val_loss: 0.0311758 | Time: 3114.51 ms [2022-03-23 22:27:05 main:574] : INFO : Epoch 1752 | loss: 0.031176 | val_loss: 0.0311767 | Time: 3201.78 ms [2022-03-23 22:27:08 main:574] : INFO : Epoch 1753 | loss: 0.031176 | val_loss: 0.0311741 | Time: 3176.17 ms [2022-03-23 22:27:12 main:574] : INFO : Epoch 1754 | loss: 0.0311747 | val_loss: 0.0311739 | Time: 3297.24 ms [2022-03-23 22:27:15 main:574] : INFO : Epoch 1755 | loss: 0.0311742 | val_loss: 0.0311743 | Time: 3034.62 ms [2022-03-23 22:27:18 main:574] : INFO : Epoch 1756 | loss: 0.0311742 | val_loss: 0.0311737 | Time: 3082.58 ms [2022-03-23 22:27:21 main:574] : INFO : Epoch 1757 | loss: 0.0311733 | val_loss: 0.0311727 | Time: 3156.77 ms [2022-03-23 22:27:24 main:574] : INFO : Epoch 1758 | loss: 0.0311727 | val_loss: 0.0311723 | Time: 3083.41 ms [2022-03-23 22:27:27 main:574] : INFO : Epoch 1759 | loss: 0.0311716 | val_loss: 0.0311719 | Time: 3113.72 ms [2022-03-23 22:27:30 main:574] : INFO : Epoch 1760 | loss: 0.0311711 | val_loss: 0.0311712 | Time: 3125.83 ms [2022-03-23 22:27:34 main:574] : INFO : Epoch 1761 | loss: 0.0311703 | val_loss: 0.0311716 | Time: 3140.78 ms [2022-03-23 22:27:37 main:574] : INFO : Epoch 1762 | loss: 0.031169 | val_loss: 0.0311698 | Time: 3237.84 ms [2022-03-23 22:27:40 main:574] : INFO : Epoch 1763 | loss: 0.0311682 | val_loss: 0.0311695 | Time: 3247.7 ms [2022-03-23 22:27:43 main:574] : INFO : Epoch 1764 | loss: 0.0311674 | val_loss: 0.0311687 | Time: 3155.5 ms [2022-03-23 22:27:46 main:574] : INFO : Epoch 1765 | loss: 0.0311672 | val_loss: 0.0311685 | Time: 3078.78 ms [2022-03-23 22:27:49 main:574] : INFO : Epoch 1766 | loss: 0.0311663 | val_loss: 0.0311683 | Time: 3033.28 ms [2022-03-23 22:27:53 main:574] : INFO : Epoch 1767 | loss: 0.0311664 | val_loss: 0.0311689 | Time: 3171.77 ms [2022-03-23 22:27:56 main:574] : INFO : Epoch 1768 | loss: 0.0311665 | val_loss: 0.0311683 | Time: 3135.28 ms [2022-03-23 22:28:00 main:574] : INFO : Epoch 1769 | loss: 0.031167 | val_loss: 0.0311682 | Time: 3743.34 ms [2022-03-23 22:28:04 main:574] : INFO : Epoch 1770 | loss: 0.0311665 | val_loss: 0.0311675 | Time: 3880.39 ms [2022-03-23 22:28:07 main:574] : INFO : Epoch 1771 | loss: 0.0311659 | val_loss: 0.0311687 | Time: 3738.01 ms [2022-03-23 22:28:12 main:574] : INFO : Epoch 1772 | loss: 0.0311664 | val_loss: 0.0311679 | Time: 4195.48 ms [2022-03-23 22:28:17 main:574] : INFO : Epoch 1773 | loss: 0.0311639 | val_loss: 0.0311654 | Time: 4946.26 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-03-23 22:31:20 main:435] : INFO : Set logging level to 1 [2022-03-23 22:31:20 main:441] : INFO : Running in BOINC Client mode [2022-03-23 22:31:20 main:444] : INFO : Resolving all filenames [2022-03-23 22:31:20 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 22:31:20 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 22:31:20 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 22:31:20 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 22:31:20 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 22:31:20 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 22:31:20 main:474] : INFO : Configuration: [2022-03-23 22:31:20 main:475] : INFO : Model type: GRU [2022-03-23 22:31:20 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 22:31:20 main:477] : INFO : Max Epochs: 2048 [2022-03-23 22:31:20 main:478] : INFO : Batch Size: 128 [2022-03-23 22:31:20 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 22:31:20 main:480] : INFO : Patience: 10 [2022-03-23 22:31:21 main:481] : INFO : Hidden Width: 12 [2022-03-23 22:31:21 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 22:31:21 main:483] : INFO : # Backend Layers: 4 [2022-03-23 22:31:21 main:484] : INFO : # Threads: 1 [2022-03-23 22:31:21 main:486] : INFO : Preparing Dataset [2022-03-23 22:31:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 22:31:21 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-23 22:31:22 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-23 22:31:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-23 22:31:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-23 22:31:22 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-23 22:31:22 main:494] : INFO : Creating Model [2022-03-23 22:31:22 main:507] : INFO : Preparing config file [2022-03-23 22:31:22 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-23 22:31:22 main:512] : INFO : Loading config [2022-03-23 22:31:22 main:514] : INFO : Loading state [2022-03-23 22:31:23 main:559] : INFO : Loading DataLoader into Memory [2022-03-23 22:31:23 main:562] : INFO : Starting Training [2022-03-23 22:31:26 main:574] : INFO : Epoch 1751 | loss: 0.0319015 | val_loss: 0.0312608 | Time: 3327.57 ms [2022-03-23 22:31:30 main:574] : INFO : Epoch 1752 | loss: 0.0312415 | val_loss: 0.0312093 | Time: 3185.96 ms [2022-03-23 22:31:33 main:574] : INFO : Epoch 1753 | loss: 0.0311983 | val_loss: 0.0311837 | Time: 3279.36 ms [2022-03-23 22:31:36 main:574] : INFO : Epoch 1754 | loss: 0.0311827 | val_loss: 0.0311749 | Time: 3071.07 ms [2022-03-23 22:31:39 main:574] : INFO : Epoch 1755 | loss: 0.0311763 | val_loss: 0.0311738 | Time: 3162.58 ms [2022-03-23 22:31:42 main:574] : INFO : Epoch 1756 | loss: 0.0311762 | val_loss: 0.0311767 | Time: 3150.25 ms [2022-03-23 22:31:45 main:574] : INFO : Epoch 1757 | loss: 0.031174 | val_loss: 0.0311782 | Time: 2962.07 ms [2022-03-23 22:31:49 main:574] : INFO : Epoch 1758 | loss: 0.0311725 | val_loss: 0.0311678 | Time: 3189.76 ms [2022-03-23 22:31:52 main:574] : INFO : Epoch 1759 | loss: 0.0311646 | val_loss: 0.0311681 | Time: 3190.35 ms [2022-03-23 22:31:55 main:574] : INFO : Epoch 1760 | loss: 0.0311629 | val_loss: 0.0311701 | Time: 3105.36 ms [2022-03-23 22:31:58 main:574] : INFO : Epoch 1761 | loss: 0.0311619 | val_loss: 0.0311633 | Time: 3240.48 ms [2022-03-23 22:32:02 main:574] : INFO : Epoch 1762 | loss: 0.031161 | val_loss: 0.0311639 | Time: 3082.67 ms [2022-03-23 22:32:05 main:574] : INFO : Epoch 1763 | loss: 0.0311561 | val_loss: 0.0311594 | Time: 3048.52 ms [2022-03-23 22:32:08 main:574] : INFO : Epoch 1764 | loss: 0.0311557 | val_loss: 0.0311616 | Time: 3266.21 ms [2022-03-23 22:32:11 main:574] : INFO : Epoch 1765 | loss: 0.0311547 | val_loss: 0.0311598 | Time: 3047.3 ms [2022-03-23 22:32:14 main:574] : INFO : Epoch 1766 | loss: 0.0311544 | val_loss: 0.0311583 | Time: 3125.83 ms [2022-03-23 22:32:17 main:574] : INFO : Epoch 1767 | loss: 0.0311541 | val_loss: 0.0311576 | Time: 3131.18 ms [2022-03-23 22:32:20 main:574] : INFO : Epoch 1768 | loss: 0.0311523 | val_loss: 0.0311575 | Time: 3086.24 ms [2022-03-23 22:32:24 main:574] : INFO : Epoch 1769 | loss: 0.0311521 | val_loss: 0.0311628 | Time: 3160.79 ms [2022-03-23 22:32:27 main:574] : INFO : Epoch 1770 | loss: 0.0311531 | val_loss: 0.0311557 | Time: 3246.39 ms [2022-03-23 22:32:30 main:574] : INFO : Epoch 1771 | loss: 0.0311502 | val_loss: 0.0311574 | Time: 3096.35 ms [2022-03-23 22:32:33 main:574] : INFO : Epoch 1772 | loss: 0.0311512 | val_loss: 0.0311584 | Time: 3071.96 ms [2022-03-23 22:32:37 main:574] : INFO : Epoch 1773 | loss: 0.0311513 | val_loss: 0.0311567 | Time: 3311.68 ms [2022-03-23 22:32:40 main:574] : INFO : Epoch 1774 | loss: 0.0311517 | val_loss: 0.0311572 | Time: 3348.75 ms [2022-03-23 22:32:43 main:574] : INFO : Epoch 1775 | loss: 0.0311507 | val_loss: 0.0311556 | Time: 3095.28 ms [2022-03-23 22:32:46 main:574] : INFO : Epoch 1776 | loss: 0.0311514 | val_loss: 0.0311562 | Time: 3114.49 ms [2022-03-23 22:32:49 main:574] : INFO : Epoch 1777 | loss: 0.0311528 | val_loss: 0.0311557 | Time: 3123.44 ms [2022-03-23 22:32:52 main:574] : INFO : Epoch 1778 | loss: 0.0311544 | val_loss: 0.0311627 | Time: 3101.77 ms [2022-03-23 22:32:56 main:574] : INFO : Epoch 1779 | loss: 0.0311527 | val_loss: 0.0311589 | Time: 3176.11 ms [2022-03-23 22:32:59 main:574] : INFO : Epoch 1780 | loss: 0.0311504 | val_loss: 0.031157 | Time: 3063.01 ms [2022-03-23 22:33:02 main:574] : INFO : Epoch 1781 | loss: 0.03115 | val_loss: 0.0311566 | Time: 3167.81 ms [2022-03-23 22:33:05 main:574] : INFO : Epoch 1782 | loss: 0.0311482 | val_loss: 0.0311526 | Time: 3030.1 ms [2022-03-23 22:33:08 main:574] : INFO : Epoch 1783 | loss: 0.0311475 | val_loss: 0.0311534 | Time: 3117.6 ms [2022-03-23 22:33:12 main:574] : INFO : Epoch 1784 | loss: 0.0311492 | val_loss: 0.0311578 | Time: 3293.19 ms [2022-03-23 22:33:15 main:574] : INFO : Epoch 1785 | loss: 0.0311471 | val_loss: 0.0311545 | Time: 3048.22 ms [2022-03-23 22:33:18 main:574] : INFO : Epoch 1786 | loss: 0.0311467 | val_loss: 0.0311561 | Time: 3058.36 ms [2022-03-23 22:33:21 main:574] : INFO : Epoch 1787 | loss: 0.0311444 | val_loss: 0.0311511 | Time: 3151.35 ms [2022-03-23 22:33:24 main:574] : INFO : Epoch 1788 | loss: 0.0311401 | val_loss: 0.03115 | Time: 3331.68 ms [2022-03-23 22:33:27 main:574] : INFO : Epoch 1789 | loss: 0.0311402 | val_loss: 0.0311533 | Time: 3090.87 ms [2022-03-23 22:33:31 main:574] : INFO : Epoch 1790 | loss: 0.0311433 | val_loss: 0.031156 | Time: 3348.98 ms [2022-03-23 22:33:34 main:574] : INFO : Epoch 1791 | loss: 0.0311451 | val_loss: 0.0311553 | Time: 3121.43 ms [2022-03-23 22:33:37 main:574] : INFO : Epoch 1792 | loss: 0.0311431 | val_loss: 0.0311583 | Time: 2990.16 ms [2022-03-23 22:33:40 main:574] : INFO : Epoch 1793 | loss: 0.0311427 | val_loss: 0.0311529 | Time: 3275.14 ms [2022-03-23 22:33:44 main:574] : INFO : Epoch 1794 | loss: 0.03114 | val_loss: 0.031153 | Time: 3161.15 ms [2022-03-23 22:33:47 main:574] : INFO : Epoch 1795 | loss: 0.0311395 | val_loss: 0.0311554 | Time: 3048 ms [2022-03-23 22:33:50 main:574] : INFO : Epoch 1796 | loss: 0.0311407 | val_loss: 0.0311551 | Time: 3011.22 ms [2022-03-23 22:33:53 main:574] : INFO : Epoch 1797 | loss: 0.0311418 | val_loss: 0.0311507 | Time: 3005.77 ms [2022-03-23 22:33:56 main:574] : INFO : Epoch 1798 | loss: 0.0311425 | val_loss: 0.0311527 | Time: 3139.15 ms [2022-03-23 22:33:59 main:574] : INFO : Epoch 1799 | loss: 0.0311428 | val_loss: 0.0311543 | Time: 3405.27 ms [2022-03-23 22:34:03 main:574] : INFO : Epoch 1800 | loss: 0.0311456 | val_loss: 0.0311493 | Time: 3259.6 ms [2022-03-23 22:34:06 main:574] : INFO : Epoch 1801 | loss: 0.0311416 | val_loss: 0.0311514 | Time: 3208.94 ms [2022-03-23 22:34:09 main:574] : INFO : Epoch 1802 | loss: 0.0311389 | val_loss: 0.0311518 | Time: 3104.22 ms [2022-03-23 22:34:12 main:574] : INFO : Epoch 1803 | loss: 0.0311448 | val_loss: 0.0311524 | Time: 3047.57 ms [2022-03-23 22:34:15 main:574] : INFO : Epoch 1804 | loss: 0.0311462 | val_loss: 0.0311605 | Time: 3208.03 ms [2022-03-23 22:34:18 main:574] : INFO : Epoch 1805 | loss: 0.0311583 | val_loss: 0.0311548 | Time: 3018.2 ms [2022-03-23 22:34:21 main:574] : INFO : Epoch 1806 | loss: 0.0311518 | val_loss: 0.0311571 | Time: 2898.97 ms [2022-03-23 22:34:24 main:574] : INFO : Epoch 1807 | loss: 0.0311521 | val_loss: 0.0311575 | Time: 3066.85 ms [2022-03-23 22:34:28 main:574] : INFO : Epoch 1808 | loss: 0.0311522 | val_loss: 0.0311548 | Time: 3134.62 ms [2022-03-23 22:34:31 main:574] : INFO : Epoch 1809 | loss: 0.0311496 | val_loss: 0.0311521 | Time: 3176.74 ms [2022-03-23 22:34:34 main:574] : INFO : Epoch 1810 | loss: 0.0311501 | val_loss: 0.0311567 | Time: 3339.16 ms [2022-03-23 22:34:38 main:574] : INFO : Epoch 1811 | loss: 0.0311554 | val_loss: 0.0311577 | Time: 3420.35 ms [2022-03-23 22:34:41 main:574] : INFO : Epoch 1812 | loss: 0.0311534 | val_loss: 0.0311598 | Time: 3216.84 ms [2022-03-23 22:34:44 main:574] : INFO : Epoch 1813 | loss: 0.0311517 | val_loss: 0.0311552 | Time: 3274.53 ms [2022-03-23 22:34:48 main:574] : INFO : Epoch 1814 | loss: 0.0311489 | val_loss: 0.0311539 | Time: 3388.4 ms [2022-03-23 22:34:51 main:574] : INFO : Epoch 1815 | loss: 0.0311475 | val_loss: 0.0311537 | Time: 3022.44 ms [2022-03-23 22:34:54 main:574] : INFO : Epoch 1816 | loss: 0.0311481 | val_loss: 0.0311524 | Time: 3127.72 ms [2022-03-23 22:34:57 main:574] : INFO : Epoch 1817 | loss: 0.0311479 | val_loss: 0.0311542 | Time: 3209.72 ms [2022-03-23 22:35:00 main:574] : INFO : Epoch 1818 | loss: 0.0311445 | val_loss: 0.0311523 | Time: 3259.5 ms [2022-03-23 22:35:04 main:574] : INFO : Epoch 1819 | loss: 0.0311429 | val_loss: 0.0311493 | Time: 3320.13 ms [2022-03-23 22:35:07 main:574] : INFO : Epoch 1820 | loss: 0.0311421 | val_loss: 0.031151 | Time: 3201.1 ms [2022-03-23 22:35:10 main:574] : INFO : Epoch 1821 | loss: 0.0311402 | val_loss: 0.03115 | Time: 3079.97 ms [2022-03-23 22:35:13 main:574] : INFO : Epoch 1822 | loss: 0.0311402 | val_loss: 0.0311511 | Time: 2976.39 ms [2022-03-23 22:35:16 main:574] : INFO : Epoch 1823 | loss: 0.0311385 | val_loss: 0.0311503 | Time: 3149.85 ms [2022-03-23 22:35:20 main:574] : INFO : Epoch 1824 | loss: 0.0311378 | val_loss: 0.0311489 | Time: 3218.4 ms [2022-03-23 22:35:23 main:574] : INFO : Epoch 1825 | loss: 0.0311363 | val_loss: 0.0311507 | Time: 3023.32 ms [2022-03-23 22:35:26 main:574] : INFO : Epoch 1826 | loss: 0.0311387 | val_loss: 0.0311515 | Time: 3088.12 ms [2022-03-23 22:35:29 main:574] : INFO : Epoch 1827 | loss: 0.0311386 | val_loss: 0.0311508 | Time: 3207.41 ms [2022-03-23 22:35:32 main:574] : INFO : Epoch 1828 | loss: 0.0311362 | val_loss: 0.0311497 | Time: 3121.56 ms [2022-03-23 22:35:35 main:574] : INFO : Epoch 1829 | loss: 0.0311359 | val_loss: 0.0311538 | Time: 3152.91 ms [2022-03-23 22:35:39 main:574] : INFO : Epoch 1830 | loss: 0.0311369 | val_loss: 0.0311528 | Time: 3145.02 ms [2022-03-23 22:35:42 main:574] : INFO : Epoch 1831 | loss: 0.0311358 | val_loss: 0.0311529 | Time: 3025.69 ms [2022-03-23 22:35:45 main:574] : INFO : Epoch 1832 | loss: 0.0311368 | val_loss: 0.0311535 | Time: 3038.25 ms [2022-03-23 22:35:48 main:574] : INFO : Epoch 1833 | loss: 0.0311353 | val_loss: 0.0311514 | Time: 3348.72 ms [2022-03-23 22:35:51 main:574] : INFO : Epoch 1834 | loss: 0.0311375 | val_loss: 0.0311507 | Time: 3116.48 ms [2022-03-23 22:35:54 main:574] : INFO : Epoch 1835 | loss: 0.0311354 | val_loss: 0.031154 | Time: 3051.64 ms [2022-03-23 22:35:57 main:574] : INFO : Epoch 1836 | loss: 0.0311376 | val_loss: 0.0311488 | Time: 3084.03 ms [2022-03-23 22:36:01 main:574] : INFO : Epoch 1837 | loss: 0.0311406 | val_loss: 0.0311478 | Time: 3166.83 ms [2022-03-23 22:36:04 main:574] : INFO : Epoch 1838 | loss: 0.0311395 | val_loss: 0.0311521 | Time: 3153.65 ms [2022-03-23 22:36:07 main:574] : INFO : Epoch 1839 | loss: 0.0311437 | val_loss: 0.0311549 | Time: 3324.68 ms [2022-03-23 22:36:10 main:574] : INFO : Epoch 1840 | loss: 0.0311441 | val_loss: 0.0311582 | Time: 2962.68 ms [2022-03-23 22:36:13 main:574] : INFO : Epoch 1841 | loss: 0.0311471 | val_loss: 0.0311524 | Time: 3138.67 ms [2022-03-23 22:36:17 main:574] : INFO : Epoch 1842 | loss: 0.0311431 | val_loss: 0.0311534 | Time: 3175.53 ms [2022-03-23 22:36:20 main:574] : INFO : Epoch 1843 | loss: 0.0311422 | val_loss: 0.0311521 | Time: 2921 ms [2022-03-23 22:36:23 main:574] : INFO : Epoch 1844 | loss: 0.0311409 | val_loss: 0.0311536 | Time: 3164.77 ms [2022-03-23 22:36:26 main:574] : INFO : Epoch 1845 | loss: 0.0311401 | val_loss: 0.0311527 | Time: 3146.01 ms [2022-03-23 22:36:29 main:574] : INFO : Epoch 1846 | loss: 0.031142 | val_loss: 0.0311531 | Time: 3138.35 ms [2022-03-23 22:36:32 main:574] : INFO : Epoch 1847 | loss: 0.0311432 | val_loss: 0.0311518 | Time: 3182.88 ms [2022-03-23 22:36:35 main:574] : INFO : Epoch 1848 | loss: 0.0311457 | val_loss: 0.0311627 | Time: 3037.55 ms [2022-03-23 22:36:39 main:574] : INFO : Epoch 1849 | loss: 0.0311676 | val_loss: 0.0311647 | Time: 3186.15 ms [2022-03-23 22:36:42 main:574] : INFO : Epoch 1850 | loss: 0.0311765 | val_loss: 0.0311838 | Time: 3004.46 ms [2022-03-23 22:36:45 main:574] : INFO : Epoch 1851 | loss: 0.0311791 | val_loss: 0.031172 | Time: 3103.65 ms [2022-03-23 22:36:48 main:574] : INFO : Epoch 1852 | loss: 0.0311733 | val_loss: 0.0311697 | Time: 3062.06 ms [2022-03-23 22:36:51 main:574] : INFO : Epoch 1853 | loss: 0.0311702 | val_loss: 0.0311707 | Time: 3272.85 ms [2022-03-23 22:36:54 main:574] : INFO : Epoch 1854 | loss: 0.0311689 | val_loss: 0.031167 | Time: 3076.25 ms [2022-03-23 22:36:58 main:574] : INFO : Epoch 1855 | loss: 0.0311687 | val_loss: 0.0311641 | Time: 3191.49 ms [2022-03-23 22:37:01 main:574] : INFO : Epoch 1856 | loss: 0.0311636 | val_loss: 0.0311611 | Time: 3070.98 ms [2022-03-23 22:37:04 main:574] : INFO : Epoch 1857 | loss: 0.0311594 | val_loss: 0.0311609 | Time: 3060.46 ms [2022-03-23 22:37:07 main:574] : INFO : Epoch 1858 | loss: 0.0311586 | val_loss: 0.0311613 | Time: 3245.47 ms [2022-03-23 22:37:10 main:574] : INFO : Epoch 1859 | loss: 0.0311592 | val_loss: 0.0311644 | Time: 3057.66 ms [2022-03-23 22:37:14 main:574] : INFO : Epoch 1860 | loss: 0.0311584 | val_loss: 0.0311619 | Time: 3425.64 ms [2022-03-23 22:37:17 main:574] : INFO : Epoch 1861 | loss: 0.0311585 | val_loss: 0.0311639 | Time: 3259.95 ms [2022-03-23 22:37:20 main:574] : INFO : Epoch 1862 | loss: 0.0311566 | val_loss: 0.03116 | Time: 3172.29 ms [2022-03-23 22:37:23 main:574] : INFO : Epoch 1863 | loss: 0.0311566 | val_loss: 0.0311599 | Time: 2970.57 ms [2022-03-23 22:37:26 main:574] : INFO : Epoch 1864 | loss: 0.0311575 | val_loss: 0.0311632 | Time: 3220.5 ms [2022-03-23 22:37:30 main:574] : INFO : Epoch 1865 | loss: 0.0311562 | val_loss: 0.0311612 | Time: 3288.11 ms [2022-03-23 22:37:33 main:574] : INFO : Epoch 1866 | loss: 0.0311535 | val_loss: 0.0311588 | Time: 3161.16 ms [2022-03-23 22:37:36 main:574] : INFO : Epoch 1867 | loss: 0.0311534 | val_loss: 0.0311652 | Time: 3024.39 ms [2022-03-23 22:37:39 main:574] : INFO : Epoch 1868 | loss: 0.0311543 | val_loss: 0.0311587 | Time: 3092.2 ms [2022-03-23 22:37:42 main:574] : INFO : Epoch 1869 | loss: 0.0311517 | val_loss: 0.0311578 | Time: 3229.82 ms [2022-03-23 22:37:46 main:574] : INFO : Epoch 1870 | loss: 0.0311541 | val_loss: 0.0311696 | Time: 3345.57 ms [2022-03-23 22:37:49 main:574] : INFO : Epoch 1871 | loss: 0.0311532 | val_loss: 0.031164 | Time: 3156.32 ms [2022-03-23 22:37:52 main:574] : INFO : Epoch 1872 | loss: 0.0311493 | val_loss: 0.0311583 | Time: 3178.87 ms [2022-03-23 22:37:55 main:574] : INFO : Epoch 1873 | loss: 0.0311479 | val_loss: 0.0311581 | Time: 3026.05 ms [2022-03-23 22:37:58 main:574] : INFO : Epoch 1874 | loss: 0.0311477 | val_loss: 0.031158 | Time: 3048.03 ms [2022-03-23 22:38:01 main:574] : INFO : Epoch 1875 | loss: 0.0311485 | val_loss: 0.0311558 | Time: 3041.3 ms [2022-03-23 22:38:04 main:574] : INFO : Epoch 1876 | loss: 0.0311462 | val_loss: 0.0311567 | Time: 2955.28 ms [2022-03-23 22:38:08 main:574] : INFO : Epoch 1877 | loss: 0.0311457 | val_loss: 0.0311562 | Time: 3261.81 ms [2022-03-23 22:38:11 main:574] : INFO : Epoch 1878 | loss: 0.0311449 | val_loss: 0.0311562 | Time: 3483.55 ms [2022-03-23 22:38:14 main:574] : INFO : Epoch 1879 | loss: 0.0311467 | val_loss: 0.03116 | Time: 2889.56 ms [2022-03-23 22:38:17 main:574] : INFO : Epoch 1880 | loss: 0.0311684 | val_loss: 0.0311785 | Time: 3261.1 ms [2022-03-23 22:38:21 main:574] : INFO : Epoch 1881 | loss: 0.031175 | val_loss: 0.0311797 | Time: 3037.77 ms [2022-03-23 22:38:25 main:574] : INFO : Epoch 1882 | loss: 0.0311734 | val_loss: 0.0311748 | Time: 3996.12 ms [2022-03-23 22:38:28 main:574] : INFO : Epoch 1883 | loss: 0.0311686 | val_loss: 0.0311734 | Time: 3656.01 ms [2022-03-23 22:38:32 main:574] : INFO : Epoch 1884 | loss: 0.0311669 | val_loss: 0.0311699 | Time: 3706.06 ms [2022-03-23 22:38:36 main:574] : INFO : Epoch 1885 | loss: 0.0311645 | val_loss: 0.0311674 | Time: 3583.54 ms [2022-03-23 22:38:40 main:574] : INFO : Epoch 1886 | loss: 0.0311631 | val_loss: 0.0311702 | Time: 4530.58 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-03-24 18:49:01 main:435] : INFO : Set logging level to 1 [2022-03-24 18:49:01 main:441] : INFO : Running in BOINC Client mode [2022-03-24 18:49:01 main:444] : INFO : Resolving all filenames [2022-03-24 18:49:01 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-24 18:49:01 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-24 18:49:01 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-24 18:49:01 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-24 18:49:01 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-24 18:49:01 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-24 18:49:01 main:474] : INFO : Configuration: [2022-03-24 18:49:01 main:475] : INFO : Model type: GRU [2022-03-24 18:49:01 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-24 18:49:01 main:477] : INFO : Max Epochs: 2048 [2022-03-24 18:49:01 main:478] : INFO : Batch Size: 128 [2022-03-24 18:49:01 main:479] : INFO : Learning Rate: 0.01 [2022-03-24 18:49:01 main:480] : INFO : Patience: 10 [2022-03-24 18:49:01 main:481] : INFO : Hidden Width: 12 [2022-03-24 18:49:01 main:482] : INFO : # Recurrent Layers: 4 [2022-03-24 18:49:01 main:483] : INFO : # Backend Layers: 4 [2022-03-24 18:49:01 main:484] : INFO : # Threads: 1 [2022-03-24 18:49:01 main:486] : INFO : Preparing Dataset [2022-03-24 18:49:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-24 18:49:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-24 18:49:03 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-24 18:49:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-24 18:49:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-24 18:49:03 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-24 18:49:03 main:494] : INFO : Creating Model [2022-03-24 18:49:03 main:507] : INFO : Preparing config file [2022-03-24 18:49:03 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-24 18:49:03 main:512] : INFO : Loading config [2022-03-24 18:49:03 main:514] : INFO : Loading state [2022-03-24 18:49:09 main:559] : INFO : Loading DataLoader into Memory [2022-03-24 18:49:10 main:562] : INFO : Starting Training [2022-03-24 18:49:15 main:574] : INFO : Epoch 1864 | loss: 0.0312726 | val_loss: 0.0311795 | Time: 5019.47 ms [2022-03-24 18:49:18 main:574] : INFO : Epoch 1865 | loss: 0.0311612 | val_loss: 0.0311665 | Time: 3145.39 ms [2022-03-24 18:49:21 main:574] : INFO : Epoch 1866 | loss: 0.0311552 | val_loss: 0.0311599 | Time: 3289.97 ms [2022-03-24 18:49:24 main:574] : INFO : Epoch 1867 | loss: 0.0311532 | val_loss: 0.0311659 | Time: 3282.48 ms [2022-03-24 18:49:28 main:574] : INFO : Epoch 1868 | loss: 0.0311566 | val_loss: 0.031159 | Time: 3122.8 ms [2022-03-24 18:49:31 main:574] : INFO : Epoch 1869 | loss: 0.031151 | val_loss: 0.0311661 | Time: 3117.8 ms [2022-03-24 18:49:34 main:574] : INFO : Epoch 1870 | loss: 0.0311499 | val_loss: 0.0311779 | Time: 3140.91 ms [2022-03-24 18:49:37 main:574] : INFO : Epoch 1871 | loss: 0.031153 | val_loss: 0.031165 | Time: 3130.67 ms [2022-03-24 18:49:40 main:574] : INFO : Epoch 1872 | loss: 0.0311504 | val_loss: 0.031159 | Time: 3185.87 ms [2022-03-24 18:49:44 main:574] : INFO : Epoch 1873 | loss: 0.0311474 | val_loss: 0.0311598 | Time: 3178.16 ms [2022-03-24 18:49:47 main:574] : INFO : Epoch 1874 | loss: 0.0311465 | val_loss: 0.0311572 | Time: 3215.67 ms [2022-03-24 18:49:50 main:574] : INFO : Epoch 1875 | loss: 0.0311458 | val_loss: 0.0311599 | Time: 3253.11 ms [2022-03-24 18:49:53 main:574] : INFO : Epoch 1876 | loss: 0.031145 | val_loss: 0.0311585 | Time: 3094.17 ms [2022-03-24 18:49:57 main:574] : INFO : Epoch 1877 | loss: 0.0311425 | val_loss: 0.0311566 | Time: 3259.31 ms [2022-03-24 18:50:00 main:574] : INFO : Epoch 1878 | loss: 0.0311455 | val_loss: 0.0311565 | Time: 3216.07 ms [2022-03-24 18:50:03 main:574] : INFO : Epoch 1879 | loss: 0.0311462 | val_loss: 0.031156 | Time: 3299.79 ms [2022-03-24 18:50:06 main:574] : INFO : Epoch 1880 | loss: 0.0311465 | val_loss: 0.0311632 | Time: 3180.13 ms [2022-03-24 18:50:10 main:574] : INFO : Epoch 1881 | loss: 0.0311468 | val_loss: 0.0311548 | Time: 3276.69 ms [2022-03-24 18:50:13 main:574] : INFO : Epoch 1882 | loss: 0.0311437 | val_loss: 0.0311545 | Time: 3286.47 ms [2022-03-24 18:50:16 main:574] : INFO : Epoch 1883 | loss: 0.031142 | val_loss: 0.0311571 | Time: 3306.81 ms [2022-03-24 18:50:20 main:574] : INFO : Epoch 1884 | loss: 0.0311436 | val_loss: 0.0311548 | Time: 3307.05 ms [2022-03-24 18:50:23 main:574] : INFO : Epoch 1885 | loss: 0.0311456 | val_loss: 0.0311636 | Time: 3195.64 ms [2022-03-24 18:50:26 main:574] : INFO : Epoch 1886 | loss: 0.0311453 | val_loss: 0.0311519 | Time: 3285.31 ms [2022-03-24 18:50:29 main:574] : INFO : Epoch 1887 | loss: 0.0311438 | val_loss: 0.0311549 | Time: 3190.07 ms [2022-03-24 18:50:33 main:574] : INFO : Epoch 1888 | loss: 0.0311441 | val_loss: 0.0311523 | Time: 3177.78 ms [2022-03-24 18:50:36 main:574] : INFO : Epoch 1889 | loss: 0.0311399 | val_loss: 0.0311542 | Time: 3176.29 ms [2022-03-24 18:50:40 main:574] : INFO : Epoch 1890 | loss: 0.0311402 | val_loss: 0.0311514 | Time: 3555.5 ms [2022-03-24 18:50:43 main:574] : INFO : Epoch 1891 | loss: 0.0311398 | val_loss: 0.0311547 | Time: 3257.9 ms [2022-03-24 18:50:46 main:574] : INFO : Epoch 1892 | loss: 0.0311402 | val_loss: 0.0311538 | Time: 3311.45 ms [2022-03-24 18:50:50 main:574] : INFO : Epoch 1893 | loss: 0.0311403 | val_loss: 0.0311533 | Time: 3473.29 ms [2022-03-24 18:50:53 main:574] : INFO : Epoch 1894 | loss: 0.0311388 | val_loss: 0.031156 | Time: 3140.65 ms [2022-03-24 18:50:56 main:574] : INFO : Epoch 1895 | loss: 0.0311374 | val_loss: 0.0311548 | Time: 3322.38 ms [2022-03-24 18:51:00 main:574] : INFO : Epoch 1896 | loss: 0.0311374 | val_loss: 0.0311545 | Time: 3290.21 ms [2022-03-24 18:51:03 main:574] : INFO : Epoch 1897 | loss: 0.0311397 | val_loss: 0.0311566 | Time: 3221.06 ms [2022-03-24 18:51:06 main:574] : INFO : Epoch 1898 | loss: 0.0311441 | val_loss: 0.0311549 | Time: 3359.43 ms [2022-03-24 18:51:09 main:574] : INFO : Epoch 1899 | loss: 0.031143 | val_loss: 0.0311557 | Time: 3169.03 ms [2022-03-24 18:51:13 main:574] : INFO : Epoch 1900 | loss: 0.0311423 | val_loss: 0.0311569 | Time: 3239.15 ms [2022-03-24 18:51:16 main:574] : INFO : Epoch 1901 | loss: 0.0311431 | val_loss: 0.0311566 | Time: 3309.4 ms [2022-03-24 18:51:19 main:574] : INFO : Epoch 1902 | loss: 0.0311398 | val_loss: 0.0311569 | Time: 3231.54 ms [2022-03-24 18:51:23 main:574] : INFO : Epoch 1903 | loss: 0.0311385 | val_loss: 0.0311545 | Time: 3235.5 ms [2022-03-24 18:51:26 main:574] : INFO : Epoch 1904 | loss: 0.031138 | val_loss: 0.0311569 | Time: 3357.58 ms [2022-03-24 18:51:29 main:574] : INFO : Epoch 1905 | loss: 0.0311361 | val_loss: 0.031154 | Time: 3384.65 ms [2022-03-24 18:51:33 main:574] : INFO : Epoch 1906 | loss: 0.0311354 | val_loss: 0.0311546 | Time: 3266.93 ms [2022-03-24 18:51:36 main:574] : INFO : Epoch 1907 | loss: 0.0311353 | val_loss: 0.0311608 | Time: 3231 ms [2022-03-24 18:51:39 main:574] : INFO : Epoch 1908 | loss: 0.0311356 | val_loss: 0.0311528 | Time: 3197.45 ms [2022-03-24 18:51:43 main:574] : INFO : Epoch 1909 | loss: 0.0311344 | val_loss: 0.0311568 | Time: 3397.04 ms [2022-03-24 18:51:46 main:574] : INFO : Epoch 1910 | loss: 0.0311321 | val_loss: 0.0311552 | Time: 3335 ms [2022-03-24 18:51:49 main:574] : INFO : Epoch 1911 | loss: 0.0311332 | val_loss: 0.0311538 | Time: 3208.31 ms [2022-03-24 18:51:52 main:574] : INFO : Epoch 1912 | loss: 0.0311352 | val_loss: 0.0311554 | Time: 3094.56 ms [2022-03-24 18:51:56 main:574] : INFO : Epoch 1913 | loss: 0.0311359 | val_loss: 0.0311561 | Time: 3185.67 ms [2022-03-24 18:51:59 main:574] : INFO : Epoch 1914 | loss: 0.0311364 | val_loss: 0.0311561 | Time: 3366.28 ms [2022-03-24 18:52:02 main:574] : INFO : Epoch 1915 | loss: 0.0311358 | val_loss: 0.0311563 | Time: 3341.91 ms [2022-03-24 18:52:06 main:574] : INFO : Epoch 1916 | loss: 0.0311352 | val_loss: 0.0311532 | Time: 3164.44 ms [2022-03-24 18:52:09 main:574] : INFO : Epoch 1917 | loss: 0.0311362 | val_loss: 0.031153 | Time: 3251.46 ms [2022-03-24 18:52:12 main:574] : INFO : Epoch 1918 | loss: 0.0311348 | val_loss: 0.0311526 | Time: 3391.05 ms [2022-03-24 18:52:16 main:574] : INFO : Epoch 1919 | loss: 0.0311376 | val_loss: 0.031151 | Time: 3401.12 ms [2022-03-24 18:52:19 main:574] : INFO : Epoch 1920 | loss: 0.0311344 | val_loss: 0.0311519 | Time: 3391.08 ms [2022-03-24 18:52:23 main:574] : INFO : Epoch 1921 | loss: 0.0311333 | val_loss: 0.0311517 | Time: 3497.72 ms [2022-03-24 18:52:26 main:574] : INFO : Epoch 1922 | loss: 0.0311338 | val_loss: 0.0311529 | Time: 3254.12 ms [2022-03-24 18:52:30 main:574] : INFO : Epoch 1923 | loss: 0.0311316 | val_loss: 0.031154 | Time: 3418.11 ms [2022-03-24 18:52:33 main:574] : INFO : Epoch 1924 | loss: 0.0311314 | val_loss: 0.031151 | Time: 3332.67 ms [2022-03-24 18:52:36 main:574] : INFO : Epoch 1925 | loss: 0.031133 | val_loss: 0.0311514 | Time: 3088.84 ms [2022-03-24 18:52:40 main:574] : INFO : Epoch 1926 | loss: 0.0311337 | val_loss: 0.0311535 | Time: 3438.27 ms [2022-03-24 18:52:43 main:574] : INFO : Epoch 1927 | loss: 0.0311326 | val_loss: 0.0311538 | Time: 3537.74 ms [2022-03-24 18:52:46 main:574] : INFO : Epoch 1928 | loss: 0.031131 | val_loss: 0.0311579 | Time: 3249.1 ms [2022-03-24 18:52:50 main:574] : INFO : Epoch 1929 | loss: 0.0311347 | val_loss: 0.0311574 | Time: 3401.28 ms [2022-03-24 18:52:53 main:574] : INFO : Epoch 1930 | loss: 0.0311323 | val_loss: 0.0311591 | Time: 3331.53 ms [2022-03-24 18:52:57 main:574] : INFO : Epoch 1931 | loss: 0.0311332 | val_loss: 0.0311567 | Time: 3655.26 ms [2022-03-24 18:53:01 main:574] : INFO : Epoch 1932 | loss: 0.0311324 | val_loss: 0.0311552 | Time: 4175.01 ms [2022-03-24 18:53:05 main:574] : INFO : Epoch 1933 | loss: 0.0311317 | val_loss: 0.0311564 | Time: 3956.9 ms [2022-03-24 18:53:09 main:574] : INFO : Epoch 1934 | loss: 0.0311344 | val_loss: 0.0311624 | Time: 4006.12 ms [2022-03-24 18:53:14 main:574] : INFO : Epoch 1935 | loss: 0.0311433 | val_loss: 0.0311705 | Time: 4427.29 ms [2022-03-24 18:53:20 main:574] : INFO : Epoch 1936 | loss: 0.0311409 | val_loss: 0.0311611 | Time: 6624.11 ms [2022-03-24 18:53:28 main:574] : INFO : Epoch 1937 | loss: 0.0311377 | val_loss: 0.0311569 | Time: 7337.38 ms [2022-03-24 18:53:35 main:574] : INFO : Epoch 1938 | loss: 0.0311336 | val_loss: 0.0311542 | Time: 7277.26 ms [2022-03-24 18:53:42 main:574] : INFO : Epoch 1939 | loss: 0.0311312 | val_loss: 0.0311547 | Time: 7322.93 ms [2022-03-24 18:53:50 main:574] : INFO : Epoch 1940 | loss: 0.0311308 | val_loss: 0.0311548 | Time: 7416.74 ms [2022-03-24 18:53:57 main:574] : INFO : Epoch 1941 | loss: 0.0311329 | val_loss: 0.0311569 | Time: 7239.94 ms [2022-03-24 18:54:05 main:574] : INFO : Epoch 1942 | loss: 0.0311319 | val_loss: 0.0311571 | Time: 7430.14 ms [2022-03-24 18:54:12 main:574] : INFO : Epoch 1943 | loss: 0.0311296 | val_loss: 0.031153 | Time: 7308.43 ms [2022-03-24 18:54:19 main:574] : INFO : Epoch 1944 | loss: 0.0311324 | val_loss: 0.0311547 | Time: 7331.66 ms [2022-03-24 18:54:27 main:574] : INFO : Epoch 1945 | loss: 0.0311307 | val_loss: 0.0311555 | Time: 7379.11 ms [2022-03-24 18:54:34 main:574] : INFO : Epoch 1946 | loss: 0.0311315 | val_loss: 0.0311548 | Time: 7395.47 ms [2022-03-24 18:54:42 main:574] : INFO : Epoch 1947 | loss: 0.031133 | val_loss: 0.031153 | Time: 7519.76 ms [2022-03-24 18:54:49 main:574] : INFO : Epoch 1948 | loss: 0.0311453 | val_loss: 0.0311557 | Time: 7505.96 ms [2022-03-24 18:54:57 main:574] : INFO : Epoch 1949 | loss: 0.0311516 | val_loss: 0.0311568 | Time: 7586.01 ms [2022-03-24 18:55:04 main:574] : INFO : Epoch 1950 | loss: 0.0311522 | val_loss: 0.0311544 | Time: 7274.88 ms [2022-03-24 18:55:12 main:574] : INFO : Epoch 1951 | loss: 0.0311502 | val_loss: 0.0311531 | Time: 7460.87 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080 Ti) [2022-03-24 18:58:16 main:435] : INFO : Set logging level to 1 [2022-03-24 18:58:16 main:441] : INFO : Running in BOINC Client mode [2022-03-24 18:58:16 main:444] : INFO : Resolving all filenames [2022-03-24 18:58:16 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-24 18:58:16 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-24 18:58:16 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-24 18:58:16 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-24 18:58:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-24 18:58:16 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-24 18:58:16 main:474] : INFO : Configuration: [2022-03-24 18:58:16 main:475] : INFO : Model type: GRU [2022-03-24 18:58:16 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-24 18:58:16 main:477] : INFO : Max Epochs: 2048 [2022-03-24 18:58:17 main:478] : INFO : Batch Size: 128 [2022-03-24 18:58:17 main:479] : INFO : Learning Rate: 0.01 [2022-03-24 18:58:17 main:480] : INFO : Patience: 10 [2022-03-24 18:58:17 main:481] : INFO : Hidden Width: 12 [2022-03-24 18:58:17 main:482] : INFO : # Recurrent Layers: 4 [2022-03-24 18:58:17 main:483] : INFO : # Backend Layers: 4 [2022-03-24 18:58:17 main:484] : INFO : # Threads: 1 [2022-03-24 18:58:17 main:486] : INFO : Preparing Dataset [2022-03-24 18:58:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-24 18:58:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-24 18:58:18 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-24 18:58:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-24 18:58:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-24 18:58:18 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-24 18:58:18 main:494] : INFO : Creating Model [2022-03-24 18:58:18 main:507] : INFO : Preparing config file [2022-03-24 18:58:18 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-24 18:58:18 main:512] : INFO : Loading config [2022-03-24 18:58:18 main:514] : INFO : Loading state [2022-03-24 18:58:19 main:559] : INFO : Loading DataLoader into Memory [2022-03-24 18:58:19 main:562] : INFO : Starting Training [2022-03-24 18:58:22 main:574] : INFO : Epoch 1951 | loss: 0.0312133 | val_loss: 0.0311693 | Time: 3313.89 ms [2022-03-24 18:58:26 main:574] : INFO : Epoch 1952 | loss: 0.0311584 | val_loss: 0.0311624 | Time: 3074.1 ms [2022-03-24 18:58:29 main:574] : INFO : Epoch 1953 | loss: 0.0311441 | val_loss: 0.0311598 | Time: 3267.45 ms [2022-03-24 18:58:32 main:574] : INFO : Epoch 1954 | loss: 0.0311402 | val_loss: 0.0311523 | Time: 3194.83 ms [2022-03-24 18:58:35 main:574] : INFO : Epoch 1955 | loss: 0.031137 | val_loss: 0.031154 | Time: 3106.37 ms [2022-03-24 18:58:39 main:574] : INFO : Epoch 1956 | loss: 0.0311383 | val_loss: 0.0311573 | Time: 3196.09 ms [2022-03-24 18:58:42 main:574] : INFO : Epoch 1957 | loss: 0.031136 | val_loss: 0.0311532 | Time: 3216.39 ms [2022-03-24 18:58:45 main:574] : INFO : Epoch 1958 | loss: 0.0311356 | val_loss: 0.0311533 | Time: 3122.61 ms [2022-03-24 18:58:48 main:574] : INFO : Epoch 1959 | loss: 0.0311354 | val_loss: 0.0311531 | Time: 3184.98 ms [2022-03-24 18:58:51 main:574] : INFO : Epoch 1960 | loss: 0.0311352 | val_loss: 0.031153 | Time: 3276.08 ms [2022-03-24 18:58:55 main:574] : INFO : Epoch 1961 | loss: 0.0311348 | val_loss: 0.0311498 | Time: 3201.45 ms [2022-03-24 18:58:58 main:574] : INFO : Epoch 1962 | loss: 0.0311344 | val_loss: 0.0311529 | Time: 3172.86 ms [2022-03-24 18:59:01 main:574] : INFO : Epoch 1963 | loss: 0.0311351 | val_loss: 0.031153 | Time: 3521.35 ms [2022-03-24 18:59:05 main:574] : INFO : Epoch 1964 | loss: 0.031137 | val_loss: 0.0311564 | Time: 3274.44 ms [2022-03-24 18:59:08 main:574] : INFO : Epoch 1965 | loss: 0.0311442 | val_loss: 0.0311542 | Time: 3045.71 ms [2022-03-24 18:59:11 main:574] : INFO : Epoch 1966 | loss: 0.0311426 | val_loss: 0.0311494 | Time: 3209.01 ms [2022-03-24 18:59:14 main:574] : INFO : Epoch 1967 | loss: 0.0311378 | val_loss: 0.0311509 | Time: 3297.86 ms [2022-03-24 18:59:18 main:574] : INFO : Epoch 1968 | loss: 0.0311361 | val_loss: 0.0311485 | Time: 3245.46 ms [2022-03-24 18:59:21 main:574] : INFO : Epoch 1969 | loss: 0.0311356 | val_loss: 0.03115 | Time: 3213.2 ms [2022-03-24 18:59:24 main:574] : INFO : Epoch 1970 | loss: 0.0311361 | val_loss: 0.0311503 | Time: 3217.56 ms [2022-03-24 18:59:27 main:574] : INFO : Epoch 1971 | loss: 0.0311348 | val_loss: 0.0311542 | Time: 3224.3 ms [2022-03-24 18:59:31 main:574] : INFO : Epoch 1972 | loss: 0.0311327 | val_loss: 0.0311504 | Time: 3190.29 ms [2022-03-24 18:59:34 main:574] : INFO : Epoch 1973 | loss: 0.0311336 | val_loss: 0.0311525 | Time: 3231.99 ms [2022-03-24 18:59:37 main:574] : INFO : Epoch 1974 | loss: 0.0311348 | val_loss: 0.031155 | Time: 3304.32 ms [2022-03-24 18:59:40 main:574] : INFO : Epoch 1975 | loss: 0.0311353 | val_loss: 0.0311557 | Time: 3064.96 ms [2022-03-24 18:59:44 main:574] : INFO : Epoch 1976 | loss: 0.0311348 | val_loss: 0.0311539 | Time: 3182.92 ms [2022-03-24 18:59:47 main:574] : INFO : Epoch 1977 | loss: 0.0311349 | val_loss: 0.0311568 | Time: 3274.7 ms [2022-03-24 18:59:50 main:574] : INFO : Epoch 1978 | loss: 0.0311328 | val_loss: 0.0311542 | Time: 3323.84 ms [2022-03-24 18:59:54 main:574] : INFO : Epoch 1979 | loss: 0.0311373 | val_loss: 0.0311561 | Time: 3112.79 ms [2022-03-24 18:59:57 main:574] : INFO : Epoch 1980 | loss: 0.0311402 | val_loss: 0.0311503 | Time: 3114.97 ms [2022-03-24 19:00:00 main:574] : INFO : Epoch 1981 | loss: 0.0311391 | val_loss: 0.0311511 | Time: 3171.71 ms [2022-03-24 19:00:03 main:574] : INFO : Epoch 1982 | loss: 0.0311374 | val_loss: 0.0311527 | Time: 3171.19 ms [2022-03-24 19:00:06 main:574] : INFO : Epoch 1983 | loss: 0.0311362 | val_loss: 0.0311511 | Time: 3141.85 ms [2022-03-24 19:00:10 main:574] : INFO : Epoch 1984 | loss: 0.0311361 | val_loss: 0.0311481 | Time: 3176.56 ms [2022-03-24 19:00:13 main:574] : INFO : Epoch 1985 | loss: 0.0311333 | val_loss: 0.0311488 | Time: 3180.54 ms [2022-03-24 19:00:16 main:574] : INFO : Epoch 1986 | loss: 0.0311326 | val_loss: 0.0311496 | Time: 3120.75 ms [2022-03-24 19:00:19 main:574] : INFO : Epoch 1987 | loss: 0.0311336 | val_loss: 0.0311505 | Time: 3139.93 ms [2022-03-24 19:00:22 main:574] : INFO : Epoch 1988 | loss: 0.031132 | val_loss: 0.0311486 | Time: 3161.81 ms [2022-03-24 19:00:25 main:574] : INFO : Epoch 1989 | loss: 0.0311292 | val_loss: 0.0311538 | Time: 3150.79 ms [2022-03-24 19:00:29 main:574] : INFO : Epoch 1990 | loss: 0.0311298 | val_loss: 0.0311511 | Time: 3259.14 ms [2022-03-24 19:00:32 main:574] : INFO : Epoch 1991 | loss: 0.0311327 | val_loss: 0.0311504 | Time: 3248.61 ms [2022-03-24 19:00:35 main:574] : INFO : Epoch 1992 | loss: 0.0311326 | val_loss: 0.0311534 | Time: 3197.18 ms [2022-03-24 19:00:38 main:574] : INFO : Epoch 1993 | loss: 0.0311297 | val_loss: 0.0311516 | Time: 3146.61 ms [2022-03-24 19:00:42 main:574] : INFO : Epoch 1994 | loss: 0.031129 | val_loss: 0.0311475 | Time: 3232.4 ms [2022-03-24 19:00:45 main:574] : INFO : Epoch 1995 | loss: 0.0311287 | val_loss: 0.0311456 | Time: 3172.07 ms [2022-03-24 19:00:48 main:574] : INFO : Epoch 1996 | loss: 0.031128 | val_loss: 0.0311443 | Time: 3263.19 ms [2022-03-24 19:00:51 main:574] : INFO : Epoch 1997 | loss: 0.0311291 | val_loss: 0.0311494 | Time: 3136.5 ms [2022-03-24 19:00:55 main:574] : INFO : Epoch 1998 | loss: 0.0311312 | val_loss: 0.0311507 | Time: 3224.39 ms [2022-03-24 19:00:58 main:574] : INFO : Epoch 1999 | loss: 0.0311314 | val_loss: 0.0311546 | Time: 3340.35 ms [2022-03-24 19:01:01 main:574] : INFO : Epoch 2000 | loss: 0.0311325 | val_loss: 0.0311519 | Time: 3223.96 ms [2022-03-24 19:01:05 main:574] : INFO : Epoch 2001 | loss: 0.0311299 | val_loss: 0.0311514 | Time: 3152.89 ms [2022-03-24 19:01:08 main:574] : INFO : Epoch 2002 | loss: 0.0311291 | val_loss: 0.0311493 | Time: 3052.56 ms [2022-03-24 19:01:11 main:574] : INFO : Epoch 2003 | loss: 0.0311277 | val_loss: 0.0311513 | Time: 3279.33 ms [2022-03-24 19:01:14 main:574] : INFO : Epoch 2004 | loss: 0.0311301 | val_loss: 0.0311547 | Time: 3120.24 ms [2022-03-24 19:01:17 main:574] : INFO : Epoch 2005 | loss: 0.0311289 | val_loss: 0.0311511 | Time: 3171.01 ms [2022-03-24 19:01:20 main:574] : INFO : Epoch 2006 | loss: 0.0311297 | val_loss: 0.031148 | Time: 3133.9 ms [2022-03-24 19:01:24 main:574] : INFO : Epoch 2007 | loss: 0.0311291 | val_loss: 0.0311493 | Time: 3214.79 ms [2022-03-24 19:01:27 main:574] : INFO : Epoch 2008 | loss: 0.0311284 | val_loss: 0.0311478 | Time: 3338.54 ms [2022-03-24 19:01:30 main:574] : INFO : Epoch 2009 | loss: 0.0311288 | val_loss: 0.0311485 | Time: 3292.11 ms [2022-03-24 19:01:34 main:574] : INFO : Epoch 2010 | loss: 0.0311277 | val_loss: 0.0311518 | Time: 3223.85 ms [2022-03-24 19:01:37 main:574] : INFO : Epoch 2011 | loss: 0.0311282 | val_loss: 0.0311515 | Time: 3202.17 ms [2022-03-24 19:01:40 main:574] : INFO : Epoch 2012 | loss: 0.0311276 | val_loss: 0.0311477 | Time: 3077.51 ms [2022-03-24 19:01:43 main:574] : INFO : Epoch 2013 | loss: 0.031131 | val_loss: 0.0311484 | Time: 3302.9 ms [2022-03-24 19:01:47 main:574] : INFO : Epoch 2014 | loss: 0.031131 | val_loss: 0.0311501 | Time: 3184.06 ms [2022-03-24 19:01:50 main:574] : INFO : Epoch 2015 | loss: 0.0311294 | val_loss: 0.0311517 | Time: 3380.75 ms [2022-03-24 19:01:53 main:574] : INFO : Epoch 2016 | loss: 0.0311297 | val_loss: 0.0311507 | Time: 3270.64 ms [2022-03-24 19:01:57 main:574] : INFO : Epoch 2017 | loss: 0.0311275 | val_loss: 0.0311474 | Time: 3212.83 ms [2022-03-24 19:02:00 main:574] : INFO : Epoch 2018 | loss: 0.0311262 | val_loss: 0.0311465 | Time: 3432.1 ms [2022-03-24 19:02:04 main:574] : INFO : Epoch 2019 | loss: 0.031125 | val_loss: 0.0311521 | Time: 3264.34 ms [2022-03-24 19:02:07 main:574] : INFO : Epoch 2020 | loss: 0.0311278 | val_loss: 0.0311519 | Time: 3233.55 ms [2022-03-24 19:02:10 main:574] : INFO : Epoch 2021 | loss: 0.0311604 | val_loss: 0.0311989 | Time: 3180.97 ms [2022-03-24 19:02:13 main:574] : INFO : Epoch 2022 | loss: 0.0354438 | val_loss: 0.0357417 | Time: 3150.97 ms [2022-03-24 19:02:16 main:574] : INFO : Epoch 2023 | loss: 0.0328537 | val_loss: 0.0314836 | Time: 3101.42 ms [2022-03-24 19:02:20 main:574] : INFO : Epoch 2024 | loss: 0.0314057 | val_loss: 0.031257 | Time: 3154.1 ms [2022-03-24 19:02:23 main:574] : INFO : Epoch 2025 | loss: 0.0312446 | val_loss: 0.0312102 | Time: 3156.11 ms [2022-03-24 19:02:26 main:574] : INFO : Epoch 2026 | loss: 0.0312098 | val_loss: 0.0311944 | Time: 3288.72 ms [2022-03-24 19:02:29 main:574] : INFO : Epoch 2027 | loss: 0.0311988 | val_loss: 0.0311932 | Time: 3256.7 ms [2022-03-24 19:02:33 main:574] : INFO : Epoch 2028 | loss: 0.0312006 | val_loss: 0.0311947 | Time: 3305.83 ms [2022-03-24 19:02:36 main:574] : INFO : Epoch 2029 | loss: 0.0312016 | val_loss: 0.0311914 | Time: 3029.87 ms [2022-03-24 19:02:39 main:574] : INFO : Epoch 2030 | loss: 0.0311957 | val_loss: 0.0311919 | Time: 3164.79 ms [2022-03-24 19:02:42 main:574] : INFO : Epoch 2031 | loss: 0.0311916 | val_loss: 0.0311856 | Time: 3081 ms [2022-03-24 19:02:45 main:574] : INFO : Epoch 2032 | loss: 0.0311894 | val_loss: 0.0311848 | Time: 3212.88 ms [2022-03-24 19:02:49 main:574] : INFO : Epoch 2033 | loss: 0.0311877 | val_loss: 0.0311832 | Time: 3127.01 ms [2022-03-24 19:02:52 main:574] : INFO : Epoch 2034 | loss: 0.0311862 | val_loss: 0.0311813 | Time: 3217.66 ms [2022-03-24 19:02:55 main:574] : INFO : Epoch 2035 | loss: 0.0311862 | val_loss: 0.0311813 | Time: 3166.01 ms [2022-03-24 19:02:58 main:574] : INFO : Epoch 2036 | loss: 0.031186 | val_loss: 0.0311812 | Time: 3294.09 ms [2022-03-24 19:03:01 main:574] : INFO : Epoch 2037 | loss: 0.0311857 | val_loss: 0.0311802 | Time: 3102.71 ms [2022-03-24 19:03:05 main:574] : INFO : Epoch 2038 | loss: 0.0311843 | val_loss: 0.0311791 | Time: 3364.98 ms [2022-03-24 19:03:08 main:574] : INFO : Epoch 2039 | loss: 0.0311828 | val_loss: 0.031179 | Time: 3247.2 ms [2022-03-24 19:03:11 main:574] : INFO : Epoch 2040 | loss: 0.0311828 | val_loss: 0.0311809 | Time: 3084.52 ms [2022-03-24 19:03:15 main:574] : INFO : Epoch 2041 | loss: 0.031185 | val_loss: 0.0311805 | Time: 3189.69 ms [2022-03-24 19:03:18 main:574] : INFO : Epoch 2042 | loss: 0.0311852 | val_loss: 0.0311804 | Time: 3204.85 ms [2022-03-24 19:03:21 main:574] : INFO : Epoch 2043 | loss: 0.0311843 | val_loss: 0.03118 | Time: 3253.83 ms [2022-03-24 19:03:24 main:574] : INFO : Epoch 2044 | loss: 0.0311828 | val_loss: 0.0311793 | Time: 3132.17 ms [2022-03-24 19:03:27 main:574] : INFO : Epoch 2045 | loss: 0.0311816 | val_loss: 0.0311788 | Time: 3117.67 ms [2022-03-24 19:03:31 main:574] : INFO : Epoch 2046 | loss: 0.0311821 | val_loss: 0.0311789 | Time: 3124.77 ms [2022-03-24 19:03:34 main:574] : INFO : Epoch 2047 | loss: 0.0311826 | val_loss: 0.0311786 | Time: 3140.84 ms [2022-03-24 19:03:37 main:574] : INFO : Epoch 2048 | loss: 0.0311822 | val_loss: 0.0311773 | Time: 3129.73 ms [2022-03-24 19:03:37 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311773 [2022-03-24 19:03:37 main:603] : INFO : Saving end state to config to file [2022-03-24 19:03:37 main:608] : INFO : Success, exiting.. 19:03:37 (25060): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)