Task 14595136
| Name | ParityModified-1647048548-17927-3-0_0 |
| Workunit | 11617392 |
| Created | 10 Apr 2022, 8:25:59 UTC |
| Sent | 11 Apr 2022, 0:18:14 UTC |
| Report deadline | 19 Apr 2022, 0:18:14 UTC |
| Received | 19 Apr 2022, 20:37:39 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 21717 |
| Run time | 3 hours 27 min 24 sec |
| CPU time | 3 hours 16 min 55 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 4,277.48 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.60 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 2022-04-19 21:25:56 main:574] : INFO : Epoch 1677 | loss: 0.0311814 | val_loss: 0.0311814 | Time: 5786.1 ms [2022-04-19 21:26:03 main:574] : INFO : Epoch 1678 | loss: 0.0311811 | val_loss: 0.0311805 | Time: 6276.51 ms [2022-04-19 21:26:08 main:574] : INFO : Epoch 1679 | loss: 0.0311805 | val_loss: 0.0311803 | Time: 5758.3 ms [2022-04-19 21:26:15 main:574] : INFO : Epoch 1680 | loss: 0.0311797 | val_loss: 0.0311802 | Time: 6429.27 ms [2022-04-19 21:26:20 main:574] : INFO : Epoch 1681 | loss: 0.0311793 | val_loss: 0.0311795 | Time: 5608.2 ms [2022-04-19 21:26:27 main:574] : INFO : Epoch 1682 | loss: 0.0311794 | val_loss: 0.0311805 | Time: 6205.65 ms [2022-04-19 21:26:32 main:574] : INFO : Epoch 1683 | loss: 0.0311784 | val_loss: 0.0311805 | Time: 5784.03 ms [2022-04-19 21:26:39 main:574] : INFO : Epoch 1684 | loss: 0.0311788 | val_loss: 0.0311793 | Time: 6351.36 ms [2022-04-19 21:26:44 main:574] : INFO : Epoch 1685 | loss: 0.031178 | val_loss: 0.0311799 | Time: 5654.55 ms [2022-04-19 21:26:51 main:574] : INFO : Epoch 1686 | loss: 0.0311775 | val_loss: 0.0311787 | Time: 6298.74 ms [2022-04-19 21:26:57 main:574] : INFO : Epoch 1687 | loss: 0.0311767 | val_loss: 0.0311803 | Time: 5750.8 ms [2022-04-19 21:27:03 main:574] : INFO : Epoch 1688 | loss: 0.0311759 | val_loss: 0.0311797 | Time: 6175.86 ms [2022-04-19 21:27:08 main:574] : INFO : Epoch 1689 | loss: 0.0311752 | val_loss: 0.0311788 | Time: 5638.07 ms [2022-04-19 21:27:15 main:574] : INFO : Epoch 1690 | loss: 0.0311748 | val_loss: 0.0311786 | Time: 6913.65 ms [2022-04-19 21:27:21 main:574] : INFO : Epoch 1691 | loss: 0.0311745 | val_loss: 0.0311801 | Time: 5890.93 ms [2022-04-19 21:27:28 main:574] : INFO : Epoch 1692 | loss: 0.0311733 | val_loss: 0.0311778 | Time: 6253.8 ms [2022-04-19 21:27:33 main:574] : INFO : Epoch 1693 | loss: 0.0311727 | val_loss: 0.0311788 | Time: 5638.85 ms [2022-04-19 21:27:40 main:574] : INFO : Epoch 1694 | loss: 0.031173 | val_loss: 0.0311809 | Time: 6302.95 ms [2022-04-19 21:27:45 main:574] : INFO : Epoch 1695 | loss: 0.0311734 | val_loss: 0.0311761 | Time: 5783.69 ms [2022-04-19 21:27:52 main:574] : INFO : Epoch 1696 | loss: 0.0311726 | val_loss: 0.0311774 | Time: 6331.56 ms [2022-04-19 21:27:57 main:574] : INFO : Epoch 1697 | loss: 0.0311755 | val_loss: 0.0311754 | Time: 5658.33 ms [2022-04-19 21:28:04 main:574] : INFO : Epoch 1698 | loss: 0.031174 | val_loss: 0.0311744 | Time: 6191.62 ms [2022-04-19 21:28:09 main:574] : INFO : Epoch 1699 | loss: 0.031173 | val_loss: 0.0311749 | Time: 5719.99 ms [2022-04-19 21:28:15 main:574] : INFO : Epoch 1700 | loss: 0.0311718 | val_loss: 0.0311752 | Time: 6164.05 ms [2022-04-19 21:28:21 main:574] : INFO : Epoch 1701 | loss: 0.0311723 | val_loss: 0.031176 | Time: 5794.16 ms [2022-04-19 21:28:28 main:574] : INFO : Epoch 1702 | loss: 0.0311721 | val_loss: 0.0311753 | Time: 6225.46 ms [2022-04-19 21:28:33 main:574] : INFO : Epoch 1703 | loss: 0.0311708 | val_loss: 0.0311744 | Time: 5758.17 ms [2022-04-19 21:28:40 main:574] : INFO : Epoch 1704 | loss: 0.0311699 | val_loss: 0.0311743 | Time: 6273.48 ms [2022-04-19 21:28:45 main:574] : INFO : Epoch 1705 | loss: 0.0311692 | val_loss: 0.0311748 | Time: 5679.38 ms [2022-04-19 21:28:52 main:574] : INFO : Epoch 1706 | loss: 0.0311688 | val_loss: 0.0311749 | Time: 6281.3 ms [2022-04-19 21:28:58 main:574] : INFO : Epoch 1707 | loss: 0.0311688 | val_loss: 0.0311772 | Time: 5836.84 ms [2022-04-19 21:29:04 main:574] : INFO : Epoch 1708 | loss: 0.0311683 | val_loss: 0.0311754 | Time: 6152.82 ms [2022-04-19 21:29:10 main:574] : INFO : Epoch 1709 | loss: 0.0311677 | val_loss: 0.0311742 | Time: 5817.12 ms [2022-04-19 21:29:16 main:574] : INFO : Epoch 1710 | loss: 0.0311675 | val_loss: 0.0311769 | Time: 6384.78 ms [2022-04-19 21:29:22 main:574] : INFO : Epoch 1711 | loss: 0.0311672 | val_loss: 0.0311747 | Time: 5841.75 ms [2022-04-19 21:29:28 main:574] : INFO : Epoch 1712 | loss: 0.0311664 | val_loss: 0.031176 | Time: 6213.9 ms [2022-04-19 21:29:34 main:574] : INFO : Epoch 1713 | loss: 0.0311668 | val_loss: 0.0311751 | Time: 5488.39 ms [2022-04-19 21:29:40 main:574] : INFO : Epoch 1714 | loss: 0.0311657 | val_loss: 0.0311752 | Time: 6359.19 ms [2022-04-19 21:29:46 main:574] : INFO : Epoch 1715 | loss: 0.0311659 | val_loss: 0.0311748 | Time: 5858.7 ms [2022-04-19 21:30:03 main:574] : INFO : Epoch 1716 | loss: 0.0311651 | val_loss: 0.0311746 | Time: 16688.1 ms [2022-04-19 21:30:08 main:574] : INFO : Epoch 1717 | loss: 0.0311657 | val_loss: 0.0311755 | Time: 5564.62 ms [2022-04-19 21:30:14 main:574] : INFO : Epoch 1718 | loss: 0.0311661 | val_loss: 0.0311778 | Time: 6038 ms [2022-04-19 21:30:20 main:574] : INFO : Epoch 1719 | loss: 0.0311665 | val_loss: 0.0311751 | Time: 5466.93 ms [2022-04-19 21:30:26 main:574] : INFO : Epoch 1720 | loss: 0.0311661 | val_loss: 0.0311752 | Time: 6097.68 ms [2022-04-19 21:30:31 main:574] : INFO : Epoch 1721 | loss: 0.0311654 | val_loss: 0.031178 | Time: 5618.33 ms [2022-04-19 21:30:38 main:574] : INFO : Epoch 1722 | loss: 0.0311645 | val_loss: 0.0311741 | Time: 6494.73 ms [2022-04-19 21:30:44 main:574] : INFO : Epoch 1723 | loss: 0.031164 | val_loss: 0.0311738 | Time: 5657.28 ms [2022-04-19 21:30:50 main:574] : INFO : Epoch 1724 | loss: 0.0311648 | val_loss: 0.0311734 | Time: 6287.96 ms [2022-04-19 21:30:55 main:574] : INFO : Epoch 1725 | loss: 0.0311631 | val_loss: 0.0311728 | Time: 5557.07 ms [2022-04-19 21:31:02 main:574] : INFO : Epoch 1726 | loss: 0.0311622 | val_loss: 0.0311693 | Time: 6259.49 ms [2022-04-19 21:31:07 main:574] : INFO : Epoch 1727 | loss: 0.0311605 | val_loss: 0.0311706 | Time: 5667.24 ms [2022-04-19 21:31:14 main:574] : INFO : Epoch 1728 | loss: 0.0311607 | val_loss: 0.0311717 | Time: 6452.42 ms [2022-04-19 21:31:21 main:574] : INFO : Epoch 1729 | loss: 0.0311612 | val_loss: 0.0311715 | Time: 7454.59 ms [2022-04-19 21:31:27 main:574] : INFO : Epoch 1730 | loss: 0.0311608 | val_loss: 0.0311705 | Time: 5879.22 ms [2022-04-19 21:31:33 main:574] : INFO : Epoch 1731 | loss: 0.0311607 | val_loss: 0.0311732 | Time: 5915.2 ms [2022-04-19 21:31:39 main:574] : INFO : Epoch 1732 | loss: 0.0311613 | val_loss: 0.0311695 | Time: 5852.94 ms [2022-04-19 21:31:45 main:574] : INFO : Epoch 1733 | loss: 0.0311602 | val_loss: 0.0311699 | Time: 5819.41 ms [2022-04-19 21:31:51 main:574] : INFO : Epoch 1734 | loss: 0.0311604 | val_loss: 0.03117 | Time: 6051.45 ms [2022-04-19 21:31:57 main:574] : INFO : Epoch 1735 | loss: 0.0311618 | val_loss: 0.0311721 | Time: 5788.91 ms [2022-04-19 21:32:03 main:574] : INFO : Epoch 1736 | loss: 0.0311615 | val_loss: 0.0311686 | Time: 6258.73 ms [2022-04-19 21:32:09 main:574] : INFO : Epoch 1737 | loss: 0.0311604 | val_loss: 0.0311674 | Time: 6118.26 ms [2022-04-19 21:32:16 main:574] : INFO : Epoch 1738 | loss: 0.0311634 | val_loss: 0.0311773 | Time: 7011.13 ms [2022-04-19 21:32:24 main:574] : INFO : Epoch 1739 | loss: 0.0311615 | val_loss: 0.0311685 | Time: 7415.94 ms [2022-04-19 21:32:30 main:574] : INFO : Epoch 1740 | loss: 0.0311582 | val_loss: 0.0311668 | Time: 6190.02 ms [2022-04-19 21:32:36 main:574] : INFO : Epoch 1741 | loss: 0.0311566 | val_loss: 0.0311672 | Time: 5794.86 ms [2022-04-19 21:32:42 main:574] : INFO : Epoch 1742 | loss: 0.0311587 | val_loss: 0.031167 | Time: 6070.44 ms [2022-04-19 21:32:48 main:574] : INFO : Epoch 1743 | loss: 0.0311601 | val_loss: 0.0311681 | Time: 5691.71 ms [2022-04-19 21:32:54 main:574] : INFO : Epoch 1744 | loss: 0.0311598 | val_loss: 0.0311693 | Time: 5950.84 ms [2022-04-19 21:33:00 main:574] : INFO : Epoch 1745 | loss: 0.0311611 | val_loss: 0.0311683 | Time: 5943.38 ms [2022-04-19 21:33:06 main:574] : INFO : Epoch 1746 | loss: 0.0311611 | val_loss: 0.0311674 | Time: 6312.64 ms [2022-04-19 21:33:12 main:574] : INFO : Epoch 1747 | loss: 0.0311618 | val_loss: 0.0311681 | Time: 5899.71 ms [2022-04-19 21:33:19 main:574] : INFO : Epoch 1748 | loss: 0.0311608 | val_loss: 0.031167 | Time: 7288.29 ms [2022-04-19 21:33:25 main:574] : INFO : Epoch 1749 | loss: 0.0311603 | val_loss: 0.0311701 | Time: 5586.59 ms [2022-04-19 21:33:31 main:574] : INFO : Epoch 1750 | loss: 0.0311632 | val_loss: 0.0311679 | Time: 6258.09 ms [2022-04-19 21:33:37 main:574] : INFO : Epoch 1751 | loss: 0.0311618 | val_loss: 0.0311696 | Time: 5644.28 ms [2022-04-19 21:33:43 main:574] : INFO : Epoch 1752 | loss: 0.0311603 | val_loss: 0.0311651 | Time: 6300.34 ms [2022-04-19 21:33:49 main:574] : INFO : Epoch 1753 | loss: 0.0311591 | val_loss: 0.0311648 | Time: 5596.94 ms [2022-04-19 21:33:55 main:574] : INFO : Epoch 1754 | loss: 0.0311581 | val_loss: 0.0311685 | Time: 6172.25 ms [2022-04-19 21:34:00 main:574] : INFO : Epoch 1755 | loss: 0.0311585 | val_loss: 0.0311667 | Time: 5612.87 ms [2022-04-19 21:34:07 main:574] : INFO : Epoch 1756 | loss: 0.0311584 | val_loss: 0.0311657 | Time: 6151.87 ms [2022-04-19 21:34:12 main:574] : INFO : Epoch 1757 | loss: 0.0311575 | val_loss: 0.031166 | Time: 5739.21 ms [2022-04-19 21:34:19 main:574] : INFO : Epoch 1758 | loss: 0.0311579 | val_loss: 0.0311643 | Time: 7029.63 ms [2022-04-19 21:34:25 main:574] : INFO : Epoch 1759 | loss: 0.0311584 | val_loss: 0.0311675 | Time: 5609.16 ms [2022-04-19 21:34:31 main:574] : INFO : Epoch 1760 | loss: 0.0311568 | val_loss: 0.0311666 | Time: 6381.43 ms [2022-04-19 21:34:37 main:574] : INFO : Epoch 1761 | loss: 0.0311561 | val_loss: 0.0311627 | Time: 5627.38 ms [2022-04-19 21:34:43 main:574] : INFO : Epoch 1762 | loss: 0.0311564 | val_loss: 0.0311685 | Time: 6168.65 ms [2022-04-19 21:34:49 main:574] : INFO : Epoch 1763 | loss: 0.0311593 | val_loss: 0.0311673 | Time: 5626.52 ms [2022-04-19 21:34:55 main:574] : INFO : Epoch 1764 | loss: 0.0311597 | val_loss: 0.0311675 | Time: 6497.24 ms [2022-04-19 21:35:01 main:574] : INFO : Epoch 1765 | loss: 0.031158 | val_loss: 0.031166 | Time: 5762.54 ms [2022-04-19 21:35:07 main:574] : INFO : Epoch 1766 | loss: 0.0311589 | val_loss: 0.0311664 | Time: 6308.62 ms [2022-04-19 21:35:13 main:574] : INFO : Epoch 1767 | loss: 0.0311583 | val_loss: 0.0311664 | Time: 5809.03 ms [2022-04-19 21:35:20 main:574] : INFO : Epoch 1768 | loss: 0.0311578 | val_loss: 0.031169 | Time: 6848.74 ms [2022-04-19 21:35:26 main:574] : INFO : Epoch 1769 | loss: 0.0311545 | val_loss: 0.0311651 | Time: 5804.05 ms [2022-04-19 21:35:32 main:574] : INFO : Epoch 1770 | loss: 0.0311561 | val_loss: 0.0311649 | Time: 5965.73 ms [2022-04-19 21:35:38 main:574] : INFO : Epoch 1771 | loss: 0.0311573 | val_loss: 0.0311638 | Time: 5648.21 ms [2022-04-19 21:35:44 main:574] : INFO : Epoch 1772 | loss: 0.0311561 | val_loss: 0.0311627 | Time: 6150.79 ms [2022-04-19 21:35:50 main:574] : INFO : Epoch 1773 | loss: 0.0311549 | val_loss: 0.0311636 | Time: 5761.66 ms [2022-04-19 21:35:56 main:574] : INFO : Epoch 1774 | loss: 0.0311534 | val_loss: 0.0311691 | Time: 6283.77 ms [2022-04-19 21:36:02 main:574] : INFO : Epoch 1775 | loss: 0.031154 | val_loss: 0.0311653 | Time: 5723.92 ms [2022-04-19 21:36:08 main:574] : INFO : Epoch 1776 | loss: 0.0311528 | val_loss: 0.0311668 | Time: 6266.57 ms [2022-04-19 21:36:14 main:574] : INFO : Epoch 1777 | loss: 0.0311528 | val_loss: 0.0311627 | Time: 5699.97 ms [2022-04-19 21:36:20 main:574] : INFO : Epoch 1778 | loss: 0.0311516 | val_loss: 0.0311638 | Time: 6539.1 ms [2022-04-19 21:36:26 main:574] : INFO : Epoch 1779 | loss: 0.031153 | val_loss: 0.0311623 | Time: 5830.91 ms [2022-04-19 21:36:32 main:574] : INFO : Epoch 1780 | loss: 0.031155 | val_loss: 0.0311653 | Time: 6129.82 ms [2022-04-19 21:36:38 main:574] : INFO : Epoch 1781 | loss: 0.0311555 | val_loss: 0.0311616 | Time: 5642.4 ms [2022-04-19 21:36:44 main:574] : INFO : Epoch 1782 | loss: 0.0311534 | val_loss: 0.0311611 | Time: 6266.93 ms [2022-04-19 21:36:50 main:574] : INFO : Epoch 1783 | loss: 0.0311522 | val_loss: 0.0311631 | Time: 5783.37 ms [2022-04-19 21:36:56 main:574] : INFO : Epoch 1784 | loss: 0.0311513 | val_loss: 0.0311597 | Time: 6273.29 ms [2022-04-19 21:37:02 main:574] : INFO : Epoch 1785 | loss: 0.0311575 | val_loss: 0.0311597 | Time: 5580.74 ms [2022-04-19 21:37:08 main:574] : INFO : Epoch 1786 | loss: 0.0311579 | val_loss: 0.0311589 | Time: 6202.47 ms [2022-04-19 21:37:14 main:574] : INFO : Epoch 1787 | loss: 0.0311576 | val_loss: 0.0311618 | Time: 5593.89 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060) [2022-04-19 21:40:00 main:435] : INFO : Set logging level to 1 [2022-04-19 21:40:00 main:441] : INFO : Running in BOINC Client mode [2022-04-19 21:40:00 main:444] : INFO : Resolving all filenames [2022-04-19 21:40:00 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-19 21:40:00 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-19 21:40:00 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-19 21:40:00 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-19 21:40:00 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-19 21:40:00 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-19 21:40:00 main:474] : INFO : Configuration: [2022-04-19 21:40:00 main:475] : INFO : Model type: GRU [2022-04-19 21:40:00 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-19 21:40:00 main:477] : INFO : Max Epochs: 2048 [2022-04-19 21:40:00 main:478] : INFO : Batch Size: 128 [2022-04-19 21:40:00 main:479] : INFO : Learning Rate: 0.01 [2022-04-19 21:40:00 main:480] : INFO : Patience: 10 [2022-04-19 21:40:00 main:481] : INFO : Hidden Width: 12 [2022-04-19 21:40:00 main:482] : INFO : # Recurrent Layers: 4 [2022-04-19 21:40:01 main:483] : INFO : # Backend Layers: 4 [2022-04-19 21:40:01 main:484] : INFO : # Threads: 1 [2022-04-19 21:40:01 main:486] : INFO : Preparing Dataset [2022-04-19 21:40:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-19 21:40:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-19 21:40:05 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-19 21:40:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-19 21:40:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-19 21:40:05 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-19 21:40:05 main:494] : INFO : Creating Model [2022-04-19 21:40:05 main:507] : INFO : Preparing config file [2022-04-19 21:40:05 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-19 21:40:05 main:512] : INFO : Loading config [2022-04-19 21:40:05 main:514] : INFO : Loading state [2022-04-19 21:40:06 main:559] : INFO : Loading DataLoader into Memory [2022-04-19 21:40:06 main:562] : INFO : Starting Training [2022-04-19 21:40:11 main:574] : INFO : Epoch 1766 | loss: 0.0316573 | val_loss: 0.0313552 | Time: 4564.49 ms [2022-04-19 21:40:16 main:574] : INFO : Epoch 1767 | loss: 0.0312547 | val_loss: 0.0312184 | Time: 4773.9 ms [2022-04-19 21:40:22 main:574] : INFO : Epoch 1768 | loss: 0.0311798 | val_loss: 0.0311715 | Time: 5866.02 ms [2022-04-19 21:40:27 main:574] : INFO : Epoch 1769 | loss: 0.0311616 | val_loss: 0.0311658 | Time: 5547.83 ms [2022-04-19 21:40:33 main:574] : INFO : Epoch 1770 | loss: 0.0311583 | val_loss: 0.0311644 | Time: 5520.95 ms [2022-04-19 21:40:39 main:574] : INFO : Epoch 1771 | loss: 0.0311571 | val_loss: 0.0311622 | Time: 5935.45 ms [2022-04-19 21:40:44 main:574] : INFO : Epoch 1772 | loss: 0.0311564 | val_loss: 0.0311624 | Time: 5620.95 ms [2022-04-19 21:40:50 main:574] : INFO : Epoch 1773 | loss: 0.0311549 | val_loss: 0.0311631 | Time: 5717.72 ms [2022-04-19 21:40:55 main:574] : INFO : Epoch 1774 | loss: 0.0311547 | val_loss: 0.0311644 | Time: 5315.73 ms [2022-04-19 21:41:02 main:574] : INFO : Epoch 1775 | loss: 0.0311572 | val_loss: 0.031163 | Time: 6799.57 ms [2022-04-19 21:41:08 main:574] : INFO : Epoch 1776 | loss: 0.031157 | val_loss: 0.0311609 | Time: 5702.62 ms [2022-04-19 21:41:14 main:574] : INFO : Epoch 1777 | loss: 0.0311584 | val_loss: 0.0311679 | Time: 6102.4 ms [2022-04-19 21:41:20 main:574] : INFO : Epoch 1778 | loss: 0.0311548 | val_loss: 0.0311621 | Time: 5570.71 ms [2022-04-19 21:41:26 main:574] : INFO : Epoch 1779 | loss: 0.0311549 | val_loss: 0.0311614 | Time: 5950.67 ms [2022-04-19 21:41:32 main:574] : INFO : Epoch 1780 | loss: 0.031153 | val_loss: 0.0311612 | Time: 6265.55 ms [2022-04-19 21:41:40 main:574] : INFO : Epoch 1781 | loss: 0.0311515 | val_loss: 0.0311629 | Time: 8195.7 ms [2022-04-19 21:41:48 main:574] : INFO : Epoch 1782 | loss: 0.0311521 | val_loss: 0.0311611 | Time: 7448.48 ms [2022-04-19 21:41:54 main:574] : INFO : Epoch 1783 | loss: 0.0311531 | val_loss: 0.0311655 | Time: 6962.29 ms [2022-04-19 21:42:01 main:574] : INFO : Epoch 1784 | loss: 0.0311563 | val_loss: 0.0311647 | Time: 6998.12 ms [2022-04-19 21:42:08 main:574] : INFO : Epoch 1785 | loss: 0.0311524 | val_loss: 0.0311621 | Time: 6940.16 ms [2022-04-19 21:42:15 main:574] : INFO : Epoch 1786 | loss: 0.0311521 | val_loss: 0.0311628 | Time: 6953.28 ms [2022-04-19 21:42:22 main:574] : INFO : Epoch 1787 | loss: 0.0311518 | val_loss: 0.0311605 | Time: 7016.04 ms [2022-04-19 21:42:29 main:574] : INFO : Epoch 1788 | loss: 0.0311484 | val_loss: 0.0311626 | Time: 7009.31 ms [2022-04-19 21:42:37 main:574] : INFO : Epoch 1789 | loss: 0.0311492 | val_loss: 0.0311639 | Time: 7038 ms [2022-04-19 21:42:44 main:574] : INFO : Epoch 1790 | loss: 0.0311495 | val_loss: 0.0311681 | Time: 6936.28 ms [2022-04-19 21:42:51 main:574] : INFO : Epoch 1791 | loss: 0.0311478 | val_loss: 0.0311614 | Time: 7282.97 ms [2022-04-19 21:42:57 main:574] : INFO : Epoch 1792 | loss: 0.0311508 | val_loss: 0.0311665 | Time: 6478.83 ms [2022-04-19 21:43:04 main:574] : INFO : Epoch 1793 | loss: 0.0311532 | val_loss: 0.0311716 | Time: 7084.77 ms [2022-04-19 21:43:11 main:574] : INFO : Epoch 1794 | loss: 0.0311503 | val_loss: 0.0311609 | Time: 6589.2 ms [2022-04-19 21:43:18 main:574] : INFO : Epoch 1795 | loss: 0.0311478 | val_loss: 0.0311617 | Time: 6566.86 ms [2022-04-19 21:43:25 main:574] : INFO : Epoch 1796 | loss: 0.0311506 | val_loss: 0.0311601 | Time: 7169.72 ms [2022-04-19 21:43:32 main:574] : INFO : Epoch 1797 | loss: 0.0311527 | val_loss: 0.0311586 | Time: 6907.25 ms [2022-04-19 21:43:39 main:574] : INFO : Epoch 1798 | loss: 0.0311489 | val_loss: 0.031164 | Time: 7168.6 ms [2022-04-19 21:43:45 main:574] : INFO : Epoch 1799 | loss: 0.0311476 | val_loss: 0.031162 | Time: 6551.71 ms [2022-04-19 21:43:52 main:574] : INFO : Epoch 1800 | loss: 0.0311455 | val_loss: 0.0311597 | Time: 6751.34 ms [2022-04-19 21:43:59 main:574] : INFO : Epoch 1801 | loss: 0.0311457 | val_loss: 0.03116 | Time: 6828.98 ms [2022-04-19 21:44:06 main:574] : INFO : Epoch 1802 | loss: 0.0311475 | val_loss: 0.0311599 | Time: 6703.36 ms [2022-04-19 21:44:13 main:574] : INFO : Epoch 1803 | loss: 0.0311469 | val_loss: 0.0311597 | Time: 6962.23 ms [2022-04-19 21:44:20 main:574] : INFO : Epoch 1804 | loss: 0.031145 | val_loss: 0.0311616 | Time: 6816.17 ms [2022-04-19 21:44:26 main:574] : INFO : Epoch 1805 | loss: 0.0311425 | val_loss: 0.0311576 | Time: 6446.57 ms [2022-04-19 21:44:34 main:574] : INFO : Epoch 1806 | loss: 0.0311429 | val_loss: 0.0311648 | Time: 7499.92 ms [2022-04-19 21:44:40 main:574] : INFO : Epoch 1807 | loss: 0.0311451 | val_loss: 0.0311641 | Time: 6710.76 ms [2022-04-19 21:44:48 main:574] : INFO : Epoch 1808 | loss: 0.0311503 | val_loss: 0.0311732 | Time: 7046.66 ms [2022-04-19 21:44:54 main:574] : INFO : Epoch 1809 | loss: 0.0311449 | val_loss: 0.031159 | Time: 6921.68 ms [2022-04-19 21:45:12 main:574] : INFO : Epoch 1810 | loss: 0.0311442 | val_loss: 0.0311574 | Time: 17049.6 ms [2022-04-19 21:45:18 main:574] : INFO : Epoch 1811 | loss: 0.0311452 | val_loss: 0.0311579 | Time: 6321.17 ms [2022-04-19 21:45:24 main:574] : INFO : Epoch 1812 | loss: 0.031146 | val_loss: 0.0311554 | Time: 6222.88 ms [2022-04-19 21:45:31 main:574] : INFO : Epoch 1813 | loss: 0.0311451 | val_loss: 0.0311565 | Time: 6630.91 ms [2022-04-19 21:45:38 main:574] : INFO : Epoch 1814 | loss: 0.031144 | val_loss: 0.0311592 | Time: 6912.88 ms [2022-04-19 21:45:45 main:574] : INFO : Epoch 1815 | loss: 0.0311452 | val_loss: 0.0311567 | Time: 6911.45 ms [2022-04-19 21:45:52 main:574] : INFO : Epoch 1816 | loss: 0.0311487 | val_loss: 0.0311607 | Time: 6877.12 ms [2022-04-19 21:45:58 main:574] : INFO : Epoch 1817 | loss: 0.0311494 | val_loss: 0.0311565 | Time: 6711.04 ms [2022-04-19 21:46:05 main:574] : INFO : Epoch 1818 | loss: 0.0311465 | val_loss: 0.0311572 | Time: 6547.94 ms [2022-04-19 21:46:12 main:574] : INFO : Epoch 1819 | loss: 0.0311461 | val_loss: 0.0311589 | Time: 6901.73 ms [2022-04-19 21:46:19 main:574] : INFO : Epoch 1820 | loss: 0.0311458 | val_loss: 0.0311585 | Time: 6827.32 ms [2022-04-19 21:46:25 main:574] : INFO : Epoch 1821 | loss: 0.0311533 | val_loss: 0.0311651 | Time: 6565.2 ms [2022-04-19 21:46:32 main:574] : INFO : Epoch 1822 | loss: 0.0311549 | val_loss: 0.0311627 | Time: 6968.81 ms [2022-04-19 21:46:39 main:574] : INFO : Epoch 1823 | loss: 0.0311559 | val_loss: 0.0311678 | Time: 6798.12 ms [2022-04-19 21:46:46 main:574] : INFO : Epoch 1824 | loss: 0.0311529 | val_loss: 0.0311594 | Time: 6692.3 ms [2022-04-19 21:46:53 main:574] : INFO : Epoch 1825 | loss: 0.031148 | val_loss: 0.031159 | Time: 6935.16 ms [2022-04-19 21:46:59 main:574] : INFO : Epoch 1826 | loss: 0.0311462 | val_loss: 0.0311606 | Time: 6559.08 ms [2022-04-19 21:47:06 main:574] : INFO : Epoch 1827 | loss: 0.0311439 | val_loss: 0.0311558 | Time: 6743.71 ms [2022-04-19 21:47:13 main:574] : INFO : Epoch 1828 | loss: 0.031143 | val_loss: 0.0311573 | Time: 6934.16 ms [2022-04-19 21:47:19 main:574] : INFO : Epoch 1829 | loss: 0.0311426 | val_loss: 0.0311594 | Time: 6387.14 ms [2022-04-19 21:47:26 main:574] : INFO : Epoch 1830 | loss: 0.0311427 | val_loss: 0.0311586 | Time: 6737.32 ms [2022-04-19 21:47:33 main:574] : INFO : Epoch 1831 | loss: 0.031144 | val_loss: 0.0311567 | Time: 6995.23 ms [2022-04-19 21:47:40 main:574] : INFO : Epoch 1832 | loss: 0.0311424 | val_loss: 0.0311551 | Time: 6561.62 ms [2022-04-19 21:47:47 main:574] : INFO : Epoch 1833 | loss: 0.031145 | val_loss: 0.0311592 | Time: 7072.81 ms [2022-04-19 21:47:54 main:574] : INFO : Epoch 1834 | loss: 0.0311426 | val_loss: 0.0311591 | Time: 7159.53 ms [2022-04-19 21:48:01 main:574] : INFO : Epoch 1835 | loss: 0.0311427 | val_loss: 0.0311602 | Time: 7127.55 ms [2022-04-19 21:48:08 main:574] : INFO : Epoch 1836 | loss: 0.0311435 | val_loss: 0.031165 | Time: 6579.07 ms [2022-04-19 21:48:14 main:574] : INFO : Epoch 1837 | loss: 0.0311451 | val_loss: 0.0311632 | Time: 6759.88 ms [2022-04-19 21:48:22 main:574] : INFO : Epoch 1838 | loss: 0.0311455 | val_loss: 0.03116 | Time: 7128.4 ms [2022-04-19 21:48:28 main:574] : INFO : Epoch 1839 | loss: 0.0311446 | val_loss: 0.0311643 | Time: 6549.44 ms [2022-04-19 21:48:35 main:574] : INFO : Epoch 1840 | loss: 0.0311493 | val_loss: 0.0311609 | Time: 7195.62 ms [2022-04-19 21:48:42 main:574] : INFO : Epoch 1841 | loss: 0.0311486 | val_loss: 0.0311587 | Time: 6528.15 ms [2022-04-19 21:48:49 main:574] : INFO : Epoch 1842 | loss: 0.0311453 | val_loss: 0.0311557 | Time: 6843.91 ms [2022-04-19 21:48:56 main:574] : INFO : Epoch 1843 | loss: 0.0311476 | val_loss: 0.031159 | Time: 6953.64 ms [2022-04-19 21:49:03 main:574] : INFO : Epoch 1844 | loss: 0.0311471 | val_loss: 0.0311585 | Time: 6652.55 ms [2022-04-19 21:49:10 main:574] : INFO : Epoch 1845 | loss: 0.0311457 | val_loss: 0.0311602 | Time: 7040.85 ms [2022-04-19 21:49:16 main:574] : INFO : Epoch 1846 | loss: 0.0311476 | val_loss: 0.0311568 | Time: 6775.25 ms [2022-04-19 21:49:23 main:574] : INFO : Epoch 1847 | loss: 0.0311438 | val_loss: 0.0311619 | Time: 6723.61 ms [2022-04-19 21:49:30 main:574] : INFO : Epoch 1848 | loss: 0.0311432 | val_loss: 0.0311558 | Time: 7083.22 ms [2022-04-19 21:49:37 main:574] : INFO : Epoch 1849 | loss: 0.0311463 | val_loss: 0.0311554 | Time: 6481.7 ms [2022-04-19 21:49:44 main:574] : INFO : Epoch 1850 | loss: 0.0311453 | val_loss: 0.031161 | Time: 6989.59 ms [2022-04-19 21:49:51 main:574] : INFO : Epoch 1851 | loss: 0.0311447 | val_loss: 0.0311575 | Time: 6865.27 ms [2022-04-19 21:49:57 main:574] : INFO : Epoch 1852 | loss: 0.0311433 | val_loss: 0.0311578 | Time: 6681.81 ms [2022-04-19 21:50:04 main:574] : INFO : Epoch 1853 | loss: 0.0311415 | val_loss: 0.0311614 | Time: 6979.35 ms [2022-04-19 21:50:11 main:574] : INFO : Epoch 1854 | loss: 0.03114 | val_loss: 0.031158 | Time: 6552.22 ms [2022-04-19 21:50:18 main:574] : INFO : Epoch 1855 | loss: 0.031144 | val_loss: 0.0311583 | Time: 7163.96 ms [2022-04-19 21:50:25 main:574] : INFO : Epoch 1856 | loss: 0.0311412 | val_loss: 0.031166 | Time: 6605.57 ms [2022-04-19 21:50:31 main:574] : INFO : Epoch 1857 | loss: 0.0311408 | val_loss: 0.0311545 | Time: 6578.45 ms [2022-04-19 21:50:38 main:574] : INFO : Epoch 1858 | loss: 0.0311417 | val_loss: 0.0311621 | Time: 6843.62 ms [2022-04-19 21:50:45 main:574] : INFO : Epoch 1859 | loss: 0.0311396 | val_loss: 0.0311622 | Time: 6870.86 ms [2022-04-19 21:50:52 main:574] : INFO : Epoch 1860 | loss: 0.031138 | val_loss: 0.0311567 | Time: 6609.92 ms [2022-04-19 21:50:59 main:574] : INFO : Epoch 1861 | loss: 0.0311382 | val_loss: 0.0311566 | Time: 7061.95 ms [2022-04-19 21:51:06 main:574] : INFO : Epoch 1862 | loss: 0.0311402 | val_loss: 0.0311626 | Time: 6956.34 ms [2022-04-19 21:51:13 main:574] : INFO : Epoch 1863 | loss: 0.0311402 | val_loss: 0.0311599 | Time: 7103.98 ms [2022-04-19 21:51:19 main:574] : INFO : Epoch 1864 | loss: 0.0311372 | val_loss: 0.0311546 | Time: 6646.17 ms [2022-04-19 21:51:26 main:574] : INFO : Epoch 1865 | loss: 0.03114 | val_loss: 0.0311652 | Time: 6541.45 ms [2022-04-19 21:51:33 main:574] : INFO : Epoch 1866 | loss: 0.0311418 | val_loss: 0.0311535 | Time: 7307.94 ms [2022-04-19 21:51:40 main:574] : INFO : Epoch 1867 | loss: 0.0311379 | val_loss: 0.0311557 | Time: 6580.6 ms [2022-04-19 21:51:47 main:574] : INFO : Epoch 1868 | loss: 0.0311367 | val_loss: 0.0311556 | Time: 7299.24 ms [2022-04-19 21:51:54 main:574] : INFO : Epoch 1869 | loss: 0.0311356 | val_loss: 0.0311592 | Time: 6695.02 ms [2022-04-19 21:52:01 main:574] : INFO : Epoch 1870 | loss: 0.0311347 | val_loss: 0.0311598 | Time: 6707.36 ms [2022-04-19 21:52:08 main:574] : INFO : Epoch 1871 | loss: 0.0311362 | val_loss: 0.0311592 | Time: 7097.81 ms [2022-04-19 21:52:14 main:574] : INFO : Epoch 1872 | loss: 0.0311392 | val_loss: 0.031158 | Time: 6624.03 ms [2022-04-19 21:52:21 main:574] : INFO : Epoch 1873 | loss: 0.0311423 | val_loss: 0.031161 | Time: 6715.14 ms [2022-04-19 21:52:28 main:574] : INFO : Epoch 1874 | loss: 0.0311384 | val_loss: 0.0311561 | Time: 7026.15 ms [2022-04-19 21:52:35 main:574] : INFO : Epoch 1875 | loss: 0.0311375 | val_loss: 0.0311556 | Time: 6716.22 ms [2022-04-19 21:52:42 main:574] : INFO : Epoch 1876 | loss: 0.0311381 | val_loss: 0.0311527 | Time: 7087.83 ms [2022-04-19 21:52:49 main:574] : INFO : Epoch 1877 | loss: 0.0311425 | val_loss: 0.0311548 | Time: 6517.66 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060) [2022-04-19 21:58:41 main:435] : INFO : Set logging level to 1 [2022-04-19 21:58:41 main:441] : INFO : Running in BOINC Client mode [2022-04-19 21:58:41 main:444] : INFO : Resolving all filenames [2022-04-19 21:58:41 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-19 21:58:41 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-19 21:58:41 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-19 21:58:41 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-19 21:58:41 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-19 21:58:41 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-19 21:58:41 main:474] : INFO : Configuration: [2022-04-19 21:58:41 main:475] : INFO : Model type: GRU [2022-04-19 21:58:41 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-19 21:58:41 main:477] : INFO : Max Epochs: 2048 [2022-04-19 21:58:42 main:478] : INFO : Batch Size: 128 [2022-04-19 21:58:42 main:479] : INFO : Learning Rate: 0.01 [2022-04-19 21:58:42 main:480] : INFO : Patience: 10 [2022-04-19 21:58:42 main:481] : INFO : Hidden Width: 12 [2022-04-19 21:58:42 main:482] : INFO : # Recurrent Layers: 4 [2022-04-19 21:58:42 main:483] : INFO : # Backend Layers: 4 [2022-04-19 21:58:42 main:484] : INFO : # Threads: 1 [2022-04-19 21:58:42 main:486] : INFO : Preparing Dataset [2022-04-19 21:58:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-19 21:58:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-19 21:58:45 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-19 21:58:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-19 21:58:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-19 21:58:46 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-19 21:58:46 main:494] : INFO : Creating Model [2022-04-19 21:58:46 main:507] : INFO : Preparing config file [2022-04-19 21:58:46 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-19 21:58:46 main:512] : INFO : Loading config [2022-04-19 21:58:46 main:514] : INFO : Loading state [2022-04-19 21:58:47 main:559] : INFO : Loading DataLoader into Memory [2022-04-19 21:58:47 main:562] : INFO : Starting Training [2022-04-19 21:58:52 main:574] : INFO : Epoch 1855 | loss: 0.031346 | val_loss: 0.0312128 | Time: 5110.94 ms [2022-04-19 21:58:57 main:574] : INFO : Epoch 1856 | loss: 0.0311843 | val_loss: 0.0311605 | Time: 5177.29 ms [2022-04-19 21:59:03 main:574] : INFO : Epoch 1857 | loss: 0.0311469 | val_loss: 0.0311556 | Time: 6128.99 ms [2022-04-19 21:59:09 main:574] : INFO : Epoch 1858 | loss: 0.0311441 | val_loss: 0.0311592 | Time: 6046.14 ms [2022-04-19 21:59:15 main:574] : INFO : Epoch 1859 | loss: 0.0311415 | val_loss: 0.031158 | Time: 5949.2 ms [2022-04-19 21:59:22 main:574] : INFO : Epoch 1860 | loss: 0.0311403 | val_loss: 0.0311511 | Time: 6259.47 ms [2022-04-19 21:59:28 main:574] : INFO : Epoch 1861 | loss: 0.0311396 | val_loss: 0.0311625 | Time: 6419.78 ms [2022-04-19 21:59:34 main:574] : INFO : Epoch 1862 | loss: 0.0311402 | val_loss: 0.0311557 | Time: 6323.54 ms [2022-04-19 21:59:41 main:574] : INFO : Epoch 1863 | loss: 0.03114 | val_loss: 0.0311575 | Time: 6401.76 ms [2022-04-19 21:59:47 main:574] : INFO : Epoch 1864 | loss: 0.0311371 | val_loss: 0.0311508 | Time: 6519.95 ms [2022-04-19 21:59:54 main:574] : INFO : Epoch 1865 | loss: 0.0311359 | val_loss: 0.0311556 | Time: 6527.34 ms [2022-04-19 22:00:01 main:574] : INFO : Epoch 1866 | loss: 0.0311385 | val_loss: 0.0311606 | Time: 6774.44 ms [2022-04-19 22:00:07 main:574] : INFO : Epoch 1867 | loss: 0.0311402 | val_loss: 0.0311519 | Time: 6647.89 ms [2022-04-19 22:00:24 main:574] : INFO : Epoch 1868 | loss: 0.0311402 | val_loss: 0.0311557 | Time: 16187.8 ms [2022-04-19 22:00:30 main:574] : INFO : Epoch 1869 | loss: 0.031139 | val_loss: 0.0311516 | Time: 6440.77 ms [2022-04-19 22:00:37 main:574] : INFO : Epoch 1870 | loss: 0.0311378 | val_loss: 0.0311519 | Time: 7091.18 ms [2022-04-19 22:00:44 main:574] : INFO : Epoch 1871 | loss: 0.0311394 | val_loss: 0.0311545 | Time: 6516.3 ms [2022-04-19 22:00:51 main:574] : INFO : Epoch 1872 | loss: 0.0311389 | val_loss: 0.031154 | Time: 7380.52 ms [2022-04-19 22:00:58 main:574] : INFO : Epoch 1873 | loss: 0.0311388 | val_loss: 0.031152 | Time: 6591.21 ms [2022-04-19 22:01:04 main:574] : INFO : Epoch 1874 | loss: 0.0311355 | val_loss: 0.031153 | Time: 6439.63 ms [2022-04-19 22:01:11 main:574] : INFO : Epoch 1875 | loss: 0.0311384 | val_loss: 0.0311578 | Time: 6910.62 ms [2022-04-19 22:01:18 main:574] : INFO : Epoch 1876 | loss: 0.031139 | val_loss: 0.0311563 | Time: 6787.96 ms [2022-04-19 22:01:24 main:574] : INFO : Epoch 1877 | loss: 0.0311406 | val_loss: 0.0311572 | Time: 6326.15 ms [2022-04-19 22:01:31 main:574] : INFO : Epoch 1878 | loss: 0.0311424 | val_loss: 0.031155 | Time: 6631.25 ms [2022-04-19 22:01:38 main:574] : INFO : Epoch 1879 | loss: 0.0311399 | val_loss: 0.031154 | Time: 6907.76 ms [2022-04-19 22:01:44 main:574] : INFO : Epoch 1880 | loss: 0.0311408 | val_loss: 0.0311527 | Time: 6334.19 ms [2022-04-19 22:01:51 main:574] : INFO : Epoch 1881 | loss: 0.0311434 | val_loss: 0.031158 | Time: 6593.71 ms [2022-04-19 22:01:58 main:574] : INFO : Epoch 1882 | loss: 0.0311408 | val_loss: 0.0311585 | Time: 6991.18 ms [2022-04-19 22:02:06 main:574] : INFO : Epoch 1883 | loss: 0.0311359 | val_loss: 0.0311567 | Time: 8610.21 ms [2022-04-19 22:02:16 main:574] : INFO : Epoch 1884 | loss: 0.0311356 | val_loss: 0.0311596 | Time: 9480.68 ms [2022-04-19 22:02:26 main:574] : INFO : Epoch 1885 | loss: 0.031134 | val_loss: 0.0311547 | Time: 9702.14 ms [2022-04-19 22:02:36 main:574] : INFO : Epoch 1886 | loss: 0.031134 | val_loss: 0.0311533 | Time: 10303.8 ms [2022-04-19 22:02:45 main:574] : INFO : Epoch 1887 | loss: 0.0311388 | val_loss: 0.0311516 | Time: 9482.39 ms [2022-04-19 22:02:55 main:574] : INFO : Epoch 1888 | loss: 0.0311382 | val_loss: 0.0311602 | Time: 9509.92 ms [2022-04-19 22:03:04 main:574] : INFO : Epoch 1889 | loss: 0.0311378 | val_loss: 0.0311598 | Time: 9242.53 ms [2022-04-19 22:03:14 main:574] : INFO : Epoch 1890 | loss: 0.0311412 | val_loss: 0.0311651 | Time: 9594.38 ms [2022-04-19 22:03:23 main:574] : INFO : Epoch 1891 | loss: 0.0311353 | val_loss: 0.0311517 | Time: 9438.03 ms [2022-04-19 22:03:33 main:574] : INFO : Epoch 1892 | loss: 0.0311376 | val_loss: 0.0311506 | Time: 9540.61 ms [2022-04-19 22:03:43 main:574] : INFO : Epoch 1893 | loss: 0.0311342 | val_loss: 0.0311542 | Time: 9621.95 ms [2022-04-19 22:03:52 main:574] : INFO : Epoch 1894 | loss: 0.0311337 | val_loss: 0.031151 | Time: 9660.89 ms [2022-04-19 22:04:02 main:574] : INFO : Epoch 1895 | loss: 0.0311387 | val_loss: 0.0311476 | Time: 9251.64 ms [2022-04-19 22:04:11 main:574] : INFO : Epoch 1896 | loss: 0.031137 | val_loss: 0.0311534 | Time: 9453.61 ms [2022-04-19 22:04:21 main:574] : INFO : Epoch 1897 | loss: 0.0311327 | val_loss: 0.0311522 | Time: 9519.12 ms [2022-04-19 22:04:30 main:574] : INFO : Epoch 1898 | loss: 0.0311347 | val_loss: 0.0311648 | Time: 9615.01 ms [2022-04-19 22:04:40 main:574] : INFO : Epoch 1899 | loss: 0.0311371 | val_loss: 0.0311586 | Time: 9559.32 ms [2022-04-19 22:04:49 main:574] : INFO : Epoch 1900 | loss: 0.0311338 | val_loss: 0.0311522 | Time: 9453.33 ms [2022-04-19 22:04:59 main:574] : INFO : Epoch 1901 | loss: 0.031133 | val_loss: 0.0311556 | Time: 9462.36 ms [2022-04-19 22:05:08 main:574] : INFO : Epoch 1902 | loss: 0.0311347 | val_loss: 0.0311554 | Time: 9246.78 ms [2022-04-19 22:05:18 main:574] : INFO : Epoch 1903 | loss: 0.0311361 | val_loss: 0.0311573 | Time: 9494.8 ms [2022-04-19 22:05:27 main:574] : INFO : Epoch 1904 | loss: 0.0311374 | val_loss: 0.0311569 | Time: 9482.57 ms [2022-04-19 22:05:36 main:574] : INFO : Epoch 1905 | loss: 0.0311367 | val_loss: 0.031151 | Time: 9354.79 ms [2022-04-19 22:05:46 main:574] : INFO : Epoch 1906 | loss: 0.0311351 | val_loss: 0.0311491 | Time: 9377.49 ms [2022-04-19 22:05:55 main:574] : INFO : Epoch 1907 | loss: 0.0311343 | val_loss: 0.0311507 | Time: 9460.74 ms [2022-04-19 22:06:05 main:574] : INFO : Epoch 1908 | loss: 0.0311345 | val_loss: 0.0311524 | Time: 9399.61 ms [2022-04-19 22:06:14 main:574] : INFO : Epoch 1909 | loss: 0.0311379 | val_loss: 0.0311528 | Time: 9531.1 ms [2022-04-19 22:06:24 main:574] : INFO : Epoch 1910 | loss: 0.03114 | val_loss: 0.0311533 | Time: 9459.55 ms [2022-04-19 22:06:33 main:574] : INFO : Epoch 1911 | loss: 0.0311382 | val_loss: 0.0311521 | Time: 9345.97 ms [2022-04-19 22:06:43 main:574] : INFO : Epoch 1912 | loss: 0.0311414 | val_loss: 0.0311568 | Time: 9563.03 ms [2022-04-19 22:06:52 main:574] : INFO : Epoch 1913 | loss: 0.0311396 | val_loss: 0.0311547 | Time: 9544.37 ms [2022-04-19 22:07:02 main:574] : INFO : Epoch 1914 | loss: 0.0311385 | val_loss: 0.0311519 | Time: 9738 ms [2022-04-19 22:07:11 main:574] : INFO : Epoch 1915 | loss: 0.0311393 | val_loss: 0.031153 | Time: 9224.33 ms [2022-04-19 22:07:21 main:574] : INFO : Epoch 1916 | loss: 0.0311376 | val_loss: 0.0311536 | Time: 9690.48 ms [2022-04-19 22:07:30 main:574] : INFO : Epoch 1917 | loss: 0.0311354 | val_loss: 0.0311522 | Time: 9175.44 ms [2022-04-19 22:07:40 main:574] : INFO : Epoch 1918 | loss: 0.0311349 | val_loss: 0.0311537 | Time: 9272.06 ms [2022-04-19 22:07:49 main:574] : INFO : Epoch 1919 | loss: 0.0311358 | val_loss: 0.0311528 | Time: 9433.11 ms [2022-04-19 22:07:58 main:574] : INFO : Epoch 1920 | loss: 0.0311377 | val_loss: 0.0311534 | Time: 9328.78 ms [2022-04-19 22:08:08 main:574] : INFO : Epoch 1921 | loss: 0.0311358 | val_loss: 0.0311528 | Time: 9232.02 ms [2022-04-19 22:08:17 main:574] : INFO : Epoch 1922 | loss: 0.0311333 | val_loss: 0.0311511 | Time: 9386.41 ms [2022-04-19 22:08:26 main:574] : INFO : Epoch 1923 | loss: 0.0311321 | val_loss: 0.0311495 | Time: 9440.56 ms [2022-04-19 22:08:35 main:574] : INFO : Epoch 1924 | loss: 0.031132 | val_loss: 0.0311519 | Time: 8921.76 ms [2022-04-19 22:08:45 main:574] : INFO : Epoch 1925 | loss: 0.0311317 | val_loss: 0.0311523 | Time: 9228.87 ms [2022-04-19 22:08:54 main:574] : INFO : Epoch 1926 | loss: 0.0311331 | val_loss: 0.031152 | Time: 9258.84 ms [2022-04-19 22:09:03 main:574] : INFO : Epoch 1927 | loss: 0.0311341 | val_loss: 0.0311516 | Time: 9496.53 ms [2022-04-19 22:09:13 main:574] : INFO : Epoch 1928 | loss: 0.031133 | val_loss: 0.0311512 | Time: 9068.62 ms [2022-04-19 22:09:22 main:574] : INFO : Epoch 1929 | loss: 0.0311359 | val_loss: 0.0311609 | Time: 9220.1 ms [2022-04-19 22:09:31 main:574] : INFO : Epoch 1930 | loss: 0.0311353 | val_loss: 0.0311545 | Time: 9106.6 ms [2022-04-19 22:09:40 main:574] : INFO : Epoch 1931 | loss: 0.03114 | val_loss: 0.0311527 | Time: 9150.61 ms [2022-04-19 22:09:49 main:574] : INFO : Epoch 1932 | loss: 0.0311355 | val_loss: 0.0311549 | Time: 9226.32 ms [2022-04-19 22:09:59 main:574] : INFO : Epoch 1933 | loss: 0.0311333 | val_loss: 0.031157 | Time: 9331.36 ms [2022-04-19 22:10:08 main:574] : INFO : Epoch 1934 | loss: 0.0311328 | val_loss: 0.0311562 | Time: 9457.91 ms [2022-04-19 22:10:17 main:574] : INFO : Epoch 1935 | loss: 0.0311316 | val_loss: 0.0311527 | Time: 9252.05 ms [2022-04-19 22:10:27 main:574] : INFO : Epoch 1936 | loss: 0.0311303 | val_loss: 0.0311516 | Time: 9209.32 ms [2022-04-19 22:10:36 main:574] : INFO : Epoch 1937 | loss: 0.0311296 | val_loss: 0.031154 | Time: 9337.7 ms [2022-04-19 22:10:45 main:574] : INFO : Epoch 1938 | loss: 0.0311287 | val_loss: 0.0311551 | Time: 9274.99 ms [2022-04-19 22:10:55 main:574] : INFO : Epoch 1939 | loss: 0.0311299 | val_loss: 0.0311527 | Time: 9243.2 ms [2022-04-19 22:11:04 main:574] : INFO : Epoch 1940 | loss: 0.0311291 | val_loss: 0.0311573 | Time: 9193.29 ms [2022-04-19 22:11:13 main:574] : INFO : Epoch 1941 | loss: 0.0311337 | val_loss: 0.0311535 | Time: 9424.07 ms [2022-04-19 22:11:23 main:574] : INFO : Epoch 1942 | loss: 0.0311333 | val_loss: 0.0311567 | Time: 9267.06 ms [2022-04-19 22:11:32 main:574] : INFO : Epoch 1943 | loss: 0.0311313 | val_loss: 0.0311558 | Time: 9293.54 ms [2022-04-19 22:11:41 main:574] : INFO : Epoch 1944 | loss: 0.0311323 | val_loss: 0.0311555 | Time: 9509.56 ms [2022-04-19 22:11:50 main:574] : INFO : Epoch 1945 | loss: 0.03114 | val_loss: 0.0311611 | Time: 9101.11 ms [2022-04-19 22:12:00 main:574] : INFO : Epoch 1946 | loss: 0.0311392 | val_loss: 0.031157 | Time: 9211.24 ms [2022-04-19 22:12:09 main:574] : INFO : Epoch 1947 | loss: 0.0311367 | val_loss: 0.0311572 | Time: 9470.15 ms [2022-04-19 22:12:19 main:574] : INFO : Epoch 1948 | loss: 0.0311339 | val_loss: 0.0311616 | Time: 9487.17 ms [2022-04-19 22:12:28 main:574] : INFO : Epoch 1949 | loss: 0.0311399 | val_loss: 0.0311646 | Time: 9143.02 ms [2022-04-19 22:12:37 main:574] : INFO : Epoch 1950 | loss: 0.0311376 | val_loss: 0.0311541 | Time: 9519.7 ms [2022-04-19 22:12:47 main:574] : INFO : Epoch 1951 | loss: 0.0311349 | val_loss: 0.0311536 | Time: 9171.45 ms [2022-04-19 22:12:56 main:574] : INFO : Epoch 1952 | loss: 0.0311325 | val_loss: 0.0311517 | Time: 9235.45 ms [2022-04-19 22:13:05 main:574] : INFO : Epoch 1953 | loss: 0.0311314 | val_loss: 0.0311549 | Time: 9405.07 ms [2022-04-19 22:13:14 main:574] : INFO : Epoch 1954 | loss: 0.0311303 | val_loss: 0.0311585 | Time: 9190.09 ms [2022-04-19 22:13:24 main:574] : INFO : Epoch 1955 | loss: 0.0311286 | val_loss: 0.031154 | Time: 9218.43 ms [2022-04-19 22:13:33 main:574] : INFO : Epoch 1956 | loss: 0.0311279 | val_loss: 0.0311528 | Time: 9287.54 ms [2022-04-19 22:13:42 main:574] : INFO : Epoch 1957 | loss: 0.0311263 | val_loss: 0.031153 | Time: 9396.14 ms [2022-04-19 22:13:52 main:574] : INFO : Epoch 1958 | loss: 0.0311262 | val_loss: 0.0311551 | Time: 9360.41 ms [2022-04-19 22:14:01 main:574] : INFO : Epoch 1959 | loss: 0.0311275 | val_loss: 0.0311572 | Time: 9533.91 ms [2022-04-19 22:14:11 main:574] : INFO : Epoch 1960 | loss: 0.0311325 | val_loss: 0.0311566 | Time: 9539.53 ms [2022-04-19 22:14:20 main:574] : INFO : Epoch 1961 | loss: 0.031134 | val_loss: 0.0311532 | Time: 9532.23 ms [2022-04-19 22:14:30 main:574] : INFO : Epoch 1962 | loss: 0.0311355 | val_loss: 0.0311531 | Time: 9587.08 ms [2022-04-19 22:14:40 main:574] : INFO : Epoch 1963 | loss: 0.0311283 | val_loss: 0.0311523 | Time: 9485.48 ms [2022-04-19 22:14:49 main:574] : INFO : Epoch 1964 | loss: 0.0311293 | val_loss: 0.0311515 | Time: 9446.1 ms [2022-04-19 22:14:59 main:574] : INFO : Epoch 1965 | loss: 0.031132 | val_loss: 0.0311542 | Time: 9637.16 ms [2022-04-19 22:15:08 main:574] : INFO : Epoch 1966 | loss: 0.0311434 | val_loss: 0.0311593 | Time: 9545.71 ms [2022-04-19 22:15:28 main:574] : INFO : Epoch 1967 | loss: 0.0311426 | val_loss: 0.0311568 | Time: 19574.8 ms [2022-04-19 22:15:36 main:574] : INFO : Epoch 1968 | loss: 0.0311409 | val_loss: 0.0311567 | Time: 8565.38 ms [2022-04-19 22:15:45 main:574] : INFO : Epoch 1969 | loss: 0.0311393 | val_loss: 0.0311552 | Time: 8538.1 ms [2022-04-19 22:15:54 main:574] : INFO : Epoch 1970 | loss: 0.0311374 | val_loss: 0.0311575 | Time: 9408.37 ms [2022-04-19 22:16:04 main:574] : INFO : Epoch 1971 | loss: 0.0311349 | val_loss: 0.0311555 | Time: 9790.63 ms [2022-04-19 22:16:14 main:574] : INFO : Epoch 1972 | loss: 0.0311371 | val_loss: 0.0311573 | Time: 9546.65 ms [2022-04-19 22:16:24 main:574] : INFO : Epoch 1973 | loss: 0.0311377 | val_loss: 0.0311583 | Time: 10220.5 ms [2022-04-19 22:16:34 main:574] : INFO : Epoch 1974 | loss: 0.0311393 | val_loss: 0.0311592 | Time: 9686.55 ms [2022-04-19 22:16:44 main:574] : INFO : Epoch 1975 | loss: 0.0311387 | val_loss: 0.0311613 | Time: 9993.06 ms [2022-04-19 22:16:54 main:574] : INFO : Epoch 1976 | loss: 0.0311393 | val_loss: 0.0311555 | Time: 9632.48 ms [2022-04-19 22:17:03 main:574] : INFO : Epoch 1977 | loss: 0.0311371 | val_loss: 0.031159 | Time: 9410.73 ms [2022-04-19 22:17:13 main:574] : INFO : Epoch 1978 | loss: 0.0311382 | val_loss: 0.0311531 | Time: 9711.44 ms [2022-04-19 22:17:22 main:574] : INFO : Epoch 1979 | loss: 0.0311361 | val_loss: 0.0311523 | Time: 9654.34 ms [2022-04-19 22:17:32 main:574] : INFO : Epoch 1980 | loss: 0.0311349 | val_loss: 0.0311552 | Time: 9440.02 ms [2022-04-19 22:17:41 main:574] : INFO : Epoch 1981 | loss: 0.0311335 | val_loss: 0.0311534 | Time: 9439.12 ms [2022-04-19 22:17:51 main:574] : INFO : Epoch 1982 | loss: 0.0311335 | val_loss: 0.0311528 | Time: 9423.36 ms [2022-04-19 22:18:00 main:574] : INFO : Epoch 1983 | loss: 0.0311335 | val_loss: 0.0311578 | Time: 9613.93 ms [2022-04-19 22:18:10 main:574] : INFO : Epoch 1984 | loss: 0.0311325 | val_loss: 0.0311527 | Time: 9614.2 ms [2022-04-19 22:18:20 main:574] : INFO : Epoch 1985 | loss: 0.0311339 | val_loss: 0.0311548 | Time: 9602.22 ms [2022-04-19 22:18:29 main:574] : INFO : Epoch 1986 | loss: 0.0311309 | val_loss: 0.0311524 | Time: 9488.9 ms [2022-04-19 22:18:39 main:574] : INFO : Epoch 1987 | loss: 0.0311357 | val_loss: 0.0311544 | Time: 9817.69 ms [2022-04-19 22:18:49 main:574] : INFO : Epoch 1988 | loss: 0.0311423 | val_loss: 0.0311567 | Time: 9773.27 ms [2022-04-19 22:18:58 main:574] : INFO : Epoch 1989 | loss: 0.0311397 | val_loss: 0.0311631 | Time: 9511.21 ms [2022-04-19 22:19:08 main:574] : INFO : Epoch 1990 | loss: 0.0311382 | val_loss: 0.031156 | Time: 9849.16 ms [2022-04-19 22:19:18 main:574] : INFO : Epoch 1991 | loss: 0.0311348 | val_loss: 0.0311612 | Time: 9766.72 ms [2022-04-19 22:19:28 main:574] : INFO : Epoch 1992 | loss: 0.031134 | val_loss: 0.031154 | Time: 9598.29 ms [2022-04-19 22:19:37 main:574] : INFO : Epoch 1993 | loss: 0.0311319 | val_loss: 0.0311578 | Time: 9536.91 ms [2022-04-19 22:19:47 main:574] : INFO : Epoch 1994 | loss: 0.0311319 | val_loss: 0.0311576 | Time: 9782.49 ms [2022-04-19 22:19:57 main:574] : INFO : Epoch 1995 | loss: 0.0311317 | val_loss: 0.0311571 | Time: 9585.55 ms [2022-04-19 22:20:06 main:574] : INFO : Epoch 1996 | loss: 0.0311349 | val_loss: 0.0311583 | Time: 9356.35 ms [2022-04-19 22:20:16 main:574] : INFO : Epoch 1997 | loss: 0.0311334 | val_loss: 0.0311551 | Time: 9670.15 ms [2022-04-19 22:20:25 main:574] : INFO : Epoch 1998 | loss: 0.0311308 | val_loss: 0.0311566 | Time: 9657.53 ms [2022-04-19 22:20:35 main:574] : INFO : Epoch 1999 | loss: 0.0311317 | val_loss: 0.0311527 | Time: 9544.13 ms [2022-04-19 22:20:44 main:574] : INFO : Epoch 2000 | loss: 0.0311338 | val_loss: 0.0311575 | Time: 9363.87 ms [2022-04-19 22:20:54 main:574] : INFO : Epoch 2001 | loss: 0.0311326 | val_loss: 0.0311623 | Time: 9303.7 ms [2022-04-19 22:21:04 main:574] : INFO : Epoch 2002 | loss: 0.0311323 | val_loss: 0.031156 | Time: 9885.99 ms [2022-04-19 22:21:13 main:574] : INFO : Epoch 2003 | loss: 0.0311308 | val_loss: 0.0311617 | Time: 9595.21 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060) [2022-04-19 22:28:36 main:435] : INFO : Set logging level to 1 [2022-04-19 22:28:36 main:441] : INFO : Running in BOINC Client mode [2022-04-19 22:28:36 main:444] : INFO : Resolving all filenames [2022-04-19 22:28:36 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-19 22:28:36 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-19 22:28:36 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-19 22:28:36 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-04-19 22:28:36 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-19 22:28:36 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-19 22:28:36 main:474] : INFO : Configuration: [2022-04-19 22:28:36 main:475] : INFO : Model type: GRU [2022-04-19 22:28:36 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-19 22:28:36 main:477] : INFO : Max Epochs: 2048 [2022-04-19 22:28:36 main:478] : INFO : Batch Size: 128 [2022-04-19 22:28:36 main:479] : INFO : Learning Rate: 0.01 [2022-04-19 22:28:36 main:480] : INFO : Patience: 10 [2022-04-19 22:28:36 main:481] : INFO : Hidden Width: 12 [2022-04-19 22:28:36 main:482] : INFO : # Recurrent Layers: 4 [2022-04-19 22:28:36 main:483] : INFO : # Backend Layers: 4 [2022-04-19 22:28:36 main:484] : INFO : # Threads: 1 [2022-04-19 22:28:36 main:486] : INFO : Preparing Dataset [2022-04-19 22:28:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-19 22:28:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-19 22:28:41 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-19 22:28:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-19 22:28:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-19 22:28:41 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-19 22:28:41 main:494] : INFO : Creating Model [2022-04-19 22:28:41 main:507] : INFO : Preparing config file [2022-04-19 22:28:41 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-19 22:28:41 main:512] : INFO : Loading config [2022-04-19 22:28:41 main:514] : INFO : Loading state [2022-04-19 22:28:43 main:559] : INFO : Loading DataLoader into Memory [2022-04-19 22:28:43 main:562] : INFO : Starting Training [2022-04-19 22:28:49 main:574] : INFO : Epoch 1989 | loss: 0.0312231 | val_loss: 0.0311968 | Time: 6796.41 ms [2022-04-19 22:28:57 main:574] : INFO : Epoch 1990 | loss: 0.03115 | val_loss: 0.031159 | Time: 7788.99 ms [2022-04-19 22:29:06 main:574] : INFO : Epoch 1991 | loss: 0.0311386 | val_loss: 0.0311576 | Time: 8305.31 ms [2022-04-19 22:29:14 main:574] : INFO : Epoch 1992 | loss: 0.0311395 | val_loss: 0.0311535 | Time: 8509.73 ms [2022-04-19 22:29:23 main:574] : INFO : Epoch 1993 | loss: 0.0311356 | val_loss: 0.0311575 | Time: 8934.48 ms [2022-04-19 22:29:33 main:574] : INFO : Epoch 1994 | loss: 0.0311345 | val_loss: 0.0311522 | Time: 9441.17 ms [2022-04-19 22:29:42 main:574] : INFO : Epoch 1995 | loss: 0.0311333 | val_loss: 0.0311529 | Time: 9217.31 ms [2022-04-19 22:29:51 main:574] : INFO : Epoch 1996 | loss: 0.0311336 | val_loss: 0.0311536 | Time: 9252.52 ms [2022-04-19 22:30:01 main:574] : INFO : Epoch 1997 | loss: 0.0311333 | val_loss: 0.031157 | Time: 9578.72 ms [2022-04-19 22:30:11 main:574] : INFO : Epoch 1998 | loss: 0.0311317 | val_loss: 0.0311557 | Time: 10082.5 ms [2022-04-19 22:30:20 main:574] : INFO : Epoch 1999 | loss: 0.0311307 | val_loss: 0.0311592 | Time: 9534.47 ms [2022-04-19 22:30:39 main:574] : INFO : Epoch 2000 | loss: 0.0311315 | val_loss: 0.0311546 | Time: 19131.1 ms [2022-04-19 22:30:48 main:574] : INFO : Epoch 2001 | loss: 0.0311343 | val_loss: 0.0311578 | Time: 8574.05 ms [2022-04-19 22:30:57 main:574] : INFO : Epoch 2002 | loss: 0.031136 | val_loss: 0.0311565 | Time: 9279.89 ms [2022-04-19 22:31:07 main:574] : INFO : Epoch 2003 | loss: 0.0311356 | val_loss: 0.0311555 | Time: 9611.81 ms [2022-04-19 22:31:17 main:574] : INFO : Epoch 2004 | loss: 0.0311341 | val_loss: 0.0311551 | Time: 9654.22 ms [2022-04-19 22:31:26 main:574] : INFO : Epoch 2005 | loss: 0.0311372 | val_loss: 0.0311582 | Time: 9758.44 ms [2022-04-19 22:31:36 main:574] : INFO : Epoch 2006 | loss: 0.0311335 | val_loss: 0.0311569 | Time: 9405.58 ms [2022-04-19 22:31:45 main:574] : INFO : Epoch 2007 | loss: 0.0311324 | val_loss: 0.0311541 | Time: 9529.27 ms [2022-04-19 22:31:55 main:574] : INFO : Epoch 2008 | loss: 0.0311287 | val_loss: 0.0311581 | Time: 9478.99 ms [2022-04-19 22:32:05 main:574] : INFO : Epoch 2009 | loss: 0.0311283 | val_loss: 0.0311566 | Time: 9880.47 ms [2022-04-19 22:32:14 main:574] : INFO : Epoch 2010 | loss: 0.0311334 | val_loss: 0.0311534 | Time: 9552.44 ms [2022-04-19 22:32:24 main:574] : INFO : Epoch 2011 | loss: 0.0311388 | val_loss: 0.0311609 | Time: 9633.44 ms [2022-04-19 22:32:33 main:574] : INFO : Epoch 2012 | loss: 0.031136 | val_loss: 0.0311535 | Time: 9429.46 ms [2022-04-19 22:32:43 main:574] : INFO : Epoch 2013 | loss: 0.0311354 | val_loss: 0.031157 | Time: 9598.45 ms [2022-04-19 22:32:53 main:574] : INFO : Epoch 2014 | loss: 0.0311382 | val_loss: 0.0311575 | Time: 9548.97 ms [2022-04-19 22:33:02 main:574] : INFO : Epoch 2015 | loss: 0.0311329 | val_loss: 0.0311567 | Time: 9594.88 ms [2022-04-19 22:33:12 main:574] : INFO : Epoch 2016 | loss: 0.0311371 | val_loss: 0.0311551 | Time: 9704.78 ms [2022-04-19 22:33:21 main:574] : INFO : Epoch 2017 | loss: 0.0311383 | val_loss: 0.0311588 | Time: 9260.22 ms [2022-04-19 22:33:31 main:574] : INFO : Epoch 2018 | loss: 0.0311398 | val_loss: 0.0311593 | Time: 9400.37 ms [2022-04-19 22:33:40 main:574] : INFO : Epoch 2019 | loss: 0.0311376 | val_loss: 0.0311589 | Time: 9797.86 ms [2022-04-19 22:33:48 main:574] : INFO : Epoch 2020 | loss: 0.0311372 | val_loss: 0.0311553 | Time: 7881.22 ms [2022-04-19 22:33:56 main:574] : INFO : Epoch 2021 | loss: 0.0311356 | val_loss: 0.0311564 | Time: 7407.85 ms [2022-04-19 22:34:03 main:574] : INFO : Epoch 2022 | loss: 0.0311379 | val_loss: 0.0311557 | Time: 7621.78 ms [2022-04-19 22:34:11 main:574] : INFO : Epoch 2023 | loss: 0.0311382 | val_loss: 0.0311572 | Time: 7414.13 ms [2022-04-19 22:34:18 main:574] : INFO : Epoch 2024 | loss: 0.0311401 | val_loss: 0.0311603 | Time: 7304.63 ms [2022-04-19 22:34:26 main:574] : INFO : Epoch 2025 | loss: 0.0311394 | val_loss: 0.0311553 | Time: 7472.63 ms [2022-04-19 22:34:33 main:574] : INFO : Epoch 2026 | loss: 0.0311411 | val_loss: 0.031158 | Time: 7380.06 ms [2022-04-19 22:34:41 main:574] : INFO : Epoch 2027 | loss: 0.0311412 | val_loss: 0.0311591 | Time: 7604.02 ms [2022-04-19 22:34:48 main:574] : INFO : Epoch 2028 | loss: 0.0311446 | val_loss: 0.0311566 | Time: 7463.49 ms [2022-04-19 22:34:56 main:574] : INFO : Epoch 2029 | loss: 0.0311408 | val_loss: 0.0311542 | Time: 7469.61 ms [2022-04-19 22:35:03 main:574] : INFO : Epoch 2030 | loss: 0.0311407 | val_loss: 0.0311545 | Time: 7287.3 ms [2022-04-19 22:35:10 main:574] : INFO : Epoch 2031 | loss: 0.031141 | val_loss: 0.0311549 | Time: 7499 ms [2022-04-19 22:35:18 main:574] : INFO : Epoch 2032 | loss: 0.0311422 | val_loss: 0.0311553 | Time: 7378.38 ms [2022-04-19 22:35:25 main:574] : INFO : Epoch 2033 | loss: 0.0311375 | val_loss: 0.031157 | Time: 7414.47 ms [2022-04-19 22:35:33 main:574] : INFO : Epoch 2034 | loss: 0.0311377 | val_loss: 0.0311613 | Time: 7492.05 ms [2022-04-19 22:35:41 main:574] : INFO : Epoch 2035 | loss: 0.0311392 | val_loss: 0.0311665 | Time: 7705.18 ms [2022-04-19 22:35:48 main:574] : INFO : Epoch 2036 | loss: 0.0311388 | val_loss: 0.0311534 | Time: 7399.29 ms [2022-04-19 22:35:55 main:574] : INFO : Epoch 2037 | loss: 0.0311367 | val_loss: 0.0311544 | Time: 7413.18 ms [2022-04-19 22:36:03 main:574] : INFO : Epoch 2038 | loss: 0.0311379 | val_loss: 0.031153 | Time: 7342.89 ms [2022-04-19 22:36:10 main:574] : INFO : Epoch 2039 | loss: 0.0311352 | val_loss: 0.031153 | Time: 7385.6 ms [2022-04-19 22:36:18 main:574] : INFO : Epoch 2040 | loss: 0.0311348 | val_loss: 0.031154 | Time: 7660.24 ms [2022-04-19 22:36:25 main:574] : INFO : Epoch 2041 | loss: 0.0311363 | val_loss: 0.0311558 | Time: 7493.5 ms [2022-04-19 22:36:33 main:574] : INFO : Epoch 2042 | loss: 0.0311378 | val_loss: 0.0311573 | Time: 7720.01 ms [2022-04-19 22:36:40 main:574] : INFO : Epoch 2043 | loss: 0.0311361 | val_loss: 0.031167 | Time: 7345.22 ms [2022-04-19 22:36:48 main:574] : INFO : Epoch 2044 | loss: 0.0311363 | val_loss: 0.0311607 | Time: 7475.77 ms [2022-04-19 22:36:55 main:574] : INFO : Epoch 2045 | loss: 0.0311348 | val_loss: 0.0311573 | Time: 7432.98 ms [2022-04-19 22:37:03 main:574] : INFO : Epoch 2046 | loss: 0.0311339 | val_loss: 0.0311614 | Time: 7634.31 ms [2022-04-19 22:37:11 main:574] : INFO : Epoch 2047 | loss: 0.0311316 | val_loss: 0.0311555 | Time: 7531.88 ms [2022-04-19 22:37:18 main:574] : INFO : Epoch 2048 | loss: 0.0311317 | val_loss: 0.0311541 | Time: 7367.54 ms [2022-04-19 22:37:18 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311541 [2022-04-19 22:37:18 main:603] : INFO : Saving end state to config to file [2022-04-19 22:37:18 main:608] : INFO : Success, exiting.. 22:37:18 (15596): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)