Task 13527968
| Name | ParityModified-1647044285-7218-0_1 |
| Workunit | 10093869 |
| Created | 22 Mar 2022, 6:30:30 UTC |
| Sent | 22 Mar 2022, 7:09:54 UTC |
| Report deadline | 30 Mar 2022, 7:09:54 UTC |
| Received | 6 Apr 2022, 17:08:43 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 8169 |
| Run time | 53 min 36 sec |
| CPU time | 48 min 5 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 9,066.41 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 3.05 GB |
| Peak swap size | 4.36 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> 9:14:59 main:507] : INFO : Preparing config file [2022-04-06 09:14:59 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-06 09:14:59 main:512] : INFO : Loading config [2022-04-06 09:14:59 main:514] : INFO : Loading state [2022-04-06 09:15:00 main:559] : INFO : Loading DataLoader into Memory [2022-04-06 09:15:00 main:562] : INFO : Starting Training [2022-04-06 09:15:02 main:574] : INFO : Epoch 1641 | loss: 0.0318728 | val_loss: 0.0313488 | Time: 1767.15 ms [2022-04-06 09:15:04 main:574] : INFO : Epoch 1642 | loss: 0.0313071 | val_loss: 0.0312782 | Time: 1495.61 ms [2022-04-06 09:15:05 main:574] : INFO : Epoch 1643 | loss: 0.0312607 | val_loss: 0.031255 | Time: 1482.22 ms [2022-04-06 09:15:07 main:574] : INFO : Epoch 1644 | loss: 0.0312508 | val_loss: 0.0312492 | Time: 1478.76 ms [2022-04-06 09:15:08 main:574] : INFO : Epoch 1645 | loss: 0.0312482 | val_loss: 0.0312476 | Time: 1476.57 ms [2022-04-06 09:15:10 main:574] : INFO : Epoch 1646 | loss: 0.0312467 | val_loss: 0.0312456 | Time: 1501.61 ms [2022-04-06 09:15:11 main:574] : INFO : Epoch 1647 | loss: 0.0312443 | val_loss: 0.0312423 | Time: 1467.22 ms [2022-04-06 09:15:13 main:574] : INFO : Epoch 1648 | loss: 0.0312394 | val_loss: 0.0312369 | Time: 1490.86 ms [2022-04-06 09:15:14 main:574] : INFO : Epoch 1649 | loss: 0.0312311 | val_loss: 0.0312246 | Time: 1489.17 ms [2022-04-06 09:15:16 main:574] : INFO : Epoch 1650 | loss: 0.0312185 | val_loss: 0.0312083 | Time: 1483.08 ms [2022-04-06 09:15:17 main:574] : INFO : Epoch 1651 | loss: 0.0312052 | val_loss: 0.031197 | Time: 1482.4 ms [2022-04-06 09:15:19 main:574] : INFO : Epoch 1652 | loss: 0.0311943 | val_loss: 0.031193 | Time: 1486.36 ms [2022-04-06 09:15:21 main:574] : INFO : Epoch 1653 | loss: 0.0311941 | val_loss: 0.0311888 | Time: 1490.06 ms [2022-04-06 09:15:22 main:574] : INFO : Epoch 1654 | loss: 0.0311892 | val_loss: 0.0311763 | Time: 1482.91 ms [2022-04-06 09:15:24 main:574] : INFO : Epoch 1655 | loss: 0.0311801 | val_loss: 0.0311741 | Time: 1478.5 ms [2022-04-06 09:15:25 main:574] : INFO : Epoch 1656 | loss: 0.0311776 | val_loss: 0.0311783 | Time: 1501.41 ms [2022-04-06 09:15:27 main:574] : INFO : Epoch 1657 | loss: 0.0311767 | val_loss: 0.0311743 | Time: 1480.47 ms [2022-04-06 09:15:28 main:574] : INFO : Epoch 1658 | loss: 0.0311745 | val_loss: 0.0311741 | Time: 1478.42 ms [2022-04-06 09:15:30 main:574] : INFO : Epoch 1659 | loss: 0.0311746 | val_loss: 0.031173 | Time: 1496.56 ms [2022-04-06 09:15:31 main:574] : INFO : Epoch 1660 | loss: 0.0311721 | val_loss: 0.0311695 | Time: 1493.53 ms [2022-04-06 09:15:33 main:574] : INFO : Epoch 1661 | loss: 0.0311718 | val_loss: 0.0311701 | Time: 1506.56 ms [2022-04-06 09:15:34 main:574] : INFO : Epoch 1662 | loss: 0.0311731 | val_loss: 0.0311692 | Time: 1501.43 ms [2022-04-06 09:15:36 main:574] : INFO : Epoch 1663 | loss: 0.0311732 | val_loss: 0.0311728 | Time: 1519.21 ms [2022-04-06 09:15:38 main:574] : INFO : Epoch 1664 | loss: 0.0311707 | val_loss: 0.0311688 | Time: 1479.17 ms [2022-04-06 09:15:39 main:574] : INFO : Epoch 1665 | loss: 0.0311697 | val_loss: 0.0311694 | Time: 1488.16 ms [2022-04-06 09:15:41 main:574] : INFO : Epoch 1666 | loss: 0.0311729 | val_loss: 0.0311697 | Time: 1483.13 ms [2022-04-06 09:15:42 main:574] : INFO : Epoch 1667 | loss: 0.0311698 | val_loss: 0.0311679 | Time: 1495.34 ms [2022-04-06 09:15:44 main:574] : INFO : Epoch 1668 | loss: 0.031169 | val_loss: 0.031168 | Time: 1487.05 ms [2022-04-06 09:15:45 main:574] : INFO : Epoch 1669 | loss: 0.0311689 | val_loss: 0.031171 | Time: 1485.42 ms [2022-04-06 09:15:47 main:574] : INFO : Epoch 1670 | loss: 0.0311705 | val_loss: 0.0311679 | Time: 1480.71 ms [2022-04-06 09:15:48 main:574] : INFO : Epoch 1671 | loss: 0.0311695 | val_loss: 0.031173 | Time: 1487.45 ms [2022-04-06 09:15:50 main:574] : INFO : Epoch 1672 | loss: 0.0311653 | val_loss: 0.0311673 | Time: 1469.71 ms [2022-04-06 09:15:52 main:574] : INFO : Epoch 1673 | loss: 0.0311633 | val_loss: 0.031164 | Time: 1488.74 ms [2022-04-06 09:15:53 main:574] : INFO : Epoch 1674 | loss: 0.0311633 | val_loss: 0.0311651 | Time: 1469.69 ms [2022-04-06 09:15:55 main:574] : INFO : Epoch 1675 | loss: 0.0311619 | val_loss: 0.0311674 | Time: 1539.61 ms [2022-04-06 09:15:56 main:574] : INFO : Epoch 1676 | loss: 0.0311614 | val_loss: 0.0311636 | Time: 1500.42 ms [2022-04-06 09:15:58 main:574] : INFO : Epoch 1677 | loss: 0.0311587 | val_loss: 0.0311647 | Time: 1472.32 ms [2022-04-06 09:15:59 main:574] : INFO : Epoch 1678 | loss: 0.0311628 | val_loss: 0.0311763 | Time: 1470.25 ms [2022-04-06 09:16:01 main:574] : INFO : Epoch 1679 | loss: 0.0311622 | val_loss: 0.0311655 | Time: 1484.71 ms [2022-04-06 09:16:02 main:574] : INFO : Epoch 1680 | loss: 0.0311577 | val_loss: 0.0311617 | Time: 1482.72 ms [2022-04-06 09:16:04 main:574] : INFO : Epoch 1681 | loss: 0.0311547 | val_loss: 0.0311608 | Time: 1478.13 ms [2022-04-06 09:16:05 main:574] : INFO : Epoch 1682 | loss: 0.0311556 | val_loss: 0.0311613 | Time: 1484.01 ms [2022-04-06 09:16:07 main:574] : INFO : Epoch 1683 | loss: 0.0311527 | val_loss: 0.0311599 | Time: 1503.42 ms [2022-04-06 09:16:09 main:574] : INFO : Epoch 1684 | loss: 0.0311526 | val_loss: 0.0311629 | Time: 1488.77 ms [2022-04-06 09:16:10 main:574] : INFO : Epoch 1685 | loss: 0.0311546 | val_loss: 0.0311608 | Time: 1473.22 ms [2022-04-06 09:16:12 main:574] : INFO : Epoch 1686 | loss: 0.0311537 | val_loss: 0.0311613 | Time: 1478.06 ms [2022-04-06 09:16:13 main:574] : INFO : Epoch 1687 | loss: 0.0311531 | val_loss: 0.031163 | Time: 1484.79 ms [2022-04-06 09:16:15 main:574] : INFO : Epoch 1688 | loss: 0.0311516 | val_loss: 0.0311601 | Time: 1498.22 ms [2022-04-06 09:16:16 main:574] : INFO : Epoch 1689 | loss: 0.0311508 | val_loss: 0.031164 | Time: 1492.25 ms [2022-04-06 09:16:18 main:574] : INFO : Epoch 1690 | loss: 0.0311507 | val_loss: 0.031161 | Time: 1485.69 ms [2022-04-06 09:16:19 main:574] : INFO : Epoch 1691 | loss: 0.0311492 | val_loss: 0.0311617 | Time: 1492.97 ms [2022-04-06 09:16:21 main:574] : INFO : Epoch 1692 | loss: 0.0311505 | val_loss: 0.0311659 | Time: 1473.47 ms [2022-04-06 09:16:22 main:574] : INFO : Epoch 1693 | loss: 0.031153 | val_loss: 0.0311667 | Time: 1478.78 ms [2022-04-06 09:16:24 main:574] : INFO : Epoch 1694 | loss: 0.0311512 | val_loss: 0.0311752 | Time: 1501.94 ms [2022-04-06 09:16:26 main:574] : INFO : Epoch 1695 | loss: 0.0311505 | val_loss: 0.0311603 | Time: 1487.35 ms [2022-04-06 09:16:27 main:574] : INFO : Epoch 1696 | loss: 0.0311497 | val_loss: 0.0311635 | Time: 1487.41 ms [2022-04-06 09:16:29 main:574] : INFO : Epoch 1697 | loss: 0.0311491 | val_loss: 0.0311647 | Time: 1477.33 ms [2022-04-06 09:16:30 main:574] : INFO : Epoch 1698 | loss: 0.0311509 | val_loss: 0.0311636 | Time: 1481.69 ms [2022-04-06 09:16:32 main:574] : INFO : Epoch 1699 | loss: 0.03115 | val_loss: 0.0311624 | Time: 1481.64 ms [2022-04-06 09:16:33 main:574] : INFO : Epoch 1700 | loss: 0.0311486 | val_loss: 0.0311679 | Time: 1477.88 ms [2022-04-06 09:16:35 main:574] : INFO : Epoch 1701 | loss: 0.0311555 | val_loss: 0.0311642 | Time: 1475.59 ms [2022-04-06 09:16:36 main:574] : INFO : Epoch 1702 | loss: 0.0311474 | val_loss: 0.0311628 | Time: 1476.22 ms [2022-04-06 09:16:38 main:574] : INFO : Epoch 1703 | loss: 0.0311482 | val_loss: 0.0311706 | Time: 1467.97 ms [2022-04-06 09:16:39 main:574] : INFO : Epoch 1704 | loss: 0.0311463 | val_loss: 0.031166 | Time: 1506.94 ms [2022-04-06 09:16:41 main:574] : INFO : Epoch 1705 | loss: 0.0311454 | val_loss: 0.0311682 | Time: 1482.39 ms [2022-04-06 09:16:43 main:574] : INFO : Epoch 1706 | loss: 0.0311436 | val_loss: 0.0311725 | Time: 1478.33 ms [2022-04-06 09:16:44 main:574] : INFO : Epoch 1707 | loss: 0.0311493 | val_loss: 0.0311665 | Time: 1473.44 ms [2022-04-06 09:16:46 main:574] : INFO : Epoch 1708 | loss: 0.0311472 | val_loss: 0.0311638 | Time: 1541.35 ms [2022-04-06 09:16:47 main:574] : INFO : Epoch 1709 | loss: 0.0311459 | val_loss: 0.0311705 | Time: 1497.65 ms [2022-04-06 09:16:49 main:574] : INFO : Epoch 1710 | loss: 0.0311444 | val_loss: 0.0311641 | Time: 1509.22 ms [2022-04-06 09:16:50 main:574] : INFO : Epoch 1711 | loss: 0.0311451 | val_loss: 0.0311658 | Time: 1483.25 ms [2022-04-06 09:16:52 main:574] : INFO : Epoch 1712 | loss: 0.0311445 | val_loss: 0.0311701 | Time: 1500.62 ms [2022-04-06 09:16:53 main:574] : INFO : Epoch 1713 | loss: 0.031142 | val_loss: 0.0311722 | Time: 1492.26 ms [2022-04-06 09:16:55 main:574] : INFO : Epoch 1714 | loss: 0.0311427 | val_loss: 0.0311665 | Time: 1488.84 ms [2022-04-06 09:16:57 main:574] : INFO : Epoch 1715 | loss: 0.0311452 | val_loss: 0.0311711 | Time: 1501.25 ms [2022-04-06 09:16:58 main:574] : INFO : Epoch 1716 | loss: 0.031146 | val_loss: 0.0311667 | Time: 1487.51 ms [2022-04-06 09:17:00 main:574] : INFO : Epoch 1717 | loss: 0.0311429 | val_loss: 0.0311694 | Time: 1485.17 ms [2022-04-06 09:17:01 main:574] : INFO : Epoch 1718 | loss: 0.0311422 | val_loss: 0.0311661 | Time: 1479.18 ms [2022-04-06 09:17:03 main:574] : INFO : Epoch 1719 | loss: 0.0311404 | val_loss: 0.0311663 | Time: 1484.43 ms [2022-04-06 09:17:04 main:574] : INFO : Epoch 1720 | loss: 0.0311422 | val_loss: 0.0311696 | Time: 1475.88 ms [2022-04-06 09:17:06 main:574] : INFO : Epoch 1721 | loss: 0.0311427 | val_loss: 0.031165 | Time: 1471.91 ms [2022-04-06 09:17:07 main:574] : INFO : Epoch 1722 | loss: 0.0311411 | val_loss: 0.0311635 | Time: 1490.96 ms [2022-04-06 09:17:09 main:574] : INFO : Epoch 1723 | loss: 0.0311413 | val_loss: 0.031167 | Time: 1473.92 ms [2022-04-06 09:17:10 main:574] : INFO : Epoch 1724 | loss: 0.031137 | val_loss: 0.0311675 | Time: 1473.06 ms [2022-04-06 09:17:12 main:574] : INFO : Epoch 1725 | loss: 0.0311365 | val_loss: 0.0311763 | Time: 1477.13 ms [2022-04-06 09:17:14 main:574] : INFO : Epoch 1726 | loss: 0.0311397 | val_loss: 0.0311674 | Time: 1487.11 ms [2022-04-06 09:17:15 main:574] : INFO : Epoch 1727 | loss: 0.0311364 | val_loss: 0.0311708 | Time: 1472.57 ms [2022-04-06 09:17:17 main:574] : INFO : Epoch 1728 | loss: 0.0311384 | val_loss: 0.0311788 | Time: 1472.24 ms [2022-04-06 09:17:18 main:574] : INFO : Epoch 1729 | loss: 0.03114 | val_loss: 0.0311709 | Time: 1485.05 ms [2022-04-06 09:17:20 main:574] : INFO : Epoch 1730 | loss: 0.0311378 | val_loss: 0.0311799 | Time: 1471.38 ms [2022-04-06 09:17:21 main:574] : INFO : Epoch 1731 | loss: 0.0311384 | val_loss: 0.0311679 | Time: 1477.88 ms [2022-04-06 09:17:23 main:574] : INFO : Epoch 1732 | loss: 0.0311369 | val_loss: 0.0311748 | Time: 1483.17 ms [2022-04-06 09:17:24 main:574] : INFO : Epoch 1733 | loss: 0.0311357 | val_loss: 0.0311676 | Time: 1489.86 ms [2022-04-06 09:17:26 main:574] : INFO : Epoch 1734 | loss: 0.0311362 | val_loss: 0.0311819 | Time: 1495.79 ms [2022-04-06 09:17:27 main:574] : INFO : Epoch 1735 | loss: 0.0311405 | val_loss: 0.0311797 | Time: 1483.57 ms [2022-04-06 09:17:29 main:574] : INFO : Epoch 1736 | loss: 0.031143 | val_loss: 0.0311693 | Time: 1479.48 ms [2022-04-06 09:17:31 main:574] : INFO : Epoch 1737 | loss: 0.0311471 | val_loss: 0.0311638 | Time: 1487.11 ms [2022-04-06 09:17:32 main:574] : INFO : Epoch 1738 | loss: 0.0311402 | val_loss: 0.0311669 | Time: 1480.45 ms [2022-04-06 09:17:34 main:574] : INFO : Epoch 1739 | loss: 0.0311378 | val_loss: 0.0311699 | Time: 1486.7 ms [2022-04-06 09:17:35 main:574] : INFO : Epoch 1740 | loss: 0.0311366 | val_loss: 0.0311683 | Time: 1465.11 ms [2022-04-06 09:17:37 main:574] : INFO : Epoch 1741 | loss: 0.0311396 | val_loss: 0.031177 | Time: 1472.67 ms [2022-04-06 09:17:38 main:574] : INFO : Epoch 1742 | loss: 0.0311448 | val_loss: 0.0311706 | Time: 1486.98 ms [2022-04-06 09:17:40 main:574] : INFO : Epoch 1743 | loss: 0.0311402 | val_loss: 0.0311667 | Time: 1479.52 ms [2022-04-06 09:17:41 main:574] : INFO : Epoch 1744 | loss: 0.0311396 | val_loss: 0.0311715 | Time: 1486.19 ms [2022-04-06 09:17:43 main:574] : INFO : Epoch 1745 | loss: 0.0311394 | val_loss: 0.031169 | Time: 1467.29 ms [2022-04-06 09:17:44 main:574] : INFO : Epoch 1746 | loss: 0.0311349 | val_loss: 0.0311731 | Time: 1477.97 ms [2022-04-06 09:17:46 main:574] : INFO : Epoch 1747 | loss: 0.0311334 | val_loss: 0.0311764 | Time: 1471.6 ms [2022-04-06 09:17:47 main:574] : INFO : Epoch 1748 | loss: 0.0311339 | val_loss: 0.031173 | Time: 1465.09 ms [2022-04-06 09:17:49 main:574] : INFO : Epoch 1749 | loss: 0.031133 | val_loss: 0.0311688 | Time: 1474.61 ms [2022-04-06 09:17:51 main:574] : INFO : Epoch 1750 | loss: 0.0311329 | val_loss: 0.0311735 | Time: 1487.11 ms [2022-04-06 09:17:52 main:574] : INFO : Epoch 1751 | loss: 0.0311352 | val_loss: 0.0311706 | Time: 1484.4 ms [2022-04-06 09:17:54 main:574] : INFO : Epoch 1752 | loss: 0.0311339 | val_loss: 0.0311783 | Time: 1462.49 ms [2022-04-06 09:17:55 main:574] : INFO : Epoch 1753 | loss: 0.0311324 | val_loss: 0.0311697 | Time: 1490.2 ms [2022-04-06 09:17:57 main:574] : INFO : Epoch 1754 | loss: 0.0311316 | val_loss: 0.0311697 | Time: 1462.69 ms [2022-04-06 09:17:58 main:574] : INFO : Epoch 1755 | loss: 0.0311363 | val_loss: 0.0311769 | Time: 1480 ms [2022-04-06 09:18:00 main:574] : INFO : Epoch 1756 | loss: 0.0311351 | val_loss: 0.0311672 | Time: 1493.51 ms [2022-04-06 09:18:01 main:574] : INFO : Epoch 1757 | loss: 0.0311339 | val_loss: 0.0311673 | Time: 1467.77 ms [2022-04-06 09:18:03 main:574] : INFO : Epoch 1758 | loss: 0.0311301 | val_loss: 0.0311745 | Time: 1469.04 ms [2022-04-06 09:18:04 main:574] : INFO : Epoch 1759 | loss: 0.0311296 | val_loss: 0.0311734 | Time: 1491.54 ms [2022-04-06 09:18:06 main:574] : INFO : Epoch 1760 | loss: 0.0311355 | val_loss: 0.0311739 | Time: 1482.31 ms [2022-04-06 09:18:08 main:574] : INFO : Epoch 1761 | loss: 0.0311321 | val_loss: 0.0311687 | Time: 1471 ms [2022-04-06 09:18:09 main:574] : INFO : Epoch 1762 | loss: 0.0311296 | val_loss: 0.031171 | Time: 1477.66 ms [2022-04-06 09:18:11 main:574] : INFO : Epoch 1763 | loss: 0.0311294 | val_loss: 0.0311686 | Time: 1488.95 ms [2022-04-06 09:18:12 main:574] : INFO : Epoch 1764 | loss: 0.0311321 | val_loss: 0.0311659 | Time: 1476.91 ms [2022-04-06 09:18:14 main:574] : INFO : Epoch 1765 | loss: 0.0311337 | val_loss: 0.031168 | Time: 1493.14 ms [2022-04-06 09:18:15 main:574] : INFO : Epoch 1766 | loss: 0.0311306 | val_loss: 0.0311735 | Time: 1489.3 ms [2022-04-06 09:18:17 main:574] : INFO : Epoch 1767 | loss: 0.0311339 | val_loss: 0.0311763 | Time: 1480.4 ms [2022-04-06 09:18:18 main:574] : INFO : Epoch 1768 | loss: 0.0311313 | val_loss: 0.0311759 | Time: 1466.39 ms [2022-04-06 09:18:20 main:574] : INFO : Epoch 1769 | loss: 0.0311332 | val_loss: 0.0311686 | Time: 1498.9 ms [2022-04-06 09:18:22 main:574] : INFO : Epoch 1770 | loss: 0.0311306 | val_loss: 0.0311689 | Time: 1466.41 ms [2022-04-06 09:18:23 main:574] : INFO : Epoch 1771 | loss: 0.0311253 | val_loss: 0.0311761 | Time: 1488.07 ms [2022-04-06 09:18:25 main:574] : INFO : Epoch 1772 | loss: 0.0311284 | val_loss: 0.0311753 | Time: 1494.34 ms [2022-04-06 09:18:26 main:574] : INFO : Epoch 1773 | loss: 0.0311252 | val_loss: 0.0311764 | Time: 1484.23 ms [2022-04-06 09:18:28 main:574] : INFO : Epoch 1774 | loss: 0.0311281 | val_loss: 0.0311761 | Time: 1463.44 ms [2022-04-06 09:18:29 main:574] : INFO : Epoch 1775 | loss: 0.0311275 | val_loss: 0.0311813 | Time: 1470.87 ms [2022-04-06 09:18:31 main:574] : INFO : Epoch 1776 | loss: 0.031132 | val_loss: 0.0311826 | Time: 1484.28 ms [2022-04-06 09:18:32 main:574] : INFO : Epoch 1777 | loss: 0.0311308 | val_loss: 0.0311768 | Time: 1469.06 ms [2022-04-06 09:18:34 main:574] : INFO : Epoch 1778 | loss: 0.0311295 | val_loss: 0.0311707 | Time: 1484.82 ms [2022-04-06 09:18:35 main:574] : INFO : Epoch 1779 | loss: 0.0311284 | val_loss: 0.0311672 | Time: 1478.98 ms [2022-04-06 09:18:37 main:574] : INFO : Epoch 1780 | loss: 0.0311278 | val_loss: 0.0311784 | Time: 1472.3 ms [2022-04-06 09:18:38 main:574] : INFO : Epoch 1781 | loss: 0.0311288 | val_loss: 0.0311679 | Time: 1468.29 ms [2022-04-06 09:18:40 main:574] : INFO : Epoch 1782 | loss: 0.0311269 | val_loss: 0.0311695 | Time: 1478.44 ms [2022-04-06 09:18:42 main:574] : INFO : Epoch 1783 | loss: 0.031125 | val_loss: 0.0311734 | Time: 1475.66 ms [2022-04-06 09:18:43 main:574] : INFO : Epoch 1784 | loss: 0.0311263 | val_loss: 0.0311719 | Time: 1479.65 ms [2022-04-06 09:18:45 main:574] : INFO : Epoch 1785 | loss: 0.0311251 | val_loss: 0.0311746 | Time: 1481.74 ms [2022-04-06 09:18:46 main:574] : INFO : Epoch 1786 | loss: 0.0311252 | val_loss: 0.0311795 | Time: 1479.03 ms [2022-04-06 09:18:48 main:574] : INFO : Epoch 1787 | loss: 0.0311263 | val_loss: 0.0311721 | Time: 1464.48 ms [2022-04-06 09:18:49 main:574] : INFO : Epoch 1788 | loss: 0.0311267 | val_loss: 0.0311789 | Time: 1485.92 ms [2022-04-06 09:18:51 main:574] : INFO : Epoch 1789 | loss: 0.0311263 | val_loss: 0.031174 | Time: 1479.46 ms [2022-04-06 09:18:52 main:574] : INFO : Epoch 1790 | loss: 0.0311246 | val_loss: 0.0311685 | Time: 1496.87 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2070 SUPER) [2022-04-06 09:30:54 main:435] : INFO : Set logging level to 1 [2022-04-06 09:30:54 main:441] : INFO : Running in BOINC Client mode [2022-04-06 09:30:54 main:444] : INFO : Resolving all filenames [2022-04-06 09:30:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-06 09:30:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-06 09:30:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-06 09:30:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 0) [2022-04-06 09:30:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-06 09:30:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-06 09:30:54 main:474] : INFO : Configuration: [2022-04-06 09:30:54 main:475] : INFO : Model type: GRU [2022-04-06 09:30:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-06 09:30:55 main:477] : INFO : Max Epochs: 2048 [2022-04-06 09:30:55 main:478] : INFO : Batch Size: 128 [2022-04-06 09:30:55 main:479] : INFO : Learning Rate: 0.01 [2022-04-06 09:30:55 main:480] : INFO : Patience: 10 [2022-04-06 09:30:55 main:481] : INFO : Hidden Width: 12 [2022-04-06 09:30:55 main:482] : INFO : # Recurrent Layers: 4 [2022-04-06 09:30:55 main:483] : INFO : # Backend Layers: 4 [2022-04-06 09:30:55 main:484] : INFO : # Threads: 1 [2022-04-06 09:30:55 main:486] : INFO : Preparing Dataset [2022-04-06 09:30:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-06 09:30:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-06 09:30:57 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-06 09:30:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-06 09:30:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-06 09:30:57 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-06 09:30:57 main:494] : INFO : Creating Model [2022-04-06 09:30:58 main:507] : INFO : Preparing config file [2022-04-06 09:30:58 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-06 09:30:58 main:512] : INFO : Loading config [2022-04-06 09:30:58 main:514] : INFO : Loading state [2022-04-06 09:30:59 main:559] : INFO : Loading DataLoader into Memory [2022-04-06 09:30:59 main:562] : INFO : Starting Training [2022-04-06 09:31:01 main:574] : INFO : Epoch 1764 | loss: 0.0314108 | val_loss: 0.0312656 | Time: 1728.65 ms [2022-04-06 09:31:02 main:574] : INFO : Epoch 1765 | loss: 0.0312141 | val_loss: 0.0311824 | Time: 1456.92 ms [2022-04-06 09:31:04 main:574] : INFO : Epoch 1766 | loss: 0.0311488 | val_loss: 0.031173 | Time: 1466.7 ms [2022-04-06 09:31:05 main:574] : INFO : Epoch 1767 | loss: 0.0311388 | val_loss: 0.0311722 | Time: 1474.06 ms [2022-04-06 09:31:07 main:574] : INFO : Epoch 1768 | loss: 0.0311361 | val_loss: 0.0311749 | Time: 1471.41 ms [2022-04-06 09:31:08 main:574] : INFO : Epoch 1769 | loss: 0.0311328 | val_loss: 0.0311699 | Time: 1503.41 ms [2022-04-06 09:31:10 main:574] : INFO : Epoch 1770 | loss: 0.0311338 | val_loss: 0.0311657 | Time: 1531.55 ms [2022-04-06 09:31:12 main:574] : INFO : Epoch 1771 | loss: 0.0311354 | val_loss: 0.0311763 | Time: 1477.34 ms [2022-04-06 09:31:13 main:574] : INFO : Epoch 1772 | loss: 0.0311358 | val_loss: 0.0311698 | Time: 1496.23 ms [2022-04-06 09:31:15 main:574] : INFO : Epoch 1773 | loss: 0.0311331 | val_loss: 0.0311675 | Time: 1471.13 ms [2022-04-06 09:31:16 main:574] : INFO : Epoch 1774 | loss: 0.0311325 | val_loss: 0.0311726 | Time: 1484.4 ms [2022-04-06 09:31:18 main:574] : INFO : Epoch 1775 | loss: 0.0311316 | val_loss: 0.0311663 | Time: 1467.85 ms [2022-04-06 09:31:19 main:574] : INFO : Epoch 1776 | loss: 0.0311326 | val_loss: 0.0311771 | Time: 1462.84 ms [2022-04-06 09:31:21 main:574] : INFO : Epoch 1777 | loss: 0.0311302 | val_loss: 0.0311768 | Time: 1461.31 ms [2022-04-06 09:31:22 main:574] : INFO : Epoch 1778 | loss: 0.0311285 | val_loss: 0.0311757 | Time: 1474.41 ms [2022-04-06 09:31:24 main:574] : INFO : Epoch 1779 | loss: 0.0311325 | val_loss: 0.0311733 | Time: 1458.15 ms [2022-04-06 09:31:25 main:574] : INFO : Epoch 1780 | loss: 0.0311273 | val_loss: 0.0311696 | Time: 1451.74 ms [2022-04-06 09:31:27 main:574] : INFO : Epoch 1781 | loss: 0.0311252 | val_loss: 0.0311765 | Time: 1458.88 ms [2022-04-06 09:31:28 main:574] : INFO : Epoch 1782 | loss: 0.0311255 | val_loss: 0.0311692 | Time: 1456.19 ms [2022-04-06 09:31:30 main:574] : INFO : Epoch 1783 | loss: 0.0311238 | val_loss: 0.03118 | Time: 1470.29 ms [2022-04-06 09:31:32 main:574] : INFO : Epoch 1784 | loss: 0.0311239 | val_loss: 0.0311746 | Time: 1452.67 ms [2022-04-06 09:31:33 main:574] : INFO : Epoch 1785 | loss: 0.0311308 | val_loss: 0.0311714 | Time: 1474.44 ms [2022-04-06 09:31:35 main:574] : INFO : Epoch 1786 | loss: 0.0311291 | val_loss: 0.0311736 | Time: 1465.09 ms [2022-04-06 09:31:36 main:574] : INFO : Epoch 1787 | loss: 0.031125 | val_loss: 0.0311729 | Time: 1453.5 ms [2022-04-06 09:31:38 main:574] : INFO : Epoch 1788 | loss: 0.031122 | val_loss: 0.0311701 | Time: 1461.05 ms [2022-04-06 09:31:39 main:574] : INFO : Epoch 1789 | loss: 0.0311234 | val_loss: 0.0311757 | Time: 1448.25 ms [2022-04-06 09:31:41 main:574] : INFO : Epoch 1790 | loss: 0.0311241 | val_loss: 0.0311767 | Time: 1460.32 ms [2022-04-06 09:31:42 main:574] : INFO : Epoch 1791 | loss: 0.031126 | val_loss: 0.0311734 | Time: 1458.94 ms [2022-04-06 09:31:44 main:574] : INFO : Epoch 1792 | loss: 0.031123 | val_loss: 0.0311782 | Time: 1457.68 ms [2022-04-06 09:31:45 main:574] : INFO : Epoch 1793 | loss: 0.0311194 | val_loss: 0.0311734 | Time: 1453.27 ms [2022-04-06 09:31:47 main:574] : INFO : Epoch 1794 | loss: 0.031119 | val_loss: 0.0311737 | Time: 1459.14 ms [2022-04-06 09:31:48 main:574] : INFO : Epoch 1795 | loss: 0.0311213 | val_loss: 0.0311768 | Time: 1455.56 ms [2022-04-06 09:31:50 main:574] : INFO : Epoch 1796 | loss: 0.0311218 | val_loss: 0.0311708 | Time: 1462.89 ms [2022-04-06 09:31:52 main:574] : INFO : Epoch 1797 | loss: 0.0311215 | val_loss: 0.0311726 | Time: 1435.15 ms [2022-04-06 09:31:53 main:574] : INFO : Epoch 1798 | loss: 0.0311219 | val_loss: 0.0311738 | Time: 1472.34 ms [2022-04-06 09:31:55 main:574] : INFO : Epoch 1799 | loss: 0.0311252 | val_loss: 0.0311738 | Time: 1543.8 ms [2022-04-06 09:31:56 main:574] : INFO : Epoch 1800 | loss: 0.0311243 | val_loss: 0.0311775 | Time: 1525.27 ms [2022-04-06 09:31:58 main:574] : INFO : Epoch 1801 | loss: 0.0311209 | val_loss: 0.0311709 | Time: 1471.53 ms [2022-04-06 09:31:59 main:574] : INFO : Epoch 1802 | loss: 0.0311201 | val_loss: 0.0311749 | Time: 1456.26 ms [2022-04-06 09:32:01 main:574] : INFO : Epoch 1803 | loss: 0.0311188 | val_loss: 0.0311725 | Time: 1461.83 ms [2022-04-06 09:32:02 main:574] : INFO : Epoch 1804 | loss: 0.031118 | val_loss: 0.0311794 | Time: 1471.94 ms [2022-04-06 09:32:04 main:574] : INFO : Epoch 1805 | loss: 0.0311209 | val_loss: 0.0311719 | Time: 1465.02 ms [2022-04-06 09:32:05 main:574] : INFO : Epoch 1806 | loss: 0.0311233 | val_loss: 0.031173 | Time: 1466.36 ms [2022-04-06 09:32:07 main:574] : INFO : Epoch 1807 | loss: 0.0311197 | val_loss: 0.0311777 | Time: 1475.99 ms [2022-04-06 09:32:08 main:574] : INFO : Epoch 1808 | loss: 0.0311184 | val_loss: 0.031173 | Time: 1473.75 ms [2022-04-06 09:32:10 main:574] : INFO : Epoch 1809 | loss: 0.0311173 | val_loss: 0.0311818 | Time: 1466.07 ms [2022-04-06 09:32:12 main:574] : INFO : Epoch 1810 | loss: 0.0311204 | val_loss: 0.0311723 | Time: 1488.91 ms [2022-04-06 09:32:13 main:574] : INFO : Epoch 1811 | loss: 0.0311206 | val_loss: 0.0311793 | Time: 1482.85 ms [2022-04-06 09:32:15 main:574] : INFO : Epoch 1812 | loss: 0.0311169 | val_loss: 0.0311729 | Time: 1456.21 ms [2022-04-06 09:32:16 main:574] : INFO : Epoch 1813 | loss: 0.031123 | val_loss: 0.0311794 | Time: 1474.12 ms [2022-04-06 09:32:18 main:574] : INFO : Epoch 1814 | loss: 0.03112 | val_loss: 0.0311694 | Time: 1471.05 ms [2022-04-06 09:32:19 main:574] : INFO : Epoch 1815 | loss: 0.0311187 | val_loss: 0.0311855 | Time: 1475.39 ms [2022-04-06 09:32:21 main:574] : INFO : Epoch 1816 | loss: 0.0311208 | val_loss: 0.0311812 | Time: 1452.49 ms [2022-04-06 09:32:22 main:574] : INFO : Epoch 1817 | loss: 0.0311207 | val_loss: 0.0311848 | Time: 1462.55 ms [2022-04-06 09:32:24 main:574] : INFO : Epoch 1818 | loss: 0.0311193 | val_loss: 0.0311822 | Time: 1462.39 ms [2022-04-06 09:32:25 main:574] : INFO : Epoch 1819 | loss: 0.0311167 | val_loss: 0.0311746 | Time: 1469.29 ms [2022-04-06 09:32:27 main:574] : INFO : Epoch 1820 | loss: 0.0311221 | val_loss: 0.0311882 | Time: 1458.36 ms [2022-04-06 09:32:28 main:574] : INFO : Epoch 1821 | loss: 0.0311217 | val_loss: 0.0311853 | Time: 1451.44 ms [2022-04-06 09:32:30 main:574] : INFO : Epoch 1822 | loss: 0.0311231 | val_loss: 0.0311772 | Time: 1483.26 ms [2022-04-06 09:32:31 main:574] : INFO : Epoch 1823 | loss: 0.0311307 | val_loss: 0.0311854 | Time: 1480.11 ms [2022-04-06 09:32:33 main:574] : INFO : Epoch 1824 | loss: 0.0311214 | val_loss: 0.0311788 | Time: 1474.06 ms [2022-04-06 09:32:34 main:574] : INFO : Epoch 1825 | loss: 0.0311146 | val_loss: 0.0311829 | Time: 1468.37 ms [2022-04-06 09:32:36 main:574] : INFO : Epoch 1826 | loss: 0.0311139 | val_loss: 0.0311817 | Time: 1470.37 ms [2022-04-06 09:32:38 main:574] : INFO : Epoch 1827 | loss: 0.0311189 | val_loss: 0.031177 | Time: 1478.06 ms [2022-04-06 09:32:39 main:574] : INFO : Epoch 1828 | loss: 0.0311208 | val_loss: 0.0311769 | Time: 1462.13 ms [2022-04-06 09:32:41 main:574] : INFO : Epoch 1829 | loss: 0.0311199 | val_loss: 0.0311791 | Time: 1451.74 ms [2022-04-06 09:32:42 main:574] : INFO : Epoch 1830 | loss: 0.0311218 | val_loss: 0.0311826 | Time: 1478.84 ms [2022-04-06 09:32:44 main:574] : INFO : Epoch 1831 | loss: 0.0311236 | val_loss: 0.0311922 | Time: 1473.48 ms [2022-04-06 09:32:45 main:574] : INFO : Epoch 1832 | loss: 0.0311175 | val_loss: 0.0311853 | Time: 1474.3 ms [2022-04-06 09:32:47 main:574] : INFO : Epoch 1833 | loss: 0.0311153 | val_loss: 0.0311905 | Time: 1457.73 ms [2022-04-06 09:32:48 main:574] : INFO : Epoch 1834 | loss: 0.0311158 | val_loss: 0.0311818 | Time: 1483.82 ms [2022-04-06 09:32:50 main:574] : INFO : Epoch 1835 | loss: 0.0311131 | val_loss: 0.0311804 | Time: 1501.29 ms [2022-04-06 09:32:51 main:574] : INFO : Epoch 1836 | loss: 0.0311123 | val_loss: 0.031193 | Time: 1520.18 ms [2022-04-06 09:32:53 main:574] : INFO : Epoch 1837 | loss: 0.0311122 | val_loss: 0.0311879 | Time: 1501.92 ms [2022-04-06 09:32:54 main:574] : INFO : Epoch 1838 | loss: 0.0311119 | val_loss: 0.0311851 | Time: 1458.57 ms [2022-04-06 09:32:56 main:574] : INFO : Epoch 1839 | loss: 0.0311099 | val_loss: 0.0311805 | Time: 1472.86 ms [2022-04-06 09:32:58 main:574] : INFO : Epoch 1840 | loss: 0.0311093 | val_loss: 0.0311903 | Time: 1454.25 ms [2022-04-06 09:32:59 main:574] : INFO : Epoch 1841 | loss: 0.0311097 | val_loss: 0.031178 | Time: 1450.66 ms [2022-04-06 09:33:01 main:574] : INFO : Epoch 1842 | loss: 0.0311124 | val_loss: 0.0311823 | Time: 1478.9 ms [2022-04-06 09:33:02 main:574] : INFO : Epoch 1843 | loss: 0.0311129 | val_loss: 0.0311789 | Time: 1474.14 ms [2022-04-06 09:33:04 main:574] : INFO : Epoch 1844 | loss: 0.0311114 | val_loss: 0.0311853 | Time: 1459.2 ms [2022-04-06 09:33:05 main:574] : INFO : Epoch 1845 | loss: 0.0311166 | val_loss: 0.0311854 | Time: 1563.23 ms [2022-04-06 09:33:07 main:574] : INFO : Epoch 1846 | loss: 0.0311157 | val_loss: 0.0311754 | Time: 1475.47 ms [2022-04-06 09:33:08 main:574] : INFO : Epoch 1847 | loss: 0.0311145 | val_loss: 0.0311774 | Time: 1485.01 ms [2022-04-06 09:33:10 main:574] : INFO : Epoch 1848 | loss: 0.0311124 | val_loss: 0.0311874 | Time: 1469.44 ms [2022-04-06 09:33:11 main:574] : INFO : Epoch 1849 | loss: 0.0311108 | val_loss: 0.0311963 | Time: 1473.02 ms [2022-04-06 09:33:13 main:574] : INFO : Epoch 1850 | loss: 0.0311146 | val_loss: 0.0311796 | Time: 1447.31 ms [2022-04-06 09:33:14 main:574] : INFO : Epoch 1851 | loss: 0.0311117 | val_loss: 0.0311726 | Time: 1460.08 ms [2022-04-06 09:33:16 main:574] : INFO : Epoch 1852 | loss: 0.0311116 | val_loss: 0.0311924 | Time: 1468.32 ms [2022-04-06 09:33:18 main:574] : INFO : Epoch 1853 | loss: 0.0311167 | val_loss: 0.031182 | Time: 1469.45 ms [2022-04-06 09:33:19 main:574] : INFO : Epoch 1854 | loss: 0.0311143 | val_loss: 0.0311829 | Time: 1473.17 ms [2022-04-06 09:33:21 main:574] : INFO : Epoch 1855 | loss: 0.0311119 | val_loss: 0.0311863 | Time: 1452.59 ms [2022-04-06 09:33:22 main:574] : INFO : Epoch 1856 | loss: 0.0311139 | val_loss: 0.0311863 | Time: 1459.02 ms [2022-04-06 09:33:24 main:574] : INFO : Epoch 1857 | loss: 0.0311091 | val_loss: 0.0311911 | Time: 1471.91 ms [2022-04-06 09:33:25 main:574] : INFO : Epoch 1858 | loss: 0.0311087 | val_loss: 0.0311809 | Time: 1479.33 ms [2022-04-06 09:33:27 main:574] : INFO : Epoch 1859 | loss: 0.0311129 | val_loss: 0.0311907 | Time: 1462.64 ms [2022-04-06 09:33:28 main:574] : INFO : Epoch 1860 | loss: 0.0311117 | val_loss: 0.0311785 | Time: 1483.45 ms [2022-04-06 09:33:30 main:574] : INFO : Epoch 1861 | loss: 0.0311111 | val_loss: 0.0311784 | Time: 1466.25 ms [2022-04-06 09:33:31 main:574] : INFO : Epoch 1862 | loss: 0.0311116 | val_loss: 0.0311795 | Time: 1476.16 ms [2022-04-06 09:33:33 main:574] : INFO : Epoch 1863 | loss: 0.0311093 | val_loss: 0.0311743 | Time: 1503.37 ms [2022-04-06 09:33:34 main:574] : INFO : Epoch 1864 | loss: 0.0311133 | val_loss: 0.0311812 | Time: 1463.67 ms [2022-04-06 09:33:36 main:574] : INFO : Epoch 1865 | loss: 0.0311182 | val_loss: 0.0311737 | Time: 1468.43 ms [2022-04-06 09:33:37 main:574] : INFO : Epoch 1866 | loss: 0.0311154 | val_loss: 0.0311848 | Time: 1468.01 ms [2022-04-06 09:33:39 main:574] : INFO : Epoch 1867 | loss: 0.0311161 | val_loss: 0.0311796 | Time: 1467.04 ms [2022-04-06 09:33:40 main:574] : INFO : Epoch 1868 | loss: 0.0311137 | val_loss: 0.0311763 | Time: 1471.48 ms [2022-04-06 09:33:42 main:574] : INFO : Epoch 1869 | loss: 0.0311162 | val_loss: 0.0311778 | Time: 1467.14 ms [2022-04-06 09:33:43 main:574] : INFO : Epoch 1870 | loss: 0.0311127 | val_loss: 0.0311887 | Time: 1461.06 ms [2022-04-06 09:33:45 main:574] : INFO : Epoch 1871 | loss: 0.031113 | val_loss: 0.0311796 | Time: 1473.6 ms [2022-04-06 09:33:47 main:574] : INFO : Epoch 1872 | loss: 0.0311137 | val_loss: 0.0311865 | Time: 1462.07 ms [2022-04-06 09:33:48 main:574] : INFO : Epoch 1873 | loss: 0.0311082 | val_loss: 0.0311797 | Time: 1460.11 ms [2022-04-06 09:33:50 main:574] : INFO : Epoch 1874 | loss: 0.0311081 | val_loss: 0.0311839 | Time: 1477.47 ms [2022-04-06 09:33:51 main:574] : INFO : Epoch 1875 | loss: 0.0311202 | val_loss: 0.0311826 | Time: 1467.85 ms [2022-04-06 09:33:53 main:574] : INFO : Epoch 1876 | loss: 0.0311238 | val_loss: 0.0311756 | Time: 1471.17 ms [2022-04-06 09:33:54 main:574] : INFO : Epoch 1877 | loss: 0.031117 | val_loss: 0.0311767 | Time: 1466.91 ms [2022-04-06 09:33:56 main:574] : INFO : Epoch 1878 | loss: 0.0311185 | val_loss: 0.0311757 | Time: 1447.33 ms [2022-04-06 09:33:57 main:574] : INFO : Epoch 1879 | loss: 0.0311153 | val_loss: 0.0311825 | Time: 1467.39 ms [2022-04-06 09:33:59 main:574] : INFO : Epoch 1880 | loss: 0.0311115 | val_loss: 0.0311896 | Time: 1458.8 ms [2022-04-06 09:34:00 main:574] : INFO : Epoch 1881 | loss: 0.0311122 | val_loss: 0.0311759 | Time: 1475.85 ms [2022-04-06 09:34:02 main:574] : INFO : Epoch 1882 | loss: 0.0311177 | val_loss: 0.0311837 | Time: 1485.24 ms [2022-04-06 09:34:03 main:574] : INFO : Epoch 1883 | loss: 0.0311209 | val_loss: 0.0311745 | Time: 1478.85 ms [2022-04-06 09:34:05 main:574] : INFO : Epoch 1884 | loss: 0.0311167 | val_loss: 0.0311898 | Time: 1459.93 ms [2022-04-06 09:34:06 main:574] : INFO : Epoch 1885 | loss: 0.0311107 | val_loss: 0.0311841 | Time: 1474.32 ms [2022-04-06 09:34:08 main:574] : INFO : Epoch 1886 | loss: 0.0311111 | val_loss: 0.0311953 | Time: 1507.61 ms [2022-04-06 09:34:10 main:574] : INFO : Epoch 1887 | loss: 0.0311085 | val_loss: 0.0311786 | Time: 1471.15 ms [2022-04-06 09:34:11 main:574] : INFO : Epoch 1888 | loss: 0.0311083 | val_loss: 0.0311929 | Time: 1481.09 ms [2022-04-06 09:34:13 main:574] : INFO : Epoch 1889 | loss: 0.0311079 | val_loss: 0.0311908 | Time: 1457.82 ms [2022-04-06 09:34:14 main:574] : INFO : Epoch 1890 | loss: 0.0311097 | val_loss: 0.0311847 | Time: 1462.95 ms [2022-04-06 09:34:16 main:574] : INFO : Epoch 1891 | loss: 0.0311077 | val_loss: 0.0311957 | Time: 1476.26 ms [2022-04-06 09:34:17 main:574] : INFO : Epoch 1892 | loss: 0.0311027 | val_loss: 0.0311853 | Time: 1483.85 ms [2022-04-06 09:34:19 main:574] : INFO : Epoch 1893 | loss: 0.0311069 | val_loss: 0.0311912 | Time: 1465.86 ms [2022-04-06 09:34:20 main:574] : INFO : Epoch 1894 | loss: 0.0311091 | val_loss: 0.03119 | Time: 1466.25 ms [2022-04-06 09:34:22 main:574] : INFO : Epoch 1895 | loss: 0.0311137 | val_loss: 0.0311813 | Time: 1467.13 ms [2022-04-06 09:34:23 main:574] : INFO : Epoch 1896 | loss: 0.0311146 | val_loss: 0.031181 | Time: 1450.24 ms [2022-04-06 09:34:25 main:574] : INFO : Epoch 1897 | loss: 0.0311085 | val_loss: 0.0311859 | Time: 1466.13 ms [2022-04-06 09:34:26 main:574] : INFO : Epoch 1898 | loss: 0.0311095 | val_loss: 0.0311927 | Time: 1466.41 ms [2022-04-06 09:34:28 main:574] : INFO : Epoch 1899 | loss: 0.031107 | val_loss: 0.0311908 | Time: 1466.29 ms [2022-04-06 09:34:29 main:574] : INFO : Epoch 1900 | loss: 0.0311073 | val_loss: 0.0311944 | Time: 1470.24 ms [2022-04-06 09:34:31 main:574] : INFO : Epoch 1901 | loss: 0.031114 | val_loss: 0.0311824 | Time: 1480.41 ms [2022-04-06 09:34:32 main:574] : INFO : Epoch 1902 | loss: 0.0311078 | val_loss: 0.0311885 | Time: 1476.87 ms [2022-04-06 09:34:34 main:574] : INFO : Epoch 1903 | loss: 0.0311082 | val_loss: 0.0311887 | Time: 1464.5 ms [2022-04-06 09:34:36 main:574] : INFO : Epoch 1904 | loss: 0.0311063 | val_loss: 0.0311776 | Time: 1488.77 ms [2022-04-06 09:34:37 main:574] : INFO : Epoch 1905 | loss: 0.0311086 | val_loss: 0.0311948 | Time: 1474.04 ms [2022-04-06 09:34:39 main:574] : INFO : Epoch 1906 | loss: 0.031111 | val_loss: 0.0311812 | Time: 1465.13 ms [2022-04-06 09:34:40 main:574] : INFO : Epoch 1907 | loss: 0.0311135 | val_loss: 0.0311955 | Time: 1483.85 ms [2022-04-06 09:34:42 main:574] : INFO : Epoch 1908 | loss: 0.0311125 | val_loss: 0.0311945 | Time: 1499.45 ms [2022-04-06 09:34:43 main:574] : INFO : Epoch 1909 | loss: 0.0311099 | val_loss: 0.0311991 | Time: 1468.89 ms [2022-04-06 09:34:45 main:574] : INFO : Epoch 1910 | loss: 0.0311039 | val_loss: 0.0311803 | Time: 1517.6 ms [2022-04-06 09:34:46 main:574] : INFO : Epoch 1911 | loss: 0.031103 | val_loss: 0.0311814 | Time: 1491.9 ms [2022-04-06 09:34:48 main:574] : INFO : Epoch 1912 | loss: 0.0311067 | val_loss: 0.0311839 | Time: 1486.89 ms [2022-04-06 09:34:49 main:574] : INFO : Epoch 1913 | loss: 0.0311031 | val_loss: 0.0311909 | Time: 1482.89 ms [2022-04-06 09:34:51 main:574] : INFO : Epoch 1914 | loss: 0.0311088 | val_loss: 0.0311869 | Time: 1481.49 ms [2022-04-06 09:34:53 main:574] : INFO : Epoch 1915 | loss: 0.0311115 | val_loss: 0.0311878 | Time: 1478.24 ms [2022-04-06 09:34:54 main:574] : INFO : Epoch 1916 | loss: 0.0311045 | val_loss: 0.0311832 | Time: 1477.13 ms [2022-04-06 09:34:56 main:574] : INFO : Epoch 1917 | loss: 0.0311046 | val_loss: 0.0311832 | Time: 1480.69 ms [2022-04-06 09:34:57 main:574] : INFO : Epoch 1918 | loss: 0.0311024 | val_loss: 0.0311841 | Time: 1485.37 ms [2022-04-06 09:34:59 main:574] : INFO : Epoch 1919 | loss: 0.0311056 | val_loss: 0.0311853 | Time: 1485.78 ms [2022-04-06 09:35:00 main:574] : INFO : Epoch 1920 | loss: 0.0311059 | val_loss: 0.0311956 | Time: 1506.75 ms [2022-04-06 09:35:02 main:574] : INFO : Epoch 1921 | loss: 0.0311083 | val_loss: 0.0311832 | Time: 1483.57 ms [2022-04-06 09:35:03 main:574] : INFO : Epoch 1922 | loss: 0.0311154 | val_loss: 0.031186 | Time: 1470.04 ms [2022-04-06 09:35:05 main:574] : INFO : Epoch 1923 | loss: 0.0311112 | val_loss: 0.0311798 | Time: 1476.39 ms [2022-04-06 09:35:06 main:574] : INFO : Epoch 1924 | loss: 0.0311129 | val_loss: 0.03118 | Time: 1482.2 ms [2022-04-06 09:35:08 main:574] : INFO : Epoch 1925 | loss: 0.031106 | val_loss: 0.0312047 | Time: 1485.5 ms [2022-04-06 09:35:09 main:574] : INFO : Epoch 1926 | loss: 0.0311012 | val_loss: 0.0311844 | Time: 1472.88 ms [2022-04-06 09:35:11 main:574] : INFO : Epoch 1927 | loss: 0.0311008 | val_loss: 0.0311943 | Time: 1496.67 ms [2022-04-06 09:35:13 main:574] : INFO : Epoch 1928 | loss: 0.0311031 | val_loss: 0.0311959 | Time: 1485.42 ms [2022-04-06 09:35:14 main:574] : INFO : Epoch 1929 | loss: 0.0311058 | val_loss: 0.0311983 | Time: 1494.01 ms [2022-04-06 09:35:16 main:574] : INFO : Epoch 1930 | loss: 0.0311034 | val_loss: 0.0311851 | Time: 1479.76 ms [2022-04-06 09:35:17 main:574] : INFO : Epoch 1931 | loss: 0.0311013 | val_loss: 0.0311936 | Time: 1472.69 ms [2022-04-06 09:35:19 main:574] : INFO : Epoch 1932 | loss: 0.0311024 | val_loss: 0.0311895 | Time: 1477.73 ms [2022-04-06 09:35:20 main:574] : INFO : Epoch 1933 | loss: 0.0311075 | val_loss: 0.0311952 | Time: 1493.37 ms [2022-04-06 09:35:22 main:574] : INFO : Epoch 1934 | loss: 0.0311086 | val_loss: 0.0311896 | Time: 1497.9 ms [2022-04-06 09:35:23 main:574] : INFO : Epoch 1935 | loss: 0.0311127 | val_loss: 0.0311941 | Time: 1473.38 ms [2022-04-06 09:35:25 main:574] : INFO : Epoch 1936 | loss: 0.0311094 | val_loss: 0.0311929 | Time: 1460.71 ms [2022-04-06 09:35:26 main:574] : INFO : Epoch 1937 | loss: 0.0311075 | val_loss: 0.0311904 | Time: 1466.86 ms [2022-04-06 09:35:28 main:574] : INFO : Epoch 1938 | loss: 0.0311029 | val_loss: 0.0311931 | Time: 1480.81 ms [2022-04-06 09:35:29 main:574] : INFO : Epoch 1939 | loss: 0.0311034 | val_loss: 0.0311997 | Time: 1463.79 ms [2022-04-06 09:35:31 main:574] : INFO : Epoch 1940 | loss: 0.0311079 | val_loss: 0.031181 | Time: 1491.27 ms [2022-04-06 09:35:33 main:574] : INFO : Epoch 1941 | loss: 0.0311051 | val_loss: 0.0311899 | Time: 1575.45 ms [2022-04-06 09:35:34 main:574] : INFO : Epoch 1942 | loss: 0.0311042 | val_loss: 0.031193 | Time: 1476.25 ms [2022-04-06 09:35:36 main:574] : INFO : Epoch 1943 | loss: 0.0311004 | val_loss: 0.0311968 | Time: 1467.4 ms [2022-04-06 09:35:37 main:574] : INFO : Epoch 1944 | loss: 0.031101 | val_loss: 0.0311853 | Time: 1470.86 ms [2022-04-06 09:35:39 main:574] : INFO : Epoch 1945 | loss: 0.0311038 | val_loss: 0.0312006 | Time: 1466.51 ms [2022-04-06 09:35:40 main:574] : INFO : Epoch 1946 | loss: 0.0311047 | val_loss: 0.0311841 | Time: 1469.53 ms [2022-04-06 09:35:42 main:574] : INFO : Epoch 1947 | loss: 0.0311018 | val_loss: 0.031189 | Time: 1490.58 ms [2022-04-06 09:35:43 main:574] : INFO : Epoch 1948 | loss: 0.031099 | val_loss: 0.0311852 | Time: 1493.99 ms [2022-04-06 09:35:45 main:574] : INFO : Epoch 1949 | loss: 0.0311006 | val_loss: 0.0312032 | Time: 1483.96 ms [2022-04-06 09:35:46 main:574] : INFO : Epoch 1950 | loss: 0.031103 | val_loss: 0.0311909 | Time: 1464.39 ms [2022-04-06 09:35:48 main:574] : INFO : Epoch 1951 | loss: 0.0311011 | val_loss: 0.0311887 | Time: 1468.6 ms [2022-04-06 09:35:50 main:574] : INFO : Epoch 1952 | loss: 0.0311013 | val_loss: 0.0312102 | Time: 1474.64 ms [2022-04-06 09:35:51 main:574] : INFO : Epoch 1953 | loss: 0.0311067 | val_loss: 0.0311943 | Time: 1470.87 ms [2022-04-06 09:35:53 main:574] : INFO : Epoch 1954 | loss: 0.0311039 | val_loss: 0.0311841 | Time: 1467.16 ms [2022-04-06 09:35:54 main:574] : INFO : Epoch 1955 | loss: 0.031102 | val_loss: 0.0311932 | Time: 1478.22 ms [2022-04-06 09:35:56 main:574] : INFO : Epoch 1956 | loss: 0.0311049 | val_loss: 0.031191 | Time: 1467.53 ms [2022-04-06 09:35:57 main:574] : INFO : Epoch 1957 | loss: 0.0311269 | val_loss: 0.0311759 | Time: 1468.33 ms [2022-04-06 09:35:59 main:574] : INFO : Epoch 1958 | loss: 0.031125 | val_loss: 0.0311825 | Time: 1472.83 ms [2022-04-06 09:36:00 main:574] : INFO : Epoch 1959 | loss: 0.031115 | val_loss: 0.031181 | Time: 1475.83 ms [2022-04-06 09:36:02 main:574] : INFO : Epoch 1960 | loss: 0.0311082 | val_loss: 0.031196 | Time: 1461.29 ms [2022-04-06 09:36:03 main:574] : INFO : Epoch 1961 | loss: 0.0311067 | val_loss: 0.0311859 | Time: 1462.61 ms [2022-04-06 09:36:05 main:574] : INFO : Epoch 1962 | loss: 0.0311004 | val_loss: 0.0311974 | Time: 1472.67 ms [2022-04-06 09:36:06 main:574] : INFO : Epoch 1963 | loss: 0.0311013 | val_loss: 0.0311907 | Time: 1488.72 ms [2022-04-06 09:36:08 main:574] : INFO : Epoch 1964 | loss: 0.0311005 | val_loss: 0.0311867 | Time: 1480.13 ms [2022-04-06 09:36:10 main:574] : INFO : Epoch 1965 | loss: 0.0311067 | val_loss: 0.0311815 | Time: 1472.11 ms [2022-04-06 09:36:11 main:574] : INFO : Epoch 1966 | loss: 0.0311094 | val_loss: 0.0311851 | Time: 1476.1 ms [2022-04-06 09:36:13 main:574] : INFO : Epoch 1967 | loss: 0.031102 | val_loss: 0.0311825 | Time: 1476.23 ms [2022-04-06 09:36:14 main:574] : INFO : Epoch 1968 | loss: 0.031105 | val_loss: 0.0311846 | Time: 1475.82 ms [2022-04-06 09:36:16 main:574] : INFO : Epoch 1969 | loss: 0.0311002 | val_loss: 0.0311963 | Time: 1482.24 ms [2022-04-06 09:36:17 main:574] : INFO : Epoch 1970 | loss: 0.0311023 | val_loss: 0.0311914 | Time: 1463.51 ms [2022-04-06 09:36:19 main:574] : INFO : Epoch 1971 | loss: 0.0311001 | val_loss: 0.0311878 | Time: 1465.35 ms [2022-04-06 09:36:20 main:574] : INFO : Epoch 1972 | loss: 0.0310995 | val_loss: 0.0311975 | Time: 1474.78 ms [2022-04-06 09:36:22 main:574] : INFO : Epoch 1973 | loss: 0.0311009 | val_loss: 0.0311884 | Time: 1469.62 ms [2022-04-06 09:36:23 main:574] : INFO : Epoch 1974 | loss: 0.0311147 | val_loss: 0.0311841 | Time: 1479.38 ms [2022-04-06 09:36:25 main:574] : INFO : Epoch 1975 | loss: 0.0311109 | val_loss: 0.0311956 | Time: 1479.32 ms [2022-04-06 09:36:26 main:574] : INFO : Epoch 1976 | loss: 0.0310994 | val_loss: 0.0311993 | Time: 1470.05 ms [2022-04-06 09:36:28 main:574] : INFO : Epoch 1977 | loss: 0.0311009 | val_loss: 0.0311923 | Time: 1467.6 ms [2022-04-06 09:36:29 main:574] : INFO : Epoch 1978 | loss: 0.0310978 | val_loss: 0.0311935 | Time: 1478.71 ms [2022-04-06 09:36:31 main:574] : INFO : Epoch 1979 | loss: 0.0310987 | val_loss: 0.0311864 | Time: 1483.06 ms [2022-04-06 09:36:33 main:574] : INFO : Epoch 1980 | loss: 0.0311054 | val_loss: 0.0312096 | Time: 1480.22 ms [2022-04-06 09:36:34 main:574] : INFO : Epoch 1981 | loss: 0.0310998 | val_loss: 0.0311801 | Time: 1478.98 ms [2022-04-06 09:36:36 main:574] : INFO : Epoch 1982 | loss: 0.0311042 | val_loss: 0.0311884 | Time: 1470.01 ms [2022-04-06 09:36:37 main:574] : INFO : Epoch 1983 | loss: 0.0311042 | val_loss: 0.0311945 | Time: 1483.85 ms [2022-04-06 09:36:39 main:574] : INFO : Epoch 1984 | loss: 0.0311074 | val_loss: 0.0311921 | Time: 1478.34 ms [2022-04-06 09:36:40 main:574] : INFO : Epoch 1985 | loss: 0.0311044 | val_loss: 0.0311824 | Time: 1475.73 ms [2022-04-06 09:36:42 main:574] : INFO : Epoch 1986 | loss: 0.0311138 | val_loss: 0.0311849 | Time: 1466.63 ms [2022-04-06 09:36:43 main:574] : INFO : Epoch 1987 | loss: 0.0311018 | val_loss: 0.0311927 | Time: 1469.77 ms [2022-04-06 09:36:45 main:574] : INFO : Epoch 1988 | loss: 0.0310984 | val_loss: 0.0311844 | Time: 1482.25 ms [2022-04-06 09:36:46 main:574] : INFO : Epoch 1989 | loss: 0.0311005 | val_loss: 0.0312016 | Time: 1487.12 ms [2022-04-06 09:36:48 main:574] : INFO : Epoch 1990 | loss: 0.0311077 | val_loss: 0.0311879 | Time: 1480.05 ms [2022-04-06 09:36:49 main:574] : INFO : Epoch 1991 | loss: 0.0311144 | val_loss: 0.0311791 | Time: 1465.35 ms [2022-04-06 09:36:51 main:574] : INFO : Epoch 1992 | loss: 0.0311061 | val_loss: 0.0311933 | Time: 1486.19 ms [2022-04-06 09:36:52 main:574] : INFO : Epoch 1993 | loss: 0.0311011 | val_loss: 0.0311942 | Time: 1473.23 ms [2022-04-06 09:36:54 main:574] : INFO : Epoch 1994 | loss: 0.0311055 | val_loss: 0.0312019 | Time: 1473.71 ms [2022-04-06 09:36:56 main:574] : INFO : Epoch 1995 | loss: 0.0311019 | val_loss: 0.0311853 | Time: 1465.82 ms [2022-04-06 09:36:57 main:574] : INFO : Epoch 1996 | loss: 0.0311014 | val_loss: 0.0311862 | Time: 1461.4 ms [2022-04-06 09:36:59 main:574] : INFO : Epoch 1997 | loss: 0.031106 | val_loss: 0.0312011 | Time: 1457.26 ms [2022-04-06 09:37:00 main:574] : INFO : Epoch 1998 | loss: 0.0311003 | val_loss: 0.0311899 | Time: 1460.85 ms [2022-04-06 09:37:02 main:574] : INFO : Epoch 1999 | loss: 0.0310967 | val_loss: 0.0311957 | Time: 1482.27 ms [2022-04-06 09:37:03 main:574] : INFO : Epoch 2000 | loss: 0.0311042 | val_loss: 0.0311735 | Time: 1474.47 ms [2022-04-06 09:37:05 main:574] : INFO : Epoch 2001 | loss: 0.0311159 | val_loss: 0.0311871 | Time: 1483.09 ms [2022-04-06 09:37:06 main:574] : INFO : Epoch 2002 | loss: 0.0311087 | val_loss: 0.0311886 | Time: 1470.1 ms [2022-04-06 09:37:08 main:574] : INFO : Epoch 2003 | loss: 0.0311004 | val_loss: 0.0311879 | Time: 1459.58 ms [2022-04-06 09:37:09 main:574] : INFO : Epoch 2004 | loss: 0.0311001 | val_loss: 0.0311887 | Time: 1478.33 ms [2022-04-06 09:37:11 main:574] : INFO : Epoch 2005 | loss: 0.0310976 | val_loss: 0.0312104 | Time: 1459.47 ms [2022-04-06 09:37:12 main:574] : INFO : Epoch 2006 | loss: 0.0311068 | val_loss: 0.0311842 | Time: 1485.74 ms [2022-04-06 09:37:14 main:574] : INFO : Epoch 2007 | loss: 0.0310975 | val_loss: 0.0311982 | Time: 1492.29 ms [2022-04-06 09:37:15 main:574] : INFO : Epoch 2008 | loss: 0.0311018 | val_loss: 0.0311892 | Time: 1480.27 ms [2022-04-06 09:37:17 main:574] : INFO : Epoch 2009 | loss: 0.0310995 | val_loss: 0.0312015 | Time: 1468.84 ms [2022-04-06 09:37:18 main:574] : INFO : Epoch 2010 | loss: 0.0310964 | val_loss: 0.0311961 | Time: 1473.89 ms [2022-04-06 09:37:20 main:574] : INFO : Epoch 2011 | loss: 0.0310947 | val_loss: 0.0311981 | Time: 1465.56 ms [2022-04-06 09:37:22 main:574] : INFO : Epoch 2012 | loss: 0.0310946 | val_loss: 0.0311992 | Time: 1460.25 ms [2022-04-06 09:37:23 main:574] : INFO : Epoch 2013 | loss: 0.0310975 | val_loss: 0.03119 | Time: 1468.54 ms [2022-04-06 09:37:25 main:574] : INFO : Epoch 2014 | loss: 0.0310923 | val_loss: 0.031193 | Time: 1463.93 ms [2022-04-06 09:37:26 main:574] : INFO : Epoch 2015 | loss: 0.0310978 | val_loss: 0.0311904 | Time: 1474.33 ms [2022-04-06 09:37:28 main:574] : INFO : Epoch 2016 | loss: 0.0310929 | val_loss: 0.0312026 | Time: 1467.46 ms [2022-04-06 09:37:29 main:574] : INFO : Epoch 2017 | loss: 0.0310991 | val_loss: 0.0311926 | Time: 1456.32 ms [2022-04-06 09:37:31 main:574] : INFO : Epoch 2018 | loss: 0.0310979 | val_loss: 0.0312093 | Time: 1472.16 ms [2022-04-06 09:37:32 main:574] : INFO : Epoch 2019 | loss: 0.0311015 | val_loss: 0.0312078 | Time: 1461.71 ms [2022-04-06 09:37:34 main:574] : INFO : Epoch 2020 | loss: 0.0311014 | val_loss: 0.0311884 | Time: 1466.58 ms [2022-04-06 09:37:35 main:574] : INFO : Epoch 2021 | loss: 0.0311004 | val_loss: 0.0311929 | Time: 1481.38 ms [2022-04-06 09:37:37 main:574] : INFO : Epoch 2022 | loss: 0.0310937 | val_loss: 0.0312017 | Time: 1474.28 ms [2022-04-06 09:37:38 main:574] : INFO : Epoch 2023 | loss: 0.0310914 | val_loss: 0.0312004 | Time: 1477.71 ms [2022-04-06 09:37:40 main:574] : INFO : Epoch 2024 | loss: 0.0310973 | val_loss: 0.0311921 | Time: 1473.28 ms [2022-04-06 09:37:41 main:574] : INFO : Epoch 2025 | loss: 0.0310912 | val_loss: 0.0312071 | Time: 1480.12 ms [2022-04-06 09:37:43 main:574] : INFO : Epoch 2026 | loss: 0.0310953 | val_loss: 0.0312036 | Time: 1470.95 ms [2022-04-06 09:37:44 main:574] : INFO : Epoch 2027 | loss: 0.0310953 | val_loss: 0.031192 | Time: 1482.53 ms [2022-04-06 09:37:46 main:574] : INFO : Epoch 2028 | loss: 0.0310996 | val_loss: 0.0311966 | Time: 1492.27 ms [2022-04-06 09:37:48 main:574] : INFO : Epoch 2029 | loss: 0.0310973 | val_loss: 0.0311972 | Time: 1475.85 ms [2022-04-06 09:37:49 main:574] : INFO : Epoch 2030 | loss: 0.0310955 | val_loss: 0.0312077 | Time: 1482.49 ms [2022-04-06 09:37:51 main:574] : INFO : Epoch 2031 | loss: 0.0310991 | val_loss: 0.0311966 | Time: 1478.42 ms [2022-04-06 09:37:52 main:574] : INFO : Epoch 2032 | loss: 0.0311014 | val_loss: 0.031197 | Time: 1470.12 ms [2022-04-06 09:37:54 main:574] : INFO : Epoch 2033 | loss: 0.0310945 | val_loss: 0.0311835 | Time: 1476.48 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2070 SUPER) Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2070 SUPER) [2022-04-06 10:07:36 main:435] : INFO : Set logging level to 1 [2022-04-06 10:07:36 main:441] : INFO : Running in BOINC Client mode [2022-04-06 10:07:36 main:444] : INFO : Resolving all filenames [2022-04-06 10:07:36 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-04-06 10:07:36 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-04-06 10:07:36 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-04-06 10:07:36 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 0) [2022-04-06 10:07:36 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-04-06 10:07:36 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-04-06 10:07:36 main:474] : INFO : Configuration: [2022-04-06 10:07:36 main:475] : INFO : Model type: GRU [2022-04-06 10:07:36 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-04-06 10:07:36 main:477] : INFO : Max Epochs: 2048 [2022-04-06 10:07:36 main:478] : INFO : Batch Size: 128 [2022-04-06 10:07:36 main:479] : INFO : Learning Rate: 0.01 [2022-04-06 10:07:36 main:480] : INFO : Patience: 10 [2022-04-06 10:07:36 main:481] : INFO : Hidden Width: 12 [2022-04-06 10:07:36 main:482] : INFO : # Recurrent Layers: 4 [2022-04-06 10:07:36 main:483] : INFO : # Backend Layers: 4 [2022-04-06 10:07:37 main:484] : INFO : # Threads: 1 [2022-04-06 10:07:37 main:486] : INFO : Preparing Dataset [2022-04-06 10:07:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-04-06 10:07:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-04-06 10:07:39 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-04-06 10:07:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-04-06 10:07:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-04-06 10:07:39 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-04-06 10:07:39 main:494] : INFO : Creating Model [2022-04-06 10:07:39 main:507] : INFO : Preparing config file [2022-04-06 10:07:39 main:511] : INFO : Found checkpoint, attempting to load... [2022-04-06 10:07:40 main:512] : INFO : Loading config [2022-04-06 10:07:40 main:514] : INFO : Loading state [2022-04-06 10:07:41 main:559] : INFO : Loading DataLoader into Memory [2022-04-06 10:07:41 main:562] : INFO : Starting Training [2022-04-06 10:07:43 main:574] : INFO : Epoch 2017 | loss: 0.031166 | val_loss: 0.0312078 | Time: 1794.42 ms [2022-04-06 10:07:44 main:574] : INFO : Epoch 2018 | loss: 0.0311192 | val_loss: 0.0311956 | Time: 1589.23 ms [2022-04-06 10:07:46 main:574] : INFO : Epoch 2019 | loss: 0.0311083 | val_loss: 0.0312014 | Time: 1578.29 ms [2022-04-06 10:07:48 main:574] : INFO : Epoch 2020 | loss: 0.0310967 | val_loss: 0.0312009 | Time: 1534.22 ms [2022-04-06 10:07:49 main:574] : INFO : Epoch 2021 | loss: 0.0310927 | val_loss: 0.0312069 | Time: 1529.36 ms [2022-04-06 10:07:51 main:574] : INFO : Epoch 2022 | loss: 0.031092 | val_loss: 0.0311978 | Time: 1547.34 ms [2022-04-06 10:07:52 main:574] : INFO : Epoch 2023 | loss: 0.0310967 | val_loss: 0.0312004 | Time: 1537.37 ms [2022-04-06 10:07:54 main:574] : INFO : Epoch 2024 | loss: 0.031101 | val_loss: 0.0312013 | Time: 1525.58 ms [2022-04-06 10:07:56 main:574] : INFO : Epoch 2025 | loss: 0.0310954 | val_loss: 0.0311977 | Time: 1519.84 ms [2022-04-06 10:07:57 main:574] : INFO : Epoch 2026 | loss: 0.0310941 | val_loss: 0.0311967 | Time: 1537.83 ms [2022-04-06 10:07:59 main:574] : INFO : Epoch 2027 | loss: 0.0310927 | val_loss: 0.0311982 | Time: 1552.76 ms [2022-04-06 10:08:00 main:574] : INFO : Epoch 2028 | loss: 0.0310964 | val_loss: 0.0311978 | Time: 1546.16 ms [2022-04-06 10:08:02 main:574] : INFO : Epoch 2029 | loss: 0.0310905 | val_loss: 0.0311977 | Time: 1542.55 ms [2022-04-06 10:08:04 main:574] : INFO : Epoch 2030 | loss: 0.0310898 | val_loss: 0.0312027 | Time: 1513.21 ms [2022-04-06 10:08:05 main:574] : INFO : Epoch 2031 | loss: 0.0310878 | val_loss: 0.031202 | Time: 1511.22 ms [2022-04-06 10:08:07 main:574] : INFO : Epoch 2032 | loss: 0.0310965 | val_loss: 0.0311885 | Time: 1522.88 ms [2022-04-06 10:08:08 main:574] : INFO : Epoch 2033 | loss: 0.0310921 | val_loss: 0.0311946 | Time: 1514.51 ms [2022-04-06 10:08:10 main:574] : INFO : Epoch 2034 | loss: 0.031093 | val_loss: 0.031191 | Time: 1531.88 ms [2022-04-06 10:08:11 main:574] : INFO : Epoch 2035 | loss: 0.0310962 | val_loss: 0.0311986 | Time: 1515.05 ms [2022-04-06 10:08:13 main:574] : INFO : Epoch 2036 | loss: 0.0310902 | val_loss: 0.031201 | Time: 1506.24 ms [2022-04-06 10:08:15 main:574] : INFO : Epoch 2037 | loss: 0.0310914 | val_loss: 0.0312031 | Time: 1536.93 ms [2022-04-06 10:08:16 main:574] : INFO : Epoch 2038 | loss: 0.0310927 | val_loss: 0.0311978 | Time: 1490.08 ms [2022-04-06 10:08:18 main:574] : INFO : Epoch 2039 | loss: 0.0310938 | val_loss: 0.0312068 | Time: 1524.6 ms [2022-04-06 10:08:19 main:574] : INFO : Epoch 2040 | loss: 0.0310944 | val_loss: 0.0311935 | Time: 1509.92 ms [2022-04-06 10:08:21 main:574] : INFO : Epoch 2041 | loss: 0.0311053 | val_loss: 0.0311869 | Time: 1523.1 ms [2022-04-06 10:08:22 main:574] : INFO : Epoch 2042 | loss: 0.0311106 | val_loss: 0.0311949 | Time: 1501.35 ms [2022-04-06 10:08:24 main:574] : INFO : Epoch 2043 | loss: 0.0311001 | val_loss: 0.0311908 | Time: 1496.99 ms [2022-04-06 10:08:26 main:574] : INFO : Epoch 2044 | loss: 0.0310955 | val_loss: 0.0312029 | Time: 1503.97 ms [2022-04-06 10:08:27 main:574] : INFO : Epoch 2045 | loss: 0.0310906 | val_loss: 0.0312026 | Time: 1500.68 ms [2022-04-06 10:08:29 main:574] : INFO : Epoch 2046 | loss: 0.031092 | val_loss: 0.0312006 | Time: 1495.87 ms [2022-04-06 10:08:30 main:574] : INFO : Epoch 2047 | loss: 0.0311101 | val_loss: 0.0311812 | Time: 1510.73 ms [2022-04-06 10:08:32 main:574] : INFO : Epoch 2048 | loss: 0.0311068 | val_loss: 0.031192 | Time: 1516.59 ms [2022-04-06 10:08:32 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.031192 [2022-04-06 10:08:32 main:603] : INFO : Saving end state to config to file [2022-04-06 10:08:32 main:608] : INFO : Success, exiting.. 10:08:32 (20644): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)