Task 13625082
| Name | ParityModified-1647041994-8482-2-0_0 |
| Workunit | 10718383 |
| Created | 23 Mar 2022, 3:46:49 UTC |
| Sent | 31 Mar 2022, 5:37:27 UTC |
| Report deadline | 8 Apr 2022, 5:37:27 UTC |
| Received | 1 Apr 2022, 5:16:16 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 6170 |
| Run time | 4 hours 56 min 41 sec |
| CPU time | 3 hours 46 min 23 sec |
| Validate state | Valid |
| Credit | 4,160.00 |
| Device peak FLOPS | 1,130.52 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.57 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> me: 8229.65 ms [2022-03-31 18:52:09 main:574] : INFO : Epoch 1655 | loss: 0.0311021 | val_loss: 0.0311385 | Time: 8856.27 ms [2022-03-31 18:52:17 main:574] : INFO : Epoch 1656 | loss: 0.0310984 | val_loss: 0.031136 | Time: 8284.34 ms [2022-03-31 18:52:29 main:574] : INFO : Epoch 1657 | loss: 0.0310983 | val_loss: 0.0311389 | Time: 12260.9 ms [2022-03-31 18:52:37 main:574] : INFO : Epoch 1658 | loss: 0.0311046 | val_loss: 0.031157 | Time: 8011.87 ms [2022-03-31 18:52:50 main:574] : INFO : Epoch 1659 | loss: 0.0311072 | val_loss: 0.0311433 | Time: 12582.5 ms [2022-03-31 18:52:57 main:574] : INFO : Epoch 1660 | loss: 0.0311113 | val_loss: 0.0311469 | Time: 7267.38 ms [2022-03-31 18:53:10 main:574] : INFO : Epoch 1661 | loss: 0.0311072 | val_loss: 0.0311431 | Time: 12625.4 ms [2022-03-31 18:53:23 main:574] : INFO : Epoch 1662 | loss: 0.031104 | val_loss: 0.0311447 | Time: 12591 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-03-31 18:56:39 main:435] : INFO : Set logging level to 1 [2022-03-31 18:56:39 main:441] : INFO : Running in BOINC Client mode [2022-03-31 18:56:39 main:444] : INFO : Resolving all filenames [2022-03-31 18:56:39 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-31 18:56:39 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-31 18:56:39 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-31 18:56:39 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-31 18:56:39 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-31 18:56:39 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-31 18:56:39 main:474] : INFO : Configuration: [2022-03-31 18:56:39 main:475] : INFO : Model type: GRU [2022-03-31 18:56:40 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-31 18:56:40 main:477] : INFO : Max Epochs: 2048 [2022-03-31 18:56:40 main:478] : INFO : Batch Size: 128 [2022-03-31 18:56:40 main:479] : INFO : Learning Rate: 0.01 [2022-03-31 18:56:40 main:480] : INFO : Patience: 10 [2022-03-31 18:56:40 main:481] : INFO : Hidden Width: 12 [2022-03-31 18:56:40 main:482] : INFO : # Recurrent Layers: 4 [2022-03-31 18:56:40 main:483] : INFO : # Backend Layers: 4 [2022-03-31 18:56:40 main:484] : INFO : # Threads: 1 [2022-03-31 18:56:40 main:486] : INFO : Preparing Dataset [2022-03-31 18:56:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-31 18:56:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-31 18:56:48 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-31 18:56:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-31 18:56:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-31 18:56:49 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-31 18:56:49 main:494] : INFO : Creating Model [2022-03-31 18:56:49 main:507] : INFO : Preparing config file [2022-03-31 18:56:49 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-31 18:56:49 main:512] : INFO : Loading config [2022-03-31 18:56:49 main:514] : INFO : Loading state [2022-03-31 18:56:52 main:559] : INFO : Loading DataLoader into Memory [2022-03-31 18:56:52 main:562] : INFO : Starting Training [2022-03-31 18:57:05 main:574] : INFO : Epoch 1663 | loss: 0.0311667 | val_loss: 0.0311684 | Time: 12747.8 ms [2022-03-31 18:57:18 main:574] : INFO : Epoch 1664 | loss: 0.0311133 | val_loss: 0.0311506 | Time: 12823.1 ms [2022-03-31 18:57:28 main:574] : INFO : Epoch 1665 | loss: 0.0311069 | val_loss: 0.0311558 | Time: 9712.18 ms [2022-03-31 18:57:36 main:574] : INFO : Epoch 1666 | loss: 0.0311079 | val_loss: 0.0311528 | Time: 8211.58 ms [2022-03-31 18:57:49 main:574] : INFO : Epoch 1667 | loss: 0.031109 | val_loss: 0.0311444 | Time: 12587.7 ms [2022-03-31 18:58:00 main:574] : INFO : Epoch 1668 | loss: 0.0311035 | val_loss: 0.0311398 | Time: 10782.6 ms [2022-03-31 18:58:06 main:574] : INFO : Epoch 1669 | loss: 0.0311016 | val_loss: 0.0311475 | Time: 6637.99 ms [2022-03-31 18:58:19 main:574] : INFO : Epoch 1670 | loss: 0.0311048 | val_loss: 0.0311449 | Time: 12870.6 ms [2022-03-31 18:58:29 main:574] : INFO : Epoch 1671 | loss: 0.0311008 | val_loss: 0.0311396 | Time: 9874.19 ms [2022-03-31 18:58:38 main:574] : INFO : Epoch 1672 | loss: 0.0311001 | val_loss: 0.0311438 | Time: 8356.69 ms [2022-03-31 18:58:46 main:574] : INFO : Epoch 1673 | loss: 0.0310983 | val_loss: 0.0311488 | Time: 7952.65 ms [2022-03-31 18:58:53 main:574] : INFO : Epoch 1674 | loss: 0.0310988 | val_loss: 0.0311523 | Time: 7088.11 ms [2022-03-31 18:59:00 main:574] : INFO : Epoch 1675 | loss: 0.0310988 | val_loss: 0.0311498 | Time: 6728.62 ms [2022-03-31 18:59:13 main:574] : INFO : Epoch 1676 | loss: 0.0310992 | val_loss: 0.0311433 | Time: 12861.1 ms [2022-03-31 18:59:25 main:574] : INFO : Epoch 1677 | loss: 0.0311 | val_loss: 0.0311411 | Time: 12541.9 ms [2022-03-31 18:59:35 main:574] : INFO : Epoch 1678 | loss: 0.0310985 | val_loss: 0.031141 | Time: 9323.62 ms [2022-03-31 18:59:43 main:574] : INFO : Epoch 1679 | loss: 0.031096 | val_loss: 0.0311451 | Time: 8122.65 ms [2022-03-31 18:59:56 main:574] : INFO : Epoch 1680 | loss: 0.0310976 | val_loss: 0.0311414 | Time: 12948.5 ms [2022-03-31 19:00:06 main:574] : INFO : Epoch 1681 | loss: 0.0311101 | val_loss: 0.03114 | Time: 10628.7 ms [2022-03-31 19:00:19 main:574] : INFO : Epoch 1682 | loss: 0.0311041 | val_loss: 0.0311379 | Time: 12270.3 ms [2022-03-31 19:00:31 main:574] : INFO : Epoch 1683 | loss: 0.0310987 | val_loss: 0.0311405 | Time: 12285.3 ms [2022-03-31 19:00:41 main:574] : INFO : Epoch 1684 | loss: 0.0310987 | val_loss: 0.0311482 | Time: 9452.87 ms [2022-03-31 19:00:52 main:574] : INFO : Epoch 1685 | loss: 0.031101 | val_loss: 0.0311361 | Time: 11120.3 ms [2022-03-31 19:01:00 main:574] : INFO : Epoch 1686 | loss: 0.0311045 | val_loss: 0.0311378 | Time: 8051.72 ms [2022-03-31 19:01:08 main:574] : INFO : Epoch 1687 | loss: 0.0311034 | val_loss: 0.0311348 | Time: 7584.77 ms [2022-03-31 19:01:19 main:574] : INFO : Epoch 1688 | loss: 0.0311028 | val_loss: 0.0311348 | Time: 11769.8 ms [2022-03-31 19:01:33 main:574] : INFO : Epoch 1689 | loss: 0.0311029 | val_loss: 0.0311378 | Time: 13302.9 ms [2022-03-31 19:01:43 main:574] : INFO : Epoch 1690 | loss: 0.0311066 | val_loss: 0.0311439 | Time: 10186.5 ms [2022-03-31 19:01:51 main:574] : INFO : Epoch 1691 | loss: 0.0310987 | val_loss: 0.0311375 | Time: 8032.48 ms [2022-03-31 19:02:03 main:574] : INFO : Epoch 1692 | loss: 0.0310973 | val_loss: 0.0311422 | Time: 12113.9 ms [2022-03-31 19:02:12 main:574] : INFO : Epoch 1693 | loss: 0.031098 | val_loss: 0.0311426 | Time: 8767.04 ms [2022-03-31 19:02:25 main:574] : INFO : Epoch 1694 | loss: 0.031105 | val_loss: 0.0311403 | Time: 12782.7 ms [2022-03-31 19:02:37 main:574] : INFO : Epoch 1695 | loss: 0.0311093 | val_loss: 0.031138 | Time: 11511.3 ms [2022-03-31 19:02:45 main:574] : INFO : Epoch 1696 | loss: 0.0311046 | val_loss: 0.0311407 | Time: 7753.72 ms [2022-03-31 19:02:57 main:574] : INFO : Epoch 1697 | loss: 0.0311033 | val_loss: 0.0311406 | Time: 12178.3 ms [2022-03-31 19:03:09 main:574] : INFO : Epoch 1698 | loss: 0.0310987 | val_loss: 0.0311404 | Time: 12054.8 ms [2022-03-31 19:03:21 main:574] : INFO : Epoch 1699 | loss: 0.0310967 | val_loss: 0.0311434 | Time: 12089.7 ms [2022-03-31 19:03:30 main:574] : INFO : Epoch 1700 | loss: 0.0310971 | val_loss: 0.03115 | Time: 8569.79 ms [2022-03-31 19:03:38 main:574] : INFO : Epoch 1701 | loss: 0.0311016 | val_loss: 0.0311417 | Time: 8093.79 ms [2022-03-31 19:03:46 main:574] : INFO : Epoch 1702 | loss: 0.0310963 | val_loss: 0.0311391 | Time: 8044.01 ms [2022-03-31 19:03:54 main:574] : INFO : Epoch 1703 | loss: 0.0310947 | val_loss: 0.031142 | Time: 8095.44 ms [2022-03-31 19:04:02 main:574] : INFO : Epoch 1704 | loss: 0.0310942 | val_loss: 0.0311414 | Time: 7602.26 ms [2022-03-31 19:04:09 main:574] : INFO : Epoch 1705 | loss: 0.031092 | val_loss: 0.0311415 | Time: 6944 ms [2022-03-31 19:04:16 main:574] : INFO : Epoch 1706 | loss: 0.0310928 | val_loss: 0.0311422 | Time: 6882.34 ms [2022-03-31 19:04:22 main:574] : INFO : Epoch 1707 | loss: 0.0310936 | val_loss: 0.0311423 | Time: 6496.31 ms [2022-03-31 19:04:28 main:574] : INFO : Epoch 1708 | loss: 0.031097 | val_loss: 0.0311426 | Time: 6209.4 ms [2022-03-31 19:04:35 main:574] : INFO : Epoch 1709 | loss: 0.0310985 | val_loss: 0.0311401 | Time: 6309.95 ms [2022-03-31 19:04:47 main:574] : INFO : Epoch 1710 | loss: 0.0311129 | val_loss: 0.0311422 | Time: 12166.9 ms [2022-03-31 19:05:00 main:574] : INFO : Epoch 1711 | loss: 0.0311137 | val_loss: 0.0311416 | Time: 12632.3 ms [2022-03-31 19:05:08 main:574] : INFO : Epoch 1712 | loss: 0.0311137 | val_loss: 0.0311383 | Time: 7845.39 ms [2022-03-31 19:05:15 main:574] : INFO : Epoch 1713 | loss: 0.0311116 | val_loss: 0.0311388 | Time: 7592.15 ms [2022-03-31 19:05:28 main:574] : INFO : Epoch 1714 | loss: 0.0311117 | val_loss: 0.0311392 | Time: 12517.3 ms [2022-03-31 19:05:37 main:574] : INFO : Epoch 1715 | loss: 0.0311071 | val_loss: 0.0311393 | Time: 9292.67 ms [2022-03-31 19:05:45 main:574] : INFO : Epoch 1716 | loss: 0.0311041 | val_loss: 0.0311369 | Time: 7927.54 ms [2022-03-31 19:05:56 main:574] : INFO : Epoch 1717 | loss: 0.0311019 | val_loss: 0.03114 | Time: 10117.9 ms [2022-03-31 19:06:03 main:574] : INFO : Epoch 1718 | loss: 0.0311023 | val_loss: 0.031138 | Time: 6880.19 ms [2022-03-31 19:06:09 main:574] : INFO : Epoch 1719 | loss: 0.031102 | val_loss: 0.0311363 | Time: 6603.5 ms [2022-03-31 19:06:16 main:574] : INFO : Epoch 1720 | loss: 0.0311008 | val_loss: 0.0311393 | Time: 6365.47 ms [2022-03-31 19:06:28 main:574] : INFO : Epoch 1721 | loss: 0.0310989 | val_loss: 0.031139 | Time: 12228.3 ms [2022-03-31 19:06:40 main:574] : INFO : Epoch 1722 | loss: 0.0311073 | val_loss: 0.0311392 | Time: 11774.5 ms [2022-03-31 19:06:49 main:574] : INFO : Epoch 1723 | loss: 0.0311122 | val_loss: 0.0311402 | Time: 8646.9 ms [2022-03-31 19:07:00 main:574] : INFO : Epoch 1724 | loss: 0.0311076 | val_loss: 0.0311401 | Time: 11680.2 ms [2022-03-31 19:07:13 main:574] : INFO : Epoch 1725 | loss: 0.0311047 | val_loss: 0.0311365 | Time: 12173.9 ms [2022-03-31 19:07:22 main:574] : INFO : Epoch 1726 | loss: 0.0311043 | val_loss: 0.0311379 | Time: 9356.66 ms [2022-03-31 19:07:31 main:574] : INFO : Epoch 1727 | loss: 0.0311029 | val_loss: 0.031143 | Time: 9261.31 ms [2022-03-31 19:07:39 main:574] : INFO : Epoch 1728 | loss: 0.0310999 | val_loss: 0.031137 | Time: 8005.93 ms [2022-03-31 19:07:47 main:574] : INFO : Epoch 1729 | loss: 0.0311003 | val_loss: 0.0311437 | Time: 7643.22 ms [2022-03-31 19:07:59 main:574] : INFO : Epoch 1730 | loss: 0.0310975 | val_loss: 0.0311444 | Time: 12040 ms [2022-03-31 19:08:12 main:574] : INFO : Epoch 1731 | loss: 0.031098 | val_loss: 0.0311413 | Time: 12800.5 ms [2022-03-31 19:08:21 main:574] : INFO : Epoch 1732 | loss: 0.0311031 | val_loss: 0.0311396 | Time: 9214.22 ms [2022-03-31 19:08:35 main:574] : INFO : Epoch 1733 | loss: 0.0311035 | val_loss: 0.0311424 | Time: 13737.5 ms [2022-03-31 19:08:48 main:574] : INFO : Epoch 1734 | loss: 0.0311029 | val_loss: 0.0311403 | Time: 13230.5 ms [2022-03-31 19:08:59 main:574] : INFO : Epoch 1735 | loss: 0.0311013 | val_loss: 0.0311467 | Time: 10451.9 ms [2022-03-31 19:09:11 main:574] : INFO : Epoch 1736 | loss: 0.0311006 | val_loss: 0.0311414 | Time: 12073.7 ms [2022-03-31 19:09:25 main:574] : INFO : Epoch 1737 | loss: 0.0311005 | val_loss: 0.0311464 | Time: 14053.4 ms [2022-03-31 19:09:35 main:574] : INFO : Epoch 1738 | loss: 0.0311004 | val_loss: 0.0311419 | Time: 9883.3 ms [2022-03-31 19:09:44 main:574] : INFO : Epoch 1739 | loss: 0.0310979 | val_loss: 0.0311428 | Time: 9216.46 ms [2022-03-31 19:09:56 main:574] : INFO : Epoch 1740 | loss: 0.0310951 | val_loss: 0.0311425 | Time: 11829.1 ms [2022-03-31 19:10:09 main:574] : INFO : Epoch 1741 | loss: 0.0310953 | val_loss: 0.0311507 | Time: 12462.7 ms [2022-03-31 19:10:21 main:574] : INFO : Epoch 1742 | loss: 0.0310965 | val_loss: 0.0311418 | Time: 11750.5 ms [2022-03-31 19:10:29 main:574] : INFO : Epoch 1743 | loss: 0.0310948 | val_loss: 0.0311501 | Time: 8757.71 ms [2022-03-31 19:10:42 main:574] : INFO : Epoch 1744 | loss: 0.0310968 | val_loss: 0.0311482 | Time: 12785.2 ms [2022-03-31 19:10:52 main:574] : INFO : Epoch 1745 | loss: 0.0310963 | val_loss: 0.0311515 | Time: 9771.02 ms [2022-03-31 19:11:01 main:574] : INFO : Epoch 1746 | loss: 0.0310942 | val_loss: 0.0311508 | Time: 8363.06 ms [2022-03-31 19:11:13 main:574] : INFO : Epoch 1747 | loss: 0.0310933 | val_loss: 0.0311466 | Time: 12394.4 ms [2022-03-31 19:11:23 main:574] : INFO : Epoch 1748 | loss: 0.0310967 | val_loss: 0.0311469 | Time: 9783.74 ms [2022-03-31 19:11:35 main:574] : INFO : Epoch 1749 | loss: 0.031099 | val_loss: 0.0311423 | Time: 11857.6 ms [2022-03-31 19:11:45 main:574] : INFO : Epoch 1750 | loss: 0.0310974 | val_loss: 0.03114 | Time: 10245 ms [2022-03-31 19:11:55 main:574] : INFO : Epoch 1751 | loss: 0.0310956 | val_loss: 0.0311529 | Time: 9873.7 ms [2022-03-31 19:12:03 main:574] : INFO : Epoch 1752 | loss: 0.0310959 | val_loss: 0.0311444 | Time: 7447.79 ms [2022-03-31 19:12:10 main:574] : INFO : Epoch 1753 | loss: 0.0311049 | val_loss: 0.0311405 | Time: 7511.05 ms [2022-03-31 19:12:18 main:574] : INFO : Epoch 1754 | loss: 0.0311023 | val_loss: 0.0311444 | Time: 7509.37 ms [2022-03-31 19:12:25 main:574] : INFO : Epoch 1755 | loss: 0.0310994 | val_loss: 0.0311404 | Time: 7444.31 ms [2022-03-31 19:12:33 main:574] : INFO : Epoch 1756 | loss: 0.0310991 | val_loss: 0.0311422 | Time: 7424.75 ms [2022-03-31 19:12:40 main:574] : INFO : Epoch 1757 | loss: 0.0311002 | val_loss: 0.0311536 | Time: 6938.07 ms [2022-03-31 19:12:52 main:574] : INFO : Epoch 1758 | loss: 0.0311026 | val_loss: 0.0311479 | Time: 12378.4 ms [2022-03-31 19:13:00 main:574] : INFO : Epoch 1759 | loss: 0.0311041 | val_loss: 0.0311552 | Time: 8175.76 ms [2022-03-31 19:13:08 main:574] : INFO : Epoch 1760 | loss: 0.031102 | val_loss: 0.0311491 | Time: 7666.12 ms [2022-03-31 19:13:20 main:574] : INFO : Epoch 1761 | loss: 0.0311019 | val_loss: 0.0311527 | Time: 11895.7 ms [2022-03-31 19:13:30 main:574] : INFO : Epoch 1762 | loss: 0.0311004 | val_loss: 0.0311408 | Time: 9556.07 ms [2022-03-31 19:13:43 main:574] : INFO : Epoch 1763 | loss: 0.0310958 | val_loss: 0.0311446 | Time: 13264 ms [2022-03-31 19:13:56 main:574] : INFO : Epoch 1764 | loss: 0.031096 | val_loss: 0.0311421 | Time: 12903.7 ms [2022-03-31 19:14:10 main:574] : INFO : Epoch 1765 | loss: 0.0310949 | val_loss: 0.0311528 | Time: 13432.5 ms [2022-03-31 19:14:18 main:574] : INFO : Epoch 1766 | loss: 0.0310937 | val_loss: 0.0311473 | Time: 8590.48 ms [2022-03-31 19:14:30 main:574] : INFO : Epoch 1767 | loss: 0.0310943 | val_loss: 0.0311445 | Time: 12127.4 ms [2022-03-31 19:14:44 main:574] : INFO : Epoch 1768 | loss: 0.0310935 | val_loss: 0.0311502 | Time: 13722 ms [2022-03-31 19:14:58 main:574] : INFO : Epoch 1769 | loss: 0.0310934 | val_loss: 0.0311456 | Time: 13447.6 ms [2022-03-31 19:15:11 main:574] : INFO : Epoch 1770 | loss: 0.0310937 | val_loss: 0.0311515 | Time: 13010.4 ms [2022-03-31 19:15:23 main:574] : INFO : Epoch 1771 | loss: 0.0310927 | val_loss: 0.0311496 | Time: 12120.6 ms [2022-03-31 19:15:33 main:574] : INFO : Epoch 1772 | loss: 0.0310937 | val_loss: 0.0311501 | Time: 10340.1 ms [2022-03-31 19:15:46 main:574] : INFO : Epoch 1773 | loss: 0.0310949 | val_loss: 0.0311516 | Time: 12676 ms [2022-03-31 19:15:55 main:574] : INFO : Epoch 1774 | loss: 0.0310977 | val_loss: 0.031151 | Time: 9070.34 ms [2022-03-31 19:16:07 main:574] : INFO : Epoch 1775 | loss: 0.0310972 | val_loss: 0.0311482 | Time: 11432.1 ms [2022-03-31 19:16:20 main:574] : INFO : Epoch 1776 | loss: 0.0311028 | val_loss: 0.0311449 | Time: 13354.4 ms [2022-03-31 19:16:30 main:574] : INFO : Epoch 1777 | loss: 0.031109 | val_loss: 0.0311372 | Time: 10084.4 ms [2022-03-31 19:16:43 main:574] : INFO : Epoch 1778 | loss: 0.0311117 | val_loss: 0.0311599 | Time: 12711.2 ms [2022-03-31 19:16:53 main:574] : INFO : Epoch 1779 | loss: 0.0311112 | val_loss: 0.0311397 | Time: 9587.79 ms [2022-03-31 19:17:01 main:574] : INFO : Epoch 1780 | loss: 0.0311037 | val_loss: 0.0311395 | Time: 8436.9 ms [2022-03-31 19:17:14 main:574] : INFO : Epoch 1781 | loss: 0.0311015 | val_loss: 0.0311448 | Time: 12489.2 ms [2022-03-31 19:17:26 main:574] : INFO : Epoch 1782 | loss: 0.0311018 | val_loss: 0.0311452 | Time: 12109.6 ms [2022-03-31 19:17:34 main:574] : INFO : Epoch 1783 | loss: 0.0310982 | val_loss: 0.0311363 | Time: 8031.03 ms [2022-03-31 19:17:47 main:574] : INFO : Epoch 1784 | loss: 0.031099 | val_loss: 0.031143 | Time: 12476.5 ms [2022-03-31 19:17:57 main:574] : INFO : Epoch 1785 | loss: 0.0310951 | val_loss: 0.0311442 | Time: 10722.9 ms [2022-03-31 19:18:10 main:574] : INFO : Epoch 1786 | loss: 0.0310943 | val_loss: 0.0311476 | Time: 12021.4 ms [2022-03-31 19:18:22 main:574] : INFO : Epoch 1787 | loss: 0.0311008 | val_loss: 0.031156 | Time: 12077.1 ms [2022-03-31 19:18:31 main:574] : INFO : Epoch 1788 | loss: 0.031105 | val_loss: 0.0311484 | Time: 9376.05 ms [2022-03-31 19:18:40 main:574] : INFO : Epoch 1789 | loss: 0.0310987 | val_loss: 0.0311437 | Time: 8386.01 ms [2022-03-31 19:18:47 main:574] : INFO : Epoch 1790 | loss: 0.0310968 | val_loss: 0.031149 | Time: 7517.39 ms [2022-03-31 19:18:55 main:574] : INFO : Epoch 1791 | loss: 0.0310999 | val_loss: 0.0311482 | Time: 7587.63 ms [2022-03-31 19:19:07 main:574] : INFO : Epoch 1792 | loss: 0.0311019 | val_loss: 0.0311471 | Time: 12453.1 ms [2022-03-31 19:19:20 main:574] : INFO : Epoch 1793 | loss: 0.0311016 | val_loss: 0.0311497 | Time: 12051.9 ms [2022-03-31 19:19:28 main:574] : INFO : Epoch 1794 | loss: 0.0310977 | val_loss: 0.0311478 | Time: 8416.18 ms [2022-03-31 19:19:36 main:574] : INFO : Epoch 1795 | loss: 0.0310983 | val_loss: 0.0311439 | Time: 8100.49 ms [2022-03-31 19:19:44 main:574] : INFO : Epoch 1796 | loss: 0.0310977 | val_loss: 0.0311428 | Time: 7904.42 ms [2022-03-31 19:19:52 main:574] : INFO : Epoch 1797 | loss: 0.0310944 | val_loss: 0.0311437 | Time: 7941.52 ms [2022-03-31 19:20:00 main:574] : INFO : Epoch 1798 | loss: 0.0310963 | val_loss: 0.0311532 | Time: 8112.95 ms [2022-03-31 19:20:13 main:574] : INFO : Epoch 1799 | loss: 0.0310949 | val_loss: 0.0311415 | Time: 12689.7 ms [2022-03-31 19:20:23 main:574] : INFO : Epoch 1800 | loss: 0.0310963 | val_loss: 0.0311518 | Time: 9667.47 ms [2022-03-31 19:20:31 main:574] : INFO : Epoch 1801 | loss: 0.0310948 | val_loss: 0.0311529 | Time: 8271.61 ms [2022-03-31 19:20:44 main:574] : INFO : Epoch 1802 | loss: 0.0310951 | val_loss: 0.0311466 | Time: 12759 ms [2022-03-31 19:20:57 main:574] : INFO : Epoch 1803 | loss: 0.0310939 | val_loss: 0.0311468 | Time: 12417.6 ms [2022-03-31 19:21:08 main:574] : INFO : Epoch 1804 | loss: 0.0310924 | val_loss: 0.0311452 | Time: 11796.4 ms [2022-03-31 19:21:18 main:574] : INFO : Epoch 1805 | loss: 0.0310918 | val_loss: 0.0311509 | Time: 9224.12 ms [2022-03-31 19:21:30 main:574] : INFO : Epoch 1806 | loss: 0.0310903 | val_loss: 0.0311412 | Time: 12580.6 ms [2022-03-31 19:21:43 main:574] : INFO : Epoch 1807 | loss: 0.0310917 | val_loss: 0.0311486 | Time: 12191.5 ms [2022-03-31 19:21:52 main:574] : INFO : Epoch 1808 | loss: 0.0310911 | val_loss: 0.0311473 | Time: 9383.14 ms [2022-03-31 19:22:00 main:574] : INFO : Epoch 1809 | loss: 0.0310967 | val_loss: 0.0311472 | Time: 8110.75 ms [2022-03-31 19:22:08 main:574] : INFO : Epoch 1810 | loss: 0.0310964 | val_loss: 0.0311557 | Time: 7963.23 ms [2022-03-31 19:22:16 main:574] : INFO : Epoch 1811 | loss: 0.0310975 | val_loss: 0.0311536 | Time: 7940.12 ms [2022-03-31 19:22:24 main:574] : INFO : Epoch 1812 | loss: 0.0310955 | val_loss: 0.031148 | Time: 7531.54 ms [2022-03-31 19:22:31 main:574] : INFO : Epoch 1813 | loss: 0.0310916 | val_loss: 0.0311416 | Time: 7525.44 ms [2022-03-31 19:22:39 main:574] : INFO : Epoch 1814 | loss: 0.0311011 | val_loss: 0.0311481 | Time: 7506.39 ms [2022-03-31 19:22:46 main:574] : INFO : Epoch 1815 | loss: 0.0311025 | val_loss: 0.0311478 | Time: 7394.77 ms [2022-03-31 19:23:00 main:574] : INFO : Epoch 1816 | loss: 0.0310968 | val_loss: 0.0311484 | Time: 14004.2 ms [2022-03-31 19:23:13 main:574] : INFO : Epoch 1817 | loss: 0.0310952 | val_loss: 0.031143 | Time: 12035.9 ms [2022-03-31 19:23:22 main:574] : INFO : Epoch 1818 | loss: 0.0310922 | val_loss: 0.0311442 | Time: 8963.69 ms [2022-03-31 19:23:30 main:574] : INFO : Epoch 1819 | loss: 0.0310927 | val_loss: 0.0311451 | Time: 8075.31 ms [2022-03-31 19:23:42 main:574] : INFO : Epoch 1820 | loss: 0.0310929 | val_loss: 0.0311444 | Time: 12552.7 ms [2022-03-31 19:23:55 main:574] : INFO : Epoch 1821 | loss: 0.0310928 | val_loss: 0.031154 | Time: 12471.4 ms [2022-03-31 19:24:05 main:574] : INFO : Epoch 1822 | loss: 0.0310903 | val_loss: 0.031145 | Time: 9773.9 ms [2022-03-31 19:24:17 main:574] : INFO : Epoch 1823 | loss: 0.0310906 | val_loss: 0.0311564 | Time: 12536.3 ms [2022-03-31 19:24:29 main:574] : INFO : Epoch 1824 | loss: 0.0310949 | val_loss: 0.0311421 | Time: 11499.9 ms [2022-03-31 19:24:37 main:574] : INFO : Epoch 1825 | loss: 0.0310932 | val_loss: 0.0311505 | Time: 8221.81 ms [2022-03-31 19:24:50 main:574] : INFO : Epoch 1826 | loss: 0.0310887 | val_loss: 0.0311511 | Time: 12871.2 ms [2022-03-31 19:25:02 main:574] : INFO : Epoch 1827 | loss: 0.0310898 | val_loss: 0.0311524 | Time: 12312.8 ms [2022-03-31 19:25:15 main:574] : INFO : Epoch 1828 | loss: 0.031088 | val_loss: 0.0311516 | Time: 12136.2 ms [2022-03-31 19:25:25 main:574] : INFO : Epoch 1829 | loss: 0.0310893 | val_loss: 0.031149 | Time: 9841.55 ms [2022-03-31 19:25:33 main:574] : INFO : Epoch 1830 | loss: 0.0310895 | val_loss: 0.0311457 | Time: 8765.02 ms [2022-03-31 19:25:40 main:574] : INFO : Epoch 1831 | loss: 0.031088 | val_loss: 0.0311475 | Time: 6929.49 ms [2022-03-31 19:25:47 main:574] : INFO : Epoch 1832 | loss: 0.03109 | val_loss: 0.0311388 | Time: 6483.88 ms [2022-03-31 19:25:53 main:574] : INFO : Epoch 1833 | loss: 0.0310969 | val_loss: 0.0311436 | Time: 6334.9 ms [2022-03-31 19:26:06 main:574] : INFO : Epoch 1834 | loss: 0.0310959 | val_loss: 0.0311461 | Time: 12606.6 ms [2022-03-31 19:26:18 main:574] : INFO : Epoch 1835 | loss: 0.0310959 | val_loss: 0.0311425 | Time: 11971 ms [2022-03-31 19:26:31 main:574] : INFO : Epoch 1836 | loss: 0.0310991 | val_loss: 0.0311476 | Time: 12806.9 ms [2022-03-31 19:26:41 main:574] : INFO : Epoch 1837 | loss: 0.031117 | val_loss: 0.0311372 | Time: 10010.9 ms [2022-03-31 19:26:50 main:574] : INFO : Epoch 1838 | loss: 0.0311185 | val_loss: 0.031146 | Time: 8683.26 ms [2022-03-31 19:26:58 main:574] : INFO : Epoch 1839 | loss: 0.0311087 | val_loss: 0.0311386 | Time: 7826.84 ms [2022-03-31 19:27:05 main:574] : INFO : Epoch 1840 | loss: 0.031108 | val_loss: 0.0311363 | Time: 7138.05 ms [2022-03-31 19:27:12 main:574] : INFO : Epoch 1841 | loss: 0.0311125 | val_loss: 0.0311335 | Time: 7238.59 ms [2022-03-31 19:27:19 main:574] : INFO : Epoch 1842 | loss: 0.0311132 | val_loss: 0.0311429 | Time: 6601.17 ms [2022-03-31 19:27:31 main:574] : INFO : Epoch 1843 | loss: 0.0311087 | val_loss: 0.0311446 | Time: 12533 ms [2022-03-31 19:27:45 main:574] : INFO : Epoch 1844 | loss: 0.0311076 | val_loss: 0.031141 | Time: 13236.1 ms [2022-03-31 19:27:55 main:574] : INFO : Epoch 1845 | loss: 0.0311054 | val_loss: 0.031145 | Time: 10781.8 ms [2022-03-31 19:28:03 main:574] : INFO : Epoch 1846 | loss: 0.0310989 | val_loss: 0.0311531 | Time: 7380.97 ms [2022-03-31 19:28:15 main:574] : INFO : Epoch 1847 | loss: 0.0311007 | val_loss: 0.0311523 | Time: 11728 ms [2022-03-31 19:28:27 main:574] : INFO : Epoch 1848 | loss: 0.0310979 | val_loss: 0.031152 | Time: 12100.1 ms [2022-03-31 19:28:36 main:574] : INFO : Epoch 1849 | loss: 0.0311004 | val_loss: 0.0311446 | Time: 9096.34 ms [2022-03-31 19:28:48 main:574] : INFO : Epoch 1850 | loss: 0.0310985 | val_loss: 0.0311492 | Time: 12391.4 ms [2022-03-31 19:28:58 main:574] : INFO : Epoch 1851 | loss: 0.0310984 | val_loss: 0.0311465 | Time: 9839.04 ms [2022-03-31 19:29:08 main:574] : INFO : Epoch 1852 | loss: 0.0310946 | val_loss: 0.0311517 | Time: 9264.47 ms [2022-03-31 19:29:19 main:574] : INFO : Epoch 1853 | loss: 0.0310944 | val_loss: 0.0311558 | Time: 10871.1 ms [2022-03-31 19:29:26 main:574] : INFO : Epoch 1854 | loss: 0.0311059 | val_loss: 0.0311561 | Time: 7743.28 ms [2022-03-31 19:29:34 main:574] : INFO : Epoch 1855 | loss: 0.0311001 | val_loss: 0.0311472 | Time: 7389.36 ms [2022-03-31 19:29:41 main:574] : INFO : Epoch 1856 | loss: 0.0310958 | val_loss: 0.0311465 | Time: 7031.61 ms [2022-03-31 19:29:48 main:574] : INFO : Epoch 1857 | loss: 0.0310923 | val_loss: 0.0311464 | Time: 6664.82 ms [2022-03-31 19:29:54 main:574] : INFO : Epoch 1858 | loss: 0.0310915 | val_loss: 0.031147 | Time: 6369.62 ms [2022-03-31 19:30:06 main:574] : INFO : Epoch 1859 | loss: 0.0310901 | val_loss: 0.0311471 | Time: 11620.9 ms [2022-03-31 19:30:20 main:574] : INFO : Epoch 1860 | loss: 0.0310882 | val_loss: 0.0311433 | Time: 13814 ms [2022-03-31 19:30:29 main:574] : INFO : Epoch 1861 | loss: 0.0310892 | val_loss: 0.0311439 | Time: 9288.52 ms [2022-03-31 19:30:37 main:574] : INFO : Epoch 1862 | loss: 0.0311052 | val_loss: 0.0311479 | Time: 8105.24 ms [2022-03-31 19:30:46 main:574] : INFO : Epoch 1863 | loss: 0.0311059 | val_loss: 0.0311453 | Time: 8117.94 ms [2022-03-31 19:30:53 main:574] : INFO : Epoch 1864 | loss: 0.0311054 | val_loss: 0.0311432 | Time: 7633.4 ms [2022-03-31 19:31:01 main:574] : INFO : Epoch 1865 | loss: 0.0311109 | val_loss: 0.0311464 | Time: 7398.71 ms [2022-03-31 19:31:14 main:574] : INFO : Epoch 1866 | loss: 0.0311048 | val_loss: 0.0311436 | Time: 12981.1 ms [2022-03-31 19:31:24 main:574] : INFO : Epoch 1867 | loss: 0.0311034 | val_loss: 0.0311485 | Time: 10492.3 ms [2022-03-31 19:31:37 main:574] : INFO : Epoch 1868 | loss: 0.0311078 | val_loss: 0.0311471 | Time: 12447.3 ms [2022-03-31 19:31:46 main:574] : INFO : Epoch 1869 | loss: 0.0311077 | val_loss: 0.0311465 | Time: 9541.71 ms [2022-03-31 19:31:55 main:574] : INFO : Epoch 1870 | loss: 0.0311027 | val_loss: 0.0311517 | Time: 8453.64 ms [2022-03-31 19:32:03 main:574] : INFO : Epoch 1871 | loss: 0.031103 | val_loss: 0.0311469 | Time: 8032.02 ms [2022-03-31 19:32:14 main:574] : INFO : Epoch 1872 | loss: 0.0311047 | val_loss: 0.0311474 | Time: 10496.5 ms [2022-03-31 19:32:27 main:574] : INFO : Epoch 1873 | loss: 0.0311031 | val_loss: 0.0311418 | Time: 13730.4 ms [2022-03-31 19:32:40 main:574] : INFO : Epoch 1874 | loss: 0.0311021 | val_loss: 0.0311483 | Time: 12530.9 ms [2022-03-31 19:32:50 main:574] : INFO : Epoch 1875 | loss: 0.0310987 | val_loss: 0.0311458 | Time: 9481.62 ms [2022-03-31 19:32:57 main:574] : INFO : Epoch 1876 | loss: 0.0310992 | val_loss: 0.0311575 | Time: 7583.14 ms [2022-03-31 19:33:04 main:574] : INFO : Epoch 1877 | loss: 0.0311001 | val_loss: 0.0311361 | Time: 7059.31 ms [2022-03-31 19:33:17 main:574] : INFO : Epoch 1878 | loss: 0.0311009 | val_loss: 0.0311387 | Time: 13027.6 ms [2022-03-31 19:33:28 main:574] : INFO : Epoch 1879 | loss: 0.0310992 | val_loss: 0.0311438 | Time: 10221.2 ms [2022-03-31 19:33:41 main:574] : INFO : Epoch 1880 | loss: 0.0310986 | val_loss: 0.0311384 | Time: 12796.2 ms [2022-03-31 19:33:53 main:574] : INFO : Epoch 1881 | loss: 0.0311008 | val_loss: 0.0311435 | Time: 12671.5 ms [2022-03-31 19:34:02 main:574] : INFO : Epoch 1882 | loss: 0.0310995 | val_loss: 0.0311442 | Time: 8620.41 ms [2022-03-31 19:34:10 main:574] : INFO : Epoch 1883 | loss: 0.0310988 | val_loss: 0.0311407 | Time: 7635.93 ms [2022-03-31 19:34:17 main:574] : INFO : Epoch 1884 | loss: 0.0311001 | val_loss: 0.0311495 | Time: 7063.13 ms [2022-03-31 19:34:24 main:574] : INFO : Epoch 1885 | loss: 0.0310973 | val_loss: 0.0311433 | Time: 7020.35 ms [2022-03-31 19:34:30 main:574] : INFO : Epoch 1886 | loss: 0.0311002 | val_loss: 0.0311444 | Time: 6379.06 ms [2022-03-31 19:34:43 main:574] : INFO : Epoch 1887 | loss: 0.0310988 | val_loss: 0.0311446 | Time: 12624.6 ms [2022-03-31 19:34:55 main:574] : INFO : Epoch 1888 | loss: 0.0310976 | val_loss: 0.0311467 | Time: 12225.9 ms [2022-03-31 19:35:08 main:574] : INFO : Epoch 1889 | loss: 0.0310969 | val_loss: 0.0311493 | Time: 12497.2 ms [2022-03-31 19:35:21 main:574] : INFO : Epoch 1890 | loss: 0.0310989 | val_loss: 0.0311401 | Time: 12896.9 ms [2022-03-31 19:35:30 main:574] : INFO : Epoch 1891 | loss: 0.0310982 | val_loss: 0.0311534 | Time: 9455.86 ms [2022-03-31 19:35:39 main:574] : INFO : Epoch 1892 | loss: 0.0311156 | val_loss: 0.0311441 | Time: 8540.02 ms [2022-03-31 19:35:47 main:574] : INFO : Epoch 1893 | loss: 0.0311135 | val_loss: 0.0311506 | Time: 7492.78 ms [2022-03-31 19:35:57 main:574] : INFO : Epoch 1894 | loss: 0.0311099 | val_loss: 0.0311451 | Time: 10346.8 ms [2022-03-31 19:36:09 main:574] : INFO : Epoch 1895 | loss: 0.0311074 | val_loss: 0.0311414 | Time: 11982.4 ms [2022-03-31 19:36:22 main:574] : INFO : Epoch 1896 | loss: 0.0311032 | val_loss: 0.0311413 | Time: 12484.1 ms [2022-03-31 19:36:32 main:574] : INFO : Epoch 1897 | loss: 0.0311055 | val_loss: 0.0311467 | Time: 9749.16 ms [2022-03-31 19:36:41 main:574] : INFO : Epoch 1898 | loss: 0.0311062 | val_loss: 0.0311464 | Time: 8879.03 ms [2022-03-31 19:36:48 main:574] : INFO : Epoch 1899 | loss: 0.0311121 | val_loss: 0.0311436 | Time: 7461.73 ms [2022-03-31 19:37:02 main:574] : INFO : Epoch 1900 | loss: 0.0311061 | val_loss: 0.0311469 | Time: 13664.7 ms [2022-03-31 19:37:09 main:574] : INFO : Epoch 1901 | loss: 0.0311065 | val_loss: 0.0311434 | Time: 7349.78 ms [2022-03-31 19:37:22 main:574] : INFO : Epoch 1902 | loss: 0.0311061 | val_loss: 0.0311453 | Time: 12536.7 ms [2022-03-31 19:37:32 main:574] : INFO : Epoch 1903 | loss: 0.0311055 | val_loss: 0.0311462 | Time: 10107.8 ms [2022-03-31 19:37:44 main:574] : INFO : Epoch 1904 | loss: 0.0311036 | val_loss: 0.0311453 | Time: 11919.8 ms [2022-03-31 19:37:56 main:574] : INFO : Epoch 1905 | loss: 0.0311033 | val_loss: 0.0311456 | Time: 11534.5 ms [2022-03-31 19:38:07 main:574] : INFO : Epoch 1906 | loss: 0.0311048 | val_loss: 0.0311427 | Time: 11061 ms [2022-03-31 19:38:19 main:574] : INFO : Epoch 1907 | loss: 0.0311017 | val_loss: 0.0311486 | Time: 12104.1 ms [2022-03-31 19:38:29 main:574] : INFO : Epoch 1908 | loss: 0.031105 | val_loss: 0.0311411 | Time: 9760.71 ms [2022-03-31 19:38:37 main:574] : INFO : Epoch 1909 | loss: 0.0311034 | val_loss: 0.0311372 | Time: 8117.22 ms [2022-03-31 19:38:45 main:574] : INFO : Epoch 1910 | loss: 0.0311008 | val_loss: 0.0311473 | Time: 7508.85 ms [2022-03-31 19:38:52 main:574] : INFO : Epoch 1911 | loss: 0.031107 | val_loss: 0.0311496 | Time: 7427.98 ms [2022-03-31 19:39:04 main:574] : INFO : Epoch 1912 | loss: 0.0311117 | val_loss: 0.0311374 | Time: 11411.2 ms [2022-03-31 19:39:13 main:574] : INFO : Epoch 1913 | loss: 0.0311074 | val_loss: 0.0311413 | Time: 9273.61 ms [2022-03-31 19:39:21 main:574] : INFO : Epoch 1914 | loss: 0.0311049 | val_loss: 0.0311437 | Time: 8124.16 ms [2022-03-31 19:39:34 main:574] : INFO : Epoch 1915 | loss: 0.0311028 | val_loss: 0.0311409 | Time: 12807.6 ms [2022-03-31 19:39:44 main:574] : INFO : Epoch 1916 | loss: 0.0311013 | val_loss: 0.0311412 | Time: 10214.5 ms [2022-03-31 19:39:56 main:574] : INFO : Epoch 1917 | loss: 0.0311011 | val_loss: 0.0311379 | Time: 12115.7 ms [2022-03-31 19:40:05 main:574] : INFO : Epoch 1918 | loss: 0.0311034 | val_loss: 0.0311426 | Time: 8815.01 ms [2022-03-31 19:40:18 main:574] : INFO : Epoch 1919 | loss: 0.0311005 | val_loss: 0.031142 | Time: 12691 ms [2022-03-31 19:40:27 main:574] : INFO : Epoch 1920 | loss: 0.0311019 | val_loss: 0.0311459 | Time: 9149.74 ms [2022-03-31 19:40:39 main:574] : INFO : Epoch 1921 | loss: 0.0311051 | val_loss: 0.031147 | Time: 11400.5 ms [2022-03-31 19:40:51 main:574] : INFO : Epoch 1922 | loss: 0.0311104 | val_loss: 0.0311434 | Time: 12661.6 ms [2022-03-31 19:41:05 main:574] : INFO : Epoch 1923 | loss: 0.0311112 | val_loss: 0.0311375 | Time: 14021.2 ms [2022-03-31 19:41:19 main:574] : INFO : Epoch 1924 | loss: 0.0311066 | val_loss: 0.0311429 | Time: 13366.9 ms [2022-03-31 19:41:28 main:574] : INFO : Epoch 1925 | loss: 0.0311057 | val_loss: 0.031141 | Time: 9319.15 ms [2022-03-31 19:41:37 main:574] : INFO : Epoch 1926 | loss: 0.0311057 | val_loss: 0.0311402 | Time: 8664.38 ms [2022-03-31 19:41:45 main:574] : INFO : Epoch 1927 | loss: 0.0311029 | val_loss: 0.0311396 | Time: 7699.6 ms [2022-03-31 19:41:57 main:574] : INFO : Epoch 1928 | loss: 0.0311009 | val_loss: 0.0311411 | Time: 12303.3 ms [2022-03-31 19:42:05 main:574] : INFO : Epoch 1929 | loss: 0.0311014 | val_loss: 0.0311466 | Time: 8112.72 ms [2022-03-31 19:42:13 main:574] : INFO : Epoch 1930 | loss: 0.0311032 | val_loss: 0.0311473 | Time: 8032.05 ms [2022-03-31 19:42:21 main:574] : INFO : Epoch 1931 | loss: 0.031103 | val_loss: 0.0311434 | Time: 7379.26 ms [2022-03-31 19:42:28 main:574] : INFO : Epoch 1932 | loss: 0.0311343 | val_loss: 0.0313399 | Time: 7191.58 ms [2022-03-31 19:42:35 main:574] : INFO : Epoch 1933 | loss: 0.0323022 | val_loss: 0.0358075 | Time: 6378.92 ms [2022-03-31 19:42:41 main:574] : INFO : Epoch 1934 | loss: 0.047489 | val_loss: 0.0357872 | Time: 6233.87 ms [2022-03-31 19:42:53 main:574] : INFO : Epoch 1935 | loss: 0.0345264 | val_loss: 0.0318153 | Time: 11728.3 ms [2022-03-31 19:43:02 main:574] : INFO : Epoch 1936 | loss: 0.03164 | val_loss: 0.0313815 | Time: 9478.7 ms [2022-03-31 19:43:15 main:574] : INFO : Epoch 1937 | loss: 0.031276 | val_loss: 0.0312314 | Time: 12316.9 ms [2022-03-31 19:43:28 main:574] : INFO : Epoch 1938 | loss: 0.0312184 | val_loss: 0.0312125 | Time: 12956.5 ms [2022-03-31 19:43:38 main:574] : INFO : Epoch 1939 | loss: 0.031202 | val_loss: 0.0312 | Time: 10569.6 ms [2022-03-31 19:43:51 main:574] : INFO : Epoch 1940 | loss: 0.0311918 | val_loss: 0.0311919 | Time: 12853.1 ms [2022-03-31 19:44:01 main:574] : INFO : Epoch 1941 | loss: 0.0311811 | val_loss: 0.0311828 | Time: 9773.94 ms [2022-03-31 19:44:08 main:574] : INFO : Epoch 1942 | loss: 0.03117 | val_loss: 0.031174 | Time: 7183.79 ms [2022-03-31 19:44:15 main:574] : INFO : Epoch 1943 | loss: 0.031153 | val_loss: 0.0311553 | Time: 6919.52 ms [2022-03-31 19:44:22 main:574] : INFO : Epoch 1944 | loss: 0.0311322 | val_loss: 0.0311538 | Time: 6450.08 ms [2022-03-31 19:44:29 main:574] : INFO : Epoch 1945 | loss: 0.0311226 | val_loss: 0.0311419 | Time: 6726.15 ms [2022-03-31 19:44:35 main:574] : INFO : Epoch 1946 | loss: 0.0311156 | val_loss: 0.0311424 | Time: 6537.29 ms [2022-03-31 19:44:42 main:574] : INFO : Epoch 1947 | loss: 0.03111 | val_loss: 0.0311435 | Time: 6753.72 ms [2022-03-31 19:44:49 main:574] : INFO : Epoch 1948 | loss: 0.0311096 | val_loss: 0.0311417 | Time: 6537.93 ms [2022-03-31 19:44:55 main:574] : INFO : Epoch 1949 | loss: 0.0311036 | val_loss: 0.0311388 | Time: 6459.02 ms [2022-03-31 19:45:07 main:574] : INFO : Epoch 1950 | loss: 0.0311031 | val_loss: 0.031152 | Time: 11462.6 ms [2022-03-31 19:45:16 main:574] : INFO : Epoch 1951 | loss: 0.0311099 | val_loss: 0.0311431 | Time: 9650.15 ms [2022-03-31 19:45:24 main:574] : INFO : Epoch 1952 | loss: 0.0311055 | val_loss: 0.0311415 | Time: 7730.02 ms [2022-03-31 19:45:36 main:574] : INFO : Epoch 1953 | loss: 0.0311024 | val_loss: 0.03114 | Time: 11771.2 ms [2022-03-31 19:45:49 main:574] : INFO : Epoch 1954 | loss: 0.0311015 | val_loss: 0.031141 | Time: 13162.1 ms [2022-03-31 19:46:03 main:574] : INFO : Epoch 1955 | loss: 0.0311007 | val_loss: 0.0311435 | Time: 13436 ms [2022-03-31 19:46:13 main:574] : INFO : Epoch 1956 | loss: 0.0310994 | val_loss: 0.0311381 | Time: 10449.9 ms [2022-03-31 19:46:24 main:574] : INFO : Epoch 1957 | loss: 0.0311017 | val_loss: 0.0311404 | Time: 10083.8 ms [2022-03-31 19:46:36 main:574] : INFO : Epoch 1958 | loss: 0.0311076 | val_loss: 0.0311411 | Time: 12586.3 ms [2022-03-31 19:46:48 main:574] : INFO : Epoch 1959 | loss: 0.0311081 | val_loss: 0.0311401 | Time: 12196.8 ms [2022-03-31 19:46:59 main:574] : INFO : Epoch 1960 | loss: 0.0311021 | val_loss: 0.0311366 | Time: 10585.7 ms [2022-03-31 19:47:11 main:574] : INFO : Epoch 1961 | loss: 0.0310995 | val_loss: 0.0311367 | Time: 11600.5 ms [2022-03-31 19:47:20 main:574] : INFO : Epoch 1962 | loss: 0.0311004 | val_loss: 0.0311433 | Time: 9287.26 ms [2022-03-31 19:47:27 main:574] : INFO : Epoch 1963 | loss: 0.0311018 | val_loss: 0.0311336 | Time: 6477.09 ms [2022-03-31 19:47:33 main:574] : INFO : Epoch 1964 | loss: 0.0311047 | val_loss: 0.0311371 | Time: 6374.48 ms [2022-03-31 19:47:47 main:574] : INFO : Epoch 1965 | loss: 0.0311169 | val_loss: 0.0311381 | Time: 13380 ms [2022-03-31 19:48:01 main:574] : INFO : Epoch 1966 | loss: 0.0311172 | val_loss: 0.0311359 | Time: 13947.9 ms [2022-03-31 19:48:15 main:574] : INFO : Epoch 1967 | loss: 0.0311119 | val_loss: 0.0311313 | Time: 13872.2 ms [2022-03-31 19:48:27 main:574] : INFO : Epoch 1968 | loss: 0.031109 | val_loss: 0.0311365 | Time: 12607.6 ms [2022-03-31 19:48:41 main:574] : INFO : Epoch 1969 | loss: 0.0311152 | val_loss: 0.0311373 | Time: 13805.5 ms [2022-03-31 19:48:51 main:574] : INFO : Epoch 1970 | loss: 0.0311139 | val_loss: 0.0311358 | Time: 9506.01 ms [2022-03-31 19:48:59 main:574] : INFO : Epoch 1971 | loss: 0.0311102 | val_loss: 0.0311347 | Time: 8037.54 ms [2022-03-31 19:49:12 main:574] : INFO : Epoch 1972 | loss: 0.0311114 | val_loss: 0.0311373 | Time: 12906.9 ms [2022-03-31 19:49:21 main:574] : INFO : Epoch 1973 | loss: 0.0311124 | val_loss: 0.0311321 | Time: 8604.83 ms [2022-03-31 19:49:28 main:574] : INFO : Epoch 1974 | loss: 0.0311085 | val_loss: 0.0311345 | Time: 7546.91 ms [2022-03-31 19:49:41 main:574] : INFO : Epoch 1975 | loss: 0.0311066 | val_loss: 0.0311342 | Time: 12692.7 ms [2022-03-31 19:49:54 main:574] : INFO : Epoch 1976 | loss: 0.0311056 | val_loss: 0.0311333 | Time: 12565.5 ms [2022-03-31 19:50:03 main:574] : INFO : Epoch 1977 | loss: 0.0311057 | val_loss: 0.0311346 | Time: 8932.94 ms [2022-03-31 19:50:11 main:574] : INFO : Epoch 1978 | loss: 0.031111 | val_loss: 0.0311383 | Time: 8045.16 ms [2022-03-31 19:50:19 main:574] : INFO : Epoch 1979 | loss: 0.0311132 | val_loss: 0.0311336 | Time: 7658.06 ms [2022-03-31 19:50:30 main:574] : INFO : Epoch 1980 | loss: 0.0311053 | val_loss: 0.0311358 | Time: 11612.1 ms [2022-03-31 19:50:39 main:574] : INFO : Epoch 1981 | loss: 0.0311032 | val_loss: 0.0311349 | Time: 8279.63 ms [2022-03-31 19:50:47 main:574] : INFO : Epoch 1982 | loss: 0.0311021 | val_loss: 0.0311331 | Time: 7975.78 ms [2022-03-31 19:50:55 main:574] : INFO : Epoch 1983 | loss: 0.0311018 | val_loss: 0.0311336 | Time: 8046.81 ms [2022-03-31 19:51:03 main:574] : INFO : Epoch 1984 | loss: 0.0311026 | val_loss: 0.0311352 | Time: 8104.22 ms [2022-03-31 19:51:15 main:574] : INFO : Epoch 1985 | loss: 0.0311006 | val_loss: 0.0311363 | Time: 11814.4 ms [2022-03-31 19:51:28 main:574] : INFO : Epoch 1986 | loss: 0.0311001 | val_loss: 0.0311387 | Time: 13319.1 ms [2022-03-31 19:51:40 main:574] : INFO : Epoch 1987 | loss: 0.0311008 | val_loss: 0.0311368 | Time: 11978.7 ms [2022-03-31 19:51:49 main:574] : INFO : Epoch 1988 | loss: 0.0311016 | val_loss: 0.0311441 | Time: 8976.39 ms [2022-03-31 19:51:57 main:574] : INFO : Epoch 1989 | loss: 0.0311012 | val_loss: 0.0311467 | Time: 8057.15 ms [2022-03-31 19:52:09 main:574] : INFO : Epoch 1990 | loss: 0.0310993 | val_loss: 0.0311357 | Time: 11614.7 ms [2022-03-31 19:52:21 main:574] : INFO : Epoch 1991 | loss: 0.0310977 | val_loss: 0.0311362 | Time: 12243.9 ms [2022-03-31 19:52:36 main:574] : INFO : Epoch 1992 | loss: 0.0310981 | val_loss: 0.0311337 | Time: 14150.5 ms [2022-03-31 19:52:49 main:574] : INFO : Epoch 1993 | loss: 0.0310984 | val_loss: 0.0311346 | Time: 12837.9 ms [2022-03-31 19:52:59 main:574] : INFO : Epoch 1994 | loss: 0.0310976 | val_loss: 0.0311406 | Time: 10125.7 ms [2022-03-31 19:53:12 main:574] : INFO : Epoch 1995 | loss: 0.0311035 | val_loss: 0.0311336 | Time: 13506.1 ms [2022-03-31 19:53:25 main:574] : INFO : Epoch 1996 | loss: 0.0311292 | val_loss: 0.0311396 | Time: 12003.4 ms [2022-03-31 19:53:37 main:574] : INFO : Epoch 1997 | loss: 0.0311277 | val_loss: 0.0311358 | Time: 12280.4 ms [2022-03-31 19:53:51 main:574] : INFO : Epoch 1998 | loss: 0.0311236 | val_loss: 0.0311411 | Time: 13809.2 ms [2022-03-31 19:54:01 main:574] : INFO : Epoch 1999 | loss: 0.0311187 | val_loss: 0.031136 | Time: 10553.7 ms [2022-03-31 19:54:12 main:574] : INFO : Epoch 2000 | loss: 0.0311218 | val_loss: 0.0311383 | Time: 10360.6 ms [2022-03-31 19:54:22 main:574] : INFO : Epoch 2001 | loss: 0.0311146 | val_loss: 0.0311379 | Time: 10564.2 ms [2022-03-31 19:54:32 main:574] : INFO : Epoch 2002 | loss: 0.0311139 | val_loss: 0.0311351 | Time: 9892.66 ms [2022-03-31 19:54:42 main:574] : INFO : Epoch 2003 | loss: 0.0311135 | val_loss: 0.0311406 | Time: 9423.71 ms [2022-03-31 19:54:55 main:574] : INFO : Epoch 2004 | loss: 0.0311107 | val_loss: 0.0311402 | Time: 12621.3 ms [2022-03-31 19:55:04 main:574] : INFO : Epoch 2005 | loss: 0.0311096 | val_loss: 0.0311403 | Time: 9129.79 ms [2022-03-31 19:55:13 main:574] : INFO : Epoch 2006 | loss: 0.031108 | val_loss: 0.0311373 | Time: 9358.46 ms [2022-03-31 19:55:27 main:574] : INFO : Epoch 2007 | loss: 0.0311075 | val_loss: 0.0311441 | Time: 13258.7 ms [2022-03-31 19:55:37 main:574] : INFO : Epoch 2008 | loss: 0.0311068 | val_loss: 0.0311405 | Time: 10051.9 ms [2022-03-31 19:55:51 main:574] : INFO : Epoch 2009 | loss: 0.0311028 | val_loss: 0.0311382 | Time: 13596 ms [2022-03-31 19:56:04 main:574] : INFO : Epoch 2010 | loss: 0.0311046 | val_loss: 0.0311425 | Time: 13377.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-03-31 20:00:38 main:435] : INFO : Set logging level to 1 [2022-03-31 20:00:38 main:441] : INFO : Running in BOINC Client mode [2022-03-31 20:00:38 main:444] : INFO : Resolving all filenames [2022-03-31 20:00:38 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-31 20:00:38 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-31 20:00:38 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-31 20:00:38 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-31 20:00:38 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-31 20:00:38 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-31 20:00:38 main:474] : INFO : Configuration: [2022-03-31 20:00:38 main:475] : INFO : Model type: GRU [2022-03-31 20:00:38 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-31 20:00:38 main:477] : INFO : Max Epochs: 2048 [2022-03-31 20:00:38 main:478] : INFO : Batch Size: 128 [2022-03-31 20:00:38 main:479] : INFO : Learning Rate: 0.01 [2022-03-31 20:00:39 main:480] : INFO : Patience: 10 [2022-03-31 20:00:39 main:481] : INFO : Hidden Width: 12 [2022-03-31 20:00:39 main:482] : INFO : # Recurrent Layers: 4 [2022-03-31 20:00:39 main:483] : INFO : # Backend Layers: 4 [2022-03-31 20:00:39 main:484] : INFO : # Threads: 1 [2022-03-31 20:00:39 main:486] : INFO : Preparing Dataset [2022-03-31 20:00:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-31 20:00:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-31 20:00:46 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-31 20:00:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-31 20:00:46 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-31 20:00:46 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-31 20:00:46 main:494] : INFO : Creating Model [2022-03-31 20:00:46 main:507] : INFO : Preparing config file [2022-03-31 20:00:46 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-31 20:00:46 main:512] : INFO : Loading config [2022-03-31 20:00:46 main:514] : INFO : Loading state [2022-03-31 20:00:49 main:559] : INFO : Loading DataLoader into Memory [2022-03-31 20:00:49 main:562] : INFO : Starting Training [2022-03-31 20:01:02 main:574] : INFO : Epoch 2006 | loss: 0.0317181 | val_loss: 0.031283 | Time: 13078.3 ms [2022-03-31 20:01:12 main:574] : INFO : Epoch 2007 | loss: 0.0312077 | val_loss: 0.0311853 | Time: 9734.33 ms [2022-03-31 20:01:22 main:574] : INFO : Epoch 2008 | loss: 0.0311406 | val_loss: 0.0311581 | Time: 9621.79 ms [2022-03-31 20:01:32 main:574] : INFO : Epoch 2009 | loss: 0.0311173 | val_loss: 0.0311422 | Time: 10642.9 ms [2022-03-31 20:01:40 main:574] : INFO : Epoch 2010 | loss: 0.0311105 | val_loss: 0.0311406 | Time: 7590.03 ms [2022-03-31 20:01:47 main:574] : INFO : Epoch 2011 | loss: 0.0311086 | val_loss: 0.0311512 | Time: 7235.99 ms [2022-03-31 20:01:55 main:574] : INFO : Epoch 2012 | loss: 0.0311082 | val_loss: 0.0311417 | Time: 7187.1 ms [2022-03-31 20:02:07 main:574] : INFO : Epoch 2013 | loss: 0.0311072 | val_loss: 0.0311446 | Time: 11812.8 ms [2022-03-31 20:02:14 main:574] : INFO : Epoch 2014 | loss: 0.0311034 | val_loss: 0.0311397 | Time: 7252.39 ms [2022-03-31 20:02:21 main:574] : INFO : Epoch 2015 | loss: 0.0311013 | val_loss: 0.031142 | Time: 7422.48 ms [2022-03-31 20:02:34 main:574] : INFO : Epoch 2016 | loss: 0.0311028 | val_loss: 0.0311417 | Time: 12405.9 ms [2022-03-31 20:02:43 main:574] : INFO : Epoch 2017 | loss: 0.0311006 | val_loss: 0.0311372 | Time: 8610.81 ms [2022-03-31 20:02:49 main:574] : INFO : Epoch 2018 | loss: 0.0310971 | val_loss: 0.031145 | Time: 6658.83 ms [2022-03-31 20:02:56 main:574] : INFO : Epoch 2019 | loss: 0.0311009 | val_loss: 0.0311426 | Time: 6382.4 ms [2022-03-31 20:03:02 main:574] : INFO : Epoch 2020 | loss: 0.0311 | val_loss: 0.031145 | Time: 6002.92 ms [2022-03-31 20:03:08 main:574] : INFO : Epoch 2021 | loss: 0.0311039 | val_loss: 0.0311495 | Time: 5874.18 ms [2022-03-31 20:03:17 main:574] : INFO : Epoch 2022 | loss: 0.0311 | val_loss: 0.0311402 | Time: 9516.14 ms [2022-03-31 20:03:23 main:574] : INFO : Epoch 2023 | loss: 0.031102 | val_loss: 0.0311362 | Time: 5702.37 ms [2022-03-31 20:03:29 main:574] : INFO : Epoch 2024 | loss: 0.0311024 | val_loss: 0.0311368 | Time: 5786.18 ms [2022-03-31 20:03:35 main:574] : INFO : Epoch 2025 | loss: 0.0311029 | val_loss: 0.0311383 | Time: 5843.75 ms [2022-03-31 20:03:41 main:574] : INFO : Epoch 2026 | loss: 0.0310997 | val_loss: 0.0311375 | Time: 5727.01 ms [2022-03-31 20:03:46 main:574] : INFO : Epoch 2027 | loss: 0.0310971 | val_loss: 0.0311358 | Time: 5811.55 ms [2022-03-31 20:03:52 main:574] : INFO : Epoch 2028 | loss: 0.0310969 | val_loss: 0.0311422 | Time: 5715.79 ms [2022-03-31 20:03:58 main:574] : INFO : Epoch 2029 | loss: 0.0310991 | val_loss: 0.0311422 | Time: 5830.64 ms [2022-03-31 20:04:11 main:574] : INFO : Epoch 2030 | loss: 0.0310989 | val_loss: 0.0311441 | Time: 12752.2 ms [2022-03-31 20:04:23 main:574] : INFO : Epoch 2031 | loss: 0.0310971 | val_loss: 0.0311432 | Time: 11945.9 ms [2022-03-31 20:04:37 main:574] : INFO : Epoch 2032 | loss: 0.0311019 | val_loss: 0.0311432 | Time: 13566.2 ms [2022-03-31 20:04:49 main:574] : INFO : Epoch 2033 | loss: 0.0311003 | val_loss: 0.0311459 | Time: 12013.9 ms [2022-03-31 20:05:01 main:574] : INFO : Epoch 2034 | loss: 0.0311005 | val_loss: 0.0311386 | Time: 11739.9 ms [2022-03-31 20:05:08 main:574] : INFO : Epoch 2035 | loss: 0.0311009 | val_loss: 0.0311354 | Time: 7407.87 ms [2022-03-31 20:05:19 main:574] : INFO : Epoch 2036 | loss: 0.0311009 | val_loss: 0.0311423 | Time: 10652.2 ms [2022-03-31 20:05:26 main:574] : INFO : Epoch 2037 | loss: 0.0310996 | val_loss: 0.0311346 | Time: 7120.34 ms [2022-03-31 20:05:39 main:574] : INFO : Epoch 2038 | loss: 0.0310963 | val_loss: 0.0311422 | Time: 13262.5 ms [2022-03-31 20:05:50 main:574] : INFO : Epoch 2039 | loss: 0.0310947 | val_loss: 0.0311455 | Time: 10049.6 ms [2022-03-31 20:06:02 main:574] : INFO : Epoch 2040 | loss: 0.0310974 | val_loss: 0.0311319 | Time: 11560.8 ms [2022-03-31 20:06:15 main:574] : INFO : Epoch 2041 | loss: 0.0310959 | val_loss: 0.0311455 | Time: 12383.1 ms [2022-03-31 20:06:28 main:574] : INFO : Epoch 2042 | loss: 0.0310943 | val_loss: 0.0311422 | Time: 11619.9 ms [2022-03-31 20:06:42 main:574] : INFO : Epoch 2043 | loss: 0.0310914 | val_loss: 0.0311405 | Time: 13523.2 ms [2022-03-31 20:06:57 main:574] : INFO : Epoch 2044 | loss: 0.0310933 | val_loss: 0.0311399 | Time: 13841.7 ms [2022-03-31 20:07:09 main:574] : INFO : Epoch 2045 | loss: 0.0310941 | val_loss: 0.0311422 | Time: 11232.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-03-31 20:10:31 main:435] : INFO : Set logging level to 1 [2022-03-31 20:10:31 main:441] : INFO : Running in BOINC Client mode [2022-03-31 20:10:31 main:444] : INFO : Resolving all filenames [2022-03-31 20:10:31 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-31 20:10:31 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-31 20:10:31 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-31 20:10:31 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-31 20:10:31 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-31 20:10:31 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-31 20:10:31 main:474] : INFO : Configuration: [2022-03-31 20:10:31 main:475] : INFO : Model type: GRU [2022-03-31 20:10:31 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-31 20:10:31 main:477] : INFO : Max Epochs: 2048 [2022-03-31 20:10:31 main:478] : INFO : Batch Size: 128 [2022-03-31 20:10:31 main:479] : INFO : Learning Rate: 0.01 [2022-03-31 20:10:31 main:480] : INFO : Patience: 10 [2022-03-31 20:10:31 main:481] : INFO : Hidden Width: 12 [2022-03-31 20:10:31 main:482] : INFO : # Recurrent Layers: 4 [2022-03-31 20:10:31 main:483] : INFO : # Backend Layers: 4 [2022-03-31 20:10:31 main:484] : INFO : # Threads: 1 [2022-03-31 20:10:31 main:486] : INFO : Preparing Dataset [2022-03-31 20:10:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-31 20:10:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-31 20:10:39 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-31 20:10:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-31 20:10:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-31 20:10:40 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-31 20:10:40 main:494] : INFO : Creating Model [2022-03-31 20:10:40 main:507] : INFO : Preparing config file [2022-03-31 20:10:40 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-31 20:10:40 main:512] : INFO : Loading config [2022-03-31 20:10:40 main:514] : INFO : Loading state [2022-03-31 20:10:43 main:559] : INFO : Loading DataLoader into Memory [2022-03-31 20:10:43 main:562] : INFO : Starting Training [2022-03-31 20:10:58 main:574] : INFO : Epoch 2043 | loss: 0.0321887 | val_loss: 0.0313031 | Time: 15038.2 ms [2022-03-31 20:11:08 main:574] : INFO : Epoch 2044 | loss: 0.0312282 | val_loss: 0.0312076 | Time: 9800.98 ms [2022-03-31 20:11:15 main:574] : INFO : Epoch 2045 | loss: 0.0311219 | val_loss: 0.0311524 | Time: 7592.9 ms [2022-03-31 20:11:22 main:574] : INFO : Epoch 2046 | loss: 0.0310995 | val_loss: 0.0311403 | Time: 6647.51 ms [2022-03-31 20:11:29 main:574] : INFO : Epoch 2047 | loss: 0.0310962 | val_loss: 0.031141 | Time: 6643.79 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GT 1030) [2022-03-31 20:13:53 main:435] : INFO : Set logging level to 1 [2022-03-31 20:13:53 main:441] : INFO : Running in BOINC Client mode [2022-03-31 20:13:53 main:444] : INFO : Resolving all filenames [2022-03-31 20:13:53 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-31 20:13:53 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-31 20:13:53 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-31 20:13:53 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-31 20:13:53 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-31 20:13:53 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-31 20:13:53 main:474] : INFO : Configuration: [2022-03-31 20:13:53 main:475] : INFO : Model type: GRU [2022-03-31 20:13:53 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-31 20:13:53 main:477] : INFO : Max Epochs: 2048 [2022-03-31 20:13:53 main:478] : INFO : Batch Size: 128 [2022-03-31 20:13:53 main:479] : INFO : Learning Rate: 0.01 [2022-03-31 20:13:53 main:480] : INFO : Patience: 10 [2022-03-31 20:13:53 main:481] : INFO : Hidden Width: 12 [2022-03-31 20:13:53 main:482] : INFO : # Recurrent Layers: 4 [2022-03-31 20:13:53 main:483] : INFO : # Backend Layers: 4 [2022-03-31 20:13:53 main:484] : INFO : # Threads: 1 [2022-03-31 20:13:53 main:486] : INFO : Preparing Dataset [2022-03-31 20:13:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-31 20:13:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-31 20:14:01 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-31 20:14:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-31 20:14:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-31 20:14:02 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-31 20:14:02 main:494] : INFO : Creating Model [2022-03-31 20:14:02 main:507] : INFO : Preparing config file [2022-03-31 20:14:02 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-31 20:14:02 main:512] : INFO : Loading config [2022-03-31 20:14:02 main:514] : INFO : Loading state [2022-03-31 20:14:05 main:559] : INFO : Loading DataLoader into Memory [2022-03-31 20:14:05 main:562] : INFO : Starting Training [2022-03-31 20:14:20 main:574] : INFO : Epoch 2043 | loss: 0.0322008 | val_loss: 0.0312695 | Time: 15218.5 ms [2022-03-31 20:14:32 main:574] : INFO : Epoch 2044 | loss: 0.0312236 | val_loss: 0.0312126 | Time: 11557.8 ms [2022-03-31 20:14:42 main:574] : INFO : Epoch 2045 | loss: 0.0311217 | val_loss: 0.031149 | Time: 10234 ms [2022-03-31 20:14:56 main:574] : INFO : Epoch 2046 | loss: 0.0310987 | val_loss: 0.0311401 | Time: 13436.9 ms [2022-03-31 20:15:10 main:574] : INFO : Epoch 2047 | loss: 0.0310919 | val_loss: 0.0311413 | Time: 14322 ms [2022-03-31 20:15:20 main:574] : INFO : Epoch 2048 | loss: 0.0310913 | val_loss: 0.0311405 | Time: 10133.3 ms [2022-03-31 20:15:20 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311405 [2022-03-31 20:15:21 main:603] : INFO : Saving end state to config to file [2022-03-31 20:15:21 main:608] : INFO : Success, exiting.. 20:15:21 (11944): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)