Task 12256777
| Name | ParityModified-1639959642-29279-3-0_1 |
| Workunit | 8682162 |
| Created | 1 Mar 2022, 0:25:35 UTC |
| Sent | 1 Mar 2022, 2:45:08 UTC |
| Report deadline | 9 Mar 2022, 2:45:08 UTC |
| Received | 29 Mar 2022, 2:04:10 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 15907 |
| Run time | 6 hours 20 min 10 sec |
| CPU time | 5 hours 59 min 57 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 786.69 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.50 GB |
| Peak swap size | 3.59 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> ry [2022-03-28 14:52:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 14:52:16 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 14:52:16 main:494] : INFO : Creating Model [2022-03-28 14:52:16 main:507] : INFO : Preparing config file [2022-03-28 14:52:16 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 14:52:16 main:512] : INFO : Loading config [2022-03-28 14:52:16 main:514] : INFO : Loading state [2022-03-28 14:52:17 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 14:52:17 main:562] : INFO : Starting Training [2022-03-28 14:52:24 main:574] : INFO : Epoch 1973 | loss: 0.0313048 | val_loss: 0.0312243 | Time: 7340.96 ms [2022-03-28 14:52:43 main:574] : INFO : Epoch 1974 | loss: 0.0312059 | val_loss: 0.0311811 | Time: 18725.8 ms [2022-03-28 14:53:07 main:574] : INFO : Epoch 1975 | loss: 0.0311765 | val_loss: 0.0311726 | Time: 23791.5 ms [2022-03-28 14:53:31 main:574] : INFO : Epoch 1976 | loss: 0.0311746 | val_loss: 0.031171 | Time: 24097.5 ms [2022-03-28 14:53:55 main:574] : INFO : Epoch 1977 | loss: 0.0311718 | val_loss: 0.0311688 | Time: 23924.2 ms [2022-03-28 14:54:19 main:574] : INFO : Epoch 1978 | loss: 0.0311703 | val_loss: 0.0311709 | Time: 24364.4 ms [2022-03-28 14:54:56 main:574] : INFO : Epoch 1979 | loss: 0.0311709 | val_loss: 0.0311703 | Time: 36635.7 ms [2022-03-28 14:55:19 main:574] : INFO : Epoch 1980 | loss: 0.0311715 | val_loss: 0.0311698 | Time: 22795.6 ms [2022-03-28 14:55:45 main:574] : INFO : Epoch 1981 | loss: 0.03117 | val_loss: 0.0311686 | Time: 26895.4 ms [2022-03-28 14:56:12 main:574] : INFO : Epoch 1982 | loss: 0.0311712 | val_loss: 0.0311722 | Time: 26544.8 ms [2022-03-28 14:56:39 main:574] : INFO : Epoch 1983 | loss: 0.0311719 | val_loss: 0.03117 | Time: 27201.8 ms [2022-03-28 14:57:14 main:574] : INFO : Epoch 1984 | loss: 0.0311709 | val_loss: 0.0311703 | Time: 35082 ms [2022-03-28 14:57:48 main:574] : INFO : Epoch 1985 | loss: 0.0311698 | val_loss: 0.0311706 | Time: 33551.5 ms [2022-03-28 14:58:15 main:574] : INFO : Epoch 1986 | loss: 0.0311733 | val_loss: 0.03117 | Time: 26735 ms [2022-03-28 14:58:42 main:574] : INFO : Epoch 1987 | loss: 0.0311731 | val_loss: 0.0311702 | Time: 26911.6 ms [2022-03-28 14:59:08 main:574] : INFO : Epoch 1988 | loss: 0.0311714 | val_loss: 0.0311713 | Time: 26043.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 15:17:58 main:435] : INFO : Set logging level to 1 [2022-03-28 15:17:58 main:441] : INFO : Running in BOINC Client mode [2022-03-28 15:17:58 main:444] : INFO : Resolving all filenames [2022-03-28 15:17:58 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 15:17:58 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 15:17:58 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 15:17:58 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 15:17:58 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 15:17:58 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 15:17:58 main:474] : INFO : Configuration: [2022-03-28 15:17:58 main:475] : INFO : Model type: GRU [2022-03-28 15:17:58 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 15:17:58 main:477] : INFO : Max Epochs: 2048 [2022-03-28 15:17:58 main:478] : INFO : Batch Size: 128 [2022-03-28 15:17:58 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 15:17:58 main:480] : INFO : Patience: 10 [2022-03-28 15:17:58 main:481] : INFO : Hidden Width: 12 [2022-03-28 15:17:58 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 15:17:58 main:483] : INFO : # Backend Layers: 4 [2022-03-28 15:17:58 main:484] : INFO : # Threads: 1 [2022-03-28 15:17:58 main:486] : INFO : Preparing Dataset [2022-03-28 15:17:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 15:17:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 15:18:00 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 15:18:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 15:18:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 15:18:00 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 15:18:00 main:494] : INFO : Creating Model [2022-03-28 15:18:00 main:507] : INFO : Preparing config file [2022-03-28 15:18:00 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 15:18:00 main:512] : INFO : Loading config [2022-03-28 15:18:00 main:514] : INFO : Loading state [2022-03-28 15:18:01 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 15:18:01 main:562] : INFO : Starting Training [2022-03-28 15:18:08 main:574] : INFO : Epoch 1989 | loss: 0.0312784 | val_loss: 0.031209 | Time: 7567.79 ms [2022-03-28 15:18:28 main:574] : INFO : Epoch 1990 | loss: 0.0311864 | val_loss: 0.0311715 | Time: 20154.6 ms [2022-03-28 15:18:53 main:574] : INFO : Epoch 1991 | loss: 0.0311758 | val_loss: 0.0311718 | Time: 24194 ms [2022-03-28 15:19:18 main:574] : INFO : Epoch 1992 | loss: 0.0311713 | val_loss: 0.031169 | Time: 25166.8 ms [2022-03-28 15:19:46 main:574] : INFO : Epoch 1993 | loss: 0.0311695 | val_loss: 0.0311675 | Time: 28273.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 15:23:00 main:435] : INFO : Set logging level to 1 [2022-03-28 15:23:00 main:441] : INFO : Running in BOINC Client mode [2022-03-28 15:23:00 main:444] : INFO : Resolving all filenames [2022-03-28 15:23:00 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 15:23:00 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 15:23:00 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 15:23:00 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 15:23:00 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 15:23:00 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 15:23:00 main:474] : INFO : Configuration: [2022-03-28 15:23:00 main:475] : INFO : Model type: GRU [2022-03-28 15:23:00 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 15:23:00 main:477] : INFO : Max Epochs: 2048 [2022-03-28 15:23:00 main:478] : INFO : Batch Size: 128 [2022-03-28 15:23:00 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 15:23:00 main:480] : INFO : Patience: 10 [2022-03-28 15:23:00 main:481] : INFO : Hidden Width: 12 [2022-03-28 15:23:00 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 15:23:00 main:483] : INFO : # Backend Layers: 4 [2022-03-28 15:23:00 main:484] : INFO : # Threads: 1 [2022-03-28 15:23:00 main:486] : INFO : Preparing Dataset [2022-03-28 15:23:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 15:23:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 15:23:02 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 15:23:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 15:23:02 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 15:23:02 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 15:23:02 main:494] : INFO : Creating Model [2022-03-28 15:23:02 main:507] : INFO : Preparing config file [2022-03-28 15:23:02 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 15:23:02 main:512] : INFO : Loading config [2022-03-28 15:23:02 main:514] : INFO : Loading state [2022-03-28 15:23:03 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 15:23:03 main:562] : INFO : Starting Training [2022-03-28 15:23:11 main:574] : INFO : Epoch 1989 | loss: 0.0312765 | val_loss: 0.0312269 | Time: 7513.84 ms [2022-03-28 15:23:30 main:574] : INFO : Epoch 1990 | loss: 0.0312 | val_loss: 0.0311773 | Time: 19699 ms [2022-03-28 15:23:55 main:574] : INFO : Epoch 1991 | loss: 0.0311748 | val_loss: 0.031175 | Time: 24430.5 ms [2022-03-28 15:24:20 main:574] : INFO : Epoch 1992 | loss: 0.0311718 | val_loss: 0.0311735 | Time: 25148.4 ms [2022-03-28 15:24:52 main:574] : INFO : Epoch 1993 | loss: 0.0311706 | val_loss: 0.0311746 | Time: 32042.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 16:24:07 main:435] : INFO : Set logging level to 1 [2022-03-28 16:24:07 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:24:07 main:444] : INFO : Resolving all filenames [2022-03-28 16:24:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:24:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:24:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:24:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:24:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:24:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:24:07 main:474] : INFO : Configuration: [2022-03-28 16:24:07 main:475] : INFO : Model type: GRU [2022-03-28 16:24:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:24:07 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:24:07 main:478] : INFO : Batch Size: 128 [2022-03-28 16:24:07 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:24:07 main:480] : INFO : Patience: 10 [2022-03-28 16:24:07 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:24:07 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:24:07 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:24:07 main:484] : INFO : # Threads: 1 [2022-03-28 16:24:07 main:486] : INFO : Preparing Dataset [2022-03-28 16:24:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:24:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 16:24:09 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 16:24:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 16:24:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 16:24:09 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 16:24:09 main:494] : INFO : Creating Model [2022-03-28 16:24:09 main:507] : INFO : Preparing config file [2022-03-28 16:24:09 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 16:24:09 main:512] : INFO : Loading config [2022-03-28 16:24:09 main:514] : INFO : Loading state [2022-03-28 16:24:10 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 16:24:10 main:562] : INFO : Starting Training [2022-03-28 16:24:17 main:574] : INFO : Epoch 1989 | loss: 0.0312784 | val_loss: 0.0312031 | Time: 7136.56 ms [2022-03-28 16:24:37 main:574] : INFO : Epoch 1990 | loss: 0.0311882 | val_loss: 0.0311766 | Time: 20241 ms [2022-03-28 16:25:07 main:574] : INFO : Epoch 1991 | loss: 0.0311738 | val_loss: 0.0311723 | Time: 29713.6 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 16:31:08 main:435] : INFO : Set logging level to 1 [2022-03-28 16:31:08 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:31:08 main:444] : INFO : Resolving all filenames [2022-03-28 16:31:08 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:31:08 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:31:08 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:31:08 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:31:08 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:31:08 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:31:08 main:474] : INFO : Configuration: [2022-03-28 16:31:08 main:475] : INFO : Model type: GRU [2022-03-28 16:31:08 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:31:08 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:31:08 main:478] : INFO : Batch Size: 128 [2022-03-28 16:31:08 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:31:08 main:480] : INFO : Patience: 10 [2022-03-28 16:31:08 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:31:08 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:31:08 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:31:08 main:484] : INFO : # Threads: 1 [2022-03-28 16:31:08 main:486] : INFO : Preparing Dataset [2022-03-28 16:31:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:31:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 16:31:10 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 16:31:10 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 16:31:10 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 16:31:10 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 16:31:10 main:494] : INFO : Creating Model [2022-03-28 16:31:10 main:507] : INFO : Preparing config file [2022-03-28 16:31:10 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 16:31:10 main:512] : INFO : Loading config [2022-03-28 16:31:10 main:514] : INFO : Loading state [2022-03-28 16:31:11 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 16:31:11 main:562] : INFO : Starting Training [2022-03-28 16:31:18 main:574] : INFO : Epoch 1989 | loss: 0.0312996 | val_loss: 0.0312376 | Time: 7356.81 ms [2022-03-28 16:31:39 main:574] : INFO : Epoch 1990 | loss: 0.0312051 | val_loss: 0.0311855 | Time: 20960.3 ms [2022-03-28 16:32:04 main:574] : INFO : Epoch 1991 | loss: 0.031177 | val_loss: 0.0311715 | Time: 24590.2 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 16:36:08 main:435] : INFO : Set logging level to 1 [2022-03-28 16:36:08 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:36:08 main:444] : INFO : Resolving all filenames [2022-03-28 16:36:08 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:36:08 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:36:08 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:36:08 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:36:08 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:36:08 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:36:08 main:474] : INFO : Configuration: [2022-03-28 16:36:08 main:475] : INFO : Model type: GRU [2022-03-28 16:36:08 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:36:08 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:36:08 main:478] : INFO : Batch Size: 128 [2022-03-28 16:36:08 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:36:08 main:480] : INFO : Patience: 10 [2022-03-28 16:36:08 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:36:08 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:36:08 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:36:08 main:484] : INFO : # Threads: 1 [2022-03-28 16:36:08 main:486] : INFO : Preparing Dataset [2022-03-28 16:36:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:36:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 16:36:10 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 16:36:10 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 16:36:10 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 16:36:10 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 16:36:10 main:494] : INFO : Creating Model [2022-03-28 16:36:10 main:507] : INFO : Preparing config file [2022-03-28 16:36:10 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 16:36:10 main:512] : INFO : Loading config [2022-03-28 16:36:10 main:514] : INFO : Loading state [2022-03-28 16:36:11 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 16:36:11 main:562] : INFO : Starting Training [2022-03-28 16:36:19 main:574] : INFO : Epoch 1989 | loss: 0.0312866 | val_loss: 0.0311957 | Time: 7865.55 ms [2022-03-28 16:36:59 main:574] : INFO : Epoch 1990 | loss: 0.0311869 | val_loss: 0.0311737 | Time: 39871.8 ms [2022-03-28 16:37:41 main:574] : INFO : Epoch 1991 | loss: 0.0311738 | val_loss: 0.0311734 | Time: 41428.8 ms [2022-03-28 16:38:25 main:574] : INFO : Epoch 1992 | loss: 0.0311721 | val_loss: 0.0311736 | Time: 44283.6 ms [2022-03-28 16:39:08 main:574] : INFO : Epoch 1993 | loss: 0.0311712 | val_loss: 0.0311692 | Time: 43469.2 ms [2022-03-28 16:40:03 main:574] : INFO : Epoch 1994 | loss: 0.0311709 | val_loss: 0.0311702 | Time: 55123.8 ms [2022-03-28 16:41:12 main:574] : INFO : Epoch 1995 | loss: 0.0311695 | val_loss: 0.0311706 | Time: 68891.5 ms [2022-03-28 16:42:12 main:574] : INFO : Epoch 1996 | loss: 0.0311706 | val_loss: 0.031171 | Time: 59800.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 16:51:40 main:435] : INFO : Set logging level to 1 [2022-03-28 16:51:40 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:51:40 main:444] : INFO : Resolving all filenames [2022-03-28 16:51:40 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:51:40 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:51:40 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:51:40 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:51:40 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:51:40 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:51:40 main:474] : INFO : Configuration: [2022-03-28 16:51:40 main:475] : INFO : Model type: GRU [2022-03-28 16:51:40 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:51:40 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:51:40 main:478] : INFO : Batch Size: 128 [2022-03-28 16:51:40 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:51:40 main:480] : INFO : Patience: 10 [2022-03-28 16:51:40 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:51:40 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:51:40 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:51:40 main:484] : INFO : # Threads: 1 [2022-03-28 16:51:40 main:486] : INFO : Preparing Dataset [2022-03-28 16:51:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:51:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 16:51:42 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 16:51:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 16:51:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 16:51:42 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 16:51:42 main:494] : INFO : Creating Model [2022-03-28 16:51:42 main:507] : INFO : Preparing config file [2022-03-28 16:51:42 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 16:51:42 main:512] : INFO : Loading config [2022-03-28 16:51:42 main:514] : INFO : Loading state [2022-03-28 16:51:43 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 16:51:43 main:562] : INFO : Starting Training [2022-03-28 16:51:50 main:574] : INFO : Epoch 1995 | loss: 0.0312579 | val_loss: 0.0311997 | Time: 7449.99 ms [2022-03-28 16:52:08 main:574] : INFO : Epoch 1996 | loss: 0.0311808 | val_loss: 0.0311742 | Time: 18121.4 ms [2022-03-28 16:52:31 main:574] : INFO : Epoch 1997 | loss: 0.0311729 | val_loss: 0.03117 | Time: 22979.8 ms [2022-03-28 16:52:55 main:574] : INFO : Epoch 1998 | loss: 0.031172 | val_loss: 0.0311717 | Time: 23309.3 ms [2022-03-28 16:53:18 main:574] : INFO : Epoch 1999 | loss: 0.0311732 | val_loss: 0.0311731 | Time: 23275 ms [2022-03-28 16:53:42 main:574] : INFO : Epoch 2000 | loss: 0.0311734 | val_loss: 0.0311701 | Time: 23696 ms [2022-03-28 16:54:06 main:574] : INFO : Epoch 2001 | loss: 0.0311715 | val_loss: 0.031172 | Time: 24246 ms [2022-03-28 16:54:31 main:574] : INFO : Epoch 2002 | loss: 0.0311705 | val_loss: 0.0311707 | Time: 25209.2 ms [2022-03-28 16:55:04 main:574] : INFO : Epoch 2003 | loss: 0.0311706 | val_loss: 0.0311706 | Time: 32694.5 ms [2022-03-28 16:55:39 main:574] : INFO : Epoch 2004 | loss: 0.0311719 | val_loss: 0.0311718 | Time: 35253.4 ms [2022-03-28 16:56:13 main:574] : INFO : Epoch 2005 | loss: 0.0311716 | val_loss: 0.0311699 | Time: 33553.8 ms [2022-03-28 16:56:40 main:574] : INFO : Epoch 2006 | loss: 0.0311716 | val_loss: 0.0311705 | Time: 27220.7 ms [2022-03-28 16:57:07 main:574] : INFO : Epoch 2007 | loss: 0.0311716 | val_loss: 0.0311761 | Time: 26675.2 ms [2022-03-28 16:57:33 main:574] : INFO : Epoch 2008 | loss: 0.0311724 | val_loss: 0.0311758 | Time: 26653.8 ms [2022-03-28 16:58:00 main:574] : INFO : Epoch 2009 | loss: 0.0311695 | val_loss: 0.0311708 | Time: 27217.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 17:01:34 main:435] : INFO : Set logging level to 1 [2022-03-28 17:01:34 main:441] : INFO : Running in BOINC Client mode [2022-03-28 17:01:34 main:444] : INFO : Resolving all filenames [2022-03-28 17:01:34 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 17:01:34 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 17:01:34 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 17:01:34 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 17:01:34 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 17:01:34 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 17:01:34 main:474] : INFO : Configuration: [2022-03-28 17:01:34 main:475] : INFO : Model type: GRU [2022-03-28 17:01:34 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 17:01:34 main:477] : INFO : Max Epochs: 2048 [2022-03-28 17:01:34 main:478] : INFO : Batch Size: 128 [2022-03-28 17:01:34 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 17:01:34 main:480] : INFO : Patience: 10 [2022-03-28 17:01:34 main:481] : INFO : Hidden Width: 12 [2022-03-28 17:01:34 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 17:01:34 main:483] : INFO : # Backend Layers: 4 [2022-03-28 17:01:34 main:484] : INFO : # Threads: 1 [2022-03-28 17:01:34 main:486] : INFO : Preparing Dataset [2022-03-28 17:01:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 17:01:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 17:01:36 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 17:01:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 17:01:36 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 17:01:36 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 17:01:36 main:494] : INFO : Creating Model [2022-03-28 17:01:36 main:507] : INFO : Preparing config file [2022-03-28 17:01:36 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 17:01:36 main:512] : INFO : Loading config [2022-03-28 17:01:36 main:514] : INFO : Loading state [2022-03-28 17:01:36 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 17:01:36 main:562] : INFO : Starting Training [2022-03-28 17:01:44 main:574] : INFO : Epoch 2004 | loss: 0.0312671 | val_loss: 0.0312187 | Time: 7407.64 ms [2022-03-28 17:02:03 main:574] : INFO : Epoch 2005 | loss: 0.0311878 | val_loss: 0.0311753 | Time: 18869 ms [2022-03-28 17:02:27 main:574] : INFO : Epoch 2006 | loss: 0.0311728 | val_loss: 0.03117 | Time: 24472 ms [2022-03-28 17:02:52 main:574] : INFO : Epoch 2007 | loss: 0.0311705 | val_loss: 0.03117 | Time: 24725.8 ms [2022-03-28 17:03:17 main:574] : INFO : Epoch 2008 | loss: 0.0311702 | val_loss: 0.03117 | Time: 24780.2 ms [2022-03-28 17:03:44 main:574] : INFO : Epoch 2009 | loss: 0.0311705 | val_loss: 0.0311681 | Time: 26856.4 ms [2022-03-28 17:04:11 main:574] : INFO : Epoch 2010 | loss: 0.0311696 | val_loss: 0.031168 | Time: 27411.6 ms [2022-03-28 17:04:39 main:574] : INFO : Epoch 2011 | loss: 0.0311702 | val_loss: 0.0311713 | Time: 27560.3 ms [2022-03-28 17:05:06 main:574] : INFO : Epoch 2012 | loss: 0.0311725 | val_loss: 0.0311713 | Time: 27445.4 ms [2022-03-28 17:05:34 main:574] : INFO : Epoch 2013 | loss: 0.0311727 | val_loss: 0.0311727 | Time: 27498.2 ms [2022-03-28 17:06:09 main:574] : INFO : Epoch 2014 | loss: 0.0311747 | val_loss: 0.0311723 | Time: 34990.2 ms [2022-03-28 17:06:43 main:574] : INFO : Epoch 2015 | loss: 0.0311716 | val_loss: 0.0311722 | Time: 33914.4 ms [2022-03-28 17:07:10 main:574] : INFO : Epoch 2016 | loss: 0.0311705 | val_loss: 0.0311702 | Time: 27464.7 ms [2022-03-28 17:07:37 main:574] : INFO : Epoch 2017 | loss: 0.0311699 | val_loss: 0.031171 | Time: 27044.2 ms [2022-03-28 17:08:04 main:574] : INFO : Epoch 2018 | loss: 0.0311712 | val_loss: 0.0311702 | Time: 27306.2 ms [2022-03-28 17:08:31 main:574] : INFO : Epoch 2019 | loss: 0.0311702 | val_loss: 0.031171 | Time: 26934.5 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 17:24:41 main:435] : INFO : Set logging level to 1 [2022-03-28 17:24:41 main:441] : INFO : Running in BOINC Client mode [2022-03-28 17:24:41 main:444] : INFO : Resolving all filenames [2022-03-28 17:24:41 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 17:24:41 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 17:24:41 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 17:24:41 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 17:24:41 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 17:24:41 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 17:24:41 main:474] : INFO : Configuration: [2022-03-28 17:24:41 main:475] : INFO : Model type: GRU [2022-03-28 17:24:41 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 17:24:41 main:477] : INFO : Max Epochs: 2048 [2022-03-28 17:24:41 main:478] : INFO : Batch Size: 128 [2022-03-28 17:24:41 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 17:24:41 main:480] : INFO : Patience: 10 [2022-03-28 17:24:41 main:481] : INFO : Hidden Width: 12 [2022-03-28 17:24:41 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 17:24:41 main:483] : INFO : # Backend Layers: 4 [2022-03-28 17:24:41 main:484] : INFO : # Threads: 1 [2022-03-28 17:24:41 main:486] : INFO : Preparing Dataset [2022-03-28 17:24:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 17:24:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 17:24:43 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 17:24:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 17:24:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 17:24:43 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 17:24:43 main:494] : INFO : Creating Model [2022-03-28 17:24:43 main:507] : INFO : Preparing config file [2022-03-28 17:24:43 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 17:24:43 main:512] : INFO : Loading config [2022-03-28 17:24:43 main:514] : INFO : Loading state [2022-03-28 17:24:44 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 17:24:44 main:562] : INFO : Starting Training [2022-03-28 17:24:51 main:574] : INFO : Epoch 2020 | loss: 0.0312906 | val_loss: 0.0312427 | Time: 7118.8 ms [2022-03-28 17:25:10 main:574] : INFO : Epoch 2021 | loss: 0.031207 | val_loss: 0.0311902 | Time: 19543 ms [2022-03-28 17:25:35 main:574] : INFO : Epoch 2022 | loss: 0.0311774 | val_loss: 0.0311706 | Time: 24520 ms [2022-03-28 17:26:00 main:574] : INFO : Epoch 2023 | loss: 0.0311723 | val_loss: 0.0311695 | Time: 25206.4 ms [2022-03-28 17:26:25 main:574] : INFO : Epoch 2024 | loss: 0.0311709 | val_loss: 0.0311694 | Time: 24574.2 ms [2022-03-28 17:26:49 main:574] : INFO : Epoch 2025 | loss: 0.0311713 | val_loss: 0.0311692 | Time: 24073.9 ms [2022-03-28 17:27:14 main:574] : INFO : Epoch 2026 | loss: 0.0311703 | val_loss: 0.0311703 | Time: 25442.9 ms [2022-03-28 17:27:41 main:574] : INFO : Epoch 2027 | loss: 0.0311713 | val_loss: 0.0311765 | Time: 26519.9 ms [2022-03-28 17:28:08 main:574] : INFO : Epoch 2028 | loss: 0.0311739 | val_loss: 0.0311698 | Time: 27317 ms [2022-03-28 17:28:36 main:574] : INFO : Epoch 2029 | loss: 0.0311727 | val_loss: 0.0311693 | Time: 27539.3 ms [2022-03-28 17:29:03 main:574] : INFO : Epoch 2030 | loss: 0.0311715 | val_loss: 0.0311684 | Time: 27594 ms [2022-03-28 17:29:31 main:574] : INFO : Epoch 2031 | loss: 0.0311697 | val_loss: 0.0311688 | Time: 27867.8 ms [2022-03-28 17:29:58 main:574] : INFO : Epoch 2032 | loss: 0.0311693 | val_loss: 0.0311689 | Time: 27356.8 ms [2022-03-28 17:30:33 main:574] : INFO : Epoch 2033 | loss: 0.0311703 | val_loss: 0.0311704 | Time: 34969.9 ms [2022-03-28 17:31:01 main:574] : INFO : Epoch 2034 | loss: 0.031169 | val_loss: 0.0311688 | Time: 27321.6 ms [2022-03-28 17:31:35 main:574] : INFO : Epoch 2035 | loss: 0.0311681 | val_loss: 0.0311703 | Time: 33898 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 17:44:43 main:435] : INFO : Set logging level to 1 [2022-03-28 17:44:43 main:441] : INFO : Running in BOINC Client mode [2022-03-28 17:44:43 main:444] : INFO : Resolving all filenames [2022-03-28 17:44:43 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 17:44:43 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 17:44:43 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 17:44:43 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 17:44:43 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 17:44:43 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 17:44:43 main:474] : INFO : Configuration: [2022-03-28 17:44:43 main:475] : INFO : Model type: GRU [2022-03-28 17:44:43 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 17:44:43 main:477] : INFO : Max Epochs: 2048 [2022-03-28 17:44:43 main:478] : INFO : Batch Size: 128 [2022-03-28 17:44:43 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 17:44:43 main:480] : INFO : Patience: 10 [2022-03-28 17:44:43 main:481] : INFO : Hidden Width: 12 [2022-03-28 17:44:43 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 17:44:43 main:483] : INFO : # Backend Layers: 4 [2022-03-28 17:44:43 main:484] : INFO : # Threads: 1 [2022-03-28 17:44:43 main:486] : INFO : Preparing Dataset [2022-03-28 17:44:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 17:44:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 17:44:45 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 17:44:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 17:44:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 17:44:45 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 17:44:45 main:494] : INFO : Creating Model [2022-03-28 17:44:45 main:507] : INFO : Preparing config file [2022-03-28 17:44:45 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 17:44:45 main:512] : INFO : Loading config [2022-03-28 17:44:45 main:514] : INFO : Loading state [2022-03-28 17:44:46 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 17:44:46 main:562] : INFO : Starting Training Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 17:52:15 main:435] : INFO : Set logging level to 1 [2022-03-28 17:52:15 main:441] : INFO : Running in BOINC Client mode [2022-03-28 17:52:15 main:444] : INFO : Resolving all filenames [2022-03-28 17:52:15 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 17:52:15 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 17:52:15 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 17:52:15 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 17:52:15 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 17:52:15 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 17:52:15 main:474] : INFO : Configuration: [2022-03-28 17:52:15 main:475] : INFO : Model type: GRU [2022-03-28 17:52:15 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 17:52:15 main:477] : INFO : Max Epochs: 2048 [2022-03-28 17:52:15 main:478] : INFO : Batch Size: 128 [2022-03-28 17:52:15 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 17:52:15 main:480] : INFO : Patience: 10 [2022-03-28 17:52:15 main:481] : INFO : Hidden Width: 12 [2022-03-28 17:52:15 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 17:52:15 main:483] : INFO : # Backend Layers: 4 [2022-03-28 17:52:15 main:484] : INFO : # Threads: 1 [2022-03-28 17:52:15 main:486] : INFO : Preparing Dataset [2022-03-28 17:52:15 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 17:52:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 17:52:17 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 17:52:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 17:52:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 17:52:18 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 17:52:18 main:494] : INFO : Creating Model [2022-03-28 17:52:18 main:507] : INFO : Preparing config file [2022-03-28 17:52:18 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 17:52:18 main:512] : INFO : Loading config [2022-03-28 17:52:18 main:514] : INFO : Loading state [2022-03-28 17:52:18 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 17:52:18 main:562] : INFO : Starting Training [2022-03-28 17:52:26 main:574] : INFO : Epoch 2036 | loss: 0.0312764 | val_loss: 0.0312131 | Time: 7591.4 ms [2022-03-28 17:52:44 main:574] : INFO : Epoch 2037 | loss: 0.0311897 | val_loss: 0.0311718 | Time: 18341.6 ms [2022-03-28 17:53:09 main:574] : INFO : Epoch 2038 | loss: 0.0311714 | val_loss: 0.0311696 | Time: 24336.9 ms [2022-03-28 17:53:33 main:574] : INFO : Epoch 2039 | loss: 0.0311678 | val_loss: 0.0311681 | Time: 24291.8 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 18:34:16 main:435] : INFO : Set logging level to 1 [2022-03-28 18:34:16 main:441] : INFO : Running in BOINC Client mode [2022-03-28 18:34:16 main:444] : INFO : Resolving all filenames [2022-03-28 18:34:16 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 18:34:16 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 18:34:16 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 18:34:16 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 18:34:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 18:34:16 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 18:34:16 main:474] : INFO : Configuration: [2022-03-28 18:34:16 main:475] : INFO : Model type: GRU [2022-03-28 18:34:16 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 18:34:16 main:477] : INFO : Max Epochs: 2048 [2022-03-28 18:34:16 main:478] : INFO : Batch Size: 128 [2022-03-28 18:34:16 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 18:34:16 main:480] : INFO : Patience: 10 [2022-03-28 18:34:16 main:481] : INFO : Hidden Width: 12 [2022-03-28 18:34:16 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 18:34:16 main:483] : INFO : # Backend Layers: 4 [2022-03-28 18:34:16 main:484] : INFO : # Threads: 1 [2022-03-28 18:34:16 main:486] : INFO : Preparing Dataset [2022-03-28 18:34:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 18:34:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 18:34:18 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 18:34:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 18:34:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 18:34:18 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 18:34:18 main:494] : INFO : Creating Model [2022-03-28 18:34:18 main:507] : INFO : Preparing config file [2022-03-28 18:34:18 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 18:34:18 main:512] : INFO : Loading config [2022-03-28 18:34:18 main:514] : INFO : Loading state [2022-03-28 18:34:18 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 18:34:18 main:562] : INFO : Starting Training [2022-03-28 18:34:26 main:574] : INFO : Epoch 2036 | loss: 0.0312873 | val_loss: 0.0312219 | Time: 7462.39 ms [2022-03-28 18:34:43 main:574] : INFO : Epoch 2037 | loss: 0.0311947 | val_loss: 0.0311825 | Time: 17403.5 ms [2022-03-28 18:35:07 main:574] : INFO : Epoch 2038 | loss: 0.0311716 | val_loss: 0.0311681 | Time: 23419.1 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 18:40:25 main:435] : INFO : Set logging level to 1 [2022-03-28 18:40:25 main:441] : INFO : Running in BOINC Client mode [2022-03-28 18:40:25 main:444] : INFO : Resolving all filenames [2022-03-28 18:40:25 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 18:40:25 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 18:40:25 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 18:40:25 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 18:40:25 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 18:40:25 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 18:40:25 main:474] : INFO : Configuration: [2022-03-28 18:40:25 main:475] : INFO : Model type: GRU [2022-03-28 18:40:25 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 18:40:25 main:477] : INFO : Max Epochs: 2048 [2022-03-28 18:40:25 main:478] : INFO : Batch Size: 128 [2022-03-28 18:40:25 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 18:40:25 main:480] : INFO : Patience: 10 [2022-03-28 18:40:25 main:481] : INFO : Hidden Width: 12 [2022-03-28 18:40:25 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 18:40:25 main:483] : INFO : # Backend Layers: 4 [2022-03-28 18:40:25 main:484] : INFO : # Threads: 1 [2022-03-28 18:40:25 main:486] : INFO : Preparing Dataset [2022-03-28 18:40:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 18:40:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 18:40:27 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 18:40:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 18:40:27 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 18:40:27 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 18:40:27 main:494] : INFO : Creating Model [2022-03-28 18:40:27 main:507] : INFO : Preparing config file [2022-03-28 18:40:27 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 18:40:27 main:512] : INFO : Loading config [2022-03-28 18:40:27 main:514] : INFO : Loading state [2022-03-28 18:40:28 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 18:40:28 main:562] : INFO : Starting Training [2022-03-28 18:40:35 main:574] : INFO : Epoch 2036 | loss: 0.0312775 | val_loss: 0.0312186 | Time: 7545.58 ms [2022-03-28 18:40:55 main:574] : INFO : Epoch 2037 | loss: 0.0311899 | val_loss: 0.0311702 | Time: 19445.1 ms [2022-03-28 18:41:19 main:574] : INFO : Epoch 2038 | loss: 0.03117 | val_loss: 0.03117 | Time: 24349.5 ms [2022-03-28 18:41:43 main:574] : INFO : Epoch 2039 | loss: 0.0311699 | val_loss: 0.0311743 | Time: 23707.9 ms [2022-03-28 18:42:06 main:574] : INFO : Epoch 2040 | loss: 0.0311703 | val_loss: 0.0311682 | Time: 23180.5 ms [2022-03-28 18:42:30 main:574] : INFO : Epoch 2041 | loss: 0.0311697 | val_loss: 0.0311698 | Time: 23349.8 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 18:45:41 main:435] : INFO : Set logging level to 1 [2022-03-28 18:45:41 main:441] : INFO : Running in BOINC Client mode [2022-03-28 18:45:41 main:444] : INFO : Resolving all filenames [2022-03-28 18:45:41 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 18:45:41 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 18:45:41 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 18:45:41 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 18:45:41 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 18:45:41 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 18:45:41 main:474] : INFO : Configuration: [2022-03-28 18:45:41 main:475] : INFO : Model type: GRU [2022-03-28 18:45:41 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 18:45:41 main:477] : INFO : Max Epochs: 2048 [2022-03-28 18:45:41 main:478] : INFO : Batch Size: 128 [2022-03-28 18:45:41 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 18:45:41 main:480] : INFO : Patience: 10 [2022-03-28 18:45:41 main:481] : INFO : Hidden Width: 12 [2022-03-28 18:45:41 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 18:45:41 main:483] : INFO : # Backend Layers: 4 [2022-03-28 18:45:41 main:484] : INFO : # Threads: 1 [2022-03-28 18:45:41 main:486] : INFO : Preparing Dataset [2022-03-28 18:45:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 18:45:42 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 18:45:43 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 18:45:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 18:45:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 18:45:44 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 18:45:44 main:494] : INFO : Creating Model [2022-03-28 18:45:44 main:507] : INFO : Preparing config file [2022-03-28 18:45:44 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 18:45:44 main:512] : INFO : Loading config [2022-03-28 18:45:44 main:514] : INFO : Loading state [2022-03-28 18:45:44 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 18:45:44 main:562] : INFO : Starting Training [2022-03-28 18:45:52 main:574] : INFO : Epoch 2036 | loss: 0.0312796 | val_loss: 0.0312159 | Time: 7698.72 ms [2022-03-28 18:46:11 main:574] : INFO : Epoch 2037 | loss: 0.031191 | val_loss: 0.0311741 | Time: 18627 ms [2022-03-28 18:46:33 main:574] : INFO : Epoch 2038 | loss: 0.0311722 | val_loss: 0.0311705 | Time: 22471.2 ms [2022-03-28 18:46:56 main:574] : INFO : Epoch 2039 | loss: 0.0311695 | val_loss: 0.0311688 | Time: 22827.9 ms [2022-03-28 18:47:18 main:574] : INFO : Epoch 2040 | loss: 0.031169 | val_loss: 0.0311699 | Time: 22157 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 18:50:56 main:435] : INFO : Set logging level to 1 [2022-03-28 18:50:56 main:441] : INFO : Running in BOINC Client mode [2022-03-28 18:50:56 main:444] : INFO : Resolving all filenames [2022-03-28 18:50:56 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 18:50:56 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 18:50:56 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 18:50:56 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 18:50:56 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 18:50:56 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 18:50:56 main:474] : INFO : Configuration: [2022-03-28 18:50:56 main:475] : INFO : Model type: GRU [2022-03-28 18:50:56 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 18:50:56 main:477] : INFO : Max Epochs: 2048 [2022-03-28 18:50:56 main:478] : INFO : Batch Size: 128 [2022-03-28 18:50:56 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 18:50:56 main:480] : INFO : Patience: 10 [2022-03-28 18:50:56 main:481] : INFO : Hidden Width: 12 [2022-03-28 18:50:56 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 18:50:56 main:483] : INFO : # Backend Layers: 4 [2022-03-28 18:50:56 main:484] : INFO : # Threads: 1 [2022-03-28 18:50:56 main:486] : INFO : Preparing Dataset [2022-03-28 18:50:56 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 18:50:57 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 18:50:58 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 18:50:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 18:50:58 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 18:50:59 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 18:50:59 main:494] : INFO : Creating Model [2022-03-28 18:50:59 main:507] : INFO : Preparing config file [2022-03-28 18:50:59 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 18:50:59 main:512] : INFO : Loading config [2022-03-28 18:50:59 main:514] : INFO : Loading state [2022-03-28 18:50:59 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 18:50:59 main:562] : INFO : Starting Training [2022-03-28 18:51:06 main:574] : INFO : Epoch 2036 | loss: 0.0312757 | val_loss: 0.0312156 | Time: 7079.47 ms [2022-03-28 18:51:22 main:574] : INFO : Epoch 2037 | loss: 0.0311908 | val_loss: 0.0311793 | Time: 16166.2 ms [2022-03-28 18:51:40 main:574] : INFO : Epoch 2038 | loss: 0.0311726 | val_loss: 0.0311711 | Time: 17934.2 ms [2022-03-28 18:52:10 main:574] : INFO : Epoch 2039 | loss: 0.0311704 | val_loss: 0.0311688 | Time: 29169.2 ms [2022-03-28 18:52:25 main:574] : INFO : Epoch 2040 | loss: 0.0311689 | val_loss: 0.0311682 | Time: 15428.3 ms [2022-03-28 18:52:43 main:574] : INFO : Epoch 2041 | loss: 0.0311691 | val_loss: 0.03117 | Time: 18240.8 ms [2022-03-28 18:53:02 main:574] : INFO : Epoch 2042 | loss: 0.0311701 | val_loss: 0.031173 | Time: 18435.3 ms [2022-03-28 18:53:21 main:574] : INFO : Epoch 2043 | loss: 0.0311691 | val_loss: 0.0311706 | Time: 19465.8 ms [2022-03-28 18:53:41 main:574] : INFO : Epoch 2044 | loss: 0.0311682 | val_loss: 0.0311702 | Time: 19863 ms [2022-03-28 18:54:01 main:574] : INFO : Epoch 2045 | loss: 0.0311692 | val_loss: 0.0311752 | Time: 20299.7 ms Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce MX230) [2022-03-28 18:58:49 main:435] : INFO : Set logging level to 1 [2022-03-28 18:58:49 main:441] : INFO : Running in BOINC Client mode [2022-03-28 18:58:49 main:444] : INFO : Resolving all filenames [2022-03-28 18:58:49 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 18:58:49 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 18:58:49 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 18:58:49 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 18:58:49 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 18:58:49 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 18:58:49 main:474] : INFO : Configuration: [2022-03-28 18:58:49 main:475] : INFO : Model type: GRU [2022-03-28 18:58:49 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 18:58:49 main:477] : INFO : Max Epochs: 2048 [2022-03-28 18:58:49 main:478] : INFO : Batch Size: 128 [2022-03-28 18:58:49 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 18:58:49 main:480] : INFO : Patience: 10 [2022-03-28 18:58:49 main:481] : INFO : Hidden Width: 12 [2022-03-28 18:58:49 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 18:58:49 main:483] : INFO : # Backend Layers: 4 [2022-03-28 18:58:49 main:484] : INFO : # Threads: 1 [2022-03-28 18:58:49 main:486] : INFO : Preparing Dataset [2022-03-28 18:58:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 18:58:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-28 18:58:51 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-28 18:58:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-28 18:58:51 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-28 18:58:51 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-28 18:58:51 main:494] : INFO : Creating Model [2022-03-28 18:58:51 main:507] : INFO : Preparing config file [2022-03-28 18:58:51 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-28 18:58:51 main:512] : INFO : Loading config [2022-03-28 18:58:51 main:514] : INFO : Loading state [2022-03-28 18:58:52 main:559] : INFO : Loading DataLoader into Memory [2022-03-28 18:58:52 main:562] : INFO : Starting Training [2022-03-28 18:59:00 main:574] : INFO : Epoch 2036 | loss: 0.0312791 | val_loss: 0.0312144 | Time: 7659.54 ms [2022-03-28 18:59:19 main:574] : INFO : Epoch 2037 | loss: 0.031191 | val_loss: 0.0311792 | Time: 19005.4 ms [2022-03-28 18:59:41 main:574] : INFO : Epoch 2038 | loss: 0.0311735 | val_loss: 0.031171 | Time: 22734.3 ms [2022-03-28 19:00:04 main:574] : INFO : Epoch 2039 | loss: 0.0311704 | val_loss: 0.0311716 | Time: 22889.7 ms [2022-03-28 19:00:28 main:574] : INFO : Epoch 2040 | loss: 0.0311688 | val_loss: 0.0311688 | Time: 23573 ms [2022-03-28 19:00:56 main:574] : INFO : Epoch 2041 | loss: 0.0311678 | val_loss: 0.031168 | Time: 28342 ms [2022-03-28 19:01:21 main:574] : INFO : Epoch 2042 | loss: 0.0311685 | val_loss: 0.0311699 | Time: 24854.6 ms [2022-03-28 19:01:47 main:574] : INFO : Epoch 2043 | loss: 0.031168 | val_loss: 0.0311677 | Time: 25504.8 ms [2022-03-28 19:02:13 main:574] : INFO : Epoch 2044 | loss: 0.0311667 | val_loss: 0.03117 | Time: 26947.6 ms [2022-03-28 19:02:40 main:574] : INFO : Epoch 2045 | loss: 0.0311684 | val_loss: 0.0311679 | Time: 26470.5 ms [2022-03-28 19:03:05 main:574] : INFO : Epoch 2046 | loss: 0.0311669 | val_loss: 0.0311683 | Time: 25294.8 ms [2022-03-28 19:03:31 main:574] : INFO : Epoch 2047 | loss: 0.0311665 | val_loss: 0.031167 | Time: 25295.4 ms [2022-03-28 19:03:56 main:574] : INFO : Epoch 2048 | loss: 0.0311662 | val_loss: 0.0311679 | Time: 25569.8 ms [2022-03-28 19:03:56 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0311679 [2022-03-28 19:03:56 main:603] : INFO : Saving end state to config to file [2022-03-28 19:03:56 main:608] : INFO : Success, exiting.. 19:03:56 (21332): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)