Task 12016931
| Name | ParityModified-1639966065-27873-3-0_0 |
| Workunit | 9206746 |
| Created | 25 Feb 2022, 3:40:03 UTC |
| Sent | 27 Feb 2022, 10:58:08 UTC |
| Report deadline | 7 Mar 2022, 10:58:08 UTC |
| Received | 29 Mar 2022, 20:24:20 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x00000000) |
| Computer ID | 11997 |
| Run time | 5 hours 35 min 28 sec |
| CPU time | 4 hours 14 min 30 sec |
| Validate state | Task was reported too late to validate |
| Credit | 0.00 |
| Device peak FLOPS | 10,510.75 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.91 GB |
| Peak swap size | 4.70 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version> <![CDATA[ <stderr_txt> ot.pt => snapshot.pt (exists = 1) [2022-03-20 16:15:52 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-20 16:15:52 main:474] : INFO : Configuration: [2022-03-20 16:15:52 main:475] : INFO : Model type: GRU [2022-03-20 16:15:52 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-20 16:15:52 main:477] : INFO : Max Epochs: 2048 [2022-03-20 16:15:52 main:478] : INFO : Batch Size: 128 [2022-03-20 16:15:52 main:479] : INFO : Learning Rate: 0.01 [2022-03-20 16:15:52 main:480] : INFO : Patience: 10 [2022-03-20 16:15:52 main:481] : INFO : Hidden Width: 12 [2022-03-20 16:15:52 main:482] : INFO : # Recurrent Layers: 4 [2022-03-20 16:15:52 main:483] : INFO : # Backend Layers: 4 [2022-03-20 16:15:52 main:484] : INFO : # Threads: 1 [2022-03-20 16:15:52 main:486] : INFO : Preparing Dataset [2022-03-20 16:15:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-20 16:15:53 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-20 16:59:31 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-20 16:59:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-20 16:59:31 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-20 16:59:31 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-20 16:59:32 main:494] : INFO : Creating Model [2022-03-20 16:59:32 main:507] : INFO : Preparing config file [2022-03-20 16:59:32 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-20 16:59:32 main:512] : INFO : Loading config [2022-03-20 16:59:32 main:514] : INFO : Loading state Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-20 22:33:11 main:435] : INFO : Set logging level to 1 [2022-03-20 22:33:11 main:441] : INFO : Running in BOINC Client mode [2022-03-20 22:33:11 main:444] : INFO : Resolving all filenames [2022-03-20 22:33:11 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-20 22:33:11 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-20 22:33:11 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-20 22:33:11 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-20 22:33:11 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-20 22:33:11 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-20 22:33:11 main:474] : INFO : Configuration: [2022-03-20 22:33:11 main:475] : INFO : Model type: GRU [2022-03-20 22:33:11 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-20 22:33:11 main:477] : INFO : Max Epochs: 2048 [2022-03-20 22:33:11 main:478] : INFO : Batch Size: 128 [2022-03-20 22:33:11 main:479] : INFO : Learning Rate: 0.01 [2022-03-20 22:33:11 main:480] : INFO : Patience: 10 [2022-03-20 22:33:11 main:481] : INFO : Hidden Width: 12 [2022-03-20 22:33:11 main:482] : INFO : # Recurrent Layers: 4 [2022-03-20 22:33:11 main:483] : INFO : # Backend Layers: 4 [2022-03-20 22:33:11 main:484] : INFO : # Threads: 1 [2022-03-20 22:33:11 main:486] : INFO : Preparing Dataset [2022-03-20 22:33:11 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-20 22:33:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-20 23:20:07 main:435] : INFO : Set logging level to 1 [2022-03-20 23:20:07 main:441] : INFO : Running in BOINC Client mode [2022-03-20 23:20:07 main:444] : INFO : Resolving all filenames [2022-03-20 23:20:07 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-20 23:20:07 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-20 23:20:07 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-20 23:20:07 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-20 23:20:07 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-20 23:20:07 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-20 23:20:07 main:474] : INFO : Configuration: [2022-03-20 23:20:07 main:475] : INFO : Model type: GRU [2022-03-20 23:20:07 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-20 23:20:07 main:477] : INFO : Max Epochs: 2048 [2022-03-20 23:20:07 main:478] : INFO : Batch Size: 128 [2022-03-20 23:20:07 main:479] : INFO : Learning Rate: 0.01 [2022-03-20 23:20:07 main:480] : INFO : Patience: 10 [2022-03-20 23:20:07 main:481] : INFO : Hidden Width: 12 [2022-03-20 23:20:07 main:482] : INFO : # Recurrent Layers: 4 [2022-03-20 23:20:07 main:483] : INFO : # Backend Layers: 4 [2022-03-20 23:20:07 main:484] : INFO : # Threads: 1 [2022-03-20 23:20:07 main:486] : INFO : Preparing Dataset [2022-03-20 23:20:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-20 23:20:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-21 00:06:55 main:435] : INFO : Set logging level to 1 [2022-03-21 00:06:55 main:441] : INFO : Running in BOINC Client mode [2022-03-21 00:06:55 main:444] : INFO : Resolving all filenames [2022-03-21 00:06:55 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-21 00:06:55 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-21 00:06:55 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-21 00:06:55 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-21 00:06:55 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-21 00:06:55 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-21 00:06:55 main:474] : INFO : Configuration: [2022-03-21 00:06:55 main:475] : INFO : Model type: GRU [2022-03-21 00:06:55 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-21 00:06:55 main:477] : INFO : Max Epochs: 2048 [2022-03-21 00:06:55 main:478] : INFO : Batch Size: 128 [2022-03-21 00:06:55 main:479] : INFO : Learning Rate: 0.01 [2022-03-21 00:06:55 main:480] : INFO : Patience: 10 [2022-03-21 00:06:55 main:481] : INFO : Hidden Width: 12 [2022-03-21 00:06:55 main:482] : INFO : # Recurrent Layers: 4 [2022-03-21 00:06:55 main:483] : INFO : # Backend Layers: 4 [2022-03-21 00:06:55 main:484] : INFO : # Threads: 1 [2022-03-21 00:06:55 main:486] : INFO : Preparing Dataset [2022-03-21 00:06:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-21 00:06:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-21 13:32:17 main:435] : INFO : Set logging level to 1 [2022-03-21 13:32:17 main:441] : INFO : Running in BOINC Client mode [2022-03-21 13:32:17 main:444] : INFO : Resolving all filenames [2022-03-21 13:32:17 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-21 13:32:17 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-21 13:32:17 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-21 13:32:17 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-21 13:32:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-21 13:32:17 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-21 13:32:17 main:474] : INFO : Configuration: [2022-03-21 13:32:17 main:475] : INFO : Model type: GRU [2022-03-21 13:32:17 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-21 13:32:17 main:477] : INFO : Max Epochs: 2048 [2022-03-21 13:32:17 main:478] : INFO : Batch Size: 128 [2022-03-21 13:32:17 main:479] : INFO : Learning Rate: 0.01 [2022-03-21 13:32:17 main:480] : INFO : Patience: 10 [2022-03-21 13:32:17 main:481] : INFO : Hidden Width: 12 [2022-03-21 13:32:17 main:482] : INFO : # Recurrent Layers: 4 [2022-03-21 13:32:17 main:483] : INFO : # Backend Layers: 4 [2022-03-21 13:32:17 main:484] : INFO : # Threads: 1 [2022-03-21 13:32:17 main:486] : INFO : Preparing Dataset [2022-03-21 13:32:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-21 13:32:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 17:25:43 main:435] : INFO : Set logging level to 1 [2022-03-22 17:25:43 main:441] : INFO : Running in BOINC Client mode [2022-03-22 17:25:43 main:444] : INFO : Resolving all filenames [2022-03-22 17:25:43 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 17:25:43 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 17:25:43 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 17:25:43 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 17:25:43 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 17:25:43 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 17:25:43 main:474] : INFO : Configuration: [2022-03-22 17:25:43 main:475] : INFO : Model type: GRU [2022-03-22 17:25:43 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 17:25:43 main:477] : INFO : Max Epochs: 2048 [2022-03-22 17:25:43 main:478] : INFO : Batch Size: 128 [2022-03-22 17:25:43 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 17:25:43 main:480] : INFO : Patience: 10 [2022-03-22 17:25:43 main:481] : INFO : Hidden Width: 12 [2022-03-22 17:25:43 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 17:25:43 main:483] : INFO : # Backend Layers: 4 [2022-03-22 17:25:43 main:484] : INFO : # Threads: 1 [2022-03-22 17:25:43 main:486] : INFO : Preparing Dataset [2022-03-22 17:25:43 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 17:25:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 18:22:54 main:435] : INFO : Set logging level to 1 [2022-03-22 18:22:54 main:441] : INFO : Running in BOINC Client mode [2022-03-22 18:22:54 main:444] : INFO : Resolving all filenames [2022-03-22 18:22:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 18:22:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 18:22:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 18:22:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 18:22:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 18:22:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 18:22:54 main:474] : INFO : Configuration: [2022-03-22 18:22:54 main:475] : INFO : Model type: GRU [2022-03-22 18:22:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 18:22:54 main:477] : INFO : Max Epochs: 2048 [2022-03-22 18:22:54 main:478] : INFO : Batch Size: 128 [2022-03-22 18:22:54 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 18:22:54 main:480] : INFO : Patience: 10 [2022-03-22 18:22:54 main:481] : INFO : Hidden Width: 12 [2022-03-22 18:22:54 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 18:22:54 main:483] : INFO : # Backend Layers: 4 [2022-03-22 18:22:54 main:484] : INFO : # Threads: 1 [2022-03-22 18:22:54 main:486] : INFO : Preparing Dataset [2022-03-22 18:22:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 18:22:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 18:33:05 main:435] : INFO : Set logging level to 1 [2022-03-22 18:33:05 main:441] : INFO : Running in BOINC Client mode [2022-03-22 18:33:05 main:444] : INFO : Resolving all filenames [2022-03-22 18:33:05 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 18:33:05 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 18:33:05 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 18:33:05 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 18:33:05 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 18:33:05 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 18:33:05 main:474] : INFO : Configuration: [2022-03-22 18:33:05 main:475] : INFO : Model type: GRU [2022-03-22 18:33:05 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 18:33:05 main:477] : INFO : Max Epochs: 2048 [2022-03-22 18:33:05 main:478] : INFO : Batch Size: 128 [2022-03-22 18:33:05 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 18:33:05 main:480] : INFO : Patience: 10 [2022-03-22 18:33:05 main:481] : INFO : Hidden Width: 12 [2022-03-22 18:33:05 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 18:33:05 main:483] : INFO : # Backend Layers: 4 [2022-03-22 18:33:05 main:484] : INFO : # Threads: 1 [2022-03-22 18:33:05 main:486] : INFO : Preparing Dataset [2022-03-22 18:33:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 18:33:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-22 19:26:59 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-22 19:26:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-22 19:27:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-22 19:27:00 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-22 19:27:00 main:494] : INFO : Creating Model [2022-03-22 19:27:00 main:507] : INFO : Preparing config file [2022-03-22 19:27:00 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-22 19:27:00 main:512] : INFO : Loading config [2022-03-22 19:27:00 main:514] : INFO : Loading state Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 20:27:03 main:435] : INFO : Set logging level to 1 [2022-03-22 20:27:03 main:441] : INFO : Running in BOINC Client mode [2022-03-22 20:27:03 main:444] : INFO : Resolving all filenames [2022-03-22 20:27:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 20:27:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 20:27:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 20:27:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 20:27:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 20:27:03 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 20:27:03 main:474] : INFO : Configuration: [2022-03-22 20:27:03 main:475] : INFO : Model type: GRU [2022-03-22 20:27:03 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 20:27:03 main:477] : INFO : Max Epochs: 2048 [2022-03-22 20:27:03 main:478] : INFO : Batch Size: 128 [2022-03-22 20:27:03 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 20:27:03 main:480] : INFO : Patience: 10 [2022-03-22 20:27:03 main:481] : INFO : Hidden Width: 12 [2022-03-22 20:27:03 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 20:27:03 main:483] : INFO : # Backend Layers: 4 [2022-03-22 20:27:03 main:484] : INFO : # Threads: 1 [2022-03-22 20:27:03 main:486] : INFO : Preparing Dataset [2022-03-22 20:27:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 20:27:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 20:45:08 main:435] : INFO : Set logging level to 1 [2022-03-22 20:45:08 main:441] : INFO : Running in BOINC Client mode [2022-03-22 20:45:08 main:444] : INFO : Resolving all filenames [2022-03-22 20:45:08 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 20:45:08 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 20:45:08 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 20:45:08 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 20:45:08 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 20:45:08 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 20:45:08 main:474] : INFO : Configuration: [2022-03-22 20:45:08 main:475] : INFO : Model type: GRU [2022-03-22 20:45:08 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 20:45:08 main:477] : INFO : Max Epochs: 2048 [2022-03-22 20:45:08 main:478] : INFO : Batch Size: 128 [2022-03-22 20:45:08 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 20:45:08 main:480] : INFO : Patience: 10 [2022-03-22 20:45:08 main:481] : INFO : Hidden Width: 12 [2022-03-22 20:45:08 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 20:45:08 main:483] : INFO : # Backend Layers: 4 [2022-03-22 20:45:08 main:484] : INFO : # Threads: 1 [2022-03-22 20:45:08 main:486] : INFO : Preparing Dataset [2022-03-22 20:45:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 20:45:09 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-22 22:32:26 main:435] : INFO : Set logging level to 1 [2022-03-22 22:32:26 main:441] : INFO : Running in BOINC Client mode [2022-03-22 22:32:26 main:444] : INFO : Resolving all filenames [2022-03-22 22:32:26 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-22 22:32:26 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-22 22:32:26 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-22 22:32:26 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-22 22:32:26 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-22 22:32:26 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-22 22:32:26 main:474] : INFO : Configuration: [2022-03-22 22:32:26 main:475] : INFO : Model type: GRU [2022-03-22 22:32:26 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-22 22:32:26 main:477] : INFO : Max Epochs: 2048 [2022-03-22 22:32:26 main:478] : INFO : Batch Size: 128 [2022-03-22 22:32:26 main:479] : INFO : Learning Rate: 0.01 [2022-03-22 22:32:26 main:480] : INFO : Patience: 10 [2022-03-22 22:32:26 main:481] : INFO : Hidden Width: 12 [2022-03-22 22:32:26 main:482] : INFO : # Recurrent Layers: 4 [2022-03-22 22:32:26 main:483] : INFO : # Backend Layers: 4 [2022-03-22 22:32:26 main:484] : INFO : # Threads: 1 [2022-03-22 22:32:26 main:486] : INFO : Preparing Dataset [2022-03-22 22:32:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-22 22:32:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-23 20:15:23 main:435] : INFO : Set logging level to 1 [2022-03-23 20:15:23 main:441] : INFO : Running in BOINC Client mode [2022-03-23 20:15:23 main:444] : INFO : Resolving all filenames [2022-03-23 20:15:23 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 20:15:23 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 20:15:23 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 20:15:23 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 20:15:23 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 20:15:23 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 20:15:23 main:474] : INFO : Configuration: [2022-03-23 20:15:23 main:475] : INFO : Model type: GRU [2022-03-23 20:15:23 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 20:15:23 main:477] : INFO : Max Epochs: 2048 [2022-03-23 20:15:23 main:478] : INFO : Batch Size: 128 [2022-03-23 20:15:23 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 20:15:23 main:480] : INFO : Patience: 10 [2022-03-23 20:15:23 main:481] : INFO : Hidden Width: 12 [2022-03-23 20:15:23 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 20:15:23 main:483] : INFO : # Backend Layers: 4 [2022-03-23 20:15:23 main:484] : INFO : # Threads: 1 [2022-03-23 20:15:23 main:486] : INFO : Preparing Dataset [2022-03-23 20:15:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 20:15:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-23 22:47:50 main:435] : INFO : Set logging level to 1 [2022-03-23 22:47:50 main:441] : INFO : Running in BOINC Client mode [2022-03-23 22:47:50 main:444] : INFO : Resolving all filenames [2022-03-23 22:47:50 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 22:47:50 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 22:47:50 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 22:47:50 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 22:47:50 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 22:47:50 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 22:47:50 main:474] : INFO : Configuration: [2022-03-23 22:47:50 main:475] : INFO : Model type: GRU [2022-03-23 22:47:50 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 22:47:50 main:477] : INFO : Max Epochs: 2048 [2022-03-23 22:47:50 main:478] : INFO : Batch Size: 128 [2022-03-23 22:47:50 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 22:47:50 main:480] : INFO : Patience: 10 [2022-03-23 22:47:50 main:481] : INFO : Hidden Width: 12 [2022-03-23 22:47:50 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 22:47:50 main:483] : INFO : # Backend Layers: 4 [2022-03-23 22:47:50 main:484] : INFO : # Threads: 1 [2022-03-23 22:47:50 main:486] : INFO : Preparing Dataset [2022-03-23 22:47:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 22:47:50 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-23 22:58:00 main:435] : INFO : Set logging level to 1 [2022-03-23 22:58:00 main:441] : INFO : Running in BOINC Client mode [2022-03-23 22:58:00 main:444] : INFO : Resolving all filenames [2022-03-23 22:58:00 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-23 22:58:00 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-23 22:58:00 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-23 22:58:00 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-23 22:58:00 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-23 22:58:00 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-23 22:58:00 main:474] : INFO : Configuration: [2022-03-23 22:58:00 main:475] : INFO : Model type: GRU [2022-03-23 22:58:00 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-23 22:58:00 main:477] : INFO : Max Epochs: 2048 [2022-03-23 22:58:00 main:478] : INFO : Batch Size: 128 [2022-03-23 22:58:00 main:479] : INFO : Learning Rate: 0.01 [2022-03-23 22:58:00 main:480] : INFO : Patience: 10 [2022-03-23 22:58:00 main:481] : INFO : Hidden Width: 12 [2022-03-23 22:58:00 main:482] : INFO : # Recurrent Layers: 4 [2022-03-23 22:58:00 main:483] : INFO : # Backend Layers: 4 [2022-03-23 22:58:00 main:484] : INFO : # Threads: 1 [2022-03-23 22:58:00 main:486] : INFO : Preparing Dataset [2022-03-23 22:58:00 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-23 22:58:01 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-24 15:57:12 main:435] : INFO : Set logging level to 1 [2022-03-24 15:57:12 main:441] : INFO : Running in BOINC Client mode [2022-03-24 15:57:12 main:444] : INFO : Resolving all filenames [2022-03-24 15:57:12 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-24 15:57:12 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-24 15:57:12 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-24 15:57:12 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-24 15:57:12 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-24 15:57:12 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-24 15:57:12 main:474] : INFO : Configuration: [2022-03-24 15:57:12 main:475] : INFO : Model type: GRU [2022-03-24 15:57:12 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-24 15:57:12 main:477] : INFO : Max Epochs: 2048 [2022-03-24 15:57:12 main:478] : INFO : Batch Size: 128 [2022-03-24 15:57:12 main:479] : INFO : Learning Rate: 0.01 [2022-03-24 15:57:12 main:480] : INFO : Patience: 10 [2022-03-24 15:57:12 main:481] : INFO : Hidden Width: 12 [2022-03-24 15:57:12 main:482] : INFO : # Recurrent Layers: 4 [2022-03-24 15:57:12 main:483] : INFO : # Backend Layers: 4 [2022-03-24 15:57:12 main:484] : INFO : # Threads: 1 [2022-03-24 15:57:12 main:486] : INFO : Preparing Dataset [2022-03-24 15:57:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-24 15:57:13 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-25 15:32:54 main:435] : INFO : Set logging level to 1 [2022-03-25 15:32:54 main:441] : INFO : Running in BOINC Client mode [2022-03-25 15:32:54 main:444] : INFO : Resolving all filenames [2022-03-25 15:32:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-25 15:32:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-25 15:32:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-25 15:32:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-25 15:32:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-25 15:32:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-25 15:32:54 main:474] : INFO : Configuration: [2022-03-25 15:32:54 main:475] : INFO : Model type: GRU [2022-03-25 15:32:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-25 15:32:54 main:477] : INFO : Max Epochs: 2048 [2022-03-25 15:32:54 main:478] : INFO : Batch Size: 128 [2022-03-25 15:32:54 main:479] : INFO : Learning Rate: 0.01 [2022-03-25 15:32:54 main:480] : INFO : Patience: 10 [2022-03-25 15:32:54 main:481] : INFO : Hidden Width: 12 [2022-03-25 15:32:54 main:482] : INFO : # Recurrent Layers: 4 [2022-03-25 15:32:54 main:483] : INFO : # Backend Layers: 4 [2022-03-25 15:32:54 main:484] : INFO : # Threads: 1 [2022-03-25 15:32:54 main:486] : INFO : Preparing Dataset [2022-03-25 15:32:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-25 15:32:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-25 17:02:38 main:435] : INFO : Set logging level to 1 [2022-03-25 17:02:38 main:441] : INFO : Running in BOINC Client mode [2022-03-25 17:02:38 main:444] : INFO : Resolving all filenames [2022-03-25 17:02:38 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-25 17:02:38 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-25 17:02:38 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-25 17:02:38 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-25 17:02:38 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-25 17:02:38 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-25 17:02:38 main:474] : INFO : Configuration: [2022-03-25 17:02:38 main:475] : INFO : Model type: GRU [2022-03-25 17:02:38 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-25 17:02:38 main:477] : INFO : Max Epochs: 2048 [2022-03-25 17:02:38 main:478] : INFO : Batch Size: 128 [2022-03-25 17:02:38 main:479] : INFO : Learning Rate: 0.01 [2022-03-25 17:02:38 main:480] : INFO : Patience: 10 [2022-03-25 17:02:38 main:481] : INFO : Hidden Width: 12 [2022-03-25 17:02:38 main:482] : INFO : # Recurrent Layers: 4 [2022-03-25 17:02:38 main:483] : INFO : # Backend Layers: 4 [2022-03-25 17:02:38 main:484] : INFO : # Threads: 1 [2022-03-25 17:02:38 main:486] : INFO : Preparing Dataset [2022-03-25 17:02:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-25 17:02:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-26 14:49:16 main:435] : INFO : Set logging level to 1 [2022-03-26 14:49:16 main:441] : INFO : Running in BOINC Client mode [2022-03-26 14:49:16 main:444] : INFO : Resolving all filenames [2022-03-26 14:49:16 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-26 14:49:16 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-26 14:49:16 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-26 14:49:16 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-26 14:49:16 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-26 14:49:16 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-26 14:49:16 main:474] : INFO : Configuration: [2022-03-26 14:49:16 main:475] : INFO : Model type: GRU [2022-03-26 14:49:16 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-26 14:49:16 main:477] : INFO : Max Epochs: 2048 [2022-03-26 14:49:16 main:478] : INFO : Batch Size: 128 [2022-03-26 14:49:16 main:479] : INFO : Learning Rate: 0.01 [2022-03-26 14:49:16 main:480] : INFO : Patience: 10 [2022-03-26 14:49:16 main:481] : INFO : Hidden Width: 12 [2022-03-26 14:49:16 main:482] : INFO : # Recurrent Layers: 4 [2022-03-26 14:49:16 main:483] : INFO : # Backend Layers: 4 [2022-03-26 14:49:16 main:484] : INFO : # Threads: 1 [2022-03-26 14:49:16 main:486] : INFO : Preparing Dataset [2022-03-26 14:49:16 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-26 14:49:17 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-26 16:57:38 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-26 16:57:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-26 16:57:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-26 16:57:38 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-26 16:57:38 main:494] : INFO : Creating Model [2022-03-26 16:57:38 main:507] : INFO : Preparing config file [2022-03-26 16:57:38 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-26 16:57:38 main:512] : INFO : Loading config [2022-03-26 16:57:38 main:514] : INFO : Loading state Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-26 23:27:32 main:435] : INFO : Set logging level to 1 [2022-03-26 23:27:32 main:441] : INFO : Running in BOINC Client mode [2022-03-26 23:27:32 main:444] : INFO : Resolving all filenames [2022-03-26 23:27:32 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-26 23:27:32 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-26 23:27:32 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-26 23:27:32 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-26 23:27:32 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-26 23:27:32 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-26 23:27:32 main:474] : INFO : Configuration: [2022-03-26 23:27:32 main:475] : INFO : Model type: GRU [2022-03-26 23:27:32 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-26 23:27:32 main:477] : INFO : Max Epochs: 2048 [2022-03-26 23:27:32 main:478] : INFO : Batch Size: 128 [2022-03-26 23:27:32 main:479] : INFO : Learning Rate: 0.01 [2022-03-26 23:27:32 main:480] : INFO : Patience: 10 [2022-03-26 23:27:32 main:481] : INFO : Hidden Width: 12 [2022-03-26 23:27:32 main:482] : INFO : # Recurrent Layers: 4 [2022-03-26 23:27:32 main:483] : INFO : # Backend Layers: 4 [2022-03-26 23:27:32 main:484] : INFO : # Threads: 1 [2022-03-26 23:27:32 main:486] : INFO : Preparing Dataset [2022-03-26 23:27:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-26 23:27:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-27 13:58:17 main:435] : INFO : Set logging level to 1 [2022-03-27 13:58:17 main:441] : INFO : Running in BOINC Client mode [2022-03-27 13:58:17 main:444] : INFO : Resolving all filenames [2022-03-27 13:58:17 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-27 13:58:17 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-27 13:58:17 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-27 13:58:17 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-27 13:58:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-27 13:58:17 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-27 13:58:17 main:474] : INFO : Configuration: [2022-03-27 13:58:17 main:475] : INFO : Model type: GRU [2022-03-27 13:58:17 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-27 13:58:18 main:477] : INFO : Max Epochs: 2048 [2022-03-27 13:58:18 main:478] : INFO : Batch Size: 128 [2022-03-27 13:58:18 main:479] : INFO : Learning Rate: 0.01 [2022-03-27 13:58:18 main:480] : INFO : Patience: 10 [2022-03-27 13:58:18 main:481] : INFO : Hidden Width: 12 [2022-03-27 13:58:18 main:482] : INFO : # Recurrent Layers: 4 [2022-03-27 13:58:18 main:483] : INFO : # Backend Layers: 4 [2022-03-27 13:58:18 main:484] : INFO : # Threads: 1 [2022-03-27 13:58:18 main:486] : INFO : Preparing Dataset [2022-03-27 13:58:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-27 13:58:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-27 16:20:35 main:435] : INFO : Set logging level to 1 [2022-03-27 16:20:35 main:441] : INFO : Running in BOINC Client mode [2022-03-27 16:20:35 main:444] : INFO : Resolving all filenames [2022-03-27 16:20:35 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-27 16:20:35 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-27 16:20:35 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-27 16:20:35 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-27 16:20:35 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-27 16:20:35 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-27 16:20:35 main:474] : INFO : Configuration: [2022-03-27 16:20:35 main:475] : INFO : Model type: GRU [2022-03-27 16:20:35 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-27 16:20:35 main:477] : INFO : Max Epochs: 2048 [2022-03-27 16:20:35 main:478] : INFO : Batch Size: 128 [2022-03-27 16:20:35 main:479] : INFO : Learning Rate: 0.01 [2022-03-27 16:20:35 main:480] : INFO : Patience: 10 [2022-03-27 16:20:35 main:481] : INFO : Hidden Width: 12 [2022-03-27 16:20:35 main:482] : INFO : # Recurrent Layers: 4 [2022-03-27 16:20:35 main:483] : INFO : # Backend Layers: 4 [2022-03-27 16:20:35 main:484] : INFO : # Threads: 1 [2022-03-27 16:20:35 main:486] : INFO : Preparing Dataset [2022-03-27 16:20:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-27 16:20:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-27 22:17:06 main:435] : INFO : Set logging level to 1 [2022-03-27 22:17:06 main:441] : INFO : Running in BOINC Client mode [2022-03-27 22:17:06 main:444] : INFO : Resolving all filenames [2022-03-27 22:17:06 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-27 22:17:06 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-27 22:17:06 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-27 22:17:06 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-27 22:17:06 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-27 22:17:06 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-27 22:17:06 main:474] : INFO : Configuration: [2022-03-27 22:17:06 main:475] : INFO : Model type: GRU [2022-03-27 22:17:06 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-27 22:17:06 main:477] : INFO : Max Epochs: 2048 [2022-03-27 22:17:06 main:478] : INFO : Batch Size: 128 [2022-03-27 22:17:06 main:479] : INFO : Learning Rate: 0.01 [2022-03-27 22:17:06 main:480] : INFO : Patience: 10 [2022-03-27 22:17:06 main:481] : INFO : Hidden Width: 12 [2022-03-27 22:17:06 main:482] : INFO : # Recurrent Layers: 4 [2022-03-27 22:17:06 main:483] : INFO : # Backend Layers: 4 [2022-03-27 22:17:06 main:484] : INFO : # Threads: 1 [2022-03-27 22:17:06 main:486] : INFO : Preparing Dataset [2022-03-27 22:17:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-27 22:17:07 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-28 15:30:55 main:435] : INFO : Set logging level to 1 [2022-03-28 15:30:55 main:441] : INFO : Running in BOINC Client mode [2022-03-28 15:30:55 main:444] : INFO : Resolving all filenames [2022-03-28 15:30:55 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 15:30:55 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 15:30:55 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 15:30:55 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 15:30:56 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 15:30:56 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 15:30:56 main:474] : INFO : Configuration: [2022-03-28 15:30:56 main:475] : INFO : Model type: GRU [2022-03-28 15:30:56 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 15:30:56 main:477] : INFO : Max Epochs: 2048 [2022-03-28 15:30:56 main:478] : INFO : Batch Size: 128 [2022-03-28 15:30:56 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 15:30:56 main:480] : INFO : Patience: 10 [2022-03-28 15:30:56 main:481] : INFO : Hidden Width: 12 [2022-03-28 15:30:56 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 15:30:56 main:483] : INFO : # Backend Layers: 4 [2022-03-28 15:30:56 main:484] : INFO : # Threads: 1 [2022-03-28 15:30:56 main:486] : INFO : Preparing Dataset [2022-03-28 15:30:56 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 15:30:56 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-28 16:15:04 main:435] : INFO : Set logging level to 1 [2022-03-28 16:15:04 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:15:04 main:444] : INFO : Resolving all filenames [2022-03-28 16:15:04 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:15:04 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:15:04 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:15:04 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:15:04 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:15:04 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:15:04 main:474] : INFO : Configuration: [2022-03-28 16:15:04 main:475] : INFO : Model type: GRU [2022-03-28 16:15:04 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:15:05 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:15:05 main:478] : INFO : Batch Size: 128 [2022-03-28 16:15:05 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:15:05 main:480] : INFO : Patience: 10 [2022-03-28 16:15:05 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:15:05 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:15:05 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:15:05 main:484] : INFO : # Threads: 1 [2022-03-28 16:15:05 main:486] : INFO : Preparing Dataset [2022-03-28 16:15:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:15:05 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-28 16:25:14 main:435] : INFO : Set logging level to 1 [2022-03-28 16:25:14 main:441] : INFO : Running in BOINC Client mode [2022-03-28 16:25:14 main:444] : INFO : Resolving all filenames [2022-03-28 16:25:14 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-28 16:25:14 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-28 16:25:14 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-28 16:25:14 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-28 16:25:14 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-28 16:25:14 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-28 16:25:14 main:474] : INFO : Configuration: [2022-03-28 16:25:14 main:475] : INFO : Model type: GRU [2022-03-28 16:25:14 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-28 16:25:14 main:477] : INFO : Max Epochs: 2048 [2022-03-28 16:25:14 main:478] : INFO : Batch Size: 128 [2022-03-28 16:25:14 main:479] : INFO : Learning Rate: 0.01 [2022-03-28 16:25:14 main:480] : INFO : Patience: 10 [2022-03-28 16:25:14 main:481] : INFO : Hidden Width: 12 [2022-03-28 16:25:14 main:482] : INFO : # Recurrent Layers: 4 [2022-03-28 16:25:14 main:483] : INFO : # Backend Layers: 4 [2022-03-28 16:25:14 main:484] : INFO : # Threads: 1 [2022-03-28 16:25:14 main:486] : INFO : Preparing Dataset [2022-03-28 16:25:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-28 16:25:14 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-29 11:34:54 main:435] : INFO : Set logging level to 1 [2022-03-29 11:34:54 main:441] : INFO : Running in BOINC Client mode [2022-03-29 11:34:54 main:444] : INFO : Resolving all filenames [2022-03-29 11:34:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-29 11:34:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-29 11:34:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-29 11:34:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-29 11:34:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-29 11:34:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-29 11:34:54 main:474] : INFO : Configuration: [2022-03-29 11:34:54 main:475] : INFO : Model type: GRU [2022-03-29 11:34:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-29 11:34:54 main:477] : INFO : Max Epochs: 2048 [2022-03-29 11:34:54 main:478] : INFO : Batch Size: 128 [2022-03-29 11:34:54 main:479] : INFO : Learning Rate: 0.01 [2022-03-29 11:34:54 main:480] : INFO : Patience: 10 [2022-03-29 11:34:54 main:481] : INFO : Hidden Width: 12 [2022-03-29 11:34:54 main:482] : INFO : # Recurrent Layers: 4 [2022-03-29 11:34:54 main:483] : INFO : # Backend Layers: 4 [2022-03-29 11:34:54 main:484] : INFO : # Threads: 1 [2022-03-29 11:34:54 main:486] : INFO : Preparing Dataset [2022-03-29 11:34:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-29 11:34:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-29 20:57:54 main:435] : INFO : Set logging level to 1 [2022-03-29 20:57:54 main:441] : INFO : Running in BOINC Client mode [2022-03-29 20:57:54 main:444] : INFO : Resolving all filenames [2022-03-29 20:57:54 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-29 20:57:54 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-29 20:57:54 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-29 20:57:54 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-29 20:57:54 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-29 20:57:54 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-29 20:57:54 main:474] : INFO : Configuration: [2022-03-29 20:57:54 main:475] : INFO : Model type: GRU [2022-03-29 20:57:54 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-29 20:57:54 main:477] : INFO : Max Epochs: 2048 [2022-03-29 20:57:54 main:478] : INFO : Batch Size: 128 [2022-03-29 20:57:54 main:479] : INFO : Learning Rate: 0.01 [2022-03-29 20:57:54 main:480] : INFO : Patience: 10 [2022-03-29 20:57:54 main:481] : INFO : Hidden Width: 12 [2022-03-29 20:57:54 main:482] : INFO : # Recurrent Layers: 4 [2022-03-29 20:57:54 main:483] : INFO : # Backend Layers: 4 [2022-03-29 20:57:54 main:484] : INFO : # Threads: 1 [2022-03-29 20:57:54 main:486] : INFO : Preparing Dataset [2022-03-29 20:57:54 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-29 20:57:55 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU) [2022-03-29 21:08:03 main:435] : INFO : Set logging level to 1 [2022-03-29 21:08:03 main:441] : INFO : Running in BOINC Client mode [2022-03-29 21:08:03 main:444] : INFO : Resolving all filenames [2022-03-29 21:08:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1) [2022-03-29 21:08:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1) [2022-03-29 21:08:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0) [2022-03-29 21:08:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1) [2022-03-29 21:08:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1) [2022-03-29 21:08:03 main:472] : INFO : Dataset filename: dataset.hdf5 [2022-03-29 21:08:03 main:474] : INFO : Configuration: [2022-03-29 21:08:03 main:475] : INFO : Model type: GRU [2022-03-29 21:08:03 main:476] : INFO : Validation Loss Threshold: 0.0001 [2022-03-29 21:08:03 main:477] : INFO : Max Epochs: 2048 [2022-03-29 21:08:03 main:478] : INFO : Batch Size: 128 [2022-03-29 21:08:03 main:479] : INFO : Learning Rate: 0.01 [2022-03-29 21:08:03 main:480] : INFO : Patience: 10 [2022-03-29 21:08:03 main:481] : INFO : Hidden Width: 12 [2022-03-29 21:08:03 main:482] : INFO : # Recurrent Layers: 4 [2022-03-29 21:08:03 main:483] : INFO : # Backend Layers: 4 [2022-03-29 21:08:03 main:484] : INFO : # Threads: 1 [2022-03-29 21:08:03 main:486] : INFO : Preparing Dataset [2022-03-29 21:08:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory [2022-03-29 21:08:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory [2022-03-29 21:44:12 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory. [2022-03-29 21:44:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory [2022-03-29 21:44:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory [2022-03-29 21:44:12 load:106] : INFO : Successfully loaded dataset of 512 examples into memory. [2022-03-29 21:44:12 main:494] : INFO : Creating Model [2022-03-29 21:44:13 main:507] : INFO : Preparing config file [2022-03-29 21:44:13 main:511] : INFO : Found checkpoint, attempting to load... [2022-03-29 21:44:13 main:512] : INFO : Loading config [2022-03-29 21:44:13 main:514] : INFO : Loading state [2022-03-29 22:20:57 main:559] : INFO : Loading DataLoader into Memory [2022-03-29 22:20:57 main:562] : INFO : Starting Training [2022-03-29 22:23:56 main:574] : INFO : Epoch 2042 | loss: 0.0312549 | val_loss: 0.0312202 | Time: 178653 ms [2022-03-29 22:23:59 main:574] : INFO : Epoch 2043 | loss: 0.0312186 | val_loss: 0.0312169 | Time: 2515.87 ms [2022-03-29 22:24:01 main:574] : INFO : Epoch 2044 | loss: 0.0312166 | val_loss: 0.0312178 | Time: 2531.22 ms [2022-03-29 22:24:04 main:574] : INFO : Epoch 2045 | loss: 0.0312178 | val_loss: 0.0312152 | Time: 2458.52 ms [2022-03-29 22:24:06 main:574] : INFO : Epoch 2046 | loss: 0.0312156 | val_loss: 0.0312155 | Time: 2538.97 ms [2022-03-29 22:24:09 main:574] : INFO : Epoch 2047 | loss: 0.0312165 | val_loss: 0.0312206 | Time: 2520.26 ms [2022-03-29 22:24:11 main:574] : INFO : Epoch 2048 | loss: 0.0312175 | val_loss: 0.0312157 | Time: 2563.71 ms [2022-03-29 22:24:11 main:597] : INFO : Saving trained model to model-final.pt, val_loss 0.0312157 [2022-03-29 22:24:11 main:603] : INFO : Saving end state to config to file [2022-03-29 22:24:11 main:608] : INFO : Success, exiting.. 22:24:11 (33596): called boinc_finish(0) </stderr_txt> ]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)