Task 5243189
| Name | ParityMachine-1619583751-17130-2_3 |
| Workunit | 3124508 |
| Created | 27 May 2021, 6:25:32 UTC |
| Sent | 27 May 2021, 8:36:45 UTC |
| Report deadline | 3 Jun 2021, 8:36:45 UTC |
| Received | 27 May 2021, 10:36:32 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 1832 |
| Run time | 15 min 27 sec |
| CPU time | 6 min 58 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 884.18 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.56 GB |
| Peak swap size | 3.46 GB |
| Peak disk usage | 1.53 GB |
Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -529697949 (0xe06d7363)</message>
<stderr_txt>
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce 940MX)
[2021-05-27 08:38:16 main:435] : INFO : Set logging level to 1
[2021-05-27 08:38:16 main:441] : INFO : Running in BOINC Client mode
[2021-05-27 08:38:16 main:444] : INFO : Resolving all filenames
[2021-05-27 08:38:16 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-05-27 08:38:16 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2_3_r808142750_1 (exists = 0)
[2021-05-27 08:38:16 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2_3_r808142750_0 (exists = 0)
[2021-05-27 08:38:17 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2 (exists = 1)
[2021-05-27 08:38:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 0)
[2021-05-27 08:38:17 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-05-27 08:38:17 main:474] : INFO : Configuration:
[2021-05-27 08:38:17 main:475] : INFO : Model type: GRU
[2021-05-27 08:38:17 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-05-27 08:38:18 main:477] : INFO : Max Epochs: 2048
[2021-05-27 08:38:19 main:478] : INFO : Batch Size: 128
[2021-05-27 08:38:20 main:479] : INFO : Learning Rate: 0.01
[2021-05-27 08:38:21 main:480] : INFO : Patience: 10
[2021-05-27 08:38:23 main:481] : INFO : Hidden Width: 12
[2021-05-27 08:38:24 main:482] : INFO : # Recurrent Layers: 4
[2021-05-27 08:38:24 main:483] : INFO : # Backend Layers: 4
[2021-05-27 08:38:24 main:484] : INFO : # Threads: 1
[2021-05-27 08:38:24 main:486] : INFO : Preparing Dataset
[2021-05-27 08:38:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 08:38:26 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 08:38:40 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-05-27 08:38:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 08:38:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 08:38:41 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-05-27 08:38:41 main:494] : INFO : Creating Model
[2021-05-27 08:38:42 main:507] : INFO : Preparing config file
[2021-05-27 08:38:42 main:519] : INFO : Creating new config file
[2021-05-27 08:38:42 main:538] : INFO : This is a continuation WU, loading previous network
[2021-05-27 08:38:54 main:559] : INFO : Loading DataLoader into Memory
[2021-05-27 08:38:54 main:562] : INFO : Starting Training
[2021-05-27 08:39:05 main:574] : INFO : Epoch 1 | loss: nan | val_loss: nan | Time: 11229.9 ms
[2021-05-27 08:39:12 main:574] : INFO : Epoch 2 | loss: nan | val_loss: nan | Time: 5748.62 ms
[2021-05-27 08:39:18 main:574] : INFO : Epoch 3 | loss: nan | val_loss: nan | Time: 5392.93 ms
[2021-05-27 08:39:25 main:574] : INFO : Epoch 4 | loss: nan | val_loss: nan | Time: 5452.69 ms
[2021-05-27 08:39:31 main:574] : INFO : Epoch 5 | loss: nan | val_loss: nan | Time: 5301.05 ms
[2021-05-27 08:39:37 main:574] : INFO : Epoch 6 | loss: nan | val_loss: nan | Time: 5307.32 ms
[2021-05-27 08:39:43 main:574] : INFO : Epoch 7 | loss: nan | val_loss: nan | Time: 5492.01 ms
[2021-05-27 08:39:49 main:574] : INFO : Epoch 8 | loss: nan | val_loss: nan | Time: 5444.42 ms
[2021-05-27 08:39:58 main:574] : INFO : Epoch 9 | loss: nan | val_loss: nan | Time: 6537.86 ms
[2021-05-27 08:40:05 main:574] : INFO : Epoch 10 | loss: nan | val_loss: nan | Time: 5480.2 ms
[2021-05-27 08:40:11 main:574] : INFO : Epoch 11 | loss: nan | val_loss: nan | Time: 6606.93 ms
[2021-05-27 08:40:18 main:574] : INFO : Epoch 12 | loss: nan | val_loss: nan | Time: 5472.49 ms
[2021-05-27 08:40:24 main:574] : INFO : Epoch 13 | loss: nan | val_loss: nan | Time: 5508.98 ms
[2021-05-27 08:40:30 main:574] : INFO : Epoch 14 | loss: nan | val_loss: nan | Time: 5493.72 ms
[2021-05-27 08:40:37 main:574] : INFO : Epoch 15 | loss: nan | val_loss: nan | Time: 5522.79 ms
[2021-05-27 08:40:52 main:574] : INFO : Epoch 16 | loss: nan | val_loss: nan | Time: 13789.1 ms
[2021-05-27 08:41:00 main:574] : INFO : Epoch 17 | loss: nan | val_loss: nan | Time: 6280.6 ms
[2021-05-27 08:41:06 main:574] : INFO : Epoch 18 | loss: nan | val_loss: nan | Time: 6106.9 ms
[2021-05-27 08:41:17 main:574] : INFO : Epoch 19 | loss: nan | val_loss: nan | Time: 9833.85 ms
[2021-05-27 08:41:25 main:574] : INFO : Epoch 20 | loss: nan | val_loss: nan | Time: 6846.71 ms
[2021-05-27 08:41:32 main:574] : INFO : Epoch 21 | loss: nan | val_loss: nan | Time: 5785.6 ms
[2021-05-27 08:41:38 main:574] : INFO : Epoch 22 | loss: nan | val_loss: nan | Time: 5606.38 ms
[2021-05-27 08:41:45 main:574] : INFO : Epoch 23 | loss: nan | val_loss: nan | Time: 6704.68 ms
[2021-05-27 08:41:54 main:574] : INFO : Epoch 24 | loss: nan | val_loss: nan | Time: 6941.62 ms
[2021-05-27 08:42:01 main:574] : INFO : Epoch 25 | loss: nan | val_loss: nan | Time: 6240.93 ms
[2021-05-27 08:42:11 main:574] : INFO : Epoch 26 | loss: nan | val_loss: nan | Time: 8882.92 ms
[2021-05-27 08:42:19 main:574] : INFO : Epoch 27 | loss: nan | val_loss: nan | Time: 6846.04 ms
[2021-05-27 08:42:26 main:574] : INFO : Epoch 28 | loss: nan | val_loss: nan | Time: 5510.38 ms
[2021-05-27 08:42:32 main:574] : INFO : Epoch 29 | loss: nan | val_loss: nan | Time: 5479.67 ms
[2021-05-27 08:42:39 main:574] : INFO : Epoch 30 | loss: nan | val_loss: nan | Time: 5738.46 ms
[2021-05-27 08:42:45 main:574] : INFO : Epoch 31 | loss: nan | val_loss: nan | Time: 5673.75 ms
[2021-05-27 08:42:53 main:574] : INFO : Epoch 32 | loss: nan | val_loss: nan | Time: 7448.74 ms
[2021-05-27 08:43:01 main:574] : INFO : Epoch 33 | loss: nan | val_loss: nan | Time: 6320.56 ms
[2021-05-27 08:43:06 main:574] : INFO : Epoch 34 | loss: nan | val_loss: nan | Time: 5501.39 ms
[2021-05-27 08:43:12 main:574] : INFO : Epoch 35 | loss: nan | val_loss: nan | Time: 5432.09 ms
[2021-05-27 08:43:19 main:574] : INFO : Epoch 36 | loss: nan | val_loss: nan | Time: 5533.16 ms
[2021-05-27 08:43:26 main:574] : INFO : Epoch 37 | loss: nan | val_loss: nan | Time: 5866.76 ms
[2021-05-27 08:43:32 main:574] : INFO : Epoch 38 | loss: nan | val_loss: nan | Time: 5611.93 ms
[2021-05-27 08:43:38 main:574] : INFO : Epoch 39 | loss: nan | val_loss: nan | Time: 5460.89 ms
[2021-05-27 08:43:45 main:574] : INFO : Epoch 40 | loss: nan | val_loss: nan | Time: 5873.14 ms
[2021-05-27 08:43:51 main:574] : INFO : Epoch 41 | loss: nan | val_loss: nan | Time: 5349.37 ms
[2021-05-27 08:43:56 main:574] : INFO : Epoch 42 | loss: nan | val_loss: nan | Time: 5351.91 ms
[2021-05-27 08:44:04 main:574] : INFO : Epoch 43 | loss: nan | val_loss: nan | Time: 7856.09 ms
[2021-05-27 08:44:10 main:574] : INFO : Epoch 44 | loss: nan | val_loss: nan | Time: 5682.3 ms
[2021-05-27 08:44:16 main:574] : INFO : Epoch 45 | loss: nan | val_loss: nan | Time: 5486.88 ms
[2021-05-27 08:44:22 main:574] : INFO : Epoch 46 | loss: nan | val_loss: nan | Time: 5975.49 ms
[2021-05-27 08:44:28 main:574] : INFO : Epoch 47 | loss: nan | val_loss: nan | Time: 5848.24 ms
[2021-05-27 08:44:34 main:574] : INFO : Epoch 48 | loss: nan | val_loss: nan | Time: 5523.36 ms
[2021-05-27 08:44:40 main:574] : INFO : Epoch 49 | loss: nan | val_loss: nan | Time: 5431.29 ms
[2021-05-27 08:44:45 main:574] : INFO : Epoch 50 | loss: nan | val_loss: nan | Time: 5760.05 ms
[2021-05-27 08:44:52 main:574] : INFO : Epoch 51 | loss: nan | val_loss: nan | Time: 6682.9 ms
[2021-05-27 08:45:04 main:574] : INFO : Epoch 52 | loss: nan | val_loss: nan | Time: 10998.1 ms
[2021-05-27 08:45:10 main:574] : INFO : Epoch 53 | loss: nan | val_loss: nan | Time: 6165.27 ms
[2021-05-27 08:45:16 main:574] : INFO : Epoch 54 | loss: nan | val_loss: nan | Time: 5650.63 ms
[2021-05-27 08:45:21 main:574] : INFO : Epoch 55 | loss: nan | val_loss: nan | Time: 5310.77 ms
[2021-05-27 08:45:27 main:574] : INFO : Epoch 56 | loss: nan | val_loss: nan | Time: 5374.27 ms
[2021-05-27 08:45:34 main:574] : INFO : Epoch 57 | loss: nan | val_loss: nan | Time: 6622.43 ms
[2021-05-27 08:45:43 main:574] : INFO : Epoch 58 | loss: nan | val_loss: nan | Time: 9481.6 ms
[2021-05-27 08:45:51 main:574] : INFO : Epoch 59 | loss: nan | val_loss: nan | Time: 5690.47 ms
[2021-05-27 08:45:58 main:574] : INFO : Epoch 60 | loss: nan | val_loss: nan | Time: 6872.1 ms
[2021-05-27 08:46:04 main:574] : INFO : Epoch 61 | loss: nan | val_loss: nan | Time: 5577.59 ms
[2021-05-27 08:46:09 main:574] : INFO : Epoch 62 | loss: nan | val_loss: nan | Time: 5498.11 ms
[2021-05-27 08:46:18 main:574] : INFO : Epoch 63 | loss: nan | val_loss: nan | Time: 8367.73 ms
[2021-05-27 08:46:24 main:574] : INFO : Epoch 64 | loss: nan | val_loss: nan | Time: 5596.44 ms
[2021-05-27 08:46:30 main:574] : INFO : Epoch 65 | loss: nan | val_loss: nan | Time: 5359.98 ms
[2021-05-27 08:46:36 main:574] : INFO : Epoch 66 | loss: nan | val_loss: nan | Time: 6404.42 ms
[2021-05-27 08:46:48 main:574] : INFO : Epoch 67 | loss: nan | val_loss: nan | Time: 10601.8 ms
[2021-05-27 08:46:54 main:574] : INFO : Epoch 68 | loss: nan | val_loss: nan | Time: 5917.36 ms
[2021-05-27 08:47:00 main:574] : INFO : Epoch 69 | loss: nan | val_loss: nan | Time: 5523.53 ms
[2021-05-27 08:47:06 main:574] : INFO : Epoch 70 | loss: nan | val_loss: nan | Time: 5703.69 ms
[2021-05-27 08:47:11 main:574] : INFO : Epoch 71 | loss: nan | val_loss: nan | Time: 5453.68 ms
[2021-05-27 08:47:17 main:574] : INFO : Epoch 72 | loss: nan | val_loss: nan | Time: 5522.75 ms
[2021-05-27 08:47:23 main:574] : INFO : Epoch 73 | loss: nan | val_loss: nan | Time: 5799.01 ms
[2021-05-27 08:47:30 main:574] : INFO : Epoch 74 | loss: nan | val_loss: nan | Time: 6164.38 ms
[2021-05-27 08:47:36 main:574] : INFO : Epoch 75 | loss: nan | val_loss: nan | Time: 5757.77 ms
[2021-05-27 08:47:43 main:574] : INFO : Epoch 76 | loss: nan | val_loss: nan | Time: 6833.28 ms
[2021-05-27 08:47:50 main:574] : INFO : Epoch 77 | loss: nan | val_loss: nan | Time: 6939.58 ms
[2021-05-27 08:47:56 main:574] : INFO : Epoch 78 | loss: nan | val_loss: nan | Time: 5426.31 ms
[2021-05-27 08:48:03 main:574] : INFO : Epoch 79 | loss: nan | val_loss: nan | Time: 5354.84 ms
[2021-05-27 08:48:10 main:574] : INFO : Epoch 80 | loss: nan | val_loss: nan | Time: 5272.96 ms
[2021-05-27 08:48:18 main:574] : INFO : Epoch 81 | loss: nan | val_loss: nan | Time: 6558.19 ms
[2021-05-27 08:48:24 main:574] : INFO : Epoch 82 | loss: nan | val_loss: nan | Time: 5412.2 ms
[2021-05-27 08:48:31 main:574] : INFO : Epoch 83 | loss: nan | val_loss: nan | Time: 5387.74 ms
[2021-05-27 08:48:38 main:574] : INFO : Epoch 84 | loss: nan | val_loss: nan | Time: 5765.2 ms
[2021-05-27 08:48:45 main:574] : INFO : Epoch 85 | loss: nan | val_loss: nan | Time: 7338 ms
[2021-05-27 08:48:51 main:574] : INFO : Epoch 86 | loss: nan | val_loss: nan | Time: 5557.94 ms
[2021-05-27 08:48:57 main:574] : INFO : Epoch 87 | loss: nan | val_loss: nan | Time: 5945.9 ms
[2021-05-27 08:49:03 main:574] : INFO : Epoch 88 | loss: nan | val_loss: nan | Time: 5613.58 ms
[2021-05-27 08:49:08 main:574] : INFO : Epoch 89 | loss: nan | val_loss: nan | Time: 5432.76 ms
[2021-05-27 08:49:14 main:574] : INFO : Epoch 90 | loss: nan | val_loss: nan | Time: 5639.75 ms
[2021-05-27 08:49:20 main:574] : INFO : Epoch 91 | loss: nan | val_loss: nan | Time: 5400.96 ms
[2021-05-27 08:49:25 main:574] : INFO : Epoch 92 | loss: nan | val_loss: nan | Time: 5285.83 ms
[2021-05-27 08:49:31 main:574] : INFO : Epoch 93 | loss: nan | val_loss: nan | Time: 5544.78 ms
[2021-05-27 08:49:38 main:574] : INFO : Epoch 94 | loss: nan | val_loss: nan | Time: 5568.82 ms
[2021-05-27 08:49:43 main:574] : INFO : Epoch 95 | loss: nan | val_loss: nan | Time: 5395.16 ms
[2021-05-27 08:49:49 main:574] : INFO : Epoch 96 | loss: nan | val_loss: nan | Time: 5497.71 ms
[2021-05-27 08:49:54 main:574] : INFO : Epoch 97 | loss: nan | val_loss: nan | Time: 5346.11 ms
[2021-05-27 08:50:00 main:574] : INFO : Epoch 98 | loss: nan | val_loss: nan | Time: 5418.93 ms
[2021-05-27 08:50:07 main:574] : INFO : Epoch 99 | loss: nan | val_loss: nan | Time: 6756.56 ms
[2021-05-27 08:50:13 main:574] : INFO : Epoch 100 | loss: nan | val_loss: nan | Time: 5771.11 ms
[2021-05-27 08:50:19 main:574] : INFO : Epoch 101 | loss: nan | val_loss: nan | Time: 6094.55 ms
[2021-05-27 08:50:25 main:574] : INFO : Epoch 102 | loss: nan | val_loss: nan | Time: 5828.72 ms
[2021-05-27 08:50:32 main:574] : INFO : Epoch 103 | loss: nan | val_loss: nan | Time: 6936.15 ms
[2021-05-27 08:50:38 main:574] : INFO : Epoch 104 | loss: nan | val_loss: nan | Time: 5564.25 ms
[2021-05-27 08:50:44 main:574] : INFO : Epoch 105 | loss: nan | val_loss: nan | Time: 5590.97 ms
[2021-05-27 08:50:50 main:574] : INFO : Epoch 106 | loss: nan | val_loss: nan | Time: 5900.59 ms
[2021-05-27 08:50:56 main:574] : INFO : Epoch 107 | loss: nan | val_loss: nan | Time: 5797.45 ms
[2021-05-27 08:51:02 main:574] : INFO : Epoch 108 | loss: nan | val_loss: nan | Time: 5657.99 ms
[2021-05-27 08:51:11 main:574] : INFO : Epoch 109 | loss: nan | val_loss: nan | Time: 8630.39 ms
[2021-05-27 08:51:17 main:574] : INFO : Epoch 110 | loss: nan | val_loss: nan | Time: 5741.39 ms
[2021-05-27 08:51:23 main:574] : INFO : Epoch 111 | loss: nan | val_loss: nan | Time: 5878.88 ms
[2021-05-27 08:51:29 main:574] : INFO : Epoch 112 | loss: nan | val_loss: nan | Time: 5376.79 ms
[2021-05-27 08:51:35 main:574] : INFO : Epoch 113 | loss: nan | val_loss: nan | Time: 6458.02 ms
[2021-05-27 08:51:42 main:574] : INFO : Epoch 114 | loss: nan | val_loss: nan | Time: 6973.91 ms
[2021-05-27 08:51:48 main:574] : INFO : Epoch 115 | loss: nan | val_loss: nan | Time: 5410.69 ms
[2021-05-27 08:51:58 main:574] : INFO : Epoch 116 | loss: nan | val_loss: nan | Time: 9157.99 ms
[2021-05-27 08:52:04 main:574] : INFO : Epoch 117 | loss: nan | val_loss: nan | Time: 5659.63 ms
[2021-05-27 08:52:10 main:574] : INFO : Epoch 118 | loss: nan | val_loss: nan | Time: 5838.33 ms
[2021-05-27 08:52:16 main:574] : INFO : Epoch 119 | loss: nan | val_loss: nan | Time: 5831.64 ms
[2021-05-27 08:52:22 main:574] : INFO : Epoch 120 | loss: nan | val_loss: nan | Time: 5404.07 ms
[2021-05-27 08:52:28 main:574] : INFO : Epoch 121 | loss: nan | val_loss: nan | Time: 5591.89 ms
[2021-05-27 08:52:34 main:574] : INFO : Epoch 122 | loss: nan | val_loss: nan | Time: 6120.02 ms
[2021-05-27 08:52:43 main:574] : INFO : Epoch 123 | loss: nan | val_loss: nan | Time: 7943.55 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce 940MX)
[2021-05-27 10:34:10 main:435] : INFO : Set logging level to 1
[2021-05-27 10:34:11 main:441] : INFO : Running in BOINC Client mode
[2021-05-27 10:34:12 main:444] : INFO : Resolving all filenames
[2021-05-27 10:34:14 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-05-27 10:34:16 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2_3_r808142750_1 (exists = 1)
[2021-05-27 10:34:17 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2_3_r808142750_0 (exists = 0)
[2021-05-27 10:34:18 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1619583751-17130-2 (exists = 1)
[2021-05-27 10:34:19 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-05-27 10:34:19 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-05-27 10:34:20 main:474] : INFO : Configuration:
[2021-05-27 10:34:21 main:475] : INFO : Model type: GRU
[2021-05-27 10:34:22 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-05-27 10:34:22 main:477] : INFO : Max Epochs: 2048
[2021-05-27 10:34:22 main:478] : INFO : Batch Size: 128
[2021-05-27 10:34:23 main:479] : INFO : Learning Rate: 0.01
[2021-05-27 10:34:23 main:480] : INFO : Patience: 10
[2021-05-27 10:34:23 main:481] : INFO : Hidden Width: 12
[2021-05-27 10:34:23 main:482] : INFO : # Recurrent Layers: 4
[2021-05-27 10:34:23 main:483] : INFO : # Backend Layers: 4
[2021-05-27 10:34:23 main:484] : INFO : # Threads: 1
[2021-05-27 10:34:24 main:486] : INFO : Preparing Dataset
[2021-05-27 10:34:24 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 10:34:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 10:34:37 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-05-27 10:34:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 10:34:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-05-27 10:34:38 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-05-27 10:34:38 main:494] : INFO : Creating Model
[2021-05-27 10:34:38 main:507] : INFO : Preparing config file
[2021-05-27 10:34:38 main:511] : INFO : Found checkpoint, attempting to load...
[2021-05-27 10:34:39 main:512] : INFO : Loading config
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFE9CE14B89
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 05/27/21 10:34:39
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\5;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000533e0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000009f390000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009da30000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009cde0000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009d7b0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009d3c0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.985) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.985 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.985
ModLoad: 000000009d4a0000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.746 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.746
ModLoad: 000000009d280000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.928 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.928
ModLoad: 000000009d160000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 000000009cc80000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 000000009f2a0000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009e730000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 000000009e9a0000 000000000009b000 C:\WINDOWS\System32\sechost.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000009d680000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000007f8e0000 0000000000058000 C:\ProgramData\BOINC\slots\5\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000001f7f0000 000000000abe7000 C:\ProgramData\BOINC\slots\5\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000007b7a0000 0000000000091000 C:\ProgramData\BOINC\slots\5\MSVCP140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 000000002a3e0000 0000000024b49000 C:\ProgramData\BOINC\slots\5\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000008c9b0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000008e750000 0000000000019000 C:\ProgramData\BOINC\slots\5\VCRUNTIME140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 000000006e930000 000000000020d000 C:\ProgramData\BOINC\slots\5\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000949c0000 000000000000c000 C:\ProgramData\BOINC\slots\5\VCRUNTIME140_1.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 000000006dff0000 00000000002db000 C:\ProgramData\BOINC\slots\5\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000086310000 0000000000045000 C:\ProgramData\BOINC\slots\5\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000057d00000 00000000030a5000 C:\ProgramData\BOINC\slots\5\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000054840000 00000000034b2000 C:\ProgramData\BOINC\slots\5\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000d9010000 000000001b3a4000 C:\ProgramData\BOINC\slots\5\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 000000008cf30000 0000000000010000 C:\ProgramData\BOINC\slots\5\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000016650000 000000000919f000 C:\ProgramData\BOINC\slots\5\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000089040000 0000000000040000 C:\ProgramData\BOINC\slots\5\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000001370000 0000000003d5b000 C:\ProgramData\BOINC\slots\5\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000052970000 0000000001eca000 C:\ProgramData\BOINC\slots\5\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000009f100000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009aa20000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009db50000 0000000000469000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009cc30000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009d390000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009c7b0000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009cd80000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.804) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 000000009cad0000 000000000015f000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009c5f0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009dfc0000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009ac20000 0000000000790000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009eaa0000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009c470000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009f050000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009ea40000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000005d970000 00000000010e7000 C:\WINDOWS\system32\nvcuda.dll (26.21.14.4614) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r445\r445_85\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 26.21.14.4614
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.0.140 driver
Product Version : 26.21.14.4614
ModLoad: 000000008e0f0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009e7d0000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009a540000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 00000000832f0000 00000000004e6000 C:\WINDOWS\SYSTEM32\nvapi64.dll (26.21.14.4614) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r445\r445_85\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 26.21.14.4614
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 26.21.14.4614
ModLoad: 000000009ee00000 000000000012a000 C:\WINDOWS\System32\OLE32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.980 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Sistema operativo Microsoft® Windows®
Product Version : 10.0.19041.980
ModLoad: 000000009b410000 00000000000f4000 C:\WINDOWS\SYSTEM32\dxgi.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : dxgi.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 000000009d200000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3607, Write: 310, Other 1044
- I/O Transfers Counters -
Read: 6534875, Write: 7271, Other 31472
- Paged Pool Usage -
QuotaPagedPoolUsage: 3453288, QuotaPeakPagedPoolUsage: 3453288
QuotaNonPagedPoolUsage: 37528, QuotaPeakNonPagedPoolUsage: 37656
- Virtual Memory Usage -
VirtualSize: -1490030592, PeakVirtualSize: -1004437504
- Pagefile Usage -
PagefileUsage: -1490030592, PeakPagefileUsage: -1490030592
- Working Set Size -
WorkingSetSize: 1152397312, PeakWorkingSetSize: 1152401408, PageFaultCount: 390921
*** Dump of thread ID 13496 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 12031250.000000, User Time: 28750000.000000, Wait Time: 64404408.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFE9CE14B89
- Registers -
rax=0000000025388b93 rbx=0000000053744910 rcx=000000008b1e5eb0 rdx=0000000000000001 rsi=000000008b1e67f0 rdi=0000000019930520
r8=000000008b1e5f48 r9=0000000000000000 r10=0000000000000000 r11=00000000d3f4a500 r12=00000000d7e908a0 r13=000000008b1e6a00
r14=00000000aa176cb0 r15=0000000000000000 rip=000000009ce14b89 rsp=000000008b1e6690 rbp=000000008b1e6980
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
- Callstack -
ChildEBP RetAddr Args to Child
8b1e6760 5365a884 00000040 0000004f d7e90300 000003d7 KERNELBASE!RaiseException+0x0
8b1e67c0 53401ca6 00000001 00000000 ccce7230 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1e6870 5340593b 00000000 00000030 00000020 00000008 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1e69c0 53401a78 00000000 00000000 8b1e6cf0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1e6a90 53401cfc 8b1e6cf0 00000000 00000080 8b1e6fc0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1e6b50 5340c4dd 8b1e6fc0 8b1e6fc0 8b1e6fc0 8b1e6fc0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1e6e40 533fb2fe 8b1e7458 8b1e6fc0 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1ffbe0 5363c6f8 867262b0 00000000 00000000 84d15b10 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1ffc20 9da47034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b1ffc50 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b1ffcd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 15824 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 64402240.000000
- Registers -
rax=00000000000001d3 rbx=0000000084d18fe0 rcx=0000000000000070 rdx=0000000084d18c60 rsi=0000000084d131b0 rdi=0000000084d18fe0
r8=000000008b4ff240 r9=0000000084d137b4 r10=000000000000006c r11=000000008b4ff040 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000009f430874 rsp=000000008b4ffad8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8b4ffad0 9f3e2dc7 00000000 00000000 84d131b0 84d18fe0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
8b4ffdd0 9da47034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
8b4ffe00 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b4ffe80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 7384 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 312500.000000, User Time: 0.000000, Wait Time: 64402240.000000
- Registers -
rax=00000000000001d3 rbx=0000000084d197f0 rcx=0000000000000070 rdx=0000000084d19470 rsi=0000000084d131b0 rdi=0000000084d197f0
r8=0000000000000008 r9=0000000000000040 r10=00000000000005c0 r11=0000000000c03000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000009f430874 rsp=000000008b5ff5f8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8b5ff5f0 9f3e2dc7 00000000 00000000 84d131b0 84d197f0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
8b5ff8f0 9da47034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
8b5ff920 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b5ff9a0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9724 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 312500.000000, User Time: 0.000000, Wait Time: 64402240.000000
- Registers -
rax=00000000000001d3 rbx=0000000084d1bdc0 rcx=0000000000000070 rdx=0000000084d1ba40 rsi=0000000084d131b0 rdi=0000000084d1bdc0
r8=00000000000002a8 r9=000000000000000e r10=0000000000000000 r11=00000000050b82a8 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000009f430874 rsp=000000008b6ff668 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8b6ff660 9f3e2dc7 00000000 00000000 84d131b0 84d1bdc0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
8b6ff960 9da47034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
8b6ff990 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b6ffa10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 2856 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 64404408.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=000000008b8ffba0 rsi=0000000000000000 rdi=0000000000000064
r8=000000008b399000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000009f42d4a4 rsp=000000008b8ffb78 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8b8ffb70 9ce290ae 8b8ffc38 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
8b8ffc10 5341a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
8b8ffc40 9da47034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8b8ffc70 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b8ffcf0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 6828 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 64402588.000000
- Registers -
rax=00000000000001d3 rbx=0000000086be65e0 rcx=0000000000000014 rdx=0000000086be6260 rsi=0000000084d10b50 rdi=0000000086be65e0
r8=00000000000000ff r9=0000000000000000 r10=0000000000000000 r11=000000008b9ffab8 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000009f430874 rsp=000000008b9ffc08 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8b9ffc00 9f3e2dc7 84d10af8 84d10af8 84d10b50 86be65e0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
8b9fff00 9da47034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
8b9fff30 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8b9fffb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8720 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 64404408.000000
- Registers -
rax=000000000000005b rbx=000000000000000a rcx=000000000000000a rdx=000000008baffb30 rsi=0000000000000000 rdi=000000000000000a
r8=0000000086e19d50 r9=000000008614a970 r10=0000000000000000 r11=000000008baffaa0 r12=0000000000000064 r13=000000008baffb30
r14=000000008baff830 r15=0000000000000000 rip=000000009f42d974 rsp=000000008baff7d8 rbp=000000008614a970
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8baff7d0 9ce2c5a0 00000000 8b38e000 9f4fa4d0 8b38e000 ntdll!ZwWaitForMultipleObjects+0x0
8baffac0 9ce2c49e 84d219c0 004e0049 0000000a 00000004 KERNELBASE!WaitForMultipleObjectsEx+0x0
8baffb00 5dae9cf4 86901f10 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
8baffd40 5db72f88 00000064 00000000 00000064 00000004 nvcuda!+0x0
8baffdf0 5dae96c3 05d8cd7e fdf4cf53 86e1f750 00000000 nvcuda!cuProfilerStop+0x0
8baffe20 5ddac000 86e1f750 00000000 00000000 00000000 nvcuda!+0x0
8baffe50 9da47034 00000000 00000000 00000000 00000000 nvcuda!cuProfilerStop+0x0
8baffe80 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8bafff00 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 4580 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 64403652.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=000000000000029c rdx=0000000000000000 rsi=0000000000000000 rdi=000000000000029c
r8=0000000086944110 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000086944110 r13=0000000000000000
r14=000000000000029c r15=0000000000000000 rip=000000009f42cea4 rsp=000000008bbffbb8 rbp=0000000000000001
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8bbffbb0 9ce0168e 00000000 00000000 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
8bbffc50 5dcca057 00000000 00000000 00000000 0000029c KERNELBASE!WaitForSingleObjectEx+0x0
8bbffca0 5dae96c3 00000000 86e1f960 00000000 5d970000 nvcuda!cuProfilerStop+0x0
8bbffcd0 5ddac000 86e1f960 00000000 00000000 00000000 nvcuda!+0x0
8bbffd00 9da47034 00000000 00000000 00000000 00000000 nvcuda!cuProfilerStop+0x0
8bbffd30 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8bbffdb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9612 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 64404380.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=0000000086ba54d8 rdx=000000008bcffa48 rsi=000000008bcff9b0 rdi=0000000000000000
r8=000000008bcff988 r9=000000008bcff9e0 r10=0000000000000000 r11=0000000000000246 r12=000000008bcffa48 r13=0000000000000000
r14=0000000086ba54d8 r15=0000000086ba5508 rip=000000009f430814 rsp=000000008bcff988 rbp=000000008bcff9e0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8bcff980 9f3f4295 00000102 00000102 00000000 8bcff9b0 ntdll!NtWaitForAlertByThreadId+0x0
8bcff9f0 9ce4c849 86ba2ce0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
8bcffa20 5da7d44a 86ba2ce0 00000000 86ba2ce0 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
8bcffa80 5dae96c3 00000004 046026c4 05e97329 00000000 nvcuda!+0x0
8bcffab0 5ddac000 86cd8890 00000000 00000000 00000000 nvcuda!+0x0
8bcffae0 9da47034 00000000 00000000 00000000 00000000 nvcuda!cuProfilerStop+0x0
8bcffb10 9f3e2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8bcffb90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)