Task 6097155
| Name | ParityMachine-1627358983-30823-1_0 |
| Workunit | 3895784 |
| Created | 27 Jul 2021, 13:27:36 UTC |
| Sent | 28 Jul 2021, 14:25:12 UTC |
| Report deadline | 4 Aug 2021, 14:25:12 UTC |
| Received | 29 Jul 2021, 23:32:49 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 157 |
| Run time | 4 hours 27 min 7 sec |
| CPU time | 3 hours 19 min 31 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 4,474.01 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.62 GB |
| Peak swap size | 3.60 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
main:574] : INFO : Epoch 982 | loss: 0.0312494 | val_loss: 0.0312494 | Time: 13013.9 ms
[2021-07-29 15:41:55 main:574] : INFO : Epoch 983 | loss: 0.0312503 | val_loss: 0.0312497 | Time: 13028.7 ms
[2021-07-29 15:42:08 main:574] : INFO : Epoch 984 | loss: 0.0312496 | val_loss: 0.0312494 | Time: 12742 ms
[2021-07-29 15:42:22 main:574] : INFO : Epoch 985 | loss: 0.0312491 | val_loss: 0.0312495 | Time: 13462.7 ms
[2021-07-29 15:42:35 main:574] : INFO : Epoch 986 | loss: 0.0312492 | val_loss: 0.0312496 | Time: 13141.7 ms
[2021-07-29 15:42:48 main:574] : INFO : Epoch 987 | loss: 0.0312493 | val_loss: 0.0312495 | Time: 13176.2 ms
[2021-07-29 15:43:02 main:574] : INFO : Epoch 988 | loss: 0.0312495 | val_loss: 0.0312494 | Time: 13442.4 ms
[2021-07-29 15:43:15 main:574] : INFO : Epoch 989 | loss: 0.0312495 | val_loss: 0.0312493 | Time: 13135 ms
[2021-07-29 15:43:28 main:574] : INFO : Epoch 990 | loss: 0.0312498 | val_loss: 0.0312494 | Time: 12845.6 ms
[2021-07-29 15:43:41 main:574] : INFO : Epoch 991 | loss: 0.0312499 | val_loss: 0.0312495 | Time: 13225.7 ms
[2021-07-29 15:43:55 main:574] : INFO : Epoch 992 | loss: 0.0312498 | val_loss: 0.0312495 | Time: 13269.8 ms
[2021-07-29 15:44:07 main:574] : INFO : Epoch 993 | loss: 0.0312494 | val_loss: 0.0312499 | Time: 12656 ms
[2021-07-29 15:44:21 main:574] : INFO : Epoch 994 | loss: 0.0312498 | val_loss: 0.0312504 | Time: 13355.5 ms
[2021-07-29 15:44:34 main:574] : INFO : Epoch 995 | loss: 0.0312495 | val_loss: 0.0312494 | Time: 13325.1 ms
[2021-07-29 15:44:46 main:574] : INFO : Epoch 996 | loss: 0.0312496 | val_loss: 0.0312501 | Time: 12080.3 ms
[2021-07-29 15:45:01 main:574] : INFO : Epoch 997 | loss: 0.0312494 | val_loss: 0.0312493 | Time: 14618.9 ms
[2021-07-29 15:45:14 main:574] : INFO : Epoch 998 | loss: 0.0312493 | val_loss: 0.0312494 | Time: 13248 ms
[2021-07-29 15:45:27 main:574] : INFO : Epoch 999 | loss: 0.0312493 | val_loss: 0.0312497 | Time: 12629.3 ms
[2021-07-29 15:45:40 main:574] : INFO : Epoch 1000 | loss: 0.0312506 | val_loss: 0.031253 | Time: 12928.9 ms
[2021-07-29 15:45:53 main:574] : INFO : Epoch 1001 | loss: 0.0312501 | val_loss: 0.0312508 | Time: 12878 ms
[2021-07-29 15:46:06 main:574] : INFO : Epoch 1002 | loss: 0.0312495 | val_loss: 0.0312497 | Time: 13193.6 ms
[2021-07-29 15:46:20 main:574] : INFO : Epoch 1003 | loss: 0.0312491 | val_loss: 0.0312494 | Time: 13348.7 ms
[2021-07-29 15:46:34 main:574] : INFO : Epoch 1004 | loss: 0.0312491 | val_loss: 0.0312494 | Time: 13993 ms
[2021-07-29 15:46:47 main:574] : INFO : Epoch 1005 | loss: 0.0312499 | val_loss: 0.0312511 | Time: 12940.8 ms
[2021-07-29 15:47:01 main:574] : INFO : Epoch 1006 | loss: 0.0312498 | val_loss: 0.0312493 | Time: 13503.4 ms
[2021-07-29 15:47:14 main:574] : INFO : Epoch 1007 | loss: 0.0312491 | val_loss: 0.0312501 | Time: 13267 ms
[2021-07-29 15:47:27 main:574] : INFO : Epoch 1008 | loss: 0.0312496 | val_loss: 0.0312495 | Time: 12915.1 ms
[2021-07-29 15:47:41 main:574] : INFO : Epoch 1009 | loss: 0.0312491 | val_loss: 0.0312501 | Time: 13494.2 ms
[2021-07-29 15:47:54 main:574] : INFO : Epoch 1010 | loss: 0.0312502 | val_loss: 0.0312508 | Time: 13291.2 ms
[2021-07-29 15:48:07 main:574] : INFO : Epoch 1011 | loss: 0.0312507 | val_loss: 0.031251 | Time: 13265.2 ms
[2021-07-29 15:48:21 main:574] : INFO : Epoch 1012 | loss: 0.0312508 | val_loss: 0.0312532 | Time: 13241.1 ms
[2021-07-29 15:48:34 main:574] : INFO : Epoch 1013 | loss: 0.0312521 | val_loss: 0.0312523 | Time: 12976.7 ms
[2021-07-29 15:48:47 main:574] : INFO : Epoch 1014 | loss: 0.0312533 | val_loss: 0.0312586 | Time: 13175.9 ms
[2021-07-29 15:49:00 main:574] : INFO : Epoch 1015 | loss: 0.0312537 | val_loss: 0.0312518 | Time: 12813.9 ms
[2021-07-29 15:49:14 main:574] : INFO : Epoch 1016 | loss: 0.03125 | val_loss: 0.0312497 | Time: 13328.8 ms
[2021-07-29 15:49:26 main:574] : INFO : Epoch 1017 | loss: 0.0312495 | val_loss: 0.0312497 | Time: 12661.2 ms
[2021-07-29 15:49:39 main:574] : INFO : Epoch 1018 | loss: 0.0312499 | val_loss: 0.0312496 | Time: 13098.3 ms
[2021-07-29 15:49:52 main:574] : INFO : Epoch 1019 | loss: 0.0312497 | val_loss: 0.0312501 | Time: 13016.2 ms
[2021-07-29 15:50:05 main:574] : INFO : Epoch 1020 | loss: 0.0312496 | val_loss: 0.0312493 | Time: 12720.5 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB)
[2021-07-29 15:54:16 main:435] : INFO : Set logging level to 1
[2021-07-29 15:54:16 main:441] : INFO : Running in BOINC Client mode
[2021-07-29 15:54:16 main:444] : INFO : Resolving all filenames
[2021-07-29 15:54:17 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-07-29 15:54:17 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_1 (exists = 1)
[2021-07-29 15:54:17 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_0 (exists = 0)
[2021-07-29 15:54:17 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1 (exists = 1)
[2021-07-29 15:54:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-07-29 15:54:17 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-07-29 15:54:17 main:474] : INFO : Configuration:
[2021-07-29 15:54:17 main:475] : INFO : Model type: GRU
[2021-07-29 15:54:17 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-07-29 15:54:17 main:477] : INFO : Max Epochs: 2048
[2021-07-29 15:54:17 main:478] : INFO : Batch Size: 128
[2021-07-29 15:54:17 main:479] : INFO : Learning Rate: 0.01
[2021-07-29 15:54:17 main:480] : INFO : Patience: 10
[2021-07-29 15:54:17 main:481] : INFO : Hidden Width: 12
[2021-07-29 15:54:17 main:482] : INFO : # Recurrent Layers: 4
[2021-07-29 15:54:17 main:483] : INFO : # Backend Layers: 4
[2021-07-29 15:54:17 main:484] : INFO : # Threads: 1
[2021-07-29 15:54:18 main:486] : INFO : Preparing Dataset
[2021-07-29 15:54:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:54:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:54:23 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-07-29 15:54:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:54:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:54:24 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-07-29 15:54:24 main:494] : INFO : Creating Model
[2021-07-29 15:54:24 main:507] : INFO : Preparing config file
[2021-07-29 15:54:24 main:511] : INFO : Found checkpoint, attempting to load...
[2021-07-29 15:54:24 main:512] : INFO : Loading config
[2021-07-29 15:54:24 main:514] : INFO : Loading state
[2021-07-29 15:54:26 main:559] : INFO : Loading DataLoader into Memory
[2021-07-29 15:54:26 main:562] : INFO : Starting Training
[2021-07-29 15:54:40 main:574] : INFO : Epoch 1020 | loss: 0.0312701 | val_loss: 0.0312568 | Time: 14228 ms
[2021-07-29 15:54:53 main:574] : INFO : Epoch 1021 | loss: 0.0312517 | val_loss: 0.0312503 | Time: 12902.9 ms
[2021-07-29 15:55:06 main:574] : INFO : Epoch 1022 | loss: 0.0312505 | val_loss: 0.0312508 | Time: 12859 ms
[2021-07-29 15:55:20 main:574] : INFO : Epoch 1023 | loss: 0.0312499 | val_loss: 0.0312489 | Time: 13534 ms
[2021-07-29 15:55:34 main:574] : INFO : Epoch 1024 | loss: 0.0312489 | val_loss: 0.0312491 | Time: 13901.1 ms
[2021-07-29 15:55:47 main:574] : INFO : Epoch 1025 | loss: 0.0312482 | val_loss: 0.0312478 | Time: 13301.9 ms
[2021-07-29 15:56:01 main:574] : INFO : Epoch 1026 | loss: 0.0312463 | val_loss: 0.0312435 | Time: 13954.4 ms
[2021-07-29 15:56:15 main:574] : INFO : Epoch 1027 | loss: 0.0312438 | val_loss: 0.0312424 | Time: 13645.5 ms
[2021-07-29 15:56:29 main:574] : INFO : Epoch 1028 | loss: 0.0312411 | val_loss: 0.0312369 | Time: 13952.8 ms
[2021-07-29 15:56:43 main:574] : INFO : Epoch 1029 | loss: 0.0312387 | val_loss: 0.0312474 | Time: 13596.1 ms
[2021-07-29 15:56:56 main:574] : INFO : Epoch 1030 | loss: 0.0312358 | val_loss: 0.031231 | Time: 13196.7 ms
[2021-07-29 15:57:10 main:574] : INFO : Epoch 1031 | loss: 0.0312278 | val_loss: 0.0312394 | Time: 14072 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB)
[2021-07-29 15:59:24 main:435] : INFO : Set logging level to 1
[2021-07-29 15:59:24 main:441] : INFO : Running in BOINC Client mode
[2021-07-29 15:59:24 main:444] : INFO : Resolving all filenames
[2021-07-29 15:59:24 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-07-29 15:59:24 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_1 (exists = 1)
[2021-07-29 15:59:24 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_0 (exists = 0)
[2021-07-29 15:59:24 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1 (exists = 1)
[2021-07-29 15:59:24 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-07-29 15:59:24 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-07-29 15:59:24 main:474] : INFO : Configuration:
[2021-07-29 15:59:24 main:475] : INFO : Model type: GRU
[2021-07-29 15:59:24 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-07-29 15:59:24 main:477] : INFO : Max Epochs: 2048
[2021-07-29 15:59:24 main:478] : INFO : Batch Size: 128
[2021-07-29 15:59:24 main:479] : INFO : Learning Rate: 0.01
[2021-07-29 15:59:24 main:480] : INFO : Patience: 10
[2021-07-29 15:59:25 main:481] : INFO : Hidden Width: 12
[2021-07-29 15:59:25 main:482] : INFO : # Recurrent Layers: 4
[2021-07-29 15:59:25 main:483] : INFO : # Backend Layers: 4
[2021-07-29 15:59:25 main:484] : INFO : # Threads: 1
[2021-07-29 15:59:25 main:486] : INFO : Preparing Dataset
[2021-07-29 15:59:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:59:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:59:29 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-07-29 15:59:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:59:29 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 15:59:29 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-07-29 15:59:29 main:494] : INFO : Creating Model
[2021-07-29 15:59:29 main:507] : INFO : Preparing config file
[2021-07-29 15:59:29 main:511] : INFO : Found checkpoint, attempting to load...
[2021-07-29 15:59:29 main:512] : INFO : Loading config
[2021-07-29 15:59:30 main:514] : INFO : Loading state
[2021-07-29 15:59:33 main:559] : INFO : Loading DataLoader into Memory
[2021-07-29 15:59:33 main:562] : INFO : Starting Training
[2021-07-29 15:59:48 main:574] : INFO : Epoch 1029 | loss: 0.0312888 | val_loss: 0.0312498 | Time: 14606.6 ms
[2021-07-29 16:00:02 main:574] : INFO : Epoch 1030 | loss: 0.031253 | val_loss: 0.0312502 | Time: 14181.1 ms
[2021-07-29 16:00:15 main:574] : INFO : Epoch 1031 | loss: 0.0312498 | val_loss: 0.0312484 | Time: 13274.3 ms
[2021-07-29 16:00:29 main:574] : INFO : Epoch 1032 | loss: 0.0312468 | val_loss: 0.0312458 | Time: 13790.6 ms
[2021-07-29 16:00:43 main:574] : INFO : Epoch 1033 | loss: 0.0312425 | val_loss: 0.0312406 | Time: 14063.8 ms
[2021-07-29 16:00:57 main:574] : INFO : Epoch 1034 | loss: 0.0312357 | val_loss: 0.0312344 | Time: 13659.3 ms
[2021-07-29 16:01:12 main:574] : INFO : Epoch 1035 | loss: 0.0312299 | val_loss: 0.031228 | Time: 14524.8 ms
[2021-07-29 16:01:25 main:574] : INFO : Epoch 1036 | loss: 0.0312268 | val_loss: 0.0312239 | Time: 13579.5 ms
[2021-07-29 16:01:39 main:574] : INFO : Epoch 1037 | loss: 0.0312223 | val_loss: 0.031222 | Time: 13744.6 ms
[2021-07-29 16:01:53 main:574] : INFO : Epoch 1038 | loss: 0.0312165 | val_loss: 0.0312179 | Time: 14131.9 ms
[2021-07-29 16:02:07 main:574] : INFO : Epoch 1039 | loss: 0.031219 | val_loss: 0.031216 | Time: 13347.6 ms
[2021-07-29 16:02:21 main:574] : INFO : Epoch 1040 | loss: 0.0312123 | val_loss: 0.0312139 | Time: 14181.5 ms
[2021-07-29 16:02:35 main:574] : INFO : Epoch 1041 | loss: 0.0312098 | val_loss: 0.0312172 | Time: 14071.3 ms
[2021-07-29 16:02:49 main:574] : INFO : Epoch 1042 | loss: 0.0312111 | val_loss: 0.0312181 | Time: 14013 ms
[2021-07-29 16:03:03 main:574] : INFO : Epoch 1043 | loss: 0.0312102 | val_loss: 0.0312118 | Time: 13809.4 ms
[2021-07-29 16:03:16 main:574] : INFO : Epoch 1044 | loss: 0.0312051 | val_loss: 0.0312116 | Time: 13411.7 ms
[2021-07-29 16:03:30 main:574] : INFO : Epoch 1045 | loss: 0.0312048 | val_loss: 0.0312148 | Time: 13661.3 ms
[2021-07-29 16:03:44 main:574] : INFO : Epoch 1046 | loss: 0.0312053 | val_loss: 0.0312138 | Time: 13822.5 ms
[2021-07-29 16:03:57 main:574] : INFO : Epoch 1047 | loss: 0.0312037 | val_loss: 0.0312148 | Time: 13310.3 ms
[2021-07-29 16:04:12 main:574] : INFO : Epoch 1048 | loss: 0.0312019 | val_loss: 0.0312242 | Time: 14186.9 ms
[2021-07-29 16:04:25 main:574] : INFO : Epoch 1049 | loss: 0.0312041 | val_loss: 0.0312126 | Time: 13471.1 ms
[2021-07-29 16:04:40 main:574] : INFO : Epoch 1050 | loss: 0.0311998 | val_loss: 0.0312157 | Time: 14710 ms
[2021-07-29 16:04:55 main:574] : INFO : Epoch 1051 | loss: 0.0312003 | val_loss: 0.0312101 | Time: 14454 ms
[2021-07-29 16:05:08 main:574] : INFO : Epoch 1052 | loss: 0.0311981 | val_loss: 0.0312075 | Time: 13238.3 ms
[2021-07-29 16:05:22 main:574] : INFO : Epoch 1053 | loss: 0.0311978 | val_loss: 0.0312084 | Time: 14023.9 ms
[2021-07-29 16:05:36 main:574] : INFO : Epoch 1054 | loss: 0.0311992 | val_loss: 0.0312094 | Time: 13307.6 ms
[2021-07-29 16:05:49 main:574] : INFO : Epoch 1055 | loss: 0.0311961 | val_loss: 0.0312203 | Time: 13302.7 ms
[2021-07-29 16:06:02 main:574] : INFO : Epoch 1056 | loss: 0.031197 | val_loss: 0.0312087 | Time: 13460.6 ms
[2021-07-29 16:06:15 main:574] : INFO : Epoch 1057 | loss: 0.0311971 | val_loss: 0.0312127 | Time: 12810.3 ms
[2021-07-29 16:06:28 main:574] : INFO : Epoch 1058 | loss: 0.0312021 | val_loss: 0.031209 | Time: 12844.5 ms
[2021-07-29 16:06:42 main:574] : INFO : Epoch 1059 | loss: 0.0311965 | val_loss: 0.0312077 | Time: 13110.3 ms
[2021-07-29 16:06:55 main:574] : INFO : Epoch 1060 | loss: 0.0311944 | val_loss: 0.0312129 | Time: 13007.7 ms
[2021-07-29 16:07:08 main:574] : INFO : Epoch 1061 | loss: 0.0311943 | val_loss: 0.0312086 | Time: 12907.3 ms
[2021-07-29 16:07:21 main:574] : INFO : Epoch 1062 | loss: 0.0311937 | val_loss: 0.0312067 | Time: 13523.4 ms
[2021-07-29 16:07:35 main:574] : INFO : Epoch 1063 | loss: 0.0311927 | val_loss: 0.0312068 | Time: 13528.3 ms
[2021-07-29 16:07:48 main:574] : INFO : Epoch 1064 | loss: 0.0311918 | val_loss: 0.0312054 | Time: 13181.8 ms
[2021-07-29 16:08:02 main:574] : INFO : Epoch 1065 | loss: 0.0311925 | val_loss: 0.0312079 | Time: 13490.8 ms
[2021-07-29 16:08:15 main:574] : INFO : Epoch 1066 | loss: 0.0311913 | val_loss: 0.0312141 | Time: 13226 ms
[2021-07-29 16:08:28 main:574] : INFO : Epoch 1067 | loss: 0.0311918 | val_loss: 0.0312057 | Time: 13066.6 ms
[2021-07-29 16:08:42 main:574] : INFO : Epoch 1068 | loss: 0.0311892 | val_loss: 0.0312056 | Time: 13270.9 ms
[2021-07-29 16:08:55 main:574] : INFO : Epoch 1069 | loss: 0.0311892 | val_loss: 0.0312049 | Time: 13036 ms
[2021-07-29 16:09:08 main:574] : INFO : Epoch 1070 | loss: 0.0311906 | val_loss: 0.0312102 | Time: 13355.9 ms
[2021-07-29 16:09:22 main:574] : INFO : Epoch 1071 | loss: 0.0311891 | val_loss: 0.0312059 | Time: 13518 ms
[2021-07-29 16:09:35 main:574] : INFO : Epoch 1072 | loss: 0.0311885 | val_loss: 0.0312054 | Time: 13257.9 ms
[2021-07-29 16:09:48 main:574] : INFO : Epoch 1073 | loss: 0.031195 | val_loss: 0.0312096 | Time: 12849.3 ms
[2021-07-29 16:10:01 main:574] : INFO : Epoch 1074 | loss: 0.031201 | val_loss: 0.0312082 | Time: 13138.4 ms
[2021-07-29 16:10:14 main:574] : INFO : Epoch 1075 | loss: 0.031198 | val_loss: 0.0312062 | Time: 13190.7 ms
[2021-07-29 16:10:28 main:574] : INFO : Epoch 1076 | loss: 0.031195 | val_loss: 0.0312043 | Time: 13512.6 ms
[2021-07-29 16:10:41 main:574] : INFO : Epoch 1077 | loss: 0.0311927 | val_loss: 0.031204 | Time: 13205.8 ms
[2021-07-29 16:10:55 main:574] : INFO : Epoch 1078 | loss: 0.0311931 | val_loss: 0.0312041 | Time: 13365.6 ms
[2021-07-29 16:11:08 main:574] : INFO : Epoch 1079 | loss: 0.0311909 | val_loss: 0.0312073 | Time: 12971.2 ms
[2021-07-29 16:11:21 main:574] : INFO : Epoch 1080 | loss: 0.0311915 | val_loss: 0.0312047 | Time: 13409.4 ms
[2021-07-29 16:11:35 main:574] : INFO : Epoch 1081 | loss: 0.0311916 | val_loss: 0.0312108 | Time: 13549.9 ms
[2021-07-29 16:11:48 main:574] : INFO : Epoch 1082 | loss: 0.0311892 | val_loss: 0.0312028 | Time: 12829.5 ms
[2021-07-29 16:12:01 main:574] : INFO : Epoch 1083 | loss: 0.0311886 | val_loss: 0.0312014 | Time: 13590.6 ms
[2021-07-29 16:12:15 main:574] : INFO : Epoch 1084 | loss: 0.031188 | val_loss: 0.0312044 | Time: 13035.4 ms
[2021-07-29 16:12:28 main:574] : INFO : Epoch 1085 | loss: 0.0311899 | val_loss: 0.0312032 | Time: 12999.3 ms
[2021-07-29 16:12:41 main:574] : INFO : Epoch 1086 | loss: 0.0311891 | val_loss: 0.0312033 | Time: 13554.3 ms
[2021-07-29 16:12:55 main:574] : INFO : Epoch 1087 | loss: 0.0311891 | val_loss: 0.031205 | Time: 13483.5 ms
[2021-07-29 16:13:08 main:574] : INFO : Epoch 1088 | loss: 0.0311877 | val_loss: 0.031202 | Time: 13053.7 ms
[2021-07-29 16:13:21 main:574] : INFO : Epoch 1089 | loss: 0.0311859 | val_loss: 0.0312026 | Time: 13252.2 ms
[2021-07-29 16:13:35 main:574] : INFO : Epoch 1090 | loss: 0.0311875 | val_loss: 0.0312044 | Time: 13590.5 ms
[2021-07-29 16:13:48 main:574] : INFO : Epoch 1091 | loss: 0.0311898 | val_loss: 0.0312048 | Time: 12683.9 ms
[2021-07-29 16:14:01 main:574] : INFO : Epoch 1092 | loss: 0.0311868 | val_loss: 0.0312019 | Time: 13680.1 ms
[2021-07-29 16:14:15 main:574] : INFO : Epoch 1093 | loss: 0.0311856 | val_loss: 0.0312045 | Time: 13266 ms
[2021-07-29 16:14:28 main:574] : INFO : Epoch 1094 | loss: 0.0311843 | val_loss: 0.0312022 | Time: 12828.8 ms
[2021-07-29 16:14:41 main:574] : INFO : Epoch 1095 | loss: 0.0311858 | val_loss: 0.0312033 | Time: 13262.1 ms
[2021-07-29 16:14:55 main:574] : INFO : Epoch 1096 | loss: 0.0311847 | val_loss: 0.0312042 | Time: 13705 ms
[2021-07-29 16:15:09 main:574] : INFO : Epoch 1097 | loss: 0.0311842 | val_loss: 0.0312023 | Time: 14230.7 ms
[2021-07-29 16:15:22 main:574] : INFO : Epoch 1098 | loss: 0.0311838 | val_loss: 0.0312053 | Time: 13076.4 ms
[2021-07-29 16:15:35 main:574] : INFO : Epoch 1099 | loss: 0.031185 | val_loss: 0.0312063 | Time: 12651.5 ms
[2021-07-29 16:15:48 main:574] : INFO : Epoch 1100 | loss: 0.0311855 | val_loss: 0.0312021 | Time: 12545.4 ms
[2021-07-29 16:16:01 main:574] : INFO : Epoch 1101 | loss: 0.0311836 | val_loss: 0.0312049 | Time: 13314.9 ms
[2021-07-29 16:16:14 main:574] : INFO : Epoch 1102 | loss: 0.0311827 | val_loss: 0.0312096 | Time: 13080.4 ms
[2021-07-29 16:16:27 main:574] : INFO : Epoch 1103 | loss: 0.0311835 | val_loss: 0.0312025 | Time: 13032.8 ms
[2021-07-29 16:16:41 main:574] : INFO : Epoch 1104 | loss: 0.0311837 | val_loss: 0.0312041 | Time: 13848 ms
[2021-07-29 16:16:55 main:574] : INFO : Epoch 1105 | loss: 0.0311859 | val_loss: 0.0312039 | Time: 13192.8 ms
[2021-07-29 16:17:08 main:574] : INFO : Epoch 1106 | loss: 0.031183 | val_loss: 0.0312053 | Time: 13482.2 ms
[2021-07-29 16:17:22 main:574] : INFO : Epoch 1107 | loss: 0.0311876 | val_loss: 0.031202 | Time: 13620.9 ms
[2021-07-29 16:17:35 main:574] : INFO : Epoch 1108 | loss: 0.0311868 | val_loss: 0.0312073 | Time: 13105.8 ms
[2021-07-29 16:17:48 main:574] : INFO : Epoch 1109 | loss: 0.0311882 | val_loss: 0.0312024 | Time: 13088.9 ms
[2021-07-29 16:18:01 main:574] : INFO : Epoch 1110 | loss: 0.0311912 | val_loss: 0.0312041 | Time: 13047.2 ms
[2021-07-29 16:18:15 main:574] : INFO : Epoch 1111 | loss: 0.0311889 | val_loss: 0.0312048 | Time: 13328.8 ms
[2021-07-29 16:18:28 main:574] : INFO : Epoch 1112 | loss: 0.0311834 | val_loss: 0.031205 | Time: 12996.2 ms
[2021-07-29 16:18:41 main:574] : INFO : Epoch 1113 | loss: 0.0311844 | val_loss: 0.0312019 | Time: 13422.1 ms
[2021-07-29 16:18:55 main:574] : INFO : Epoch 1114 | loss: 0.0311826 | val_loss: 0.0312044 | Time: 13191.3 ms
[2021-07-29 16:19:08 main:574] : INFO : Epoch 1115 | loss: 0.0311812 | val_loss: 0.0312028 | Time: 12894.4 ms
[2021-07-29 16:19:21 main:574] : INFO : Epoch 1116 | loss: 0.031181 | val_loss: 0.0312043 | Time: 13340.3 ms
[2021-07-29 16:19:34 main:574] : INFO : Epoch 1117 | loss: 0.0311806 | val_loss: 0.0312015 | Time: 13294.6 ms
[2021-07-29 16:19:48 main:574] : INFO : Epoch 1118 | loss: 0.0311819 | val_loss: 0.0312063 | Time: 13373.9 ms
[2021-07-29 16:20:01 main:574] : INFO : Epoch 1119 | loss: 0.0311792 | val_loss: 0.0312023 | Time: 13017.3 ms
[2021-07-29 16:20:14 main:574] : INFO : Epoch 1120 | loss: 0.031178 | val_loss: 0.0312031 | Time: 13346.1 ms
[2021-07-29 16:20:28 main:574] : INFO : Epoch 1121 | loss: 0.0311797 | val_loss: 0.0312081 | Time: 13161.2 ms
[2021-07-29 16:20:41 main:574] : INFO : Epoch 1122 | loss: 0.0311828 | val_loss: 0.0312048 | Time: 13278.7 ms
[2021-07-29 16:20:54 main:574] : INFO : Epoch 1123 | loss: 0.0311808 | val_loss: 0.0312021 | Time: 13180.4 ms
[2021-07-29 16:21:07 main:574] : INFO : Epoch 1124 | loss: 0.0311793 | val_loss: 0.0312014 | Time: 12865 ms
[2021-07-29 16:21:20 main:574] : INFO : Epoch 1125 | loss: 0.0311806 | val_loss: 0.0312033 | Time: 13318.7 ms
[2021-07-29 16:21:34 main:574] : INFO : Epoch 1126 | loss: 0.0311789 | val_loss: 0.0312026 | Time: 13039.1 ms
[2021-07-29 16:21:47 main:574] : INFO : Epoch 1127 | loss: 0.0311822 | val_loss: 0.0312152 | Time: 12973 ms
[2021-07-29 16:22:00 main:574] : INFO : Epoch 1128 | loss: 0.0311924 | val_loss: 0.0312197 | Time: 13268.2 ms
[2021-07-29 16:22:14 main:574] : INFO : Epoch 1129 | loss: 0.0312023 | val_loss: 0.0312057 | Time: 13516.2 ms
[2021-07-29 16:22:27 main:574] : INFO : Epoch 1130 | loss: 0.0311913 | val_loss: 0.0312186 | Time: 13143.1 ms
[2021-07-29 16:22:40 main:574] : INFO : Epoch 1131 | loss: 0.0312652 | val_loss: 0.0313983 | Time: 13069.3 ms
[2021-07-29 16:22:54 main:574] : INFO : Epoch 1132 | loss: nan | val_loss: nan | Time: 13741 ms
[2021-07-29 16:23:07 main:574] : INFO : Epoch 1133 | loss: nan | val_loss: nan | Time: 13015.2 ms
[2021-07-29 16:23:20 main:574] : INFO : Epoch 1134 | loss: nan | val_loss: nan | Time: 13045.7 ms
[2021-07-29 16:23:33 main:574] : INFO : Epoch 1135 | loss: nan | val_loss: nan | Time: 13091.5 ms
[2021-07-29 16:23:46 main:574] : INFO : Epoch 1136 | loss: nan | val_loss: nan | Time: 12700.3 ms
[2021-07-29 16:23:59 main:574] : INFO : Epoch 1137 | loss: nan | val_loss: nan | Time: 13150.1 ms
[2021-07-29 16:24:12 main:574] : INFO : Epoch 1138 | loss: nan | val_loss: nan | Time: 13046 ms
[2021-07-29 16:24:25 main:574] : INFO : Epoch 1139 | loss: nan | val_loss: nan | Time: 12877.8 ms
[2021-07-29 16:24:38 main:574] : INFO : Epoch 1140 | loss: nan | val_loss: nan | Time: 13031 ms
[2021-07-29 16:24:52 main:574] : INFO : Epoch 1141 | loss: nan | val_loss: nan | Time: 13235.5 ms
[2021-07-29 16:25:05 main:574] : INFO : Epoch 1142 | loss: nan | val_loss: nan | Time: 13457.4 ms
[2021-07-29 16:25:19 main:574] : INFO : Epoch 1143 | loss: nan | val_loss: nan | Time: 13232.8 ms
[2021-07-29 16:25:32 main:574] : INFO : Epoch 1144 | loss: nan | val_loss: nan | Time: 13440.9 ms
[2021-07-29 16:25:45 main:574] : INFO : Epoch 1145 | loss: nan | val_loss: nan | Time: 13083.5 ms
[2021-07-29 16:25:58 main:574] : INFO : Epoch 1146 | loss: nan | val_loss: nan | Time: 13086.6 ms
[2021-07-29 16:26:11 main:574] : INFO : Epoch 1147 | loss: nan | val_loss: nan | Time: 13084.1 ms
[2021-07-29 16:26:25 main:574] : INFO : Epoch 1148 | loss: nan | val_loss: nan | Time: 13292.9 ms
[2021-07-29 16:26:38 main:574] : INFO : Epoch 1149 | loss: nan | val_loss: nan | Time: 12783.3 ms
[2021-07-29 16:26:51 main:574] : INFO : Epoch 1150 | loss: nan | val_loss: nan | Time: 13523.6 ms
[2021-07-29 16:27:05 main:574] : INFO : Epoch 1151 | loss: nan | val_loss: nan | Time: 13543.2 ms
[2021-07-29 16:27:18 main:574] : INFO : Epoch 1152 | loss: nan | val_loss: nan | Time: 13222.9 ms
[2021-07-29 16:27:32 main:574] : INFO : Epoch 1153 | loss: nan | val_loss: nan | Time: 14000.3 ms
[2021-07-29 16:27:45 main:574] : INFO : Epoch 1154 | loss: nan | val_loss: nan | Time: 12843.7 ms
[2021-07-29 16:27:59 main:574] : INFO : Epoch 1155 | loss: nan | val_loss: nan | Time: 13295.3 ms
[2021-07-29 16:28:12 main:574] : INFO : Epoch 1156 | loss: nan | val_loss: nan | Time: 13405 ms
[2021-07-29 16:28:25 main:574] : INFO : Epoch 1157 | loss: nan | val_loss: nan | Time: 12678.1 ms
[2021-07-29 16:28:37 main:574] : INFO : Epoch 1158 | loss: nan | val_loss: nan | Time: 12473.4 ms
[2021-07-29 16:28:50 main:574] : INFO : Epoch 1159 | loss: nan | val_loss: nan | Time: 12878.4 ms
[2021-07-29 16:29:04 main:574] : INFO : Epoch 1160 | loss: nan | val_loss: nan | Time: 13212 ms
[2021-07-29 16:29:16 main:574] : INFO : Epoch 1161 | loss: nan | val_loss: nan | Time: 12662.1 ms
[2021-07-29 16:29:30 main:574] : INFO : Epoch 1162 | loss: nan | val_loss: nan | Time: 13301.6 ms
[2021-07-29 16:29:47 main:574] : INFO : Epoch 1163 | loss: nan | val_loss: nan | Time: 16951.8 ms
[2021-07-29 16:30:01 main:574] : INFO : Epoch 1164 | loss: nan | val_loss: nan | Time: 13842.4 ms
[2021-07-29 16:30:14 main:574] : INFO : Epoch 1165 | loss: nan | val_loss: nan | Time: 13461.5 ms
[2021-07-29 16:30:27 main:574] : INFO : Epoch 1166 | loss: nan | val_loss: nan | Time: 13166.6 ms
[2021-07-29 16:30:40 main:574] : INFO : Epoch 1167 | loss: nan | val_loss: nan | Time: 12978.8 ms
[2021-07-29 16:30:54 main:574] : INFO : Epoch 1168 | loss: nan | val_loss: nan | Time: 13533.9 ms
[2021-07-29 16:31:07 main:574] : INFO : Epoch 1169 | loss: nan | val_loss: nan | Time: 13080.1 ms
[2021-07-29 16:31:20 main:574] : INFO : Epoch 1170 | loss: nan | val_loss: nan | Time: 12911.7 ms
[2021-07-29 16:31:34 main:574] : INFO : Epoch 1171 | loss: nan | val_loss: nan | Time: 13273 ms
[2021-07-29 16:31:46 main:574] : INFO : Epoch 1172 | loss: nan | val_loss: nan | Time: 12280.5 ms
[2021-07-29 16:31:59 main:574] : INFO : Epoch 1173 | loss: nan | val_loss: nan | Time: 13198.6 ms
[2021-07-29 16:32:13 main:574] : INFO : Epoch 1174 | loss: nan | val_loss: nan | Time: 13422.3 ms
[2021-07-29 16:32:26 main:574] : INFO : Epoch 1175 | loss: nan | val_loss: nan | Time: 13082.5 ms
[2021-07-29 16:32:39 main:574] : INFO : Epoch 1176 | loss: nan | val_loss: nan | Time: 13180.9 ms
[2021-07-29 16:32:52 main:574] : INFO : Epoch 1177 | loss: nan | val_loss: nan | Time: 12927.7 ms
[2021-07-29 16:33:05 main:574] : INFO : Epoch 1178 | loss: nan | val_loss: nan | Time: 13012.3 ms
[2021-07-29 16:33:18 main:574] : INFO : Epoch 1179 | loss: nan | val_loss: nan | Time: 13191.9 ms
[2021-07-29 16:33:32 main:574] : INFO : Epoch 1180 | loss: nan | val_loss: nan | Time: 13005.9 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 1060 6GB)
[2021-07-29 17:32:16 main:435] : INFO : Set logging level to 1
[2021-07-29 17:32:16 main:441] : INFO : Running in BOINC Client mode
[2021-07-29 17:32:16 main:444] : INFO : Resolving all filenames
[2021-07-29 17:32:16 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-07-29 17:32:16 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_1 (exists = 1)
[2021-07-29 17:32:17 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1_0_r1129496179_0 (exists = 0)
[2021-07-29 17:32:17 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1627358983-30823-1 (exists = 1)
[2021-07-29 17:32:17 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-07-29 17:32:17 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-07-29 17:32:17 main:474] : INFO : Configuration:
[2021-07-29 17:32:17 main:475] : INFO : Model type: GRU
[2021-07-29 17:32:17 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-07-29 17:32:17 main:477] : INFO : Max Epochs: 2048
[2021-07-29 17:32:17 main:478] : INFO : Batch Size: 128
[2021-07-29 17:32:17 main:479] : INFO : Learning Rate: 0.01
[2021-07-29 17:32:17 main:480] : INFO : Patience: 10
[2021-07-29 17:32:17 main:481] : INFO : Hidden Width: 12
[2021-07-29 17:32:18 main:482] : INFO : # Recurrent Layers: 4
[2021-07-29 17:32:18 main:483] : INFO : # Backend Layers: 4
[2021-07-29 17:32:18 main:484] : INFO : # Threads: 1
[2021-07-29 17:32:18 main:486] : INFO : Preparing Dataset
[2021-07-29 17:32:18 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 17:32:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 17:32:24 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-07-29 17:32:24 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 17:32:25 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-07-29 17:32:25 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-07-29 17:32:25 main:494] : INFO : Creating Model
[2021-07-29 17:32:25 main:507] : INFO : Preparing config file
[2021-07-29 17:32:25 main:511] : INFO : Found checkpoint, attempting to load...
[2021-07-29 17:32:25 main:512] : INFO : Loading config
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFCD3454ED9
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 07/29/21 17:32:25
Install Directory : D:\Program Files\
Data Directory : D:\ProgramData
Project Symstore :
LoadLibraryA( D:\ProgramData\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( D:\ProgramData\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: D:\ProgramData\slots\9;D:\ProgramData\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000d57e0000 00000000003df000 D:\ProgramData\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d5cb0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1023 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1023
ModLoad: 00000000d5990000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1023 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1023
ModLoad: 00000000d3420000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1023 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1023
ModLoad: 00000000a3710000 00000000000f1000 C:\Program Files\Bitdefender\Bitdefender Security\atcuf\265267636905997236\atcuf64.dll (1.40.291.0) (-exported- Symbols Loaded)
Linked PDB Filename : E:\builds\ATC-DEFAULT-SOURCES\bin\x64\Release\atcuf64\atcuf64.pdb
File Version : 1.40.291.0 #0xd59eaa1
Company Name : Bitdefender S.R.L. Bucharest, ROMANIA
Product Name : Bitdefender® ATC
Product Version : 13
ModLoad: 00000000d3f80000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d33f0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1110
ModLoad: 00000000d4a50000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.746 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.746
ModLoad: 00000000d39e0000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1110
ModLoad: 00000000d3c00000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000d38e0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000d4560000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1052) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d4a90000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 00000000d44c0000 000000000009b000 C:\WINDOWS\System32\sechost.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d5b40000 000000000012a000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000b89d0000 0000000000058000 D:\ProgramData\slots\9\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000080000000 0000000024b49000 D:\ProgramData\slots\9\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000038230000 000000000abe7000 D:\ProgramData\slots\9\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000b9c20000 000000000008d000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000cbe10000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 00000000bb330000 000000000001a000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000cbde0000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000bb280000 0000000000045000 D:\ProgramData\slots\9\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000749a0000 00000000030a5000 D:\ProgramData\slots\9\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 000000006dbf0000 00000000034b2000 D:\ProgramData\slots\9\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000fbf70000 000000001b3a4000 D:\ProgramData\slots\9\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000cdc10000 0000000000010000 D:\ProgramData\slots\9\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 000000002f090000 000000000919f000 D:\ProgramData\slots\9\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 000000002b330000 0000000003d5b000 D:\ProgramData\slots\9\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000088670000 000000000020d000 D:\ProgramData\slots\9\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 0000000082ca0000 00000000002db000 D:\ProgramData\slots\9\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000072ad0000 0000000001eca000 D:\ProgramData\slots\9\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000b9cc0000 0000000000040000 D:\ProgramData\slots\9\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d4970000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d1340000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d4ba0000 0000000000469000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d3ca0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d3cf0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d3130000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d3af0000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1110) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1110 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1110
ModLoad: 00000000d36f0000 000000000015f000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d2f70000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d50a0000 000000000073f000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 00000000d1540000 0000000000791000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d4610000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d2d90000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d3e00000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d4440000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000b7b10000 00000000002cc000 C:\WINDOWS\system32\nvcuda.dll (30.0.14.7111) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_96\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.14.7111
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.4.56 driver
Product Version : 30.0.14.7111
ModLoad: 00000000cdad0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000078890000 000000000161a000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispsi.inf_amd64_d610222ce397fb36\nvcuda64.dll (30.0.14.7111) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_96\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.14.7111
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.4.56 driver
Product Version : 30.0.14.7111
ModLoad: 00000000cadd0000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 00000000d2cd0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d0e80000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.1081) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000007b470000 0000000000718000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.14.7111) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_96\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.14.7111
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.14.7111
ModLoad: 00000000d4240000 000000000012a000 C:\WINDOWS\System32\ole32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000d3850000 0000000000083000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1023 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1023
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3635, Write: 310, Other 885
- I/O Transfers Counters -
Read: 6674129, Write: 7303, Other 38668
- Paged Pool Usage -
QuotaPagedPoolUsage: 3443024, QuotaPeakPagedPoolUsage: 3443024
QuotaNonPagedPoolUsage: 41000, QuotaPeakNonPagedPoolUsage: 41128
- Virtual Memory Usage -
VirtualSize: -1436102656, PeakVirtualSize: -842674176
- Pagefile Usage -
PagefileUsage: -1436102656, PeakPagefileUsage: -1436102656
- Working Set Size -
WorkingSetSize: 1155153920, PeakWorkingSetSize: 1155158016, PageFaultCount: 393611
*** Dump of thread ID 7968 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 12031250.000000, User Time: 32812500.000000, Wait Time: 88495448.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFCD3454ED9
- Registers -
rax=00000000faa20150 rbx=00000000d5b44910 rcx=0000000000000000 rdx=00000000000003ff rsi=00000000886e69b0 rdi=0000000019930520
r8=000000000000009c r9=0000000000000040 r10=00000000fab90cc0 r11=00000000faa20320 r12=00000000dd13eef0 r13=00000000886e6bc0
r14=0000000004303710 r15=0000000000000000 rip=00000000d3454ed9 rsp=00000000886e6850 rbp=00000000886e6b40
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
886e6920 d5a5a884 00000040 0000004f dd13f800 886e69e8 KERNELBASE!RaiseException+0x0
886e6980 d5801ca6 01a95540 8f849b2d 01a978b8 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886e6a30 d580593b 00000074 886e6c98 00000800 00000001 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886e6b80 d5801a78 00000000 00000000 886e6eb0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886e6c50 d5801cfc 886e6eb0 00000000 00000080 886e7180 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886e6d10 d580c4dd 886e7180 886e7180 886e7180 886e7180 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886e7000 d57fb2fe 886e7618 886e7180 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886ffda0 d5a3c6f8 fce238a0 00000000 00000000 fce4e0b0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886ffde0 d59a7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
886ffe10 d5d02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
886ffe90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 13440 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 88495448.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000888ffce0 rsi=0000000000000000 rdi=0000000000000064
r8=00000000888ffcb8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000d5d4d4a4 rsp=00000000888ffcb8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
888ffcb0 d34695be 888ffd78 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
888ffd50 d581a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
888ffd80 d59a7034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
888ffdb0 d5d02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
888ffe30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9156 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 88495448.000000
- Registers -
rax=000000000000005b rbx=0000000000000006 rcx=0000000000000006 rdx=00000000889ff980 rsi=0000000000000000 rdi=0000000000000006
r8=00000000889ff628 r9=00000000fd037980 r10=0000000000000000 r11=00000000889ff8f0 r12=0000000000000064 r13=00000000889ff980
r14=00000000889ff680 r15=0000000000000000 rip=00000000d5d4d974 rsp=00000000889ff628 rbp=00000000fd037980
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
889ff620 d346cab0 889ffb50 fac70000 a37f2160 a3721487 ntdll!ZwWaitForMultipleObjects+0x0
889ff910 d346c9ae faa2f470 00000000 00000006 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
889ff950 78a611f4 fd03a320 00000000 00000000 fb2bd290 KERNELBASE!WaitForMultipleObjects+0x0
889ffb90 78b07eed 00000000 fd037980 00000000 00000000 nvcuda64!+0x0
889ffc40 78a60bc3 72c6eedc 6ee55a22 fd618460 00000000 nvcuda64!cuProfilerStop+0x0
889ffc70 78d96294 fd618460 00000000 00000000 00000000 nvcuda64!+0x0
889ffca0 d59a7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
889ffcd0 d5d02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
889ffd50 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 15168 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 88495440.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000fd2bff80 rdx=0000000088aff708 rsi=0000000088aff670 rdi=0000000000000000
r8=0000000088aff648 r9=0000000088aff6a0 r10=0000000000000000 r11=0000000000000246 r12=0000000088aff708 r13=0000000000000000
r14=00000000fd2bff80 r15=00000000fd2c4730 rip=00000000d5d50814 rsp=0000000088aff648 rbp=0000000088aff6a0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
88aff640 d5d14295 00000102 00000102 00000000 88aff670 ntdll!NtWaitForAlertByThreadId+0x0
88aff6b0 d348cd79 fd2c1f10 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
88aff6e0 789cef6c fd2c1f10 fd2c1f10 fd2c1f10 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
88aff740 78a60bc3 00000005 721cbaa6 72b58225 00000000 nvcuda64!+0x0
88aff770 78d96294 fd6178c0 00000000 00000000 00000000 nvcuda64!+0x0
88aff7a0 d59a7034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
88aff7d0 d5d02651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
88aff850 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)