Task 5306274
| Name | ParityMachine-1622600713-19006-1_1 |
| Workunit | 3183607 |
| Created | 6 Jun 2021, 18:27:24 UTC |
| Sent | 6 Jun 2021, 22:59:07 UTC |
| Report deadline | 13 Jun 2021, 22:59:07 UTC |
| Received | 8 Jun 2021, 12:54:39 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 157 |
| Run time | 1 hours 49 min 56 sec |
| CPU time | 50 min 47 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 4,474.04 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.64 GB |
| Peak swap size | 3.62 GB |
| Peak disk usage | 1.53 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
37 main:484] : INFO : # Threads: 1
[2021-06-08 04:36:37 main:486] : INFO : Preparing Dataset
[2021-06-08 04:36:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 04:36:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 04:36:41 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-06-08 04:36:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 04:36:41 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 04:36:42 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-06-08 04:36:42 main:494] : INFO : Creating Model
[2021-06-08 04:36:42 main:507] : INFO : Preparing config file
[2021-06-08 04:36:42 main:519] : INFO : Creating new config file
[2021-06-08 04:36:42 main:538] : INFO : This is a continuation WU, loading previous network
[2021-06-08 04:36:48 main:559] : INFO : Loading DataLoader into Memory
[2021-06-08 04:36:48 main:562] : INFO : Starting Training
[2021-06-08 04:37:06 main:574] : INFO : Epoch 1 | loss: 0.0312651 | val_loss: 0.0312511 | Time: 18443 ms
[2021-06-08 04:37:21 main:574] : INFO : Epoch 2 | loss: 0.0312517 | val_loss: 0.0312505 | Time: 14909.3 ms
[2021-06-08 04:37:38 main:574] : INFO : Epoch 3 | loss: 0.0312507 | val_loss: 0.0312504 | Time: 16463.4 ms
[2021-06-08 04:37:53 main:574] : INFO : Epoch 4 | loss: 0.0312506 | val_loss: 0.03125 | Time: 15429.8 ms
[2021-06-08 04:38:09 main:574] : INFO : Epoch 5 | loss: 0.0312501 | val_loss: 0.0312505 | Time: 15342.2 ms
[2021-06-08 04:38:25 main:574] : INFO : Epoch 6 | loss: 0.031251 | val_loss: 0.0312513 | Time: 15831 ms
[2021-06-08 04:38:39 main:574] : INFO : Epoch 7 | loss: 0.0312502 | val_loss: 0.0312496 | Time: 14747.7 ms
[2021-06-08 04:38:56 main:574] : INFO : Epoch 8 | loss: 0.0312496 | val_loss: 0.0312452 | Time: 16036.5 ms
[2021-06-08 04:39:12 main:574] : INFO : Epoch 9 | loss: 0.0312453 | val_loss: 0.0312357 | Time: 16298.1 ms
[2021-06-08 04:39:28 main:574] : INFO : Epoch 10 | loss: 0.0312552 | val_loss: 0.0312579 | Time: 15882.9 ms
[2021-06-08 04:39:43 main:574] : INFO : Epoch 11 | loss: 0.0312336 | val_loss: 0.0312059 | Time: 15279 ms
[2021-06-08 04:39:59 main:574] : INFO : Epoch 12 | loss: 0.0311944 | val_loss: 0.0312022 | Time: 15396.4 ms
[2021-06-08 04:40:14 main:574] : INFO : Epoch 13 | loss: 0.0311933 | val_loss: 0.0312025 | Time: 15451.6 ms
[2021-06-08 04:40:29 main:574] : INFO : Epoch 14 | loss: 0.0311922 | val_loss: 0.0311991 | Time: 14895 ms
[2021-06-08 04:40:45 main:574] : INFO : Epoch 15 | loss: 0.0311987 | val_loss: 0.0312121 | Time: 16101.4 ms
[2021-06-08 04:41:01 main:574] : INFO : Epoch 16 | loss: 0.031193 | val_loss: 0.0311938 | Time: 15436 ms
[2021-06-08 04:41:17 main:574] : INFO : Epoch 17 | loss: 0.0311882 | val_loss: 0.0311891 | Time: 15916.9 ms
[2021-06-08 04:41:33 main:574] : INFO : Epoch 18 | loss: 0.0311895 | val_loss: 0.0311888 | Time: 15679.3 ms
[2021-06-08 04:41:49 main:574] : INFO : Epoch 19 | loss: 0.0311858 | val_loss: 0.0311894 | Time: 16025.1 ms
[2021-06-08 04:42:04 main:574] : INFO : Epoch 20 | loss: 0.031188 | val_loss: 0.0311885 | Time: 15519.6 ms
[2021-06-08 04:42:20 main:574] : INFO : Epoch 21 | loss: 0.0311872 | val_loss: 0.0311868 | Time: 15538.2 ms
[2021-06-08 04:42:36 main:574] : INFO : Epoch 22 | loss: 0.0311832 | val_loss: 0.0311843 | Time: 16165.9 ms
[2021-06-08 04:42:52 main:574] : INFO : Epoch 23 | loss: 0.0311806 | val_loss: 0.0311801 | Time: 15416.2 ms
[2021-06-08 04:43:08 main:574] : INFO : Epoch 24 | loss: 0.0311757 | val_loss: 0.0311757 | Time: 16190.7 ms
[2021-06-08 04:43:24 main:574] : INFO : Epoch 25 | loss: 0.0311689 | val_loss: 0.0311715 | Time: 15462.9 ms
[2021-06-08 04:43:39 main:574] : INFO : Epoch 26 | loss: 0.0311662 | val_loss: 0.0311698 | Time: 15381.9 ms
[2021-06-08 04:43:55 main:574] : INFO : Epoch 27 | loss: 0.031164 | val_loss: 0.0311687 | Time: 16060.5 ms
[2021-06-08 04:44:10 main:574] : INFO : Epoch 28 | loss: 0.031165 | val_loss: 0.0311695 | Time: 14753.7 ms
[2021-06-08 04:44:26 main:574] : INFO : Epoch 29 | loss: 0.0311643 | val_loss: 0.031165 | Time: 15802.3 ms
[2021-06-08 04:44:41 main:574] : INFO : Epoch 30 | loss: 0.0311623 | val_loss: 0.0311632 | Time: 15184.4 ms
[2021-06-08 04:44:57 main:574] : INFO : Epoch 31 | loss: 0.0311593 | val_loss: 0.0311665 | Time: 15882.4 ms
[2021-06-08 04:45:13 main:574] : INFO : Epoch 32 | loss: 0.0311556 | val_loss: 0.031162 | Time: 15490.5 ms
[2021-06-08 04:45:29 main:574] : INFO : Epoch 33 | loss: 0.0311539 | val_loss: 0.0311625 | Time: 16054.9 ms
[2021-06-08 04:45:44 main:574] : INFO : Epoch 34 | loss: 0.0311543 | val_loss: 0.0311641 | Time: 15556.8 ms
[2021-06-08 04:46:00 main:574] : INFO : Epoch 35 | loss: 0.0311543 | val_loss: 0.0311622 | Time: 15520.4 ms
[2021-06-08 04:46:15 main:574] : INFO : Epoch 36 | loss: 0.0311549 | val_loss: 0.0311752 | Time: 15334 ms
[2021-06-08 04:46:31 main:574] : INFO : Epoch 37 | loss: 0.0311552 | val_loss: 0.0311604 | Time: 15077 ms
[2021-06-08 04:46:46 main:574] : INFO : Epoch 38 | loss: 0.0311493 | val_loss: 0.0311606 | Time: 15840.4 ms
[2021-06-08 04:47:02 main:574] : INFO : Epoch 39 | loss: 0.0311474 | val_loss: 0.0311599 | Time: 15325.4 ms
[2021-06-08 04:47:18 main:574] : INFO : Epoch 40 | loss: 0.0311467 | val_loss: 0.0311631 | Time: 15955.4 ms
[2021-06-08 04:47:33 main:574] : INFO : Epoch 41 | loss: 0.03115 | val_loss: 0.0311677 | Time: 15124.8 ms
[2021-06-08 04:47:49 main:574] : INFO : Epoch 42 | loss: 0.0311464 | val_loss: 0.0311637 | Time: 16105.6 ms
[2021-06-08 04:48:05 main:574] : INFO : Epoch 43 | loss: 0.0311438 | val_loss: 0.0311673 | Time: 15739.9 ms
[2021-06-08 04:48:20 main:574] : INFO : Epoch 44 | loss: 0.0311473 | val_loss: 0.0311684 | Time: 15162 ms
[2021-06-08 04:48:36 main:574] : INFO : Epoch 45 | loss: 0.0311459 | val_loss: 0.0311634 | Time: 15892.5 ms
[2021-06-08 04:48:52 main:574] : INFO : Epoch 46 | loss: 0.0311407 | val_loss: 0.0311614 | Time: 15072 ms
[2021-06-08 04:49:08 main:574] : INFO : Epoch 47 | loss: 0.0311846 | val_loss: 0.031251 | Time: 16063.7 ms
[2021-06-08 04:49:23 main:574] : INFO : Epoch 48 | loss: nan | val_loss: nan | Time: 15413.4 ms
[2021-06-08 04:49:39 main:574] : INFO : Epoch 49 | loss: nan | val_loss: nan | Time: 15740 ms
[2021-06-08 04:49:55 main:574] : INFO : Epoch 50 | loss: nan | val_loss: nan | Time: 15463.5 ms
[2021-06-08 04:50:10 main:574] : INFO : Epoch 51 | loss: nan | val_loss: nan | Time: 15805.1 ms
[2021-06-08 04:50:26 main:574] : INFO : Epoch 52 | loss: nan | val_loss: nan | Time: 15709.6 ms
[2021-06-08 04:50:41 main:574] : INFO : Epoch 53 | loss: nan | val_loss: nan | Time: 14802 ms
[2021-06-08 04:50:58 main:574] : INFO : Epoch 54 | loss: nan | val_loss: nan | Time: 16614 ms
[2021-06-08 04:51:14 main:574] : INFO : Epoch 55 | loss: nan | val_loss: nan | Time: 15907.1 ms
[2021-06-08 04:51:30 main:574] : INFO : Epoch 56 | loss: nan | val_loss: nan | Time: 16073.2 ms
[2021-06-08 04:51:45 main:574] : INFO : Epoch 57 | loss: nan | val_loss: nan | Time: 15125.1 ms
[2021-06-08 04:52:01 main:574] : INFO : Epoch 58 | loss: nan | val_loss: nan | Time: 15497.1 ms
[2021-06-08 04:52:17 main:574] : INFO : Epoch 59 | loss: nan | val_loss: nan | Time: 15922.8 ms
[2021-06-08 04:52:32 main:574] : INFO : Epoch 60 | loss: nan | val_loss: nan | Time: 15620.1 ms
[2021-06-08 04:52:49 main:574] : INFO : Epoch 61 | loss: nan | val_loss: nan | Time: 16538.1 ms
[2021-06-08 04:53:04 main:574] : INFO : Epoch 62 | loss: nan | val_loss: nan | Time: 14905.5 ms
[2021-06-08 04:53:20 main:574] : INFO : Epoch 63 | loss: nan | val_loss: nan | Time: 16167.4 ms
[2021-06-08 04:53:36 main:574] : INFO : Epoch 64 | loss: nan | val_loss: nan | Time: 15637.9 ms
[2021-06-08 04:53:51 main:574] : INFO : Epoch 65 | loss: nan | val_loss: nan | Time: 15420.9 ms
[2021-06-08 04:54:08 main:574] : INFO : Epoch 66 | loss: nan | val_loss: nan | Time: 16460.8 ms
[2021-06-08 04:54:24 main:574] : INFO : Epoch 67 | loss: nan | val_loss: nan | Time: 15592.6 ms
[2021-06-08 04:54:40 main:574] : INFO : Epoch 68 | loss: nan | val_loss: nan | Time: 16221.2 ms
[2021-06-08 04:54:55 main:574] : INFO : Epoch 69 | loss: nan | val_loss: nan | Time: 15453.4 ms
[2021-06-08 04:55:12 main:574] : INFO : Epoch 70 | loss: nan | val_loss: nan | Time: 16003.1 ms
[2021-06-08 04:55:27 main:574] : INFO : Epoch 71 | loss: nan | val_loss: nan | Time: 15619.3 ms
[2021-06-08 04:55:43 main:574] : INFO : Epoch 72 | loss: nan | val_loss: nan | Time: 15286.7 ms
[2021-06-08 04:55:59 main:574] : INFO : Epoch 73 | loss: nan | val_loss: nan | Time: 16541.2 ms
[2021-06-08 04:56:15 main:574] : INFO : Epoch 74 | loss: nan | val_loss: nan | Time: 15761 ms
[2021-06-08 04:56:31 main:574] : INFO : Epoch 75 | loss: nan | val_loss: nan | Time: 15913.1 ms
[2021-06-08 04:56:47 main:574] : INFO : Epoch 76 | loss: nan | val_loss: nan | Time: 15421.5 ms
[2021-06-08 04:57:02 main:574] : INFO : Epoch 77 | loss: nan | val_loss: nan | Time: 15544.9 ms
[2021-06-08 04:57:18 main:574] : INFO : Epoch 78 | loss: nan | val_loss: nan | Time: 16023.7 ms
[2021-06-08 04:57:34 main:574] : INFO : Epoch 79 | loss: nan | val_loss: nan | Time: 15403.4 ms
[2021-06-08 04:57:50 main:574] : INFO : Epoch 80 | loss: nan | val_loss: nan | Time: 16411.6 ms
[2021-06-08 04:58:06 main:574] : INFO : Epoch 81 | loss: nan | val_loss: nan | Time: 15334 ms
[2021-06-08 04:58:22 main:574] : INFO : Epoch 82 | loss: nan | val_loss: nan | Time: 15969.7 ms
[2021-06-08 04:58:37 main:574] : INFO : Epoch 83 | loss: nan | val_loss: nan | Time: 15534.8 ms
[2021-06-08 04:58:53 main:574] : INFO : Epoch 84 | loss: nan | val_loss: nan | Time: 15542.9 ms
[2021-06-08 04:59:09 main:574] : INFO : Epoch 85 | loss: nan | val_loss: nan | Time: 16203.3 ms
[2021-06-08 04:59:25 main:574] : INFO : Epoch 86 | loss: nan | val_loss: nan | Time: 15846.3 ms
[2021-06-08 04:59:42 main:574] : INFO : Epoch 87 | loss: nan | val_loss: nan | Time: 16556.4 ms
[2021-06-08 04:59:57 main:574] : INFO : Epoch 88 | loss: nan | val_loss: nan | Time: 15601.4 ms
[2021-06-08 05:00:13 main:574] : INFO : Epoch 89 | loss: nan | val_loss: nan | Time: 15549.5 ms
[2021-06-08 05:00:32 main:574] : INFO : Epoch 90 | loss: nan | val_loss: nan | Time: 18861.1 ms
[2021-06-08 05:00:48 main:574] : INFO : Epoch 91 | loss: nan | val_loss: nan | Time: 15931 ms
[2021-06-08 05:01:04 main:574] : INFO : Epoch 92 | loss: nan | val_loss: nan | Time: 15965.2 ms
[2021-06-08 05:01:20 main:574] : INFO : Epoch 93 | loss: nan | val_loss: nan | Time: 16257.8 ms
[2021-06-08 05:01:36 main:574] : INFO : Epoch 94 | loss: nan | val_loss: nan | Time: 15884.5 ms
[2021-06-08 05:01:53 main:574] : INFO : Epoch 95 | loss: nan | val_loss: nan | Time: 16534.8 ms
[2021-06-08 05:02:09 main:574] : INFO : Epoch 96 | loss: nan | val_loss: nan | Time: 16511.8 ms
[2021-06-08 05:02:26 main:574] : INFO : Epoch 97 | loss: nan | val_loss: nan | Time: 16397.7 ms
[2021-06-08 05:02:42 main:574] : INFO : Epoch 98 | loss: nan | val_loss: nan | Time: 16432.9 ms
[2021-06-08 05:02:58 main:574] : INFO : Epoch 99 | loss: nan | val_loss: nan | Time: 15779.3 ms
[2021-06-08 05:03:14 main:574] : INFO : Epoch 100 | loss: nan | val_loss: nan | Time: 15411.6 ms
[2021-06-08 05:03:30 main:574] : INFO : Epoch 101 | loss: nan | val_loss: nan | Time: 15695.8 ms
[2021-06-08 05:03:45 main:574] : INFO : Epoch 102 | loss: nan | val_loss: nan | Time: 15532.7 ms
[2021-06-08 05:04:02 main:574] : INFO : Epoch 103 | loss: nan | val_loss: nan | Time: 16381.8 ms
[2021-06-08 05:04:17 main:574] : INFO : Epoch 104 | loss: nan | val_loss: nan | Time: 15546 ms
[2021-06-08 05:04:34 main:574] : INFO : Epoch 105 | loss: nan | val_loss: nan | Time: 16293.8 ms
[2021-06-08 05:04:49 main:574] : INFO : Epoch 106 | loss: nan | val_loss: nan | Time: 15255.9 ms
[2021-06-08 05:05:04 main:574] : INFO : Epoch 107 | loss: nan | val_loss: nan | Time: 15447.9 ms
[2021-06-08 05:05:21 main:574] : INFO : Epoch 108 | loss: nan | val_loss: nan | Time: 16520.7 ms
[2021-06-08 05:05:37 main:574] : INFO : Epoch 109 | loss: nan | val_loss: nan | Time: 15735.6 ms
[2021-06-08 05:05:53 main:574] : INFO : Epoch 110 | loss: nan | val_loss: nan | Time: 16472.2 ms
[2021-06-08 05:06:09 main:574] : INFO : Epoch 111 | loss: nan | val_loss: nan | Time: 15619 ms
[2021-06-08 05:06:25 main:574] : INFO : Epoch 112 | loss: nan | val_loss: nan | Time: 15633.4 ms
[2021-06-08 05:06:41 main:574] : INFO : Epoch 113 | loss: nan | val_loss: nan | Time: 15745.6 ms
[2021-06-08 05:06:56 main:574] : INFO : Epoch 114 | loss: nan | val_loss: nan | Time: 15002.7 ms
[2021-06-08 05:07:12 main:574] : INFO : Epoch 115 | loss: nan | val_loss: nan | Time: 16539.6 ms
[2021-06-08 05:07:28 main:574] : INFO : Epoch 116 | loss: nan | val_loss: nan | Time: 15473.5 ms
[2021-06-08 05:07:44 main:574] : INFO : Epoch 117 | loss: nan | val_loss: nan | Time: 16114.9 ms
[2021-06-08 05:07:59 main:574] : INFO : Epoch 118 | loss: nan | val_loss: nan | Time: 15353.9 ms
[2021-06-08 05:08:15 main:574] : INFO : Epoch 119 | loss: nan | val_loss: nan | Time: 15240.1 ms
[2021-06-08 05:08:31 main:574] : INFO : Epoch 120 | loss: nan | val_loss: nan | Time: 15984 ms
[2021-06-08 05:08:46 main:574] : INFO : Epoch 121 | loss: nan | val_loss: nan | Time: 15283.6 ms
[2021-06-08 05:09:03 main:574] : INFO : Epoch 122 | loss: nan | val_loss: nan | Time: 17001.8 ms
[2021-06-08 05:09:19 main:574] : INFO : Epoch 123 | loss: nan | val_loss: nan | Time: 15393.3 ms
[2021-06-08 05:09:35 main:574] : INFO : Epoch 124 | loss: nan | val_loss: nan | Time: 15909.1 ms
[2021-06-08 05:09:50 main:574] : INFO : Epoch 125 | loss: nan | val_loss: nan | Time: 15676.2 ms
[2021-06-08 05:10:06 main:574] : INFO : Epoch 126 | loss: nan | val_loss: nan | Time: 15424.1 ms
[2021-06-08 05:10:22 main:574] : INFO : Epoch 127 | loss: nan | val_loss: nan | Time: 16270.4 ms
[2021-06-08 05:10:38 main:574] : INFO : Epoch 128 | loss: nan | val_loss: nan | Time: 15507.6 ms
[2021-06-08 05:10:54 main:574] : INFO : Epoch 129 | loss: nan | val_loss: nan | Time: 15753.5 ms
[2021-06-08 05:11:09 main:574] : INFO : Epoch 130 | loss: nan | val_loss: nan | Time: 15658.4 ms
[2021-06-08 05:11:25 main:574] : INFO : Epoch 131 | loss: nan | val_loss: nan | Time: 15656.2 ms
[2021-06-08 05:11:41 main:574] : INFO : Epoch 132 | loss: nan | val_loss: nan | Time: 15470.8 ms
[2021-06-08 05:11:56 main:574] : INFO : Epoch 133 | loss: nan | val_loss: nan | Time: 15284.1 ms
[2021-06-08 05:12:13 main:574] : INFO : Epoch 134 | loss: nan | val_loss: nan | Time: 16462.1 ms
[2021-06-08 05:12:28 main:574] : INFO : Epoch 135 | loss: nan | val_loss: nan | Time: 15205.1 ms
[2021-06-08 05:12:44 main:574] : INFO : Epoch 136 | loss: nan | val_loss: nan | Time: 16031.3 ms
[2021-06-08 05:12:59 main:574] : INFO : Epoch 137 | loss: nan | val_loss: nan | Time: 15348.4 ms
[2021-06-08 05:13:15 main:574] : INFO : Epoch 138 | loss: nan | val_loss: nan | Time: 15658.2 ms
[2021-06-08 05:13:31 main:574] : INFO : Epoch 139 | loss: nan | val_loss: nan | Time: 15720.9 ms
[2021-06-08 05:13:46 main:574] : INFO : Epoch 140 | loss: nan | val_loss: nan | Time: 15521.2 ms
[2021-06-08 05:14:03 main:574] : INFO : Epoch 141 | loss: nan | val_loss: nan | Time: 16265.8 ms
[2021-06-08 05:14:18 main:574] : INFO : Epoch 142 | loss: nan | val_loss: nan | Time: 14782.8 ms
[2021-06-08 05:14:34 main:574] : INFO : Epoch 143 | loss: nan | val_loss: nan | Time: 15990.9 ms
[2021-06-08 05:14:49 main:574] : INFO : Epoch 144 | loss: nan | val_loss: nan | Time: 15051.6 ms
[2021-06-08 05:15:05 main:574] : INFO : Epoch 145 | loss: nan | val_loss: nan | Time: 15958.1 ms
[2021-06-08 05:15:20 main:574] : INFO : Epoch 146 | loss: nan | val_loss: nan | Time: 15365.9 ms
[2021-06-08 05:15:36 main:574] : INFO : Epoch 147 | loss: nan | val_loss: nan | Time: 15577.4 ms
[2021-06-08 05:15:52 main:574] : INFO : Epoch 148 | loss: nan | val_loss: nan | Time: 16073.6 ms
[2021-06-08 05:16:07 main:574] : INFO : Epoch 149 | loss: nan | val_loss: nan | Time: 15041.2 ms
[2021-06-08 05:16:23 main:574] : INFO : Epoch 150 | loss: nan | val_loss: nan | Time: 16124.6 ms
[2021-06-08 05:16:38 main:574] : INFO : Epoch 151 | loss: nan | val_loss: nan | Time: 14994.8 ms
[2021-06-08 05:16:54 main:574] : INFO : Epoch 152 | loss: nan | val_loss: nan | Time: 15863.3 ms
[2021-06-08 05:17:10 main:574] : INFO : Epoch 153 | loss: nan | val_loss: nan | Time: 15725.3 ms
[2021-06-08 05:17:26 main:574] : INFO : Epoch 154 | loss: nan | val_loss: nan | Time: 15783.1 ms
[2021-06-08 05:17:42 main:574] : INFO : Epoch 155 | loss: nan | val_loss: nan | Time: 15929 ms
[2021-06-08 05:17:58 main:574] : INFO : Epoch 156 | loss: nan | val_loss: nan | Time: 15520.4 ms
[2021-06-08 05:18:14 main:574] : INFO : Epoch 157 | loss: nan | val_loss: nan | Time: 16009.7 ms
[2021-06-08 05:18:29 main:574] : INFO : Epoch 158 | loss: nan | val_loss: nan | Time: 15441.3 ms
[2021-06-08 05:18:46 main:574] : INFO : Epoch 159 | loss: nan | val_loss: nan | Time: 16509.8 ms
[2021-06-08 05:19:02 main:574] : INFO : Epoch 160 | loss: nan | val_loss: nan | Time: 16007.6 ms
[2021-06-08 05:19:17 main:574] : INFO : Epoch 161 | loss: nan | val_loss: nan | Time: 15352.1 ms
[2021-06-08 05:19:33 main:574] : INFO : Epoch 162 | loss: nan | val_loss: nan | Time: 15943 ms
[2021-06-08 05:19:49 main:574] : INFO : Epoch 163 | loss: nan | val_loss: nan | Time: 15918.5 ms
[2021-06-08 05:20:06 main:574] : INFO : Epoch 164 | loss: nan | val_loss: nan | Time: 16139.2 ms
[2021-06-08 05:20:21 main:574] : INFO : Epoch 165 | loss: nan | val_loss: nan | Time: 15542.9 ms
[2021-06-08 05:20:38 main:574] : INFO : Epoch 166 | loss: nan | val_loss: nan | Time: 16328.2 ms
[2021-06-08 05:20:54 main:574] : INFO : Epoch 167 | loss: nan | val_loss: nan | Time: 16517.7 ms
[2021-06-08 05:21:09 main:574] : INFO : Epoch 168 | loss: nan | val_loss: nan | Time: 15189.1 ms
[2021-06-08 05:21:26 main:574] : INFO : Epoch 169 | loss: nan | val_loss: nan | Time: 16300.3 ms
[2021-06-08 05:21:41 main:574] : INFO : Epoch 170 | loss: nan | val_loss: nan | Time: 15340.4 ms
[2021-06-08 05:21:57 main:574] : INFO : Epoch 171 | loss: nan | val_loss: nan | Time: 16096.3 ms
[2021-06-08 05:22:14 main:574] : INFO : Epoch 172 | loss: nan | val_loss: nan | Time: 16658.7 ms
[2021-06-08 05:22:30 main:574] : INFO : Epoch 173 | loss: nan | val_loss: nan | Time: 16107 ms
[2021-06-08 05:22:47 main:574] : INFO : Epoch 174 | loss: nan | val_loss: nan | Time: 16701.6 ms
[2021-06-08 05:23:03 main:574] : INFO : Epoch 175 | loss: nan | val_loss: nan | Time: 15854 ms
[2021-06-08 05:23:18 main:574] : INFO : Epoch 176 | loss: nan | val_loss: nan | Time: 15180.2 ms
[2021-06-08 05:23:34 main:574] : INFO : Epoch 177 | loss: nan | val_loss: nan | Time: 15889.8 ms
[2021-06-08 05:23:50 main:574] : INFO : Epoch 178 | loss: nan | val_loss: nan | Time: 15542.7 ms
[2021-06-08 05:24:07 main:574] : INFO : Epoch 179 | loss: nan | val_loss: nan | Time: 17219.9 ms
[2021-06-08 05:24:23 main:574] : INFO : Epoch 180 | loss: nan | val_loss: nan | Time: 15808.7 ms
[2021-06-08 05:24:39 main:574] : INFO : Epoch 181 | loss: nan | val_loss: nan | Time: 15548.2 ms
[2021-06-08 05:24:55 main:574] : INFO : Epoch 182 | loss: nan | val_loss: nan | Time: 16519.3 ms
[2021-06-08 05:25:11 main:574] : INFO : Epoch 183 | loss: nan | val_loss: nan | Time: 16023.1 ms
[2021-06-08 05:25:28 main:574] : INFO : Epoch 184 | loss: nan | val_loss: nan | Time: 16450.1 ms
[2021-06-08 05:25:44 main:574] : INFO : Epoch 185 | loss: nan | val_loss: nan | Time: 15989.8 ms
[2021-06-08 05:25:59 main:574] : INFO : Epoch 186 | loss: nan | val_loss: nan | Time: 15301.8 ms
[2021-06-08 05:26:15 main:574] : INFO : Epoch 187 | loss: nan | val_loss: nan | Time: 16122.4 ms
[2021-06-08 05:26:31 main:574] : INFO : Epoch 188 | loss: nan | val_loss: nan | Time: 15476.7 ms
[2021-06-08 05:26:48 main:574] : INFO : Epoch 189 | loss: nan | val_loss: nan | Time: 16333.2 ms
[2021-06-08 05:27:02 main:574] : INFO : Epoch 190 | loss: nan | val_loss: nan | Time: 14336.2 ms
[2021-06-08 05:27:19 main:574] : INFO : Epoch 191 | loss: nan | val_loss: nan | Time: 17375.2 ms
[2021-06-08 05:27:38 main:574] : INFO : Epoch 192 | loss: nan | val_loss: nan | Time: 18331.7 ms
[2021-06-08 05:27:56 main:574] : INFO : Epoch 193 | loss: nan | val_loss: nan | Time: 17948.4 ms
[2021-06-08 05:28:14 main:574] : INFO : Epoch 194 | loss: nan | val_loss: nan | Time: 17999.4 ms
[2021-06-08 05:28:32 main:574] : INFO : Epoch 195 | loss: nan | val_loss: nan | Time: 18148.1 ms
[2021-06-08 05:28:50 main:574] : INFO : Epoch 196 | loss: nan | val_loss: nan | Time: 17840.1 ms
[2021-06-08 05:29:09 main:574] : INFO : Epoch 197 | loss: nan | val_loss: nan | Time: 18463.2 ms
[2021-06-08 05:29:27 main:574] : INFO : Epoch 198 | loss: nan | val_loss: nan | Time: 18122.1 ms
[2021-06-08 05:29:45 main:574] : INFO : Epoch 199 | loss: nan | val_loss: nan | Time: 18011.2 ms
[2021-06-08 05:30:03 main:574] : INFO : Epoch 200 | loss: nan | val_loss: nan | Time: 18206.6 ms
[2021-06-08 05:30:22 main:574] : INFO : Epoch 201 | loss: nan | val_loss: nan | Time: 18475 ms
[2021-06-08 05:30:40 main:574] : INFO : Epoch 202 | loss: nan | val_loss: nan | Time: 17937.4 ms
[2021-06-08 05:30:58 main:574] : INFO : Epoch 203 | loss: nan | val_loss: nan | Time: 18228 ms
[2021-06-08 05:31:17 main:574] : INFO : Epoch 204 | loss: nan | val_loss: nan | Time: 18912.4 ms
[2021-06-08 05:31:36 main:574] : INFO : Epoch 205 | loss: nan | val_loss: nan | Time: 18801.5 ms
[2021-06-08 05:31:54 main:574] : INFO : Epoch 206 | loss: nan | val_loss: nan | Time: 18198.7 ms
[2021-06-08 05:32:11 main:574] : INFO : Epoch 207 | loss: nan | val_loss: nan | Time: 17222 ms
[2021-06-08 05:32:30 main:574] : INFO : Epoch 208 | loss: nan | val_loss: nan | Time: 18164.9 ms
[2021-06-08 05:32:48 main:574] : INFO : Epoch 209 | loss: nan | val_loss: nan | Time: 18196.9 ms
[2021-06-08 05:33:06 main:574] : INFO : Epoch 210 | loss: nan | val_loss: nan | Time: 18344.7 ms
[2021-06-08 05:33:23 main:574] : INFO : Epoch 211 | loss: nan | val_loss: nan | Time: 16959.5 ms
[2021-06-08 05:33:47 main:574] : INFO : Epoch 212 | loss: nan | val_loss: nan | Time: 24102.5 ms
[2021-06-08 05:34:12 main:574] : INFO : Epoch 213 | loss: nan | val_loss: nan | Time: 24810.3 ms
[2021-06-08 05:34:38 main:574] : INFO : Epoch 214 | loss: nan | val_loss: nan | Time: 25446.8 ms
[2021-06-08 05:35:03 main:574] : INFO : Epoch 215 | loss: nan | val_loss: nan | Time: 25278.9 ms
[2021-06-08 05:35:31 main:574] : INFO : Epoch 216 | loss: nan | val_loss: nan | Time: 27519.8 ms
[2021-06-08 05:35:56 main:574] : INFO : Epoch 217 | loss: nan | val_loss: nan | Time: 24656.6 ms
[2021-06-08 05:36:22 main:574] : INFO : Epoch 218 | loss: nan | val_loss: nan | Time: 26389.2 ms
[2021-06-08 05:36:49 main:574] : INFO : Epoch 219 | loss: nan | val_loss: nan | Time: 26626.2 ms
[2021-06-08 05:37:13 main:574] : INFO : Epoch 220 | loss: nan | val_loss: nan | Time: 24607.3 ms
[2021-06-08 05:37:40 main:574] : INFO : Epoch 221 | loss: nan | val_loss: nan | Time: 26697.3 ms
[2021-06-08 05:38:07 main:574] : INFO : Epoch 222 | loss: nan | val_loss: nan | Time: 26160.8 ms
[2021-06-08 05:38:33 main:574] : INFO : Epoch 223 | loss: nan | val_loss: nan | Time: 26078 ms
[2021-06-08 05:38:58 main:574] : INFO : Epoch 224 | loss: nan | val_loss: nan | Time: 25209.6 ms
[2021-06-08 05:39:25 main:574] : INFO : Epoch 225 | loss: nan | val_loss: nan | Time: 27257.5 ms
[2021-06-08 05:39:52 main:574] : INFO : Epoch 226 | loss: nan | val_loss: nan | Time: 26090.9 ms
[2021-06-08 05:40:17 main:574] : INFO : Epoch 227 | loss: nan | val_loss: nan | Time: 25524.7 ms
[2021-06-08 05:40:44 main:574] : INFO : Epoch 228 | loss: nan | val_loss: nan | Time: 26353 ms
[2021-06-08 05:41:10 main:574] : INFO : Epoch 229 | loss: nan | val_loss: nan | Time: 25964.2 ms
[2021-06-08 05:41:34 main:574] : INFO : Epoch 230 | loss: nan | val_loss: nan | Time: 24772.9 ms
[2021-06-08 05:42:00 main:574] : INFO : Epoch 231 | loss: nan | val_loss: nan | Time: 25638.1 ms
[2021-06-08 05:42:27 main:574] : INFO : Epoch 232 | loss: nan | val_loss: nan | Time: 26314.5 ms
[2021-06-08 05:42:52 main:574] : INFO : Epoch 233 | loss: nan | val_loss: nan | Time: 25587.8 ms
[2021-06-08 05:43:18 main:574] : INFO : Epoch 234 | loss: nan | val_loss: nan | Time: 25902.6 ms
[2021-06-08 05:43:44 main:574] : INFO : Epoch 235 | loss: nan | val_loss: nan | Time: 25491.6 ms
[2021-06-08 05:44:10 main:574] : INFO : Epoch 236 | loss: nan | val_loss: nan | Time: 26461 ms
[2021-06-08 05:44:35 main:574] : INFO : Epoch 237 | loss: nan | val_loss: nan | Time: 25020.7 ms
[2021-06-08 05:45:01 main:574] : INFO : Epoch 238 | loss: nan | val_loss: nan | Time: 25856.1 ms
[2021-06-08 05:45:28 main:574] : INFO : Epoch 239 | loss: nan | val_loss: nan | Time: 26212.6 ms
[2021-06-08 05:45:54 main:574] : INFO : Epoch 240 | loss: nan | val_loss: nan | Time: 26035.4 ms
[2021-06-08 05:46:19 main:574] : INFO : Epoch 241 | loss: nan | val_loss: nan | Time: 25237.4 ms
[2021-06-08 05:46:44 main:574] : INFO : Epoch 242 | loss: nan | val_loss: nan | Time: 25353.4 ms
[2021-06-08 05:47:10 main:574] : INFO : Epoch 243 | loss: nan | val_loss: nan | Time: 25569.2 ms
[2021-06-08 05:47:37 main:574] : INFO : Epoch 244 | loss: nan | val_loss: nan | Time: 27121.1 ms
[2021-06-08 05:48:03 main:574] : INFO : Epoch 245 | loss: nan | val_loss: nan | Time: 26140.2 ms
[2021-06-08 05:48:30 main:574] : INFO : Epoch 246 | loss: nan | val_loss: nan | Time: 26144.5 ms
[2021-06-08 05:48:55 main:574] : INFO : Epoch 247 | loss: nan | val_loss: nan | Time: 25440.1 ms
[2021-06-08 05:49:18 main:574] : INFO : Epoch 248 | loss: nan | val_loss: nan | Time: 23072.4 ms
[2021-06-08 05:49:32 main:574] : INFO : Epoch 249 | loss: nan | val_loss: nan | Time: 13537.2 ms
[2021-06-08 05:52:40 main:574] : INFO : Epoch 250 | loss: nan | val_loss: nan | Time: 185372 ms
[2021-06-08 05:55:39 main:574] : INFO : Epoch 251 | loss: nan | val_loss: nan | Time: 178443 ms
[2021-06-08 05:56:10 main:574] : INFO : Epoch 252 | loss: nan | val_loss: nan | Time: 30667.3 ms
[2021-06-08 05:56:37 main:574] : INFO : Epoch 253 | loss: nan | val_loss: nan | Time: 27826.5 ms
[2021-06-08 05:57:29 main:574] : INFO : Epoch 254 | loss: nan | val_loss: nan | Time: 51004.2 ms
[2021-06-08 05:58:22 main:574] : INFO : Epoch 255 | loss: nan | val_loss: nan | Time: 53459.5 ms
[2021-06-08 06:00:23 main:574] : INFO : Epoch 256 | loss: nan | val_loss: nan | Time: 121050 ms
[2021-06-08 06:02:34 main:574] : INFO : Epoch 257 | loss: nan | val_loss: nan | Time: 130538 ms
[2021-06-08 06:06:06 main:574] : INFO : Epoch 258 | loss: nan | val_loss: nan | Time: 206246 ms
[2021-06-08 06:09:08 main:574] : INFO : Epoch 259 | loss: nan | val_loss: nan | Time: 181867 ms
[2021-06-08 06:11:03 main:574] : INFO : Epoch 260 | loss: nan | val_loss: nan | Time: 115360 ms
[2021-06-08 06:12:51 main:574] : INFO : Epoch 261 | loss: nan | val_loss: nan | Time: 107721 ms
[2021-06-08 06:15:40 main:574] : INFO : Epoch 262 | loss: nan | val_loss: nan | Time: 168467 ms
[2021-06-08 06:17:29 main:574] : INFO : Epoch 263 | loss: nan | val_loss: nan | Time: 109412 ms
[2021-06-08 06:20:56 main:574] : INFO : Epoch 264 | loss: nan | val_loss: nan | Time: 207023 ms
[2021-06-08 06:22:18 main:574] : INFO : Epoch 265 | loss: nan | val_loss: nan | Time: 81350.5 ms
[2021-06-08 06:25:46 main:574] : INFO : Epoch 266 | loss: nan | val_loss: nan | Time: 208197 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1060 6GB)
[2021-06-08 06:53:52 main:435] : INFO : Set logging level to 1
[2021-06-08 06:53:52 main:441] : INFO : Running in BOINC Client mode
[2021-06-08 06:53:52 main:444] : INFO : Resolving all filenames
[2021-06-08 06:53:52 main:452] : INFO : Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 (exists = 1)
[2021-06-08 06:53:52 main:452] : INFO : Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1622600713-19006-1_1_r1625013431_1 (exists = 1)
[2021-06-08 06:53:52 main:452] : INFO : Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1622600713-19006-1_1_r1625013431_0 (exists = 0)
[2021-06-08 06:53:52 main:452] : INFO : Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityMachine-1622600713-19006-1 (exists = 1)
[2021-06-08 06:53:52 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-06-08 06:53:53 main:472] : INFO : Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5
[2021-06-08 06:53:58 main:474] : INFO : Configuration:
[2021-06-08 06:53:58 main:475] : INFO : Model type: GRU
[2021-06-08 06:53:58 main:476] : INFO : Validation Loss Threshold: 0.0001
[2021-06-08 06:53:58 main:477] : INFO : Max Epochs: 2048
[2021-06-08 06:53:58 main:478] : INFO : Batch Size: 128
[2021-06-08 06:53:58 main:479] : INFO : Learning Rate: 0.01
[2021-06-08 06:53:58 main:480] : INFO : Patience: 10
[2021-06-08 06:53:58 main:481] : INFO : Hidden Width: 12
[2021-06-08 06:53:58 main:482] : INFO : # Recurrent Layers: 4
[2021-06-08 06:53:58 main:483] : INFO : # Backend Layers: 4
[2021-06-08 06:53:58 main:484] : INFO : # Threads: 1
[2021-06-08 06:53:59 main:486] : INFO : Preparing Dataset
[2021-06-08 06:53:59 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 06:54:04 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 06:54:12 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2021-06-08 06:54:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 06:54:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityMachine-train-val-dataset.hdf5 into memory
[2021-06-08 06:54:18 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2021-06-08 06:54:18 main:494] : INFO : Creating Model
[2021-06-08 06:54:18 main:507] : INFO : Preparing config file
[2021-06-08 06:54:18 main:511] : INFO : Found checkpoint, attempting to load...
[2021-06-08 06:54:18 main:512] : INFO : Loading config
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFFCD0D4B89
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 06/08/21 06:54:18
Install Directory : D:\Program Files\
Data Directory : D:\ProgramData
Project Symstore :
LoadLibraryA( D:\ProgramData\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( D:\ProgramData\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: D:\ProgramData\slots\5;D:\ProgramData\projects\www.mlcathome.org_mlcathome
ModLoad: 0000000070910000 00000000003df000 D:\ProgramData\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000cf470000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 00000000cd570000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 00000000cd0a0000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 00000000a7940000 00000000000f1000 C:\Program Files\Bitdefender\Bitdefender Security\atcuf\265267636905997236\atcuf64.dll (1.40.291.0) (-exported- Symbols Loaded)
Linked PDB Filename : E:\builds\ATC-DEFAULT-SOURCES\bin\x64\Release\atcuf64\atcuf64.pdb
File Version : 1.40.291.0 #0xd59eaa1
Company Name : Bitdefender S.R.L. Bucharest, ROMANIA
Product Name : Bitdefender® ATC
Product Version : 13
ModLoad: 00000000ceae0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cd370000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.985) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.985 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.985
ModLoad: 00000000ce790000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.746 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.746
ModLoad: 00000000ccdb0000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.928 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.928
ModLoad: 00000000ccd10000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000ccc10000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000ceed0000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cdb10000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 00000000ced50000 000000000009b000 C:\WINDOWS\System32\sechost.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000ce9b0000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000c6bf0000 0000000000058000 D:\ProgramData\slots\5\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000000000000 0000000024b49000 D:\ProgramData\slots\5\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000003a190000 000000000abe7000 D:\ProgramData\slots\5\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000c6b00000 000000000008d000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000c6490000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 00000000c88d0000 000000000001a000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000c88c0000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29910.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29910.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29910.0
ModLoad: 00000000c7190000 0000000000045000 D:\ProgramData\slots\5\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000078b20000 00000000030a5000 D:\ProgramData\slots\5\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000075660000 00000000034b2000 D:\ProgramData\slots\5\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 000000001ede0000 000000001b3a4000 D:\ProgramData\slots\5\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000c72a0000 0000000000010000 D:\ProgramData\slots\5\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000015c40000 000000000919f000 D:\ProgramData\slots\5\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000011ee0000 0000000003d5b000 D:\ProgramData\slots\5\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000a0d50000 000000000020d000 D:\ProgramData\slots\5\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 000000008e100000 00000000002db000 D:\ProgramData\slots\5\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000006e2a0000 0000000001eca000 D:\ProgramData\slots\5\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000c6ac0000 0000000000040000 D:\ProgramData\slots\5\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000cda70000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000cab20000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000cdbb0000 0000000000469000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000ccec0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000ccf10000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cc8f0000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000ccbb0000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.804) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.804 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.804
ModLoad: 00000000ccf40000 000000000015f000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.844) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cc730000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000ce020000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 00000000cad20000 0000000000790000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cd6b0000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.928) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cc550000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cf250000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.964) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cecf0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000886f0000 000000000029d000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.2.135 driver
Product Version : 27.21.14.6140
ModLoad: 00000000c7c90000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000010910000 00000000015c2000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispsi.inf_amd64_5e5ddf14f45dc8f6\nvcuda64.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.2.135 driver
Product Version : 27.21.14.6140
ModLoad: 00000000cec80000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000c5280000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 00000000cc4b0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000ca660000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000b6140000 00000000006f1000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.6140) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r460\r461_36\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 27.21.14.6140
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 27.21.14.6140
ModLoad: 00000000cf300000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.746) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000cd450000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.662 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.662
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3620, Write: 309, Other 885
- I/O Transfers Counters -
Read: 6566596, Write: 7289, Other 38732
- Paged Pool Usage -
QuotaPagedPoolUsage: 3442384, QuotaPeakPagedPoolUsage: 3442384
QuotaNonPagedPoolUsage: 41272, QuotaPeakNonPagedPoolUsage: 41400
- Virtual Memory Usage -
VirtualSize: -1422434304, PeakVirtualSize: -816451584
- Pagefile Usage -
PagefileUsage: -1422434304, PeakPagefileUsage: -1422434304
- Working Set Size -
WorkingSetSize: 1164685312, PeakWorkingSetSize: 1164689408, PageFaultCount: 397814
*** Dump of thread ID 24392 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 10312500.000000, User Time: 28750000.000000, Wait Time: 100945208.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFFCD0D4B89
- Registers -
rax=000000007c1b1140 rbx=0000000070c74910 rcx=0000000000000000 rdx=00000000cf47a36f rsi=00000000bf3b6970 rdi=0000000019930520
r8=0000000000000050 r9=0000000000000040 r10=000000007c1b0cc0 r11=0000000000000141 r12=000000005cacd350 r13=00000000bf3b6b80
r14=00000000734d93f0 r15=0000000000000000 rip=00000000cd0d4b89 rsp=00000000bf3b6810 rbp=00000000bf3b6b00
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
bf3b68e0 70b8a884 00000040 0000004f 5cacc200 bf3b69a8 KERNELBASE!RaiseException+0x0
bf3b6940 70931ca6 7c9b0a50 70bafd48 7b5c9b18 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3b69f0 7093593b bf3b6aa8 bf3b6bb9 bf3b6e70 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3b6b40 70931a78 00000000 00000000 bf3b6e70 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3b6c10 70931cfc bf3b6e70 00000000 00000080 bf3b7140 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3b6cd0 7093c4dd bf3b7140 bf3b7140 bf3b7140 bf3b7140 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3b6fc0 7092b2fe bf3b75d8 bf3b7140 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3cfd60 70b6c6f8 7e3d2360 00000000 00000000 7e3fd5b0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3cfda0 cd587034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf3cfdd0 cf4c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
bf3cfe50 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 12820 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 100945200.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000bf7ff8c0 rsi=0000000000000000 rdi=0000000000000064
r8=00000000ffffffff r9=0000000070bc86e0 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000cf50d4a4 rsp=00000000bf7ff898 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
bf7ff890 cd0e90ae bf7ff958 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
bf7ff930 7094a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
bf7ff960 cd587034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
bf7ff990 cf4c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
bf7ffa10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 24068 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 100945200.000000
- Registers -
rax=000000000000005b rbx=0000000000000006 rcx=0000000000000006 rdx=00000000bf8ff5c0 rsi=0000000000000000 rdi=0000000000000006
r8=0000000000000001 r9=000000007e5e6a00 r10=0000000000000000 r11=00000000bf8ff530 r12=0000000000000064 r13=00000000bf8ff5c0
r14=00000000bf8ff2c0 r15=0000000000000000 rip=00000000cf50d974 rsp=00000000bf8ff268 rbp=000000007e5e6a80
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
bf8ff260 cd0ec5a0 bf8ff790 7c2d0000 a7a22160 a7951487 ntdll!ZwWaitForMultipleObjects+0x0
bf8ff550 cd0ec49e 7c060770 00000000 00000006 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
bf8ff590 10ac6714 7e5e8ee0 00000000 00000000 7c9bd080 KERNELBASE!WaitForMultipleObjects+0x0
bf8ff7d0 10b5e55d 00000000 7e5e6a80 00000000 00000000 nvcuda64!+0x0
bf8ff880 10ac60e3 5e331de7 555b2221 7ebc7770 00000000 nvcuda64!cuProfilerStop+0x0
bf8ff8b0 10ddd9d4 7ebc7770 00000000 00000000 00000000 nvcuda64!+0x0
bf8ff8e0 cd587034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
bf8ff910 cf4c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
bf8ff990 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 16548 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 100945176.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=000000007e86f298 rdx=00000000bf9ffca8 rsi=00000000bf9ffc10 rdi=0000000000000000
r8=0000000000000004 r9=0000000000000004 r10=0000000000000003 r11=00000000bf9ffc40 r12=00000000bf9ffca8 r13=0000000000000000
r14=000000007e86f298 r15=000000007e873e70 rip=00000000cf510814 rsp=00000000bf9ffbe8 rbp=00000000bf9ffc40
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
bf9ffbe0 cf4d4295 00000102 00000102 00000000 bf9ffc10 ntdll!NtWaitForAlertByThreadId+0x0
bf9ffc50 cd10c849 7e871650 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
bf9ffc80 10a44a9c 7e871650 7e871650 7e871650 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
bf9ffce0 10ac60e3 00000004 5d9cebb3 5e3a44d5 00000000 nvcuda64!+0x0
bf9ffd10 10ddd9d4 7ebc7230 00000000 00000000 00000000 nvcuda64!+0x0
bf9ffd40 cd587034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
bf9ffd70 cf4c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
bf9ffdf0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)