Task 13709453
| Name | ParityModified-1647041704-22546-2-0_0 |
| Workunit | 10799282 |
| Created | 24 Mar 2022, 13:34:57 UTC |
| Sent | 31 Mar 2022, 11:11:29 UTC |
| Report deadline | 8 Apr 2022, 11:11:29 UTC |
| Received | 4 Apr 2022, 13:25:19 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED |
| Computer ID | 4503 |
| Run time | 8 hours 36 min 13 sec |
| CPU time | 15 min 43 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 10,073.24 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.37 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
exceeded elapsed time limit 30969.09 (4000000.00G/129.16G)</message>
<stderr_txt>
Batch Size: 128
[2022-04-04 01:15:37 main:479] : INFO : Learning Rate: 0.01
[2022-04-04 01:15:37 main:480] : INFO : Patience: 10
[2022-04-04 01:15:37 main:481] : INFO : Hidden Width: 12
[2022-04-04 01:15:37 main:482] : INFO : # Recurrent Layers: 4
[2022-04-04 01:15:37 main:483] : INFO : # Backend Layers: 4
[2022-04-04 01:15:37 main:484] : INFO : # Threads: 1
[2022-04-04 01:15:37 main:486] : INFO : Preparing Dataset
[2022-04-04 01:15:37 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-04-04 01:15:38 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-04-04 01:15:42 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-04-04 01:15:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-04-04 01:15:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-04-04 01:15:46 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-04-04 01:15:49 main:494] : INFO : Creating Model
[2022-04-04 01:15:49 main:507] : INFO : Preparing config file
[2022-04-04 01:15:51 main:511] : INFO : Found checkpoint, attempting to load...
[2022-04-04 01:15:52 main:512] : INFO : Loading config
[2022-04-04 01:15:52 main:514] : INFO : Loading state
[2022-04-04 01:15:56 main:559] : INFO : Loading DataLoader into Memory
[2022-04-04 01:16:13 main:562] : INFO : Starting Training
[2022-04-04 01:18:57 main:574] : INFO : Epoch 105 | loss: 0.031234 | val_loss: 0.0312025 | Time: 163903 ms
[2022-04-04 01:19:29 main:574] : INFO : Epoch 106 | loss: 0.0311662 | val_loss: 0.0311803 | Time: 26851 ms
[2022-04-04 01:21:20 main:574] : INFO : Epoch 107 | loss: 0.031155 | val_loss: 0.0311744 | Time: 111337 ms
[2022-04-04 01:22:52 main:574] : INFO : Epoch 108 | loss: 0.0311516 | val_loss: 0.0311746 | Time: 92153 ms
[2022-04-04 01:25:45 main:574] : INFO : Epoch 109 | loss: 0.0311515 | val_loss: 0.0311763 | Time: 172172 ms
[2022-04-04 01:28:38 main:574] : INFO : Epoch 110 | loss: 0.0311513 | val_loss: 0.0311737 | Time: 173372 ms
[2022-04-04 01:30:10 main:574] : INFO : Epoch 111 | loss: 0.0311494 | val_loss: 0.0311759 | Time: 92173.5 ms
[2022-04-04 01:32:13 main:574] : INFO : Epoch 112 | loss: 0.0311495 | val_loss: 0.0311746 | Time: 117736 ms
[2022-04-04 01:33:50 main:574] : INFO : Epoch 113 | loss: 0.0311533 | val_loss: 0.0311783 | Time: 91829.7 ms
[2022-04-04 01:35:02 main:574] : INFO : Epoch 114 | loss: 0.0311506 | val_loss: 0.0311793 | Time: 66053 ms
[2022-04-04 01:37:21 main:574] : INFO : Epoch 115 | loss: 0.0311499 | val_loss: 0.0311777 | Time: 133081 ms
[2022-04-04 01:38:25 main:574] : INFO : Epoch 116 | loss: 0.0311494 | val_loss: 0.031174 | Time: 58622.9 ms
[2022-04-04 01:41:53 main:574] : INFO : Epoch 117 | loss: 0.0311507 | val_loss: 0.031176 | Time: 207980 ms
[2022-04-04 01:42:59 main:574] : INFO : Epoch 118 | loss: 0.0311499 | val_loss: 0.03118 | Time: 60760.5 ms
[2022-04-04 01:46:58 main:574] : INFO : Epoch 119 | loss: 0.0311548 | val_loss: 0.0311784 | Time: 233963 ms
[2022-04-04 01:49:33 main:574] : INFO : Epoch 120 | loss: 0.0311599 | val_loss: 0.0311743 | Time: 150450 ms
[2022-04-04 01:50:55 main:574] : INFO : Epoch 121 | loss: 0.0311581 | val_loss: 0.0311748 | Time: 81421.2 ms
[2022-04-04 01:52:49 main:574] : INFO : Epoch 122 | loss: 0.0311549 | val_loss: 0.0311721 | Time: 114261 ms
[2022-04-04 01:55:06 main:574] : INFO : Epoch 123 | loss: 0.0311521 | val_loss: 0.0311727 | Time: 137146 ms
[2022-04-04 01:57:38 main:574] : INFO : Epoch 124 | loss: 0.0311499 | val_loss: 0.0311723 | Time: 151825 ms
[2022-04-04 02:00:06 main:574] : INFO : Epoch 125 | loss: 0.03115 | val_loss: 0.031177 | Time: 143420 ms
[2022-04-04 02:01:14 main:574] : INFO : Epoch 126 | loss: 0.0311504 | val_loss: 0.0311761 | Time: 62158.8 ms
[2022-04-04 02:03:25 main:574] : INFO : Epoch 127 | loss: 0.0311506 | val_loss: 0.0311746 | Time: 131157 ms
[2022-04-04 02:03:28 main:574] : INFO : Epoch 128 | loss: 0.0311502 | val_loss: 0.0311826 | Time: 3533.4 ms
[2022-04-04 02:03:34 main:574] : INFO : Epoch 129 | loss: 0.0311489 | val_loss: 0.031173 | Time: 5980.66 ms
[2022-04-04 02:05:12 main:574] : INFO : Epoch 130 | loss: 0.0311485 | val_loss: 0.0311745 | Time: 90817.8 ms
[2022-04-04 02:05:16 main:574] : INFO : Epoch 131 | loss: 0.0311488 | val_loss: 0.0311774 | Time: 3703.84 ms
[2022-04-04 02:05:57 main:574] : INFO : Epoch 132 | loss: 0.0311543 | val_loss: 0.0311765 | Time: 41130.4 ms
[2022-04-04 02:08:19 main:574] : INFO : Epoch 133 | loss: 0.0311507 | val_loss: 0.0311783 | Time: 141979 ms
[2022-04-04 02:11:17 main:574] : INFO : Epoch 134 | loss: 0.0311521 | val_loss: 0.0311768 | Time: 177053 ms
[2022-04-04 02:13:41 main:574] : INFO : Epoch 135 | loss: 0.0311514 | val_loss: 0.0311748 | Time: 136559 ms
[2022-04-04 02:16:31 main:574] : INFO : Epoch 136 | loss: 0.0311504 | val_loss: 0.0311757 | Time: 169791 ms
[2022-04-04 02:19:04 main:574] : INFO : Epoch 137 | loss: 0.031149 | val_loss: 0.0311716 | Time: 153207 ms
[2022-04-04 02:19:45 main:574] : INFO : Epoch 138 | loss: 0.0311498 | val_loss: 0.0311723 | Time: 31976.1 ms
[2022-04-04 02:20:11 main:574] : INFO : Epoch 139 | loss: 0.0311489 | val_loss: 0.0311728 | Time: 20644.2 ms
[2022-04-04 02:20:14 main:574] : INFO : Epoch 140 | loss: 0.0311486 | val_loss: 0.0311734 | Time: 3544.66 ms
[2022-04-04 02:20:18 main:574] : INFO : Epoch 141 | loss: 0.0311483 | val_loss: 0.0311737 | Time: 3422.89 ms
[2022-04-04 02:20:22 main:574] : INFO : Epoch 142 | loss: 0.031149 | val_loss: 0.0311771 | Time: 4575.65 ms
[2022-04-04 02:22:25 main:574] : INFO : Epoch 143 | loss: 0.0311485 | val_loss: 0.0311721 | Time: 116265 ms
[2022-04-04 02:22:59 main:574] : INFO : Epoch 144 | loss: 0.0311466 | val_loss: 0.031175 | Time: 27311.6 ms
[2022-04-04 02:23:03 main:574] : INFO : Epoch 145 | loss: 0.0311452 | val_loss: 0.0311762 | Time: 4351.5 ms
[2022-04-04 02:26:33 main:574] : INFO : Epoch 146 | loss: 0.0311462 | val_loss: 0.031173 | Time: 207947 ms
[2022-04-04 02:27:48 main:574] : INFO : Epoch 147 | loss: 0.0311465 | val_loss: 0.0311768 | Time: 75598.2 ms
[2022-04-04 02:27:52 main:574] : INFO : Epoch 148 | loss: 0.0311487 | val_loss: 0.0311791 | Time: 3552.86 ms
[2022-04-04 02:28:02 main:574] : INFO : Epoch 149 | loss: 0.0311527 | val_loss: 0.0311805 | Time: 10174.1 ms
[2022-04-04 02:28:06 main:574] : INFO : Epoch 150 | loss: 0.0311532 | val_loss: 0.0311792 | Time: 3861.5 ms
[2022-04-04 02:28:10 main:574] : INFO : Epoch 151 | loss: 0.0311523 | val_loss: 0.031182 | Time: 3862.78 ms
[2022-04-04 02:30:11 main:574] : INFO : Epoch 152 | loss: 0.0311513 | val_loss: 0.0311773 | Time: 120637 ms
[2022-04-04 02:31:59 main:574] : INFO : Epoch 153 | loss: 0.0311508 | val_loss: 0.031178 | Time: 102727 ms
[2022-04-04 02:32:16 main:574] : INFO : Epoch 154 | loss: 0.0311512 | val_loss: 0.0311725 | Time: 17285.3 ms
[2022-04-04 02:32:20 main:574] : INFO : Epoch 155 | loss: 0.0311483 | val_loss: 0.031175 | Time: 3614.05 ms
[2022-04-04 02:32:24 main:574] : INFO : Epoch 156 | loss: 0.031148 | val_loss: 0.0311761 | Time: 4526.31 ms
[2022-04-04 02:33:03 main:574] : INFO : Epoch 157 | loss: 0.0311463 | val_loss: 0.0311793 | Time: 37727.3 ms
[2022-04-04 02:33:07 main:574] : INFO : Epoch 158 | loss: 0.0311467 | val_loss: 0.0311773 | Time: 3653.27 ms
[2022-04-04 02:33:12 main:574] : INFO : Epoch 159 | loss: 0.0311445 | val_loss: 0.0311766 | Time: 4627.15 ms
[2022-04-04 02:35:38 main:574] : INFO : Epoch 160 | loss: 0.0311441 | val_loss: 0.0311724 | Time: 146704 ms
[2022-04-04 02:37:43 main:574] : INFO : Epoch 161 | loss: 0.0311443 | val_loss: 0.0311761 | Time: 119011 ms
[2022-04-04 02:39:56 main:574] : INFO : Epoch 162 | loss: 0.0311423 | val_loss: 0.031176 | Time: 127088 ms
[2022-04-04 02:42:20 main:574] : INFO : Epoch 163 | loss: 0.0311436 | val_loss: 0.0311796 | Time: 135388 ms
[2022-04-04 02:44:56 main:574] : INFO : Epoch 164 | loss: 0.0311446 | val_loss: 0.0311731 | Time: 152023 ms
[2022-04-04 02:45:07 main:574] : INFO : Epoch 165 | loss: 0.0311492 | val_loss: 0.0311747 | Time: 3595.75 ms
[2022-04-04 02:45:12 main:574] : INFO : Epoch 166 | loss: 0.031147 | val_loss: 0.0311767 | Time: 5569.02 ms
[2022-04-04 02:45:58 main:574] : INFO : Epoch 167 | loss: 0.0311438 | val_loss: 0.0311803 | Time: 46158.8 ms
[2022-04-04 02:46:04 main:574] : INFO : Epoch 168 | loss: 0.0311442 | val_loss: 0.0311813 | Time: 5136.31 ms
[2022-04-04 02:47:34 main:574] : INFO : Epoch 169 | loss: 0.0311513 | val_loss: 0.0311761 | Time: 89455.7 ms
[2022-04-04 02:47:38 main:574] : INFO : Epoch 170 | loss: 0.031149 | val_loss: 0.0311785 | Time: 4301.03 ms
[2022-04-04 02:49:18 main:574] : INFO : Epoch 171 | loss: 0.0311428 | val_loss: 0.0311754 | Time: 99409.3 ms
[2022-04-04 02:50:30 main:574] : INFO : Epoch 172 | loss: 0.0311462 | val_loss: 0.0311811 | Time: 72451.8 ms
[2022-04-04 02:51:10 main:574] : INFO : Epoch 173 | loss: 0.0311497 | val_loss: 0.0311779 | Time: 35282.3 ms
[2022-04-04 02:53:44 main:574] : INFO : Epoch 174 | loss: 0.0311517 | val_loss: 0.0311806 | Time: 148436 ms
[2022-04-04 02:56:13 main:574] : INFO : Epoch 175 | loss: 0.0311524 | val_loss: 0.031173 | Time: 148451 ms
[2022-04-04 02:58:58 main:574] : INFO : Epoch 176 | loss: 0.0311469 | val_loss: 0.0311782 | Time: 160307 ms
[2022-04-04 03:01:16 main:574] : INFO : Epoch 177 | loss: 0.0311427 | val_loss: 0.0311808 | Time: 138389 ms
[2022-04-04 03:03:48 main:574] : INFO : Epoch 178 | loss: 0.0311456 | val_loss: 0.0311771 | Time: 151393 ms
[2022-04-04 03:06:01 main:574] : INFO : Epoch 179 | loss: 0.0311438 | val_loss: 0.031185 | Time: 127280 ms
[2022-04-04 03:07:52 main:574] : INFO : Epoch 180 | loss: 0.0311439 | val_loss: 0.0311808 | Time: 110816 ms
[2022-04-04 03:09:51 main:574] : INFO : Epoch 181 | loss: 0.0311458 | val_loss: 0.0311763 | Time: 114609 ms
[2022-04-04 03:12:21 main:574] : INFO : Epoch 182 | loss: 0.0311436 | val_loss: 0.0311797 | Time: 150440 ms
[2022-04-04 03:14:53 main:574] : INFO : Epoch 183 | loss: 0.0311413 | val_loss: 0.0311807 | Time: 146168 ms
[2022-04-04 03:16:46 main:574] : INFO : Epoch 184 | loss: 0.0311426 | val_loss: 0.0311781 | Time: 107704 ms
[2022-04-04 03:17:58 main:574] : INFO : Epoch 185 | loss: 0.0311399 | val_loss: 0.0311805 | Time: 60527.3 ms
[2022-04-04 03:18:02 main:574] : INFO : Epoch 186 | loss: 0.0311376 | val_loss: 0.0311805 | Time: 4072.19 ms
[2022-04-04 03:19:32 main:574] : INFO : Epoch 187 | loss: 0.0311371 | val_loss: 0.0311832 | Time: 90016.6 ms
[2022-04-04 03:21:06 main:574] : INFO : Epoch 188 | loss: 0.0311379 | val_loss: 0.0311828 | Time: 93581 ms
[2022-04-04 03:23:52 main:574] : INFO : Epoch 189 | loss: 0.0311376 | val_loss: 0.0311792 | Time: 161500 ms
[2022-04-04 03:25:36 main:574] : INFO : Epoch 190 | loss: 0.0311373 | val_loss: 0.0311785 | Time: 103364 ms
[2022-04-04 03:28:12 main:574] : INFO : Epoch 191 | loss: 0.0311401 | val_loss: 0.0311774 | Time: 150837 ms
[2022-04-04 03:30:07 main:574] : INFO : Epoch 192 | loss: 0.0311427 | val_loss: 0.0311838 | Time: 110072 ms
[2022-04-04 03:32:49 main:574] : INFO : Epoch 193 | loss: 0.031141 | val_loss: 0.0311767 | Time: 156780 ms
[2022-04-04 03:35:16 main:574] : INFO : Epoch 194 | loss: 0.0311471 | val_loss: 0.0311807 | Time: 140137 ms
[2022-04-04 03:36:26 main:574] : INFO : Epoch 195 | loss: 0.0311454 | val_loss: 0.0311766 | Time: 69452.5 ms
[2022-04-04 03:38:28 main:574] : INFO : Epoch 196 | loss: 0.0311434 | val_loss: 0.0311768 | Time: 122287 ms
[2022-04-04 03:40:52 main:574] : INFO : Epoch 197 | loss: 0.0311452 | val_loss: 0.0311737 | Time: 144276 ms
[2022-04-04 03:43:46 main:574] : INFO : Epoch 198 | loss: 0.0311426 | val_loss: 0.0311745 | Time: 173354 ms
[2022-04-04 03:46:28 main:574] : INFO : Epoch 199 | loss: 0.031142 | val_loss: 0.0311847 | Time: 162689 ms
[2022-04-04 03:49:09 main:574] : INFO : Epoch 200 | loss: 0.0311434 | val_loss: 0.031179 | Time: 150843 ms
[2022-04-04 03:50:58 main:574] : INFO : Epoch 201 | loss: 0.0311444 | val_loss: 0.0311772 | Time: 104811 ms
[2022-04-04 03:52:10 main:574] : INFO : Epoch 202 | loss: 0.0311426 | val_loss: 0.0311771 | Time: 71540.8 ms
[2022-04-04 03:54:43 main:574] : INFO : Epoch 203 | loss: 0.0311398 | val_loss: 0.0311804 | Time: 153176 ms
[2022-04-04 03:57:38 main:574] : INFO : Epoch 204 | loss: 0.0311403 | val_loss: 0.0311779 | Time: 174539 ms
[2022-04-04 03:59:48 main:574] : INFO : Epoch 205 | loss: 0.0311383 | val_loss: 0.0311852 | Time: 130619 ms
[2022-04-04 04:01:27 main:574] : INFO : Epoch 206 | loss: 0.0311382 | val_loss: 0.0311795 | Time: 94361.6 ms
[2022-04-04 04:03:22 main:574] : INFO : Epoch 207 | loss: 0.0311389 | val_loss: 0.0311762 | Time: 115050 ms
[2022-04-04 04:06:01 main:574] : INFO : Epoch 208 | loss: 0.0311363 | val_loss: 0.0311804 | Time: 148697 ms
[2022-04-04 04:08:23 main:574] : INFO : Epoch 209 | loss: 0.0311384 | val_loss: 0.0311783 | Time: 141994 ms
[2022-04-04 04:09:01 main:574] : INFO : Epoch 210 | loss: 0.0311412 | val_loss: 0.0311828 | Time: 38123.5 ms
[2022-04-04 04:11:35 main:574] : INFO : Epoch 211 | loss: 0.0311397 | val_loss: 0.0311753 | Time: 148476 ms
[2022-04-04 04:13:51 main:574] : INFO : Epoch 212 | loss: 0.0311388 | val_loss: 0.0311803 | Time: 130708 ms
[2022-04-04 04:14:23 main:574] : INFO : Epoch 213 | loss: 0.03114 | val_loss: 0.0311795 | Time: 26096.4 ms
[2022-04-04 04:14:27 main:574] : INFO : Epoch 214 | loss: 0.0311382 | val_loss: 0.0311821 | Time: 4536.58 ms
[2022-04-04 04:15:58 main:574] : INFO : Epoch 215 | loss: 0.0311376 | val_loss: 0.0311813 | Time: 90550.6 ms
[2022-04-04 04:16:02 main:574] : INFO : Epoch 216 | loss: 0.0311384 | val_loss: 0.0311769 | Time: 3498.93 ms
[2022-04-04 04:16:06 main:574] : INFO : Epoch 217 | loss: 0.0311369 | val_loss: 0.0311836 | Time: 4439.61 ms
[2022-04-04 04:19:33 main:574] : INFO : Epoch 218 | loss: 0.0311352 | val_loss: 0.0311797 | Time: 204725 ms
[2022-04-04 04:20:54 main:574] : INFO : Epoch 219 | loss: 0.0311442 | val_loss: 0.0311882 | Time: 81176.5 ms
[2022-04-04 04:21:25 main:574] : INFO : Epoch 220 | loss: 0.0311528 | val_loss: 0.0311787 | Time: 30724.8 ms
[2022-04-04 04:24:03 main:574] : INFO : Epoch 221 | loss: 0.0311487 | val_loss: 0.0311762 | Time: 158166 ms
[2022-04-04 04:26:23 main:574] : INFO : Epoch 222 | loss: 0.0311493 | val_loss: 0.031184 | Time: 140108 ms
[2022-04-04 04:26:45 main:574] : INFO : Epoch 223 | loss: 0.0311495 | val_loss: 0.0311754 | Time: 21609.2 ms
[2022-04-04 04:28:56 main:574] : INFO : Epoch 224 | loss: 0.0311495 | val_loss: 0.0311783 | Time: 125900 ms
[2022-04-04 04:31:02 main:574] : INFO : Epoch 225 | loss: 0.0311479 | val_loss: 0.0311777 | Time: 120846 ms
[2022-04-04 04:32:46 main:574] : INFO : Epoch 226 | loss: 0.0311427 | val_loss: 0.0311758 | Time: 98966.1 ms
[2022-04-04 04:33:49 main:574] : INFO : Epoch 227 | loss: 0.0311447 | val_loss: 0.031179 | Time: 57498.8 ms
[2022-04-04 04:33:53 main:574] : INFO : Epoch 228 | loss: 0.0311506 | val_loss: 0.0311723 | Time: 4607.4 ms
[2022-04-04 04:35:56 main:574] : INFO : Epoch 229 | loss: 0.0311456 | val_loss: 0.031182 | Time: 122811 ms
[2022-04-04 04:38:06 main:574] : INFO : Epoch 230 | loss: 0.0311421 | val_loss: 0.0311775 | Time: 129643 ms
[2022-04-04 04:38:47 main:574] : INFO : Epoch 231 | loss: 0.0311423 | val_loss: 0.031177 | Time: 35970.6 ms
[2022-04-04 04:38:50 main:574] : INFO : Epoch 232 | loss: 0.0311418 | val_loss: 0.031178 | Time: 3583.12 ms
[2022-04-04 04:38:55 main:574] : INFO : Epoch 233 | loss: 0.031142 | val_loss: 0.0311757 | Time: 4575.27 ms
[2022-04-04 04:39:44 main:574] : INFO : Epoch 234 | loss: 0.0311463 | val_loss: 0.0311766 | Time: 47053.5 ms
[2022-04-04 04:39:47 main:574] : INFO : Epoch 235 | loss: 0.0311477 | val_loss: 0.0311762 | Time: 3543.64 ms
[2022-04-04 04:40:53 main:574] : INFO : Epoch 236 | loss: 0.0311545 | val_loss: 0.0311829 | Time: 65188.9 ms
[2022-04-04 04:41:03 main:574] : INFO : Epoch 237 | loss: 0.0311533 | val_loss: 0.0311788 | Time: 3522.48 ms
[2022-04-04 04:41:07 main:574] : INFO : Epoch 238 | loss: 0.0311489 | val_loss: 0.0311744 | Time: 4084.3 ms
[2022-04-04 04:42:55 main:574] : INFO : Epoch 239 | loss: 0.0311488 | val_loss: 0.0311871 | Time: 109417 ms
[2022-04-04 04:44:25 main:574] : INFO : Epoch 240 | loss: 0.0311468 | val_loss: 0.0311766 | Time: 79199.4 ms
[2022-04-04 04:44:28 main:574] : INFO : Epoch 241 | loss: 0.0311452 | val_loss: 0.0311773 | Time: 3772.31 ms
[2022-04-04 04:46:05 main:574] : INFO : Epoch 242 | loss: 0.0311448 | val_loss: 0.0311783 | Time: 96098.7 ms
[2022-04-04 04:48:42 main:574] : INFO : Epoch 243 | loss: 0.0311464 | val_loss: 0.0311772 | Time: 151883 ms
[2022-04-04 04:50:34 main:574] : INFO : Epoch 244 | loss: 0.0311444 | val_loss: 0.0311768 | Time: 106885 ms
[2022-04-04 04:51:31 main:574] : INFO : Epoch 245 | loss: 0.0311402 | val_loss: 0.0311755 | Time: 56971.1 ms
[2022-04-04 04:52:56 main:574] : INFO : Epoch 246 | loss: 0.0311394 | val_loss: 0.0311788 | Time: 85292.5 ms
[2022-04-04 04:53:01 main:574] : INFO : Epoch 247 | loss: 0.0311395 | val_loss: 0.0311773 | Time: 4370.5 ms
[2022-04-04 04:55:23 main:574] : INFO : Epoch 248 | loss: 0.0311371 | val_loss: 0.0311776 | Time: 142288 ms
[2022-04-04 04:56:59 main:574] : INFO : Epoch 249 | loss: 0.0311385 | val_loss: 0.0311746 | Time: 90583.5 ms
[2022-04-04 04:57:03 main:574] : INFO : Epoch 250 | loss: 0.0311367 | val_loss: 0.0311869 | Time: 3861 ms
[2022-04-04 04:57:30 main:574] : INFO : Epoch 251 | loss: 0.0311369 | val_loss: 0.0311821 | Time: 27158.6 ms
[2022-04-04 04:57:35 main:574] : INFO : Epoch 252 | loss: 0.0311402 | val_loss: 0.0311761 | Time: 4119.95 ms
[2022-04-04 04:58:43 main:574] : INFO : Epoch 253 | loss: 0.0311389 | val_loss: 0.0311807 | Time: 66988.3 ms
[2022-04-04 04:58:48 main:574] : INFO : Epoch 254 | loss: 0.0311368 | val_loss: 0.0311807 | Time: 4053.34 ms
[2022-04-04 05:00:57 main:574] : INFO : Epoch 255 | loss: 0.0311359 | val_loss: 0.0311794 | Time: 129132 ms
[2022-04-04 05:02:26 main:574] : INFO : Epoch 256 | loss: 0.0311334 | val_loss: 0.0311875 | Time: 84315.2 ms
[2022-04-04 05:04:45 main:574] : INFO : Epoch 257 | loss: 0.0311319 | val_loss: 0.0311851 | Time: 138633 ms
[2022-04-04 05:05:53 main:574] : INFO : Epoch 258 | loss: 0.0311318 | val_loss: 0.0311801 | Time: 63486.7 ms
[2022-04-04 05:08:05 main:574] : INFO : Epoch 259 | loss: 0.0311333 | val_loss: 0.0311828 | Time: 131839 ms
[2022-04-04 05:09:00 main:574] : INFO : Epoch 260 | loss: 0.0311363 | val_loss: 0.031178 | Time: 54620.9 ms
[2022-04-04 05:11:15 main:574] : INFO : Epoch 261 | loss: 0.0311486 | val_loss: 0.0311732 | Time: 135386 ms
[2022-04-04 05:13:48 main:574] : INFO : Epoch 262 | loss: 0.0311466 | val_loss: 0.0311835 | Time: 146072 ms
[2022-04-04 05:16:24 main:574] : INFO : Epoch 263 | loss: 0.0311466 | val_loss: 0.0311816 | Time: 150812 ms
[2022-04-04 05:18:55 main:574] : INFO : Epoch 264 | loss: 0.0311422 | val_loss: 0.0311758 | Time: 150646 ms
[2022-04-04 05:19:35 main:574] : INFO : Epoch 265 | loss: 0.0311398 | val_loss: 0.0311807 | Time: 31968.1 ms
[2022-04-04 05:20:56 main:574] : INFO : Epoch 266 | loss: 0.0311362 | val_loss: 0.0311824 | Time: 80929 ms
[2022-04-04 05:21:27 main:574] : INFO : Epoch 267 | loss: 0.0311348 | val_loss: 0.0311747 | Time: 30673.1 ms
[2022-04-04 05:23:40 main:574] : INFO : Epoch 268 | loss: 0.0311363 | val_loss: 0.0311811 | Time: 128973 ms
[2022-04-04 05:26:33 main:574] : INFO : Epoch 269 | loss: 0.0311371 | val_loss: 0.0311802 | Time: 167461 ms
[2022-04-04 05:29:22 main:574] : INFO : Epoch 270 | loss: 0.0311345 | val_loss: 0.0311795 | Time: 163873 ms
[2022-04-04 05:30:05 main:574] : INFO : Epoch 271 | loss: 0.0311327 | val_loss: 0.0311842 | Time: 43325.2 ms
[2022-04-04 05:30:09 main:574] : INFO : Epoch 272 | loss: 0.0311348 | val_loss: 0.0311791 | Time: 4000.18 ms
[2022-04-04 05:31:24 main:574] : INFO : Epoch 273 | loss: 0.0311437 | val_loss: 0.0311762 | Time: 75142.2 ms
[2022-04-04 05:32:17 main:574] : INFO : Epoch 274 | loss: 0.0311408 | val_loss: 0.0311838 | Time: 53060.6 ms
[2022-04-04 05:34:49 main:574] : INFO : Epoch 275 | loss: 0.03114 | val_loss: 0.0311802 | Time: 152003 ms
[2022-04-04 05:36:55 main:574] : INFO : Epoch 276 | loss: 0.0311395 | val_loss: 0.03118 | Time: 120527 ms
[2022-04-04 05:38:31 main:574] : INFO : Epoch 277 | loss: 0.0311437 | val_loss: 0.0311775 | Time: 96328.1 ms
[2022-04-04 05:41:30 main:574] : INFO : Epoch 278 | loss: 0.0311405 | val_loss: 0.0311771 | Time: 179143 ms
[2022-04-04 05:43:53 main:574] : INFO : Epoch 279 | loss: 0.0311382 | val_loss: 0.0311802 | Time: 142510 ms
[2022-04-04 05:46:00 main:574] : INFO : Epoch 280 | loss: 0.0311396 | val_loss: 0.0311766 | Time: 122322 ms
[2022-04-04 05:47:52 main:574] : INFO : Epoch 281 | loss: 0.0311368 | val_loss: 0.0311908 | Time: 111946 ms
[2022-04-04 05:49:41 main:574] : INFO : Epoch 282 | loss: 0.0311407 | val_loss: 0.0311798 | Time: 100313 ms
[2022-04-04 05:50:52 main:574] : INFO : Epoch 283 | loss: 0.031135 | val_loss: 0.0311786 | Time: 65033.6 ms
[2022-04-04 05:51:32 main:574] : INFO : Epoch 284 | loss: 0.0311341 | val_loss: 0.0311826 | Time: 40640 ms
[2022-04-04 05:53:52 main:574] : INFO : Epoch 285 | loss: 0.0311342 | val_loss: 0.0311761 | Time: 139540 ms
[2022-04-04 05:56:37 main:574] : INFO : Epoch 286 | loss: 0.0311356 | val_loss: 0.031181 | Time: 165043 ms
[2022-04-04 05:59:09 main:574] : INFO : Epoch 287 | loss: 0.0311359 | val_loss: 0.0311793 | Time: 151965 ms
[2022-04-04 06:00:57 main:574] : INFO : Epoch 288 | loss: 0.0311365 | val_loss: 0.0311886 | Time: 108432 ms
[2022-04-04 06:02:10 main:574] : INFO : Epoch 289 | loss: 0.031133 | val_loss: 0.0311855 | Time: 57610.6 ms
[2022-04-04 06:04:54 main:574] : INFO : Epoch 290 | loss: 0.0311325 | val_loss: 0.0311848 | Time: 159137 ms
[2022-04-04 06:05:55 main:574] : INFO : Epoch 291 | loss: 0.031131 | val_loss: 0.0311809 | Time: 61306.7 ms
[2022-04-04 06:08:42 main:574] : INFO : Epoch 292 | loss: 0.0311315 | val_loss: 0.0311832 | Time: 167272 ms
[2022-04-04 06:11:19 main:574] : INFO : Epoch 293 | loss: 0.0311522 | val_loss: 0.031173 | Time: 151231 ms
[2022-04-04 06:13:55 main:574] : INFO : Epoch 294 | loss: 0.0311517 | val_loss: 0.0311757 | Time: 150838 ms
[2022-04-04 06:16:19 main:574] : INFO : Epoch 295 | loss: 0.0311507 | val_loss: 0.0311743 | Time: 138958 ms
[2022-04-04 06:19:35 main:574] : INFO : Epoch 296 | loss: 0.0311445 | val_loss: 0.0311775 | Time: 184351 ms
[2022-04-04 06:20:13 main:574] : INFO : Epoch 297 | loss: 0.0311424 | val_loss: 0.0311764 | Time: 38204.2 ms
[2022-04-04 06:21:37 main:574] : INFO : Epoch 298 | loss: 0.031139 | val_loss: 0.0311771 | Time: 77826.8 ms
[2022-04-04 06:22:47 main:574] : INFO : Epoch 299 | loss: 0.0311365 | val_loss: 0.0311859 | Time: 70035.6 ms
[2022-04-04 06:24:56 main:574] : INFO : Epoch 300 | loss: 0.0311388 | val_loss: 0.0311706 | Time: 119016 ms
[2022-04-04 06:25:05 main:574] : INFO : Epoch 301 | loss: 0.0311409 | val_loss: 0.0311806 | Time: 2917.72 ms
[2022-04-04 06:25:08 main:574] : INFO : Epoch 302 | loss: 0.0311384 | val_loss: 0.0311747 | Time: 2948.69 ms
[2022-04-04 06:26:16 main:574] : INFO : Epoch 303 | loss: 0.0311368 | val_loss: 0.0311778 | Time: 66247.8 ms
[2022-04-04 06:26:39 main:574] : INFO : Epoch 304 | loss: 0.0311376 | val_loss: 0.0311886 | Time: 23499.1 ms
[2022-04-04 06:26:42 main:574] : INFO : Epoch 305 | loss: 0.0311376 | val_loss: 0.0311773 | Time: 3100.38 ms
[2022-04-04 06:26:47 main:574] : INFO : Epoch 306 | loss: 0.0311356 | val_loss: 0.0311789 | Time: 4081.86 ms
[2022-04-04 06:29:55 main:574] : INFO : Epoch 307 | loss: 0.031136 | val_loss: 0.0311815 | Time: 187159 ms
[2022-04-04 06:31:26 main:574] : INFO : Epoch 308 | loss: 0.0311357 | val_loss: 0.0311785 | Time: 85844.1 ms
[2022-04-04 06:34:33 main:574] : INFO : Epoch 309 | loss: 0.0311336 | val_loss: 0.0311779 | Time: 187264 ms
[2022-04-04 06:35:59 main:574] : INFO : Epoch 310 | loss: 0.0311319 | val_loss: 0.0311824 | Time: 86172.1 ms
[2022-04-04 06:38:31 main:574] : INFO : Epoch 311 | loss: 0.031131 | val_loss: 0.0311822 | Time: 145819 ms
[2022-04-04 06:40:14 main:574] : INFO : Epoch 312 | loss: 0.0311332 | val_loss: 0.031176 | Time: 102744 ms
[2022-04-04 06:43:05 main:574] : INFO : Epoch 313 | loss: 0.0311331 | val_loss: 0.0311904 | Time: 165067 ms
[2022-04-04 06:44:14 main:574] : INFO : Epoch 314 | loss: 0.0311328 | val_loss: 0.0311863 | Time: 69294.9 ms
[2022-04-04 06:44:18 main:574] : INFO : Epoch 315 | loss: 0.0311293 | val_loss: 0.031182 | Time: 3789.96 ms
[2022-04-04 06:46:13 main:574] : INFO : Epoch 316 | loss: 0.0311318 | val_loss: 0.0311825 | Time: 114592 ms
[2022-04-04 06:48:05 main:574] : INFO : Epoch 317 | loss: 0.0311295 | val_loss: 0.0311808 | Time: 111892 ms
[2022-04-04 06:48:09 main:574] : INFO : Epoch 318 | loss: 0.031131 | val_loss: 0.0311836 | Time: 3691.64 ms
[2022-04-04 06:48:13 main:574] : INFO : Epoch 319 | loss: 0.0311289 | val_loss: 0.0311833 | Time: 4009.5 ms
[2022-04-04 06:49:18 main:574] : INFO : Epoch 320 | loss: 0.0311297 | val_loss: 0.0311879 | Time: 65036.7 ms
[2022-04-04 06:49:42 main:574] : INFO : Epoch 321 | loss: 0.0311292 | val_loss: 0.0311827 | Time: 18551.3 ms
[2022-04-04 06:51:05 main:574] : INFO : Epoch 322 | loss: 0.0311305 | val_loss: 0.0311816 | Time: 77933.9 ms
[2022-04-04 06:51:09 main:574] : INFO : Epoch 323 | loss: 0.0311323 | val_loss: 0.0311836 | Time: 3706.18 ms
[2022-04-04 06:51:14 main:574] : INFO : Epoch 324 | loss: 0.0311296 | val_loss: 0.031181 | Time: 4920.31 ms
[2022-04-04 06:52:45 main:574] : INFO : Epoch 325 | loss: 0.0311276 | val_loss: 0.0311897 | Time: 89830.6 ms
[2022-04-04 06:54:52 main:574] : INFO : Epoch 326 | loss: 0.0311272 | val_loss: 0.0311786 | Time: 126466 ms
[2022-04-04 06:55:05 main:574] : INFO : Epoch 327 | loss: 0.0311275 | val_loss: 0.0311857 | Time: 7959.68 ms
[2022-04-04 06:55:09 main:574] : INFO : Epoch 328 | loss: 0.031133 | val_loss: 0.0311801 | Time: 4082.3 ms
[2022-04-04 06:58:21 main:574] : INFO : Epoch 329 | loss: 0.031131 | val_loss: 0.0311834 | Time: 190512 ms
[2022-04-04 07:00:25 main:574] : INFO : Epoch 330 | loss: 0.0311277 | val_loss: 0.031179 | Time: 123237 ms
[2022-04-04 07:01:41 main:574] : INFO : Epoch 331 | loss: 0.0311304 | val_loss: 0.0311881 | Time: 61867.1 ms
[2022-04-04 07:01:44 main:574] : INFO : Epoch 332 | loss: 0.0311275 | val_loss: 0.0311839 | Time: 3531.57 ms
[2022-04-04 07:02:37 main:574] : INFO : Epoch 333 | loss: 0.0311266 | val_loss: 0.0311873 | Time: 50917.7 ms
[2022-04-04 07:04:07 main:574] : INFO : Epoch 334 | loss: 0.0311244 | val_loss: 0.0311872 | Time: 85521.4 ms
[2022-04-04 07:05:55 main:574] : INFO : Epoch 335 | loss: 0.0311261 | val_loss: 0.0311785 | Time: 108088 ms
[2022-04-04 07:07:46 main:574] : INFO : Epoch 336 | loss: 0.0311269 | val_loss: 0.0311879 | Time: 110634 ms
[2022-04-04 07:07:49 main:574] : INFO : Epoch 337 | loss: 0.0311284 | val_loss: 0.0311853 | Time: 3671.37 ms
[2022-04-04 07:07:53 main:574] : INFO : Epoch 338 | loss: 0.0311255 | val_loss: 0.0311804 | Time: 3706.22 ms
[2022-04-04 07:08:11 main:574] : INFO : Epoch 339 | loss: 0.0311267 | val_loss: 0.031186 | Time: 17313.7 ms
[2022-04-04 07:08:15 main:574] : INFO : Epoch 340 | loss: 0.0311267 | val_loss: 0.0311837 | Time: 3984.04 ms
[2022-04-04 07:09:14 main:574] : INFO : Epoch 341 | loss: 0.0311274 | val_loss: 0.0311999 | Time: 58877.1 ms
[2022-04-04 07:09:48 main:574] : INFO : Epoch 342 | loss: 0.0311328 | val_loss: 0.0311832 | Time: 28869.5 ms
[2022-04-04 07:09:52 main:574] : INFO : Epoch 343 | loss: 0.031131 | val_loss: 0.0311821 | Time: 4031.08 ms
[2022-04-04 07:12:09 main:574] : INFO : Epoch 344 | loss: 0.0311259 | val_loss: 0.0311851 | Time: 135704 ms
[2022-04-04 07:12:14 main:574] : INFO : Epoch 345 | loss: 0.0311257 | val_loss: 0.031197 | Time: 4824.24 ms
[2022-04-04 07:13:47 main:574] : INFO : Epoch 346 | loss: 0.0311286 | val_loss: 0.0311849 | Time: 92207.8 ms
[2022-04-04 07:15:51 main:574] : INFO : Epoch 347 | loss: 0.0311312 | val_loss: 0.0311906 | Time: 119100 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 04/04/22 07:16:02
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\12;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 000000002c280000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000294d0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000028900000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000027260000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000283d0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026c90000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1586 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1586
ModLoad: 00000000faf30000 0000000000058000 C:\ProgramData\BOINC\slots\12\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000027740000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000026f90000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000270a0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000007c40000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000001a230000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000000cd10000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000000d570000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 00000000aad10000 000000000abe7000 C:\ProgramData\BOINC\slots\12\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000f62f0000 000000000020d000 C:\ProgramData\BOINC\slots\12\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000e8c50000 00000000002db000 C:\ProgramData\BOINC\slots\12\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000fac00000 0000000000040000 C:\ProgramData\BOINC\slots\12\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000026bf0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000028850000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000293e0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000027770000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027870000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000061a60000 0000000024b49000 C:\ProgramData\BOINC\slots\12\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000fa870000 0000000000045000 C:\ProgramData\BOINC\slots\12\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bd5f0000 00000000030a5000 C:\ProgramData\BOINC\slots\12\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000ba130000 00000000034b2000 C:\ProgramData\BOINC\slots\12\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000140b0000 0000000000010000 C:\ProgramData\BOINC\slots\12\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000033310000 000000001b3a4000 C:\ProgramData\BOINC\slots\12\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 000000009bd50000 0000000003d5b000 C:\ProgramData\BOINC\slots\12\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000a8e40000 0000000001eca000 C:\ProgramData\BOINC\slots\12\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000092bb0000 000000000919f000 C:\ProgramData\BOINC\slots\12\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000028710000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000024a40000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000289c0000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000271a0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000026cc0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000269b0000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000271f0000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026d80000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000267d0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000027a50000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 0000000024d20000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000028e30000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000026650000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027690000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000275a0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000b7ab0000 00000000002cc000 C:\WINDOWS\system32\nvcuda.dll (30.0.14.7208) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_00\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.14.7208
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.4.136 driver
Product Version : 30.0.14.7208
ModLoad: 00000000216e0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000b68e0000 00000000011c5000 C:\WINDOWS\system32\DriverStore\FileRepository\nvhdc.inf_amd64_312d953c81546f4f\nvcuda64.dll (30.0.14.7208) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_00\drivers\gpgpu\_out\wddm2_ms_shipping_wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.14.7208
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.4.136 driver
Product Version : 30.0.14.7208
ModLoad: 000000001dc70000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 0000000018c60000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000265c0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000024640000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.1266) (-exported- Symbols Loaded)
Linked PDB Filename : UxTheme.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000005070000 0000000000719000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.14.7208) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r470\r470_00\drivers\nvapi\gpu\_out\wddm2_ms_shipping_wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.14.7208
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.14.7208
ModLoad: 00000000285e0000 000000000012a000 C:\WINDOWS\System32\ole32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000006710000 0000000000009000 C:\ProgramData\BOINC\slots\12\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000091be0000 0000000000fca000 C:\ProgramData\BOINC\slots\12\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000026cf0000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3870, Write: 3022, Other 8026
- I/O Transfers Counters -
Read: 7722280, Write: 6810374, Other 57772
- Paged Pool Usage -
QuotaPagedPoolUsage: 3536472, QuotaPeakPagedPoolUsage: 3536744
QuotaNonPagedPoolUsage: 59016, QuotaPeakNonPagedPoolUsage: 59424
- Virtual Memory Usage -
VirtualSize: 396709888, PeakVirtualSize: 1019875328
- Pagefile Usage -
PagefileUsage: 396709888, PeakPagefileUsage: 396709888
- Working Set Size -
WorkingSetSize: 936783872, PeakWorkingSetSize: -2053980160, PageFaultCount: 898493
*** Dump of thread ID 19808 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 143901552.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=000000002c538c40 rdx=00000000e50fe6f0 rsi=0000000000000000 rdi=0000000000000000
r8=00000000e50fe6f0 r9=000000002c538c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000273290d2 rsp=00000000e50fe6c8 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e50fe6c0 2c2b9100 00000001 e50fe6f0 e50fe6f0 2c538c30 KERNELBASE!DebugBreak+0x0
e50feb00 2c2b9ce4 cd800000 e50fee64 00000000 00000378 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e50ffa60 2c2ba084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e50ffa90 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e50ffac0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e50ffb40 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9032 (state: Running): ***
- Information -
Status: Base Priority: Above Normal, Priority: Time Critical, , Kernel Time: 155156256.000000, User Time: 3576093696.000000, Wait Time: 143901552.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000000e0 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000000e0
r8=00000000e4936858 r9=00000000e4936b70 r10=0000000000000000 r11=0000000000000246 r12=0000000000000800 r13=000000000000015b
r14=00000000000000e0 r15=0000000000000001 rip=000000002956cdf4 rsp=00000000e49367e8 rbp=00000000e4936b70
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e49367e0 27281a5e 00000000 00000000 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
e4936880 2c2c4ca3 2c2cd95f 00000000 00000000 000000e0 KERNELBASE!WaitForSingleObjectEx+0x0
e49368b0 2c2bf3be 00000000 2c2cd95f 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e49368e0 2c2b8916 2c2cd95f 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e4936a30 2c2ba3aa ba902274 d6370ce0 00000000 750f548a mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e4936a60 2c29bb68 00000000 2c2cd95f 00000000 75b0dc90 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e494f800 2c4dc6f8 17c0fc70 00000000 00000000 17960d70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e494f840 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
e494f870 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e494f8f0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 16628 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 143901552.000000
- Registers -
rax=000000000000005b rbx=000000000000000e rcx=000000000000000e rdx=00000000e51ff490 rsi=0000000000000000 rdi=000000000000000e
r8=00000000e51ff138 r9=0000000017e45260 r10=0000000000000000 r11=0000000000000246 r12=0000000000000064 r13=00000000e51ff490
r14=00000000e51ff190 r15=0000000000000000 rip=000000002956d8c4 rsp=00000000e51ff138 rbp=0000000017e45260
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e51ff130 272acb20 00000000 e4a98000 2963a4d0 e4a98000 ntdll!ZwWaitForMultipleObjects+0x0
e51ff420 272aca1e 15f93160 004e0049 0000000e 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
e51ff460 b6ab1224 17e48200 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
e51ff6a0 b6b57fcd 00000000 17e45260 00000000 00000000 nvcuda64!+0x0
e51ff750 b6ab0bf3 1c1aee9a c30eff3a 183fbd30 00000000 nvcuda64!cuProfilerStop+0x0
e51ff780 b6de58a4 183fbd30 00000000 00000000 00000000 nvcuda64!+0x0
e51ff7b0 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
e51ff7e0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e51ff860 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 6220 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 142517632.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002c8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002c8
r8=00000000e52ff948 r9=0000000000000000 r10=0000000000000000 r11=00000000e52ff9a8 r12=0000000000000000 r13=0000000000000000
r14=00000000000002c8 r15=0000000000000000 rip=000000002956cdf4 rsp=00000000e52ff6a8 rbp=0000000017e2ac60
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e52ff6a0 27281a5e 00000000 29522630 0000027f 00000000 ntdll!ZwWaitForSingleObject+0x0
e52ff740 b6cdd50d 183fbf40 00000000 00000000 000002c8 KERNELBASE!WaitForSingleObjectEx+0x0
e52ff9a0 b6ab0bf3 00000000 183fbf40 b68e0000 17e65700 nvcuda64!cuProfilerStop+0x0
e52ff9d0 b6de58a4 183fbf40 00000000 00000000 00000000 nvcuda64!+0x0
e52ffa00 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
e52ffa30 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e52ffab0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 16564 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 143901536.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000180bff80 rdx=00000000e53ff778 rsi=00000000e53ff6e0 rdi=0000000000000000
r8=0000000000000083 r9=0000000000000004 r10=0000000000000003 r11=00000000e53ff710 r12=00000000e53ff778 r13=0000000000000000
r14=00000000180bff80 r15=00000000180c4730 rip=0000000029570764 rsp=00000000e53ff6b8 rbp=00000000e53ff710
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e53ff6b0 29534295 00000102 00000102 00000000 e53ff6e0 ntdll!NtWaitForAlertByThreadId+0x0
e53ff720 272ccef9 180c1f10 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0
e53ff750 b6a1f0ec 180c1f10 180c1f10 180c1f10 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0
e53ff7b0 b6ab0bf3 00000005 1beda353 1beda353 00000000 nvcuda64!+0x0
e53ff7e0 b6de58a4 183fc600 00000000 00000000 00000000 nvcuda64!+0x0
e53ff810 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
e53ff840 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e53ff8c0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 16420 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 104531248.000000, User Time: 2682812416.000000, Wait Time: 143900416.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000adf039d8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000adf039d8
r8=00000000ffffffd8 r9=00000000ffffffff r10=0000000000000000 r11=0000000000000003 r12=0000000000000000 r13=0000000000000000
r14=00000000e54ff340 r15=00000000adf03988 rip=0000000029570764 rsp=00000000e54ff318 rbp=00000000e54ff370
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
e54ff310 29534021 bc9e5208 07c52f97 c9db3200 c9db31f0 ntdll!NtWaitForAlertByThreadId+0x0
e54ff390 272ccfa9 adf039d0 adf039c8 adf03980 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
e54ff3d0 07c7bf3d c9db3220 00000000 00000000 b1ef05b4 KERNELBASE!SleepConditionVariableSRW+0x0
e54ff410 07c52798 00000000 c9db3220 00000000 00000000 MSVCP140!__crtSetThreadpoolWait+0x0
e54ff440 07c52a4a fbe4f56c 00000000 e54ff650 adf039c8 MSVCP140!std::_Winerror_message+0x0
e54ff470 b1ef064b adf03980 e54ff570 adf03980 c9db3200 MSVCP140!_Cnd_wait+0x0
e54ff4b0 b1ef25b1 cfbe9a00 ffffffff c9db3200 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
e54ff7a0 b1ef2471 b5416d10 e54ff7e0 b5416d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
e54ff7f0 b1ee9744 00080001 cf4a8820 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
e54ff820 270c1bb2 cf4a8820 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
e54ff850 28917034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
e54ff880 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
e54ff900 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)