Task 14175525
| Name | ParityModified-1647041678-20869-3-0_0 |
| Workunit | 11237781 |
| Created | 1 Apr 2022, 4:40:44 UTC |
| Sent | 5 Apr 2022, 18:57:25 UTC |
| Report deadline | 13 Apr 2022, 18:57:25 UTC |
| Received | 7 Apr 2022, 5:28:55 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED |
| Computer ID | 4503 |
| Run time | 8 hours 8 min 30 sec |
| CPU time | 13 min 56 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 10,073.24 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.40 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
exceeded elapsed time limit 29305.33 (4000000.00G/136.49G)</message>
<stderr_txt>
hine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080)
[2022-04-06 14:03:03 main:435] : INFO : Set logging level to 1
[2022-04-06 14:03:03 main:441] : INFO : Running in BOINC Client mode
[2022-04-06 14:03:03 main:444] : INFO : Resolving all filenames
[2022-04-06 14:03:03 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-04-06 14:03:03 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 0)
[2022-04-06 14:03:03 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-04-06 14:03:03 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-04-06 14:03:03 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 0)
[2022-04-06 14:03:03 main:472] : INFO : Dataset filename: dataset.hdf5
[2022-04-06 14:03:03 main:474] : INFO : Configuration:
[2022-04-06 14:03:03 main:475] : INFO : Model type: GRU
[2022-04-06 14:03:03 main:476] : INFO : Validation Loss Threshold: 0.0001
[2022-04-06 14:03:03 main:477] : INFO : Max Epochs: 2048
[2022-04-06 14:03:03 main:478] : INFO : Batch Size: 128
[2022-04-06 14:03:03 main:479] : INFO : Learning Rate: 0.01
[2022-04-06 14:03:03 main:480] : INFO : Patience: 10
[2022-04-06 14:03:03 main:481] : INFO : Hidden Width: 12
[2022-04-06 14:03:03 main:482] : INFO : # Recurrent Layers: 4
[2022-04-06 14:03:03 main:483] : INFO : # Backend Layers: 4
[2022-04-06 14:03:03 main:484] : INFO : # Threads: 1
[2022-04-06 14:03:03 main:486] : INFO : Preparing Dataset
[2022-04-06 14:03:03 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-04-06 14:03:12 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-04-06 14:03:18 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-04-06 14:03:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-04-06 14:03:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-04-06 14:03:21 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-04-06 14:03:21 main:494] : INFO : Creating Model
[2022-04-06 14:03:22 main:507] : INFO : Preparing config file
[2022-04-06 14:03:22 main:519] : INFO : Creating new config file
[2022-04-06 14:03:22 main:538] : INFO : This is a continuation WU, loading previous network
[2022-04-06 14:03:29 main:559] : INFO : Loading DataLoader into Memory
[2022-04-06 14:03:37 main:562] : INFO : Starting Training
[2022-04-06 14:07:56 main:574] : INFO : Epoch 1 | loss: 0.0316971 | val_loss: 0.0312987 | Time: 259050 ms
[2022-04-06 14:10:18 main:574] : INFO : Epoch 2 | loss: 0.0312444 | val_loss: 0.0311948 | Time: 142376 ms
[2022-04-06 14:10:57 main:574] : INFO : Epoch 3 | loss: 0.0311577 | val_loss: 0.0311576 | Time: 38546.7 ms
[2022-04-06 14:11:03 main:574] : INFO : Epoch 4 | loss: 0.0311303 | val_loss: 0.0311523 | Time: 6490.28 ms
[2022-04-06 14:13:02 main:574] : INFO : Epoch 5 | loss: 0.0311234 | val_loss: 0.0311492 | Time: 118103 ms
[2022-04-06 14:16:42 main:574] : INFO : Epoch 6 | loss: 0.0311238 | val_loss: 0.0311548 | Time: 219703 ms
[2022-04-06 14:20:30 main:574] : INFO : Epoch 7 | loss: 0.0311234 | val_loss: 0.0311496 | Time: 217325 ms
[2022-04-06 14:21:29 main:574] : INFO : Epoch 8 | loss: 0.0311244 | val_loss: 0.0311529 | Time: 54863.4 ms
[2022-04-06 14:23:51 main:574] : INFO : Epoch 9 | loss: 0.0311235 | val_loss: 0.0311489 | Time: 136350 ms
[2022-04-06 14:23:57 main:574] : INFO : Epoch 10 | loss: 0.0311205 | val_loss: 0.0311547 | Time: 6259.88 ms
[2022-04-06 14:25:56 main:574] : INFO : Epoch 11 | loss: 0.0311212 | val_loss: 0.0311497 | Time: 119002 ms
[2022-04-06 14:29:17 main:574] : INFO : Epoch 12 | loss: 0.0311234 | val_loss: 0.0311511 | Time: 195932 ms
[2022-04-06 14:30:53 main:574] : INFO : Epoch 13 | loss: 0.0311213 | val_loss: 0.0311515 | Time: 96413.3 ms
[2022-04-06 14:31:00 main:574] : INFO : Epoch 14 | loss: 0.0311277 | val_loss: 0.031153 | Time: 6921.1 ms
[2022-04-06 14:32:44 main:574] : INFO : Epoch 15 | loss: 0.0311249 | val_loss: 0.0311504 | Time: 102663 ms
[2022-04-06 14:33:33 main:574] : INFO : Epoch 16 | loss: 0.0311249 | val_loss: 0.0311486 | Time: 49155 ms
[2022-04-06 14:33:55 main:574] : INFO : Epoch 17 | loss: 0.0311216 | val_loss: 0.0311489 | Time: 16577.5 ms
[2022-04-06 14:35:29 main:574] : INFO : Epoch 18 | loss: 0.0311204 | val_loss: 0.031151 | Time: 92588.1 ms
[2022-04-06 14:37:36 main:574] : INFO : Epoch 19 | loss: 0.031119 | val_loss: 0.031154 | Time: 127657 ms
[2022-04-06 14:40:27 main:574] : INFO : Epoch 20 | loss: 0.03112 | val_loss: 0.0311509 | Time: 161205 ms
[2022-04-06 14:42:28 main:574] : INFO : Epoch 21 | loss: 0.0311204 | val_loss: 0.0311535 | Time: 121263 ms
[2022-04-06 14:45:07 main:574] : INFO : Epoch 22 | loss: 0.0311199 | val_loss: 0.0311524 | Time: 153196 ms
[2022-04-06 14:48:07 main:574] : INFO : Epoch 23 | loss: 0.0311179 | val_loss: 0.0311497 | Time: 173226 ms
[2022-04-06 14:50:57 main:574] : INFO : Epoch 24 | loss: 0.0311173 | val_loss: 0.031153 | Time: 164425 ms
[2022-04-06 14:53:10 main:574] : INFO : Epoch 25 | loss: 0.0311173 | val_loss: 0.0311495 | Time: 133654 ms
[2022-04-06 14:56:59 main:574] : INFO : Epoch 26 | loss: 0.0311172 | val_loss: 0.0311488 | Time: 210351 ms
[2022-04-06 14:59:59 main:574] : INFO : Epoch 27 | loss: 0.0311178 | val_loss: 0.0311503 | Time: 180472 ms
[2022-04-06 15:02:56 main:574] : INFO : Epoch 28 | loss: 0.0311214 | val_loss: 0.0311491 | Time: 176672 ms
[2022-04-06 15:05:32 main:574] : INFO : Epoch 29 | loss: 0.0311202 | val_loss: 0.0311509 | Time: 145995 ms
[2022-04-06 15:05:38 main:574] : INFO : Epoch 30 | loss: 0.0311185 | val_loss: 0.031153 | Time: 6284.55 ms
[2022-04-06 15:07:14 main:574] : INFO : Epoch 31 | loss: 0.0311165 | val_loss: 0.031158 | Time: 84104.7 ms
[2022-04-06 15:10:24 main:574] : INFO : Epoch 32 | loss: 0.031117 | val_loss: 0.0311521 | Time: 182394 ms
[2022-04-06 15:10:36 main:574] : INFO : Epoch 33 | loss: 0.0311152 | val_loss: 0.0311619 | Time: 11877.4 ms
[2022-04-06 15:11:54 main:574] : INFO : Epoch 34 | loss: 0.0311177 | val_loss: 0.0311559 | Time: 77648 ms
[2022-04-06 15:15:56 main:574] : INFO : Epoch 35 | loss: 0.0311166 | val_loss: 0.0311517 | Time: 232735 ms
[2022-04-06 15:19:44 main:574] : INFO : Epoch 36 | loss: 0.0311151 | val_loss: 0.0311525 | Time: 227969 ms
[2022-04-06 15:20:55 main:574] : INFO : Epoch 37 | loss: 0.0311159 | val_loss: 0.0311505 | Time: 70594.1 ms
[2022-04-06 15:23:26 main:574] : INFO : Epoch 38 | loss: 0.0311164 | val_loss: 0.03115 | Time: 146298 ms
[2022-04-06 15:27:02 main:574] : INFO : Epoch 39 | loss: 0.0311163 | val_loss: 0.0311505 | Time: 216034 ms
[2022-04-06 15:28:55 main:574] : INFO : Epoch 40 | loss: 0.0311168 | val_loss: 0.0311506 | Time: 107324 ms
[2022-04-06 15:29:03 main:574] : INFO : Epoch 41 | loss: 0.0311185 | val_loss: 0.03115 | Time: 7230.16 ms
[2022-04-06 15:32:57 main:574] : INFO : Epoch 42 | loss: 0.031124 | val_loss: 0.0311526 | Time: 233398 ms
[2022-04-06 15:35:59 main:574] : INFO : Epoch 43 | loss: 0.031125 | val_loss: 0.0311458 | Time: 181397 ms
[2022-04-06 15:40:11 main:574] : INFO : Epoch 44 | loss: 0.0311252 | val_loss: 0.0311482 | Time: 246985 ms
[2022-04-06 15:43:50 main:574] : INFO : Epoch 45 | loss: 0.0311239 | val_loss: 0.0311487 | Time: 219692 ms
[2022-04-06 15:44:37 main:574] : INFO : Epoch 46 | loss: 0.031122 | val_loss: 0.0311484 | Time: 41446.5 ms
[2022-04-06 15:45:56 main:574] : INFO : Epoch 47 | loss: 0.0311178 | val_loss: 0.0311467 | Time: 79808.1 ms
[2022-04-06 15:46:57 main:574] : INFO : Epoch 48 | loss: 0.0311184 | val_loss: 0.0311494 | Time: 54663.4 ms
[2022-04-06 15:47:52 main:574] : INFO : Epoch 49 | loss: 0.0311194 | val_loss: 0.0311483 | Time: 54825.9 ms
[2022-04-06 15:50:53 main:574] : INFO : Epoch 50 | loss: 0.0311184 | val_loss: 0.0311494 | Time: 173024 ms
[2022-04-06 15:51:35 main:574] : INFO : Epoch 51 | loss: 0.0311176 | val_loss: 0.03115 | Time: 42176.7 ms
[2022-04-06 15:51:40 main:574] : INFO : Epoch 52 | loss: 0.031116 | val_loss: 0.0311518 | Time: 4735.47 ms
[2022-04-06 15:53:25 main:574] : INFO : Epoch 53 | loss: 0.0311192 | val_loss: 0.0311527 | Time: 104940 ms
[2022-04-06 15:56:51 main:574] : INFO : Epoch 54 | loss: 0.0311188 | val_loss: 0.0311465 | Time: 205478 ms
[2022-04-06 15:59:24 main:574] : INFO : Epoch 55 | loss: 0.0311205 | val_loss: 0.031145 | Time: 153521 ms
[2022-04-06 16:02:49 main:574] : INFO : Epoch 56 | loss: 0.0311187 | val_loss: 0.0311474 | Time: 199503 ms
[2022-04-06 16:03:25 main:574] : INFO : Epoch 57 | loss: 0.0311186 | val_loss: 0.0311552 | Time: 36234.4 ms
[2022-04-06 16:05:57 main:574] : INFO : Epoch 58 | loss: 0.0311196 | val_loss: 0.0311587 | Time: 145786 ms
[2022-04-06 16:09:18 main:574] : INFO : Epoch 59 | loss: 0.0311196 | val_loss: 0.0311497 | Time: 201560 ms
[2022-04-06 16:10:45 main:574] : INFO : Epoch 60 | loss: 0.0311192 | val_loss: 0.0311509 | Time: 87273.3 ms
[2022-04-06 16:13:46 main:574] : INFO : Epoch 61 | loss: 0.0311182 | val_loss: 0.0311532 | Time: 175832 ms
[2022-04-06 16:17:35 main:574] : INFO : Epoch 62 | loss: 0.0311209 | val_loss: 0.0311573 | Time: 222959 ms
[2022-04-06 16:20:24 main:574] : INFO : Epoch 63 | loss: 0.031119 | val_loss: 0.0311574 | Time: 159007 ms
[2022-04-06 16:21:32 main:574] : INFO : Epoch 64 | loss: 0.0311195 | val_loss: 0.0311527 | Time: 63327.4 ms
[2022-04-06 16:23:32 main:574] : INFO : Epoch 65 | loss: 0.0311174 | val_loss: 0.0311572 | Time: 120415 ms
[2022-04-06 16:26:52 main:574] : INFO : Epoch 66 | loss: 0.031119 | val_loss: 0.0311596 | Time: 199663 ms
[2022-04-06 16:30:34 main:574] : INFO : Epoch 67 | loss: 0.0311167 | val_loss: 0.0311541 | Time: 217330 ms
[2022-04-06 16:33:23 main:574] : INFO : Epoch 68 | loss: 0.0311166 | val_loss: 0.0311513 | Time: 163857 ms
[2022-04-06 16:35:54 main:574] : INFO : Epoch 69 | loss: 0.0311181 | val_loss: 0.031153 | Time: 146691 ms
[2022-04-06 16:38:37 main:574] : INFO : Epoch 70 | loss: 0.0311194 | val_loss: 0.0311493 | Time: 162632 ms
[2022-04-06 16:41:06 main:574] : INFO : Epoch 71 | loss: 0.0311199 | val_loss: 0.031148 | Time: 149700 ms
[2022-04-06 16:41:13 main:574] : INFO : Epoch 72 | loss: 0.0311201 | val_loss: 0.0311519 | Time: 6763.28 ms
[2022-04-06 16:45:22 main:574] : INFO : Epoch 73 | loss: 0.0311169 | val_loss: 0.0311493 | Time: 248172 ms
[2022-04-06 16:48:15 main:574] : INFO : Epoch 74 | loss: 0.0311161 | val_loss: 0.0311486 | Time: 172591 ms
[2022-04-06 16:50:58 main:574] : INFO : Epoch 75 | loss: 0.031117 | val_loss: 0.0311513 | Time: 157838 ms
[2022-04-06 16:52:32 main:574] : INFO : Epoch 76 | loss: 0.0311166 | val_loss: 0.0311533 | Time: 93915 ms
[2022-04-06 16:53:48 main:574] : INFO : Epoch 77 | loss: 0.0311178 | val_loss: 0.0311549 | Time: 71113.7 ms
[2022-04-06 16:56:51 main:574] : INFO : Epoch 78 | loss: 0.0311172 | val_loss: 0.0311526 | Time: 177593 ms
[2022-04-06 16:57:43 main:574] : INFO : Epoch 79 | loss: 0.0311169 | val_loss: 0.031153 | Time: 52215.8 ms
[2022-04-06 17:01:18 main:574] : INFO : Epoch 80 | loss: 0.0311167 | val_loss: 0.0311568 | Time: 210215 ms
[2022-04-06 17:04:26 main:574] : INFO : Epoch 81 | loss: 0.0311172 | val_loss: 0.0311528 | Time: 183450 ms
[2022-04-06 17:05:53 main:574] : INFO : Epoch 82 | loss: 0.0311161 | val_loss: 0.0311543 | Time: 86511.7 ms
[2022-04-06 17:07:41 main:574] : INFO : Epoch 83 | loss: 0.0311157 | val_loss: 0.0311542 | Time: 103423 ms
[2022-04-06 17:08:44 main:574] : INFO : Epoch 84 | loss: 0.0311162 | val_loss: 0.0311569 | Time: 62691.1 ms
[2022-04-06 17:10:41 main:574] : INFO : Epoch 85 | loss: 0.0311157 | val_loss: 0.0311535 | Time: 112386 ms
[2022-04-06 17:10:47 main:574] : INFO : Epoch 86 | loss: 0.0311139 | val_loss: 0.0311557 | Time: 5741.57 ms
[2022-04-06 17:12:49 main:574] : INFO : Epoch 87 | loss: 0.0311182 | val_loss: 0.0311492 | Time: 120055 ms
[2022-04-06 17:15:41 main:574] : INFO : Epoch 88 | loss: 0.0311181 | val_loss: 0.0311496 | Time: 172160 ms
[2022-04-06 17:19:27 main:574] : INFO : Epoch 89 | loss: 0.0311162 | val_loss: 0.0311516 | Time: 225618 ms
[2022-04-06 17:21:15 main:574] : INFO : Epoch 90 | loss: 0.0311177 | val_loss: 0.0311456 | Time: 103999 ms
[2022-04-06 17:21:22 main:574] : INFO : Epoch 91 | loss: 0.0311183 | val_loss: 0.0311461 | Time: 7002.51 ms
[2022-04-06 17:23:01 main:574] : INFO : Epoch 92 | loss: 0.0311159 | val_loss: 0.0311514 | Time: 97346.2 ms
[2022-04-06 17:25:48 main:574] : INFO : Epoch 93 | loss: 0.0311161 | val_loss: 0.0311503 | Time: 166828 ms
[2022-04-06 17:28:28 main:574] : INFO : Epoch 94 | loss: 0.031139 | val_loss: 0.0311685 | Time: 155619 ms
[2022-04-06 17:31:52 main:574] : INFO : Epoch 95 | loss: 0.03116 | val_loss: 0.031169 | Time: 198343 ms
[2022-04-06 17:33:48 main:574] : INFO : Epoch 96 | loss: 0.0311627 | val_loss: 0.0311645 | Time: 111025 ms
[2022-04-06 17:36:41 main:574] : INFO : Epoch 97 | loss: 0.031164 | val_loss: 0.0311613 | Time: 173354 ms
[2022-04-06 17:39:06 main:574] : INFO : Epoch 98 | loss: 0.031162 | val_loss: 0.0311624 | Time: 144913 ms
[2022-04-06 17:40:25 main:574] : INFO : Epoch 99 | loss: 0.0311621 | val_loss: 0.031162 | Time: 68827.6 ms
[2022-04-06 17:43:50 main:574] : INFO : Epoch 100 | loss: 0.0311672 | val_loss: 0.0311643 | Time: 204906 ms
[2022-04-06 17:43:55 main:574] : INFO : Epoch 101 | loss: 0.0311653 | val_loss: 0.0311672 | Time: 5539.63 ms
[2022-04-06 17:46:58 main:574] : INFO : Epoch 102 | loss: 0.0311663 | val_loss: 0.0311659 | Time: 181886 ms
[2022-04-06 17:50:38 main:574] : INFO : Epoch 103 | loss: 0.0311618 | val_loss: 0.0311601 | Time: 214921 ms
[2022-04-06 17:51:35 main:574] : INFO : Epoch 104 | loss: 0.031158 | val_loss: 0.0311591 | Time: 57256 ms
[2022-04-06 17:53:27 main:574] : INFO : Epoch 105 | loss: 0.0311583 | val_loss: 0.0311588 | Time: 111559 ms
[2022-04-06 17:56:38 main:574] : INFO : Epoch 106 | loss: 0.0311573 | val_loss: 0.0311576 | Time: 191547 ms
[2022-04-06 18:00:05 main:574] : INFO : Epoch 107 | loss: 0.03116 | val_loss: 0.0311631 | Time: 206719 ms
[2022-04-06 18:03:26 main:574] : INFO : Epoch 108 | loss: 0.0311648 | val_loss: 0.0311668 | Time: 194924 ms
[2022-04-06 18:04:58 main:574] : INFO : Epoch 109 | loss: 0.0311642 | val_loss: 0.0311684 | Time: 91938.8 ms
[2022-04-06 18:05:59 main:574] : INFO : Epoch 110 | loss: 0.0311621 | val_loss: 0.0311682 | Time: 59829.7 ms
[2022-04-06 18:08:44 main:574] : INFO : Epoch 111 | loss: 0.0311607 | val_loss: 0.031167 | Time: 165382 ms
[2022-04-06 18:08:52 main:574] : INFO : Epoch 112 | loss: 0.0311602 | val_loss: 0.0311661 | Time: 7932.73 ms
[2022-04-06 18:10:34 main:574] : INFO : Epoch 113 | loss: 0.0311617 | val_loss: 0.0311671 | Time: 95828.9 ms
[2022-04-06 18:12:36 main:574] : INFO : Epoch 114 | loss: 0.0311614 | val_loss: 0.0311636 | Time: 122666 ms
[2022-04-06 18:16:03 main:574] : INFO : Epoch 115 | loss: 0.0311602 | val_loss: 0.0311646 | Time: 201732 ms
[2022-04-06 18:19:54 main:574] : INFO : Epoch 116 | loss: 0.0311609 | val_loss: 0.0311623 | Time: 226827 ms
[2022-04-06 18:21:33 main:574] : INFO : Epoch 117 | loss: 0.0311607 | val_loss: 0.0311598 | Time: 93606 ms
[2022-04-06 18:23:01 main:574] : INFO : Epoch 118 | loss: 0.0311593 | val_loss: 0.0311597 | Time: 88180.8 ms
[2022-04-06 18:23:10 main:574] : INFO : Epoch 119 | loss: 0.0311588 | val_loss: 0.0311602 | Time: 8854.27 ms
[2022-04-06 18:24:04 main:574] : INFO : Epoch 120 | loss: 0.0311599 | val_loss: 0.0311605 | Time: 43033.7 ms
[2022-04-06 18:27:03 main:574] : INFO : Epoch 121 | loss: 0.0311589 | val_loss: 0.0311615 | Time: 173349 ms
[2022-04-06 18:27:27 main:574] : INFO : Epoch 122 | loss: 0.0311605 | val_loss: 0.0311636 | Time: 24133.9 ms
[2022-04-06 18:29:39 main:574] : INFO : Epoch 123 | loss: 0.0311581 | val_loss: 0.0311624 | Time: 129490 ms
[2022-04-06 18:30:08 main:574] : INFO : Epoch 124 | loss: 0.0311592 | val_loss: 0.0311612 | Time: 24435.9 ms
[2022-04-06 18:31:33 main:574] : INFO : Epoch 125 | loss: 0.0311583 | val_loss: 0.0311614 | Time: 84183.3 ms
[2022-04-06 18:33:25 main:574] : INFO : Epoch 126 | loss: 0.0311575 | val_loss: 0.0311664 | Time: 107250 ms
[2022-04-06 18:35:57 main:574] : INFO : Epoch 127 | loss: 0.0311618 | val_loss: 0.0311715 | Time: 152448 ms
[2022-04-06 18:40:08 main:574] : INFO : Epoch 128 | loss: 0.0311633 | val_loss: 0.0311643 | Time: 251222 ms
[2022-04-06 18:41:32 main:574] : INFO : Epoch 129 | loss: 0.0311595 | val_loss: 0.0311617 | Time: 78714.5 ms
[2022-04-06 18:44:53 main:574] : INFO : Epoch 130 | loss: 0.0311595 | val_loss: 0.0311676 | Time: 195994 ms
[2022-04-06 18:47:58 main:574] : INFO : Epoch 131 | loss: 0.0311594 | val_loss: 0.0311622 | Time: 178694 ms
[2022-04-06 18:50:51 main:574] : INFO : Epoch 132 | loss: 0.0311588 | val_loss: 0.0311642 | Time: 167925 ms
[2022-04-06 18:53:04 main:574] : INFO : Epoch 133 | loss: 0.0311603 | val_loss: 0.0311663 | Time: 133110 ms
[2022-04-06 18:53:34 main:574] : INFO : Epoch 134 | loss: 0.0311562 | val_loss: 0.0311642 | Time: 25062.8 ms
[2022-04-06 18:57:25 main:574] : INFO : Epoch 135 | loss: 0.0311571 | val_loss: 0.0311667 | Time: 230811 ms
[2022-04-06 19:00:30 main:574] : INFO : Epoch 136 | loss: 0.0311563 | val_loss: 0.0311642 | Time: 185784 ms
[2022-04-06 19:00:37 main:574] : INFO : Epoch 137 | loss: 0.031157 | val_loss: 0.0311649 | Time: 6571.78 ms
[2022-04-06 19:04:11 main:574] : INFO : Epoch 138 | loss: 0.0311562 | val_loss: 0.031166 | Time: 211221 ms
[2022-04-06 19:07:18 main:574] : INFO : Epoch 139 | loss: 0.0311569 | val_loss: 0.0311652 | Time: 187574 ms
[2022-04-06 19:10:29 main:574] : INFO : Epoch 140 | loss: 0.0311591 | val_loss: 0.0311705 | Time: 190854 ms
[2022-04-06 19:12:18 main:574] : INFO : Epoch 141 | loss: 0.0311607 | val_loss: 0.0311738 | Time: 108984 ms
[2022-04-06 19:15:03 main:574] : INFO : Epoch 142 | loss: 0.0311575 | val_loss: 0.0311687 | Time: 159800 ms
[2022-04-06 19:18:56 main:574] : INFO : Epoch 143 | loss: 0.0311549 | val_loss: 0.0311668 | Time: 224408 ms
[2022-04-06 19:20:25 main:574] : INFO : Epoch 144 | loss: 0.031156 | val_loss: 0.0311658 | Time: 88742.2 ms
[2022-04-06 19:20:59 main:574] : INFO : Epoch 145 | loss: 0.0311559 | val_loss: 0.0311654 | Time: 33664.2 ms
[2022-04-06 19:21:40 main:574] : INFO : Epoch 146 | loss: 0.0311548 | val_loss: 0.0311671 | Time: 40959.2 ms
[2022-04-06 19:23:31 main:574] : INFO : Epoch 147 | loss: 0.0311563 | val_loss: 0.0311656 | Time: 111524 ms
[2022-04-06 19:25:11 main:574] : INFO : Epoch 148 | loss: 0.0311568 | val_loss: 0.0311663 | Time: 93894.7 ms
[2022-04-06 19:28:52 main:574] : INFO : Epoch 149 | loss: 0.0311567 | val_loss: 0.0311649 | Time: 210187 ms
[2022-04-06 19:32:10 main:574] : INFO : Epoch 150 | loss: 0.0311543 | val_loss: 0.0311645 | Time: 198273 ms
[2022-04-06 19:33:33 main:574] : INFO : Epoch 151 | loss: 0.0311538 | val_loss: 0.0311667 | Time: 82638.8 ms
[2022-04-06 19:36:20 main:574] : INFO : Epoch 152 | loss: 0.0311529 | val_loss: 0.0311677 | Time: 166916 ms
[2022-04-06 19:36:26 main:574] : INFO : Epoch 153 | loss: 0.0311525 | val_loss: 0.0311648 | Time: 6100.15 ms
[2022-04-06 19:40:26 main:574] : INFO : Epoch 154 | loss: 0.0311531 | val_loss: 0.0311705 | Time: 238926 ms
[2022-04-06 19:42:20 main:574] : INFO : Epoch 155 | loss: 0.0311528 | val_loss: 0.0311671 | Time: 114558 ms
[2022-04-06 19:44:40 main:574] : INFO : Epoch 156 | loss: 0.0311511 | val_loss: 0.0311656 | Time: 139311 ms
[2022-04-06 19:47:31 main:574] : INFO : Epoch 157 | loss: 0.0311528 | val_loss: 0.0311666 | Time: 165467 ms
[2022-04-06 19:47:43 main:574] : INFO : Epoch 158 | loss: 0.0311568 | val_loss: 0.0311627 | Time: 12524.3 ms
[2022-04-06 19:49:59 main:574] : INFO : Epoch 159 | loss: 0.0311506 | val_loss: 0.0311629 | Time: 135715 ms
[2022-04-06 19:50:46 main:574] : INFO : Epoch 160 | loss: 0.0311525 | val_loss: 0.0311637 | Time: 47025.7 ms
[2022-04-06 19:51:39 main:574] : INFO : Epoch 161 | loss: 0.0311537 | val_loss: 0.0311674 | Time: 53255.2 ms
[2022-04-06 19:55:34 main:574] : INFO : Epoch 162 | loss: 0.0311579 | val_loss: 0.0311631 | Time: 234659 ms
[2022-04-06 19:59:33 main:574] : INFO : Epoch 163 | loss: 0.0311559 | val_loss: 0.0311632 | Time: 238642 ms
[2022-04-06 20:00:59 main:574] : INFO : Epoch 164 | loss: 0.0311549 | val_loss: 0.0311631 | Time: 86082 ms
[2022-04-06 20:03:10 main:574] : INFO : Epoch 165 | loss: 0.0311558 | val_loss: 0.031162 | Time: 131159 ms
[2022-04-06 20:04:26 main:574] : INFO : Epoch 166 | loss: 0.0311533 | val_loss: 0.0311635 | Time: 75503.9 ms
[2022-04-06 20:07:22 main:574] : INFO : Epoch 167 | loss: 0.0311521 | val_loss: 0.031164 | Time: 170742 ms
[2022-04-06 20:10:36 main:574] : INFO : Epoch 168 | loss: 0.0311529 | val_loss: 0.0311657 | Time: 188948 ms
[2022-04-06 20:13:14 main:574] : INFO : Epoch 169 | loss: 0.0311536 | val_loss: 0.0311626 | Time: 158774 ms
[2022-04-06 20:16:11 main:574] : INFO : Epoch 170 | loss: 0.0311566 | val_loss: 0.0311671 | Time: 171293 ms
[2022-04-06 20:19:53 main:574] : INFO : Epoch 171 | loss: 0.0311648 | val_loss: 0.0311664 | Time: 216150 ms
[2022-04-06 20:20:56 main:574] : INFO : Epoch 172 | loss: 0.0311663 | val_loss: 0.0311669 | Time: 58027.1 ms
[2022-04-06 20:22:26 main:574] : INFO : Epoch 173 | loss: 0.0311643 | val_loss: 0.0311656 | Time: 90286 ms
[2022-04-06 20:22:42 main:574] : INFO : Epoch 174 | loss: 0.0311641 | val_loss: 0.0311641 | Time: 10891 ms
[2022-04-06 20:23:38 main:574] : INFO : Epoch 175 | loss: 0.0311637 | val_loss: 0.0311623 | Time: 55393 ms
[2022-04-06 20:26:16 main:574] : INFO : Epoch 176 | loss: 0.0311601 | val_loss: 0.0311619 | Time: 158080 ms
[2022-04-06 20:29:50 main:574] : INFO : Epoch 177 | loss: 0.0311589 | val_loss: 0.0311637 | Time: 208993 ms
[2022-04-06 20:32:41 main:574] : INFO : Epoch 178 | loss: 0.031158 | val_loss: 0.0311624 | Time: 170956 ms
[2022-04-06 20:33:28 main:574] : INFO : Epoch 179 | loss: 0.0311572 | val_loss: 0.031166 | Time: 46887.8 ms
[2022-04-06 20:36:10 main:574] : INFO : Epoch 180 | loss: 0.0311577 | val_loss: 0.0311671 | Time: 157342 ms
[2022-04-06 20:38:42 main:574] : INFO : Epoch 181 | loss: 0.0311583 | val_loss: 0.031166 | Time: 152053 ms
[2022-04-06 20:40:33 main:574] : INFO : Epoch 182 | loss: 0.0311553 | val_loss: 0.0311656 | Time: 105599 ms
[2022-04-06 20:44:02 main:574] : INFO : Epoch 183 | loss: 0.0311563 | val_loss: 0.0311639 | Time: 208459 ms
[2022-04-06 20:44:22 main:574] : INFO : Epoch 184 | loss: 0.0311583 | val_loss: 0.0311655 | Time: 20103.4 ms
[2022-04-06 20:44:28 main:574] : INFO : Epoch 185 | loss: 0.031157 | val_loss: 0.0311668 | Time: 6339.9 ms
[2022-04-06 20:48:19 main:574] : INFO : Epoch 186 | loss: 0.0311558 | val_loss: 0.0311676 | Time: 230022 ms
[2022-04-06 20:50:53 main:574] : INFO : Epoch 187 | loss: 0.0311566 | val_loss: 0.031167 | Time: 149204 ms
[2022-04-06 20:53:35 main:574] : INFO : Epoch 188 | loss: 0.031157 | val_loss: 0.0311681 | Time: 151313 ms
[2022-04-06 20:56:41 main:574] : INFO : Epoch 189 | loss: 0.0311575 | val_loss: 0.0311639 | Time: 185329 ms
[2022-04-06 20:59:46 main:574] : INFO : Epoch 190 | loss: 0.0311564 | val_loss: 0.0311646 | Time: 185011 ms
[2022-04-06 21:02:29 main:574] : INFO : Epoch 191 | loss: 0.0311582 | val_loss: 0.0311669 | Time: 156751 ms
[2022-04-06 21:04:34 main:574] : INFO : Epoch 192 | loss: 0.0311611 | val_loss: 0.0311705 | Time: 125845 ms
[2022-04-06 21:05:17 main:574] : INFO : Epoch 193 | loss: 0.0311613 | val_loss: 0.0311684 | Time: 42096.5 ms
[2022-04-06 21:05:24 main:574] : INFO : Epoch 194 | loss: 0.0311594 | val_loss: 0.0311681 | Time: 6141.11 ms
[2022-04-06 21:08:01 main:574] : INFO : Epoch 195 | loss: 0.0311579 | val_loss: 0.0311651 | Time: 156532 ms
[2022-04-06 21:08:54 main:574] : INFO : Epoch 196 | loss: 0.0311568 | val_loss: 0.0311688 | Time: 47205.1 ms
[2022-04-06 21:10:07 main:574] : INFO : Epoch 197 | loss: 0.0311604 | val_loss: 0.0311676 | Time: 72686.4 ms
[2022-04-06 21:11:45 main:574] : INFO : Epoch 198 | loss: 0.031159 | val_loss: 0.0311665 | Time: 92602.5 ms
[2022-04-06 21:14:02 main:574] : INFO : Epoch 199 | loss: 0.031162 | val_loss: 0.031169 | Time: 137221 ms
[2022-04-06 21:17:51 main:574] : INFO : Epoch 200 | loss: 0.0311581 | val_loss: 0.0311718 | Time: 223248 ms
[2022-04-06 21:20:08 main:574] : INFO : Epoch 201 | loss: 0.0311614 | val_loss: 0.0311752 | Time: 137599 ms
[2022-04-06 21:21:27 main:574] : INFO : Epoch 202 | loss: 0.0311656 | val_loss: 0.0311761 | Time: 74337.8 ms
[2022-04-06 21:24:27 main:574] : INFO : Epoch 203 | loss: 0.0311775 | val_loss: 0.0311781 | Time: 166492 ms
[2022-04-06 21:24:38 main:574] : INFO : Epoch 204 | loss: 0.0311801 | val_loss: 0.0311792 | Time: 10518.3 ms
[2022-04-06 21:26:25 main:574] : INFO : Epoch 205 | loss: 0.0311783 | val_loss: 0.0311807 | Time: 106518 ms
[2022-04-06 21:30:19 main:574] : INFO : Epoch 206 | loss: 0.0311776 | val_loss: 0.0311827 | Time: 229183 ms
[2022-04-06 21:33:33 main:574] : INFO : Epoch 207 | loss: 0.0311767 | val_loss: 0.0311756 | Time: 194047 ms
[2022-04-06 21:35:55 main:574] : INFO : Epoch 208 | loss: 0.0311749 | val_loss: 0.0311763 | Time: 142433 ms
[2022-04-06 21:36:50 main:574] : INFO : Epoch 209 | loss: 0.0311732 | val_loss: 0.0311761 | Time: 54243.3 ms
[2022-04-06 21:38:00 main:574] : INFO : Epoch 210 | loss: 0.0311733 | val_loss: 0.031176 | Time: 70786.1 ms
[2022-04-06 21:40:40 main:574] : INFO : Epoch 211 | loss: 0.0311715 | val_loss: 0.031179 | Time: 153812 ms
[2022-04-06 21:44:04 main:574] : INFO : Epoch 212 | loss: 0.0311702 | val_loss: 0.0311748 | Time: 204202 ms
[2022-04-06 21:46:21 main:574] : INFO : Epoch 213 | loss: 0.0311697 | val_loss: 0.0311759 | Time: 137404 ms
[2022-04-06 21:49:34 main:574] : INFO : Epoch 214 | loss: 0.0311715 | val_loss: 0.0311769 | Time: 187251 ms
[2022-04-06 21:51:39 main:574] : INFO : Epoch 215 | loss: 0.0311705 | val_loss: 0.0311735 | Time: 120356 ms
[2022-04-06 21:53:42 main:574] : INFO : Epoch 216 | loss: 0.0311673 | val_loss: 0.0311737 | Time: 122440 ms
[2022-04-06 21:57:04 main:574] : INFO : Epoch 217 | loss: 0.0311684 | val_loss: 0.0311744 | Time: 202495 ms
[2022-04-06 22:00:17 main:574] : INFO : Epoch 218 | loss: 0.0311673 | val_loss: 0.0311726 | Time: 192355 ms
[2022-04-06 22:02:56 main:574] : INFO : Epoch 219 | loss: 0.0311655 | val_loss: 0.0311727 | Time: 154388 ms
[2022-04-06 22:04:03 main:574] : INFO : Epoch 220 | loss: 0.0311676 | val_loss: 0.0311754 | Time: 54534.7 ms
[2022-04-06 22:05:53 main:574] : INFO : Epoch 221 | loss: 0.0311673 | val_loss: 0.0311725 | Time: 104785 ms
[2022-04-06 22:07:17 main:574] : INFO : Epoch 222 | loss: 0.031164 | val_loss: 0.0311729 | Time: 84323.5 ms
[2022-04-06 22:10:37 main:574] : INFO : Epoch 223 | loss: 0.0311641 | val_loss: 0.0311713 | Time: 193972 ms
[2022-04-06 22:13:21 main:574] : INFO : Epoch 224 | loss: 0.0311644 | val_loss: 0.0311738 | Time: 164337 ms
[2022-04-06 22:15:07 main:574] : INFO : Epoch 225 | loss: 0.0311656 | val_loss: 0.0311743 | Time: 100289 ms
[2022-04-06 22:15:39 main:574] : INFO : Epoch 226 | loss: 0.0311649 | val_loss: 0.031179 | Time: 31455.6 ms
[2022-04-06 22:19:31 main:574] : INFO : Epoch 227 | loss: 0.0311675 | val_loss: 0.0311769 | Time: 227993 ms
[2022-04-06 22:21:34 main:574] : INFO : Epoch 228 | loss: 0.0311658 | val_loss: 0.0311747 | Time: 122912 ms
[2022-04-06 22:23:39 main:574] : INFO : Epoch 229 | loss: 0.0311646 | val_loss: 0.0311751 | Time: 112105 ms
[2022-04-06 22:25:15 main:574] : INFO : Epoch 230 | loss: 0.0311629 | val_loss: 0.0311764 | Time: 95418.2 ms
[2022-04-06 22:28:07 main:574] : INFO : Epoch 231 | loss: 0.0311622 | val_loss: 0.0311726 | Time: 166883 ms
[2022-04-06 22:28:15 main:574] : INFO : Epoch 232 | loss: 0.0311628 | val_loss: 0.0311726 | Time: 6916.67 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 04/06/22 22:28:32
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\15;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 000000002c280000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000294d0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000028900000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000027260000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000283d0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026c90000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1586 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1586
ModLoad: 0000000027740000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000026f90000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026bf0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000270a0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000028850000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000293e0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000027770000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027870000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019820000 0000000000058000 C:\ProgramData\BOINC\slots\15\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000007c40000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000001a230000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000000cd10000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000000d570000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 00000000aad10000 000000000abe7000 C:\ProgramData\BOINC\slots\15\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000ca300000 00000000002db000 C:\ProgramData\BOINC\slots\15\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000e4610000 000000000020d000 C:\ProgramData\BOINC\slots\15\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 0000000006160000 0000000000040000 C:\ProgramData\BOINC\slots\15\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000007af60000 0000000024b49000 C:\ProgramData\BOINC\slots\15\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000046d0000 0000000000045000 C:\ProgramData\BOINC\slots\15\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bd1e0000 00000000034b2000 C:\ProgramData\BOINC\slots\15\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000219e0000 0000000000010000 C:\ProgramData\BOINC\slots\15\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 000000005fbb0000 000000001b3a4000 C:\ProgramData\BOINC\slots\15\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000ba130000 00000000030a5000 C:\ProgramData\BOINC\slots\15\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000056a10000 000000000919f000 C:\ProgramData\BOINC\slots\15\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000052cb0000 0000000003d5b000 C:\ProgramData\BOINC\slots\15\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000b6ae0000 0000000001eca000 C:\ProgramData\BOINC\slots\15\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000028710000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000024a40000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000289c0000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000271a0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000026cc0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000269b0000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000271f0000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026d80000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000267d0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000027a50000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 0000000024d20000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000028e30000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000026650000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027690000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000275a0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000c9340000 00000000002cd000 C:\WINDOWS\system32\nvcuda.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000a7620000 00000000016a0000 C:\WINDOWS\system32\DriverStore\FileRepository\nvhdc.inf_amd64_d11413f6fd8fe259\nvcuda64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000216e0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001dc70000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 0000000018c60000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000265c0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000052540000 0000000000768000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1165
ModLoad: 0000000019dc0000 0000000000009000 C:\ProgramData\BOINC\slots\15\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000a9d40000 0000000000fca000 C:\ProgramData\BOINC\slots\15\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000026cf0000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3852, Write: 3069, Other 8273
- I/O Transfers Counters -
Read: 7546275, Write: 6869115, Other 57560
- Paged Pool Usage -
QuotaPagedPoolUsage: 3533544, QuotaPeakPagedPoolUsage: 3533816
QuotaNonPagedPoolUsage: 58336, QuotaPeakNonPagedPoolUsage: 58744
- Virtual Memory Usage -
VirtualSize: 398434304, PeakVirtualSize: 1023766528
- Pagefile Usage -
PagefileUsage: 398434304, PeakPagefileUsage: 398434304
- Working Set Size -
WorkingSetSize: -2125213696, PeakWorkingSetSize: -2060353536, PageFaultCount: 883401
*** Dump of thread ID 7564 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 158464848.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=000000002c538c40 rdx=00000000091fe3f0 rsi=0000000000000000 rdi=0000000000000000
r8=00000000091fe3f0 r9=000000002c538c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000273290d2 rsp=00000000091fe3c8 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
091fe3c0 2c2b9100 00000001 091fe3f0 091fe3f0 2c538c30 KERNELBASE!DebugBreak+0x0
091fe800 2c2b9ce4 cd800000 091feb64 00000000 0000032c mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
091ff760 2c2ba084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
091ff790 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
091ff7c0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
091ff840 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 5940 (state: Ready): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 224531248.000000, User Time: 4849999872.000000, Wait Time: 158464832.000000
- Registers -
rax=0000000000000024 rbx=0000000008ce0f80 rcx=0000000000000000 rdx=0000000008ce1448 rsi=0000000000000000 rdi=000000000000000c
r8=0000000000000024 r9=0000000000000024 r10=0000000008ce14b8 r11=0000000008ce1480 r12=0000000008ce1448 r13=0000000008ce14f0
r14=00000000bb843010 r15=0000000008ce13e8 rip=0000000060a967ca rsp=0000000008ce0ec0 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
08ce0ef0 60a97139 bb843010 08ce14b8 08ce13e8 00000000 cudnn64_7!cudnnSetDropoutDescriptor+0x0
08ce12d0 609b4098 000002ee 08ce13bc 000002ee 00000003 cudnn64_7!cudnnSetDropoutDescriptor+0x0
08ce15a0 609b700e 08ce15e8 00000001 d6734e70 6681ff40 cudnn64_7!cudnnSetDropoutDescriptor+0x0
08ce1690 608b82ca bb843010 00000000 00000000 00000024 cudnn64_7!cudnnSetDropoutDescriptor+0x0
08ce1760 608ca9bb 0000000c 3c287000 00000003 00000000 cudnn64_7!cudnnSetAttnDescriptor+0x0
08ce1c10 608beff1 08ce3ef8 512a5fc0 00000005 ced4afb0 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
08ce1db0 608c7d8e cc6b1300 00000000 ced4afb0 08ce2078 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
08ce1f50 608ddcd3 00000000 00000000 d0321700 08ce2070 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
08ce40e0 7dcd89fd cc6aace0 3c200000 cc6aace0 512a5fc0 cudnn64_7!cudnnRNNForwardTraining+0x0
08ce45c0 7dd325ea 08ce4688 00000001 08ce4688 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
08ce46a0 7dd40cbf 08ce4d90 08ce6598 08ce56e0 294fb86b torch_cuda!at::native::set_storage_cuda_+0x0
08ce4780 b075b9fc 08ce4d90 00000000 bb888170 bb888140 torch_cuda!at::native::set_storage_cuda_+0x0
08ce4900 b07931a2 00000001 b1f067c2 08ce4934 294fb86b torch_cpu!at::native::mkldnn_sigmoid_+0x0
08ce4be0 b1a42613 00000000 00000000 00000003 aa5ebf08 torch_cpu!at::_cudnn_rnn+0x0
08ce50e0 b1a5080b 782eaa86 cfa52c90 08ce5210 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
08ce51c0 b075b9fc 08ce5740 b1b75878 08ce5320 a7647a77 torch_cpu!torch::autograd::GraphRoot::apply+0x0
08ce5340 b07931a2 ced4a4e0 00000000 782eb436 7ebfa3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
08ce5620 7dcd69a1 08ce0001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
08ce57f0 7dce0117 00000000 08ce6598 046d2e0d 08ce5910 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
08ce5880 b04e107b b5631ac0 08ce5ba0 19a18a3e a7817783 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
08ce5af0 b0513b16 00000001 08ce6598 00000080 cff09d68 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
08ce5c80 b088ae89 08ce5cf8 00000001 08ce5cf8 00000001 torch_cpu!at::native::gru+0x0
08ce5d10 b1b79fa3 08ce5e10 00000001 08ce5f90 08ce6598 torch_cpu!at::zeros_out+0x0
08ce5da0 b087dd35 19871ea8 6768c460 00000000 19871ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
08ce5e20 b07588ec b56375c8 4ec7ab70 d007a770 08ce6040 torch_cpu!at::zeros_out+0x0
08ce5f20 b07cc1bb 0000000c 00000080 00000004 cfa52c00 torch_cpu!at::native::mkldnn_sigmoid_+0x0
08ce5fa0 b22b7294 08ce60c0 4ec7ab70 08ce60b0 08ce6258 torch_cpu!at::gru+0x0
08ce61d0 b22b62ab 08ce6570 b06e030e 08ce6570 08ce6598 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
08ce62c0 2c292621 00000002 08ce6fd0 00000000 19871ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
08ce64c0 2c29705a 00000000 08ce0000 08ce0601 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
08ce69c0 2c29b950 08ce6ac0 08ce6ac0 00000000 2f5636a0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::then+0x0
08cff760 2c4dc6f8 67c36880 00000000 00000000 67b7cbf0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
08cff7a0 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
08cff7d0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
08cff850 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 14136 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 158464848.000000
- Registers -
rax=000000000000005b rbx=000000000000000e rcx=000000000000000e rdx=00000000092ff6b0 rsi=0000000000000000 rdi=000000000000000e
r8=0000000000000001 r9=0000000067d3f500 r10=0000000000000000 r11=00000000092ff620 r12=0000000000000064 r13=00000000092ff6b0
r14=00000000092ff3b0 r15=0000000000000000 rip=000000002956d8c4 rsp=00000000092ff358 rbp=0000000067d3f5a0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
092ff350 272acb20 00000000 08abc000 2963a4d0 08abc000 ntdll!ZwWaitForMultipleObjects+0x0
092ff640 272aca1e 66216470 004e0049 0000000e 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
092ff680 a77fd554 67d4a1c0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
092ff8c0 a78a58cd 00000000 67d3f5a0 00000000 00000000 nvcuda64!+0x0
092ff970 a77fcf23 ed413acb 52d4a7de 682e57c0 00000000 nvcuda64!cuProfilerStop+0x0
092ff9a0 a7b344c4 682e57c0 00000000 00000000 00000000 nvcuda64!+0x0
092ff9d0 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
092ffa00 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
092ffa80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 17256 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 156524352.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002bc rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002bc
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000067d13fa0 r13=0000000000000000
r14=00000000000002bc r15=0000000000000000 rip=000000002956cdf4 rsp=00000000093ff998 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
093ff990 27281a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
093ffa30 a7a25d0e 682e5280 00000000 00000000 000002bc KERNELBASE!WaitForSingleObjectEx+0x0
093ffc70 a77fcf23 00000000 682e5280 a7620000 67d29240 nvcuda64!cuProfilerStop+0x0
093ffca0 a7b344c4 682e5280 00000000 00000000 00000000 nvcuda64!+0x0
093ffcd0 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
093ffd00 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
093ffd80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 12672 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 135312496.000000, User Time: 3155000064.000000, Wait Time: 158464832.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000b36d6bb8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000b36d6bb8
r8=0000000066301740 r9=0000000000000001 r10=0000000000008000 r11=00000000094fe290 r12=0000000000000000 r13=0000000000000000
r14=00000000094ff880 r15=00000000b36d6b68 rip=0000000029570764 rsp=00000000094ff858 rbp=00000000094ff8b0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
094ff850 29534021 cab06da8 07c52f97 bbd28da0 bbd28d90 ntdll!NtWaitForAlertByThreadId+0x0
094ff8d0 272ccfa9 b36d6bb0 b36d6ba8 b36d6b60 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
094ff910 07c7bf3d bbd28dc0 00000000 00000000 b1ef05b4 KERNELBASE!SleepConditionVariableSRW+0x0
094ff950 07c52798 00000000 bbd28dc0 00000000 00000000 MSVCP140!__crtSetThreadpoolWait+0x0
094ff980 07c52a4a 15648454 00000000 094ffb90 b36d6ba8 MSVCP140!std::_Winerror_message+0x0
094ff9b0 b1ef064b b36d6b60 094ffab0 b36d6b60 bbd28da0 MSVCP140!_Cnd_wait+0x0
094ff9f0 b1ef25b1 d07bcd40 ffffffff bbd28da0 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
094ffce0 b1ef2471 b5416d10 094ffd20 b5416d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
094ffd30 b1ee9744 00080001 d00671a0 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
094ffd60 270c1bb2 d00671a0 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
094ffd90 28917034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
094ffdc0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
094ffe40 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)