Task 14092772
| Name | ParityModified-1647042820-20694-3-0_0 |
| Workunit | 11160865 |
| Created | 30 Mar 2022, 13:58:11 UTC |
| Sent | 5 Apr 2022, 20:18:08 UTC |
| Report deadline | 13 Apr 2022, 20:18:08 UTC |
| Received | 7 Apr 2022, 21:38:45 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED |
| Computer ID | 4503 |
| Run time | 8 hours 8 min 31 sec |
| CPU time | 14 min 11 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 10,073.24 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.37 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
exceeded elapsed time limit 29305.33 (4000000.00G/136.49G)</message>
<stderr_txt>
shold: 0.0001
[2022-04-07 06:55:19 main:477] : INFO : Max Epochs: 2048
[2022-04-07 06:55:19 main:478] : INFO : Batch Size: 128
[2022-04-07 06:55:19 main:479] : INFO : Learning Rate: 0.01
[2022-04-07 06:55:19 main:480] : INFO : Patience: 10
[2022-04-07 06:55:19 main:481] : INFO : Hidden Width: 12
[2022-04-07 06:55:19 main:482] : INFO : # Recurrent Layers: 4
[2022-04-07 06:55:19 main:483] : INFO : # Backend Layers: 4
[2022-04-07 06:55:19 main:484] : INFO : # Threads: 1
[2022-04-07 06:55:19 main:486] : INFO : Preparing Dataset
[2022-04-07 06:55:19 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-04-07 06:55:20 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-04-07 06:55:30 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-04-07 06:55:30 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-04-07 06:55:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-04-07 06:55:33 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-04-07 06:55:33 main:494] : INFO : Creating Model
[2022-04-07 06:55:33 main:507] : INFO : Preparing config file
[2022-04-07 06:55:33 main:519] : INFO : Creating new config file
[2022-04-07 06:55:33 main:538] : INFO : This is a continuation WU, loading previous network
[2022-04-07 06:55:42 main:559] : INFO : Loading DataLoader into Memory
[2022-04-07 06:55:59 main:562] : INFO : Starting Training
[2022-04-07 07:00:18 main:574] : INFO : Epoch 1 | loss: 0.0312628 | val_loss: 0.0312469 | Time: 258865 ms
[2022-04-07 07:02:56 main:574] : INFO : Epoch 2 | loss: 0.0312476 | val_loss: 0.0312461 | Time: 158791 ms
[2022-04-07 07:05:58 main:574] : INFO : Epoch 3 | loss: 0.0312459 | val_loss: 0.0312457 | Time: 171861 ms
[2022-04-07 07:07:58 main:574] : INFO : Epoch 4 | loss: 0.0312446 | val_loss: 0.0312423 | Time: 119658 ms
[2022-04-07 07:10:53 main:574] : INFO : Epoch 5 | loss: 0.0312443 | val_loss: 0.0312438 | Time: 174989 ms
[2022-04-07 07:13:02 main:574] : INFO : Epoch 6 | loss: 0.0312453 | val_loss: 0.0312482 | Time: 123658 ms
[2022-04-07 07:13:09 main:574] : INFO : Epoch 7 | loss: 0.0312484 | val_loss: 0.0312463 | Time: 6874.12 ms
[2022-04-07 07:17:02 main:574] : INFO : Epoch 8 | loss: 0.0312471 | val_loss: 0.0312474 | Time: 225619 ms
[2022-04-07 07:20:52 main:574] : INFO : Epoch 9 | loss: 0.0312476 | val_loss: 0.0312457 | Time: 230038 ms
[2022-04-07 07:23:54 main:574] : INFO : Epoch 10 | loss: 0.0312468 | val_loss: 0.0312465 | Time: 182862 ms
[2022-04-07 07:24:26 main:574] : INFO : Epoch 11 | loss: 0.0312474 | val_loss: 0.0312477 | Time: 31055 ms
[2022-04-07 07:26:45 main:574] : INFO : Epoch 12 | loss: 0.0312482 | val_loss: 0.0312453 | Time: 134451 ms
[2022-04-07 07:30:01 main:574] : INFO : Epoch 13 | loss: 0.0312467 | val_loss: 0.0312476 | Time: 190101 ms
[2022-04-07 07:30:06 main:574] : INFO : Epoch 14 | loss: 0.0312471 | val_loss: 0.0312447 | Time: 4993.85 ms
[2022-04-07 07:31:45 main:574] : INFO : Epoch 15 | loss: 0.0312476 | val_loss: 0.0312515 | Time: 98304.6 ms
[2022-04-07 07:34:38 main:574] : INFO : Epoch 16 | loss: 0.0312472 | val_loss: 0.0312453 | Time: 168049 ms
[2022-04-07 07:35:56 main:574] : INFO : Epoch 17 | loss: 0.0312461 | val_loss: 0.0312452 | Time: 73331.4 ms
[2022-04-07 07:38:19 main:574] : INFO : Epoch 18 | loss: 0.031246 | val_loss: 0.0312456 | Time: 142788 ms
[2022-04-07 07:41:00 main:574] : INFO : Epoch 19 | loss: 0.031246 | val_loss: 0.0312448 | Time: 156837 ms
[2022-04-07 07:44:10 main:574] : INFO : Epoch 20 | loss: 0.0312461 | val_loss: 0.0312448 | Time: 189648 ms
[2022-04-07 07:48:14 main:574] : INFO : Epoch 21 | loss: 0.0312452 | val_loss: 0.0312447 | Time: 235100 ms
[2022-04-07 07:50:59 main:574] : INFO : Epoch 22 | loss: 0.0312449 | val_loss: 0.0312439 | Time: 165045 ms
[2022-04-07 07:52:01 main:574] : INFO : Epoch 23 | loss: 0.0312453 | val_loss: 0.0312469 | Time: 61366.3 ms
[2022-04-07 07:53:16 main:574] : INFO : Epoch 24 | loss: 0.0312456 | val_loss: 0.0312441 | Time: 75069.8 ms
[2022-04-07 07:54:03 main:574] : INFO : Epoch 25 | loss: 0.0312448 | val_loss: 0.031244 | Time: 44900.6 ms
[2022-04-07 07:56:50 main:574] : INFO : Epoch 26 | loss: 0.031245 | val_loss: 0.0312442 | Time: 167743 ms
[2022-04-07 08:00:47 main:574] : INFO : Epoch 27 | loss: 0.0312447 | val_loss: 0.0312446 | Time: 231542 ms
[2022-04-07 08:03:51 main:574] : INFO : Epoch 28 | loss: 0.031245 | val_loss: 0.0312439 | Time: 184231 ms
[2022-04-07 08:05:32 main:574] : INFO : Epoch 29 | loss: 0.0312442 | val_loss: 0.0312437 | Time: 100279 ms
[2022-04-07 08:06:44 main:574] : INFO : Epoch 30 | loss: 0.0312436 | val_loss: 0.0312448 | Time: 72180.3 ms
[2022-04-07 08:10:25 main:574] : INFO : Epoch 31 | loss: 0.0312443 | val_loss: 0.0312431 | Time: 220856 ms
[2022-04-07 08:13:43 main:574] : INFO : Epoch 32 | loss: 0.0312437 | val_loss: 0.031243 | Time: 198302 ms
[2022-04-07 08:16:39 main:574] : INFO : Epoch 33 | loss: 0.0312435 | val_loss: 0.031243 | Time: 175469 ms
[2022-04-07 08:16:51 main:574] : INFO : Epoch 34 | loss: 0.031245 | val_loss: 0.0312448 | Time: 7572.26 ms
[2022-04-07 08:18:03 main:574] : INFO : Epoch 35 | loss: 0.0312447 | val_loss: 0.0312447 | Time: 71957.8 ms
[2022-04-07 08:20:58 main:574] : INFO : Epoch 36 | loss: 0.0312442 | val_loss: 0.0312438 | Time: 169990 ms
[2022-04-07 08:21:50 main:574] : INFO : Epoch 37 | loss: 0.0312441 | val_loss: 0.0312447 | Time: 52387.9 ms
[2022-04-07 08:22:43 main:574] : INFO : Epoch 38 | loss: 0.0312437 | val_loss: 0.0312428 | Time: 52169.1 ms
[2022-04-07 08:24:49 main:574] : INFO : Epoch 39 | loss: 0.0312435 | val_loss: 0.0312427 | Time: 120517 ms
[2022-04-07 08:27:52 main:574] : INFO : Epoch 40 | loss: 0.031244 | val_loss: 0.0312431 | Time: 182862 ms
[2022-04-07 08:31:22 main:574] : INFO : Epoch 41 | loss: 0.0312458 | val_loss: 0.0312487 | Time: 210153 ms
[2022-04-07 08:33:50 main:574] : INFO : Epoch 42 | loss: 0.0312495 | val_loss: 0.0312504 | Time: 147325 ms
[2022-04-07 08:34:26 main:574] : INFO : Epoch 43 | loss: 0.0312496 | val_loss: 0.0312495 | Time: 36395.1 ms
[2022-04-07 08:35:25 main:574] : INFO : Epoch 44 | loss: 0.0312495 | val_loss: 0.0312474 | Time: 58864.5 ms
[2022-04-07 08:37:55 main:574] : INFO : Epoch 45 | loss: 0.0312492 | val_loss: 0.0312503 | Time: 145431 ms
[2022-04-07 08:40:32 main:574] : INFO : Epoch 46 | loss: 0.0312499 | val_loss: 0.0312503 | Time: 156743 ms
[2022-04-07 08:42:11 main:574] : INFO : Epoch 47 | loss: 0.0312509 | val_loss: 0.0312507 | Time: 98560.5 ms
[2022-04-07 08:44:33 main:574] : INFO : Epoch 48 | loss: 0.0312511 | val_loss: 0.0312501 | Time: 137145 ms
[2022-04-07 08:48:24 main:574] : INFO : Epoch 49 | loss: 0.0312497 | val_loss: 0.0312496 | Time: 225595 ms
[2022-04-07 08:50:56 main:574] : INFO : Epoch 50 | loss: 0.0312497 | val_loss: 0.0312491 | Time: 151858 ms
[2022-04-07 08:52:08 main:574] : INFO : Epoch 51 | loss: 0.0312495 | val_loss: 0.0312492 | Time: 67465.5 ms
[2022-04-07 08:53:59 main:574] : INFO : Epoch 52 | loss: 0.0312495 | val_loss: 0.0312482 | Time: 105123 ms
[2022-04-07 08:58:02 main:574] : INFO : Epoch 53 | loss: 0.0312495 | val_loss: 0.0312505 | Time: 243414 ms
[2022-04-07 09:01:13 main:574] : INFO : Epoch 54 | loss: 0.0312492 | val_loss: 0.0312494 | Time: 186451 ms
[2022-04-07 09:03:58 main:574] : INFO : Epoch 55 | loss: 0.0312495 | val_loss: 0.0312503 | Time: 164582 ms
[2022-04-07 09:06:31 main:574] : INFO : Epoch 56 | loss: 0.03125 | val_loss: 0.0312516 | Time: 147631 ms
[2022-04-07 09:09:16 main:574] : INFO : Epoch 57 | loss: 0.0312498 | val_loss: 0.0312493 | Time: 159876 ms
[2022-04-07 09:11:05 main:574] : INFO : Epoch 58 | loss: 0.0312491 | val_loss: 0.0312496 | Time: 103364 ms
[2022-04-07 09:12:04 main:574] : INFO : Epoch 59 | loss: 0.0312494 | val_loss: 0.03125 | Time: 54301.6 ms
[2022-04-07 09:13:21 main:574] : INFO : Epoch 60 | loss: 0.0312502 | val_loss: 0.0312512 | Time: 76664.4 ms
[2022-04-07 09:15:56 main:574] : INFO : Epoch 61 | loss: 0.0312504 | val_loss: 0.0312492 | Time: 144901 ms
[2022-04-07 09:19:45 main:574] : INFO : Epoch 62 | loss: 0.0312491 | val_loss: 0.0312491 | Time: 216285 ms
[2022-04-07 09:19:52 main:574] : INFO : Epoch 63 | loss: 0.0312493 | val_loss: 0.0312495 | Time: 4887.86 ms
[2022-04-07 09:20:54 main:574] : INFO : Epoch 64 | loss: 0.0312494 | val_loss: 0.0312499 | Time: 61014.7 ms
[2022-04-07 09:21:43 main:574] : INFO : Epoch 65 | loss: 0.0312495 | val_loss: 0.0312498 | Time: 48513.9 ms
[2022-04-07 09:22:05 main:574] : INFO : Epoch 66 | loss: 0.0312499 | val_loss: 0.0312504 | Time: 21242.1 ms
[2022-04-07 09:23:57 main:574] : INFO : Epoch 67 | loss: 0.0312494 | val_loss: 0.0312491 | Time: 111487 ms
[2022-04-07 09:25:58 main:574] : INFO : Epoch 68 | loss: 0.0312495 | val_loss: 0.0312487 | Time: 120547 ms
[2022-04-07 09:29:11 main:574] : INFO : Epoch 69 | loss: 0.03125 | val_loss: 0.0312492 | Time: 187642 ms
[2022-04-07 09:30:55 main:574] : INFO : Epoch 70 | loss: 0.0312482 | val_loss: 0.0312488 | Time: 100184 ms
[2022-04-07 09:31:02 main:574] : INFO : Epoch 71 | loss: 0.0312482 | val_loss: 0.0312481 | Time: 6178.6 ms
[2022-04-07 09:33:56 main:574] : INFO : Epoch 72 | loss: 0.0312461 | val_loss: 0.0312501 | Time: 174255 ms
[2022-04-07 09:35:59 main:574] : INFO : Epoch 73 | loss: 0.0312483 | val_loss: 0.0312471 | Time: 122270 ms
[2022-04-07 09:39:39 main:574] : INFO : Epoch 74 | loss: 0.0312493 | val_loss: 0.0312491 | Time: 220059 ms
[2022-04-07 09:41:03 main:574] : INFO : Epoch 75 | loss: 0.0312491 | val_loss: 0.0312508 | Time: 76286.6 ms
[2022-04-07 09:45:17 main:574] : INFO : Epoch 76 | loss: 0.0312471 | val_loss: 0.031249 | Time: 248182 ms
[2022-04-07 09:48:51 main:574] : INFO : Epoch 77 | loss: 0.0312465 | val_loss: 0.0312473 | Time: 214907 ms
[2022-04-07 09:50:52 main:574] : INFO : Epoch 78 | loss: 0.0312482 | val_loss: 0.0312495 | Time: 120150 ms
[2022-04-07 09:52:13 main:574] : INFO : Epoch 79 | loss: 0.0312472 | val_loss: 0.0312498 | Time: 81204.3 ms
[2022-04-07 09:53:15 main:574] : INFO : Epoch 80 | loss: 0.0312492 | val_loss: 0.0312483 | Time: 62406.8 ms
[2022-04-07 09:54:17 main:574] : INFO : Epoch 81 | loss: 0.0312473 | val_loss: 0.0312508 | Time: 62012.9 ms
[2022-04-07 09:57:57 main:574] : INFO : Epoch 82 | loss: 0.0312478 | val_loss: 0.0312481 | Time: 214935 ms
[2022-04-07 10:01:29 main:574] : INFO : Epoch 83 | loss: 0.0312461 | val_loss: 0.031247 | Time: 211357 ms
[2022-04-07 10:04:24 main:574] : INFO : Epoch 84 | loss: 0.0312457 | val_loss: 0.0312519 | Time: 170567 ms
[2022-04-07 10:05:56 main:574] : INFO : Epoch 85 | loss: 0.0312463 | val_loss: 0.0312491 | Time: 87010.8 ms
[2022-04-07 10:09:01 main:574] : INFO : Epoch 86 | loss: 0.0312464 | val_loss: 0.0312499 | Time: 179285 ms
[2022-04-07 10:11:01 main:574] : INFO : Epoch 87 | loss: 0.0312482 | val_loss: 0.0312487 | Time: 120453 ms
[2022-04-07 10:13:27 main:574] : INFO : Epoch 88 | loss: 0.0312473 | val_loss: 0.0312494 | Time: 140775 ms
[2022-04-07 10:17:25 main:574] : INFO : Epoch 89 | loss: 0.0312467 | val_loss: 0.0312492 | Time: 232760 ms
[2022-04-07 10:20:28 main:574] : INFO : Epoch 90 | loss: 0.0312457 | val_loss: 0.0312481 | Time: 183362 ms
[2022-04-07 10:21:39 main:574] : INFO : Epoch 91 | loss: 0.0312465 | val_loss: 0.0312475 | Time: 64864.6 ms
[2022-04-07 10:21:44 main:574] : INFO : Epoch 92 | loss: 0.0312462 | val_loss: 0.0312498 | Time: 5481.86 ms
[2022-04-07 10:23:09 main:574] : INFO : Epoch 93 | loss: 0.0312467 | val_loss: 0.0312467 | Time: 84805.7 ms
[2022-04-07 10:25:48 main:574] : INFO : Epoch 94 | loss: 0.0312461 | val_loss: 0.0312505 | Time: 149015 ms
[2022-04-07 10:29:30 main:574] : INFO : Epoch 95 | loss: 0.031246 | val_loss: 0.0312499 | Time: 222064 ms
[2022-04-07 10:33:39 main:574] : INFO : Epoch 96 | loss: 0.0312456 | val_loss: 0.0312468 | Time: 244661 ms
[2022-04-07 10:35:29 main:574] : INFO : Epoch 97 | loss: 0.0312453 | val_loss: 0.0312476 | Time: 104772 ms
[2022-04-07 10:39:11 main:574] : INFO : Epoch 98 | loss: 0.0312457 | val_loss: 0.031247 | Time: 217289 ms
[2022-04-07 10:41:08 main:574] : INFO : Epoch 99 | loss: 0.0312474 | val_loss: 0.0312484 | Time: 116349 ms
[2022-04-07 10:43:33 main:574] : INFO : Epoch 100 | loss: 0.0312466 | val_loss: 0.0312475 | Time: 145723 ms
[2022-04-07 10:43:40 main:574] : INFO : Epoch 101 | loss: 0.0312483 | val_loss: 0.0312495 | Time: 6124.15 ms
[2022-04-07 10:45:19 main:574] : INFO : Epoch 102 | loss: 0.0312481 | val_loss: 0.0312481 | Time: 99191.8 ms
[2022-04-07 10:45:33 main:574] : INFO : Epoch 103 | loss: 0.0312467 | val_loss: 0.0312475 | Time: 8538.32 ms
[2022-04-07 10:47:22 main:574] : INFO : Epoch 104 | loss: 0.031246 | val_loss: 0.0312485 | Time: 108777 ms
[2022-04-07 10:50:02 main:574] : INFO : Epoch 105 | loss: 0.0312477 | val_loss: 0.0312477 | Time: 155774 ms
[2022-04-07 10:51:34 main:574] : INFO : Epoch 106 | loss: 0.0312495 | val_loss: 0.031247 | Time: 83494.5 ms
[2022-04-07 10:53:11 main:574] : INFO : Epoch 107 | loss: 0.0312469 | val_loss: 0.0312475 | Time: 86947.1 ms
[2022-04-07 10:53:36 main:574] : INFO : Epoch 108 | loss: 0.0312478 | val_loss: 0.0312534 | Time: 15332.9 ms
[2022-04-07 10:53:44 main:574] : INFO : Epoch 109 | loss: 0.0312483 | val_loss: 0.0312479 | Time: 7587.7 ms
[2022-04-07 10:57:34 main:574] : INFO : Epoch 110 | loss: 0.0312465 | val_loss: 0.0312465 | Time: 214447 ms
[2022-04-07 10:58:35 main:574] : INFO : Epoch 111 | loss: 0.0312465 | val_loss: 0.0312488 | Time: 56392.1 ms
[2022-04-07 10:58:41 main:574] : INFO : Epoch 112 | loss: 0.031246 | val_loss: 0.0312466 | Time: 5765.16 ms
[2022-04-07 11:02:44 main:574] : INFO : Epoch 113 | loss: 0.0312462 | val_loss: 0.0312485 | Time: 242637 ms
[2022-04-07 11:04:00 main:574] : INFO : Epoch 114 | loss: 0.0312458 | val_loss: 0.0312466 | Time: 71279.4 ms
[2022-04-07 11:07:13 main:574] : INFO : Epoch 115 | loss: 0.0312452 | val_loss: 0.0312472 | Time: 188051 ms
[2022-04-07 11:07:19 main:574] : INFO : Epoch 116 | loss: 0.0312455 | val_loss: 0.0312472 | Time: 6206.06 ms
[2022-04-07 11:09:36 main:574] : INFO : Epoch 117 | loss: 0.0312469 | val_loss: 0.03125 | Time: 134643 ms
[2022-04-07 11:11:06 main:574] : INFO : Epoch 118 | loss: 0.0312462 | val_loss: 0.0312469 | Time: 90460.6 ms
[2022-04-07 11:13:29 main:574] : INFO : Epoch 119 | loss: 0.0312454 | val_loss: 0.0312463 | Time: 131785 ms
[2022-04-07 11:16:45 main:574] : INFO : Epoch 120 | loss: 0.0312457 | val_loss: 0.0312464 | Time: 191760 ms
[2022-04-07 11:19:27 main:574] : INFO : Epoch 121 | loss: 0.0312452 | val_loss: 0.0312468 | Time: 156731 ms
[2022-04-07 11:20:55 main:574] : INFO : Epoch 122 | loss: 0.031245 | val_loss: 0.0312464 | Time: 87965.9 ms
[2022-04-07 11:24:08 main:574] : INFO : Epoch 123 | loss: 0.0312461 | val_loss: 0.0312481 | Time: 187339 ms
[2022-04-07 11:27:05 main:574] : INFO : Epoch 124 | loss: 0.0312455 | val_loss: 0.0312476 | Time: 171687 ms
[2022-04-07 11:27:12 main:574] : INFO : Epoch 125 | loss: 0.031246 | val_loss: 0.0312471 | Time: 6945.71 ms
[2022-04-07 11:29:50 main:574] : INFO : Epoch 126 | loss: 0.0312461 | val_loss: 0.0312469 | Time: 158060 ms
[2022-04-07 11:33:37 main:574] : INFO : Epoch 127 | loss: 0.031245 | val_loss: 0.0312465 | Time: 222030 ms
[2022-04-07 11:35:48 main:574] : INFO : Epoch 128 | loss: 0.031245 | val_loss: 0.0312462 | Time: 130018 ms
[2022-04-07 11:38:49 main:574] : INFO : Epoch 129 | loss: 0.031245 | val_loss: 0.0312464 | Time: 181103 ms
[2022-04-07 11:41:04 main:574] : INFO : Epoch 130 | loss: 0.0312472 | val_loss: 0.0312481 | Time: 130345 ms
[2022-04-07 11:44:09 main:574] : INFO : Epoch 131 | loss: 0.031246 | val_loss: 0.0312487 | Time: 184938 ms
[2022-04-07 11:47:38 main:574] : INFO : Epoch 132 | loss: 0.0312456 | val_loss: 0.0312476 | Time: 204227 ms
[2022-04-07 11:50:17 main:574] : INFO : Epoch 133 | loss: 0.0312461 | val_loss: 0.0312472 | Time: 158856 ms
[2022-04-07 11:50:54 main:574] : INFO : Epoch 134 | loss: 0.031246 | val_loss: 0.0312483 | Time: 28493.3 ms
[2022-04-07 11:53:36 main:574] : INFO : Epoch 135 | loss: 0.031246 | val_loss: 0.0312472 | Time: 157933 ms
[2022-04-07 11:55:01 main:574] : INFO : Epoch 136 | loss: 0.0312452 | val_loss: 0.0312471 | Time: 84882.4 ms
[2022-04-07 11:57:45 main:574] : INFO : Epoch 137 | loss: 0.0312455 | val_loss: 0.0312468 | Time: 163524 ms
[2022-04-07 11:57:50 main:574] : INFO : Epoch 138 | loss: 0.0312464 | val_loss: 0.0312499 | Time: 5022.29 ms
[2022-04-07 11:59:53 main:574] : INFO : Epoch 139 | loss: 0.0312462 | val_loss: 0.0312465 | Time: 121923 ms
[2022-04-07 12:03:30 main:574] : INFO : Epoch 140 | loss: 0.0312462 | val_loss: 0.0312471 | Time: 199961 ms
[2022-04-07 12:03:35 main:574] : INFO : Epoch 141 | loss: 0.0312453 | val_loss: 0.0312465 | Time: 5083.58 ms
[2022-04-07 12:04:07 main:574] : INFO : Epoch 142 | loss: 0.0312455 | val_loss: 0.0312484 | Time: 32251.3 ms
[2022-04-07 12:06:57 main:574] : INFO : Epoch 143 | loss: 0.0312475 | val_loss: 0.0312475 | Time: 169931 ms
[2022-04-07 12:10:20 main:574] : INFO : Epoch 144 | loss: 0.0312461 | val_loss: 0.0312465 | Time: 202359 ms
[2022-04-07 12:12:46 main:574] : INFO : Epoch 145 | loss: 0.0312463 | val_loss: 0.0312479 | Time: 141046 ms
[2022-04-07 12:14:37 main:574] : INFO : Epoch 146 | loss: 0.0312455 | val_loss: 0.0312473 | Time: 106426 ms
[2022-04-07 12:14:44 main:574] : INFO : Epoch 147 | loss: 0.031246 | val_loss: 0.031247 | Time: 7068.11 ms
[2022-04-07 12:19:08 main:574] : INFO : Epoch 148 | loss: 0.0312463 | val_loss: 0.0312473 | Time: 262992 ms
[2022-04-07 12:21:30 main:574] : INFO : Epoch 149 | loss: 0.0312459 | val_loss: 0.0312477 | Time: 131952 ms
[2022-04-07 12:24:03 main:574] : INFO : Epoch 150 | loss: 0.0312455 | val_loss: 0.0312469 | Time: 148517 ms
[2022-04-07 12:26:51 main:574] : INFO : Epoch 151 | loss: 0.0312457 | val_loss: 0.0312467 | Time: 167757 ms
[2022-04-07 12:26:58 main:574] : INFO : Epoch 152 | loss: 0.0312457 | val_loss: 0.031248 | Time: 6066.75 ms
[2022-04-07 12:30:13 main:574] : INFO : Epoch 153 | loss: 0.0312461 | val_loss: 0.0312479 | Time: 193315 ms
[2022-04-07 12:30:59 main:574] : INFO : Epoch 154 | loss: 0.0312464 | val_loss: 0.0312518 | Time: 40996.7 ms
[2022-04-07 12:31:50 main:574] : INFO : Epoch 155 | loss: 0.0312477 | val_loss: 0.0312495 | Time: 49521.3 ms
[2022-04-07 12:34:04 main:574] : INFO : Epoch 156 | loss: 0.0312461 | val_loss: 0.0312473 | Time: 128255 ms
[2022-04-07 12:35:55 main:574] : INFO : Epoch 157 | loss: 0.0312451 | val_loss: 0.0312473 | Time: 111323 ms
[2022-04-07 12:37:42 main:574] : INFO : Epoch 158 | loss: 0.0312455 | val_loss: 0.0312495 | Time: 107170 ms
[2022-04-07 12:40:50 main:574] : INFO : Epoch 159 | loss: 0.0312455 | val_loss: 0.0312471 | Time: 182887 ms
[2022-04-07 12:41:11 main:574] : INFO : Epoch 160 | loss: 0.0312452 | val_loss: 0.0312471 | Time: 16340.7 ms
[2022-04-07 12:45:14 main:574] : INFO : Epoch 161 | loss: 0.0312455 | val_loss: 0.0312473 | Time: 242937 ms
[2022-04-07 12:48:24 main:574] : INFO : Epoch 162 | loss: 0.0312468 | val_loss: 0.031247 | Time: 185234 ms
[2022-04-07 12:50:56 main:574] : INFO : Epoch 163 | loss: 0.031246 | val_loss: 0.0312466 | Time: 151960 ms
[2022-04-07 12:52:11 main:574] : INFO : Epoch 164 | loss: 0.0312466 | val_loss: 0.0312472 | Time: 74391.4 ms
[2022-04-07 12:55:52 main:574] : INFO : Epoch 165 | loss: 0.0312459 | val_loss: 0.0312478 | Time: 216000 ms
[2022-04-07 12:58:45 main:574] : INFO : Epoch 166 | loss: 0.0312464 | val_loss: 0.0312478 | Time: 166427 ms
[2022-04-07 13:00:39 main:574] : INFO : Epoch 167 | loss: 0.0312458 | val_loss: 0.0312472 | Time: 108248 ms
[2022-04-07 13:00:45 main:574] : INFO : Epoch 168 | loss: 0.031246 | val_loss: 0.031249 | Time: 6768.03 ms
[2022-04-07 13:04:01 main:574] : INFO : Epoch 169 | loss: 0.0312465 | val_loss: 0.0312498 | Time: 193892 ms
[2022-04-07 13:05:52 main:574] : INFO : Epoch 170 | loss: 0.0312479 | val_loss: 0.0312491 | Time: 111463 ms
[2022-04-07 13:08:14 main:574] : INFO : Epoch 171 | loss: 0.0312464 | val_loss: 0.0312474 | Time: 141226 ms
[2022-04-07 13:10:09 main:574] : INFO : Epoch 172 | loss: 0.031253 | val_loss: 0.0312519 | Time: 115169 ms
[2022-04-07 13:12:29 main:574] : INFO : Epoch 173 | loss: 0.0312485 | val_loss: 0.0312483 | Time: 140114 ms
[2022-04-07 13:16:10 main:574] : INFO : Epoch 174 | loss: 0.0312466 | val_loss: 0.0312477 | Time: 220852 ms
[2022-04-07 13:19:52 main:574] : INFO : Epoch 175 | loss: 0.0312455 | val_loss: 0.0312471 | Time: 222045 ms
[2022-04-07 13:22:17 main:574] : INFO : Epoch 176 | loss: 0.0312457 | val_loss: 0.0312478 | Time: 144793 ms
[2022-04-07 13:22:25 main:574] : INFO : Epoch 177 | loss: 0.0312457 | val_loss: 0.0312472 | Time: 8575.36 ms
[2022-04-07 13:24:09 main:574] : INFO : Epoch 178 | loss: 0.0312456 | val_loss: 0.0312487 | Time: 103352 ms
[2022-04-07 13:24:45 main:574] : INFO : Epoch 179 | loss: 0.0312458 | val_loss: 0.0312473 | Time: 35640.7 ms
[2022-04-07 13:25:26 main:574] : INFO : Epoch 180 | loss: 0.0312458 | val_loss: 0.0312484 | Time: 41585.5 ms
[2022-04-07 13:28:36 main:574] : INFO : Epoch 181 | loss: 0.0312458 | val_loss: 0.0312472 | Time: 185232 ms
[2022-04-07 13:31:28 main:574] : INFO : Epoch 182 | loss: 0.0312459 | val_loss: 0.0312506 | Time: 172182 ms
[2022-04-07 13:33:20 main:574] : INFO : Epoch 183 | loss: 0.0312464 | val_loss: 0.031247 | Time: 111004 ms
[2022-04-07 13:33:28 main:574] : INFO : Epoch 184 | loss: 0.0312453 | val_loss: 0.0312472 | Time: 8500.13 ms
[2022-04-07 13:34:40 main:574] : INFO : Epoch 185 | loss: 0.0312454 | val_loss: 0.031247 | Time: 67206 ms
[2022-04-07 13:38:15 main:574] : INFO : Epoch 186 | loss: 0.0312452 | val_loss: 0.0312472 | Time: 209587 ms
[2022-04-07 13:41:05 main:574] : INFO : Epoch 187 | loss: 0.0312454 | val_loss: 0.0312471 | Time: 169849 ms
[2022-04-07 13:43:10 main:574] : INFO : Epoch 188 | loss: 0.0312458 | val_loss: 0.0312468 | Time: 125240 ms
[2022-04-07 13:45:26 main:574] : INFO : Epoch 189 | loss: 0.0312471 | val_loss: 0.0312473 | Time: 130933 ms
[2022-04-07 13:45:33 main:574] : INFO : Epoch 190 | loss: 0.0312469 | val_loss: 0.031247 | Time: 6749.5 ms
[2022-04-07 13:50:01 main:574] : INFO : Epoch 191 | loss: 0.0312455 | val_loss: 0.031247 | Time: 267183 ms
[2022-04-07 13:52:16 main:574] : INFO : Epoch 192 | loss: 0.0312461 | val_loss: 0.0312472 | Time: 134385 ms
[2022-04-07 13:53:42 main:574] : INFO : Epoch 193 | loss: 0.0312463 | val_loss: 0.0312479 | Time: 85958.7 ms
[2022-04-07 13:54:17 main:574] : INFO : Epoch 194 | loss: 0.0312455 | val_loss: 0.0312476 | Time: 29758.7 ms
[2022-04-07 13:56:13 main:574] : INFO : Epoch 195 | loss: 0.0312459 | val_loss: 0.0312471 | Time: 114472 ms
[2022-04-07 13:59:53 main:574] : INFO : Epoch 196 | loss: 0.0312455 | val_loss: 0.031247 | Time: 212565 ms
[2022-04-07 14:03:35 main:574] : INFO : Epoch 197 | loss: 0.0312455 | val_loss: 0.0312491 | Time: 217303 ms
[2022-04-07 14:05:58 main:574] : INFO : Epoch 198 | loss: 0.0312457 | val_loss: 0.0312469 | Time: 137954 ms
[2022-04-07 14:08:09 main:574] : INFO : Epoch 199 | loss: 0.0312457 | val_loss: 0.03125 | Time: 130762 ms
[2022-04-07 14:11:14 main:574] : INFO : Epoch 200 | loss: 0.0312461 | val_loss: 0.0312482 | Time: 180000 ms
[2022-04-07 14:15:10 main:574] : INFO : Epoch 201 | loss: 0.0312458 | val_loss: 0.0312473 | Time: 231546 ms
[2022-04-07 14:18:57 main:574] : INFO : Epoch 202 | loss: 0.0312472 | val_loss: 0.0312471 | Time: 226817 ms
[2022-04-07 14:20:21 main:574] : INFO : Epoch 203 | loss: 0.0312466 | val_loss: 0.0312476 | Time: 79241.5 ms
[2022-04-07 14:20:52 main:574] : INFO : Epoch 204 | loss: 0.0312454 | val_loss: 0.0312472 | Time: 30292.9 ms
[2022-04-07 14:22:14 main:574] : INFO : Epoch 205 | loss: 0.0312455 | val_loss: 0.031247 | Time: 82073.4 ms
[2022-04-07 14:24:49 main:574] : INFO : Epoch 206 | loss: 0.0312454 | val_loss: 0.0312476 | Time: 155412 ms
[2022-04-07 14:28:32 main:574] : INFO : Epoch 207 | loss: 0.0312457 | val_loss: 0.0312474 | Time: 217308 ms
[2022-04-07 14:31:37 main:574] : INFO : Epoch 208 | loss: 0.0312471 | val_loss: 0.031252 | Time: 184481 ms
[2022-04-07 14:34:05 main:574] : INFO : Epoch 209 | loss: 0.0312483 | val_loss: 0.031253 | Time: 142556 ms
[2022-04-07 14:36:55 main:574] : INFO : Epoch 210 | loss: 0.0312485 | val_loss: 0.0312508 | Time: 164488 ms
[2022-04-07 14:37:41 main:574] : INFO : Epoch 211 | loss: 0.0312475 | val_loss: 0.0312501 | Time: 41315.7 ms
[2022-04-07 14:37:47 main:574] : INFO : Epoch 212 | loss: 0.0312459 | val_loss: 0.031248 | Time: 5391.04 ms
[2022-04-07 14:40:50 main:574] : INFO : Epoch 213 | loss: 0.0312468 | val_loss: 0.0312474 | Time: 181613 ms
[2022-04-07 14:41:44 main:574] : INFO : Epoch 214 | loss: 0.0312461 | val_loss: 0.0312485 | Time: 48678.2 ms
[2022-04-07 14:41:50 main:574] : INFO : Epoch 215 | loss: 0.0312451 | val_loss: 0.0312474 | Time: 6039 ms
[2022-04-07 14:45:37 main:574] : INFO : Epoch 216 | loss: 0.0312455 | val_loss: 0.0312473 | Time: 225534 ms
[2022-04-07 14:48:36 main:574] : INFO : Epoch 217 | loss: 0.0312457 | val_loss: 0.0312531 | Time: 179489 ms
[2022-04-07 14:51:53 main:574] : INFO : Epoch 218 | loss: 0.0312481 | val_loss: 0.031248 | Time: 190606 ms
[2022-04-07 14:53:14 main:574] : INFO : Epoch 219 | loss: 0.0312455 | val_loss: 0.0312483 | Time: 81264 ms
[2022-04-07 14:53:21 main:574] : INFO : Epoch 220 | loss: 0.031246 | val_loss: 0.0312468 | Time: 6654.21 ms
[2022-04-07 14:55:17 main:574] : INFO : Epoch 221 | loss: 0.0312453 | val_loss: 0.0312475 | Time: 115352 ms
[2022-04-07 14:58:02 main:574] : INFO : Epoch 222 | loss: 0.0312455 | val_loss: 0.0312502 | Time: 164967 ms
[2022-04-07 15:00:55 main:574] : INFO : Epoch 223 | loss: 0.0312456 | val_loss: 0.0312468 | Time: 168643 ms
[2022-04-07 15:04:02 main:574] : INFO : Epoch 224 | loss: 0.0312448 | val_loss: 0.0312486 | Time: 181402 ms
[2022-04-07 15:05:54 main:574] : INFO : Epoch 225 | loss: 0.0312453 | val_loss: 0.0312484 | Time: 112346 ms
[2022-04-07 15:06:09 main:574] : INFO : Epoch 226 | loss: 0.0312454 | val_loss: 0.0312489 | Time: 15125.1 ms
[2022-04-07 15:09:38 main:574] : INFO : Epoch 227 | loss: 0.031245 | val_loss: 0.0312471 | Time: 208602 ms
[2022-04-07 15:11:09 main:574] : INFO : Epoch 228 | loss: 0.0312448 | val_loss: 0.0312474 | Time: 90690 ms
[2022-04-07 15:12:10 main:574] : INFO : Epoch 229 | loss: 0.0312448 | val_loss: 0.031247 | Time: 61179.1 ms
[2022-04-07 15:12:24 main:574] : INFO : Epoch 230 | loss: 0.0312446 | val_loss: 0.0312475 | Time: 14212.9 ms
[2022-04-07 15:13:54 main:574] : INFO : Epoch 231 | loss: 0.0312446 | val_loss: 0.0312484 | Time: 90238.8 ms
[2022-04-07 15:17:42 main:574] : INFO : Epoch 232 | loss: 0.0312453 | val_loss: 0.0312479 | Time: 227988 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 04/07/22 15:20:38
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\15;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 000000002c280000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000294d0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000028900000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000027260000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000283d0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026c90000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1586 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1586
ModLoad: 0000000027740000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000026f90000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026bf0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000270a0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000028850000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000293e0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000027770000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027870000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000019820000 0000000000058000 C:\ProgramData\BOINC\slots\15\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000007c40000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000001a230000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000000cd10000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000000d570000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000007af60000 0000000024b49000 C:\ProgramData\BOINC\slots\15\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000004700000 0000000000045000 C:\ProgramData\BOINC\slots\15\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000aad10000 000000000abe7000 C:\ProgramData\BOINC\slots\15\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bd1e0000 00000000034b2000 C:\ProgramData\BOINC\slots\15\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000219e0000 0000000000010000 C:\ProgramData\BOINC\slots\15\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000ba130000 00000000030a5000 C:\ProgramData\BOINC\slots\15\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 000000005fbb0000 000000001b3a4000 C:\ProgramData\BOINC\slots\15\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000e4610000 000000000020d000 C:\ProgramData\BOINC\slots\15\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000ca300000 00000000002db000 C:\ProgramData\BOINC\slots\15\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000006160000 0000000000040000 C:\ProgramData\BOINC\slots\15\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000005be50000 0000000003d5b000 C:\ProgramData\BOINC\slots\15\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000b71f0000 0000000001eca000 C:\ProgramData\BOINC\slots\15\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000052cb0000 000000000919f000 C:\ProgramData\BOINC\slots\15\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000028710000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000024a40000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000289c0000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000271a0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000026cc0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000269b0000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000271f0000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026d80000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000267d0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000027a50000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 0000000024d20000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000028e30000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000026650000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027690000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000275a0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000c9340000 00000000002cd000 C:\WINDOWS\system32\nvcuda.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000a7620000 00000000016a0000 C:\WINDOWS\system32\DriverStore\FileRepository\nvhdc.inf_amd64_d11413f6fd8fe259\nvcuda64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000216e0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001dc70000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 0000000018c60000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000265c0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000052540000 0000000000768000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1165
ModLoad: 0000000019dc0000 0000000000009000 C:\ProgramData\BOINC\slots\15\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000a9d40000 0000000000fca000 C:\ProgramData\BOINC\slots\15\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000026cf0000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3852, Write: 3069, Other 8301
- I/O Transfers Counters -
Read: 7575534, Write: 6869122, Other 57632
- Paged Pool Usage -
QuotaPagedPoolUsage: 3533320, QuotaPeakPagedPoolUsage: 3533544
QuotaNonPagedPoolUsage: 58200, QuotaPeakNonPagedPoolUsage: 58608
- Virtual Memory Usage -
VirtualSize: 399765504, PeakVirtualSize: 1015795712
- Pagefile Usage -
PagefileUsage: 399765504, PeakPagefileUsage: 399765504
- Working Set Size -
WorkingSetSize: -2121854976, PeakWorkingSetSize: -2057728000, PageFaultCount: 840122
*** Dump of thread ID 1144 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 162351280.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=000000002c538c40 rdx=000000005e7fe900 rsi=0000000000000000 rdi=0000000000000000
r8=000000005e7fe900 r9=000000002c538c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000273290d2 rsp=000000005e7fe8d8 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5e7fe8d0 2c2b9100 00000001 5e7fe900 5e7fe900 2c538c30 KERNELBASE!DebugBreak+0x0
5e7fed10 2c2b9ce4 cd800000 5e7ff074 00000000 0000036c mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
5e7ffc70 2c2ba084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
5e7ffca0 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
5e7ffcd0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5e7ffd50 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8208 (state: Ready): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 218124992.000000, User Time: 4951875072.000000, Wait Time: 162351008.000000
- Registers -
rax=00000000fffffec0 rbx=00000000ab6aa5dc rcx=0000000000000003 rdx=00000000ab6aa510 rsi=00000000698fcdb0 rdi=0000000000000160
r8=0000000000000100 r9=00000000a7aff993 r10=00000000a7620000 r11=00000000616c9588 r12=00000000ab6aa510 r13=00000000698fcd00
r14=0000000000000160 r15=0000000000000002 rip=00000000a76c5afe rsp=000000005e2dfaa0 rbp=0000000000000014
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
5e2dfac0 a79d90ae 00000160 698fcd24 00000000 a79d596e nvcuda64!+0x0
5e2dfb10 a79d71a8 5e2e0770 969ba630 ba902410 00000002 nvcuda64!cuProfilerStop+0x0
5e2dfd80 a77b1206 53bd3090 5e2dfe90 969ba630 616c9510 nvcuda64!cuProfilerStop+0x0
5e2e03c0 a7962e95 00000000 00000000 00000000 00000000 nvcuda64!+0x0
5e2e0500 a765c464 969ba630 5e2e0770 969ba630 5e2e0990 nvcuda64!cuProfilerStop+0x0
5e2e05d0 a765ca57 5e2e06e0 53bd3090 5e2e06e0 969ba630 nvcuda64!+0x0
5e2e0940 a785c8bc 00000001 00004000 60bd0b00 60954de7 nvcuda64!+0x0
5e2e0ab0 609529b6 00000000 00000001 00000100 60bd0b00 nvcuda64!cuEventRecordWithFlags+0x0
5e2e0bc0 6094fdf4 60bd0b00 5252ffd0 969ba630 5e2e0de0 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e0d40 60bbbb21 60bd0b00 00000000 60bd0b00 00000000 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e0eb0 60abde74 3c290510 3bff8000 3c200000 00000090 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e10e0 60acf99d 00000000 61dc1400 00000000 60bd0b00 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e1140 60a9703b 00000000 00000000 3c290510 5e2e1250 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e15d0 609b43bc 00000000 5e2e16bc 00000000 ba905020 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e18a0 609b700e 00000100 d074ef60 00000006 00000001 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e1990 608b82ca ba905020 00000000 00000000 00000024 cudnn64_7!cudnnSetDropoutDescriptor+0x0
5e2e1a60 608b62b3 3c212000 00000024 5e2e1ca0 00000350 cudnn64_7!cudnnSetAttnDescriptor+0x0
5e2e1bb0 608b9536 3c212000 308ac1c0 00000000 00000000 cudnn64_7!cudnnSetAttnDescriptor+0x0
5e2e1cc0 608cb06f d05ed280 00000000 00000000 00000000 cudnn64_7!cudnnSetAttnDescriptor+0x0
5e2e2170 608beff1 00000680 308ac1c0 5e2e4438 cdc516d0 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
5e2e2310 608c7d8e d0600300 00000000 cdc516d0 5e2e25b8 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
5e2e24b0 608dd24a 5e2e25b8 5e2e25b8 5e2e25b8 5e2e25b8 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
5e2e4620 7dcd8bd1 d05fe100 3c200000 d05fe100 308ac1c0 cudnn64_7!cudnnRNNForwardInference+0x0
5e2e4b00 7dd325ea 5e2e4bc8 00000001 5e2e4bc8 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
5e2e4be0 7dd40cbf 5e2e52d0 5e2e6ac8 5e2e5c20 00000001 torch_cuda!at::native::set_storage_cuda_+0x0
5e2e4cc0 b075b9fc 5e2e52d0 5e2e59b0 00000000 5e2e4dd8 torch_cuda!at::native::set_storage_cuda_+0x0
5e2e4e40 b07931a2 5e2e51d0 00000001 00000000 00000000 torch_cpu!at::native::mkldnn_sigmoid_+0x0
5e2e5120 b1a42613 00000000 5e2e5c30 5e2e5c30 5e2e5230 torch_cpu!at::_cudnn_rnn+0x0
5e2e5620 b1a5080b 280a0ceb d0ca1120 5e2e5750 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
5e2e5700 b075b9fc 5e2e5c80 b1b75878 5e2e5860 a7647a77 torch_cpu!torch::autograd::GraphRoot::apply+0x0
5e2e5880 b07931a2 cdc52ee0 00000000 280a0f5b 7ebfa3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
5e2e5b60 7dcd69a1 5e2e0001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
5e2e5d30 7dce0117 00000000 5e2e6ac8 04702e0d 5e2e5e50 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
5e2e5dc0 b04e107b b5631ac0 5e2e60e0 c94319e7 a7817783 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
5e2e6030 b0513b16 00000001 5e2e6ac8 00000080 d0ace6f8 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
5e2e61c0 b088ae89 5e2e6238 00000001 5e2e6238 00000001 torch_cpu!at::native::gru+0x0
5e2e6250 b1b79fa3 5e2e6350 00000001 5e2e64d0 5e2e6ac8 torch_cpu!at::zeros_out+0x0
5e2e62e0 b087dd35 19871ea8 532bd6b0 00000000 19871ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
5e2e6360 b07588ec b56375c8 3d77ae20 d074efd0 5e2e6580 torch_cpu!at::zeros_out+0x0
5e2e6460 b07cc1bb 0000000c 00000080 00000004 d0ca0b00 torch_cpu!at::native::mkldnn_sigmoid_+0x0
5e2e64e0 b22b7294 5e2e6600 3d77ae20 5e2e65f0 5e2e6798 torch_cpu!at::gru+0x0
5e2e6710 b22b62ab 5e2e6aa8 b06e030e 5e2e6aa8 5e2e6ac8 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
5e2e6800 2c292621 00000002 d0c9cb70 00000000 19871ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
5e2e6a00 2c297e98 5e2e6a58 5e2e0000 d0c90601 d0c9cb70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
5e2e6ea0 2c29b964 2c2cd95f 2c2cd95f 00000000 3e0c2350 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::unsafeRemoveSlot+0x0
5e2ffc40 2c4dc6f8 5373c520 00000000 00000000 537841a0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
5e2ffc80 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
5e2ffcb0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5e2ffd30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9848 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 162351280.000000
- Registers -
rax=000000000000005b rbx=000000000000000e rcx=000000000000000e rdx=000000005e8ff9f0 rsi=0000000000000000 rdi=000000000000000e
r8=0000000000000001 r9=0000000053950a00 r10=0000000000000000 r11=000000005e8ff960 r12=0000000000000064 r13=000000005e8ff9f0
r14=000000005e8ff6f0 r15=0000000000000000 rip=000000002956d8c4 rsp=000000005e8ff698 rbp=0000000053950a50
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5e8ff690 272acb20 00000000 5e034000 2963a4d0 5e034000 ntdll!ZwWaitForMultipleObjects+0x0
5e8ff980 272aca1e 51d73440 004e0049 0000000e 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
5e8ff9c0 a77fd554 53953930 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
5e8ffc00 a78a58cd 00000000 53950a50 00000000 00000000 nvcuda64!+0x0
5e8ffcb0 a77fcf23 515ace0d b8672cd3 53efe950 00000000 nvcuda64!cuProfilerStop+0x0
5e8ffce0 a7b344c4 53efe950 00000000 00000000 00000000 nvcuda64!+0x0
5e8ffd10 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
5e8ffd40 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5e8ffdc0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 15704 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 160411024.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002b8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002b8
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000053938700 r13=0000000000000000
r14=00000000000002b8 r15=0000000000000000 rip=000000002956cdf4 rsp=000000005e9ff6e8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5e9ff6e0 27281a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
5e9ff780 a7a25d0e 53efe350 00000000 00000000 000002b8 KERNELBASE!WaitForSingleObjectEx+0x0
5e9ff9c0 a77fcf23 00000000 53efe350 a7620000 5396efb0 nvcuda64!cuProfilerStop+0x0
5e9ff9f0 a7b344c4 53efe350 00000000 00000000 00000000 nvcuda64!+0x0
5e9ffa20 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
5e9ffa50 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5e9ffad0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 18112 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 130937504.000000, User Time: 3216250112.000000, Wait Time: 162349952.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000b3b11358 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000b3b11358
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=000000005eaff0c0 r12=0000000000000000 r13=0000000000000000
r14=000000005eaff8c0 r15=00000000b3b11308 rip=0000000029570764 rsp=000000005eaff898 rbp=000000005eaff8f0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5eaff890 29534021 baca28c8 07c52f97 cbb59af0 cbb59ae0 ntdll!NtWaitForAlertByThreadId+0x0
5eaff910 272ccfa9 b3b11350 b3b11348 b3b11300 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
5eaff950 07c7bf3d cbb59b10 00000000 00000000 b1ef05b4 KERNELBASE!SleepConditionVariableSRW+0x0
5eaff990 07c52798 00000000 cbb59b10 00000000 00000000 MSVCP140!__crtSetThreadpoolWait+0x0
5eaff9c0 07c52a4a 75a29078 00000000 5eaffbd0 b3b11348 MSVCP140!std::_Winerror_message+0x0
5eaff9f0 b1ef064b b3b11300 5eaffaf0 b3b11300 cbb59af0 MSVCP140!_Cnd_wait+0x0
5eaffa30 b1ef25b1 d10fcff0 ffffffff cbb59af0 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
5eaffd20 b1ef2471 b5416d10 5eaffd60 b5416d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
5eaffd70 b1ee9744 00080001 d0744370 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
5eaffda0 270c1bb2 d0744370 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
5eaffdd0 28917034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
5eaffe00 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5eaffe80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)