Task 12199146
| Name | ParityModified-1645996729-18341-0_0 |
| Workunit | 9380222 |
| Created | 27 Feb 2022, 21:18:53 UTC |
| Sent | 1 Mar 2022, 2:09:14 UTC |
| Report deadline | 9 Mar 2022, 2:09:14 UTC |
| Received | 1 Mar 2022, 6:06:42 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Aborted by user |
| Exit status | 203 (0x000000CB) EXIT_ABORTED_VIA_GUI |
| Computer ID | 10156 |
| Run time | 3 hours 56 min 12 sec |
| CPU time | 1 hours 17 min 23 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 15,672.14 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.87 GB |
| Peak swap size | 4.72 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080)
[2022-02-28 18:09:21 main:435] : INFO : Set logging level to 1
[2022-02-28 18:09:21 main:441] : INFO : Running in BOINC Client mode
[2022-02-28 18:09:21 main:444] : INFO : Resolving all filenames
[2022-02-28 18:09:21 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-28 18:09:21 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 0)
[2022-02-28 18:09:21 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-28 18:09:21 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 0)
[2022-02-28 18:09:21 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 0)
[2022-02-28 18:09:22 main:472] : INFO : Dataset filename: dataset.hdf5
[2022-02-28 18:09:22 main:474] : INFO : Configuration:
[2022-02-28 18:09:22 main:475] : INFO : Model type: GRU
[2022-02-28 18:09:22 main:476] : INFO : Validation Loss Threshold: 0.0001
[2022-02-28 18:09:22 main:477] : INFO : Max Epochs: 2048
[2022-02-28 18:09:22 main:478] : INFO : Batch Size: 128
[2022-02-28 18:09:22 main:479] : INFO : Learning Rate: 0.01
[2022-02-28 18:09:22 main:480] : INFO : Patience: 10
[2022-02-28 18:09:22 main:481] : INFO : Hidden Width: 12
[2022-02-28 18:09:22 main:482] : INFO : # Recurrent Layers: 4
[2022-02-28 18:09:22 main:483] : INFO : # Backend Layers: 4
[2022-02-28 18:09:22 main:484] : INFO : # Threads: 1
[2022-02-28 18:09:22 main:486] : INFO : Preparing Dataset
[2022-02-28 18:09:22 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-28 18:09:23 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-02-28 20:06:32 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-02-28 20:06:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-02-28 20:06:32 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-02-28 20:06:33 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-02-28 20:06:33 main:494] : INFO : Creating Model
[2022-02-28 20:06:33 main:507] : INFO : Preparing config file
[2022-02-28 20:06:33 main:519] : INFO : Creating new config file
[2022-02-28 21:44:18 main:559] : INFO : Loading DataLoader into Memory
[2022-02-28 21:44:18 main:562] : INFO : Starting Training
[2022-02-28 21:53:37 main:574] : INFO : Epoch 1 | loss: 0.0532955 | val_loss: 0.0328433 | Time: 558954 ms
[2022-02-28 21:53:40 main:574] : INFO : Epoch 2 | loss: 0.0318892 | val_loss: 0.0313619 | Time: 3247.71 ms
[2022-02-28 21:53:43 main:574] : INFO : Epoch 3 | loss: 0.0313722 | val_loss: 0.0313191 | Time: 3532.96 ms
[2022-02-28 21:53:47 main:574] : INFO : Epoch 4 | loss: 0.0312859 | val_loss: 0.0312661 | Time: 3215.51 ms
[2022-02-28 21:53:50 main:574] : INFO : Epoch 5 | loss: 0.0312594 | val_loss: 0.0312596 | Time: 3297.9 ms
[2022-02-28 21:53:53 main:574] : INFO : Epoch 6 | loss: 0.0312547 | val_loss: 0.0312538 | Time: 3373.79 ms
[2022-02-28 21:53:57 main:574] : INFO : Epoch 7 | loss: 0.0312535 | val_loss: 0.0312527 | Time: 3364.04 ms
[2022-02-28 21:54:00 main:574] : INFO : Epoch 8 | loss: 0.0312513 | val_loss: 0.0312516 | Time: 3234.31 ms
[2022-02-28 21:54:04 main:574] : INFO : Epoch 9 | loss: 0.0312525 | val_loss: 0.0312567 | Time: 3618.76 ms
[2022-02-28 21:54:07 main:574] : INFO : Epoch 10 | loss: 0.0312533 | val_loss: 0.0312541 | Time: 3255.24 ms
[2022-02-28 21:54:10 main:574] : INFO : Epoch 11 | loss: 0.0312517 | val_loss: 0.0312511 | Time: 3224.37 ms
[2022-02-28 21:54:14 main:574] : INFO : Epoch 12 | loss: 0.0312529 | val_loss: 0.0312541 | Time: 3768.46 ms
[2022-02-28 21:54:17 main:574] : INFO : Epoch 13 | loss: 0.031253 | val_loss: 0.0312508 | Time: 3341.54 ms
[2022-02-28 21:54:21 main:574] : INFO : Epoch 14 | loss: 0.0312507 | val_loss: 0.0312509 | Time: 3387.06 ms
[2022-02-28 21:54:24 main:574] : INFO : Epoch 15 | loss: 0.0312515 | val_loss: 0.031251 | Time: 3566.74 ms
[2022-02-28 21:54:28 main:574] : INFO : Epoch 16 | loss: 0.0312505 | val_loss: 0.0312591 | Time: 3504.84 ms
[2022-02-28 21:54:31 main:574] : INFO : Epoch 17 | loss: 0.0312532 | val_loss: 0.0312522 | Time: 3615.25 ms
[2022-02-28 21:54:35 main:574] : INFO : Epoch 18 | loss: 0.031253 | val_loss: 0.0312532 | Time: 3354.68 ms
[2022-02-28 21:54:38 main:574] : INFO : Epoch 19 | loss: 0.0312541 | val_loss: 0.0312511 | Time: 3493.96 ms
[2022-02-28 21:54:41 main:574] : INFO : Epoch 20 | loss: 0.0312525 | val_loss: 0.0312516 | Time: 3397.92 ms
[2022-02-28 21:54:45 main:574] : INFO : Epoch 21 | loss: 0.0312533 | val_loss: 0.031253 | Time: 3236.95 ms
[2022-02-28 21:54:48 main:574] : INFO : Epoch 22 | loss: 0.0312526 | val_loss: 0.0312548 | Time: 3444.92 ms
[2022-02-28 21:54:52 main:574] : INFO : Epoch 23 | loss: 0.0312526 | val_loss: 0.0312527 | Time: 3475.26 ms
[2022-02-28 21:54:55 main:574] : INFO : Epoch 24 | loss: 0.0312555 | val_loss: 0.031256 | Time: 3579.68 ms
[2022-02-28 21:54:58 main:574] : INFO : Epoch 25 | loss: 0.0312537 | val_loss: 0.0312599 | Time: 3228.45 ms
[2022-02-28 21:55:02 main:574] : INFO : Epoch 26 | loss: 0.0312531 | val_loss: 0.0312539 | Time: 3508.47 ms
[2022-02-28 21:55:05 main:574] : INFO : Epoch 27 | loss: 0.0312503 | val_loss: 0.0312499 | Time: 2994.47 ms
[2022-02-28 21:55:08 main:574] : INFO : Epoch 28 | loss: 0.0312501 | val_loss: 0.0312498 | Time: 3267.98 ms
[2022-02-28 21:55:11 main:574] : INFO : Epoch 29 | loss: 0.0312505 | val_loss: 0.0312578 | Time: 3087.23 ms
[2022-02-28 21:55:15 main:574] : INFO : Epoch 30 | loss: 0.0312555 | val_loss: 0.0312512 | Time: 3140.72 ms
[2022-02-28 21:55:18 main:574] : INFO : Epoch 31 | loss: 0.0312506 | val_loss: 0.0312502 | Time: 3576.15 ms
[2022-02-28 21:55:22 main:574] : INFO : Epoch 32 | loss: 0.0312503 | val_loss: 0.0312498 | Time: 3456.84 ms
[2022-02-28 21:55:25 main:574] : INFO : Epoch 33 | loss: 0.0312545 | val_loss: 0.0312582 | Time: 3629.58 ms
[2022-02-28 21:55:29 main:574] : INFO : Epoch 34 | loss: 0.0312562 | val_loss: 0.0312501 | Time: 3432.48 ms
[2022-02-28 21:55:32 main:574] : INFO : Epoch 35 | loss: 0.0312529 | val_loss: 0.031252 | Time: 3290.18 ms
[2022-02-28 21:55:35 main:574] : INFO : Epoch 36 | loss: 0.0312518 | val_loss: 0.0312515 | Time: 3542.91 ms
[2022-02-28 21:55:39 main:574] : INFO : Epoch 37 | loss: 0.031251 | val_loss: 0.0312507 | Time: 3426.57 ms
[2022-02-28 21:55:42 main:574] : INFO : Epoch 38 | loss: 0.0312511 | val_loss: 0.0312494 | Time: 3402.71 ms
[2022-02-28 21:55:46 main:574] : INFO : Epoch 39 | loss: 0.0312495 | val_loss: 0.0312491 | Time: 3241.42 ms
[2022-02-28 21:55:49 main:574] : INFO : Epoch 40 | loss: 0.0312504 | val_loss: 0.0312509 | Time: 3123.39 ms
[2022-02-28 21:55:52 main:574] : INFO : Epoch 41 | loss: 0.0312504 | val_loss: 0.0312497 | Time: 3436.86 ms
[2022-02-28 21:55:56 main:574] : INFO : Epoch 42 | loss: 0.0312497 | val_loss: 0.0312492 | Time: 3673.16 ms
[2022-02-28 21:55:59 main:574] : INFO : Epoch 43 | loss: 0.0312497 | val_loss: 0.0312503 | Time: 3662.63 ms
[2022-02-28 21:56:03 main:574] : INFO : Epoch 44 | loss: 0.0312504 | val_loss: 0.0312499 | Time: 3171.8 ms
[2022-02-28 21:56:06 main:574] : INFO : Epoch 45 | loss: 0.0312515 | val_loss: 0.0312511 | Time: 3047.47 ms
[2022-02-28 21:56:09 main:574] : INFO : Epoch 46 | loss: 0.0312509 | val_loss: 0.031256 | Time: 3462.02 ms
[2022-02-28 21:56:12 main:574] : INFO : Epoch 47 | loss: 0.0312502 | val_loss: 0.0312497 | Time: 3031.46 ms
[2022-02-28 21:56:15 main:574] : INFO : Epoch 48 | loss: 0.0312511 | val_loss: 0.0312502 | Time: 3090.97 ms
[2022-02-28 21:56:19 main:574] : INFO : Epoch 49 | loss: 0.0312499 | val_loss: 0.0312483 | Time: 3295.18 ms
[2022-02-28 21:56:22 main:574] : INFO : Epoch 50 | loss: 0.0312525 | val_loss: 0.0312478 | Time: 3238.09 ms
[2022-02-28 21:56:25 main:574] : INFO : Epoch 51 | loss: 0.0312516 | val_loss: 0.0312497 | Time: 3423.6 ms
[2022-02-28 21:56:29 main:574] : INFO : Epoch 52 | loss: 0.0312493 | val_loss: 0.0312497 | Time: 3323.2 ms
[2022-02-28 21:56:32 main:574] : INFO : Epoch 53 | loss: 0.0312511 | val_loss: 0.0312529 | Time: 3583.34 ms
[2022-02-28 21:56:36 main:574] : INFO : Epoch 54 | loss: 0.031251 | val_loss: 0.0312472 | Time: 3299.79 ms
[2022-02-28 21:56:39 main:574] : INFO : Epoch 55 | loss: 0.0312489 | val_loss: 0.0312466 | Time: 3163.3 ms
[2022-02-28 21:56:42 main:574] : INFO : Epoch 56 | loss: 0.0312469 | val_loss: 0.0312469 | Time: 3420.12 ms
[2022-02-28 21:56:45 main:574] : INFO : Epoch 57 | loss: 0.0312481 | val_loss: 0.0312572 | Time: 3137.36 ms
[2022-02-28 21:56:49 main:574] : INFO : Epoch 58 | loss: 0.0312485 | val_loss: 0.0312446 | Time: 3789.35 ms
[2022-02-28 21:56:52 main:574] : INFO : Epoch 59 | loss: 0.0312457 | val_loss: 0.0312477 | Time: 3379.61 ms
[2022-02-28 21:56:56 main:574] : INFO : Epoch 60 | loss: 0.0312489 | val_loss: 0.0312595 | Time: 3604.11 ms
[2022-02-28 21:56:59 main:574] : INFO : Epoch 61 | loss: 0.0312505 | val_loss: 0.0312606 | Time: 3187.83 ms
[2022-02-28 21:57:03 main:574] : INFO : Epoch 62 | loss: 0.031246 | val_loss: 0.0312443 | Time: 3526.92 ms
[2022-02-28 21:57:06 main:574] : INFO : Epoch 63 | loss: 0.0312457 | val_loss: 0.0312471 | Time: 3555.13 ms
[2022-02-28 21:57:10 main:574] : INFO : Epoch 64 | loss: 0.0312454 | val_loss: 0.0312468 | Time: 3312.73 ms
[2022-02-28 21:57:13 main:574] : INFO : Epoch 65 | loss: 0.0312472 | val_loss: 0.0312487 | Time: 3295.4 ms
[2022-02-28 21:57:16 main:574] : INFO : Epoch 66 | loss: 0.0312475 | val_loss: 0.0312444 | Time: 3143.22 ms
[2022-02-28 21:57:19 main:574] : INFO : Epoch 67 | loss: 0.0312508 | val_loss: 0.0312512 | Time: 3316.86 ms
[2022-02-28 21:57:23 main:574] : INFO : Epoch 68 | loss: 0.0312455 | val_loss: 0.0312428 | Time: 3618.87 ms
[2022-02-28 21:57:27 main:574] : INFO : Epoch 69 | loss: 0.0312428 | val_loss: 0.0312394 | Time: 3599.41 ms
[2022-02-28 21:57:30 main:574] : INFO : Epoch 70 | loss: 0.0312384 | val_loss: 0.0312344 | Time: 3372.72 ms
[2022-02-28 21:57:33 main:574] : INFO : Epoch 71 | loss: 0.0312355 | val_loss: 0.031236 | Time: 2826.71 ms
[2022-02-28 21:57:37 main:574] : INFO : Epoch 72 | loss: 0.0312404 | val_loss: 0.031231 | Time: 3802.69 ms
[2022-02-28 21:57:40 main:574] : INFO : Epoch 73 | loss: 0.0312307 | val_loss: 0.031229 | Time: 3152.03 ms
[2022-02-28 21:57:43 main:574] : INFO : Epoch 74 | loss: 0.0312327 | val_loss: 0.0312348 | Time: 3159.91 ms
[2022-02-28 21:57:46 main:574] : INFO : Epoch 75 | loss: 0.0312356 | val_loss: 0.0312311 | Time: 3449.52 ms
[2022-02-28 21:57:50 main:574] : INFO : Epoch 76 | loss: 0.0312361 | val_loss: 0.0312272 | Time: 3281.58 ms
[2022-02-28 21:57:53 main:574] : INFO : Epoch 77 | loss: 0.0312447 | val_loss: 0.0312279 | Time: 3345.37 ms
[2022-02-28 21:57:56 main:574] : INFO : Epoch 78 | loss: 0.0312327 | val_loss: 0.0312222 | Time: 3338.37 ms
[2022-02-28 21:58:00 main:574] : INFO : Epoch 79 | loss: 0.0312249 | val_loss: 0.0312202 | Time: 3383.72 ms
[2022-02-28 21:58:04 main:574] : INFO : Epoch 80 | loss: 0.0312249 | val_loss: 0.0312245 | Time: 3713.27 ms
[2022-02-28 21:58:07 main:574] : INFO : Epoch 81 | loss: 0.0312479 | val_loss: 0.0312178 | Time: 3417.75 ms
[2022-02-28 21:58:11 main:574] : INFO : Epoch 82 | loss: 0.0312263 | val_loss: 0.0312213 | Time: 3647.26 ms
[2022-02-28 21:58:14 main:574] : INFO : Epoch 83 | loss: 0.0312184 | val_loss: 0.0312124 | Time: 3574.88 ms
[2022-02-28 21:58:18 main:574] : INFO : Epoch 84 | loss: 0.0312124 | val_loss: 0.0312081 | Time: 3549.23 ms
[2022-02-28 21:58:21 main:574] : INFO : Epoch 85 | loss: 0.031217 | val_loss: 0.0312114 | Time: 3029.08 ms
[2022-02-28 21:58:24 main:574] : INFO : Epoch 86 | loss: 0.0312156 | val_loss: 0.0312022 | Time: 3440.45 ms
[2022-02-28 21:58:28 main:574] : INFO : Epoch 87 | loss: 0.0312353 | val_loss: 0.0312043 | Time: 3339.79 ms
[2022-02-28 21:58:31 main:574] : INFO : Epoch 88 | loss: 0.0312085 | val_loss: 0.0312043 | Time: 3166.96 ms
[2022-02-28 21:58:35 main:574] : INFO : Epoch 89 | loss: 0.031202 | val_loss: 0.0311953 | Time: 3687.22 ms
[2022-02-28 21:58:38 main:574] : INFO : Epoch 90 | loss: 0.0312161 | val_loss: 0.0311941 | Time: 3536.71 ms
[2022-02-28 21:58:41 main:574] : INFO : Epoch 91 | loss: 0.0312303 | val_loss: 0.031255 | Time: 3393.06 ms
[2022-02-28 21:58:45 main:574] : INFO : Epoch 92 | loss: 0.0312076 | val_loss: 0.0311933 | Time: 3362.54 ms
[2022-02-28 21:58:48 main:574] : INFO : Epoch 93 | loss: 0.0311905 | val_loss: 0.0311832 | Time: 3112.47 ms
[2022-02-28 21:58:51 main:574] : INFO : Epoch 94 | loss: 0.0311951 | val_loss: 0.0312125 | Time: 3127.33 ms
[2022-02-28 21:58:55 main:574] : INFO : Epoch 95 | loss: 0.0311982 | val_loss: 0.0311972 | Time: 3444.73 ms
[2022-02-28 21:58:58 main:574] : INFO : Epoch 96 | loss: 0.0311912 | val_loss: 0.0311801 | Time: 3452.55 ms
[2022-02-28 21:59:01 main:574] : INFO : Epoch 97 | loss: 0.0311825 | val_loss: 0.0311798 | Time: 3197.65 ms
[2022-02-28 21:59:05 main:574] : INFO : Epoch 98 | loss: 0.0311936 | val_loss: 0.03119 | Time: 3465.48 ms
[2022-02-28 21:59:08 main:574] : INFO : Epoch 99 | loss: 0.031195 | val_loss: 0.0311805 | Time: 3652.98 ms
[2022-02-28 21:59:12 main:574] : INFO : Epoch 100 | loss: 0.0311883 | val_loss: 0.0311938 | Time: 3308.12 ms
[2022-02-28 21:59:15 main:574] : INFO : Epoch 101 | loss: 0.03119 | val_loss: 0.0311996 | Time: 3132.14 ms
[2022-02-28 21:59:19 main:574] : INFO : Epoch 102 | loss: 0.0311835 | val_loss: 0.0311819 | Time: 3730.52 ms
[2022-02-28 21:59:22 main:574] : INFO : Epoch 103 | loss: 0.0311801 | val_loss: 0.0311821 | Time: 3371.17 ms
[2022-02-28 21:59:26 main:574] : INFO : Epoch 104 | loss: 0.0311802 | val_loss: 0.0311806 | Time: 4014.92 ms
[2022-02-28 21:59:29 main:574] : INFO : Epoch 105 | loss: 0.0311789 | val_loss: 0.0311971 | Time: 3219.52 ms
[2022-02-28 21:59:33 main:574] : INFO : Epoch 106 | loss: 0.0311822 | val_loss: 0.0312326 | Time: 3536.44 ms
[2022-02-28 21:59:36 main:574] : INFO : Epoch 107 | loss: 0.0311958 | val_loss: 0.0311784 | Time: 3508.76 ms
[2022-02-28 21:59:40 main:574] : INFO : Epoch 108 | loss: 0.0311752 | val_loss: 0.0311795 | Time: 3450.97 ms
[2022-02-28 21:59:43 main:574] : INFO : Epoch 109 | loss: 0.0311747 | val_loss: 0.0311774 | Time: 3445.42 ms
[2022-02-28 21:59:46 main:574] : INFO : Epoch 110 | loss: 0.0311713 | val_loss: 0.0311765 | Time: 3198.04 ms
[2022-02-28 21:59:50 main:574] : INFO : Epoch 111 | loss: 0.0311725 | val_loss: 0.0311793 | Time: 3625 ms
[2022-02-28 21:59:54 main:574] : INFO : Epoch 112 | loss: 0.0311735 | val_loss: 0.0311745 | Time: 3685.88 ms
[2022-02-28 21:59:56 main:574] : INFO : Epoch 113 | loss: 0.0311783 | val_loss: 0.0311817 | Time: 2318.04 ms
[2022-02-28 21:59:57 main:574] : INFO : Epoch 114 | loss: 0.0311764 | val_loss: 0.0311934 | Time: 1436.08 ms
[2022-02-28 21:59:59 main:574] : INFO : Epoch 115 | loss: 0.0311786 | val_loss: 0.0311764 | Time: 1429.48 ms
[2022-02-28 22:00:00 main:574] : INFO : Epoch 116 | loss: 0.0311717 | val_loss: 0.0311818 | Time: 1457.25 ms
[2022-02-28 22:00:02 main:574] : INFO : Epoch 117 | loss: 0.0311707 | val_loss: 0.0311754 | Time: 1474.78 ms
[2022-02-28 22:00:03 main:574] : INFO : Epoch 118 | loss: 0.0311708 | val_loss: 0.0311764 | Time: 1484.18 ms
[2022-02-28 22:00:05 main:574] : INFO : Epoch 119 | loss: 0.0311796 | val_loss: 0.0311765 | Time: 1457.81 ms
[2022-02-28 22:00:06 main:574] : INFO : Epoch 120 | loss: 0.0311789 | val_loss: 0.031176 | Time: 1455.64 ms
[2022-02-28 22:00:08 main:574] : INFO : Epoch 121 | loss: 0.0311683 | val_loss: 0.0311749 | Time: 1454.79 ms
[2022-02-28 22:00:11 main:574] : INFO : Epoch 122 | loss: 0.0311701 | val_loss: 0.0311755 | Time: 3019.08 ms
[2022-02-28 22:00:14 main:574] : INFO : Epoch 123 | loss: 0.0311687 | val_loss: 0.0311746 | Time: 3517.24 ms
[2022-02-28 22:00:18 main:574] : INFO : Epoch 124 | loss: 0.0311678 | val_loss: 0.0311734 | Time: 3446.62 ms
[2022-02-28 22:00:21 main:574] : INFO : Epoch 125 | loss: 0.0311663 | val_loss: 0.0311786 | Time: 3378.09 ms
[2022-02-28 22:00:24 main:574] : INFO : Epoch 126 | loss: 0.0311709 | val_loss: 0.0311759 | Time: 3365.44 ms
[2022-02-28 22:00:28 main:574] : INFO : Epoch 127 | loss: 0.0311686 | val_loss: 0.0311741 | Time: 3289.9 ms
[2022-02-28 22:00:31 main:574] : INFO : Epoch 128 | loss: 0.0311682 | val_loss: 0.0311755 | Time: 3535.76 ms
[2022-02-28 22:00:34 main:574] : INFO : Epoch 129 | loss: 0.0311692 | val_loss: 0.0311798 | Time: 3230.19 ms
[2022-02-28 22:00:38 main:574] : INFO : Epoch 130 | loss: 0.0311649 | val_loss: 0.0311744 | Time: 3308.24 ms
[2022-02-28 22:00:41 main:574] : INFO : Epoch 131 | loss: 0.0311657 | val_loss: 0.0311862 | Time: 3628.51 ms
[2022-02-28 22:00:45 main:574] : INFO : Epoch 132 | loss: 0.0311745 | val_loss: 0.0311745 | Time: 3254.64 ms
[2022-02-28 22:00:48 main:574] : INFO : Epoch 133 | loss: 0.0311638 | val_loss: 0.0311731 | Time: 3404.4 ms
[2022-02-28 22:00:51 main:574] : INFO : Epoch 134 | loss: 0.031164 | val_loss: 0.0311762 | Time: 3353.8 ms
[2022-02-28 22:00:55 main:574] : INFO : Epoch 135 | loss: 0.0311637 | val_loss: 0.0311714 | Time: 3474.72 ms
[2022-02-28 22:00:58 main:574] : INFO : Epoch 136 | loss: 0.0311642 | val_loss: 0.0311734 | Time: 3447.33 ms
[2022-02-28 22:01:02 main:574] : INFO : Epoch 137 | loss: 0.0311633 | val_loss: 0.0311725 | Time: 3338.11 ms
[2022-02-28 22:01:05 main:574] : INFO : Epoch 138 | loss: 0.0311603 | val_loss: 0.0311703 | Time: 3093.31 ms
[2022-02-28 22:01:08 main:574] : INFO : Epoch 139 | loss: 0.0311599 | val_loss: 0.0311711 | Time: 3207.94 ms
[2022-02-28 22:01:12 main:574] : INFO : Epoch 140 | loss: 0.0311631 | val_loss: 0.0311722 | Time: 3593.09 ms
[2022-02-28 22:01:15 main:574] : INFO : Epoch 141 | loss: 0.0311635 | val_loss: 0.0311769 | Time: 3516.39 ms
[2022-02-28 22:01:19 main:574] : INFO : Epoch 142 | loss: 0.0311604 | val_loss: 0.0311711 | Time: 3439.83 ms
[2022-02-28 22:01:22 main:574] : INFO : Epoch 143 | loss: 0.0311618 | val_loss: 0.0311851 | Time: 3314.55 ms
[2022-02-28 22:01:25 main:574] : INFO : Epoch 144 | loss: 0.0311695 | val_loss: 0.0311791 | Time: 3306.97 ms
[2022-02-28 22:01:29 main:574] : INFO : Epoch 145 | loss: 0.0311605 | val_loss: 0.0311707 | Time: 3434.63 ms
[2022-02-28 22:01:32 main:574] : INFO : Epoch 146 | loss: 0.0311605 | val_loss: 0.0311732 | Time: 3129.21 ms
[2022-02-28 22:01:35 main:574] : INFO : Epoch 147 | loss: 0.0311573 | val_loss: 0.0311796 | Time: 3609.49 ms
[2022-02-28 22:01:39 main:574] : INFO : Epoch 148 | loss: 0.0311586 | val_loss: 0.0311756 | Time: 3585.4 ms
[2022-02-28 22:01:42 main:574] : INFO : Epoch 149 | loss: 0.0311587 | val_loss: 0.0311734 | Time: 3402.91 ms
[2022-02-28 22:01:46 main:574] : INFO : Epoch 150 | loss: 0.0311565 | val_loss: 0.0311703 | Time: 3269.71 ms
[2022-02-28 22:01:49 main:574] : INFO : Epoch 151 | loss: 0.0311545 | val_loss: 0.0311729 | Time: 3329.67 ms
[2022-02-28 22:01:52 main:574] : INFO : Epoch 152 | loss: 0.0311543 | val_loss: 0.0311721 | Time: 3094.17 ms
[2022-02-28 22:01:56 main:574] : INFO : Epoch 153 | loss: 0.031156 | val_loss: 0.0311738 | Time: 3619.36 ms
[2022-02-28 22:01:59 main:574] : INFO : Epoch 154 | loss: 0.0311526 | val_loss: 0.0311752 | Time: 3671 ms
[2022-02-28 22:02:03 main:574] : INFO : Epoch 155 | loss: 0.0311536 | val_loss: 0.0311805 | Time: 3512.25 ms
[2022-02-28 22:02:06 main:574] : INFO : Epoch 156 | loss: 0.0311646 | val_loss: 0.0311805 | Time: 3435.73 ms
[2022-02-28 22:02:10 main:574] : INFO : Epoch 157 | loss: 0.0311569 | val_loss: 0.0311718 | Time: 3459.29 ms
[2022-02-28 22:02:13 main:574] : INFO : Epoch 158 | loss: 0.0311604 | val_loss: 0.0311785 | Time: 3419.86 ms
[2022-02-28 22:02:17 main:574] : INFO : Epoch 159 | loss: 0.0311572 | val_loss: 0.0311702 | Time: 3515.93 ms
[2022-02-28 22:02:20 main:574] : INFO : Epoch 160 | loss: 0.0311576 | val_loss: 0.0311778 | Time: 3507.96 ms
[2022-02-28 22:02:24 main:574] : INFO : Epoch 161 | loss: 0.0311559 | val_loss: 0.0311774 | Time: 3186.21 ms
[2022-02-28 22:02:27 main:574] : INFO : Epoch 162 | loss: 0.031152 | val_loss: 0.0311974 | Time: 3299.27 ms
[2022-02-28 22:02:30 main:574] : INFO : Epoch 163 | loss: 0.031157 | val_loss: 0.0311749 | Time: 3491.65 ms
[2022-02-28 22:02:34 main:574] : INFO : Epoch 164 | loss: 0.0311513 | val_loss: 0.0311697 | Time: 3254.13 ms
[2022-02-28 22:02:37 main:574] : INFO : Epoch 165 | loss: 0.0311531 | val_loss: 0.0311764 | Time: 3092.54 ms
[2022-02-28 22:02:40 main:574] : INFO : Epoch 166 | loss: 0.0311509 | val_loss: 0.0311795 | Time: 3076.6 ms
[2022-02-28 22:02:43 main:574] : INFO : Epoch 167 | loss: 0.0311517 | val_loss: 0.0311717 | Time: 3577.59 ms
[2022-02-28 22:02:47 main:574] : INFO : Epoch 168 | loss: 0.0311498 | val_loss: 0.0311817 | Time: 3652.98 ms
[2022-02-28 22:02:50 main:574] : INFO : Epoch 169 | loss: 0.0311473 | val_loss: 0.0311758 | Time: 3387.05 ms
[2022-02-28 22:02:54 main:574] : INFO : Epoch 170 | loss: 0.0311525 | val_loss: 0.0311757 | Time: 3099.25 ms
[2022-02-28 22:02:57 main:574] : INFO : Epoch 171 | loss: 0.0311497 | val_loss: 0.0311782 | Time: 3579.39 ms
[2022-02-28 22:03:00 main:574] : INFO : Epoch 172 | loss: 0.0311479 | val_loss: 0.0311807 | Time: 3152.79 ms
[2022-02-28 22:03:04 main:574] : INFO : Epoch 173 | loss: 0.0311476 | val_loss: 0.0311746 | Time: 3421.96 ms
[2022-02-28 22:03:07 main:574] : INFO : Epoch 174 | loss: 0.0311458 | val_loss: 0.0311775 | Time: 3178.16 ms
[2022-02-28 22:03:10 main:574] : INFO : Epoch 175 | loss: 0.0311466 | val_loss: 0.0311803 | Time: 3638.16 ms
[2022-02-28 22:03:14 main:574] : INFO : Epoch 176 | loss: 0.0311475 | val_loss: 0.0311821 | Time: 3266.02 ms
[2022-02-28 22:03:17 main:574] : INFO : Epoch 177 | loss: 0.0311497 | val_loss: 0.031179 | Time: 3242.76 ms
[2022-02-28 22:03:20 main:574] : INFO : Epoch 178 | loss: 0.0311464 | val_loss: 0.0311796 | Time: 3268.58 ms
[2022-02-28 22:03:24 main:574] : INFO : Epoch 179 | loss: 0.0311456 | val_loss: 0.0311847 | Time: 3218.1 ms
[2022-02-28 22:03:27 main:574] : INFO : Epoch 180 | loss: 0.031152 | val_loss: 0.0311846 | Time: 3094.06 ms
[2022-02-28 22:03:30 main:574] : INFO : Epoch 181 | loss: 0.0311489 | val_loss: 0.0311758 | Time: 3601.84 ms
[2022-02-28 22:03:34 main:574] : INFO : Epoch 182 | loss: 0.0311455 | val_loss: 0.0311773 | Time: 3457.63 ms
[2022-02-28 22:03:37 main:574] : INFO : Epoch 183 | loss: 0.0311452 | val_loss: 0.0311947 | Time: 3138.69 ms
[2022-02-28 22:03:40 main:574] : INFO : Epoch 184 | loss: 0.0311534 | val_loss: 0.0311695 | Time: 3101.37 ms
[2022-02-28 22:03:43 main:574] : INFO : Epoch 185 | loss: 0.031149 | val_loss: 0.0311833 | Time: 3423.14 ms
[2022-02-28 22:03:47 main:574] : INFO : Epoch 186 | loss: 0.031147 | val_loss: 0.0311836 | Time: 3414.03 ms
[2022-02-28 22:03:50 main:574] : INFO : Epoch 187 | loss: 0.031142 | val_loss: 0.0311829 | Time: 3663.92 ms
[2022-02-28 22:03:54 main:574] : INFO : Epoch 188 | loss: 0.0311442 | val_loss: 0.0311773 | Time: 3369.88 ms
[2022-02-28 22:03:58 main:574] : INFO : Epoch 189 | loss: 0.0311433 | val_loss: 0.0311786 | Time: 3723.44 ms
[2022-02-28 22:04:01 main:574] : INFO : Epoch 190 | loss: 0.031145 | val_loss: 0.0311814 | Time: 3031.15 ms
[2022-02-28 22:04:04 main:574] : INFO : Epoch 191 | loss: 0.0311523 | val_loss: 0.0311738 | Time: 3386.87 ms
[2022-02-28 22:04:07 main:574] : INFO : Epoch 192 | loss: 0.031154 | val_loss: 0.0311794 | Time: 3031.78 ms
[2022-02-28 22:04:10 main:574] : INFO : Epoch 193 | loss: 0.0311467 | val_loss: 0.0311973 | Time: 3410.04 ms
[2022-02-28 22:04:14 main:574] : INFO : Epoch 194 | loss: 0.031146 | val_loss: 0.0311847 | Time: 3271.03 ms
[2022-02-28 22:04:17 main:574] : INFO : Epoch 195 | loss: 0.0311403 | val_loss: 0.0311813 | Time: 3052.4 ms
[2022-02-28 22:04:20 main:574] : INFO : Epoch 196 | loss: 0.0311488 | val_loss: 0.0311793 | Time: 3494.92 ms
[2022-02-28 22:04:24 main:574] : INFO : Epoch 197 | loss: 0.0311387 | val_loss: 0.0311824 | Time: 3232.21 ms
[2022-02-28 22:04:27 main:574] : INFO : Epoch 198 | loss: 0.0311401 | val_loss: 0.031182 | Time: 3320.32 ms
[2022-02-28 22:04:30 main:574] : INFO : Epoch 199 | loss: 0.031141 | val_loss: 0.031178 | Time: 3281.72 ms
[2022-02-28 22:04:33 main:574] : INFO : Epoch 200 | loss: 0.0311437 | val_loss: 0.0312124 | Time: 3152.94 ms
[2022-02-28 22:04:37 main:574] : INFO : Epoch 201 | loss: 0.0311493 | val_loss: 0.0311863 | Time: 3737.74 ms
[2022-02-28 22:04:40 main:574] : INFO : Epoch 202 | loss: 0.0311405 | val_loss: 0.0311736 | Time: 3131.02 ms
[2022-02-28 22:04:43 main:574] : INFO : Epoch 203 | loss: 0.0311424 | val_loss: 0.0311904 | Time: 3230.86 ms
[2022-02-28 22:04:47 main:574] : INFO : Epoch 204 | loss: 0.0311478 | val_loss: 0.0311828 | Time: 3149.65 ms
[2022-02-28 22:04:50 main:574] : INFO : Epoch 205 | loss: 0.0311444 | val_loss: 0.0311975 | Time: 3493.62 ms
[2022-02-28 22:04:53 main:574] : INFO : Epoch 206 | loss: 0.0311375 | val_loss: 0.0311827 | Time: 3232.77 ms
[2022-02-28 22:04:57 main:574] : INFO : Epoch 207 | loss: 0.031134 | val_loss: 0.0311855 | Time: 3470.48 ms
[2022-02-28 22:05:00 main:574] : INFO : Epoch 208 | loss: 0.0311358 | val_loss: 0.0311956 | Time: 3379.09 ms
[2022-02-28 22:05:04 main:574] : INFO : Epoch 209 | loss: 0.031135 | val_loss: 0.0311879 | Time: 3373.96 ms
[2022-02-28 22:05:07 main:574] : INFO : Epoch 210 | loss: 0.0311351 | val_loss: 0.0311799 | Time: 3651.18 ms
[2022-02-28 22:05:10 main:574] : INFO : Epoch 211 | loss: 0.0311401 | val_loss: 0.031179 | Time: 3112.71 ms
[2022-02-28 22:05:14 main:574] : INFO : Epoch 212 | loss: 0.0311454 | val_loss: 0.0311919 | Time: 3306.42 ms
[2022-02-28 22:05:17 main:574] : INFO : Epoch 213 | loss: 0.0311389 | val_loss: 0.0311856 | Time: 3200.03 ms
[2022-02-28 22:05:20 main:574] : INFO : Epoch 214 | loss: 0.0311389 | val_loss: 0.0311804 | Time: 3378.9 ms
[2022-02-28 22:05:23 main:574] : INFO : Epoch 215 | loss: 0.0311336 | val_loss: 0.0311865 | Time: 3214.08 ms
[2022-02-28 22:05:26 main:574] : INFO : Epoch 216 | loss: 0.0311331 | val_loss: 0.0311751 | Time: 3044.66 ms
[2022-02-28 22:05:30 main:574] : INFO : Epoch 217 | loss: 0.0311396 | val_loss: 0.031197 | Time: 3555.37 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFE251990D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 02/28/22 22:05:32
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\0;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 0000000016e50000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000027670000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000025f60000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000250d0000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000022360000 0000000000090000 C:\WINDOWS\SYSTEM32\apphelp.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : apphelp.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027160000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000256a0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026fc0000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 00000000254f0000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000253a0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000024fd0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000266e0000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027300000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000026020000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026460000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 000000000b660000 0000000000058000 C:\ProgramData\BOINC\slots\0\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000000eb20000 0000000000091000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000132b0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000000ebc0000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 000000000eb10000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 000000008c220000 000000000abe7000 C:\ProgramData\BOINC\slots\0\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000098600000 00000000002db000 C:\ProgramData\BOINC\slots\0\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000becd0000 000000000020d000 C:\ProgramData\BOINC\slots\0\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 000000000b5e0000 0000000000040000 C:\ProgramData\BOINC\slots\0\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000676d0000 0000000024b49000 C:\ProgramData\BOINC\slots\0\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000fef10000 0000000000045000 C:\ProgramData\BOINC\slots\0\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000064210000 00000000034b2000 C:\ProgramData\BOINC\slots\0\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 000000000ff60000 0000000000010000 C:\ProgramData\BOINC\slots\0\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000061160000 00000000030a5000 C:\ProgramData\BOINC\slots\0\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000045db0000 000000001b3a4000 C:\ProgramData\BOINC\slots\0\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 0000000042050000 0000000003d5b000 C:\ProgramData\BOINC\slots\0\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 0000000038eb0000 000000000919f000 C:\ProgramData\BOINC\slots\0\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 0000000036fe0000 0000000001eca000 C:\ProgramData\BOINC\slots\0\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000260c0000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000022cc0000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000026850000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000024f80000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000025670000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000024b30000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000025600000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000024d90000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000024970000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000025810000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 0000000022ec0000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026100000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 00000000247f0000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026f10000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027100000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000beee0000 00000000002cd000 C:\WINDOWS\system32\nvcuda.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000a22f0000 00000000016a0000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispig.inf_amd64_31a2adf8c49e7799\nvcuda64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 000000001e4e0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001e4a0000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 000000001c0e0000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000024740000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000018040000 0000000000768000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1165
ModLoad: 0000000036720000 00000000008b3000 C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispig.inf_amd64_31a2adf8c49e7799\nvptxJitCompiler64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows PTX JIT Compiler
Product Version : 30.0.15.1165
ModLoad: 0000000021680000 0000000000009000 C:\ProgramData\BOINC\slots\0\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000f4c70000 0000000000fca000 C:\ProgramData\BOINC\slots\0\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000024ef0000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 5353, Write: 4719, Other 21517
- I/O Transfers Counters -
Read: 26449755, Write: -1939559358, Other 68188
- Paged Pool Usage -
QuotaPagedPoolUsage: 3536072, QuotaPeakPagedPoolUsage: 3536072
QuotaNonPagedPoolUsage: 59288, QuotaPeakNonPagedPoolUsage: 80776
- Virtual Memory Usage -
VirtualSize: 460341248, PeakVirtualSize: 1637085184
- Pagefile Usage -
PagefileUsage: 460341248, PeakPagefileUsage: 831066112
- Working Set Size -
WorkingSetSize: -1863725056, PeakWorkingSetSize: -1174249472, PageFaultCount: 41678826
*** Dump of thread ID 11684 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 44779740.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFE251990D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=0000000017108c40 rdx=000000008cffe750 rsi=0000000000000000 rdi=0000000000000000
r8=000000008cffe750 r9=0000000017108c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000251990d2 rsp=000000008cffe728 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8cffe720 16e89100 00000001 8cffe750 8cffe750 17108c30 KERNELBASE!DebugBreak+0x0
8cffeb60 16e89ce4 0d400000 8cffeec4 00000000 000003d0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8cfffac0 16e8a084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8cfffaf0 25f77034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8cfffb20 276c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8cfffba0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 7996 (state: Ready): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 4075468800.000000, User Time: 40902967296.000000, Wait Time: 44779740.000000
- Registers -
rax=0000000000000001 rbx=0000000014b750a0 rcx=0000000000000001 rdx=0000000089e8df80 rsi=000000006ead59e0 rdi=0000000000000000
r8=0000000001e13220 r9=0000000000000000 r10=000000000000c000 r11=0000000000000000 r12=0000000001e13220 r13=0000000000000000
r14=000000008cae1200 r15=000000008cae1260 rip=00000000a24808ff rsp=000000008cae0880 rbp=000000008cae0980
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
8cae0eb0 a2632e95 00000024 8cae1480 00000001 8cae1560 nvcuda64!+0x0
8cae0ff0 a232c464 6ead59e0 8cae1260 6ead59e0 8cae1480 nvcuda64!cuProfilerStop+0x0
8cae10c0 a232ca57 8cae11d0 14b750a0 8cae11d0 6ead59e0 nvcuda64!+0x0
8cae1430 a252c8bc 00000006 a259d056 8cae1be8 00000000 nvcuda64!+0x0
8cae15a0 46b529b6 00000000 00000006 00000100 46abd5d0 nvcuda64!cuEventRecordWithFlags+0x0
8cae16b0 46b4fdf4 46abd5d0 1330ffd0 6ead59e0 8cae18c0 cudnn64_7!cudnnSetDropoutDescriptor+0x0
8cae1830 46ac2719 46abd5d0 00000002 46abd5d0 0000000c cudnn64_7!cudnnSetDropoutDescriptor+0x0
8cae1950 46abce23 0000000c 0000000c 00000080 00000080 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
8cae1a30 46acb483 00000000 00000000 3db12000 00000000 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
8cae1ee0 46abeff1 000001c0 6907e9b0 8cae41a8 ea3912c0 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
8cae2080 46ac7d8e e8afab00 00000000 ea3912c0 8cae2328 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
8cae2220 46add24a 8cae2328 8cae2328 8cae2328 8cae2328 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
8cae4390 6a448bd1 e8af0100 3e200000 e8af0100 6907e9b0 cudnn64_7!cudnnRNNForwardInference+0x0
8cae4870 6a4a25ea 8cae4938 00000001 8cae4938 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
8cae4950 6a4b0cbf 8cae5040 8cae6838 8cae5990 00000001 torch_cuda!at::native::set_storage_cuda_+0x0
8cae4a30 91c6b9fc 8cae5040 8cae5720 00000000 8cae4b48 torch_cuda!at::native::set_storage_cuda_+0x0
8cae4bb0 91ca31a2 8cae4f40 00000001 00000000 00000000 torch_cpu!at::native::mkldnn_sigmoid_+0x0
8cae4e90 92f52613 00000000 8cae59a0 8cae59a0 8cae4fa0 torch_cpu!at::_cudnn_rnn+0x0
8cae5390 92f6080b b61ff3ef 4f674f20 8cae54c0 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
8cae5470 91c6b9fc 8cae59f0 93085878 8cae55d0 a2317a77 torch_cpu!torch::autograd::GraphRoot::apply+0x0
8cae55f0 91ca31a2 ea391840 00000000 b61ff57f 6b36a3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
8cae58d0 6a4469a1 8cae0001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
8cae5aa0 6a450117 00000000 8cae6838 fef12e0d 8cae5bc0 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
8cae5b30 919f107b 96b41ac0 8cae5e50 9df74872 a24e7783 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
8cae5da0 91a23b16 00000001 8cae6838 00000080 a2cc3958 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
8cae5f30 91d9ae89 8cae5fa8 00000001 8cae5fa8 00000001 torch_cpu!at::native::gru+0x0
8cae5fc0 93089fa3 8cae60c0 00000001 8cae6240 8cae6838 torch_cpu!at::zeros_out+0x0
8cae6050 91d8dd35 0b6b1ea8 1425f600 00000000 0b6b1ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
8cae60d0 91c688ec 96b475c8 6e9ffe20 d7c910e0 8cae62f0 torch_cpu!at::zeros_out+0x0
8cae61d0 91cdc1bb 0000000c 00000080 00000004 4f675500 torch_cpu!at::native::mkldnn_sigmoid_+0x0
8cae6250 937c7294 8cae6370 6e9ffe20 8cae6360 8cae6508 torch_cpu!at::gru+0x0
8cae6480 937c62ab 8cae6818 91bf030e 8cae6818 8cae6838 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
8cae6570 16e62621 00000002 adaa6bc0 00000000 0b6b1ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
8cae6770 16e67e98 8cae67c8 8cae0000 adaa0601 adaa6bc0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
8cae6c10 16e6b964 8cae6d10 8cae6d10 00000000 593bbfd0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::unsafeRemoveSlot+0x0
8caff9b0 170ac6f8 1476f850 00000000 00000000 141ea360 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8caff9f0 25f77034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8caffa20 276c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8caffaa0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 9528 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 44779736.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=000000008d0ff900 rsi=0000000000000000 rdi=000000000000000d
r8=000000008d0ff5a8 r9=00000000148a5f80 r10=0000000000000000 r11=0000000000000246 r12=0000000000000064 r13=000000008d0ff900
r14=000000008d0ff600 r15=0000000000000000 rip=000000002770d8c4 rsp=000000008d0ff5a8 rbp=00000000148a5f80
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8d0ff5a0 2511cb20 00000000 8c810000 277da4d0 8c810000 ntdll!ZwWaitForMultipleObjects+0x0
8d0ff890 2511ca1e 12b29420 004e0049 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
8d0ff8d0 a24cd554 148a88c0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
8d0ffb10 a25758cd 00000000 148a5f80 00000000 00000000 nvcuda64!+0x0
8d0ffbc0 a24ccf23 144f22cf 1df426c4 14eaf880 00000000 nvcuda64!cuProfilerStop+0x0
8d0ffbf0 a28044c4 14eaf880 00000000 00000000 00000000 nvcuda64!+0x0
8d0ffc20 25f77034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
8d0ffc50 276c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8d0ffcd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 2976 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 43872976.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002f8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002f8
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000014902b30 r13=0000000000000000
r14=00000000000002f8 r15=0000000000000000 rip=000000002770cdf4 rsp=000000008d1ff588 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8d1ff580 250f1a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
8d1ff620 a26f5d0e 14eb0060 00000000 00000000 000002f8 KERNELBASE!WaitForSingleObjectEx+0x0
8d1ff860 a24ccf23 00000000 14eb0060 a22f0000 149179e0 nvcuda64!cuProfilerStop+0x0
8d1ff890 a28044c4 14eb0060 00000000 00000000 00000000 nvcuda64!+0x0
8d1ff8c0 25f77034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
8d1ff8f0 276c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8d1ff970 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 13460 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 57968752.000000, User Time: 1396718720.000000, Wait Time: 44779736.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000f007d708 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000f007d708
r8=0000000091aa4a40 r9=00000000134d6e78 r10=0000000000000038 r11=000000008cbfc230 r12=0000000000000000 r13=0000000000000000
r14=000000008cbff560 r15=00000000f007d6b8 rip=0000000027710764 rsp=000000008cbff538 rbp=000000008cbff590
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
8cbff530 276d4021 68c626b8 68c626b8 8cbff670 68c626b8 ntdll!NtWaitForAlertByThreadId+0x0
8cbff5b0 2513cfa9 f007d6b0 f007d6f8 00000000 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
8cbff5f0 0eb5ba8d 0e8f1d00 0e8f1d00 0e8f1d00 00000000 KERNELBASE!SleepConditionVariableSRW+0x0
8cbff630 0eb32848 6beafa10 00000000 00000000 934005b4 MSVCP140!__crtSetThreadpoolWait+0x0
8cbff660 0eb32710 00000000 0eb32f17 00000000 6beafa10 MSVCP140!std::_Winerror_message+0x0
8cbff6d0 9340064b 6beaf9f0 000001c0 f007d6b0 f007d6b0 MSVCP140!std::_Winerror_message+0x0
8cbff710 934025b1 e9a44090 ffffffff 6beaf9f0 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
8cbffa00 93402471 96926d10 8cbffa40 96926d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
8cbffa50 933f9744 00080001 a2ca42c0 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
8cbffa80 24ff1bb2 a2ca42c0 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
8cbffab0 25f77034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
8cbffae0 276c2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8cbffb60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)