Task 14553051
| Name | ParityModified-1647045646-21643-3-0_1 |
| Workunit | 11371572 |
| Created | 9 Apr 2022, 10:57:12 UTC |
| Sent | 9 Apr 2022, 12:01:05 UTC |
| Report deadline | 17 Apr 2022, 12:01:05 UTC |
| Received | 10 Apr 2022, 3:45:58 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED |
| Computer ID | 4503 |
| Run time | 6 hours 59 min 56 sec |
| CPU time | 31 min 22 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 10,073.24 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.08 GB |
| Peak swap size | 4.40 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
exceeded elapsed time limit 25192.79 (4000000.00G/158.78G)</message>
<stderr_txt>
038 | loss: 0.0312454 | val_loss: 0.0312439 | Time: 1184.68 ms
[2022-04-09 12:43:13 main:574] : INFO : Epoch 1039 | loss: 0.0312453 | val_loss: 0.0312438 | Time: 1207.73 ms
[2022-04-09 12:43:14 main:574] : INFO : Epoch 1040 | loss: 0.0312453 | val_loss: 0.0312441 | Time: 1613.33 ms
[2022-04-09 12:43:16 main:574] : INFO : Epoch 1041 | loss: 0.0312453 | val_loss: 0.0312439 | Time: 1705.38 ms
[2022-04-09 12:43:18 main:574] : INFO : Epoch 1042 | loss: 0.0312452 | val_loss: 0.0312438 | Time: 1720.54 ms
[2022-04-09 12:43:20 main:574] : INFO : Epoch 1043 | loss: 0.0312454 | val_loss: 0.0312443 | Time: 1807.03 ms
[2022-04-09 12:43:21 main:574] : INFO : Epoch 1044 | loss: 0.0312458 | val_loss: 0.0312442 | Time: 1762.71 ms
[2022-04-09 12:43:23 main:574] : INFO : Epoch 1045 | loss: 0.0312452 | val_loss: 0.0312439 | Time: 1738.29 ms
[2022-04-09 12:43:25 main:574] : INFO : Epoch 1046 | loss: 0.0312451 | val_loss: 0.0312439 | Time: 1787.35 ms
[2022-04-09 12:43:27 main:574] : INFO : Epoch 1047 | loss: 0.0312454 | val_loss: 0.0312448 | Time: 1735.41 ms
[2022-04-09 12:43:29 main:574] : INFO : Epoch 1048 | loss: 0.0312456 | val_loss: 0.031244 | Time: 1827.86 ms
[2022-04-09 12:43:30 main:574] : INFO : Epoch 1049 | loss: 0.0312456 | val_loss: 0.0312437 | Time: 1767.22 ms
[2022-04-09 12:43:32 main:574] : INFO : Epoch 1050 | loss: 0.0312451 | val_loss: 0.0312435 | Time: 1776.14 ms
[2022-04-09 12:43:34 main:574] : INFO : Epoch 1051 | loss: 0.0312449 | val_loss: 0.0312435 | Time: 1737.08 ms
[2022-04-09 12:43:36 main:574] : INFO : Epoch 1052 | loss: 0.0312451 | val_loss: 0.0312435 | Time: 1758.31 ms
[2022-04-09 12:43:37 main:574] : INFO : Epoch 1053 | loss: 0.0312449 | val_loss: 0.0312435 | Time: 1693.02 ms
[2022-04-09 12:43:39 main:574] : INFO : Epoch 1054 | loss: 0.0312449 | val_loss: 0.0312439 | Time: 1646.66 ms
[2022-04-09 12:43:41 main:574] : INFO : Epoch 1055 | loss: 0.0312453 | val_loss: 0.0312441 | Time: 1759.37 ms
[2022-04-09 12:43:42 main:574] : INFO : Epoch 1056 | loss: 0.0312452 | val_loss: 0.0312436 | Time: 1730.5 ms
[2022-04-09 12:43:44 main:574] : INFO : Epoch 1057 | loss: 0.0312448 | val_loss: 0.0312435 | Time: 1783.01 ms
[2022-04-09 12:43:46 main:574] : INFO : Epoch 1058 | loss: 0.0312448 | val_loss: 0.0312442 | Time: 1800.28 ms
[2022-04-09 12:43:48 main:574] : INFO : Epoch 1059 | loss: 0.0312447 | val_loss: 0.0312435 | Time: 1754.25 ms
[2022-04-09 12:43:50 main:574] : INFO : Epoch 1060 | loss: 0.0312449 | val_loss: 0.0312435 | Time: 2200.18 ms
[2022-04-09 12:43:53 main:574] : INFO : Epoch 1061 | loss: 0.0312449 | val_loss: 0.0312439 | Time: 2393.09 ms
[2022-04-09 12:45:14 main:574] : INFO : Epoch 1062 | loss: 0.0312446 | val_loss: 0.0312435 | Time: 81599.6 ms
[2022-04-09 12:47:26 main:574] : INFO : Epoch 1063 | loss: 0.0312457 | val_loss: 0.0312449 | Time: 124386 ms
[2022-04-09 12:49:33 main:574] : INFO : Epoch 1064 | loss: 0.0312455 | val_loss: 0.0312436 | Time: 121419 ms
[2022-04-09 12:52:32 main:574] : INFO : Epoch 1065 | loss: 0.0312449 | val_loss: 0.0312438 | Time: 179002 ms
[2022-04-09 12:54:48 main:574] : INFO : Epoch 1066 | loss: 0.0312448 | val_loss: 0.0312435 | Time: 136275 ms
[2022-04-09 12:57:44 main:574] : INFO : Epoch 1067 | loss: 0.031245 | val_loss: 0.0312441 | Time: 170392 ms
[2022-04-09 13:00:32 main:574] : INFO : Epoch 1068 | loss: 0.0312451 | val_loss: 0.0312437 | Time: 163863 ms
[2022-04-09 13:03:53 main:574] : INFO : Epoch 1069 | loss: 0.0312448 | val_loss: 0.0312437 | Time: 200661 ms
[2022-04-09 13:06:37 main:574] : INFO : Epoch 1070 | loss: 0.0312449 | val_loss: 0.0312437 | Time: 163838 ms
[2022-04-09 13:10:28 main:574] : INFO : Epoch 1071 | loss: 0.0312451 | val_loss: 0.0312439 | Time: 226847 ms
[2022-04-09 13:12:44 main:574] : INFO : Epoch 1072 | loss: 0.031245 | val_loss: 0.0312438 | Time: 131078 ms
[2022-04-09 13:15:25 main:574] : INFO : Epoch 1073 | loss: 0.0312451 | val_loss: 0.0312437 | Time: 160443 ms
[2022-04-09 13:18:19 main:574] : INFO : Epoch 1074 | loss: 0.031245 | val_loss: 0.0312439 | Time: 174286 ms
[2022-04-09 13:20:51 main:574] : INFO : Epoch 1075 | loss: 0.0312447 | val_loss: 0.0312438 | Time: 144055 ms
[2022-04-09 13:23:54 main:574] : INFO : Epoch 1076 | loss: 0.0312452 | val_loss: 0.0312445 | Time: 182186 ms
[2022-04-09 13:25:46 main:574] : INFO : Epoch 1077 | loss: 0.0312449 | val_loss: 0.0312436 | Time: 112176 ms
[2022-04-09 13:29:30 main:574] : INFO : Epoch 1078 | loss: 0.0312448 | val_loss: 0.0312443 | Time: 224089 ms
[2022-04-09 13:31:40 main:574] : INFO : Epoch 1079 | loss: 0.0312452 | val_loss: 0.0312436 | Time: 129441 ms
[2022-04-09 13:33:53 main:574] : INFO : Epoch 1080 | loss: 0.031245 | val_loss: 0.0312437 | Time: 133465 ms
[2022-04-09 13:37:10 main:574] : INFO : Epoch 1081 | loss: 0.0312449 | val_loss: 0.0312445 | Time: 182863 ms
[2022-04-09 13:40:14 main:574] : INFO : Epoch 1082 | loss: 0.0312449 | val_loss: 0.0312441 | Time: 184136 ms
[2022-04-09 13:43:49 main:574] : INFO : Epoch 1083 | loss: 0.0312448 | val_loss: 0.0312439 | Time: 209576 ms
[2022-04-09 13:47:36 main:574] : INFO : Epoch 1084 | loss: 0.0312452 | val_loss: 0.0312443 | Time: 226787 ms
[2022-04-09 13:50:48 main:574] : INFO : Epoch 1085 | loss: 0.031246 | val_loss: 0.0312454 | Time: 192112 ms
[2022-04-09 13:52:33 main:574] : INFO : Epoch 1086 | loss: 0.0312449 | val_loss: 0.0312437 | Time: 105045 ms
[2022-04-09 13:55:42 main:574] : INFO : Epoch 1087 | loss: 0.0312446 | val_loss: 0.0312439 | Time: 184237 ms
[2022-04-09 13:57:14 main:574] : INFO : Epoch 1088 | loss: 0.0312448 | val_loss: 0.0312445 | Time: 91135.8 ms
[2022-04-09 13:59:55 main:574] : INFO : Epoch 1089 | loss: 0.0312453 | val_loss: 0.0312451 | Time: 156389 ms
[2022-04-09 14:03:52 main:574] : INFO : Epoch 1090 | loss: 0.0312454 | val_loss: 0.0312454 | Time: 231572 ms
[2022-04-09 14:05:42 main:574] : INFO : Epoch 1091 | loss: 0.0312458 | val_loss: 0.0312438 | Time: 105817 ms
[2022-04-09 14:09:35 main:574] : INFO : Epoch 1092 | loss: 0.0312447 | val_loss: 0.0312441 | Time: 232628 ms
[2022-04-09 14:12:49 main:574] : INFO : Epoch 1093 | loss: 0.0312453 | val_loss: 0.0312439 | Time: 189201 ms
[2022-04-09 14:15:34 main:574] : INFO : Epoch 1094 | loss: 0.0312449 | val_loss: 0.0312442 | Time: 159982 ms
[2022-04-09 14:19:24 main:574] : INFO : Epoch 1095 | loss: 0.031245 | val_loss: 0.0312438 | Time: 224407 ms
[2022-04-09 14:22:56 main:574] : INFO : Epoch 1096 | loss: 0.031245 | val_loss: 0.0312441 | Time: 211966 ms
[2022-04-09 14:25:28 main:574] : INFO : Epoch 1097 | loss: 0.0312448 | val_loss: 0.031244 | Time: 152100 ms
[2022-04-09 14:27:36 main:574] : INFO : Epoch 1098 | loss: 0.0312452 | val_loss: 0.031244 | Time: 122314 ms
[2022-04-09 14:31:30 main:574] : INFO : Epoch 1099 | loss: 0.0312451 | val_loss: 0.031244 | Time: 229204 ms
[2022-04-09 14:34:38 main:574] : INFO : Epoch 1100 | loss: 0.0312451 | val_loss: 0.0312443 | Time: 182848 ms
[2022-04-09 14:35:40 main:574] : INFO : Epoch 1101 | loss: 0.0312448 | val_loss: 0.0312444 | Time: 62216.9 ms
[2022-04-09 14:39:51 main:574] : INFO : Epoch 1102 | loss: 0.0312451 | val_loss: 0.0312438 | Time: 251272 ms
[2022-04-09 14:42:46 main:574] : INFO : Epoch 1103 | loss: 0.0312456 | val_loss: 0.0312438 | Time: 174526 ms
[2022-04-09 14:44:43 main:574] : INFO : Epoch 1104 | loss: 0.031245 | val_loss: 0.0312442 | Time: 117574 ms
[2022-04-09 14:47:54 main:574] : INFO : Epoch 1105 | loss: 0.0312451 | val_loss: 0.031244 | Time: 185716 ms
[2022-04-09 14:50:53 main:574] : INFO : Epoch 1106 | loss: 0.0312448 | val_loss: 0.0312441 | Time: 173788 ms
[2022-04-09 14:53:56 main:574] : INFO : Epoch 1107 | loss: 0.0312447 | val_loss: 0.0312437 | Time: 182634 ms
[2022-04-09 14:55:47 main:574] : INFO : Epoch 1108 | loss: 0.0312448 | val_loss: 0.0312438 | Time: 111893 ms
[2022-04-09 14:58:50 main:574] : INFO : Epoch 1109 | loss: 0.0312453 | val_loss: 0.0312451 | Time: 182981 ms
[2022-04-09 15:02:46 main:574] : INFO : Epoch 1110 | loss: 0.0312456 | val_loss: 0.0312443 | Time: 235132 ms
[2022-04-09 15:05:36 main:574] : INFO : Epoch 1111 | loss: 0.0312447 | val_loss: 0.0312441 | Time: 161134 ms
[2022-04-09 15:07:31 main:574] : INFO : Epoch 1112 | loss: 0.0312449 | val_loss: 0.0312438 | Time: 114378 ms
[2022-04-09 15:10:19 main:574] : INFO : Epoch 1113 | loss: 0.0312452 | val_loss: 0.0312446 | Time: 164030 ms
[2022-04-09 15:12:43 main:574] : INFO : Epoch 1114 | loss: 0.031245 | val_loss: 0.031244 | Time: 137447 ms
[2022-04-09 15:14:59 main:574] : INFO : Epoch 1115 | loss: 0.0312447 | val_loss: 0.0312437 | Time: 136856 ms
[2022-04-09 15:18:19 main:574] : INFO : Epoch 1116 | loss: 0.0312452 | val_loss: 0.0312444 | Time: 199514 ms
[2022-04-09 15:18:57 main:574] : INFO : Epoch 1117 | loss: 0.031245 | val_loss: 0.0312442 | Time: 32595.3 ms
[2022-04-09 15:19:24 main:574] : INFO : Epoch 1118 | loss: 0.0312451 | val_loss: 0.0312448 | Time: 25550.2 ms
[2022-04-09 15:19:30 main:574] : INFO : Epoch 1119 | loss: 0.0312463 | val_loss: 0.0312448 | Time: 5821 ms
[2022-04-09 15:19:35 main:574] : INFO : Epoch 1120 | loss: 0.0312452 | val_loss: 0.0312439 | Time: 5427.17 ms
[2022-04-09 15:21:30 main:574] : INFO : Epoch 1121 | loss: 0.0312448 | val_loss: 0.0312441 | Time: 113781 ms
[2022-04-09 15:21:37 main:574] : INFO : Epoch 1122 | loss: 0.0312458 | val_loss: 0.0312446 | Time: 6859.58 ms
[2022-04-09 15:21:43 main:574] : INFO : Epoch 1123 | loss: 0.0312447 | val_loss: 0.0312449 | Time: 6478.68 ms
[2022-04-09 15:22:48 main:574] : INFO : Epoch 1124 | loss: 0.0312452 | val_loss: 0.031245 | Time: 63219.2 ms
[2022-04-09 15:23:18 main:574] : INFO : Epoch 1125 | loss: 0.0312454 | val_loss: 0.0312451 | Time: 29570.5 ms
[2022-04-09 15:23:54 main:574] : INFO : Epoch 1126 | loss: 0.0312451 | val_loss: 0.0312442 | Time: 36358.8 ms
[2022-04-09 15:25:46 main:574] : INFO : Epoch 1127 | loss: 0.0312449 | val_loss: 0.0312442 | Time: 111660 ms
[2022-04-09 15:28:39 main:574] : INFO : Epoch 1128 | loss: 0.0312447 | val_loss: 0.0312437 | Time: 172561 ms
[2022-04-09 15:32:20 main:574] : INFO : Epoch 1129 | loss: 0.031245 | val_loss: 0.0312442 | Time: 216135 ms
[2022-04-09 15:32:57 main:574] : INFO : Epoch 1130 | loss: 0.031245 | val_loss: 0.0312445 | Time: 32349.2 ms
[2022-04-09 15:34:05 main:574] : INFO : Epoch 1131 | loss: 0.031245 | val_loss: 0.0312444 | Time: 68296.6 ms
[2022-04-09 15:34:11 main:574] : INFO : Epoch 1132 | loss: 0.0312447 | val_loss: 0.031244 | Time: 5515.65 ms
[2022-04-09 15:37:14 main:574] : INFO : Epoch 1133 | loss: 0.0312446 | val_loss: 0.0312445 | Time: 181304 ms
[2022-04-09 15:37:44 main:574] : INFO : Epoch 1134 | loss: 0.0312452 | val_loss: 0.0312459 | Time: 24307.2 ms
[2022-04-09 15:38:26 main:574] : INFO : Epoch 1135 | loss: 0.0312471 | val_loss: 0.0312451 | Time: 41692.2 ms
[2022-04-09 15:41:43 main:574] : INFO : Epoch 1136 | loss: 0.0312452 | val_loss: 0.031244 | Time: 196852 ms
[2022-04-09 15:44:58 main:574] : INFO : Epoch 1137 | loss: 0.031245 | val_loss: 0.0312444 | Time: 194729 ms
[2022-04-09 15:47:46 main:574] : INFO : Epoch 1138 | loss: 0.0312454 | val_loss: 0.0312441 | Time: 167750 ms
[2022-04-09 15:50:34 main:574] : INFO : Epoch 1139 | loss: 0.0312456 | val_loss: 0.0312447 | Time: 162665 ms
[2022-04-09 15:51:30 main:574] : INFO : Epoch 1140 | loss: 0.0312454 | val_loss: 0.0312443 | Time: 56358.6 ms
[2022-04-09 15:53:53 main:574] : INFO : Epoch 1141 | loss: 0.031245 | val_loss: 0.0312439 | Time: 138243 ms
[2022-04-09 15:55:45 main:574] : INFO : Epoch 1142 | loss: 0.031245 | val_loss: 0.0312441 | Time: 106016 ms
[2022-04-09 15:58:44 main:574] : INFO : Epoch 1143 | loss: 0.0312459 | val_loss: 0.0312438 | Time: 178956 ms
[2022-04-09 16:02:18 main:574] : INFO : Epoch 1144 | loss: 0.0312455 | val_loss: 0.0312441 | Time: 213743 ms
[2022-04-09 16:04:39 main:574] : INFO : Epoch 1145 | loss: 0.0312449 | val_loss: 0.0312443 | Time: 141436 ms
[2022-04-09 16:05:58 main:574] : INFO : Epoch 1146 | loss: 0.0312449 | val_loss: 0.031246 | Time: 73651.9 ms
[2022-04-09 16:09:51 main:574] : INFO : Epoch 1147 | loss: 0.031245 | val_loss: 0.0312447 | Time: 228025 ms
[2022-04-09 16:12:18 main:574] : INFO : Epoch 1148 | loss: 0.031245 | val_loss: 0.0312444 | Time: 142460 ms
[2022-04-09 16:12:52 main:574] : INFO : Epoch 1149 | loss: 0.0312449 | val_loss: 0.0312441 | Time: 28981.2 ms
[2022-04-09 16:16:31 main:574] : INFO : Epoch 1150 | loss: 0.0312448 | val_loss: 0.0312441 | Time: 213787 ms
[2022-04-09 16:19:18 main:574] : INFO : Epoch 1151 | loss: 0.0312448 | val_loss: 0.0312439 | Time: 161494 ms
[2022-04-09 16:20:58 main:574] : INFO : Epoch 1152 | loss: 0.031245 | val_loss: 0.0312447 | Time: 100687 ms
[2022-04-09 16:23:31 main:574] : INFO : Epoch 1153 | loss: 0.0312455 | val_loss: 0.0312438 | Time: 147211 ms
[2022-04-09 16:25:52 main:574] : INFO : Epoch 1154 | loss: 0.0312447 | val_loss: 0.0312461 | Time: 135709 ms
[2022-04-09 16:27:58 main:574] : INFO : Epoch 1155 | loss: 0.0312454 | val_loss: 0.0312439 | Time: 126708 ms
[2022-04-09 16:31:40 main:574] : INFO : Epoch 1156 | loss: 0.0312451 | val_loss: 0.0312443 | Time: 222094 ms
[2022-04-09 16:35:15 main:574] : INFO : Epoch 1157 | loss: 0.0312451 | val_loss: 0.0312447 | Time: 209015 ms
[2022-04-09 16:37:35 main:574] : INFO : Epoch 1158 | loss: 0.0312453 | val_loss: 0.0312445 | Time: 134789 ms
[2022-04-09 16:41:22 main:574] : INFO : Epoch 1159 | loss: 0.0312452 | val_loss: 0.0312445 | Time: 217295 ms
[2022-04-09 16:43:58 main:574] : INFO : Epoch 1160 | loss: 0.0312454 | val_loss: 0.0312444 | Time: 156218 ms
[2022-04-09 16:47:31 main:574] : INFO : Epoch 1161 | loss: 0.0312462 | val_loss: 0.0312478 | Time: 207825 ms
[2022-04-09 16:50:54 main:574] : INFO : Epoch 1162 | loss: 0.0312467 | val_loss: 0.0312437 | Time: 193938 ms
[2022-04-09 16:55:09 main:574] : INFO : Epoch 1163 | loss: 0.0312454 | val_loss: 0.0312442 | Time: 254915 ms
[2022-04-09 16:55:52 main:574] : INFO : Epoch 1164 | loss: 0.031245 | val_loss: 0.0312443 | Time: 42124.1 ms
[2022-04-09 16:58:18 main:574] : INFO : Epoch 1165 | loss: 0.0312453 | val_loss: 0.031244 | Time: 141081 ms
[2022-04-09 17:01:42 main:574] : INFO : Epoch 1166 | loss: 0.0312451 | val_loss: 0.0312444 | Time: 199515 ms
[2022-04-09 17:04:51 main:574] : INFO : Epoch 1167 | loss: 0.0312453 | val_loss: 0.0312459 | Time: 184040 ms
[2022-04-09 17:06:54 main:574] : INFO : Epoch 1168 | loss: 0.0312462 | val_loss: 0.0312446 | Time: 122331 ms
[2022-04-09 17:09:47 main:574] : INFO : Epoch 1169 | loss: 0.0312453 | val_loss: 0.0312453 | Time: 168831 ms
[2022-04-09 17:12:27 main:574] : INFO : Epoch 1170 | loss: 0.0312449 | val_loss: 0.0312442 | Time: 154382 ms
[2022-04-09 17:14:43 main:574] : INFO : Epoch 1171 | loss: 0.0312448 | val_loss: 0.0312443 | Time: 131238 ms
[2022-04-09 17:17:16 main:574] : INFO : Epoch 1172 | loss: 0.031245 | val_loss: 0.0312438 | Time: 148077 ms
[2022-04-09 17:20:52 main:574] : INFO : Epoch 1173 | loss: 0.0312459 | val_loss: 0.0312486 | Time: 210877 ms
[2022-04-09 17:23:55 main:574] : INFO : Epoch 1174 | loss: 0.0312454 | val_loss: 0.0312441 | Time: 182700 ms
[2022-04-09 17:25:59 main:574] : INFO : Epoch 1175 | loss: 0.0312453 | val_loss: 0.0312446 | Time: 123928 ms
[2022-04-09 17:29:07 main:574] : INFO : Epoch 1176 | loss: 0.0312448 | val_loss: 0.0312441 | Time: 183509 ms
[2022-04-09 17:31:56 main:574] : INFO : Epoch 1177 | loss: 0.0312457 | val_loss: 0.0312455 | Time: 163886 ms
[2022-04-09 17:35:45 main:574] : INFO : Epoch 1178 | loss: 0.031246 | val_loss: 0.0312445 | Time: 224255 ms
[2022-04-09 17:39:03 main:574] : INFO : Epoch 1179 | loss: 0.031245 | val_loss: 0.031244 | Time: 197294 ms
[2022-04-09 17:42:52 main:574] : INFO : Epoch 1180 | loss: 0.0312454 | val_loss: 0.0312439 | Time: 224443 ms
[2022-04-09 17:42:55 main:574] : INFO : Epoch 1181 | loss: 0.0312457 | val_loss: 0.0312453 | Time: 3574.2 ms
[2022-04-09 17:47:18 main:574] : INFO : Epoch 1182 | loss: 0.0312452 | val_loss: 0.0312444 | Time: 262376 ms
[2022-04-09 17:50:31 main:574] : INFO : Epoch 1183 | loss: 0.0312446 | val_loss: 0.0312446 | Time: 186428 ms
[2022-04-09 17:51:38 main:574] : INFO : Epoch 1184 | loss: 0.031245 | val_loss: 0.0312463 | Time: 66822.9 ms
[2022-04-09 17:52:48 main:574] : INFO : Epoch 1185 | loss: 0.0312462 | val_loss: 0.031246 | Time: 64611.8 ms
[2022-04-09 17:55:54 main:574] : INFO : Epoch 1186 | loss: 0.0312463 | val_loss: 0.0312452 | Time: 179947 ms
[2022-04-09 17:56:49 main:574] : INFO : Epoch 1187 | loss: 0.0312461 | val_loss: 0.0312448 | Time: 55035.6 ms
[2022-04-09 17:57:05 main:574] : INFO : Epoch 1188 | loss: 0.0312454 | val_loss: 0.0312441 | Time: 15967.7 ms
[2022-04-09 17:58:42 main:574] : INFO : Epoch 1189 | loss: 0.0312452 | val_loss: 0.0312447 | Time: 96827.4 ms
[2022-04-09 17:59:45 main:574] : INFO : Epoch 1190 | loss: 0.0312456 | val_loss: 0.0312461 | Time: 58268.5 ms
[2022-04-09 17:59:51 main:574] : INFO : Epoch 1191 | loss: 0.0312452 | val_loss: 0.0312439 | Time: 6233.59 ms
[2022-04-09 18:00:34 main:574] : INFO : Epoch 1192 | loss: 0.0312449 | val_loss: 0.0312451 | Time: 40550.5 ms
[2022-04-09 18:01:08 main:574] : INFO : Epoch 1193 | loss: 0.031245 | val_loss: 0.0312448 | Time: 33601.9 ms
[2022-04-09 18:01:14 main:574] : INFO : Epoch 1194 | loss: 0.0312453 | val_loss: 0.0312454 | Time: 5459.53 ms
[2022-04-09 18:03:02 main:574] : INFO : Epoch 1195 | loss: 0.0312451 | val_loss: 0.0312439 | Time: 108357 ms
[2022-04-09 18:05:47 main:574] : INFO : Epoch 1196 | loss: 0.0312452 | val_loss: 0.0312439 | Time: 157706 ms
[2022-04-09 18:06:19 main:574] : INFO : Epoch 1197 | loss: 0.0312448 | val_loss: 0.0312453 | Time: 30321.1 ms
[2022-04-09 18:08:21 main:574] : INFO : Epoch 1198 | loss: 0.0312458 | val_loss: 0.0312442 | Time: 120898 ms
[2022-04-09 18:11:58 main:574] : INFO : Epoch 1199 | loss: 0.0312453 | val_loss: 0.0312459 | Time: 212562 ms
[2022-04-09 18:12:53 main:574] : INFO : Epoch 1200 | loss: 0.0312451 | val_loss: 0.0312439 | Time: 54979.8 ms
[2022-04-09 18:16:04 main:574] : INFO : Epoch 1201 | loss: 0.0312458 | val_loss: 0.031244 | Time: 188902 ms
[2022-04-09 18:16:10 main:574] : INFO : Epoch 1202 | loss: 0.031245 | val_loss: 0.0312453 | Time: 6000.7 ms
[2022-04-09 18:17:40 main:574] : INFO : Epoch 1203 | loss: 0.0312453 | val_loss: 0.0312457 | Time: 87822.6 ms
[2022-04-09 18:18:53 main:574] : INFO : Epoch 1204 | loss: 0.0312472 | val_loss: 0.0312454 | Time: 72489.2 ms
[2022-04-09 18:21:23 main:574] : INFO : Epoch 1205 | loss: 0.0312453 | val_loss: 0.0312471 | Time: 145525 ms
[2022-04-09 18:24:03 main:574] : INFO : Epoch 1206 | loss: 0.0312457 | val_loss: 0.0312444 | Time: 153600 ms
[2022-04-09 18:25:55 main:574] : INFO : Epoch 1207 | loss: 0.0312454 | val_loss: 0.031246 | Time: 112203 ms
[2022-04-09 18:30:06 main:574] : INFO : Epoch 1208 | loss: 0.0312461 | val_loss: 0.0312441 | Time: 251317 ms
[2022-04-09 18:33:25 main:574] : INFO : Epoch 1209 | loss: 0.0312454 | val_loss: 0.0312451 | Time: 198308 ms
[2022-04-09 18:35:44 main:574] : INFO : Epoch 1210 | loss: 0.0312456 | val_loss: 0.031244 | Time: 134878 ms
[2022-04-09 18:37:02 main:574] : INFO : Epoch 1211 | loss: 0.031245 | val_loss: 0.0312444 | Time: 78047.3 ms
[2022-04-09 18:41:06 main:574] : INFO : Epoch 1212 | loss: 0.0312452 | val_loss: 0.0312443 | Time: 235096 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 2080)
[2022-04-09 19:52:06 main:435] : INFO : Set logging level to 1
[2022-04-09 19:52:06 main:441] : INFO : Running in BOINC Client mode
[2022-04-09 19:52:06 main:444] : INFO : Resolving all filenames
[2022-04-09 19:52:06 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-04-09 19:52:06 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1)
[2022-04-09 19:52:06 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-04-09 19:52:06 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-04-09 19:52:06 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-04-09 19:52:06 main:472] : INFO : Dataset filename: dataset.hdf5
[2022-04-09 19:52:06 main:474] : INFO : Configuration:
[2022-04-09 19:52:06 main:475] : INFO : Model type: GRU
[2022-04-09 19:52:06 main:476] : INFO : Validation Loss Threshold: 0.0001
[2022-04-09 19:52:06 main:477] : INFO : Max Epochs: 2048
[2022-04-09 19:52:06 main:478] : INFO : Batch Size: 128
[2022-04-09 19:52:06 main:479] : INFO : Learning Rate: 0.01
[2022-04-09 19:52:06 main:480] : INFO : Patience: 10
[2022-04-09 19:52:06 main:481] : INFO : Hidden Width: 12
[2022-04-09 19:52:06 main:482] : INFO : # Recurrent Layers: 4
[2022-04-09 19:52:06 main:483] : INFO : # Backend Layers: 4
[2022-04-09 19:52:06 main:484] : INFO : # Threads: 1
[2022-04-09 19:52:06 main:486] : INFO : Preparing Dataset
[2022-04-09 19:52:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-04-09 19:52:06 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-04-09 19:52:08 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-04-09 19:52:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-04-09 19:52:08 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-04-09 19:52:08 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-04-09 19:52:08 main:494] : INFO : Creating Model
[2022-04-09 19:52:08 main:507] : INFO : Preparing config file
[2022-04-09 19:52:08 main:511] : INFO : Found checkpoint, attempting to load...
[2022-04-09 19:52:08 main:512] : INFO : Loading config
[2022-04-09 19:52:08 main:514] : INFO : Loading state
[2022-04-09 19:52:09 main:559] : INFO : Loading DataLoader into Memory
[2022-04-09 19:52:09 main:562] : INFO : Starting Training
[2022-04-09 19:52:11 main:574] : INFO : Epoch 1213 | loss: 0.0315584 | val_loss: 0.0314198 | Time: 1831.57 ms
[2022-04-09 19:52:13 main:574] : INFO : Epoch 1214 | loss: 0.0312907 | val_loss: 0.0312622 | Time: 1513.38 ms
[2022-04-09 19:52:14 main:574] : INFO : Epoch 1215 | loss: 0.0312564 | val_loss: 0.0312505 | Time: 1442.86 ms
[2022-04-09 19:52:16 main:574] : INFO : Epoch 1216 | loss: 0.0312499 | val_loss: 0.0312489 | Time: 1491 ms
[2022-04-09 19:52:17 main:574] : INFO : Epoch 1217 | loss: 0.031249 | val_loss: 0.0312482 | Time: 1559.15 ms
[2022-04-09 19:52:19 main:574] : INFO : Epoch 1218 | loss: 0.0312476 | val_loss: 0.0312468 | Time: 1437.54 ms
[2022-04-09 19:52:20 main:574] : INFO : Epoch 1219 | loss: 0.0312474 | val_loss: 0.0312464 | Time: 1486.37 ms
[2022-04-09 19:52:21 main:574] : INFO : Epoch 1220 | loss: 0.0312472 | val_loss: 0.0312462 | Time: 1410.66 ms
[2022-04-09 19:52:23 main:574] : INFO : Epoch 1221 | loss: 0.0312472 | val_loss: 0.0312461 | Time: 1455.56 ms
[2022-04-09 19:52:24 main:574] : INFO : Epoch 1222 | loss: 0.0312468 | val_loss: 0.0312459 | Time: 1385.46 ms
[2022-04-09 19:52:26 main:574] : INFO : Epoch 1223 | loss: 0.0312466 | val_loss: 0.0312497 | Time: 1420.79 ms
[2022-04-09 19:52:27 main:574] : INFO : Epoch 1224 | loss: 0.0312484 | val_loss: 0.0312466 | Time: 1599.73 ms
[2022-04-09 19:52:29 main:574] : INFO : Epoch 1225 | loss: 0.0312474 | val_loss: 0.031245 | Time: 1426.38 ms
[2022-04-09 19:52:30 main:574] : INFO : Epoch 1226 | loss: 0.0312465 | val_loss: 0.0312451 | Time: 1425.8 ms
[2022-04-09 19:52:32 main:574] : INFO : Epoch 1227 | loss: 0.0312463 | val_loss: 0.0312444 | Time: 1452.56 ms
[2022-04-09 19:52:33 main:574] : INFO : Epoch 1228 | loss: 0.0312461 | val_loss: 0.0312442 | Time: 1455.99 ms
[2022-04-09 19:52:35 main:574] : INFO : Epoch 1229 | loss: 0.0312461 | val_loss: 0.031244 | Time: 1418.13 ms
[2022-04-09 19:52:36 main:574] : INFO : Epoch 1230 | loss: 0.0312458 | val_loss: 0.0312444 | Time: 1421.82 ms
[2022-04-09 19:52:37 main:574] : INFO : Epoch 1231 | loss: 0.0312464 | val_loss: 0.0312439 | Time: 1411.55 ms
[2022-04-09 19:52:39 main:574] : INFO : Epoch 1232 | loss: 0.0312458 | val_loss: 0.0312436 | Time: 1412.15 ms
[2022-04-09 19:52:40 main:574] : INFO : Epoch 1233 | loss: 0.031246 | val_loss: 0.0312443 | Time: 1357.03 ms
[2022-04-09 19:52:41 main:574] : INFO : Epoch 1234 | loss: 0.0312461 | val_loss: 0.0312435 | Time: 1360.18 ms
[2022-04-09 19:52:43 main:574] : INFO : Epoch 1235 | loss: 0.0312454 | val_loss: 0.0312433 | Time: 1427.78 ms
[2022-04-09 19:52:44 main:574] : INFO : Epoch 1236 | loss: 0.0312454 | val_loss: 0.0312435 | Time: 1330.79 ms
[2022-04-09 19:52:46 main:574] : INFO : Epoch 1237 | loss: 0.0312456 | val_loss: 0.0312438 | Time: 1456.87 ms
[2022-04-09 19:52:47 main:574] : INFO : Epoch 1238 | loss: 0.0312465 | val_loss: 0.031244 | Time: 1379.87 ms
[2022-04-09 19:52:49 main:574] : INFO : Epoch 1239 | loss: 0.0312459 | val_loss: 0.0312433 | Time: 1402.53 ms
[2022-04-09 19:52:50 main:574] : INFO : Epoch 1240 | loss: 0.0312459 | val_loss: 0.0312443 | Time: 1382.99 ms
[2022-04-09 19:52:51 main:574] : INFO : Epoch 1241 | loss: 0.0312456 | val_loss: 0.0312432 | Time: 1405.83 ms
[2022-04-09 19:52:53 main:574] : INFO : Epoch 1242 | loss: 0.0312458 | val_loss: 0.0312434 | Time: 1404.57 ms
[2022-04-09 19:52:54 main:574] : INFO : Epoch 1243 | loss: 0.0312454 | val_loss: 0.0312436 | Time: 1425.31 ms
[2022-04-09 19:52:56 main:574] : INFO : Epoch 1244 | loss: 0.0312462 | val_loss: 0.0312438 | Time: 1414.28 ms
[2022-04-09 19:52:57 main:574] : INFO : Epoch 1245 | loss: 0.031246 | val_loss: 0.0312457 | Time: 1406.91 ms
[2022-04-09 19:52:58 main:574] : INFO : Epoch 1246 | loss: 0.0312458 | val_loss: 0.0312444 | Time: 1388.95 ms
[2022-04-09 19:53:00 main:574] : INFO : Epoch 1247 | loss: 0.0312455 | val_loss: 0.0312437 | Time: 1382.27 ms
[2022-04-09 19:53:01 main:574] : INFO : Epoch 1248 | loss: 0.0312452 | val_loss: 0.0312436 | Time: 1455.23 ms
[2022-04-09 19:53:03 main:574] : INFO : Epoch 1249 | loss: 0.0312456 | val_loss: 0.0312433 | Time: 1403.7 ms
[2022-04-09 19:53:04 main:574] : INFO : Epoch 1250 | loss: 0.0312459 | val_loss: 0.0312463 | Time: 1418.63 ms
[2022-04-09 19:53:05 main:574] : INFO : Epoch 1251 | loss: 0.0312458 | val_loss: 0.0312434 | Time: 1421.82 ms
[2022-04-09 19:53:07 main:574] : INFO : Epoch 1252 | loss: 0.0312458 | val_loss: 0.0312466 | Time: 1428.16 ms
[2022-04-09 19:53:08 main:574] : INFO : Epoch 1253 | loss: 0.0312462 | val_loss: 0.0312447 | Time: 1366 ms
[2022-04-09 19:53:10 main:574] : INFO : Epoch 1254 | loss: 0.0312462 | val_loss: 0.0312433 | Time: 1404.93 ms
[2022-04-09 19:54:08 main:574] : INFO : Epoch 1255 | loss: 0.031246 | val_loss: 0.0312441 | Time: 58675.2 ms
[2022-04-09 19:58:09 main:574] : INFO : Epoch 1256 | loss: 0.0312456 | val_loss: 0.0312431 | Time: 240671 ms
[2022-04-09 20:01:55 main:574] : INFO : Epoch 1257 | loss: 0.0312463 | val_loss: 0.0312436 | Time: 220860 ms
[2022-04-09 20:04:43 main:574] : INFO : Epoch 1258 | loss: 0.0312453 | val_loss: 0.0312437 | Time: 168044 ms
[2022-04-09 20:07:01 main:574] : INFO : Epoch 1259 | loss: 0.0312455 | val_loss: 0.0312447 | Time: 128424 ms
[2022-04-09 20:11:20 main:574] : INFO : Epoch 1260 | loss: 0.0312462 | val_loss: 0.0312436 | Time: 254097 ms
[2022-04-09 20:12:56 main:574] : INFO : Epoch 1261 | loss: 0.0312465 | val_loss: 0.0312436 | Time: 95167 ms
[2022-04-09 20:16:46 main:574] : INFO : Epoch 1262 | loss: 0.0312465 | val_loss: 0.0312434 | Time: 230211 ms
[2022-04-09 20:20:31 main:574] : INFO : Epoch 1263 | loss: 0.0312453 | val_loss: 0.0312442 | Time: 220894 ms
[2022-04-09 20:22:20 main:574] : INFO : Epoch 1264 | loss: 0.0312458 | val_loss: 0.0312431 | Time: 103287 ms
[2022-04-09 20:25:57 main:574] : INFO : Epoch 1265 | loss: 0.0312453 | val_loss: 0.0312432 | Time: 217213 ms
[2022-04-09 20:29:05 main:574] : INFO : Epoch 1266 | loss: 0.0312453 | val_loss: 0.0312431 | Time: 188310 ms
[2022-04-09 20:33:13 main:574] : INFO : Epoch 1267 | loss: 0.031245 | val_loss: 0.0312433 | Time: 243455 ms
[2022-04-09 20:36:47 main:574] : INFO : Epoch 1268 | loss: 0.031245 | val_loss: 0.0312434 | Time: 209007 ms
[2022-04-09 20:40:37 main:574] : INFO : Epoch 1269 | loss: 0.0312464 | val_loss: 0.031247 | Time: 225596 ms
[2022-04-09 20:42:53 main:574] : INFO : Epoch 1270 | loss: 0.0312466 | val_loss: 0.0312458 | Time: 135191 ms
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 04/09/22 20:45:53
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\0;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 000000002c280000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000294d0000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000028900000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000027260000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000283d0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000026c90000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1586 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1586
ModLoad: 0000000027740000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 0000000026f90000 000000000010b000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026bf0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000270a0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 0000000003490000 0000000000058000 C:\ProgramData\BOINC\slots\0\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000028850000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000293e0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 0000000027770000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027870000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000007c40000 0000000000092000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000001a230000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 000000000cd10000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 000000000d570000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.28.29334.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\6\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.28.29334.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.28.29334.0
ModLoad: 00000000aad10000 000000000abe7000 C:\ProgramData\BOINC\slots\0\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000e8bb0000 000000000020d000 C:\ProgramData\BOINC\slots\0\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 00000000e36b0000 00000000002db000 C:\ProgramData\BOINC\slots\0\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000004b0000 0000000000040000 C:\ProgramData\BOINC\slots\0\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000007af60000 0000000024b49000 C:\ProgramData\BOINC\slots\0\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000000460000 0000000000045000 C:\ProgramData\BOINC\slots\0\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000ba000000 00000000034b2000 C:\ProgramData\BOINC\slots\0\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000a7c60000 00000000030a5000 C:\ProgramData\BOINC\slots\0\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 0000000019a10000 0000000000010000 C:\ProgramData\BOINC\slots\0\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000056a10000 000000000919f000 C:\ProgramData\BOINC\slots\0\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 000000004a960000 0000000003d5b000 C:\ProgramData\BOINC\slots\0\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000b71f0000 0000000001eca000 C:\ProgramData\BOINC\slots\0\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 000000005fbb0000 000000001b3a4000 C:\ProgramData\BOINC\slots\0\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 0000000028710000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000024a40000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000289c0000 0000000000470000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000271a0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 0000000026cc0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000269b0000 0000000000033000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000271f0000 0000000000068000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 0000000026d80000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000267d0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000027a50000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 0000000024d20000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000028e30000 0000000000354000 C:\WINDOWS\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 0000000026650000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 0000000027690000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000275a0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000e01e0000 00000000002cd000 C:\WINDOWS\system32\nvcuda.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000492c0000 00000000016a0000 C:\WINDOWS\system32\DriverStore\FileRepository\nvhdc.inf_amd64_d11413f6fd8fe259\nvcuda64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.99 driver
Product Version : 30.0.15.1165
ModLoad: 00000000216e0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000001dc70000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 0000000018c60000 0000000000148000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1566 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1566
ModLoad: 00000000265c0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 0000000052540000 0000000000768000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.15.1165) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_37\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1165
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1165
ModLoad: 00000000140b0000 0000000000009000 C:\ProgramData\BOINC\slots\0\caffe2_nvrtc.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000055a40000 0000000000fca000 C:\ProgramData\BOINC\slots\0\nvrtc64_102_0.dll (6.14.11.9000) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6.14.11.9000
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.89 NVRTC Library
Product Version : 6.14.11.9000
ModLoad: 0000000026cf0000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3703, Write: 954, Other 2268
- I/O Transfers Counters -
Read: 7059179, Write: 1724690, Other 34922
- Paged Pool Usage -
QuotaPagedPoolUsage: 3533320, QuotaPeakPagedPoolUsage: 3533320
QuotaNonPagedPoolUsage: 58336, QuotaPeakNonPagedPoolUsage: 58808
- Virtual Memory Usage -
VirtualSize: 395407360, PeakVirtualSize: 1017380864
- Pagefile Usage -
PagefileUsage: 395407360, PeakPagefileUsage: 397500416
- Working Set Size -
WorkingSetSize: -2124111872, PeakWorkingSetSize: -2056105984, PageFaultCount: 839779
*** Dump of thread ID 4744 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174659520.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFF273290D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=000000002c538c40 rdx=0000000059efea20 rsi=0000000000000000 rdi=0000000000000000
r8=0000000059efea20 r9=000000002c538c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000273290d2 rsp=0000000059efe9f8 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
59efe9f0 2c2b9100 00000001 59efea20 59efea20 2c538c30 KERNELBASE!DebugBreak+0x0
59efee30 2c2b9ce4 cd800000 59eff194 00000000 00000318 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
59effd90 2c2ba084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
59effdc0 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
59effdf0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
59effe70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8208 (state: Ready): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 30625000.000000, User Time: 710625024.000000, Wait Time: 174659472.000000
- Registers -
rax=0000000000000000 rbx=000000000000011d rcx=00000000067c0000 rdx=000000006e896fb0 rsi=0000000000000001 rdi=0000000000000001
r8=000000006e8970c8 r9=000000006e8970a0 r10=0000000000000000 r11=00000000597e1b90 r12=000000000000011c r13=00000000cd3c6810
r14=000000000000011d r15=00000000597e2110 rip=00000000608ca43e rsp=00000000597e1bb0 rbp=00000000597e1d30
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000202
- Callstack -
ChildEBP RetAddr Args to Child
597e2050 608beff1 000006c0 6434f6c0 597e4318 cd3c6810 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
597e21f0 608c7d8e cfdc4300 00000000 cd3c6810 597e2498 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
597e2390 608dd24a 597e2498 597e2498 597e2498 597e2498 cudnn64_7!cudnnSetRNNProjectionLayers+0x0
597e4500 7dcd8bd1 cfdc2100 3c200000 cfdc2100 6434f6c0 cudnn64_7!cudnnRNNForwardInference+0x0
597e49e0 7dd325ea 597e4aa8 00000001 597e4aa8 00000001 torch_cuda!at::native::_cudnn_rnn+0x0
597e4ac0 7dd40cbf 597e51b0 597e69a8 597e5b00 00000001 torch_cuda!at::native::set_storage_cuda_+0x0
597e4ba0 b075b9fc 597e51b0 597e5890 00000000 597e4cb8 torch_cuda!at::native::set_storage_cuda_+0x0
597e4d20 b07931a2 597e50b0 00000001 00000000 00000000 torch_cpu!at::native::mkldnn_sigmoid_+0x0
597e5000 b1a42613 00000000 597e5b10 597e5b10 597e5110 torch_cpu!at::_cudnn_rnn+0x0
597e5500 b1a5080b 5715e69d ce9df4e0 597e5630 00000004 torch_cpu!torch::autograd::NotImplemented::~NotImplemented+0x0
597e55e0 b075b9fc 597e5b60 b1b75878 597e5740 492e7a77 torch_cpu!torch::autograd::GraphRoot::apply+0x0
597e5760 b07931a2 cd3c8800 00000000 5715e0ed 7ebfa3f0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
597e5a40 7dcd69a1 597e0001 00000001 00000004 00000004 torch_cpu!at::_cudnn_rnn+0x0
597e5c10 7dce0117 00000000 597e69a8 00462e0d 597e5d30 torch_cuda!at::native::_use_cudnn_ctc_loss+0x0
597e5ca0 b04e107b b5631ac0 597e5fc0 cb42bbff 494b7783 torch_cuda!at::native::_cudnn_rnn_flatten_weight+0x0
597e5f10 b0513b16 00000001 597e69a8 00000080 d03d7e68 torch_cpu!at::native::fbgemm_pack_quantized_matrix+0x0
597e60a0 b088ae89 597e6118 00000001 597e6118 00000001 torch_cpu!at::native::gru+0x0
597e6130 b1b79fa3 597e6230 00000001 597e63b0 597e69a8 torch_cpu!at::zeros_out+0x0
597e61c0 b087dd35 034e1ea8 0450cbb0 00000000 034e1ea8 torch_cpu!torch::autograd::GraphRoot::apply+0x0
597e6240 b07588ec b56375c8 64434900 ceaa7460 597e6460 torch_cpu!at::zeros_out+0x0
597e6340 b07cc1bb 0000000c 00000080 00000004 ce9dff00 torch_cpu!at::native::mkldnn_sigmoid_+0x0
597e63c0 b22b7294 597e64e0 64434900 597e64d0 597e6678 torch_cpu!at::gru+0x0
597e65f0 b22b62ab 597e6988 b06e030e 597e6988 597e69a8 torch_cpu!torch::nn::GRUImpl::forward_helper+0x0
597e66e0 2c292621 00000002 cf543820 00000000 034e1ea8 torch_cpu!torch::nn::GRUImpl::forward+0x0
597e68e0 2c297e98 597e6938 597e0000 cf540601 cf543820 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::error+0x0
597e6d80 2c29b964 2c2cd95f 2c2cd95f 00000000 60c0bb70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Object::unsafeRemoveSlot+0x0
597ffb20 2c4dc6f8 0497cba0 00000000 00000000 04993c00 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
597ffb60 28917034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
597ffb90 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
597ffc10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 16804 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174659520.000000
- Registers -
rax=000000000000005b rbx=000000000000000e rcx=000000000000000e rdx=0000000059fffb00 rsi=0000000000000000 rdi=000000000000000e
r8=0000000000000001 r9=0000000004bad900 r10=0000000000000000 r11=0000000059fffa70 r12=0000000000000064 r13=0000000059fffb00
r14=0000000059fff800 r15=0000000000000000 rip=000000002956d8c4 rsp=0000000059fff7a8 rbp=0000000004bad900
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
59fff7a0 272acb20 00000000 5983c000 2963a4d0 5983c000 ntdll!ZwWaitForMultipleObjects+0x0
59fffa90 272aca1e 02f70850 004e0049 0000000e 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
59fffad0 4949d554 04bb2a60 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
59fffd10 495458cd 00000000 04bad900 00000000 00000000 nvcuda64!+0x0
59fffdc0 4949cf23 16454ffc 90c8ee8c 051375e0 00000000 nvcuda64!cuProfilerStop+0x0
59fffdf0 497d44c4 051375e0 00000000 00000000 00000000 nvcuda64!+0x0
59fffe20 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
59fffe50 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
59fffed0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 8688 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174453056.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=0000000000000084 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000084
r8=000000005a0ffba8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000004b93d60 r13=0000000000000000
r14=0000000000000084 r15=0000000000000000 rip=000000002956cdf4 rsp=000000005a0ff928 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5a0ff920 27281a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
5a0ff9c0 496c5d0e 05137550 00000000 00000000 00000084 KERNELBASE!WaitForSingleObjectEx+0x0
5a0ffc00 4949cf23 00000000 05137550 492c0000 04bcaeb0 nvcuda64!cuProfilerStop+0x0
5a0ffc30 497d44c4 05137550 00000000 00000000 00000000 nvcuda64!+0x0
5a0ffc60 28917034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
5a0ffc90 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5a0ffd10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 18724 (state: Waiting): ***
- Information -
Status: Wait Reason: Unknown, , Kernel Time: 20000000.000000, User Time: 476875008.000000, Wait Time: 174657440.000000
- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000a6185af8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000a6185af8
r8=000000005a1ff0c4 r9=0000000000000001 r10=0000000000008000 r11=000000005a1fedb0 r12=0000000000000000 r13=0000000000000000
r14=000000005a1ff390 r15=00000000a6185aa8 rip=0000000029570764 rsp=000000005a1ff368 rbp=000000005a1ff3c0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
5a1ff360 29534021 bcbf8d08 07c52f97 bcd18e10 bcd18e00 ntdll!NtWaitForAlertByThreadId+0x0
5a1ff3e0 272ccfa9 a6185af0 a6185ae8 a6185aa0 00000000 ntdll!RtlSleepConditionVariableSRW+0x0
5a1ff420 07c7bf3d bcd18e30 00000000 00000000 b1ef05b4 KERNELBASE!SleepConditionVariableSRW+0x0
5a1ff460 07c52798 00000000 bcd18e30 00000000 00000000 MSVCP140!__crtSetThreadpoolWait+0x0
5a1ff490 07c52a4a 6f8d5a39 00000000 5a1ff6a0 a6185ae8 MSVCP140!std::_Winerror_message+0x0
5a1ff4c0 b1ef064b a6185aa0 5a1ff5c0 a6185aa0 bcd18e10 MSVCP140!_Cnd_wait+0x0
5a1ff500 b1ef25b1 d02d85c0 ffffffff bcd18e10 ffffffff torch_cpu!torch::utils::Future<std::vector<at::Tensor,std::allocator<at::Tensor> > >::markCompletedInternal+0x0
5a1ff7f0 b1ef2471 b5416d10 5a1ff830 b5416d10 00000001 torch_cpu!torch::autograd::Engine::thread_main+0x0
5a1ff840 b1ee9744 00080001 cfbaa9e0 00000000 00000000 torch_cpu!torch::autograd::Engine::thread_init+0x0
5a1ff870 270c1bb2 cfbaa9e0 00000000 00000000 00000000 torch_cpu!torch::autograd::Engine::get_base_engine+0x0
5a1ff8a0 28917034 00000000 00000000 00000000 00000000 ucrtbase!_configthreadlocale+0x0
5a1ff8d0 29522651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
5a1ff950 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)