Task 4152886

Name ParityModified-1612595701-11795-1_2
Workunit 2133421
Created 22 Feb 2021, 18:28:55 UTC
Sent 22 Feb 2021, 21:03:24 UTC
Report deadline 1 Mar 2021, 21:03:24 UTC
Received 23 Feb 2021, 7:21:50 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 7171
Run time 30 min 2 sec
CPU time 26 min 50 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 2,569.86 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 1.55 GB
Peak swap size 3.47 GB
Peak disk usage 1.53 GB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -529697949 (0xe06d7363)</message>
<stderr_txt>
2021-02-22 22:53:23	                main:574]	:	INFO	:	Epoch 392 | loss: nan | val_loss: nan | Time: 2574.31 ms
[2021-02-22 22:53:26	                main:574]	:	INFO	:	Epoch 393 | loss: nan | val_loss: nan | Time: 2481.52 ms
[2021-02-22 22:53:29	                main:574]	:	INFO	:	Epoch 394 | loss: nan | val_loss: nan | Time: 2531.88 ms
[2021-02-22 22:53:31	                main:574]	:	INFO	:	Epoch 395 | loss: nan | val_loss: nan | Time: 2555.69 ms
[2021-02-22 22:53:34	                main:574]	:	INFO	:	Epoch 396 | loss: nan | val_loss: nan | Time: 2644.25 ms
[2021-02-22 22:53:37	                main:574]	:	INFO	:	Epoch 397 | loss: nan | val_loss: nan | Time: 2494.18 ms
[2021-02-22 22:53:39	                main:574]	:	INFO	:	Epoch 398 | loss: nan | val_loss: nan | Time: 2688.77 ms
[2021-02-22 22:53:42	                main:574]	:	INFO	:	Epoch 399 | loss: nan | val_loss: nan | Time: 2480.62 ms
[2021-02-22 22:53:44	                main:574]	:	INFO	:	Epoch 400 | loss: nan | val_loss: nan | Time: 2363.69 ms
[2021-02-22 22:53:47	                main:574]	:	INFO	:	Epoch 401 | loss: nan | val_loss: nan | Time: 2483.29 ms
[2021-02-22 22:53:49	                main:574]	:	INFO	:	Epoch 402 | loss: nan | val_loss: nan | Time: 2469.26 ms
[2021-02-22 22:53:52	                main:574]	:	INFO	:	Epoch 403 | loss: nan | val_loss: nan | Time: 2435.12 ms
[2021-02-22 22:53:54	                main:574]	:	INFO	:	Epoch 404 | loss: nan | val_loss: nan | Time: 2520.87 ms
[2021-02-22 22:53:57	                main:574]	:	INFO	:	Epoch 405 | loss: nan | val_loss: nan | Time: 2607.08 ms
[2021-02-22 22:54:00	                main:574]	:	INFO	:	Epoch 406 | loss: nan | val_loss: nan | Time: 2639.27 ms
[2021-02-22 22:54:02	                main:574]	:	INFO	:	Epoch 407 | loss: nan | val_loss: nan | Time: 2569.05 ms
[2021-02-22 22:54:05	                main:574]	:	INFO	:	Epoch 408 | loss: nan | val_loss: nan | Time: 2640.85 ms
[2021-02-22 22:54:08	                main:574]	:	INFO	:	Epoch 409 | loss: nan | val_loss: nan | Time: 2709.71 ms
[2021-02-22 22:54:10	                main:574]	:	INFO	:	Epoch 410 | loss: nan | val_loss: nan | Time: 2623.15 ms
[2021-02-22 22:54:13	                main:574]	:	INFO	:	Epoch 411 | loss: nan | val_loss: nan | Time: 2501.33 ms
[2021-02-22 22:54:16	                main:574]	:	INFO	:	Epoch 412 | loss: nan | val_loss: nan | Time: 2543.39 ms
[2021-02-22 22:54:18	                main:574]	:	INFO	:	Epoch 413 | loss: nan | val_loss: nan | Time: 2815.77 ms
[2021-02-22 22:54:21	                main:574]	:	INFO	:	Epoch 414 | loss: nan | val_loss: nan | Time: 2554.64 ms
[2021-02-22 22:54:24	                main:574]	:	INFO	:	Epoch 415 | loss: nan | val_loss: nan | Time: 2548.47 ms
[2021-02-22 22:54:26	                main:574]	:	INFO	:	Epoch 416 | loss: nan | val_loss: nan | Time: 2565.74 ms
[2021-02-22 22:54:29	                main:574]	:	INFO	:	Epoch 417 | loss: nan | val_loss: nan | Time: 2836.06 ms
[2021-02-22 22:54:32	                main:574]	:	INFO	:	Epoch 418 | loss: nan | val_loss: nan | Time: 2665.06 ms
[2021-02-22 22:54:34	                main:574]	:	INFO	:	Epoch 419 | loss: nan | val_loss: nan | Time: 2537.63 ms
[2021-02-22 22:54:37	                main:574]	:	INFO	:	Epoch 420 | loss: nan | val_loss: nan | Time: 2657.89 ms
[2021-02-22 22:54:40	                main:574]	:	INFO	:	Epoch 421 | loss: nan | val_loss: nan | Time: 2596.13 ms
[2021-02-22 22:54:42	                main:574]	:	INFO	:	Epoch 422 | loss: nan | val_loss: nan | Time: 2596.4 ms
[2021-02-22 22:54:45	                main:574]	:	INFO	:	Epoch 423 | loss: nan | val_loss: nan | Time: 2474.59 ms
[2021-02-22 22:54:48	                main:574]	:	INFO	:	Epoch 424 | loss: nan | val_loss: nan | Time: 3045.43 ms
[2021-02-22 22:54:50	                main:574]	:	INFO	:	Epoch 425 | loss: nan | val_loss: nan | Time: 2532.42 ms
[2021-02-22 22:54:53	                main:574]	:	INFO	:	Epoch 426 | loss: nan | val_loss: nan | Time: 2436.81 ms
[2021-02-22 22:54:55	                main:574]	:	INFO	:	Epoch 427 | loss: nan | val_loss: nan | Time: 2517.19 ms
[2021-02-22 22:54:58	                main:574]	:	INFO	:	Epoch 428 | loss: nan | val_loss: nan | Time: 2818.79 ms
[2021-02-22 22:55:01	                main:574]	:	INFO	:	Epoch 429 | loss: nan | val_loss: nan | Time: 2780.43 ms
[2021-02-22 22:55:04	                main:574]	:	INFO	:	Epoch 430 | loss: nan | val_loss: nan | Time: 2585.77 ms
[2021-02-22 22:55:06	                main:574]	:	INFO	:	Epoch 431 | loss: nan | val_loss: nan | Time: 2638.41 ms
[2021-02-22 22:55:09	                main:574]	:	INFO	:	Epoch 432 | loss: nan | val_loss: nan | Time: 2759.37 ms
[2021-02-22 22:55:12	                main:574]	:	INFO	:	Epoch 433 | loss: nan | val_loss: nan | Time: 2663.49 ms
[2021-02-22 22:55:14	                main:574]	:	INFO	:	Epoch 434 | loss: nan | val_loss: nan | Time: 2712.81 ms
[2021-02-22 22:55:17	                main:574]	:	INFO	:	Epoch 435 | loss: nan | val_loss: nan | Time: 3008.78 ms
[2021-02-22 22:55:20	                main:574]	:	INFO	:	Epoch 436 | loss: nan | val_loss: nan | Time: 2668.55 ms
[2021-02-22 22:55:23	                main:574]	:	INFO	:	Epoch 437 | loss: nan | val_loss: nan | Time: 2537.7 ms
[2021-02-22 22:55:25	                main:574]	:	INFO	:	Epoch 438 | loss: nan | val_loss: nan | Time: 2455.87 ms
[2021-02-22 22:55:28	                main:574]	:	INFO	:	Epoch 439 | loss: nan | val_loss: nan | Time: 2660.4 ms
[2021-02-22 22:55:31	                main:574]	:	INFO	:	Epoch 440 | loss: nan | val_loss: nan | Time: 2670.28 ms
[2021-02-22 22:55:33	                main:574]	:	INFO	:	Epoch 441 | loss: nan | val_loss: nan | Time: 2869.29 ms
[2021-02-22 22:55:36	                main:574]	:	INFO	:	Epoch 442 | loss: nan | val_loss: nan | Time: 2617.34 ms
[2021-02-22 22:55:39	                main:574]	:	INFO	:	Epoch 443 | loss: nan | val_loss: nan | Time: 2759.52 ms
[2021-02-22 22:55:42	                main:574]	:	INFO	:	Epoch 444 | loss: nan | val_loss: nan | Time: 2618.73 ms
[2021-02-22 22:55:45	                main:574]	:	INFO	:	Epoch 445 | loss: nan | val_loss: nan | Time: 2575.93 ms
[2021-02-22 22:55:47	                main:574]	:	INFO	:	Epoch 446 | loss: nan | val_loss: nan | Time: 2722.18 ms
[2021-02-22 22:55:50	                main:574]	:	INFO	:	Epoch 447 | loss: nan | val_loss: nan | Time: 2658.72 ms
[2021-02-22 22:55:53	                main:574]	:	INFO	:	Epoch 448 | loss: nan | val_loss: nan | Time: 2551.27 ms
[2021-02-22 22:55:55	                main:574]	:	INFO	:	Epoch 449 | loss: nan | val_loss: nan | Time: 2609.47 ms
[2021-02-22 22:55:58	                main:574]	:	INFO	:	Epoch 450 | loss: nan | val_loss: nan | Time: 2699.36 ms
[2021-02-22 22:56:01	                main:574]	:	INFO	:	Epoch 451 | loss: nan | val_loss: nan | Time: 2764.8 ms
[2021-02-22 22:56:04	                main:574]	:	INFO	:	Epoch 452 | loss: nan | val_loss: nan | Time: 2985.57 ms
[2021-02-22 22:56:07	                main:574]	:	INFO	:	Epoch 453 | loss: nan | val_loss: nan | Time: 2873.27 ms
[2021-02-22 22:56:10	                main:574]	:	INFO	:	Epoch 454 | loss: nan | val_loss: nan | Time: 2938.07 ms
[2021-02-22 22:56:12	                main:574]	:	INFO	:	Epoch 455 | loss: nan | val_loss: nan | Time: 2638.98 ms
[2021-02-22 22:56:15	                main:574]	:	INFO	:	Epoch 456 | loss: nan | val_loss: nan | Time: 2506.82 ms
[2021-02-22 22:56:17	                main:574]	:	INFO	:	Epoch 457 | loss: nan | val_loss: nan | Time: 2638.9 ms
[2021-02-22 22:56:20	                main:574]	:	INFO	:	Epoch 458 | loss: nan | val_loss: nan | Time: 2677.82 ms
[2021-02-22 22:56:23	                main:574]	:	INFO	:	Epoch 459 | loss: nan | val_loss: nan | Time: 2499.57 ms
[2021-02-22 22:56:25	                main:574]	:	INFO	:	Epoch 460 | loss: nan | val_loss: nan | Time: 2537.78 ms
[2021-02-22 22:56:28	                main:574]	:	INFO	:	Epoch 461 | loss: nan | val_loss: nan | Time: 2616.06 ms
[2021-02-22 22:56:31	                main:574]	:	INFO	:	Epoch 462 | loss: nan | val_loss: nan | Time: 2723.79 ms
[2021-02-22 22:56:33	                main:574]	:	INFO	:	Epoch 463 | loss: nan | val_loss: nan | Time: 2622.48 ms
[2021-02-22 22:56:36	                main:574]	:	INFO	:	Epoch 464 | loss: nan | val_loss: nan | Time: 2682.98 ms
[2021-02-22 22:56:39	                main:574]	:	INFO	:	Epoch 465 | loss: nan | val_loss: nan | Time: 2681.74 ms
[2021-02-22 22:56:42	                main:574]	:	INFO	:	Epoch 466 | loss: nan | val_loss: nan | Time: 2756.7 ms
[2021-02-22 22:56:44	                main:574]	:	INFO	:	Epoch 467 | loss: nan | val_loss: nan | Time: 2558.85 ms
[2021-02-22 22:56:47	                main:574]	:	INFO	:	Epoch 468 | loss: nan | val_loss: nan | Time: 2594.77 ms
[2021-02-22 22:56:50	                main:574]	:	INFO	:	Epoch 469 | loss: nan | val_loss: nan | Time: 2981.99 ms
[2021-02-22 22:56:52	                main:574]	:	INFO	:	Epoch 470 | loss: nan | val_loss: nan | Time: 2623.91 ms
[2021-02-22 22:56:55	                main:574]	:	INFO	:	Epoch 471 | loss: nan | val_loss: nan | Time: 2420.88 ms
[2021-02-22 22:56:58	                main:574]	:	INFO	:	Epoch 472 | loss: nan | val_loss: nan | Time: 2763.65 ms
[2021-02-22 22:57:01	                main:574]	:	INFO	:	Epoch 473 | loss: nan | val_loss: nan | Time: 2822.53 ms
[2021-02-22 22:57:04	                main:574]	:	INFO	:	Epoch 474 | loss: nan | val_loss: nan | Time: 2639.85 ms
[2021-02-22 22:57:06	                main:574]	:	INFO	:	Epoch 475 | loss: nan | val_loss: nan | Time: 2655.73 ms
[2021-02-22 22:57:09	                main:574]	:	INFO	:	Epoch 476 | loss: nan | val_loss: nan | Time: 2798.96 ms
[2021-02-22 22:57:12	                main:574]	:	INFO	:	Epoch 477 | loss: nan | val_loss: nan | Time: 2660.42 ms
[2021-02-22 22:57:14	                main:574]	:	INFO	:	Epoch 478 | loss: nan | val_loss: nan | Time: 2534.55 ms
[2021-02-22 22:57:17	                main:574]	:	INFO	:	Epoch 479 | loss: nan | val_loss: nan | Time: 2481.43 ms
[2021-02-22 22:57:20	                main:574]	:	INFO	:	Epoch 480 | loss: nan | val_loss: nan | Time: 2728.51 ms
[2021-02-22 22:57:22	                main:574]	:	INFO	:	Epoch 481 | loss: nan | val_loss: nan | Time: 2598.83 ms
[2021-02-22 22:57:25	                main:574]	:	INFO	:	Epoch 482 | loss: nan | val_loss: nan | Time: 2364.43 ms
[2021-02-22 22:57:27	                main:574]	:	INFO	:	Epoch 483 | loss: nan | val_loss: nan | Time: 2498.02 ms
[2021-02-22 22:57:30	                main:574]	:	INFO	:	Epoch 484 | loss: nan | val_loss: nan | Time: 2800.55 ms
[2021-02-22 22:57:33	                main:574]	:	INFO	:	Epoch 485 | loss: nan | val_loss: nan | Time: 2656.13 ms
[2021-02-22 22:57:35	                main:574]	:	INFO	:	Epoch 486 | loss: nan | val_loss: nan | Time: 2559.87 ms
[2021-02-22 22:57:38	                main:574]	:	INFO	:	Epoch 487 | loss: nan | val_loss: nan | Time: 2627.31 ms
[2021-02-22 22:57:41	                main:574]	:	INFO	:	Epoch 488 | loss: nan | val_loss: nan | Time: 2809.98 ms
[2021-02-22 22:57:43	                main:574]	:	INFO	:	Epoch 489 | loss: nan | val_loss: nan | Time: 2584.42 ms
[2021-02-22 22:57:46	                main:574]	:	INFO	:	Epoch 490 | loss: nan | val_loss: nan | Time: 2455.42 ms
[2021-02-22 22:57:49	                main:574]	:	INFO	:	Epoch 491 | loss: nan | val_loss: nan | Time: 2802.88 ms
[2021-02-22 22:57:51	                main:574]	:	INFO	:	Epoch 492 | loss: nan | val_loss: nan | Time: 2549.22 ms
[2021-02-22 22:57:54	                main:574]	:	INFO	:	Epoch 493 | loss: nan | val_loss: nan | Time: 2599.12 ms
[2021-02-22 22:57:56	                main:574]	:	INFO	:	Epoch 494 | loss: nan | val_loss: nan | Time: 2409.63 ms
[2021-02-22 22:57:59	                main:574]	:	INFO	:	Epoch 495 | loss: nan | val_loss: nan | Time: 2668.12 ms
[2021-02-22 22:58:02	                main:574]	:	INFO	:	Epoch 496 | loss: nan | val_loss: nan | Time: 2721.99 ms
[2021-02-22 22:58:04	                main:574]	:	INFO	:	Epoch 497 | loss: nan | val_loss: nan | Time: 2610.34 ms
[2021-02-22 22:58:07	                main:574]	:	INFO	:	Epoch 498 | loss: nan | val_loss: nan | Time: 2581.25 ms
[2021-02-22 22:58:10	                main:574]	:	INFO	:	Epoch 499 | loss: nan | val_loss: nan | Time: 2921.18 ms
[2021-02-22 22:58:13	                main:574]	:	INFO	:	Epoch 500 | loss: nan | val_loss: nan | Time: 2640.79 ms
[2021-02-22 22:58:15	                main:574]	:	INFO	:	Epoch 501 | loss: nan | val_loss: nan | Time: 2591.58 ms
[2021-02-22 22:58:18	                main:574]	:	INFO	:	Epoch 502 | loss: nan | val_loss: nan | Time: 2569.88 ms
[2021-02-22 22:58:21	                main:574]	:	INFO	:	Epoch 503 | loss: nan | val_loss: nan | Time: 2760.09 ms
[2021-02-22 22:58:23	                main:574]	:	INFO	:	Epoch 504 | loss: nan | val_loss: nan | Time: 2738.43 ms
[2021-02-22 22:58:26	                main:574]	:	INFO	:	Epoch 505 | loss: nan | val_loss: nan | Time: 2508.13 ms
[2021-02-22 22:58:29	                main:574]	:	INFO	:	Epoch 506 | loss: nan | val_loss: nan | Time: 2596.96 ms
[2021-02-22 22:58:31	                main:574]	:	INFO	:	Epoch 507 | loss: nan | val_loss: nan | Time: 2937.8 ms
[2021-02-22 22:58:34	                main:574]	:	INFO	:	Epoch 508 | loss: nan | val_loss: nan | Time: 2630.15 ms
[2021-02-22 22:58:37	                main:574]	:	INFO	:	Epoch 509 | loss: nan | val_loss: nan | Time: 2550.27 ms
[2021-02-22 22:58:39	                main:574]	:	INFO	:	Epoch 510 | loss: nan | val_loss: nan | Time: 2744.36 ms
[2021-02-22 22:58:42	                main:574]	:	INFO	:	Epoch 511 | loss: nan | val_loss: nan | Time: 2664.33 ms
[2021-02-22 22:58:45	                main:574]	:	INFO	:	Epoch 512 | loss: nan | val_loss: nan | Time: 2534.71 ms
[2021-02-22 22:58:47	                main:574]	:	INFO	:	Epoch 513 | loss: nan | val_loss: nan | Time: 2481.18 ms
[2021-02-22 22:58:50	                main:574]	:	INFO	:	Epoch 514 | loss: nan | val_loss: nan | Time: 2675.76 ms
[2021-02-22 22:58:53	                main:574]	:	INFO	:	Epoch 515 | loss: nan | val_loss: nan | Time: 2556.32 ms
[2021-02-22 22:58:55	                main:574]	:	INFO	:	Epoch 516 | loss: nan | val_loss: nan | Time: 2369.94 ms
[2021-02-22 22:58:57	                main:574]	:	INFO	:	Epoch 517 | loss: nan | val_loss: nan | Time: 2546.62 ms
[2021-02-22 22:59:00	                main:574]	:	INFO	:	Epoch 518 | loss: nan | val_loss: nan | Time: 2653.19 ms
[2021-02-22 22:59:03	                main:574]	:	INFO	:	Epoch 519 | loss: nan | val_loss: nan | Time: 2746.81 ms
[2021-02-22 22:59:06	                main:574]	:	INFO	:	Epoch 520 | loss: nan | val_loss: nan | Time: 2615.46 ms
[2021-02-22 22:59:08	                main:574]	:	INFO	:	Epoch 521 | loss: nan | val_loss: nan | Time: 2699.42 ms
[2021-02-22 22:59:11	                main:574]	:	INFO	:	Epoch 522 | loss: nan | val_loss: nan | Time: 2796.92 ms
[2021-02-22 22:59:14	                main:574]	:	INFO	:	Epoch 523 | loss: nan | val_loss: nan | Time: 2617.49 ms
[2021-02-22 22:59:16	                main:574]	:	INFO	:	Epoch 524 | loss: nan | val_loss: nan | Time: 2548.73 ms
[2021-02-22 22:59:19	                main:574]	:	INFO	:	Epoch 525 | loss: nan | val_loss: nan | Time: 2581.48 ms
[2021-02-22 22:59:22	                main:574]	:	INFO	:	Epoch 526 | loss: nan | val_loss: nan | Time: 2641.48 ms
[2021-02-22 22:59:24	                main:574]	:	INFO	:	Epoch 527 | loss: nan | val_loss: nan | Time: 2643.06 ms
[2021-02-22 22:59:27	                main:574]	:	INFO	:	Epoch 528 | loss: nan | val_loss: nan | Time: 2752.15 ms
[2021-02-22 22:59:30	                main:574]	:	INFO	:	Epoch 529 | loss: nan | val_loss: nan | Time: 2647.63 ms
[2021-02-22 22:59:32	                main:574]	:	INFO	:	Epoch 530 | loss: nan | val_loss: nan | Time: 2662.69 ms
[2021-02-22 22:59:35	                main:574]	:	INFO	:	Epoch 531 | loss: nan | val_loss: nan | Time: 2614.74 ms
[2021-02-22 22:59:38	                main:574]	:	INFO	:	Epoch 532 | loss: nan | val_loss: nan | Time: 2633.39 ms
[2021-02-22 22:59:40	                main:574]	:	INFO	:	Epoch 533 | loss: nan | val_loss: nan | Time: 2722.67 ms
[2021-02-22 22:59:43	                main:574]	:	INFO	:	Epoch 534 | loss: nan | val_loss: nan | Time: 2754.22 ms
[2021-02-22 22:59:46	                main:574]	:	INFO	:	Epoch 535 | loss: nan | val_loss: nan | Time: 2545.71 ms
[2021-02-22 22:59:48	                main:574]	:	INFO	:	Epoch 536 | loss: nan | val_loss: nan | Time: 2617.62 ms
[2021-02-22 22:59:51	                main:574]	:	INFO	:	Epoch 537 | loss: nan | val_loss: nan | Time: 2624.05 ms
[2021-02-22 22:59:54	                main:574]	:	INFO	:	Epoch 538 | loss: nan | val_loss: nan | Time: 2595.82 ms
[2021-02-22 22:59:56	                main:574]	:	INFO	:	Epoch 539 | loss: nan | val_loss: nan | Time: 2448.79 ms
[2021-02-22 22:59:59	                main:574]	:	INFO	:	Epoch 540 | loss: nan | val_loss: nan | Time: 2521.93 ms
[2021-02-22 23:00:01	                main:574]	:	INFO	:	Epoch 541 | loss: nan | val_loss: nan | Time: 2438.32 ms
[2021-02-22 23:00:04	                main:574]	:	INFO	:	Epoch 542 | loss: nan | val_loss: nan | Time: 2746.64 ms
[2021-02-22 23:00:07	                main:574]	:	INFO	:	Epoch 543 | loss: nan | val_loss: nan | Time: 2633.97 ms
[2021-02-22 23:00:09	                main:574]	:	INFO	:	Epoch 544 | loss: nan | val_loss: nan | Time: 2712.97 ms
[2021-02-22 23:00:12	                main:574]	:	INFO	:	Epoch 545 | loss: nan | val_loss: nan | Time: 2752.43 ms
[2021-02-22 23:00:15	                main:574]	:	INFO	:	Epoch 546 | loss: nan | val_loss: nan | Time: 2614.18 ms
[2021-02-22 23:00:18	                main:574]	:	INFO	:	Epoch 547 | loss: nan | val_loss: nan | Time: 2802.07 ms
[2021-02-22 23:00:20	                main:574]	:	INFO	:	Epoch 548 | loss: nan | val_loss: nan | Time: 2691.84 ms
[2021-02-22 23:00:23	                main:574]	:	INFO	:	Epoch 549 | loss: nan | val_loss: nan | Time: 2676.19 ms
[2021-02-22 23:00:26	                main:574]	:	INFO	:	Epoch 550 | loss: nan | val_loss: nan | Time: 2456.32 ms
[2021-02-22 23:00:28	                main:574]	:	INFO	:	Epoch 551 | loss: nan | val_loss: nan | Time: 2685.72 ms
[2021-02-22 23:00:31	                main:574]	:	INFO	:	Epoch 552 | loss: nan | val_loss: nan | Time: 2650.39 ms
[2021-02-22 23:00:34	                main:574]	:	INFO	:	Epoch 553 | loss: nan | val_loss: nan | Time: 2672.57 ms
[2021-02-22 23:00:36	                main:574]	:	INFO	:	Epoch 554 | loss: nan | val_loss: nan | Time: 2759.94 ms
[2021-02-22 23:00:39	                main:574]	:	INFO	:	Epoch 555 | loss: nan | val_loss: nan | Time: 2817.75 ms
[2021-02-22 23:00:42	                main:574]	:	INFO	:	Epoch 556 | loss: nan | val_loss: nan | Time: 2679.65 ms
[2021-02-22 23:00:45	                main:574]	:	INFO	:	Epoch 557 | loss: nan | val_loss: nan | Time: 2674.85 ms
[2021-02-22 23:00:47	                main:574]	:	INFO	:	Epoch 558 | loss: nan | val_loss: nan | Time: 2730.2 ms
[2021-02-22 23:00:50	                main:574]	:	INFO	:	Epoch 559 | loss: nan | val_loss: nan | Time: 2584.79 ms
[2021-02-22 23:00:53	                main:574]	:	INFO	:	Epoch 560 | loss: nan | val_loss: nan | Time: 2563.87 ms
[2021-02-22 23:00:55	                main:574]	:	INFO	:	Epoch 561 | loss: nan | val_loss: nan | Time: 2470.59 ms
[2021-02-22 23:00:58	                main:574]	:	INFO	:	Epoch 562 | loss: nan | val_loss: nan | Time: 2749.03 ms
[2021-02-22 23:01:01	                main:574]	:	INFO	:	Epoch 563 | loss: nan | val_loss: nan | Time: 3318.04 ms
[2021-02-22 23:01:04	                main:574]	:	INFO	:	Epoch 564 | loss: nan | val_loss: nan | Time: 2815.49 ms
[2021-02-22 23:01:07	                main:574]	:	INFO	:	Epoch 565 | loss: nan | val_loss: nan | Time: 2728.29 ms
[2021-02-22 23:01:11	                main:574]	:	INFO	:	Epoch 566 | loss: nan | val_loss: nan | Time: 2868.03 ms
[2021-02-22 23:01:14	                main:574]	:	INFO	:	Epoch 567 | loss: nan | val_loss: nan | Time: 2528.19 ms
[2021-02-22 23:01:17	                main:574]	:	INFO	:	Epoch 568 | loss: nan | val_loss: nan | Time: 2839.27 ms
[2021-02-22 23:01:20	                main:574]	:	INFO	:	Epoch 569 | loss: nan | val_loss: nan | Time: 2640.56 ms
[2021-02-22 23:01:22	                main:574]	:	INFO	:	Epoch 570 | loss: nan | val_loss: nan | Time: 2465.02 ms
[2021-02-22 23:01:24	                main:574]	:	INFO	:	Epoch 571 | loss: nan | val_loss: nan | Time: 1942.21 ms
[2021-02-22 23:01:26	                main:574]	:	INFO	:	Epoch 572 | loss: nan | val_loss: nan | Time: 2252.97 ms
[2021-02-22 23:01:28	                main:574]	:	INFO	:	Epoch 573 | loss: nan | val_loss: nan | Time: 2078.66 ms
[2021-02-22 23:01:32	                main:574]	:	INFO	:	Epoch 574 | loss: nan | val_loss: nan | Time: 2046.14 ms
[2021-02-22 23:01:38	                main:574]	:	INFO	:	Epoch 575 | loss: nan | val_loss: nan | Time: 2087.15 ms
[2021-02-22 23:01:43	                main:574]	:	INFO	:	Epoch 576 | loss: nan | val_loss: nan | Time: 2117.52 ms
[2021-02-22 23:01:47	                main:574]	:	INFO	:	Epoch 577 | loss: nan | val_loss: nan | Time: 1954.42 ms
[2021-02-22 23:01:49	                main:574]	:	INFO	:	Epoch 578 | loss: nan | val_loss: nan | Time: 1927.34 ms
[2021-02-22 23:01:52	                main:574]	:	INFO	:	Epoch 579 | loss: nan | val_loss: nan | Time: 2000.11 ms
[2021-02-22 23:01:56	                main:574]	:	INFO	:	Epoch 580 | loss: nan | val_loss: nan | Time: 2073.1 ms
[2021-02-22 23:01:58	                main:574]	:	INFO	:	Epoch 581 | loss: nan | val_loss: nan | Time: 1958.2 ms
[2021-02-22 23:02:03	                main:574]	:	INFO	:	Epoch 582 | loss: nan | val_loss: nan | Time: 2064.27 ms
[2021-02-22 23:02:06	                main:574]	:	INFO	:	Epoch 583 | loss: nan | val_loss: nan | Time: 2023.79 ms
[2021-02-22 23:02:09	                main:574]	:	INFO	:	Epoch 584 | loss: nan | val_loss: nan | Time: 2073.77 ms
[2021-02-22 23:02:13	                main:574]	:	INFO	:	Epoch 585 | loss: nan | val_loss: nan | Time: 2111.73 ms
[2021-02-22 23:02:15	                main:574]	:	INFO	:	Epoch 586 | loss: nan | val_loss: nan | Time: 1982.91 ms
[2021-02-22 23:02:20	                main:574]	:	INFO	:	Epoch 587 | loss: nan | val_loss: nan | Time: 2022.59 ms
[2021-02-22 23:02:23	                main:574]	:	INFO	:	Epoch 588 | loss: nan | val_loss: nan | Time: 1948.95 ms
[2021-02-22 23:02:25	                main:574]	:	INFO	:	Epoch 589 | loss: nan | val_loss: nan | Time: 1970.24 ms
[2021-02-22 23:02:28	                main:574]	:	INFO	:	Epoch 590 | loss: nan | val_loss: nan | Time: 1954.83 ms
[2021-02-22 23:02:32	                main:574]	:	INFO	:	Epoch 591 | loss: nan | val_loss: nan | Time: 1944.27 ms
[2021-02-22 23:02:37	                main:574]	:	INFO	:	Epoch 592 | loss: nan | val_loss: nan | Time: 1985.37 ms
[2021-02-22 23:02:42	                main:574]	:	INFO	:	Epoch 593 | loss: nan | val_loss: nan | Time: 1965.8 ms
[2021-02-22 23:02:45	                main:574]	:	INFO	:	Epoch 594 | loss: nan | val_loss: nan | Time: 2008.86 ms
[2021-02-22 23:02:50	                main:574]	:	INFO	:	Epoch 595 | loss: nan | val_loss: nan | Time: 1982.98 ms
[2021-02-22 23:02:56	                main:574]	:	INFO	:	Epoch 596 | loss: nan | val_loss: nan | Time: 2067.71 ms
[2021-02-22 23:03:03	                main:574]	:	INFO	:	Epoch 597 | loss: nan | val_loss: nan | Time: 1942.13 ms
[2021-02-22 23:03:05	                main:574]	:	INFO	:	Epoch 598 | loss: nan | val_loss: nan | Time: 1927.94 ms
[2021-02-22 23:03:07	                main:574]	:	INFO	:	Epoch 599 | loss: nan | val_loss: nan | Time: 1978.27 ms
[2021-02-22 23:03:09	                main:574]	:	INFO	:	Epoch 600 | loss: nan | val_loss: nan | Time: 1956.58 ms
[2021-02-22 23:03:14	                main:574]	:	INFO	:	Epoch 601 | loss: nan | val_loss: nan | Time: 1917.98 ms
[2021-02-22 23:03:17	                main:574]	:	INFO	:	Epoch 602 | loss: nan | val_loss: nan | Time: 1975.09 ms
[2021-02-22 23:03:19	                main:574]	:	INFO	:	Epoch 603 | loss: nan | val_loss: nan | Time: 1923.25 ms
[2021-02-22 23:03:21	                main:574]	:	INFO	:	Epoch 604 | loss: nan | val_loss: nan | Time: 1955.47 ms
[2021-02-22 23:03:27	                main:574]	:	INFO	:	Epoch 605 | loss: nan | val_loss: nan | Time: 1945.36 ms
[2021-02-22 23:03:29	                main:574]	:	INFO	:	Epoch 606 | loss: nan | val_loss: nan | Time: 1940.48 ms
[2021-02-22 23:03:32	                main:574]	:	INFO	:	Epoch 607 | loss: nan | val_loss: nan | Time: 1939.24 ms
[2021-02-22 23:03:34	                main:574]	:	INFO	:	Epoch 608 | loss: nan | val_loss: nan | Time: 1927.74 ms
[2021-02-22 23:03:36	                main:574]	:	INFO	:	Epoch 609 | loss: nan | val_loss: nan | Time: 1939.83 ms
[2021-02-22 23:03:41	                main:574]	:	INFO	:	Epoch 610 | loss: nan | val_loss: nan | Time: 2070.08 ms
[2021-02-22 23:03:45	                main:574]	:	INFO	:	Epoch 611 | loss: nan | val_loss: nan | Time: 2016.98 ms
[2021-02-22 23:03:47	                main:574]	:	INFO	:	Epoch 612 | loss: nan | val_loss: nan | Time: 1909.47 ms
[2021-02-22 23:03:49	                main:574]	:	INFO	:	Epoch 613 | loss: nan | val_loss: nan | Time: 1906.21 ms
[2021-02-22 23:03:52	                main:574]	:	INFO	:	Epoch 614 | loss: nan | val_loss: nan | Time: 1932.94 ms
[2021-02-22 23:03:55	                main:574]	:	INFO	:	Epoch 615 | loss: nan | val_loss: nan | Time: 2032.25 ms
[2021-02-22 23:03:59	                main:574]	:	INFO	:	Epoch 616 | loss: nan | val_loss: nan | Time: 2100.16 ms
[2021-02-22 23:04:03	                main:574]	:	INFO	:	Epoch 617 | loss: nan | val_loss: nan | Time: 1937.35 ms
[2021-02-22 23:04:09	                main:574]	:	INFO	:	Epoch 618 | loss: nan | val_loss: nan | Time: 1970.8 ms
[2021-02-22 23:04:12	                main:574]	:	INFO	:	Epoch 619 | loss: nan | val_loss: nan | Time: 1952.27 ms
[2021-02-22 23:04:15	                main:574]	:	INFO	:	Epoch 620 | loss: nan | val_loss: nan | Time: 2158.28 ms
[2021-02-22 23:04:17	                main:574]	:	INFO	:	Epoch 621 | loss: nan | val_loss: nan | Time: 2016.34 ms
[2021-02-22 23:04:19	                main:574]	:	INFO	:	Epoch 622 | loss: nan | val_loss: nan | Time: 1859.71 ms
[2021-02-22 23:04:24	                main:574]	:	INFO	:	Epoch 623 | loss: nan | val_loss: nan | Time: 1960.4 ms
[2021-02-22 23:04:26	                main:574]	:	INFO	:	Epoch 624 | loss: nan | val_loss: nan | Time: 1899.12 ms
[2021-02-22 23:04:28	                main:574]	:	INFO	:	Epoch 625 | loss: nan | val_loss: nan | Time: 1859.94 ms
[2021-02-22 23:04:30	                main:574]	:	INFO	:	Epoch 626 | loss: nan | val_loss: nan | Time: 1875.66 ms
[2021-02-22 23:04:32	                main:574]	:	INFO	:	Epoch 627 | loss: nan | val_loss: nan | Time: 1917.68 ms
[2021-02-22 23:04:34	                main:574]	:	INFO	:	Epoch 628 | loss: nan | val_loss: nan | Time: 1882.96 ms
[2021-02-22 23:04:36	                main:574]	:	INFO	:	Epoch 629 | loss: nan | val_loss: nan | Time: 1868.86 ms
[2021-02-22 23:04:38	                main:574]	:	INFO	:	Epoch 630 | loss: nan | val_loss: nan | Time: 1906.23 ms
[2021-02-22 23:04:39	                main:574]	:	INFO	:	Epoch 631 | loss: nan | val_loss: nan | Time: 1876.42 ms
[2021-02-22 23:04:41	                main:574]	:	INFO	:	Epoch 632 | loss: nan | val_loss: nan | Time: 1918.81 ms
[2021-02-22 23:04:43	                main:574]	:	INFO	:	Epoch 633 | loss: nan | val_loss: nan | Time: 1886.13 ms
[2021-02-22 23:04:46	                main:574]	:	INFO	:	Epoch 634 | loss: nan | val_loss: nan | Time: 2021.43 ms
[2021-02-22 23:04:48	                main:574]	:	INFO	:	Epoch 635 | loss: nan | val_loss: nan | Time: 1992.39 ms
[2021-02-22 23:04:51	                main:574]	:	INFO	:	Epoch 636 | loss: nan | val_loss: nan | Time: 2996.47 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 960)
[2021-02-23 08:19:58	                main:435]	:	INFO	:	Set logging level to 1
[2021-02-23 08:19:58	                main:441]	:	INFO	:	Running in BOINC Client mode
[2021-02-23 08:19:58	                main:444]	:	INFO	:	Resolving all filenames
[2021-02-23 08:19:58	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 (exists = 1)
[2021-02-23 08:19:58	                main:452]	:	INFO	:	Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1612595701-11795-1_2_r562329074_1 (exists = 1)
[2021-02-23 08:19:58	                main:452]	:	INFO	:	Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1612595701-11795-1_2_r562329074_0 (exists = 0)
[2021-02-23 08:19:58	                main:452]	:	INFO	:	Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1612595701-11795-1 (exists = 1)
[2021-02-23 08:19:58	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-02-23 08:19:58	                main:472]	:	INFO	:	Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5
[2021-02-23 08:19:58	                main:474]	:	INFO	:	Configuration: 
[2021-02-23 08:19:58	                main:475]	:	INFO	:	    Model type: GRU
[2021-02-23 08:19:58	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2021-02-23 08:19:59	                main:477]	:	INFO	:	    Max Epochs: 2048
[2021-02-23 08:19:59	                main:478]	:	INFO	:	    Batch Size: 128
[2021-02-23 08:19:59	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2021-02-23 08:19:59	                main:480]	:	INFO	:	    Patience: 10
[2021-02-23 08:19:59	                main:481]	:	INFO	:	    Hidden Width: 12
[2021-02-23 08:19:59	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2021-02-23 08:19:59	                main:483]	:	INFO	:	    # Backend Layers: 4
[2021-02-23 08:19:59	                main:484]	:	INFO	:	    # Threads: 1
[2021-02-23 08:19:59	                main:486]	:	INFO	:	Preparing Dataset
[2021-02-23 08:20:00	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-02-23 08:20:01	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-02-23 08:20:11	                load:106]	:	INFO	:	Successfully loaded dataset of 2048 examples into memory.
[2021-02-23 08:20:11	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-02-23 08:20:11	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-02-23 08:20:12	                load:106]	:	INFO	:	Successfully loaded dataset of 512 examples into memory.
[2021-02-23 08:20:12	                main:494]	:	INFO	:	Creating Model
[2021-02-23 08:20:12	                main:507]	:	INFO	:	Preparing config file
[2021-02-23 08:20:12	                main:511]	:	INFO	:	Found checkpoint, attempting to load... 
[2021-02-23 08:20:12	                main:512]	:	INFO	:	Loading config


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFBE9DFD759

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 02/23/21 08:20:12
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\2;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 00000000365d0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000ec210000 00000000001f6000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.662

ModLoad: 00000000eb740000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.662

ModLoad: 00000000e9dd0000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.662

ModLoad: 00000000eb4e0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000ea1e0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 00000000ebc70000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.19041.685 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.685

ModLoad: 00000000ea0d0000 0000000000109000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.19041.685 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.685

ModLoad: 00000000e9a50000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000e9950000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000ec120000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000eb440000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.19041.546

ModLoad: 00000000bb100000 0000000000058000 C:\ProgramData\BOINC\slots\2\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000eaec0000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 00000000ea280000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000d4be0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.488) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.19041.488 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.488

ModLoad: 00000000d7ab0000 0000000000091000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.27.29016.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.27.29016.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.27.29016.0

ModLoad: 00000000d7b80000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.27.29016.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.27.29016.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.27.29016.0

ModLoad: 00000000d89e0000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.27.29016.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.27.29016.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.27.29016.0

ModLoad: 000000000e730000 0000000024b49000 C:\ProgramData\BOINC\slots\2\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000096a90000 00000000030a5000 C:\ProgramData\BOINC\slots\2\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 00000000b9bf0000 0000000000045000 C:\ProgramData\BOINC\slots\2\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000007e170000 00000000034b2000 C:\ProgramData\BOINC\slots\2\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000d73b0000 0000000000010000 C:\ProgramData\BOINC\slots\2\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 00000000f3380000 000000001b3a4000 C:\ProgramData\BOINC\slots\2\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 0000000081630000 000000000abe7000 C:\ProgramData\BOINC\slots\2\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000069b20000 000000000919f000 C:\ProgramData\BOINC\slots\2\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000ae540000 000000000020d000 C:\ProgramData\BOINC\slots\2\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 0000000073370000 0000000003d5b000 C:\ProgramData\BOINC\slots\2\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000ab430000 00000000002db000 C:\ProgramData\BOINC\slots\2\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000008e520000 0000000001eca000 C:\ProgramData\BOINC\slots\2\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000bc530000 0000000000040000 C:\ProgramData\BOINC\slots\2\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000eb680000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000e78a0000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000eb800000 0000000000467000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e9af0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000ea0a0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcrypt.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e9700000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000ea210000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 00000000e9c70000 000000000015d000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e9540000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000ea4c0000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e7aa0000 0000000000795000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000ebd10000 0000000000356000 C:\WINDOWS\System32\combase.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e93c0000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : WLDP.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000ec070000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000eb3e0000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 000000009d8f0000 0000000000267000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.5655) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
    File Version          : 27.21.14.5655
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.1.70 driver
    Product Version       : 27.21.14.5655

ModLoad: 00000000e26f0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000007cc40000 0000000001529000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_9cb6a07d60163933\nvcuda64.dll (27.21.14.5655) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
    File Version          : 27.21.14.5655
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.1.70 driver
    Product Version       : 27.21.14.5655

ModLoad: 00000000ebca0000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ws2_32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e7340000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.610) (-exported- Symbols Loaded)
    Linked PDB Filename   : UxTheme.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000ddb90000 00000000006d3000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.5655) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_26\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
    File Version          : 27.21.14.5655
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA Windows drivers
    Product Version       : 27.21.14.5655

ModLoad: 00000000eac10000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 10.0.19041.860 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.860

ModLoad: 00000000e9b40000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3609, Write: 307, Other 966

- I/O Transfers Counters -
Read: 6547609, Write: 7293, Other 31990

- Paged Pool Usage -
QuotaPagedPoolUsage: 3470528, QuotaPeakPagedPoolUsage: 3491088
QuotaNonPagedPoolUsage: 37936, QuotaPeakNonPagedPoolUsage: 37936

- Virtual Memory Usage -
VirtualSize: -1479741440, PeakVirtualSize: -979128320

- Pagefile Usage -
PagefileUsage: -1479741440, PeakPagefileUsage: -1479680000

- Working Set Size -
WorkingSetSize: 1146310656, PeakWorkingSetSize: 1146314752, PageFaultCount: 378546

*** Dump of thread ID 26208 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 13437500.000000, User Time: 22656250.000000, Wait Time: 174232016.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFBE9DFD759

- Registers -
rax=00000000b18e5b38 rbx=0000000036934910 rcx=00000000e2bd2e00 rdx=00000000ec229500 rsi=00000000b18e63a0 rdi=0000000019930520
r8=00000000b18e5b60 r9=00000000b18e5b88 r10=0000000000000004 r11=00000000b18e5b98 r12=00000000dc74e3d0 r13=00000000b18e65b0
r14=00000000e3019b20 r15=0000000000000000 rip=00000000e9dfd759 rsp=00000000b18e6240 rbp=00000000b18e6530
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
b18e6310 3684a884 00000040 0000004f dc74f200 000003f4 KERNELBASE!RaiseException+0x0 
b18e6370 365f1ca6 90b00000 ec226e2c df5b5330 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18e6420 365f593b 00000020 00000003 00000000 b18e6519 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18e6570 365f1a78 00000000 00000000 b18e68a0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18e6640 365f1cfc b18e68a0 00000000 00000080 b18e6b70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18e6700 365fc4dd b18e6b70 b18e6b70 b18e6b70 b18e6b70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18e69f0 365eb2fe b18e7008 b18e6b70 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18ff790 3682c6f8 9248d4a0 00000000 00000000 9222bdb0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18ff7d0 eb757034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b18ff800 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b18ff880 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 29792 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 174231104.000000

- Registers -
rax=00000000000001d3 rbx=0000000090b06020 rcx=0000000000000070 rdx=0000000090b05ca0 rsi=0000000090b03950 rdi=0000000090b06020
r8=0000000000000008 r9=0000000000000000 r10=0000000092558f50 r11=00000000b19ff750 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000ec2afa04 rsp=00000000b19ff878 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b19ff870 ec2223d7 00000000 00000000 90b03950 90b06020 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
b19ffb70 eb757034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
b19ffba0 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b19ffc20 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 33360 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174231104.000000

- Registers -
rax=00000000000001d3 rbx=0000000090b0b740 rcx=0000000000000070 rdx=0000000090b0b3c0 rsi=0000000090b03950 rdi=0000000090b0b740
r8=0000000000000008 r9=0000000000000040 r10=00000000000005c0 r11=00000000bb9be3bf r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000ec2afa04 rsp=00000000b1affb48 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b1affb40 ec2223d7 00000000 00000000 90b03950 90b0b740 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
b1affe40 eb757034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
b1affe70 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b1affef0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 28992 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 174231104.000000

- Registers -
rax=00000000000001d3 rbx=0000000090b0cbd0 rcx=0000000000000070 rdx=0000000090b0c850 rsi=0000000090b03950 rdi=0000000090b0cbd0
r8=00000000e26f5008 r9=00000000b1bff540 r10=00000000e26f00f0 r11=0000000000000001 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000ec2afa04 rsp=00000000b1bffa98 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b1bffa90 ec2223d7 00000000 00000000 90b03950 90b0cbd0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
b1bffd90 eb757034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
b1bffdc0 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b1bffe40 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 29152 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174232000.000000

- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000b1dff840 rsi=0000000000000000 rdi=0000000000000064
r8=00000000b1dff818 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000ec2ac634 rsp=00000000b1dff818 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b1dff810 e9e1d84e b1dff8d8 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0 
b1dff8b0 3660a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0 
b1dff8e0 eb757034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
b1dff910 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b1dff990 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 26648 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174231216.000000

- Registers -
rax=00000000000001d3 rbx=0000000092913de0 rcx=0000000000000018 rdx=0000000092913a60 rsi=0000000090b00b50 rdi=0000000092913de0
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000ec2afa04 rsp=00000000b1eff488 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b1eff480 ec2223d7 90b00af8 90b00af8 90b00b50 92913de0 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
b1eff780 eb757034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
b1eff7b0 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b1eff830 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 9560 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174232000.000000

- Registers -
rax=000000000000005b rbx=000000000000000b rcx=000000000000000b rdx=00000000b1fffb30 rsi=0000000000000000 rdi=000000000000000b
r8=0000000092a21580 r9=000000009262e860 r10=0000000000000000 r11=00000000b1fffaa0 r12=0000000000000064 r13=00000000b1fffb30
r14=00000000b1fff830 r15=0000000000000000 rip=00000000ec2acb04 rsp=00000000b1fff7d8 rbp=000000009262e860
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b1fff7d0 e9e20d40 00000000 b17ea000 ec37b4d0 b17ea000 ntdll!NtWaitForMultipleObjects+0x0 
b1fffac0 e9e20c3e 90b127d0 004e0049 0000000b 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
b1fffb00 7cdeb924 92631da0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0 
b1fffd40 7ce807bd 00000000 00000004 00000000 00000000 nvcuda64!+0x0 
b1fffdf0 7cdeb2f3 76b88fb6 713cf7b5 92bacc70 00000000 nvcuda64!cuProfilerStop+0x0 
b1fffe20 7d0f8ea0 92bacc70 00000000 00000000 00000000 nvcuda64!+0x0 
b1fffe50 eb757034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
b1fffe80 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b1ffff00 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 16432 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174231440.000000

- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=0000000000000268 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000268
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=000000009266e370 r13=0000000000000000
r14=0000000000000268 r15=0000000000000000 rip=00000000ec2ac034 rsp=00000000b20ff6d8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b20ff6d0 e9dea34e 00000000 00000000 00000000 00000000 ntdll!NtWaitForSingleObject+0x0 
b20ff770 7cffa698 00000000 e9df2cb5 00000000 00000268 KERNELBASE!WaitForSingleObjectEx+0x0 
b20ff7e0 7cdeb2f3 00000001 929c6100 92bac250 00000000 nvcuda64!cuProfilerStop+0x0 
b20ff810 7d0f8ea0 92bac250 00000000 00000000 00000000 nvcuda64!+0x0 
b20ff840 eb757034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
b20ff870 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b20ff8f0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 10888 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 174232000.000000

- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000928e3258 rdx=00000000b21ffe18 rsi=00000000b21ffd80 rdi=0000000000000000
r8=0000000000000001 r9=0000000000002a88 r10=00000000928e79d8 r11=00000000b6b9f14f r12=00000000b21ffe18 r13=0000000000000000
r14=00000000928e3258 r15=00000000928e79d8 rip=00000000ec2af9a4 rsp=00000000b21ffd58 rbp=00000000b21ffdb0
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
b21ffd50 ec2754f5 00000102 00000102 00000000 b21ffd80 ntdll!NtWaitForAlertByThreadId+0x0 
b21ffdc0 e9e3b9d9 928e51c0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0 
b21ffdf0 7cd720ba 928e51c0 ec25a9f0 928e51c0 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0 
b21ffe50 7cdeb2f3 00000001 7690e4cc 00000000 00000000 nvcuda64!+0x0 
b21ffe80 7d0f8ea0 92bad3c0 00000000 00000000 00000000 nvcuda64!+0x0 
b21ffeb0 eb757034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
b21ffee0 ec25d0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
b21fff60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)