Task 10975337

Name ParityModified-1639957296-9924-1-0_1
Workunit 7828232
Created 24 Jan 2022, 0:42:01 UTC
Sent 24 Jan 2022, 1:34:14 UTC
Report deadline 1 Feb 2022, 1:34:14 UTC
Received 24 Jan 2022, 23:44:27 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 17991
Run time 2 hours 0 min 14 sec
CPU time 1 hours 52 min 50 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4,458.46 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 2.09 GB
Peak swap size 4.32 GB
Peak disk usage 1.54 GB

Stderr output

<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
5 | Time: 8780.17 ms
[2022-01-24 13:49:46	                main:574]	:	INFO	:	Epoch 588 | loss: 0.0311615 | val_loss: 0.0311704 | Time: 8883.58 ms
[2022-01-24 13:49:54	                main:574]	:	INFO	:	Epoch 589 | loss: 0.0311592 | val_loss: 0.0311701 | Time: 8517.75 ms
[2022-01-24 13:50:03	                main:574]	:	INFO	:	Epoch 590 | loss: 0.0311564 | val_loss: 0.0311705 | Time: 8686.75 ms
[2022-01-24 13:50:11	                main:574]	:	INFO	:	Epoch 591 | loss: 0.0311546 | val_loss: 0.0311714 | Time: 8452.01 ms
[2022-01-24 13:50:20	                main:574]	:	INFO	:	Epoch 592 | loss: 0.0311549 | val_loss: 0.0311668 | Time: 8810.12 ms
[2022-01-24 13:50:28	                main:574]	:	INFO	:	Epoch 593 | loss: 0.0311543 | val_loss: 0.03117 | Time: 8277.26 ms
[2022-01-24 13:50:37	                main:574]	:	INFO	:	Epoch 594 | loss: 0.0311535 | val_loss: 0.0311674 | Time: 8619.14 ms
[2022-01-24 13:50:46	                main:574]	:	INFO	:	Epoch 595 | loss: 0.0311502 | val_loss: 0.0311667 | Time: 8779.24 ms
[2022-01-24 13:50:54	                main:574]	:	INFO	:	Epoch 596 | loss: 0.0311484 | val_loss: 0.0311682 | Time: 8284 ms
[2022-01-24 13:51:03	                main:574]	:	INFO	:	Epoch 597 | loss: 0.0311596 | val_loss: 0.031178 | Time: 8857.48 ms
[2022-01-24 13:51:11	                main:574]	:	INFO	:	Epoch 598 | loss: 0.0311775 | val_loss: 0.0311754 | Time: 8074.65 ms
[2022-01-24 13:51:20	                main:574]	:	INFO	:	Epoch 599 | loss: 0.0311718 | val_loss: 0.0311737 | Time: 8943.62 ms
[2022-01-24 13:51:28	                main:574]	:	INFO	:	Epoch 600 | loss: 0.0311662 | val_loss: 0.0311671 | Time: 8279.23 ms
[2022-01-24 13:51:37	                main:574]	:	INFO	:	Epoch 601 | loss: 0.0311609 | val_loss: 0.0311682 | Time: 8645.49 ms
[2022-01-24 13:51:46	                main:574]	:	INFO	:	Epoch 602 | loss: 0.0311586 | val_loss: 0.0311651 | Time: 8908.66 ms
[2022-01-24 13:51:55	                main:574]	:	INFO	:	Epoch 603 | loss: 0.0311564 | val_loss: 0.0311625 | Time: 8888.62 ms
[2022-01-24 13:52:04	                main:574]	:	INFO	:	Epoch 604 | loss: 0.0311559 | val_loss: 0.0311629 | Time: 8836.12 ms
[2022-01-24 13:52:12	                main:574]	:	INFO	:	Epoch 605 | loss: 0.0311549 | val_loss: 0.0311626 | Time: 8211.91 ms
[2022-01-24 13:52:21	                main:574]	:	INFO	:	Epoch 606 | loss: 0.0311529 | val_loss: 0.0311614 | Time: 8885.12 ms
[2022-01-24 13:52:29	                main:574]	:	INFO	:	Epoch 607 | loss: 0.0311519 | val_loss: 0.0311623 | Time: 8153.96 ms
[2022-01-24 13:52:38	                main:574]	:	INFO	:	Epoch 608 | loss: 0.0311513 | val_loss: 0.0311655 | Time: 8885.76 ms
[2022-01-24 13:52:47	                main:574]	:	INFO	:	Epoch 609 | loss: 0.0311537 | val_loss: 0.0311669 | Time: 8772.77 ms
[2022-01-24 13:52:55	                main:574]	:	INFO	:	Epoch 610 | loss: 0.0311527 | val_loss: 0.0311658 | Time: 8196.2 ms
[2022-01-24 13:53:04	                main:574]	:	INFO	:	Epoch 611 | loss: 0.0311501 | val_loss: 0.0311625 | Time: 8645.56 ms
[2022-01-24 13:53:12	                main:574]	:	INFO	:	Epoch 612 | loss: 0.0311497 | val_loss: 0.0311673 | Time: 8163.73 ms
[2022-01-24 13:53:20	                main:574]	:	INFO	:	Epoch 613 | loss: 0.0311503 | val_loss: 0.0311607 | Time: 8728.49 ms
[2022-01-24 13:53:29	                main:574]	:	INFO	:	Epoch 614 | loss: 0.0311489 | val_loss: 0.0311645 | Time: 8357.39 ms
[2022-01-24 13:53:38	                main:574]	:	INFO	:	Epoch 615 | loss: 0.0311484 | val_loss: 0.0311628 | Time: 9091.69 ms
[2022-01-24 13:53:47	                main:574]	:	INFO	:	Epoch 616 | loss: 0.0311495 | val_loss: 0.0311664 | Time: 8918.4 ms
[2022-01-24 13:53:55	                main:574]	:	INFO	:	Epoch 617 | loss: 0.0311485 | val_loss: 0.0311641 | Time: 8238.47 ms
[2022-01-24 13:54:04	                main:574]	:	INFO	:	Epoch 618 | loss: 0.0311461 | val_loss: 0.0311614 | Time: 8870.29 ms
[2022-01-24 13:54:12	                main:574]	:	INFO	:	Epoch 619 | loss: 0.0311459 | val_loss: 0.031161 | Time: 8061.26 ms
[2022-01-24 13:54:21	                main:574]	:	INFO	:	Epoch 620 | loss: 0.0311446 | val_loss: 0.0311638 | Time: 8618.85 ms
[2022-01-24 13:54:29	                main:574]	:	INFO	:	Epoch 621 | loss: 0.0311468 | val_loss: 0.0311664 | Time: 8274.3 ms
[2022-01-24 13:54:38	                main:574]	:	INFO	:	Epoch 622 | loss: 0.031152 | val_loss: 0.0311726 | Time: 8915.27 ms
[2022-01-24 13:54:47	                main:574]	:	INFO	:	Epoch 623 | loss: 0.0311481 | val_loss: 0.0311626 | Time: 8838.38 ms
[2022-01-24 13:54:55	                main:574]	:	INFO	:	Epoch 624 | loss: 0.0311484 | val_loss: 0.0311663 | Time: 8357.57 ms
[2022-01-24 13:55:04	                main:574]	:	INFO	:	Epoch 625 | loss: 0.0311463 | val_loss: 0.0311609 | Time: 8891.63 ms
[2022-01-24 13:55:12	                main:574]	:	INFO	:	Epoch 626 | loss: 0.0311453 | val_loss: 0.0311644 | Time: 8349.54 ms
[2022-01-24 13:55:21	                main:574]	:	INFO	:	Epoch 627 | loss: 0.031145 | val_loss: 0.0311705 | Time: 8745.02 ms
[2022-01-24 13:55:29	                main:574]	:	INFO	:	Epoch 628 | loss: 0.031144 | val_loss: 0.031163 | Time: 8168.8 ms
[2022-01-24 13:55:38	                main:574]	:	INFO	:	Epoch 629 | loss: 0.031142 | val_loss: 0.0311628 | Time: 8802.25 ms
[2022-01-24 13:55:47	                main:574]	:	INFO	:	Epoch 630 | loss: 0.0311413 | val_loss: 0.0311644 | Time: 8585.12 ms
[2022-01-24 13:55:55	                main:574]	:	INFO	:	Epoch 631 | loss: 0.0311406 | val_loss: 0.0311614 | Time: 8275.25 ms
[2022-01-24 13:56:04	                main:574]	:	INFO	:	Epoch 632 | loss: 0.0311421 | val_loss: 0.0311604 | Time: 8719.29 ms
[2022-01-24 13:56:12	                main:574]	:	INFO	:	Epoch 633 | loss: 0.0311409 | val_loss: 0.031161 | Time: 8569.21 ms
[2022-01-24 13:56:21	                main:574]	:	INFO	:	Epoch 634 | loss: 0.0311407 | val_loss: 0.0311669 | Time: 8799.95 ms
[2022-01-24 13:56:30	                main:574]	:	INFO	:	Epoch 635 | loss: 0.0311804 | val_loss: 0.0311908 | Time: 8291.26 ms
[2022-01-24 13:56:38	                main:574]	:	INFO	:	Epoch 636 | loss: 0.031189 | val_loss: 0.031182 | Time: 8952.52 ms
[2022-01-24 13:56:47	                main:574]	:	INFO	:	Epoch 637 | loss: 0.03118 | val_loss: 0.0311708 | Time: 8696.29 ms
[2022-01-24 13:56:56	                main:574]	:	INFO	:	Epoch 638 | loss: 0.0311728 | val_loss: 0.0311733 | Time: 8366.34 ms
[2022-01-24 13:57:04	                main:574]	:	INFO	:	Epoch 639 | loss: 0.0311684 | val_loss: 0.0311664 | Time: 8855.84 ms
[2022-01-24 13:57:13	                main:574]	:	INFO	:	Epoch 640 | loss: 0.0311669 | val_loss: 0.0311651 | Time: 8386.84 ms
[2022-01-24 13:57:22	                main:574]	:	INFO	:	Epoch 641 | loss: 0.0311649 | val_loss: 0.031164 | Time: 9076.72 ms
[2022-01-24 13:57:30	                main:574]	:	INFO	:	Epoch 642 | loss: 0.0311629 | val_loss: 0.031165 | Time: 8463.44 ms
[2022-01-24 13:57:39	                main:574]	:	INFO	:	Epoch 643 | loss: 0.0311619 | val_loss: 0.0311622 | Time: 9046.97 ms
[2022-01-24 13:57:48	                main:574]	:	INFO	:	Epoch 644 | loss: 0.0311619 | val_loss: 0.0311683 | Time: 8459.71 ms
[2022-01-24 13:57:57	                main:574]	:	INFO	:	Epoch 645 | loss: 0.031163 | val_loss: 0.0311627 | Time: 8777.08 ms
[2022-01-24 13:58:05	                main:574]	:	INFO	:	Epoch 646 | loss: 0.0311662 | val_loss: 0.0311721 | Time: 8755.27 ms
[2022-01-24 13:58:14	                main:574]	:	INFO	:	Epoch 647 | loss: 0.031162 | val_loss: 0.0311618 | Time: 8496.73 ms
[2022-01-24 13:58:23	                main:574]	:	INFO	:	Epoch 648 | loss: 0.0311598 | val_loss: 0.0311636 | Time: 8744.16 ms
[2022-01-24 13:58:31	                main:574]	:	INFO	:	Epoch 649 | loss: 0.0311591 | val_loss: 0.0311631 | Time: 8435.55 ms
[2022-01-24 13:58:40	                main:574]	:	INFO	:	Epoch 650 | loss: 0.0311576 | val_loss: 0.0311622 | Time: 8804.67 ms
[2022-01-24 13:58:48	                main:574]	:	INFO	:	Epoch 651 | loss: 0.0311585 | val_loss: 0.0311637 | Time: 8372.38 ms
[2022-01-24 13:58:57	                main:574]	:	INFO	:	Epoch 652 | loss: 0.0311583 | val_loss: 0.0311638 | Time: 8786.55 ms
[2022-01-24 13:59:06	                main:574]	:	INFO	:	Epoch 653 | loss: 0.0311557 | val_loss: 0.0311609 | Time: 8908.41 ms
[2022-01-24 13:59:14	                main:574]	:	INFO	:	Epoch 654 | loss: 0.0311558 | val_loss: 0.0311627 | Time: 8296.9 ms
[2022-01-24 13:59:23	                main:574]	:	INFO	:	Epoch 655 | loss: 0.0311596 | val_loss: 0.0311613 | Time: 8835.2 ms
[2022-01-24 13:59:32	                main:574]	:	INFO	:	Epoch 656 | loss: 0.0311557 | val_loss: 0.0311625 | Time: 8431.04 ms
[2022-01-24 13:59:41	                main:574]	:	INFO	:	Epoch 657 | loss: 0.0311545 | val_loss: 0.031164 | Time: 8844.59 ms
[2022-01-24 13:59:49	                main:574]	:	INFO	:	Epoch 658 | loss: 0.031154 | val_loss: 0.0311665 | Time: 8141.99 ms
[2022-01-24 13:59:58	                main:574]	:	INFO	:	Epoch 659 | loss: 0.0311554 | val_loss: 0.0311665 | Time: 8769.74 ms
[2022-01-24 14:00:06	                main:574]	:	INFO	:	Epoch 660 | loss: 0.0311538 | val_loss: 0.0311718 | Time: 8844 ms
[2022-01-24 14:00:15	                main:574]	:	INFO	:	Epoch 661 | loss: 0.0311564 | val_loss: 0.0311652 | Time: 8302.43 ms
[2022-01-24 14:00:24	                main:574]	:	INFO	:	Epoch 662 | loss: 0.0311532 | val_loss: 0.0311632 | Time: 8858.47 ms
[2022-01-24 14:00:32	                main:574]	:	INFO	:	Epoch 663 | loss: 0.0311574 | val_loss: 0.0311634 | Time: 8687.75 ms
[2022-01-24 14:00:41	                main:574]	:	INFO	:	Epoch 664 | loss: nan | val_loss: nan | Time: 9096.99 ms
[2022-01-24 14:00:50	                main:574]	:	INFO	:	Epoch 665 | loss: nan | val_loss: nan | Time: 8319.41 ms
[2022-01-24 14:00:59	                main:574]	:	INFO	:	Epoch 666 | loss: nan | val_loss: nan | Time: 8942.49 ms
[2022-01-24 14:01:08	                main:574]	:	INFO	:	Epoch 667 | loss: nan | val_loss: nan | Time: 8869.15 ms
[2022-01-24 14:01:16	                main:574]	:	INFO	:	Epoch 668 | loss: nan | val_loss: nan | Time: 8529.51 ms
[2022-01-24 14:01:25	                main:574]	:	INFO	:	Epoch 669 | loss: nan | val_loss: nan | Time: 8822.4 ms
[2022-01-24 14:01:33	                main:574]	:	INFO	:	Epoch 670 | loss: nan | val_loss: nan | Time: 8481.15 ms
[2022-01-24 14:01:42	                main:574]	:	INFO	:	Epoch 671 | loss: nan | val_loss: nan | Time: 8829.13 ms
[2022-01-24 14:01:51	                main:574]	:	INFO	:	Epoch 672 | loss: nan | val_loss: nan | Time: 8744.91 ms
[2022-01-24 14:02:00	                main:574]	:	INFO	:	Epoch 673 | loss: nan | val_loss: nan | Time: 8869.02 ms
[2022-01-24 14:02:08	                main:574]	:	INFO	:	Epoch 674 | loss: nan | val_loss: nan | Time: 8383.02 ms
[2022-01-24 14:02:17	                main:574]	:	INFO	:	Epoch 675 | loss: nan | val_loss: nan | Time: 8762.67 ms
[2022-01-24 14:02:26	                main:574]	:	INFO	:	Epoch 676 | loss: nan | val_loss: nan | Time: 8944.46 ms
[2022-01-24 14:02:34	                main:574]	:	INFO	:	Epoch 677 | loss: nan | val_loss: nan | Time: 8373.89 ms
[2022-01-24 14:02:43	                main:574]	:	INFO	:	Epoch 678 | loss: nan | val_loss: nan | Time: 8955.61 ms
[2022-01-24 14:02:52	                main:574]	:	INFO	:	Epoch 679 | loss: nan | val_loss: nan | Time: 8397.83 ms
[2022-01-24 14:03:01	                main:574]	:	INFO	:	Epoch 680 | loss: nan | val_loss: nan | Time: 8673.02 ms
[2022-01-24 14:03:09	                main:574]	:	INFO	:	Epoch 681 | loss: nan | val_loss: nan | Time: 8376.64 ms
[2022-01-24 14:03:18	                main:574]	:	INFO	:	Epoch 682 | loss: nan | val_loss: nan | Time: 9001.42 ms
[2022-01-24 14:03:27	                main:574]	:	INFO	:	Epoch 683 | loss: nan | val_loss: nan | Time: 8703.95 ms
[2022-01-24 14:03:35	                main:574]	:	INFO	:	Epoch 684 | loss: nan | val_loss: nan | Time: 8280.9 ms
[2022-01-24 14:03:44	                main:574]	:	INFO	:	Epoch 685 | loss: nan | val_loss: nan | Time: 8784.4 ms
[2022-01-24 14:03:52	                main:574]	:	INFO	:	Epoch 686 | loss: nan | val_loss: nan | Time: 8299.49 ms
[2022-01-24 14:04:01	                main:574]	:	INFO	:	Epoch 687 | loss: nan | val_loss: nan | Time: 8882.43 ms
[2022-01-24 14:04:09	                main:574]	:	INFO	:	Epoch 688 | loss: nan | val_loss: nan | Time: 8455.49 ms
[2022-01-24 14:04:18	                main:574]	:	INFO	:	Epoch 689 | loss: nan | val_loss: nan | Time: 8897.85 ms
[2022-01-24 14:04:27	                main:574]	:	INFO	:	Epoch 690 | loss: nan | val_loss: nan | Time: 8991.07 ms
[2022-01-24 14:04:36	                main:574]	:	INFO	:	Epoch 691 | loss: nan | val_loss: nan | Time: 8352.9 ms
[2022-01-24 14:04:44	                main:574]	:	INFO	:	Epoch 692 | loss: nan | val_loss: nan | Time: 8794.93 ms
[2022-01-24 14:04:53	                main:574]	:	INFO	:	Epoch 693 | loss: nan | val_loss: nan | Time: 8484.73 ms
[2022-01-24 14:05:02	                main:574]	:	INFO	:	Epoch 694 | loss: nan | val_loss: nan | Time: 8885.18 ms
[2022-01-24 14:05:10	                main:574]	:	INFO	:	Epoch 695 | loss: nan | val_loss: nan | Time: 8262.93 ms
[2022-01-24 14:05:19	                main:574]	:	INFO	:	Epoch 696 | loss: nan | val_loss: nan | Time: 9198.79 ms
[2022-01-24 14:05:28	                main:574]	:	INFO	:	Epoch 697 | loss: nan | val_loss: nan | Time: 8667.61 ms
[2022-01-24 14:05:36	                main:574]	:	INFO	:	Epoch 698 | loss: nan | val_loss: nan | Time: 8365.24 ms
[2022-01-24 14:05:45	                main:574]	:	INFO	:	Epoch 699 | loss: nan | val_loss: nan | Time: 8887.5 ms
[2022-01-24 14:05:54	                main:574]	:	INFO	:	Epoch 700 | loss: nan | val_loss: nan | Time: 8632.39 ms
[2022-01-24 14:06:03	                main:574]	:	INFO	:	Epoch 701 | loss: nan | val_loss: nan | Time: 8930.48 ms
[2022-01-24 14:06:12	                main:574]	:	INFO	:	Epoch 702 | loss: nan | val_loss: nan | Time: 8601.82 ms
[2022-01-24 14:06:20	                main:574]	:	INFO	:	Epoch 703 | loss: nan | val_loss: nan | Time: 8683.23 ms
[2022-01-24 14:06:29	                main:574]	:	INFO	:	Epoch 704 | loss: nan | val_loss: nan | Time: 8448.85 ms
[2022-01-24 14:06:38	                main:574]	:	INFO	:	Epoch 705 | loss: nan | val_loss: nan | Time: 8935.13 ms
[2022-01-24 14:06:46	                main:574]	:	INFO	:	Epoch 706 | loss: nan | val_loss: nan | Time: 8777.75 ms
[2022-01-24 14:06:55	                main:574]	:	INFO	:	Epoch 707 | loss: nan | val_loss: nan | Time: 8195.03 ms
[2022-01-24 14:07:04	                main:574]	:	INFO	:	Epoch 708 | loss: nan | val_loss: nan | Time: 9040.32 ms
[2022-01-24 14:07:12	                main:574]	:	INFO	:	Epoch 709 | loss: nan | val_loss: nan | Time: 8605.6 ms
[2022-01-24 14:07:21	                main:574]	:	INFO	:	Epoch 710 | loss: nan | val_loss: nan | Time: 9072.7 ms
[2022-01-24 14:07:30	                main:574]	:	INFO	:	Epoch 711 | loss: nan | val_loss: nan | Time: 8591.94 ms
[2022-01-24 14:07:39	                main:574]	:	INFO	:	Epoch 712 | loss: nan | val_loss: nan | Time: 9063.93 ms
[2022-01-24 14:07:48	                main:574]	:	INFO	:	Epoch 713 | loss: nan | val_loss: nan | Time: 8834.93 ms
[2022-01-24 14:07:56	                main:574]	:	INFO	:	Epoch 714 | loss: nan | val_loss: nan | Time: 8495.66 ms
[2022-01-24 14:08:05	                main:574]	:	INFO	:	Epoch 715 | loss: nan | val_loss: nan | Time: 8660.03 ms
[2022-01-24 14:08:14	                main:574]	:	INFO	:	Epoch 716 | loss: nan | val_loss: nan | Time: 8464.81 ms
[2022-01-24 14:08:23	                main:574]	:	INFO	:	Epoch 717 | loss: nan | val_loss: nan | Time: 9041.33 ms
[2022-01-24 14:08:31	                main:574]	:	INFO	:	Epoch 718 | loss: nan | val_loss: nan | Time: 8319.79 ms
[2022-01-24 14:08:40	                main:574]	:	INFO	:	Epoch 719 | loss: nan | val_loss: nan | Time: 8741.65 ms
[2022-01-24 14:08:49	                main:574]	:	INFO	:	Epoch 720 | loss: nan | val_loss: nan | Time: 8881.58 ms
[2022-01-24 14:08:58	                main:574]	:	INFO	:	Epoch 721 | loss: nan | val_loss: nan | Time: 8841.49 ms
[2022-01-24 14:09:06	                main:574]	:	INFO	:	Epoch 722 | loss: nan | val_loss: nan | Time: 8944.85 ms
[2022-01-24 14:09:15	                main:574]	:	INFO	:	Epoch 723 | loss: nan | val_loss: nan | Time: 8563.76 ms
[2022-01-24 14:09:24	                main:574]	:	INFO	:	Epoch 724 | loss: nan | val_loss: nan | Time: 8980.61 ms
[2022-01-24 14:09:33	                main:574]	:	INFO	:	Epoch 725 | loss: nan | val_loss: nan | Time: 8526.81 ms
[2022-01-24 14:09:41	                main:574]	:	INFO	:	Epoch 726 | loss: nan | val_loss: nan | Time: 8885.8 ms
[2022-01-24 14:09:50	                main:574]	:	INFO	:	Epoch 727 | loss: nan | val_loss: nan | Time: 8420.38 ms
[2022-01-24 14:09:59	                main:574]	:	INFO	:	Epoch 728 | loss: nan | val_loss: nan | Time: 9181.36 ms
[2022-01-24 14:10:08	                main:574]	:	INFO	:	Epoch 729 | loss: nan | val_loss: nan | Time: 8899.58 ms
[2022-01-24 14:10:16	                main:574]	:	INFO	:	Epoch 730 | loss: nan | val_loss: nan | Time: 8442.61 ms
[2022-01-24 14:10:25	                main:574]	:	INFO	:	Epoch 731 | loss: nan | val_loss: nan | Time: 8996.85 ms
[2022-01-24 14:10:34	                main:574]	:	INFO	:	Epoch 732 | loss: nan | val_loss: nan | Time: 8458.06 ms
[2022-01-24 14:10:43	                main:574]	:	INFO	:	Epoch 733 | loss: nan | val_loss: nan | Time: 9090.04 ms
[2022-01-24 14:10:52	                main:574]	:	INFO	:	Epoch 734 | loss: nan | val_loss: nan | Time: 8448.61 ms
[2022-01-24 14:11:01	                main:574]	:	INFO	:	Epoch 735 | loss: nan | val_loss: nan | Time: 8989.26 ms
[2022-01-24 14:11:09	                main:574]	:	INFO	:	Epoch 736 | loss: nan | val_loss: nan | Time: 8685.28 ms
[2022-01-24 14:11:18	                main:574]	:	INFO	:	Epoch 737 | loss: nan | val_loss: nan | Time: 8780.86 ms
[2022-01-24 14:11:27	                main:574]	:	INFO	:	Epoch 738 | loss: nan | val_loss: nan | Time: 8864.5 ms
[2022-01-24 14:11:35	                main:574]	:	INFO	:	Epoch 739 | loss: nan | val_loss: nan | Time: 8426.93 ms
[2022-01-24 14:11:44	                main:574]	:	INFO	:	Epoch 740 | loss: nan | val_loss: nan | Time: 8700.02 ms
[2022-01-24 14:11:52	                main:574]	:	INFO	:	Epoch 741 | loss: nan | val_loss: nan | Time: 8261.05 ms
[2022-01-24 14:12:01	                main:574]	:	INFO	:	Epoch 742 | loss: nan | val_loss: nan | Time: 8975.46 ms
[2022-01-24 14:12:10	                main:574]	:	INFO	:	Epoch 743 | loss: nan | val_loss: nan | Time: 8532.09 ms
[2022-01-24 14:12:19	                main:574]	:	INFO	:	Epoch 744 | loss: nan | val_loss: nan | Time: 8785.1 ms
[2022-01-24 14:12:28	                main:574]	:	INFO	:	Epoch 745 | loss: nan | val_loss: nan | Time: 8892.25 ms
[2022-01-24 14:12:36	                main:574]	:	INFO	:	Epoch 746 | loss: nan | val_loss: nan | Time: 8181.13 ms
[2022-01-24 14:12:45	                main:574]	:	INFO	:	Epoch 747 | loss: nan | val_loss: nan | Time: 8658.16 ms
[2022-01-24 14:12:53	                main:574]	:	INFO	:	Epoch 748 | loss: nan | val_loss: nan | Time: 8470.48 ms
[2022-01-24 14:13:02	                main:574]	:	INFO	:	Epoch 749 | loss: nan | val_loss: nan | Time: 8729.9 ms
[2022-01-24 14:13:10	                main:574]	:	INFO	:	Epoch 750 | loss: nan | val_loss: nan | Time: 8219.1 ms
[2022-01-24 14:13:19	                main:574]	:	INFO	:	Epoch 751 | loss: nan | val_loss: nan | Time: 8685.25 ms
[2022-01-24 14:13:28	                main:574]	:	INFO	:	Epoch 752 | loss: nan | val_loss: nan | Time: 8905.35 ms
[2022-01-24 14:13:36	                main:574]	:	INFO	:	Epoch 753 | loss: nan | val_loss: nan | Time: 8398.51 ms
[2022-01-24 14:13:45	                main:574]	:	INFO	:	Epoch 754 | loss: nan | val_loss: nan | Time: 8682.42 ms
[2022-01-24 14:13:54	                main:574]	:	INFO	:	Epoch 755 | loss: nan | val_loss: nan | Time: 8523.19 ms
[2022-01-24 14:14:02	                main:574]	:	INFO	:	Epoch 756 | loss: nan | val_loss: nan | Time: 8846.67 ms
[2022-01-24 14:14:11	                main:574]	:	INFO	:	Epoch 757 | loss: nan | val_loss: nan | Time: 8301.38 ms
[2022-01-24 14:14:19	                main:574]	:	INFO	:	Epoch 758 | loss: nan | val_loss: nan | Time: 8702.44 ms
[2022-01-24 14:14:28	                main:574]	:	INFO	:	Epoch 759 | loss: nan | val_loss: nan | Time: 8967.69 ms
[2022-01-24 14:14:37	                main:574]	:	INFO	:	Epoch 760 | loss: nan | val_loss: nan | Time: 8448.91 ms
[2022-01-24 14:14:46	                main:574]	:	INFO	:	Epoch 761 | loss: nan | val_loss: nan | Time: 8989.22 ms
[2022-01-24 14:14:54	                main:574]	:	INFO	:	Epoch 762 | loss: nan | val_loss: nan | Time: 8251.94 ms
[2022-01-24 14:15:03	                main:574]	:	INFO	:	Epoch 763 | loss: nan | val_loss: nan | Time: 8994.84 ms
[2022-01-24 14:15:11	                main:574]	:	INFO	:	Epoch 764 | loss: nan | val_loss: nan | Time: 8302.87 ms
[2022-01-24 14:15:20	                main:574]	:	INFO	:	Epoch 765 | loss: nan | val_loss: nan | Time: 8718.78 ms
[2022-01-24 14:15:29	                main:574]	:	INFO	:	Epoch 766 | loss: nan | val_loss: nan | Time: 8742.8 ms
[2022-01-24 14:15:37	                main:574]	:	INFO	:	Epoch 767 | loss: nan | val_loss: nan | Time: 8501.92 ms
[2022-01-24 14:15:46	                main:574]	:	INFO	:	Epoch 768 | loss: nan | val_loss: nan | Time: 8920.83 ms
[2022-01-24 14:15:55	                main:574]	:	INFO	:	Epoch 769 | loss: nan | val_loss: nan | Time: 8501.14 ms
[2022-01-24 14:16:04	                main:574]	:	INFO	:	Epoch 770 | loss: nan | val_loss: nan | Time: 8909.46 ms
[2022-01-24 14:16:12	                main:574]	:	INFO	:	Epoch 771 | loss: nan | val_loss: nan | Time: 8382.56 ms
[2022-01-24 14:16:21	                main:574]	:	INFO	:	Epoch 772 | loss: nan | val_loss: nan | Time: 8817.59 ms
[2022-01-24 14:16:30	                main:574]	:	INFO	:	Epoch 773 | loss: nan | val_loss: nan | Time: 8544.49 ms
[2022-01-24 14:16:38	                main:574]	:	INFO	:	Epoch 774 | loss: nan | val_loss: nan | Time: 8807.01 ms
[2022-01-24 14:16:47	                main:574]	:	INFO	:	Epoch 775 | loss: nan | val_loss: nan | Time: 8726.12 ms
[2022-01-24 14:16:55	                main:574]	:	INFO	:	Epoch 776 | loss: nan | val_loss: nan | Time: 8284.88 ms
[2022-01-24 14:17:04	                main:574]	:	INFO	:	Epoch 777 | loss: nan | val_loss: nan | Time: 8985.42 ms
[2022-01-24 14:17:13	                main:574]	:	INFO	:	Epoch 778 | loss: nan | val_loss: nan | Time: 8651.76 ms
[2022-01-24 14:17:22	                main:574]	:	INFO	:	Epoch 779 | loss: nan | val_loss: nan | Time: 8858.48 ms
[2022-01-24 14:17:30	                main:574]	:	INFO	:	Epoch 780 | loss: nan | val_loss: nan | Time: 8328.09 ms
[2022-01-24 14:17:39	                main:574]	:	INFO	:	Epoch 781 | loss: nan | val_loss: nan | Time: 8890.31 ms
[2022-01-24 14:17:48	                main:574]	:	INFO	:	Epoch 782 | loss: nan | val_loss: nan | Time: 8748.07 ms
[2022-01-24 14:17:57	                main:574]	:	INFO	:	Epoch 783 | loss: nan | val_loss: nan | Time: 8542.68 ms
[2022-01-24 14:18:06	                main:574]	:	INFO	:	Epoch 784 | loss: nan | val_loss: nan | Time: 8918.26 ms
[2022-01-24 14:18:14	                main:574]	:	INFO	:	Epoch 785 | loss: nan | val_loss: nan | Time: 8325.8 ms
[2022-01-24 14:18:23	                main:574]	:	INFO	:	Epoch 786 | loss: nan | val_loss: nan | Time: 8941.56 ms
[2022-01-24 14:18:31	                main:574]	:	INFO	:	Epoch 787 | loss: nan | val_loss: nan | Time: 8524.13 ms
[2022-01-24 14:18:40	                main:574]	:	INFO	:	Epoch 788 | loss: nan | val_loss: nan | Time: 8830.69 ms
[2022-01-24 14:18:49	                main:574]	:	INFO	:	Epoch 789 | loss: nan | val_loss: nan | Time: 8792.4 ms
[2022-01-24 14:18:57	                main:574]	:	INFO	:	Epoch 790 | loss: nan | val_loss: nan | Time: 8450.61 ms
[2022-01-24 14:19:06	                main:574]	:	INFO	:	Epoch 791 | loss: nan | val_loss: nan | Time: 8933.9 ms
[2022-01-24 14:19:15	                main:574]	:	INFO	:	Epoch 792 | loss: nan | val_loss: nan | Time: 8400.34 ms
[2022-01-24 14:19:24	                main:574]	:	INFO	:	Epoch 793 | loss: nan | val_loss: nan | Time: 9184.72 ms
[2022-01-24 14:19:32	                main:574]	:	INFO	:	Epoch 794 | loss: nan | val_loss: nan | Time: 8471.56 ms
[2022-01-24 14:19:41	                main:574]	:	INFO	:	Epoch 795 | loss: nan | val_loss: nan | Time: 8894.22 ms
[2022-01-24 14:19:50	                main:574]	:	INFO	:	Epoch 796 | loss: nan | val_loss: nan | Time: 8511.67 ms
[2022-01-24 14:19:59	                main:574]	:	INFO	:	Epoch 797 | loss: nan | val_loss: nan | Time: 8895.01 ms
[2022-01-24 14:20:08	                main:574]	:	INFO	:	Epoch 798 | loss: nan | val_loss: nan | Time: 8917.29 ms
[2022-01-24 14:20:17	                main:574]	:	INFO	:	Epoch 799 | loss: nan | val_loss: nan | Time: 8763.49 ms
[2022-01-24 14:20:25	                main:574]	:	INFO	:	Epoch 800 | loss: nan | val_loss: nan | Time: 8885.14 ms
[2022-01-24 14:20:34	                main:574]	:	INFO	:	Epoch 801 | loss: nan | val_loss: nan | Time: 8370.63 ms
[2022-01-24 14:20:43	                main:574]	:	INFO	:	Epoch 802 | loss: nan | val_loss: nan | Time: 8739.25 ms
[2022-01-24 14:20:51	                main:574]	:	INFO	:	Epoch 803 | loss: nan | val_loss: nan | Time: 8400.77 ms
[2022-01-24 14:21:00	                main:574]	:	INFO	:	Epoch 804 | loss: nan | val_loss: nan | Time: 9080.95 ms
[2022-01-24 14:21:09	                main:574]	:	INFO	:	Epoch 805 | loss: nan | val_loss: nan | Time: 8786.22 ms
[2022-01-24 14:21:17	                main:574]	:	INFO	:	Epoch 806 | loss: nan | val_loss: nan | Time: 8428.14 ms
[2022-01-24 14:21:26	                main:574]	:	INFO	:	Epoch 807 | loss: nan | val_loss: nan | Time: 9131.99 ms
[2022-01-24 14:21:35	                main:574]	:	INFO	:	Epoch 808 | loss: nan | val_loss: nan | Time: 8278.25 ms
[2022-01-24 14:21:44	                main:574]	:	INFO	:	Epoch 809 | loss: nan | val_loss: nan | Time: 9066.69 ms
[2022-01-24 14:21:52	                main:574]	:	INFO	:	Epoch 810 | loss: nan | val_loss: nan | Time: 8214.93 ms
[2022-01-24 14:22:01	                main:574]	:	INFO	:	Epoch 811 | loss: nan | val_loss: nan | Time: 8918.66 ms
[2022-01-24 14:22:10	                main:574]	:	INFO	:	Epoch 812 | loss: nan | val_loss: nan | Time: 8708.89 ms
[2022-01-24 14:22:18	                main:574]	:	INFO	:	Epoch 813 | loss: nan | val_loss: nan | Time: 8707.49 ms
[2022-01-24 14:22:27	                main:574]	:	INFO	:	Epoch 814 | loss: nan | val_loss: nan | Time: 8998.93 ms
[2022-01-24 14:22:36	                main:574]	:	INFO	:	Epoch 815 | loss: nan | val_loss: nan | Time: 8389.07 ms
[2022-01-24 14:22:45	                main:574]	:	INFO	:	Epoch 816 | loss: nan | val_loss: nan | Time: 9075.46 ms
[2022-01-24 14:22:53	                main:574]	:	INFO	:	Epoch 817 | loss: nan | val_loss: nan | Time: 8424.39 ms
[2022-01-24 14:23:02	                main:574]	:	INFO	:	Epoch 818 | loss: nan | val_loss: nan | Time: 8901.81 ms
[2022-01-24 14:23:11	                main:574]	:	INFO	:	Epoch 819 | loss: nan | val_loss: nan | Time: 8496.29 ms
[2022-01-24 14:23:20	                main:574]	:	INFO	:	Epoch 820 | loss: nan | val_loss: nan | Time: 9188.19 ms
[2022-01-24 14:23:29	                main:574]	:	INFO	:	Epoch 821 | loss: nan | val_loss: nan | Time: 8695.13 ms
[2022-01-24 14:23:37	                main:574]	:	INFO	:	Epoch 822 | loss: nan | val_loss: nan | Time: 8327.29 ms
[2022-01-24 14:23:46	                main:574]	:	INFO	:	Epoch 823 | loss: nan | val_loss: nan | Time: 8919.97 ms
[2022-01-24 14:23:54	                main:574]	:	INFO	:	Epoch 824 | loss: nan | val_loss: nan | Time: 8369.62 ms
[2022-01-24 14:24:03	                main:574]	:	INFO	:	Epoch 825 | loss: nan | val_loss: nan | Time: 8777.7 ms
[2022-01-24 14:24:12	                main:574]	:	INFO	:	Epoch 826 | loss: nan | val_loss: nan | Time: 8337.85 ms
[2022-01-24 14:24:21	                main:574]	:	INFO	:	Epoch 827 | loss: nan | val_loss: nan | Time: 9020.38 ms
[2022-01-24 14:24:29	                main:574]	:	INFO	:	Epoch 828 | loss: nan | val_loss: nan | Time: 8821.56 ms
[2022-01-24 14:24:38	                main:574]	:	INFO	:	Epoch 829 | loss: nan | val_loss: nan | Time: 8380.65 ms
[2022-01-24 14:24:47	                main:574]	:	INFO	:	Epoch 830 | loss: nan | val_loss: nan | Time: 9012.29 ms
[2022-01-24 14:24:55	                main:574]	:	INFO	:	Epoch 831 | loss: nan | val_loss: nan | Time: 8534.27 ms
[2022-01-24 14:25:04	                main:574]	:	INFO	:	Epoch 832 | loss: nan | val_loss: nan | Time: 8756.17 ms
[2022-01-24 14:25:13	                main:574]	:	INFO	:	Epoch 833 | loss: nan | val_loss: nan | Time: 8594.29 ms
[2022-01-24 14:25:21	                main:574]	:	INFO	:	Epoch 834 | loss: nan | val_loss: nan | Time: 8700.06 ms
[2022-01-24 14:25:30	                main:574]	:	INFO	:	Epoch 835 | loss: nan | val_loss: nan | Time: 8636.21 ms
[2022-01-24 14:25:39	                main:574]	:	INFO	:	Epoch 836 | loss: nan | val_loss: nan | Time: 8921.54 ms
[2022-01-24 14:25:48	                main:574]	:	INFO	:	Epoch 837 | loss: nan | val_loss: nan | Time: 8681.15 ms
[2022-01-24 14:25:56	                main:574]	:	INFO	:	Epoch 838 | loss: nan | val_loss: nan | Time: 8291.38 ms
[2022-01-24 14:26:05	                main:574]	:	INFO	:	Epoch 839 | loss: nan | val_loss: nan | Time: 8844.23 ms
[2022-01-24 14:26:13	                main:574]	:	INFO	:	Epoch 840 | loss: nan | val_loss: nan | Time: 8211.43 ms
[2022-01-24 14:26:22	                main:574]	:	INFO	:	Epoch 841 | loss: nan | val_loss: nan | Time: 8627.97 ms
[2022-01-24 14:26:30	                main:574]	:	INFO	:	Epoch 842 | loss: nan | val_loss: nan | Time: 8473.85 ms
[2022-01-24 14:26:39	                main:574]	:	INFO	:	Epoch 843 | loss: nan | val_loss: nan | Time: 8990.03 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1650 SUPER)
[2022-01-24 15:44:17	                main:435]	:	INFO	:	Set logging level to 1
[2022-01-24 15:44:17	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-01-24 15:44:17	                main:444]	:	INFO	:	Resolving all filenames
[2022-01-24 15:44:17	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-01-24 15:44:17	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-01-24 15:44:17	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-01-24 15:44:17	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-01-24 15:44:17	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-01-24 15:44:17	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-01-24 15:44:17	                main:474]	:	INFO	:	Configuration: 
[2022-01-24 15:44:17	                main:475]	:	INFO	:	    Model type: GRU
[2022-01-24 15:44:17	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-01-24 15:44:17	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-01-24 15:44:17	                main:478]	:	INFO	:	    Batch Size: 128
[2022-01-24 15:44:17	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-01-24 15:44:17	                main:480]	:	INFO	:	    Patience: 10
[2022-01-24 15:44:17	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-01-24 15:44:17	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-01-24 15:44:17	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-01-24 15:44:17	                main:484]	:	INFO	:	    # Threads: 1
[2022-01-24 15:44:17	                main:486]	:	INFO	:	Preparing Dataset
[2022-01-24 15:44:17	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-01-24 15:44:18	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
[2022-01-24 15:44:20	                load:106]	:	INFO	:	Successfully loaded dataset of 2048 examples into memory.
[2022-01-24 15:44:20	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xv from dataset.hdf5 into memory
[2022-01-24 15:44:20	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yv from dataset.hdf5 into memory
[2022-01-24 15:44:20	                load:106]	:	INFO	:	Successfully loaded dataset of 512 examples into memory.
[2022-01-24 15:44:20	                main:494]	:	INFO	:	Creating Model
[2022-01-24 15:44:20	                main:507]	:	INFO	:	Preparing config file
[2022-01-24 15:44:20	                main:511]	:	INFO	:	Found checkpoint, attempting to load... 
[2022-01-24 15:44:20	                main:512]	:	INFO	:	Loading config


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFD6905478C

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 01/24/22 15:44:20
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\6;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 00000000c9500000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000006b980000 0000000000207000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.22000.434 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.434

ModLoad: 000000006a620000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.22000.434 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.434

ModLoad: 0000000069010000 0000000000374000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.22000.434 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.434

ModLoad: 0000000069e60000 00000000001ac000 C:\WINDOWS\System32\USER32.dll (6.2.22000.282) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000068fe0000 0000000000026000 C:\WINDOWS\System32\win32u.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.22000.434 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.434

ModLoad: 000000006b910000 0000000000029000 C:\WINDOWS\System32\GDI32.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000049810000 0000000000058000 C:\ProgramData\BOINC\slots\6\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000069680000 0000000000112000 C:\WINDOWS\System32\gdi32full.dll (6.2.22000.71) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.22000.71 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.71

ModLoad: 00000000695e0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000069390000 0000000000111000 C:\WINDOWS\System32\ucrtbase.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000069830000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 000000006a250000 00000000000a3000 C:\WINDOWS\System32\msvcrt.dll (7.0.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.22000.1

ModLoad: 000000006adb0000 000000000009e000 C:\WINDOWS\System32\sechost.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 000000006a910000 0000000000120000 C:\WINDOWS\System32\RPCRT4.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000057fa0000 0000000000221000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000045110000 0000000000099000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.23.27820.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.23.27820.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174; 2017
    Product Version       : 14.23.27820.0

ModLoad: 0000000066290000 0000000000016000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.23.27820.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.23.27820.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174; 2017
    Product Version       : 14.23.27820.0

ModLoad: 000000005c300000 000000000000b000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.23.27820.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.23.27820.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174; 2017
    Product Version       : 14.23.27820.0

ModLoad: 00000000d9a30000 000000000abe7000 C:\ProgramData\BOINC\slots\6\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000027b10000 00000000002db000 C:\ProgramData\BOINC\slots\6\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000029ed0000 000000000020d000 C:\ProgramData\BOINC\slots\6\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 0000000049ae0000 0000000000040000 C:\ProgramData\BOINC\slots\6\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e4620000 0000000024b49000 C:\ProgramData\BOINC\slots\6\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000497c0000 0000000000045000 C:\ProgramData\BOINC\slots\6\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000d34c0000 00000000034b2000 C:\ProgramData\BOINC\slots\6\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000d6980000 00000000030a5000 C:\ProgramData\BOINC\slots\6\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 0000000058ef0000 0000000000010000 C:\ProgramData\BOINC\slots\6\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 00000000ab210000 0000000003d5b000 C:\ProgramData\BOINC\slots\6\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000aef70000 000000000919f000 C:\ProgramData\BOINC\slots\6\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000a9340000 0000000001eca000 C:\ProgramData\BOINC\slots\6\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000b8110000 000000001b3a4000 C:\ProgramData\BOINC\slots\6\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 000000006a4d0000 0000000000031000 C:\WINDOWS\System32\IMM32.DLL (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000068060000 0000000000018000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.22000.71) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.22000.71 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.71

ModLoad: 0000000066690000 0000000000038000 C:\WINDOWS\SYSTEM32\dxcore.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : DXCore.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000699d0000 000000000046b000 C:\WINDOWS\System32\Setupapi.dll (6.2.22000.194) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000068b90000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000068b40000 000000000004c000 C:\WINDOWS\SYSTEM32\cfgmgr32.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000694b0000 0000000000066000 C:\WINDOWS\System32\WINTRUST.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.22000.434 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.434

ModLoad: 0000000068e70000 0000000000162000 C:\WINDOWS\System32\CRYPT32.dll (6.2.22000.348) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000068730000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 000000006ae50000 00000000007ae000 C:\WINDOWS\System32\Shell32.dll (6.2.22000.434) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.22000.184 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.184

ModLoad: 0000000066fb0000 0000000000867000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.22000.348) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.22000.184 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.184

ModLoad: 000000006aa30000 0000000000378000 C:\WINDOWS\System32\combase.dll (6.2.22000.282) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000066e40000 0000000000166000 C:\WINDOWS\SYSTEM32\wintypes.dll (6.2.22000.41) (-exported- Symbols Loaded)
    Linked PDB Filename   : WinTypes.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 000000006b7c0000 00000000000ea000 C:\WINDOWS\System32\SHCORE.dll (6.2.22000.71) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 000000006b8b0000 000000000005d000 C:\WINDOWS\System32\shlwapi.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000028430000 0000000000267000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.5671) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
    File Version          : 27.21.14.5671
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.1.96 driver
    Product Version       : 27.21.14.5671

ModLoad: 00000000650a0000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000a7e10000 000000000152a000 C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_1c83a5d7cffd7bff\nvcuda64.dll (27.21.14.5671) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
    File Version          : 27.21.14.5671
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.1.96 driver
    Product Version       : 27.21.14.5671

ModLoad: 000000006a020000 000000000006f000 C:\WINDOWS\System32\WS2_32.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : ws2_32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000632b0000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptnet.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000686f0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.22000.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000665b0000 00000000000ac000 C:\WINDOWS\system32\uxtheme.dll (6.2.22000.120) (-exported- Symbols Loaded)
    Linked PDB Filename   : uxtheme.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 0000000060120000 00000000006d5000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.5671) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r455\r456_61\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
    File Version          : 27.21.14.5671
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA Windows drivers
    Product Version       : 27.21.14.5671

ModLoad: 000000006b600000 000000000019a000 C:\WINDOWS\System32\OLE32.DLL (6.2.22000.120) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 10.0.22000.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.1

ModLoad: 00000000697a0000 000000000007f000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.22000.376) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.22000.376 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.22000.376



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3614, Write: 307, Other 897

- I/O Transfers Counters -
Read: 6637617, Write: 6049, Other 30186

- Paged Pool Usage -
QuotaPagedPoolUsage: 3460192, QuotaPeakPagedPoolUsage: 3460192
QuotaNonPagedPoolUsage: 41576, QuotaPeakNonPagedPoolUsage: 41640

- Virtual Memory Usage -
VirtualSize: -1282031616, PeakVirtualSize: -675647488

- Pagefile Usage -
PagefileUsage: -1282031616, PeakPagefileUsage: -1282031616

- Working Set Size -
WorkingSetSize: 1228034048, PeakWorkingSetSize: 1228034048, PageFaultCount: 427436

*** Dump of thread ID 3440 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 6250000.000000, User Time: 23437500.000000, Wait Time: 7126501.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFD6905478C

- Registers -
rax=000000006b9a8200 rbx=00000000c9864910 rcx=000000004590331d rdx=00000000f8d66040 rsi=00000000f8d668a0 rdi=0000000019930520
r8=0000000000000004 r9=00000000f8d66098 r10=0000000000000004 r11=0000000079a3aed0 r12=0000000067296c90 r13=00000000f8d66ab0
r14=00000000119eaa10 r15=0000000000000000 rip=000000006905478c rsp=00000000f8d66740 rbp=00000000f8d66a30
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
f8d66810 c977a884 00000040 0000004f 67296300 c979fd48 KERNELBASE!RaiseException+0x0 
f8d66870 c9521ca6 00000000 000001a7 6cf90b78 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d66920 c952593b 00000000 00000020 00000000 00000030 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d66a70 c9521a78 00000000 00000000 f8d66da0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d66b40 c9521cfc f8d66da0 00000000 00000080 f8d67070 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d66c00 c952c4dd f8d67070 f8d67070 f8d67070 f8d67070 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d66ef0 c951b2fe f8d67508 f8d67070 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d7fc90 c975c6f8 2c842800 00000000 00000000 2c9328d0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d7fcd0 6a6354e0 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f8d7fd00 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f8d7fd80 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 6324 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 7126299.000000

- Registers -
rax=00000000000001de rbx=000000002adf9d90 rcx=0000000000000074 rdx=000000002adf9a10 rsi=000000002adf7590 rdi=000000002adf9d90
r8=00000000650a5004 r9=0000000000000000 r10=00000000650a00f0 r11=0000000000000001 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000006ba26f14 rsp=00000000f90ff578 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f90ff570 6b996b4f 00000000 00000000 2adf7590 2adf9d90 ntdll!NtWaitForWorkViaWorkerFactory+0x0 
f90ff860 6a6354e0 00000000 00000000 00000000 00000000 ntdll!EtwNotificationRegister+0x0 
f90ff890 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f90ff910 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 9956 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 7126299.000000

- Registers -
rax=00000000000001de rbx=000000002adfa1c0 rcx=0000000000000074 rdx=000000002adf9e40 rsi=000000002adf7590 rdi=000000002adfa1c0
r8=0000000000000288 r9=0000000000000288 r10=0000000000000287 r11=000000000000028a r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000006ba26f14 rsp=00000000f91ffc08 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f91ffc00 6b996b4f 00000000 00000000 2adf7590 2adfa1c0 ntdll!NtWaitForWorkViaWorkerFactory+0x0 
f91ffef0 6a6354e0 00000000 00000000 00000000 00000000 ntdll!EtwNotificationRegister+0x0 
f91fff20 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f91fffa0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 3436 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 7126300.000000

- Registers -
rax=00000000000001de rbx=000000002adfbe60 rcx=0000000000000074 rdx=000000002adfbae0 rsi=000000002adf7590 rdi=000000002adfbe60
r8=0000000000000008 r9=0000000000000040 r10=0000000000000580 r11=0000000000f40000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000006ba26f14 rsp=00000000f92ff7d8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f92ff7d0 6b996b4f 00000000 00000000 2adf7590 2adfbe60 ntdll!NtWaitForWorkViaWorkerFactory+0x0 
f92ffac0 6a6354e0 00000000 00000000 00000000 00000000 ntdll!EtwNotificationRegister+0x0 
f92ffaf0 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f92ffb70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 4248 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 7126495.000000

- Registers -
rax=0000000000000034 rbx=00000000f94ff7b0 rcx=0000000000000000 rdx=00000000f94ff7b0 rsi=00000000f8e16000 rdi=0000000000000000
r8=00000000f94fdd3c r9=000000000000003c r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000006ba239e4 rsp=00000000f94ff6e8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f94ff6e0 6b9db4c3 00000000 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0 
f94ff710 6905b3a1 00000000 00000000 00000000 00000000 ntdll!RtlDelayExecution+0x0 
f94ff790 c953a07f 00000000 00000000 fff0bdc0 00000000 KERNELBASE!SleepEx+0x0 
f94ff7c0 6a6354e0 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
f94ff7f0 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f94ff870 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 10476 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 7126496.000000

- Registers -
rax=000000000000005b rbx=0000000000000007 rcx=0000000000000007 rdx=00000000f95ffbc0 rsi=0000000000000000 rdi=0000000000000007
r8=000000002ccd7090 r9=000000002ca8a100 r10=0000000000000000 r11=00000000f95ffb30 r12=0000000000000064 r13=00000000f95ffbc0
r14=00000000f95ff8c0 r15=0000000000000000 rip=000000006ba23eb4 rsp=00000000f95ff868 rbp=000000002ca8a120
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f95ff860 6905fb10 00000000 00000000 00000000 00000000 ntdll!NtWaitForMultipleObjects+0x0 
f95ffb50 6905fa0e a8303b48 2cb19780 00000007 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
f95ffb90 a7fbba44 2ca8fa00 00000000 2ca889e0 00000000 KERNELBASE!WaitForMultipleObjects+0x0 
f95ffdd0 a80508dd 00000000 00000004 00000000 00000000 nvcuda64!+0x0 
f95ffe80 a7fbb413 43169adf 41a72585 2cf5d150 00000000 nvcuda64!cuProfilerStop+0x0 
f95ffeb0 a82c9200 2cf5d150 00000000 00000000 00000000 nvcuda64!+0x0 
f95ffee0 6a6354e0 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
f95fff10 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f95fff90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 11016 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 7126472.000000

- Registers -
rax=00000000000001db rbx=0000000000000000 rcx=000000002ccda258 rdx=00000000f96ffe68 rsi=0000000000000102 rdi=0000000000000000
r8=0000000000000004 r9=0000000000000004 r10=0000000000000003 r11=00000000f96ffe00 r12=00000000f96ffe68 r13=0000000000000000
r14=000000002ccde9d8 r15=000000002ccda258 rip=000000006ba26eb4 rsp=00000000f96ffda8 rbp=00000000f96ffe00
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
f96ffda0 6b9ea1da 00000102 00000102 00000000 f96ffdd0 ntdll!NtWaitForAlertByThreadId+0x0 
f96ffe10 6906d449 2ccdc1c0 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0 
f96ffe40 a7f421da 2ccdc1c0 6b986770 2ccdc1c0 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0 
f96ffea0 a7fbb413 00000001 42d89fae 00000000 00000000 nvcuda64!+0x0 
f96ffed0 a82c9200 2cf5dd80 00000000 00000000 00000000 nvcuda64!+0x0 
f96fff00 6a6354e0 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
f96fff30 6b98485b 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
f96fffb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)