Task 10543577

Name ParityModified-1639966399-6257-1-0_0
Workunit 7878442
Created 7 Jan 2022, 18:02:33 UTC
Sent 25 Jan 2022, 14:57:42 UTC
Report deadline 2 Feb 2022, 14:57:42 UTC
Received 9 Feb 2022, 4:23:08 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 14113
Run time 1 hours 41 min 50 sec
CPU time 1 hours 41 min 7 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 8,299.00 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 2.74 GB
Peak swap size 4.63 GB
Peak disk usage 1.54 GB

Stderr output

<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
Time: 2169.79 ms
[2022-02-07 18:31:53	                main:574]	:	INFO	:	Epoch 428 | loss: 0.0311777 | val_loss: 0.0311816 | Time: 2022.78 ms
[2022-02-07 18:31:55	                main:574]	:	INFO	:	Epoch 429 | loss: 0.0311837 | val_loss: 0.0311892 | Time: 2066.39 ms
[2022-02-07 18:31:57	                main:574]	:	INFO	:	Epoch 430 | loss: 0.0311879 | val_loss: 0.0311845 | Time: 2113.48 ms
[2022-02-07 18:31:59	                main:574]	:	INFO	:	Epoch 431 | loss: 0.0311846 | val_loss: 0.0311833 | Time: 2081.3 ms
[2022-02-07 18:32:01	                main:574]	:	INFO	:	Epoch 432 | loss: 0.0311836 | val_loss: 0.0311828 | Time: 2087.87 ms
[2022-02-07 18:32:03	                main:574]	:	INFO	:	Epoch 433 | loss: 0.0311826 | val_loss: 0.0311823 | Time: 1951.16 ms
[2022-02-07 18:32:05	                main:574]	:	INFO	:	Epoch 434 | loss: 0.031182 | val_loss: 0.0311811 | Time: 1977.95 ms
[2022-02-07 18:32:07	                main:574]	:	INFO	:	Epoch 435 | loss: 0.0311819 | val_loss: 0.0311825 | Time: 2140.33 ms
[2022-02-07 18:32:09	                main:574]	:	INFO	:	Epoch 436 | loss: 0.0311823 | val_loss: 0.0311814 | Time: 1979.22 ms
[2022-02-07 18:32:11	                main:574]	:	INFO	:	Epoch 437 | loss: 0.0311814 | val_loss: 0.0311809 | Time: 2092.92 ms
[2022-02-07 18:32:13	                main:574]	:	INFO	:	Epoch 438 | loss: 0.0311809 | val_loss: 0.0311826 | Time: 1966.91 ms
[2022-02-07 18:32:15	                main:574]	:	INFO	:	Epoch 439 | loss: 0.0311815 | val_loss: 0.0311816 | Time: 1925.41 ms
[2022-02-07 18:32:17	                main:574]	:	INFO	:	Epoch 440 | loss: 0.0311811 | val_loss: 0.031184 | Time: 2080.1 ms
[2022-02-07 18:32:19	                main:574]	:	INFO	:	Epoch 441 | loss: 0.0311815 | val_loss: 0.0311813 | Time: 2073.86 ms
[2022-02-07 18:32:21	                main:574]	:	INFO	:	Epoch 442 | loss: 0.0311823 | val_loss: 0.0311876 | Time: 2008.02 ms
[2022-02-07 18:32:23	                main:574]	:	INFO	:	Epoch 443 | loss: 0.031185 | val_loss: 0.031184 | Time: 1999.36 ms
[2022-02-07 18:32:25	                main:574]	:	INFO	:	Epoch 444 | loss: 0.0311835 | val_loss: 0.0311895 | Time: 1905.78 ms
[2022-02-07 18:32:27	                main:574]	:	INFO	:	Epoch 445 | loss: 0.0311843 | val_loss: 0.03119 | Time: 2064.18 ms
[2022-02-07 18:32:30	                main:574]	:	INFO	:	Epoch 446 | loss: 0.031182 | val_loss: 0.0311805 | Time: 2149.26 ms
[2022-02-07 18:32:32	                main:574]	:	INFO	:	Epoch 447 | loss: 0.0311801 | val_loss: 0.0311809 | Time: 2009.56 ms
[2022-02-07 18:32:34	                main:574]	:	INFO	:	Epoch 448 | loss: 0.031181 | val_loss: 0.0311836 | Time: 2089.03 ms
[2022-02-07 18:32:36	                main:574]	:	INFO	:	Epoch 449 | loss: 0.0311803 | val_loss: 0.0311814 | Time: 2109.56 ms
[2022-02-07 18:32:38	                main:574]	:	INFO	:	Epoch 450 | loss: 0.0311802 | val_loss: 0.0311804 | Time: 2085.57 ms
[2022-02-07 18:32:40	                main:574]	:	INFO	:	Epoch 451 | loss: 0.0311815 | val_loss: 0.0311816 | Time: 2076.93 ms
[2022-02-07 18:32:42	                main:574]	:	INFO	:	Epoch 452 | loss: 0.0311799 | val_loss: 0.0311818 | Time: 1973.65 ms
[2022-02-07 18:32:44	                main:574]	:	INFO	:	Epoch 453 | loss: 0.0311803 | val_loss: 0.0311802 | Time: 2019.02 ms
[2022-02-07 18:32:46	                main:574]	:	INFO	:	Epoch 454 | loss: 0.0311796 | val_loss: 0.0311806 | Time: 2131.68 ms
[2022-02-07 18:32:48	                main:574]	:	INFO	:	Epoch 455 | loss: 0.0311797 | val_loss: 0.0311874 | Time: 2046.17 ms
[2022-02-07 18:32:50	                main:574]	:	INFO	:	Epoch 456 | loss: 0.0311809 | val_loss: 0.0311834 | Time: 1979.51 ms
[2022-02-07 18:32:52	                main:574]	:	INFO	:	Epoch 457 | loss: 0.0311791 | val_loss: 0.0311808 | Time: 2112.93 ms
[2022-02-07 18:32:54	                main:574]	:	INFO	:	Epoch 458 | loss: 0.0311787 | val_loss: 0.0311814 | Time: 2024.8 ms
[2022-02-07 18:32:56	                main:574]	:	INFO	:	Epoch 459 | loss: 0.0311805 | val_loss: 0.0311876 | Time: 2035.89 ms
[2022-02-07 18:32:58	                main:574]	:	INFO	:	Epoch 460 | loss: 0.0311809 | val_loss: 0.031182 | Time: 2109.33 ms
[2022-02-07 18:33:00	                main:574]	:	INFO	:	Epoch 461 | loss: 0.0311795 | val_loss: 0.0311819 | Time: 2066.77 ms
[2022-02-07 18:33:03	                main:574]	:	INFO	:	Epoch 462 | loss: 0.0311797 | val_loss: 0.0311804 | Time: 2028.65 ms
[2022-02-07 18:33:04	                main:574]	:	INFO	:	Epoch 463 | loss: 0.0311813 | val_loss: 0.0311852 | Time: 1943.15 ms
[2022-02-07 18:33:07	                main:574]	:	INFO	:	Epoch 464 | loss: 0.0311815 | val_loss: 0.0311804 | Time: 2065.58 ms
[2022-02-07 18:33:09	                main:574]	:	INFO	:	Epoch 465 | loss: 0.0311803 | val_loss: 0.031182 | Time: 2191.93 ms
[2022-02-07 18:33:11	                main:574]	:	INFO	:	Epoch 466 | loss: 0.0311787 | val_loss: 0.0311806 | Time: 1973.82 ms
[2022-02-07 18:33:13	                main:574]	:	INFO	:	Epoch 467 | loss: 0.0311783 | val_loss: 0.0311809 | Time: 2099.48 ms
[2022-02-07 18:33:15	                main:574]	:	INFO	:	Epoch 468 | loss: 0.0311789 | val_loss: 0.0311831 | Time: 2112.97 ms
[2022-02-07 18:33:17	                main:574]	:	INFO	:	Epoch 469 | loss: 0.0311799 | val_loss: 0.0311839 | Time: 1924.14 ms
[2022-02-07 18:33:19	                main:574]	:	INFO	:	Epoch 470 | loss: 0.0311799 | val_loss: 0.031183 | Time: 2071.64 ms
[2022-02-07 18:33:21	                main:574]	:	INFO	:	Epoch 471 | loss: 0.0311791 | val_loss: 0.0311818 | Time: 2022.74 ms
[2022-02-07 18:33:23	                main:574]	:	INFO	:	Epoch 472 | loss: 0.0311786 | val_loss: 0.03118 | Time: 2003.69 ms
[2022-02-07 18:33:25	                main:574]	:	INFO	:	Epoch 473 | loss: 0.0311775 | val_loss: 0.0311805 | Time: 2155.72 ms
[2022-02-07 18:33:27	                main:574]	:	INFO	:	Epoch 474 | loss: 0.0311794 | val_loss: 0.0311803 | Time: 2137.13 ms
[2022-02-07 18:33:29	                main:574]	:	INFO	:	Epoch 475 | loss: 0.0311783 | val_loss: 0.0311829 | Time: 2160.17 ms
[2022-02-07 18:33:32	                main:574]	:	INFO	:	Epoch 476 | loss: 0.0311772 | val_loss: 0.0311803 | Time: 2183.4 ms
[2022-02-07 18:33:34	                main:574]	:	INFO	:	Epoch 477 | loss: 0.0311768 | val_loss: 0.0311806 | Time: 2058.1 ms
[2022-02-07 18:33:36	                main:574]	:	INFO	:	Epoch 478 | loss: 0.0311791 | val_loss: 0.0311841 | Time: 2131.15 ms
[2022-02-07 18:33:38	                main:574]	:	INFO	:	Epoch 479 | loss: 0.0311799 | val_loss: 0.0311798 | Time: 2104.84 ms
[2022-02-07 18:33:40	                main:574]	:	INFO	:	Epoch 480 | loss: 0.0311782 | val_loss: 0.0311818 | Time: 2069.14 ms
[2022-02-07 18:33:42	                main:574]	:	INFO	:	Epoch 481 | loss: 0.0311809 | val_loss: 0.0311848 | Time: 2137.34 ms
[2022-02-07 18:33:44	                main:574]	:	INFO	:	Epoch 482 | loss: 0.031178 | val_loss: 0.03118 | Time: 2058.14 ms
[2022-02-07 18:33:46	                main:574]	:	INFO	:	Epoch 483 | loss: 0.0311776 | val_loss: 0.0311804 | Time: 2052.08 ms
[2022-02-07 18:33:48	                main:574]	:	INFO	:	Epoch 484 | loss: 0.031178 | val_loss: 0.0311816 | Time: 2103.34 ms
[2022-02-07 18:33:50	                main:574]	:	INFO	:	Epoch 485 | loss: 0.0311778 | val_loss: 0.0311803 | Time: 2016.1 ms
[2022-02-07 18:33:52	                main:574]	:	INFO	:	Epoch 486 | loss: 0.0311779 | val_loss: 0.0311805 | Time: 2047.97 ms
[2022-02-07 18:33:54	                main:574]	:	INFO	:	Epoch 487 | loss: 0.0311774 | val_loss: 0.0311805 | Time: 2026.93 ms
[2022-02-07 18:33:57	                main:574]	:	INFO	:	Epoch 488 | loss: 0.0311774 | val_loss: 0.0311805 | Time: 2041.76 ms
[2022-02-07 18:33:59	                main:574]	:	INFO	:	Epoch 489 | loss: 0.031177 | val_loss: 0.031182 | Time: 2191.81 ms
[2022-02-07 18:34:01	                main:574]	:	INFO	:	Epoch 490 | loss: 0.0311779 | val_loss: 0.0311805 | Time: 2081.52 ms
[2022-02-07 18:34:03	                main:574]	:	INFO	:	Epoch 491 | loss: 0.0311791 | val_loss: 0.0311814 | Time: 2013.88 ms
[2022-02-07 18:34:05	                main:574]	:	INFO	:	Epoch 492 | loss: 0.0311768 | val_loss: 0.03118 | Time: 2016.79 ms
[2022-02-07 18:34:07	                main:574]	:	INFO	:	Epoch 493 | loss: 0.0311767 | val_loss: 0.0311841 | Time: 2003.49 ms
[2022-02-07 18:34:09	                main:574]	:	INFO	:	Epoch 494 | loss: 0.0311774 | val_loss: 0.0311814 | Time: 2061.7 ms
[2022-02-07 18:34:11	                main:574]	:	INFO	:	Epoch 495 | loss: 0.0311766 | val_loss: 0.0311792 | Time: 2061.99 ms
[2022-02-07 18:34:13	                main:574]	:	INFO	:	Epoch 496 | loss: 0.0311759 | val_loss: 0.0311806 | Time: 2049.82 ms
[2022-02-07 18:34:15	                main:574]	:	INFO	:	Epoch 497 | loss: 0.0311758 | val_loss: 0.0311804 | Time: 2113.27 ms
[2022-02-07 18:34:17	                main:574]	:	INFO	:	Epoch 498 | loss: 0.0311755 | val_loss: 0.03118 | Time: 1980.69 ms
[2022-02-07 18:34:19	                main:574]	:	INFO	:	Epoch 499 | loss: 0.031176 | val_loss: 0.031182 | Time: 2096.22 ms
[2022-02-07 18:34:21	                main:574]	:	INFO	:	Epoch 500 | loss: 0.0311765 | val_loss: 0.0311804 | Time: 2059.56 ms
[2022-02-07 18:34:23	                main:574]	:	INFO	:	Epoch 501 | loss: 0.031179 | val_loss: 0.0311831 | Time: 2094.74 ms
[2022-02-07 18:34:25	                main:574]	:	INFO	:	Epoch 502 | loss: 0.0311759 | val_loss: 0.0311799 | Time: 2012.8 ms
[2022-02-07 18:34:28	                main:574]	:	INFO	:	Epoch 503 | loss: 0.0311768 | val_loss: 0.0311815 | Time: 2107.36 ms
[2022-02-07 18:34:30	                main:574]	:	INFO	:	Epoch 504 | loss: 0.0311782 | val_loss: 0.0311817 | Time: 2063.03 ms
[2022-02-07 18:34:32	                main:574]	:	INFO	:	Epoch 505 | loss: 0.0311761 | val_loss: 0.0311799 | Time: 2121.59 ms
[2022-02-07 18:34:34	                main:574]	:	INFO	:	Epoch 506 | loss: 0.0311758 | val_loss: 0.0311808 | Time: 1973.3 ms
[2022-02-07 18:34:36	                main:574]	:	INFO	:	Epoch 507 | loss: 0.0311763 | val_loss: 0.0311798 | Time: 1989.66 ms
[2022-02-07 18:34:38	                main:574]	:	INFO	:	Epoch 508 | loss: 0.0311761 | val_loss: 0.0311796 | Time: 2119.45 ms
[2022-02-07 18:34:40	                main:574]	:	INFO	:	Epoch 509 | loss: 0.0311757 | val_loss: 0.031183 | Time: 2075.86 ms
[2022-02-07 18:34:42	                main:574]	:	INFO	:	Epoch 510 | loss: 0.0311763 | val_loss: 0.0311795 | Time: 2027.34 ms
[2022-02-07 18:34:44	                main:574]	:	INFO	:	Epoch 511 | loss: 0.0311761 | val_loss: 0.0311801 | Time: 2034.61 ms
[2022-02-07 18:34:46	                main:574]	:	INFO	:	Epoch 512 | loss: 0.031175 | val_loss: 0.031179 | Time: 2016.6 ms
[2022-02-07 18:34:48	                main:574]	:	INFO	:	Epoch 513 | loss: 0.0311757 | val_loss: 0.0311793 | Time: 2067.68 ms
[2022-02-07 18:34:50	                main:574]	:	INFO	:	Epoch 514 | loss: 0.0311763 | val_loss: 0.0311797 | Time: 2085.15 ms
[2022-02-07 18:34:52	                main:574]	:	INFO	:	Epoch 515 | loss: 0.0311764 | val_loss: 0.0311812 | Time: 2008.98 ms
[2022-02-07 18:34:54	                main:574]	:	INFO	:	Epoch 516 | loss: 0.0311762 | val_loss: 0.0311794 | Time: 2125.37 ms
[2022-02-07 18:34:56	                main:574]	:	INFO	:	Epoch 517 | loss: 0.0311762 | val_loss: 0.0311793 | Time: 2075.53 ms
[2022-02-07 18:34:58	                main:574]	:	INFO	:	Epoch 518 | loss: 0.0311758 | val_loss: 0.0311805 | Time: 2076.46 ms
[2022-02-07 18:35:01	                main:574]	:	INFO	:	Epoch 519 | loss: 0.0311778 | val_loss: 0.0311794 | Time: 2054.43 ms
[2022-02-07 18:35:03	                main:574]	:	INFO	:	Epoch 520 | loss: 0.0311762 | val_loss: 0.0311807 | Time: 2080.95 ms
[2022-02-07 18:35:05	                main:574]	:	INFO	:	Epoch 521 | loss: 0.031177 | val_loss: 0.0311807 | Time: 2015.37 ms
[2022-02-07 18:35:07	                main:574]	:	INFO	:	Epoch 522 | loss: 0.0311762 | val_loss: 0.0311785 | Time: 2013.86 ms
[2022-02-07 18:35:09	                main:574]	:	INFO	:	Epoch 523 | loss: 0.0311759 | val_loss: 0.0311789 | Time: 1983.58 ms
[2022-02-07 18:35:11	                main:574]	:	INFO	:	Epoch 524 | loss: 0.0311787 | val_loss: 0.0311851 | Time: 2121.37 ms
[2022-02-07 18:35:13	                main:574]	:	INFO	:	Epoch 525 | loss: 0.0311769 | val_loss: 0.0311795 | Time: 2064.38 ms
[2022-02-07 18:35:15	                main:574]	:	INFO	:	Epoch 526 | loss: 0.0311756 | val_loss: 0.0311785 | Time: 1988.71 ms
[2022-02-07 18:35:17	                main:574]	:	INFO	:	Epoch 527 | loss: 0.0311751 | val_loss: 0.0311784 | Time: 2014.84 ms
[2022-02-07 18:35:19	                main:574]	:	INFO	:	Epoch 528 | loss: 0.0311752 | val_loss: 0.0311785 | Time: 1992.76 ms
[2022-02-07 18:35:21	                main:574]	:	INFO	:	Epoch 529 | loss: 0.0311749 | val_loss: 0.0311785 | Time: 2100.53 ms
[2022-02-07 18:35:23	                main:574]	:	INFO	:	Epoch 530 | loss: 0.0311762 | val_loss: 0.0311783 | Time: 2057.45 ms
[2022-02-07 18:35:25	                main:574]	:	INFO	:	Epoch 531 | loss: 0.0311777 | val_loss: 0.0311796 | Time: 2078.29 ms
[2022-02-07 18:35:27	                main:574]	:	INFO	:	Epoch 532 | loss: 0.0311776 | val_loss: 0.0311816 | Time: 2168.27 ms
[2022-02-07 18:35:29	                main:574]	:	INFO	:	Epoch 533 | loss: 0.0311774 | val_loss: 0.0311794 | Time: 2014.51 ms
[2022-02-07 18:35:31	                main:574]	:	INFO	:	Epoch 534 | loss: 0.0311762 | val_loss: 0.0311788 | Time: 1982.96 ms
[2022-02-07 18:35:33	                main:574]	:	INFO	:	Epoch 535 | loss: 0.0311754 | val_loss: 0.0311816 | Time: 2129.99 ms
[2022-02-07 18:35:35	                main:574]	:	INFO	:	Epoch 536 | loss: 0.0311763 | val_loss: 0.0311788 | Time: 1983.38 ms
[2022-02-07 18:35:37	                main:574]	:	INFO	:	Epoch 537 | loss: 0.0311768 | val_loss: 0.0311792 | Time: 2050.78 ms
[2022-02-07 18:35:40	                main:574]	:	INFO	:	Epoch 538 | loss: 0.0311774 | val_loss: 0.0311799 | Time: 2119.22 ms
[2022-02-07 18:35:42	                main:574]	:	INFO	:	Epoch 539 | loss: 0.0311761 | val_loss: 0.0311785 | Time: 2119.5 ms
[2022-02-07 18:35:44	                main:574]	:	INFO	:	Epoch 540 | loss: 0.0311752 | val_loss: 0.0311821 | Time: 2244.92 ms
[2022-02-07 18:35:46	                main:574]	:	INFO	:	Epoch 541 | loss: 0.0311764 | val_loss: 0.0311806 | Time: 2068.74 ms
[2022-02-07 18:35:48	                main:574]	:	INFO	:	Epoch 542 | loss: 0.0311749 | val_loss: 0.0311789 | Time: 2090.97 ms
[2022-02-07 18:35:50	                main:574]	:	INFO	:	Epoch 543 | loss: 0.0311759 | val_loss: 0.0311795 | Time: 2019.17 ms
[2022-02-07 18:35:52	                main:574]	:	INFO	:	Epoch 544 | loss: 0.0311777 | val_loss: 0.0311794 | Time: 1989.23 ms
[2022-02-07 18:35:54	                main:574]	:	INFO	:	Epoch 545 | loss: 0.0311745 | val_loss: 0.0311794 | Time: 2074.51 ms
[2022-02-07 18:35:56	                main:574]	:	INFO	:	Epoch 546 | loss: 0.0311756 | val_loss: 0.0311786 | Time: 2125.82 ms
[2022-02-07 18:35:58	                main:574]	:	INFO	:	Epoch 547 | loss: 0.031177 | val_loss: 0.0311816 | Time: 1967.72 ms
[2022-02-07 18:36:00	                main:574]	:	INFO	:	Epoch 548 | loss: 0.0311769 | val_loss: 0.0311789 | Time: 2143.06 ms
[2022-02-07 18:36:02	                main:574]	:	INFO	:	Epoch 549 | loss: 0.0311748 | val_loss: 0.0311817 | Time: 2038.63 ms
[2022-02-07 18:36:05	                main:574]	:	INFO	:	Epoch 550 | loss: 0.0311743 | val_loss: 0.0311793 | Time: 2017.6 ms
[2022-02-07 18:36:07	                main:574]	:	INFO	:	Epoch 551 | loss: 0.0311741 | val_loss: 0.0311785 | Time: 2101.72 ms
[2022-02-07 18:36:09	                main:574]	:	INFO	:	Epoch 552 | loss: 0.0311747 | val_loss: 0.0311793 | Time: 1952.98 ms
[2022-02-07 18:36:11	                main:574]	:	INFO	:	Epoch 553 | loss: 0.0311741 | val_loss: 0.0311787 | Time: 2044.23 ms
[2022-02-07 18:36:13	                main:574]	:	INFO	:	Epoch 554 | loss: 0.0311746 | val_loss: 0.0311829 | Time: 1994.99 ms
[2022-02-07 18:36:15	                main:574]	:	INFO	:	Epoch 555 | loss: 0.031175 | val_loss: 0.031185 | Time: 2089.7 ms
[2022-02-07 18:36:17	                main:574]	:	INFO	:	Epoch 556 | loss: 0.0311754 | val_loss: 0.0311789 | Time: 2385.78 ms
[2022-02-07 18:36:19	                main:574]	:	INFO	:	Epoch 557 | loss: 0.0311759 | val_loss: 0.0311797 | Time: 2175.96 ms
[2022-02-07 18:36:21	                main:574]	:	INFO	:	Epoch 558 | loss: 0.0311742 | val_loss: 0.0311784 | Time: 2187.23 ms
[2022-02-07 18:36:24	                main:574]	:	INFO	:	Epoch 559 | loss: 0.0311734 | val_loss: 0.0311784 | Time: 2197.08 ms
[2022-02-07 18:36:26	                main:574]	:	INFO	:	Epoch 560 | loss: 0.0311741 | val_loss: 0.0311793 | Time: 2058.58 ms
[2022-02-07 18:36:28	                main:574]	:	INFO	:	Epoch 561 | loss: 0.0311768 | val_loss: 0.0311792 | Time: 2084.03 ms
[2022-02-07 18:36:30	                main:574]	:	INFO	:	Epoch 562 | loss: 0.031174 | val_loss: 0.0311786 | Time: 2078.65 ms
[2022-02-07 18:36:32	                main:574]	:	INFO	:	Epoch 563 | loss: 0.0311734 | val_loss: 0.0311815 | Time: 1984.02 ms
[2022-02-07 18:36:34	                main:574]	:	INFO	:	Epoch 564 | loss: 0.0311748 | val_loss: 0.0311817 | Time: 2053.2 ms
[2022-02-07 18:36:36	                main:574]	:	INFO	:	Epoch 565 | loss: 0.0311743 | val_loss: 0.0311784 | Time: 2100.96 ms
[2022-02-07 18:36:38	                main:574]	:	INFO	:	Epoch 566 | loss: 0.0311736 | val_loss: 0.0311833 | Time: 2044.5 ms
[2022-02-07 18:36:40	                main:574]	:	INFO	:	Epoch 567 | loss: 0.0311751 | val_loss: 0.0311792 | Time: 2076.87 ms
[2022-02-07 18:36:42	                main:574]	:	INFO	:	Epoch 568 | loss: 0.0311737 | val_loss: 0.0311799 | Time: 2019.54 ms
[2022-02-07 18:36:44	                main:574]	:	INFO	:	Epoch 569 | loss: 0.0311741 | val_loss: 0.0311791 | Time: 2033.93 ms
[2022-02-07 18:36:46	                main:574]	:	INFO	:	Epoch 570 | loss: 0.0311744 | val_loss: 0.0311798 | Time: 2074.38 ms
[2022-02-07 18:36:48	                main:574]	:	INFO	:	Epoch 571 | loss: 0.0311732 | val_loss: 0.0311805 | Time: 1970.09 ms
[2022-02-07 18:36:50	                main:574]	:	INFO	:	Epoch 572 | loss: 0.0311739 | val_loss: 0.0311795 | Time: 2049.51 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-07 18:37:29	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-07 18:37:29	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-07 18:37:29	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-07 18:37:29	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-07 18:37:29	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-07 18:37:29	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-07 18:37:29	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-07 18:37:29	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-07 18:37:29	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-07 18:37:29	                main:474]	:	INFO	:	Configuration: 
[2022-02-07 18:37:29	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-07 18:37:29	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-07 18:37:29	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-07 18:37:29	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-07 18:37:29	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-07 18:37:29	                main:480]	:	INFO	:	    Patience: 10
[2022-02-07 18:37:29	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-07 18:37:30	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-07 18:37:30	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-07 18:37:30	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-07 18:37:30	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-07 18:37:30	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-07 18:37:30	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-07 19:02:27	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-07 19:02:27	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-07 19:02:27	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-07 19:02:27	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-07 19:02:27	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-07 19:02:27	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-07 19:02:27	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-07 19:02:27	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-07 19:02:27	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-07 19:02:27	                main:474]	:	INFO	:	Configuration: 
[2022-02-07 19:02:27	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-07 19:02:27	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-07 19:02:27	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-07 19:02:27	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-07 19:02:27	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-07 19:02:27	                main:480]	:	INFO	:	    Patience: 10
[2022-02-07 19:02:27	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-07 19:02:27	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-07 19:02:27	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-07 19:02:27	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-07 19:02:27	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-07 19:02:27	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-07 19:02:28	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-07 19:31:50	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-07 19:31:50	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-07 19:31:50	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-07 19:31:50	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-07 19:31:50	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-07 19:31:50	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-07 19:31:50	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-07 19:31:50	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-07 19:31:50	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-07 19:31:50	                main:474]	:	INFO	:	Configuration: 
[2022-02-07 19:31:50	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-07 19:31:50	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-07 19:31:50	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-07 19:31:50	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-07 19:31:50	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-07 19:31:50	                main:480]	:	INFO	:	    Patience: 10
[2022-02-07 19:31:50	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-07 19:31:50	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-07 19:31:50	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-07 19:31:50	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-07 19:31:50	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-07 19:31:50	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-07 19:31:50	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-07 20:16:43	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-07 20:16:43	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-07 20:16:43	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-07 20:16:43	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-07 20:16:43	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-07 20:16:43	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-07 20:16:43	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-07 20:16:43	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-07 20:16:43	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-07 20:16:43	                main:474]	:	INFO	:	Configuration: 
[2022-02-07 20:16:43	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-07 20:16:43	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-07 20:16:43	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-07 20:16:43	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-07 20:16:43	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-07 20:16:43	                main:480]	:	INFO	:	    Patience: 10
[2022-02-07 20:16:43	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-07 20:16:43	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-07 20:16:43	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-07 20:16:43	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-07 20:16:43	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-07 20:16:43	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-07 20:16:43	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-08 06:23:27	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-08 06:23:27	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-08 06:23:27	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-08 06:23:27	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-08 06:23:27	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-08 06:23:27	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-08 06:23:28	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-08 06:23:28	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-08 06:23:28	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-08 06:23:28	                main:474]	:	INFO	:	Configuration: 
[2022-02-08 06:23:28	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-08 06:23:28	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-08 06:23:28	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-08 06:23:28	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-08 06:23:28	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-08 06:23:28	                main:480]	:	INFO	:	    Patience: 10
[2022-02-08 06:23:28	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-08 06:23:28	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-08 06:23:28	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-08 06:23:28	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-08 06:23:28	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-08 06:23:28	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-08 06:23:28	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-08 19:14:45	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-08 19:14:45	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-08 19:14:45	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-08 19:14:45	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-08 19:14:45	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-08 19:14:45	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-08 19:14:45	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-08 19:14:45	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-08 19:14:45	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-08 19:14:45	                main:474]	:	INFO	:	Configuration: 
[2022-02-08 19:14:45	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-08 19:14:45	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-08 19:14:45	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-08 19:14:45	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-08 19:14:45	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-08 19:14:45	                main:480]	:	INFO	:	    Patience: 10
[2022-02-08 19:14:45	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-08 19:14:45	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-08 19:14:45	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-08 19:14:45	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-08 19:14:45	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-08 19:14:45	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-08 19:14:46	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-08 19:24:48	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-08 19:24:48	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-08 19:24:48	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-08 19:24:48	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-08 19:24:48	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-08 19:24:48	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-08 19:24:48	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-08 19:24:48	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-08 19:24:48	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-08 19:24:48	                main:474]	:	INFO	:	Configuration: 
[2022-02-08 19:24:48	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-08 19:24:48	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-08 19:24:48	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-08 19:24:48	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-08 19:24:48	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-08 19:24:48	                main:480]	:	INFO	:	    Patience: 10
[2022-02-08 19:24:48	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-08 19:24:48	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-08 19:24:48	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-08 19:24:48	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-08 19:24:48	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-08 19:24:48	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-08 19:24:48	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3070 Laptop GPU)
[2022-02-08 20:11:22	                main:435]	:	INFO	:	Set logging level to 1
[2022-02-08 20:11:22	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-02-08 20:11:22	                main:444]	:	INFO	:	Resolving all filenames
[2022-02-08 20:11:22	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-02-08 20:11:22	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-02-08 20:11:22	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-02-08 20:11:22	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-02-08 20:11:22	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-02-08 20:11:22	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-02-08 20:11:22	                main:474]	:	INFO	:	Configuration: 
[2022-02-08 20:11:22	                main:475]	:	INFO	:	    Model type: GRU
[2022-02-08 20:11:22	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-02-08 20:11:22	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-02-08 20:11:22	                main:478]	:	INFO	:	    Batch Size: 128
[2022-02-08 20:11:22	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-02-08 20:11:22	                main:480]	:	INFO	:	    Patience: 10
[2022-02-08 20:11:22	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-02-08 20:11:22	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-02-08 20:11:22	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-02-08 20:11:22	                main:484]	:	INFO	:	    # Threads: 1
[2022-02-08 20:11:22	                main:486]	:	INFO	:	Preparing Dataset
[2022-02-08 20:11:22	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-02-08 20:11:22	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFE79FD4F69

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 02/08/22 20:21:14
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\4;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 00000000e4ec0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000007c430000 00000000001f5000 C:\Windows\SYSTEM32\ntdll.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 000000007a580000 00000000000be000 C:\Windows\System32\KERNEL32.DLL (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 0000000079fa0000 00000000002c8000 C:\Windows\System32\KERNELBASE.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 000000007af90000 00000000001a1000 C:\Windows\System32\USER32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000079f70000 0000000000022000 C:\Windows\System32\win32u.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 000000007a640000 000000000002b000 C:\Windows\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.19041.1202 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1202

ModLoad: 0000000079e60000 000000000010d000 C:\Windows\System32\gdi32full.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 000000007a270000 000000000009d000 C:\Windows\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.19041.789 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.789

ModLoad: 0000000079d00000 0000000000100000 C:\Windows\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.19041.789 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.789

ModLoad: 000000007c110000 00000000000ae000 C:\Windows\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000007aef0000 000000000009e000 C:\Windows\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.19041.546

ModLoad: 000000007a670000 000000000009c000 C:\Windows\System32\sechost.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000007aa00000 0000000000125000 C:\Windows\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000069480000 0000000000058000 C:\ProgramData\BOINC\slots\4\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000048090000 000000000008d000 C:\Windows\SYSTEM32\MSVCP140.dll (14.29.30133.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.29.30133.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.29.30133.0

ModLoad: 0000000078050000 00000000001e4000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.19041.867 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.867

ModLoad: 0000000048170000 000000000001b000 C:\Windows\SYSTEM32\VCRUNTIME140.dll (14.29.30133.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.29.30133.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.29.30133.0

ModLoad: 0000000047fd0000 000000000000c000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll (14.29.30133.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.29.30133.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.29.30133.0

ModLoad: 000000009e140000 000000000abe7000 C:\ProgramData\BOINC\slots\4\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000021970000 000000000020d000 C:\ProgramData\BOINC\slots\4\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 0000000021310000 00000000002db000 C:\ProgramData\BOINC\slots\4\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000071540000 0000000000040000 C:\ProgramData\BOINC\slots\4\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000a8d30000 0000000024b49000 C:\ProgramData\BOINC\slots\4\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000069430000 0000000000045000 C:\ProgramData\BOINC\slots\4\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000097bd0000 00000000034b2000 C:\ProgramData\BOINC\slots\4\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 000000009b090000 00000000030a5000 C:\ProgramData\BOINC\slots\4\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 0000000073940000 0000000000010000 C:\ProgramData\BOINC\slots\4\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 000000006f920000 0000000003d5b000 C:\ProgramData\BOINC\slots\4\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 0000000073680000 000000000919f000 C:\ProgramData\BOINC\slots\4\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000e37d0000 0000000001eca000 C:\ProgramData\BOINC\slots\4\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 000000007c820000 000000001b3a4000 C:\ProgramData\BOINC\slots\4\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 000000007a470000 0000000000030000 C:\Windows\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000783c0000 0000000000012000 C:\Windows\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000007b8b0000 0000000000472000 C:\Windows\System32\Setupapi.dll (6.2.19041.1237) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000079c20000 000000000004e000 C:\Windows\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.19041.1151 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1151

ModLoad: 0000000079bf0000 0000000000027000 C:\Windows\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcrypt.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 00000000798e0000 0000000000034000 C:\Windows\SYSTEM32\DEVOBJ.dll (6.2.19041.1151) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.19041.1151 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1151

ModLoad: 0000000079e00000 0000000000060000 C:\Windows\System32\WINTRUST.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.19041.1466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1466

ModLoad: 000000007a310000 0000000000156000 C:\Windows\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000079720000 0000000000012000 C:\Windows\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000007b140000 0000000000744000 C:\Windows\System32\Shell32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.19041.964 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.964

ModLoad: 0000000077680000 0000000000795000 C:\Windows\SYSTEM32\windows.storage.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000007ab90000 0000000000355000 C:\Windows\System32\combase.dll (6.2.19041.1348) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.19041.1320 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1320

ModLoad: 0000000079590000 000000000002e000 C:\Windows\SYSTEM32\Wldp.dll (6.2.19041.1320) (-exported- Symbols Loaded)
    Linked PDB Filename   : WLDP.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000007c260000 00000000000ad000 C:\Windows\System32\SHCORE.dll (6.2.19041.1387) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000007bf00000 0000000000055000 C:\Windows\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000fbd0000 00000000002b9000 C:\Windows\system32\nvcuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb

ModLoad: 00000000da990000 0000000001679000 C:\Windows\system32\DriverStore\FileRepository\nvdmi.inf_amd64_86a5cd9d74f0a80e\nvcuda64.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb

ModLoad: 0000000072f10000 000000000000a000 C:\Windows\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000006ea20000 0000000000031000 C:\Windows\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptnet.pdb
    File Version          : 10.0.19041.906 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.906

ModLoad: 000000006e6e0000 000000000014a000 C:\Windows\SYSTEM32\drvstore.dll (6.2.19041.1320) (-exported- Symbols Loaded)
    Linked PDB Filename   : drvstore.pdb
    File Version          : 10.0.19041.1320 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1320

ModLoad: 00000000794e0000 000000000000c000 C:\Windows\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000006c1a0000 0000000000762000 C:\Windows\SYSTEM32\nvapi64.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb

ModLoad: 00000000f6da0000 00000000008a5000 C:\Windows\System32\DriverStore\FileRepository\nvdmi.inf_amd64_86a5cd9d74f0a80e\nvptxJitCompiler64.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000079c70000 0000000000082000 C:\Windows\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.19041.1415 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1415



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 2670, Write: 891, Other 3754

- I/O Transfers Counters -
Read: 32660712, Write: 433299637, Other 27628

- Paged Pool Usage -
QuotaPagedPoolUsage: 3433776, QuotaPeakPagedPoolUsage: 3450160
QuotaNonPagedPoolUsage: 31680, QuotaPeakNonPagedPoolUsage: 37664

- Virtual Memory Usage -
VirtualSize: -2000228352, PeakVirtualSize: -805052416

- Pagefile Usage -
PagefileUsage: -2000228352, PeakPagefileUsage: -1184894976

- Working Set Size -
WorkingSetSize: 758521856, PeakWorkingSetSize: 1602043904, PageFaultCount: 6338171

*** Dump of thread ID 23908 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 76562496.000000, User Time: 5740312576.000000, Wait Time: 21149936.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFE79FD4F69

- Registers -
rax=00000000face4e34 rbx=00000000ac7aaf00 rcx=00000000face4e00 rdx=0000000000000003 rsi=00000000face5290 rdi=0000000019930520
r8=00000000f9de0000 r9=00000000face4989 r10=0000000000000000 r11=0000000000743000 r12=0000000008000000 r13=0000000000000001
r14=0000000004800000 r15=0000000000000000 rip=0000000079fd4f69 rsp=00000000face5040 rbp=00000000face5280
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202

- Callstack -
ChildEBP RetAddr  Args to Child
face5110 48176480 face5250 e3e98720 00000000 00000894 KERNELBASE!RaiseException+0x0 
face5170 aae94ee9 ac0d3020 00000000 face5230 face57c0 VCRUNTIME140!_CxxThrowException+0x0 
face5380 a384f058 2efeaeb4 face7450 a8a616c0 face57c0 torch_cuda!at::cuda::CUDAEvent::block+0x0 
face5580 a38522d2 face57c0 face56b8 face63f0 face63f0 torch_cpu!at::native::conv_tbc_backward+0x0 
face5ae0 a3851eff 00000000 face63f0 14000000 a372bfac torch_cpu!at::native::copy_+0x0 
face5b40 a3ce4a0d a8a60101 a8a60640 ffffffff a3710231 torch_cpu!at::native::copy_+0x0 
face5d60 a56115e6 10000000 00000000 a6c2ac10 00000000 torch_cpu!at::Tensor::copy_+0x0 
face61a0 a3ce4a0d a8a60640 7c45b3c7 face6228 a8a60640 torch_cpu!torch::autograd::VariableType::allCUDATypes+0x0 
face63c0 a39aa38f face6580 6949a8c0 f9decb60 face63f0 torch_cpu!at::Tensor::copy_+0x0 
face6470 a39a9c19 face6a18 face7450 face6780 face6a18 torch_cpu!at::native::to_dense_backward+0x0 
face6680 a3cce363 c11e4f40 00000000 00000002 00000002 torch_cpu!at::native::to+0x0 
face66c0 a4f36573 face0000 fbb05370 face6700 0000657a torch_cpu!at::zeros_out+0x0 
face6700 a3b102f2 face0000 a69824d6 2efe99b4 face7450 torch_cpu!torch::autograd::GraphRoot::apply+0x0 
face6740 a3d01222 face0000 00000000 face67c8 face6850 torch_cpu!at::native::mkldnn_sigmoid_+0x0 
face6980 e4edcd74 face73e0 face73e0 face73e0 00000000 torch_cpu!at::Tensor::to+0x0 
face6b90 e4edaf96 face7470 face6d10 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
facff930 e511c6f8 fb861dc0 00000000 00000000 fb873890 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
facff970 7a597034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
facff9a0 7c482651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
facffa20 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 23416 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 312500.000000, Wait Time: 21149930.000000

- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000fb1ff6e0 rsi=0000000000000000 rdi=0000000000000064
r8=00000000fb1ff6b8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000007c4cd3f4 rsp=00000000fb1ff6b8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
fb1ff6b0 79fe962e fb1ff778 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0 
fb1ff750 e4efa07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0 
fb1ff780 7a597034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
fb1ff7b0 7c482651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
fb1ff830 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 21816 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 21149936.000000

- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=00000000fb2ff740 rsi=0000000000000000 rdi=000000000000000d
r8=00000000fb2ff3e8 r9=0000000000000000 r10=0000000000000000 r11=00000000fb2ff6b0 r12=0000000000000064 r13=00000000fb2ff740
r14=00000000fb2ff440 r15=0000000000000000 rip=000000007c4cd8c4 rsp=00000000fb2ff3e8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
fb2ff3e0 79fecb20 00000000 faaef000 7c59a4d0 faaef000 ntdll!ZwWaitForMultipleObjects+0x0 
fb2ff6d0 79feca1e f9df55e0 006e0069 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
fb2ff710 dab65be4 fba97d20 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0 
fb2ff950 dac0d15d 00000000 fba94ea0 00000000 00000000 nvcuda64!+0x0 
fb2ffa00 dab655b3 7ce429e4 710c7970 fc05e130 00000000 nvcuda64!cuProfilerStop+0x0 
fb2ffa30 dae94264 fc05e130 00000000 00000000 00000000 nvcuda64!+0x0 
fb2ffa60 7a597034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
fb2ffa90 7c482651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
fb2ffb10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 10208 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 21112118.000000

- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002b0 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002b0
r8=00000000fb3ff6e8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=00000000fba58dd0 r13=0000000000000000
r14=00000000000002b0 r15=0000000000000000 rip=000000007c4ccdf4 rsp=00000000fb3ff468 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
fb3ff460 79fc1a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0 
fb3ff500 dad895ee fc05e340 00000000 00000000 000002b0 KERNELBASE!WaitForSingleObjectEx+0x0 
fb3ff740 dab655b3 00000000 fc05e340 da990000 fbaaf960 nvcuda64!cuProfilerStop+0x0 
fb3ff770 dae94264 fc05e340 00000000 00000000 00000000 nvcuda64!+0x0 
fb3ff7a0 7a597034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
fb3ff7d0 7c482651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
fb3ff850 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 21924 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 21149904.000000

- Registers -
rax=00000000000001d3 rbx=00000000e622bb40 rcx=0000000000000070 rdx=00000000e622b7c0 rsi=00000000f9de3310 rdi=00000000e622bb40
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000007c4d07c4 rsp=00000000fadff438 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
fadff430 7c482dc7 00000000 00000000 f9de3310 00000000 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
fadff730 7a597034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0 
fadff760 7c482651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
fadff7e0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)