Task 12941384

Name ParityModified-1645996061-6307-2-0_0
Workunit 10074457
Created 11 Mar 2022, 18:57:27 UTC
Sent 11 Mar 2022, 19:03:39 UTC
Report deadline 19 Mar 2022, 19:03:39 UTC
Received 19 Mar 2022, 19:35:50 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 17360
Run time 4 hours 30 min 8 sec
CPU time 4 hours 2 min 43 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 865.70 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 1.54 GB
Peak swap size 3.43 GB
Peak disk usage 1.54 GB

Stderr output

<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
n | val_loss: nan | Time: 9114.45 ms
[2022-03-19 02:16:35	                main:574]	:	INFO	:	Epoch 1577 | loss: nan | val_loss: nan | Time: 9109.58 ms
[2022-03-19 02:16:45	                main:574]	:	INFO	:	Epoch 1578 | loss: nan | val_loss: nan | Time: 9103.27 ms
[2022-03-19 02:16:54	                main:574]	:	INFO	:	Epoch 1579 | loss: nan | val_loss: nan | Time: 9113.43 ms
[2022-03-19 02:17:03	                main:574]	:	INFO	:	Epoch 1580 | loss: nan | val_loss: nan | Time: 9102.31 ms
[2022-03-19 02:17:12	                main:574]	:	INFO	:	Epoch 1581 | loss: nan | val_loss: nan | Time: 9124.79 ms
[2022-03-19 02:17:21	                main:574]	:	INFO	:	Epoch 1582 | loss: nan | val_loss: nan | Time: 9116.36 ms
[2022-03-19 02:17:30	                main:574]	:	INFO	:	Epoch 1583 | loss: nan | val_loss: nan | Time: 9102.2 ms
[2022-03-19 02:17:39	                main:574]	:	INFO	:	Epoch 1584 | loss: nan | val_loss: nan | Time: 9102.84 ms
[2022-03-19 02:17:49	                main:574]	:	INFO	:	Epoch 1585 | loss: nan | val_loss: nan | Time: 9260.51 ms
[2022-03-19 02:17:58	                main:574]	:	INFO	:	Epoch 1586 | loss: nan | val_loss: nan | Time: 9203.86 ms
[2022-03-19 02:18:07	                main:574]	:	INFO	:	Epoch 1587 | loss: nan | val_loss: nan | Time: 9202.1 ms
[2022-03-19 02:18:16	                main:574]	:	INFO	:	Epoch 1588 | loss: nan | val_loss: nan | Time: 9144.49 ms
[2022-03-19 02:18:25	                main:574]	:	INFO	:	Epoch 1589 | loss: nan | val_loss: nan | Time: 9128.57 ms
[2022-03-19 02:18:35	                main:574]	:	INFO	:	Epoch 1590 | loss: nan | val_loss: nan | Time: 9131.39 ms
[2022-03-19 02:18:44	                main:574]	:	INFO	:	Epoch 1591 | loss: nan | val_loss: nan | Time: 9110.38 ms
[2022-03-19 02:18:53	                main:574]	:	INFO	:	Epoch 1592 | loss: nan | val_loss: nan | Time: 9116.8 ms
[2022-03-19 02:19:02	                main:574]	:	INFO	:	Epoch 1593 | loss: nan | val_loss: nan | Time: 9148.06 ms
[2022-03-19 02:19:11	                main:574]	:	INFO	:	Epoch 1594 | loss: nan | val_loss: nan | Time: 9134.88 ms
[2022-03-19 02:19:20	                main:574]	:	INFO	:	Epoch 1595 | loss: nan | val_loss: nan | Time: 9117.37 ms
[2022-03-19 02:19:29	                main:574]	:	INFO	:	Epoch 1596 | loss: nan | val_loss: nan | Time: 9100.83 ms
[2022-03-19 02:19:38	                main:574]	:	INFO	:	Epoch 1597 | loss: nan | val_loss: nan | Time: 9107.02 ms
[2022-03-19 02:19:48	                main:574]	:	INFO	:	Epoch 1598 | loss: nan | val_loss: nan | Time: 9113.82 ms
[2022-03-19 02:19:57	                main:574]	:	INFO	:	Epoch 1599 | loss: nan | val_loss: nan | Time: 9110.92 ms
[2022-03-19 02:20:06	                main:574]	:	INFO	:	Epoch 1600 | loss: nan | val_loss: nan | Time: 9107.45 ms
[2022-03-19 02:20:15	                main:574]	:	INFO	:	Epoch 1601 | loss: nan | val_loss: nan | Time: 9122.22 ms
[2022-03-19 02:20:24	                main:574]	:	INFO	:	Epoch 1602 | loss: nan | val_loss: nan | Time: 9097.07 ms
[2022-03-19 02:20:33	                main:574]	:	INFO	:	Epoch 1603 | loss: nan | val_loss: nan | Time: 9121.29 ms
[2022-03-19 02:20:42	                main:574]	:	INFO	:	Epoch 1604 | loss: nan | val_loss: nan | Time: 9125.79 ms
[2022-03-19 02:20:51	                main:574]	:	INFO	:	Epoch 1605 | loss: nan | val_loss: nan | Time: 9123.73 ms
[2022-03-19 02:21:01	                main:574]	:	INFO	:	Epoch 1606 | loss: nan | val_loss: nan | Time: 9114.56 ms
[2022-03-19 02:21:10	                main:574]	:	INFO	:	Epoch 1607 | loss: nan | val_loss: nan | Time: 9120.49 ms
[2022-03-19 02:21:19	                main:574]	:	INFO	:	Epoch 1608 | loss: nan | val_loss: nan | Time: 9110.47 ms
[2022-03-19 02:21:28	                main:574]	:	INFO	:	Epoch 1609 | loss: nan | val_loss: nan | Time: 9098.75 ms
[2022-03-19 02:21:37	                main:574]	:	INFO	:	Epoch 1610 | loss: nan | val_loss: nan | Time: 9126.7 ms
[2022-03-19 02:21:46	                main:574]	:	INFO	:	Epoch 1611 | loss: nan | val_loss: nan | Time: 9090.69 ms
[2022-03-19 02:21:55	                main:574]	:	INFO	:	Epoch 1612 | loss: nan | val_loss: nan | Time: 9128.27 ms
[2022-03-19 02:22:05	                main:574]	:	INFO	:	Epoch 1613 | loss: nan | val_loss: nan | Time: 9200.7 ms
[2022-03-19 02:22:14	                main:574]	:	INFO	:	Epoch 1614 | loss: nan | val_loss: nan | Time: 9119.58 ms
[2022-03-19 02:22:23	                main:574]	:	INFO	:	Epoch 1615 | loss: nan | val_loss: nan | Time: 9114.93 ms
[2022-03-19 02:22:32	                main:574]	:	INFO	:	Epoch 1616 | loss: nan | val_loss: nan | Time: 9181.01 ms
[2022-03-19 02:22:41	                main:574]	:	INFO	:	Epoch 1617 | loss: nan | val_loss: nan | Time: 9129.67 ms
[2022-03-19 02:22:50	                main:574]	:	INFO	:	Epoch 1618 | loss: nan | val_loss: nan | Time: 9115.2 ms
[2022-03-19 02:22:59	                main:574]	:	INFO	:	Epoch 1619 | loss: nan | val_loss: nan | Time: 9110.47 ms
[2022-03-19 02:23:08	                main:574]	:	INFO	:	Epoch 1620 | loss: nan | val_loss: nan | Time: 9118.6 ms
[2022-03-19 02:23:18	                main:574]	:	INFO	:	Epoch 1621 | loss: nan | val_loss: nan | Time: 9150.71 ms
[2022-03-19 02:23:27	                main:574]	:	INFO	:	Epoch 1622 | loss: nan | val_loss: nan | Time: 9135.02 ms
[2022-03-19 02:23:36	                main:574]	:	INFO	:	Epoch 1623 | loss: nan | val_loss: nan | Time: 9148.26 ms
[2022-03-19 02:23:45	                main:574]	:	INFO	:	Epoch 1624 | loss: nan | val_loss: nan | Time: 9118.42 ms
[2022-03-19 02:23:54	                main:574]	:	INFO	:	Epoch 1625 | loss: nan | val_loss: nan | Time: 9130.41 ms
[2022-03-19 02:24:03	                main:574]	:	INFO	:	Epoch 1626 | loss: nan | val_loss: nan | Time: 9104.25 ms
[2022-03-19 02:24:12	                main:574]	:	INFO	:	Epoch 1627 | loss: nan | val_loss: nan | Time: 9131.73 ms
[2022-03-19 02:24:22	                main:574]	:	INFO	:	Epoch 1628 | loss: nan | val_loss: nan | Time: 9125.61 ms
[2022-03-19 02:24:31	                main:574]	:	INFO	:	Epoch 1629 | loss: nan | val_loss: nan | Time: 9151.74 ms
[2022-03-19 02:24:40	                main:574]	:	INFO	:	Epoch 1630 | loss: nan | val_loss: nan | Time: 9108.87 ms
[2022-03-19 02:24:49	                main:574]	:	INFO	:	Epoch 1631 | loss: nan | val_loss: nan | Time: 9125.88 ms
[2022-03-19 02:24:58	                main:574]	:	INFO	:	Epoch 1632 | loss: nan | val_loss: nan | Time: 9110.76 ms
[2022-03-19 02:25:07	                main:574]	:	INFO	:	Epoch 1633 | loss: nan | val_loss: nan | Time: 9146.26 ms
[2022-03-19 02:25:16	                main:574]	:	INFO	:	Epoch 1634 | loss: nan | val_loss: nan | Time: 9133.43 ms
[2022-03-19 02:25:26	                main:574]	:	INFO	:	Epoch 1635 | loss: nan | val_loss: nan | Time: 9115.38 ms
[2022-03-19 02:25:35	                main:574]	:	INFO	:	Epoch 1636 | loss: nan | val_loss: nan | Time: 9124.92 ms
[2022-03-19 02:25:44	                main:574]	:	INFO	:	Epoch 1637 | loss: nan | val_loss: nan | Time: 9106.16 ms
[2022-03-19 02:25:53	                main:574]	:	INFO	:	Epoch 1638 | loss: nan | val_loss: nan | Time: 9110.21 ms
[2022-03-19 02:26:02	                main:574]	:	INFO	:	Epoch 1639 | loss: nan | val_loss: nan | Time: 9131.67 ms
[2022-03-19 02:26:11	                main:574]	:	INFO	:	Epoch 1640 | loss: nan | val_loss: nan | Time: 9132.23 ms
[2022-03-19 02:26:20	                main:574]	:	INFO	:	Epoch 1641 | loss: nan | val_loss: nan | Time: 9119.25 ms
[2022-03-19 02:26:29	                main:574]	:	INFO	:	Epoch 1642 | loss: nan | val_loss: nan | Time: 9115.18 ms
[2022-03-19 02:26:39	                main:574]	:	INFO	:	Epoch 1643 | loss: nan | val_loss: nan | Time: 9113.58 ms
[2022-03-19 02:26:48	                main:574]	:	INFO	:	Epoch 1644 | loss: nan | val_loss: nan | Time: 9128.48 ms
[2022-03-19 02:26:57	                main:574]	:	INFO	:	Epoch 1645 | loss: nan | val_loss: nan | Time: 9124.59 ms
[2022-03-19 02:27:06	                main:574]	:	INFO	:	Epoch 1646 | loss: nan | val_loss: nan | Time: 9093.67 ms
[2022-03-19 02:27:15	                main:574]	:	INFO	:	Epoch 1647 | loss: nan | val_loss: nan | Time: 9118.88 ms
[2022-03-19 02:27:24	                main:574]	:	INFO	:	Epoch 1648 | loss: nan | val_loss: nan | Time: 9132.2 ms
[2022-03-19 02:27:33	                main:574]	:	INFO	:	Epoch 1649 | loss: nan | val_loss: nan | Time: 9141.77 ms
[2022-03-19 02:27:43	                main:574]	:	INFO	:	Epoch 1650 | loss: nan | val_loss: nan | Time: 9122.8 ms
[2022-03-19 02:27:52	                main:574]	:	INFO	:	Epoch 1651 | loss: nan | val_loss: nan | Time: 9162.63 ms
[2022-03-19 02:28:01	                main:574]	:	INFO	:	Epoch 1652 | loss: nan | val_loss: nan | Time: 9140.98 ms
[2022-03-19 02:28:10	                main:574]	:	INFO	:	Epoch 1653 | loss: nan | val_loss: nan | Time: 9113.56 ms
[2022-03-19 02:28:19	                main:574]	:	INFO	:	Epoch 1654 | loss: nan | val_loss: nan | Time: 9152.4 ms
[2022-03-19 02:28:28	                main:574]	:	INFO	:	Epoch 1655 | loss: nan | val_loss: nan | Time: 9115.57 ms
[2022-03-19 02:28:37	                main:574]	:	INFO	:	Epoch 1656 | loss: nan | val_loss: nan | Time: 9120.73 ms
[2022-03-19 02:28:47	                main:574]	:	INFO	:	Epoch 1657 | loss: nan | val_loss: nan | Time: 9120.08 ms
[2022-03-19 02:28:56	                main:574]	:	INFO	:	Epoch 1658 | loss: nan | val_loss: nan | Time: 9179.86 ms
[2022-03-19 02:29:05	                main:574]	:	INFO	:	Epoch 1659 | loss: nan | val_loss: nan | Time: 9133.84 ms
[2022-03-19 02:29:14	                main:574]	:	INFO	:	Epoch 1660 | loss: nan | val_loss: nan | Time: 9116.69 ms
[2022-03-19 02:29:23	                main:574]	:	INFO	:	Epoch 1661 | loss: nan | val_loss: nan | Time: 9142.47 ms
[2022-03-19 02:29:32	                main:574]	:	INFO	:	Epoch 1662 | loss: nan | val_loss: nan | Time: 9148.9 ms
[2022-03-19 02:29:41	                main:574]	:	INFO	:	Epoch 1663 | loss: nan | val_loss: nan | Time: 9113.86 ms
[2022-03-19 02:29:51	                main:574]	:	INFO	:	Epoch 1664 | loss: nan | val_loss: nan | Time: 9154.5 ms
[2022-03-19 02:30:00	                main:574]	:	INFO	:	Epoch 1665 | loss: nan | val_loss: nan | Time: 9125.28 ms
[2022-03-19 02:30:09	                main:574]	:	INFO	:	Epoch 1666 | loss: nan | val_loss: nan | Time: 9118.1 ms
[2022-03-19 02:30:18	                main:574]	:	INFO	:	Epoch 1667 | loss: nan | val_loss: nan | Time: 9103.35 ms
[2022-03-19 02:30:27	                main:574]	:	INFO	:	Epoch 1668 | loss: nan | val_loss: nan | Time: 9118.79 ms
[2022-03-19 02:30:36	                main:574]	:	INFO	:	Epoch 1669 | loss: nan | val_loss: nan | Time: 9111.15 ms
[2022-03-19 02:30:45	                main:574]	:	INFO	:	Epoch 1670 | loss: nan | val_loss: nan | Time: 9107.08 ms
[2022-03-19 02:30:55	                main:574]	:	INFO	:	Epoch 1671 | loss: nan | val_loss: nan | Time: 9108.57 ms
[2022-03-19 02:31:04	                main:574]	:	INFO	:	Epoch 1672 | loss: nan | val_loss: nan | Time: 9107.9 ms
[2022-03-19 02:31:13	                main:574]	:	INFO	:	Epoch 1673 | loss: nan | val_loss: nan | Time: 9115.08 ms
[2022-03-19 02:31:22	                main:574]	:	INFO	:	Epoch 1674 | loss: nan | val_loss: nan | Time: 9119.7 ms
[2022-03-19 02:31:31	                main:574]	:	INFO	:	Epoch 1675 | loss: nan | val_loss: nan | Time: 9111.51 ms
[2022-03-19 02:31:40	                main:574]	:	INFO	:	Epoch 1676 | loss: nan | val_loss: nan | Time: 9111.81 ms
[2022-03-19 02:31:49	                main:574]	:	INFO	:	Epoch 1677 | loss: nan | val_loss: nan | Time: 9116.16 ms
[2022-03-19 02:31:58	                main:574]	:	INFO	:	Epoch 1678 | loss: nan | val_loss: nan | Time: 9117.07 ms
[2022-03-19 02:32:07	                main:574]	:	INFO	:	Epoch 1679 | loss: nan | val_loss: nan | Time: 9120.34 ms
[2022-03-19 02:32:17	                main:574]	:	INFO	:	Epoch 1680 | loss: nan | val_loss: nan | Time: 9156.48 ms
[2022-03-19 02:32:26	                main:574]	:	INFO	:	Epoch 1681 | loss: nan | val_loss: nan | Time: 9127.79 ms
[2022-03-19 02:32:35	                main:574]	:	INFO	:	Epoch 1682 | loss: nan | val_loss: nan | Time: 9216.11 ms
[2022-03-19 02:32:44	                main:574]	:	INFO	:	Epoch 1683 | loss: nan | val_loss: nan | Time: 9392.56 ms
[2022-03-19 02:32:54	                main:574]	:	INFO	:	Epoch 1684 | loss: nan | val_loss: nan | Time: 9225.34 ms
[2022-03-19 02:33:03	                main:574]	:	INFO	:	Epoch 1685 | loss: nan | val_loss: nan | Time: 9187.42 ms
[2022-03-19 02:33:12	                main:574]	:	INFO	:	Epoch 1686 | loss: nan | val_loss: nan | Time: 9180.68 ms
[2022-03-19 02:33:21	                main:574]	:	INFO	:	Epoch 1687 | loss: nan | val_loss: nan | Time: 9121.53 ms
[2022-03-19 02:33:30	                main:574]	:	INFO	:	Epoch 1688 | loss: nan | val_loss: nan | Time: 9103.97 ms
[2022-03-19 02:33:39	                main:574]	:	INFO	:	Epoch 1689 | loss: nan | val_loss: nan | Time: 9109.88 ms
[2022-03-19 02:33:49	                main:574]	:	INFO	:	Epoch 1690 | loss: nan | val_loss: nan | Time: 9118.33 ms
[2022-03-19 02:33:58	                main:574]	:	INFO	:	Epoch 1691 | loss: nan | val_loss: nan | Time: 9118.11 ms
[2022-03-19 02:34:07	                main:574]	:	INFO	:	Epoch 1692 | loss: nan | val_loss: nan | Time: 9117.39 ms
[2022-03-19 02:34:16	                main:574]	:	INFO	:	Epoch 1693 | loss: nan | val_loss: nan | Time: 9110.04 ms
[2022-03-19 02:34:25	                main:574]	:	INFO	:	Epoch 1694 | loss: nan | val_loss: nan | Time: 9132.48 ms
[2022-03-19 02:34:34	                main:574]	:	INFO	:	Epoch 1695 | loss: nan | val_loss: nan | Time: 9103.47 ms
[2022-03-19 02:34:43	                main:574]	:	INFO	:	Epoch 1696 | loss: nan | val_loss: nan | Time: 9117.3 ms
[2022-03-19 02:34:53	                main:574]	:	INFO	:	Epoch 1697 | loss: nan | val_loss: nan | Time: 9146.17 ms
[2022-03-19 02:35:02	                main:574]	:	INFO	:	Epoch 1698 | loss: nan | val_loss: nan | Time: 9138.94 ms
[2022-03-19 02:35:11	                main:574]	:	INFO	:	Epoch 1699 | loss: nan | val_loss: nan | Time: 9106.11 ms
[2022-03-19 02:35:20	                main:574]	:	INFO	:	Epoch 1700 | loss: nan | val_loss: nan | Time: 9127.79 ms
[2022-03-19 02:35:29	                main:574]	:	INFO	:	Epoch 1701 | loss: nan | val_loss: nan | Time: 9114.84 ms
[2022-03-19 02:35:38	                main:574]	:	INFO	:	Epoch 1702 | loss: nan | val_loss: nan | Time: 9106.88 ms
[2022-03-19 02:35:47	                main:574]	:	INFO	:	Epoch 1703 | loss: nan | val_loss: nan | Time: 9111.92 ms
[2022-03-19 02:35:56	                main:574]	:	INFO	:	Epoch 1704 | loss: nan | val_loss: nan | Time: 9104.14 ms
[2022-03-19 02:36:06	                main:574]	:	INFO	:	Epoch 1705 | loss: nan | val_loss: nan | Time: 9125.27 ms
[2022-03-19 02:36:15	                main:574]	:	INFO	:	Epoch 1706 | loss: nan | val_loss: nan | Time: 9115.58 ms
[2022-03-19 02:36:24	                main:574]	:	INFO	:	Epoch 1707 | loss: nan | val_loss: nan | Time: 9107.43 ms
[2022-03-19 02:36:33	                main:574]	:	INFO	:	Epoch 1708 | loss: nan | val_loss: nan | Time: 9123.71 ms
[2022-03-19 02:36:42	                main:574]	:	INFO	:	Epoch 1709 | loss: nan | val_loss: nan | Time: 9126.58 ms
[2022-03-19 02:36:51	                main:574]	:	INFO	:	Epoch 1710 | loss: nan | val_loss: nan | Time: 9111.97 ms
[2022-03-19 02:37:00	                main:574]	:	INFO	:	Epoch 1711 | loss: nan | val_loss: nan | Time: 9111.23 ms
[2022-03-19 02:37:09	                main:574]	:	INFO	:	Epoch 1712 | loss: nan | val_loss: nan | Time: 9104.7 ms
[2022-03-19 02:37:19	                main:574]	:	INFO	:	Epoch 1713 | loss: nan | val_loss: nan | Time: 9117.84 ms
[2022-03-19 02:37:28	                main:574]	:	INFO	:	Epoch 1714 | loss: nan | val_loss: nan | Time: 9123.17 ms
[2022-03-19 02:37:37	                main:574]	:	INFO	:	Epoch 1715 | loss: nan | val_loss: nan | Time: 9104.09 ms
[2022-03-19 02:37:46	                main:574]	:	INFO	:	Epoch 1716 | loss: nan | val_loss: nan | Time: 9233.62 ms
[2022-03-19 02:37:56	                main:574]	:	INFO	:	Epoch 1717 | loss: nan | val_loss: nan | Time: 9880.75 ms
[2022-03-19 02:38:05	                main:574]	:	INFO	:	Epoch 1718 | loss: nan | val_loss: nan | Time: 9135.57 ms
[2022-03-19 02:38:14	                main:574]	:	INFO	:	Epoch 1719 | loss: nan | val_loss: nan | Time: 9120.82 ms
[2022-03-19 02:38:23	                main:574]	:	INFO	:	Epoch 1720 | loss: nan | val_loss: nan | Time: 9110.32 ms
[2022-03-19 02:38:32	                main:574]	:	INFO	:	Epoch 1721 | loss: nan | val_loss: nan | Time: 9101.04 ms
[2022-03-19 02:38:42	                main:574]	:	INFO	:	Epoch 1722 | loss: nan | val_loss: nan | Time: 9115.24 ms
[2022-03-19 02:38:51	                main:574]	:	INFO	:	Epoch 1723 | loss: nan | val_loss: nan | Time: 9097.53 ms
[2022-03-19 02:39:00	                main:574]	:	INFO	:	Epoch 1724 | loss: nan | val_loss: nan | Time: 9112.97 ms
[2022-03-19 02:39:09	                main:574]	:	INFO	:	Epoch 1725 | loss: nan | val_loss: nan | Time: 9110.61 ms
[2022-03-19 02:39:18	                main:574]	:	INFO	:	Epoch 1726 | loss: nan | val_loss: nan | Time: 9096.22 ms
[2022-03-19 02:39:27	                main:574]	:	INFO	:	Epoch 1727 | loss: nan | val_loss: nan | Time: 9124.63 ms
[2022-03-19 02:39:36	                main:574]	:	INFO	:	Epoch 1728 | loss: nan | val_loss: nan | Time: 9105.39 ms
[2022-03-19 02:39:45	                main:574]	:	INFO	:	Epoch 1729 | loss: nan | val_loss: nan | Time: 9116.16 ms
[2022-03-19 02:39:55	                main:574]	:	INFO	:	Epoch 1730 | loss: nan | val_loss: nan | Time: 9094.66 ms
[2022-03-19 02:40:04	                main:574]	:	INFO	:	Epoch 1731 | loss: nan | val_loss: nan | Time: 9145.7 ms
[2022-03-19 02:40:13	                main:574]	:	INFO	:	Epoch 1732 | loss: nan | val_loss: nan | Time: 9107.63 ms
[2022-03-19 02:40:22	                main:574]	:	INFO	:	Epoch 1733 | loss: nan | val_loss: nan | Time: 9123.49 ms
[2022-03-19 02:40:31	                main:574]	:	INFO	:	Epoch 1734 | loss: nan | val_loss: nan | Time: 9116.2 ms
[2022-03-19 02:40:40	                main:574]	:	INFO	:	Epoch 1735 | loss: nan | val_loss: nan | Time: 9121.89 ms
[2022-03-19 02:40:49	                main:574]	:	INFO	:	Epoch 1736 | loss: nan | val_loss: nan | Time: 9113.23 ms
[2022-03-19 02:40:58	                main:574]	:	INFO	:	Epoch 1737 | loss: nan | val_loss: nan | Time: 9115.15 ms
[2022-03-19 02:41:08	                main:574]	:	INFO	:	Epoch 1738 | loss: nan | val_loss: nan | Time: 9102.94 ms
[2022-03-19 02:41:17	                main:574]	:	INFO	:	Epoch 1739 | loss: nan | val_loss: nan | Time: 9114.89 ms
[2022-03-19 02:41:26	                main:574]	:	INFO	:	Epoch 1740 | loss: nan | val_loss: nan | Time: 9109.57 ms
[2022-03-19 02:41:35	                main:574]	:	INFO	:	Epoch 1741 | loss: nan | val_loss: nan | Time: 9109.64 ms
[2022-03-19 02:41:44	                main:574]	:	INFO	:	Epoch 1742 | loss: nan | val_loss: nan | Time: 9132.86 ms
[2022-03-19 02:41:53	                main:574]	:	INFO	:	Epoch 1743 | loss: nan | val_loss: nan | Time: 9109.08 ms
[2022-03-19 02:42:02	                main:574]	:	INFO	:	Epoch 1744 | loss: nan | val_loss: nan | Time: 9092.93 ms
[2022-03-19 02:42:11	                main:574]	:	INFO	:	Epoch 1745 | loss: nan | val_loss: nan | Time: 9119.46 ms
[2022-03-19 02:42:21	                main:574]	:	INFO	:	Epoch 1746 | loss: nan | val_loss: nan | Time: 9095.6 ms
[2022-03-19 02:42:30	                main:574]	:	INFO	:	Epoch 1747 | loss: nan | val_loss: nan | Time: 9119.08 ms
[2022-03-19 02:42:39	                main:574]	:	INFO	:	Epoch 1748 | loss: nan | val_loss: nan | Time: 9116.07 ms
[2022-03-19 02:42:48	                main:574]	:	INFO	:	Epoch 1749 | loss: nan | val_loss: nan | Time: 9102.26 ms
[2022-03-19 02:42:57	                main:574]	:	INFO	:	Epoch 1750 | loss: nan | val_loss: nan | Time: 9102.11 ms
[2022-03-19 02:43:06	                main:574]	:	INFO	:	Epoch 1751 | loss: nan | val_loss: nan | Time: 9104.9 ms
[2022-03-19 02:43:15	                main:574]	:	INFO	:	Epoch 1752 | loss: nan | val_loss: nan | Time: 9105.17 ms
[2022-03-19 02:43:24	                main:574]	:	INFO	:	Epoch 1753 | loss: nan | val_loss: nan | Time: 9138.88 ms
[2022-03-19 02:43:34	                main:574]	:	INFO	:	Epoch 1754 | loss: nan | val_loss: nan | Time: 9130.41 ms
[2022-03-19 02:43:43	                main:574]	:	INFO	:	Epoch 1755 | loss: nan | val_loss: nan | Time: 9112.59 ms
[2022-03-19 02:43:52	                main:574]	:	INFO	:	Epoch 1756 | loss: nan | val_loss: nan | Time: 9160.68 ms
[2022-03-19 02:44:01	                main:574]	:	INFO	:	Epoch 1757 | loss: nan | val_loss: nan | Time: 9125.43 ms
[2022-03-19 02:44:10	                main:574]	:	INFO	:	Epoch 1758 | loss: nan | val_loss: nan | Time: 9118.01 ms
[2022-03-19 02:44:19	                main:574]	:	INFO	:	Epoch 1759 | loss: nan | val_loss: nan | Time: 9116.6 ms
[2022-03-19 02:44:28	                main:574]	:	INFO	:	Epoch 1760 | loss: nan | val_loss: nan | Time: 9115.19 ms
[2022-03-19 02:44:37	                main:574]	:	INFO	:	Epoch 1761 | loss: nan | val_loss: nan | Time: 9112.99 ms
[2022-03-19 02:44:47	                main:574]	:	INFO	:	Epoch 1762 | loss: nan | val_loss: nan | Time: 9124.68 ms
[2022-03-19 02:44:56	                main:574]	:	INFO	:	Epoch 1763 | loss: nan | val_loss: nan | Time: 9112.12 ms
[2022-03-19 02:45:05	                main:574]	:	INFO	:	Epoch 1764 | loss: nan | val_loss: nan | Time: 9097.43 ms
[2022-03-19 02:45:14	                main:574]	:	INFO	:	Epoch 1765 | loss: nan | val_loss: nan | Time: 9107.31 ms
[2022-03-19 02:45:23	                main:574]	:	INFO	:	Epoch 1766 | loss: nan | val_loss: nan | Time: 9111.54 ms
[2022-03-19 02:45:32	                main:574]	:	INFO	:	Epoch 1767 | loss: nan | val_loss: nan | Time: 9096.51 ms
[2022-03-19 02:45:41	                main:574]	:	INFO	:	Epoch 1768 | loss: nan | val_loss: nan | Time: 9141.04 ms
[2022-03-19 02:45:50	                main:574]	:	INFO	:	Epoch 1769 | loss: nan | val_loss: nan | Time: 9096.74 ms
[2022-03-19 02:46:00	                main:574]	:	INFO	:	Epoch 1770 | loss: nan | val_loss: nan | Time: 9123.64 ms
[2022-03-19 02:46:09	                main:574]	:	INFO	:	Epoch 1771 | loss: nan | val_loss: nan | Time: 9114.93 ms
[2022-03-19 02:46:18	                main:574]	:	INFO	:	Epoch 1772 | loss: nan | val_loss: nan | Time: 9111.5 ms
[2022-03-19 02:46:27	                main:574]	:	INFO	:	Epoch 1773 | loss: nan | val_loss: nan | Time: 9092.02 ms
[2022-03-19 02:46:36	                main:574]	:	INFO	:	Epoch 1774 | loss: nan | val_loss: nan | Time: 9116.53 ms
[2022-03-19 02:46:45	                main:574]	:	INFO	:	Epoch 1775 | loss: nan | val_loss: nan | Time: 9113.42 ms
[2022-03-19 02:46:54	                main:574]	:	INFO	:	Epoch 1776 | loss: nan | val_loss: nan | Time: 9129.69 ms
[2022-03-19 02:47:03	                main:574]	:	INFO	:	Epoch 1777 | loss: nan | val_loss: nan | Time: 9102.52 ms
[2022-03-19 02:47:13	                main:574]	:	INFO	:	Epoch 1778 | loss: nan | val_loss: nan | Time: 9116.25 ms
[2022-03-19 02:47:22	                main:574]	:	INFO	:	Epoch 1779 | loss: nan | val_loss: nan | Time: 9119.49 ms
[2022-03-19 02:47:31	                main:574]	:	INFO	:	Epoch 1780 | loss: nan | val_loss: nan | Time: 9117.13 ms
[2022-03-19 02:47:40	                main:574]	:	INFO	:	Epoch 1781 | loss: nan | val_loss: nan | Time: 9112.91 ms
[2022-03-19 02:47:49	                main:574]	:	INFO	:	Epoch 1782 | loss: nan | val_loss: nan | Time: 9105.91 ms
[2022-03-19 02:47:58	                main:574]	:	INFO	:	Epoch 1783 | loss: nan | val_loss: nan | Time: 9119.79 ms
[2022-03-19 02:48:07	                main:574]	:	INFO	:	Epoch 1784 | loss: nan | val_loss: nan | Time: 9152.68 ms
[2022-03-19 02:48:16	                main:574]	:	INFO	:	Epoch 1785 | loss: nan | val_loss: nan | Time: 9119.56 ms
[2022-03-19 02:48:26	                main:574]	:	INFO	:	Epoch 1786 | loss: nan | val_loss: nan | Time: 9178.91 ms
[2022-03-19 02:48:35	                main:574]	:	INFO	:	Epoch 1787 | loss: nan | val_loss: nan | Time: 9121.08 ms
[2022-03-19 02:48:44	                main:574]	:	INFO	:	Epoch 1788 | loss: nan | val_loss: nan | Time: 9097.03 ms
[2022-03-19 02:48:53	                main:574]	:	INFO	:	Epoch 1789 | loss: nan | val_loss: nan | Time: 9122.43 ms
[2022-03-19 02:49:02	                main:574]	:	INFO	:	Epoch 1790 | loss: nan | val_loss: nan | Time: 9115.61 ms
[2022-03-19 02:49:11	                main:574]	:	INFO	:	Epoch 1791 | loss: nan | val_loss: nan | Time: 9122.89 ms
[2022-03-19 02:49:20	                main:574]	:	INFO	:	Epoch 1792 | loss: nan | val_loss: nan | Time: 9144.24 ms
[2022-03-19 02:49:30	                main:574]	:	INFO	:	Epoch 1793 | loss: nan | val_loss: nan | Time: 9100.77 ms
[2022-03-19 02:49:39	                main:574]	:	INFO	:	Epoch 1794 | loss: nan | val_loss: nan | Time: 9118.36 ms
[2022-03-19 02:49:48	                main:574]	:	INFO	:	Epoch 1795 | loss: nan | val_loss: nan | Time: 9108.43 ms
[2022-03-19 02:49:57	                main:574]	:	INFO	:	Epoch 1796 | loss: nan | val_loss: nan | Time: 9106.6 ms
[2022-03-19 02:50:06	                main:574]	:	INFO	:	Epoch 1797 | loss: nan | val_loss: nan | Time: 9152.89 ms
[2022-03-19 02:50:15	                main:574]	:	INFO	:	Epoch 1798 | loss: nan | val_loss: nan | Time: 9116.13 ms
[2022-03-19 02:50:24	                main:574]	:	INFO	:	Epoch 1799 | loss: nan | val_loss: nan | Time: 9128.74 ms
[2022-03-19 02:50:33	                main:574]	:	INFO	:	Epoch 1800 | loss: nan | val_loss: nan | Time: 9112.88 ms
[2022-03-19 02:50:43	                main:574]	:	INFO	:	Epoch 1801 | loss: nan | val_loss: nan | Time: 9136.84 ms
[2022-03-19 02:50:52	                main:574]	:	INFO	:	Epoch 1802 | loss: nan | val_loss: nan | Time: 9114.07 ms
[2022-03-19 02:51:01	                main:574]	:	INFO	:	Epoch 1803 | loss: nan | val_loss: nan | Time: 9122.26 ms
[2022-03-19 02:51:10	                main:574]	:	INFO	:	Epoch 1804 | loss: nan | val_loss: nan | Time: 9121.77 ms
[2022-03-19 02:51:19	                main:574]	:	INFO	:	Epoch 1805 | loss: nan | val_loss: nan | Time: 9104.26 ms
[2022-03-19 02:51:28	                main:574]	:	INFO	:	Epoch 1806 | loss: nan | val_loss: nan | Time: 9115.36 ms
[2022-03-19 02:51:37	                main:574]	:	INFO	:	Epoch 1807 | loss: nan | val_loss: nan | Time: 9108.99 ms
[2022-03-19 02:51:47	                main:574]	:	INFO	:	Epoch 1808 | loss: nan | val_loss: nan | Time: 9122.71 ms
[2022-03-19 02:51:56	                main:574]	:	INFO	:	Epoch 1809 | loss: nan | val_loss: nan | Time: 9098.09 ms
[2022-03-19 02:52:05	                main:574]	:	INFO	:	Epoch 1810 | loss: nan | val_loss: nan | Time: 9110.97 ms
[2022-03-19 02:52:14	                main:574]	:	INFO	:	Epoch 1811 | loss: nan | val_loss: nan | Time: 9103.62 ms
[2022-03-19 02:52:23	                main:574]	:	INFO	:	Epoch 1812 | loss: nan | val_loss: nan | Time: 9153.27 ms
[2022-03-19 02:52:32	                main:574]	:	INFO	:	Epoch 1813 | loss: nan | val_loss: nan | Time: 9111.82 ms
[2022-03-19 02:52:41	                main:574]	:	INFO	:	Epoch 1814 | loss: nan | val_loss: nan | Time: 9124.01 ms
[2022-03-19 02:52:50	                main:574]	:	INFO	:	Epoch 1815 | loss: nan | val_loss: nan | Time: 9108.97 ms
[2022-03-19 02:53:00	                main:574]	:	INFO	:	Epoch 1816 | loss: nan | val_loss: nan | Time: 9132.08 ms
[2022-03-19 02:53:09	                main:574]	:	INFO	:	Epoch 1817 | loss: nan | val_loss: nan | Time: 9108.56 ms
[2022-03-19 02:53:18	                main:574]	:	INFO	:	Epoch 1818 | loss: nan | val_loss: nan | Time: 9107.98 ms
[2022-03-19 02:53:27	                main:574]	:	INFO	:	Epoch 1819 | loss: nan | val_loss: nan | Time: 9118.76 ms
[2022-03-19 02:53:36	                main:574]	:	INFO	:	Epoch 1820 | loss: nan | val_loss: nan | Time: 9118.74 ms
[2022-03-19 02:53:45	                main:574]	:	INFO	:	Epoch 1821 | loss: nan | val_loss: nan | Time: 9115.32 ms
[2022-03-19 02:53:54	                main:574]	:	INFO	:	Epoch 1822 | loss: nan | val_loss: nan | Time: 9163.01 ms
[2022-03-19 02:54:03	                main:574]	:	INFO	:	Epoch 1823 | loss: nan | val_loss: nan | Time: 9120.27 ms
[2022-03-19 02:54:13	                main:574]	:	INFO	:	Epoch 1824 | loss: nan | val_loss: nan | Time: 9121.03 ms
[2022-03-19 02:54:22	                main:574]	:	INFO	:	Epoch 1825 | loss: nan | val_loss: nan | Time: 9102.28 ms
[2022-03-19 02:54:31	                main:574]	:	INFO	:	Epoch 1826 | loss: nan | val_loss: nan | Time: 9109.2 ms
[2022-03-19 02:54:40	                main:574]	:	INFO	:	Epoch 1827 | loss: nan | val_loss: nan | Time: 9105.7 ms
[2022-03-19 02:54:49	                main:574]	:	INFO	:	Epoch 1828 | loss: nan | val_loss: nan | Time: 9116.6 ms
[2022-03-19 02:54:58	                main:574]	:	INFO	:	Epoch 1829 | loss: nan | val_loss: nan | Time: 9119.39 ms
[2022-03-19 02:55:07	                main:574]	:	INFO	:	Epoch 1830 | loss: nan | val_loss: nan | Time: 9120.68 ms
[2022-03-19 02:55:16	                main:574]	:	INFO	:	Epoch 1831 | loss: nan | val_loss: nan | Time: 9128.96 ms
[2022-03-19 02:55:26	                main:574]	:	INFO	:	Epoch 1832 | loss: nan | val_loss: nan | Time: 9110.64 ms
[2022-03-19 02:55:35	                main:574]	:	INFO	:	Epoch 1833 | loss: nan | val_loss: nan | Time: 9106.21 ms
[2022-03-19 02:55:44	                main:574]	:	INFO	:	Epoch 1834 | loss: nan | val_loss: nan | Time: 9112.22 ms
[2022-03-19 02:55:53	                main:574]	:	INFO	:	Epoch 1835 | loss: nan | val_loss: nan | Time: 9122.11 ms
[2022-03-19 02:56:02	                main:574]	:	INFO	:	Epoch 1836 | loss: nan | val_loss: nan | Time: 9105.5 ms
[2022-03-19 02:56:11	                main:574]	:	INFO	:	Epoch 1837 | loss: nan | val_loss: nan | Time: 9114.88 ms
[2022-03-19 02:56:20	                main:574]	:	INFO	:	Epoch 1838 | loss: nan | val_loss: nan | Time: 9123.47 ms
[2022-03-19 02:56:29	                main:574]	:	INFO	:	Epoch 1839 | loss: nan | val_loss: nan | Time: 9123.14 ms
[2022-03-19 02:56:39	                main:574]	:	INFO	:	Epoch 1840 | loss: nan | val_loss: nan | Time: 9106.2 ms
[2022-03-19 02:56:48	                main:574]	:	INFO	:	Epoch 1841 | loss: nan | val_loss: nan | Time: 9106.13 ms
[2022-03-19 02:56:57	                main:574]	:	INFO	:	Epoch 1842 | loss: nan | val_loss: nan | Time: 9106.72 ms
[2022-03-19 02:57:06	                main:574]	:	INFO	:	Epoch 1843 | loss: nan | val_loss: nan | Time: 9124.27 ms
[2022-03-19 02:57:15	                main:574]	:	INFO	:	Epoch 1844 | loss: nan | val_loss: nan | Time: 9119.59 ms
[2022-03-19 02:57:24	                main:574]	:	INFO	:	Epoch 1845 | loss: nan | val_loss: nan | Time: 9115.54 ms
[2022-03-19 02:57:33	                main:574]	:	INFO	:	Epoch 1846 | loss: nan | val_loss: nan | Time: 9118.74 ms
[2022-03-19 02:57:43	                main:574]	:	INFO	:	Epoch 1847 | loss: nan | val_loss: nan | Time: 9143.82 ms
[2022-03-19 02:57:52	                main:574]	:	INFO	:	Epoch 1848 | loss: nan | val_loss: nan | Time: 9518.72 ms
[2022-03-19 02:58:01	                main:574]	:	INFO	:	Epoch 1849 | loss: nan | val_loss: nan | Time: 9119.01 ms
[2022-03-19 02:58:10	                main:574]	:	INFO	:	Epoch 1850 | loss: nan | val_loss: nan | Time: 9122.81 ms
[2022-03-19 02:58:20	                main:574]	:	INFO	:	Epoch 1851 | loss: nan | val_loss: nan | Time: 9370.25 ms
[2022-03-19 02:58:29	                main:574]	:	INFO	:	Epoch 1852 | loss: nan | val_loss: nan | Time: 9112.14 ms
[2022-03-19 02:58:38	                main:574]	:	INFO	:	Epoch 1853 | loss: nan | val_loss: nan | Time: 9127.85 ms
[2022-03-19 02:58:47	                main:574]	:	INFO	:	Epoch 1854 | loss: nan | val_loss: nan | Time: 9114.65 ms
[2022-03-19 02:58:56	                main:574]	:	INFO	:	Epoch 1855 | loss: nan | val_loss: nan | Time: 9101.39 ms
[2022-03-19 02:59:05	                main:574]	:	INFO	:	Epoch 1856 | loss: nan | val_loss: nan | Time: 9116.37 ms
[2022-03-19 02:59:14	                main:574]	:	INFO	:	Epoch 1857 | loss: nan | val_loss: nan | Time: 9108.52 ms
[2022-03-19 02:59:24	                main:574]	:	INFO	:	Epoch 1858 | loss: nan | val_loss: nan | Time: 9147.2 ms
[2022-03-19 02:59:33	                main:574]	:	INFO	:	Epoch 1859 | loss: nan | val_loss: nan | Time: 9121.34 ms
[2022-03-19 02:59:42	                main:574]	:	INFO	:	Epoch 1860 | loss: nan | val_loss: nan | Time: 9105.49 ms
[2022-03-19 02:59:51	                main:574]	:	INFO	:	Epoch 1861 | loss: nan | val_loss: nan | Time: 9106.97 ms
[2022-03-19 03:00:00	                main:574]	:	INFO	:	Epoch 1862 | loss: nan | val_loss: nan | Time: 9130.77 ms
[2022-03-19 03:00:09	                main:574]	:	INFO	:	Epoch 1863 | loss: nan | val_loss: nan | Time: 9135.85 ms
[2022-03-19 03:00:18	                main:574]	:	INFO	:	Epoch 1864 | loss: nan | val_loss: nan | Time: 9130.83 ms
[2022-03-19 03:00:28	                main:574]	:	INFO	:	Epoch 1865 | loss: nan | val_loss: nan | Time: 9106.61 ms
[2022-03-19 03:00:37	                main:574]	:	INFO	:	Epoch 1866 | loss: nan | val_loss: nan | Time: 9130.97 ms
[2022-03-19 03:00:46	                main:574]	:	INFO	:	Epoch 1867 | loss: nan | val_loss: nan | Time: 9108.81 ms
[2022-03-19 03:00:55	                main:574]	:	INFO	:	Epoch 1868 | loss: nan | val_loss: nan | Time: 9709.26 ms
[2022-03-19 03:01:05	                main:574]	:	INFO	:	Epoch 1869 | loss: nan | val_loss: nan | Time: 9287.48 ms
[2022-03-19 03:01:14	                main:574]	:	INFO	:	Epoch 1870 | loss: nan | val_loss: nan | Time: 9099.74 ms
[2022-03-19 03:01:23	                main:574]	:	INFO	:	Epoch 1871 | loss: nan | val_loss: nan | Time: 9499.51 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce 840M)
[2022-03-19 20:32:07	                main:435]	:	INFO	:	Set logging level to 1
[2022-03-19 20:32:07	                main:441]	:	INFO	:	Running in BOINC Client mode
[2022-03-19 20:32:07	                main:444]	:	INFO	:	Resolving all filenames
[2022-03-19 20:32:07	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-03-19 20:32:07	                main:452]	:	INFO	:	Resolved: model.cfg => model.cfg (exists = 1)
[2022-03-19 20:32:07	                main:452]	:	INFO	:	Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-03-19 20:32:07	                main:452]	:	INFO	:	Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-03-19 20:32:07	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-03-19 20:32:07	                main:472]	:	INFO	:	Dataset filename: dataset.hdf5
[2022-03-19 20:32:07	                main:474]	:	INFO	:	Configuration: 
[2022-03-19 20:32:07	                main:475]	:	INFO	:	    Model type: GRU
[2022-03-19 20:32:07	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2022-03-19 20:32:07	                main:477]	:	INFO	:	    Max Epochs: 2048
[2022-03-19 20:32:07	                main:478]	:	INFO	:	    Batch Size: 128
[2022-03-19 20:32:07	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2022-03-19 20:32:07	                main:480]	:	INFO	:	    Patience: 10
[2022-03-19 20:32:07	                main:481]	:	INFO	:	    Hidden Width: 12
[2022-03-19 20:32:07	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2022-03-19 20:32:07	                main:483]	:	INFO	:	    # Backend Layers: 4
[2022-03-19 20:32:07	                main:484]	:	INFO	:	    # Threads: 1
[2022-03-19 20:32:07	                main:486]	:	INFO	:	Preparing Dataset
[2022-03-19 20:32:07	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from dataset.hdf5 into memory
[2022-03-19 20:32:11	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from dataset.hdf5 into memory
[2022-03-19 20:33:11	                load:106]	:	INFO	:	Successfully loaded dataset of 2048 examples into memory.
[2022-03-19 20:33:12	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xv from dataset.hdf5 into memory
[2022-03-19 20:34:16	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yv from dataset.hdf5 into memory
[2022-03-19 20:34:17	                load:106]	:	INFO	:	Successfully loaded dataset of 512 examples into memory.
[2022-03-19 20:34:17	                main:494]	:	INFO	:	Creating Model
[2022-03-19 20:34:18	                main:507]	:	INFO	:	Preparing config file
[2022-03-19 20:34:18	                main:511]	:	INFO	:	Found checkpoint, attempting to load... 
[2022-03-19 20:34:18	                main:512]	:	INFO	:	Loading config


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFB9E6A4F69

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 03/19/22 20:34:18
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\5;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 000000006fbc0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000a0b90000 00000000001f5000 C:\Windows\SYSTEM32\ntdll.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1566

ModLoad: 000000009ebf0000 00000000000be000 C:\Windows\System32\KERNEL32.DLL (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1566

ModLoad: 000000009e670000 00000000002c8000 C:\Windows\System32\KERNELBASE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1566

ModLoad: 000000009f340000 00000000001a0000 C:\Windows\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009e590000 0000000000022000 C:\Windows\System32\win32u.dll (6.2.19041.1586) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.19041.1586 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1586

ModLoad: 00000000a0230000 000000000002b000 C:\Windows\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.19041.1202 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1202

ModLoad: 0000000061f50000 0000000000058000 C:\ProgramData\BOINC\slots\5\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000009e480000 000000000010b000 C:\Windows\System32\gdi32full.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1566

ModLoad: 000000009e2e0000 000000000009d000 C:\Windows\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.19041.789 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.789

ModLoad: 000000009e380000 0000000000100000 C:\Windows\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.19041.789 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.789

ModLoad: 0000000085e40000 000000000008d000 C:\Windows\SYSTEM32\MSVCP140.dll (14.28.29910.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.28.29910.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.28.29910.0

ModLoad: 000000008a7e0000 00000000001e4000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.19041.867 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.867

ModLoad: 0000000087020000 000000000001a000 C:\Windows\SYSTEM32\VCRUNTIME140.dll (14.28.29910.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.28.29910.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.28.29910.0

ModLoad: 0000000087010000 000000000000c000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll (14.28.29910.0) (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\a01\_work\9\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.28.29910.0 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.28.29910.0

ModLoad: 000000009f550000 00000000000ae000 C:\Windows\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009ee20000 000000000009e000 C:\Windows\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.19041.546

ModLoad: 000000009eec0000 000000000009c000 C:\Windows\System32\sechost.dll (6.2.19041.1586) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000009ecb0000 0000000000125000 C:\Windows\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 0000000051bb0000 000000000abe7000 C:\ProgramData\BOINC\slots\5\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000061d40000 000000000020d000 C:\ProgramData\BOINC\slots\5\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 0000000061a60000 00000000002db000 C:\ProgramData\BOINC\slots\5\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000061a20000 0000000000040000 C:\ProgramData\BOINC\slots\5\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000014280000 0000000024b49000 C:\ProgramData\BOINC\slots\5\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000619d0000 0000000000045000 C:\ProgramData\BOINC\slots\5\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000004eb00000 00000000030a5000 C:\ProgramData\BOINC\slots\5\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 0000000044ab0000 00000000034b2000 C:\ProgramData\BOINC\slots\5\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 0000000095e00000 0000000000010000 C:\ProgramData\BOINC\slots\5\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 000000003b910000 000000000919f000 C:\ProgramData\BOINC\slots\5\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 00000000f5170000 0000000003d5b000 C:\ProgramData\BOINC\slots\5\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000f8ed0000 000000001b3a4000 C:\ProgramData\BOINC\slots\5\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 000000005d910000 0000000001eca000 C:\ProgramData\BOINC\slots\5\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 000000009edf0000 0000000000030000 C:\Windows\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000009c1c0000 0000000000012000 C:\Windows\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000a03a0000 0000000000470000 C:\Windows\System32\Setupapi.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009e940000 000000000004e000 C:\Windows\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.19041.1151 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1151

ModLoad: 000000009e2b0000 0000000000027000 C:\Windows\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcrypt.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009e050000 0000000000033000 C:\Windows\SYSTEM32\DEVOBJ.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1566

ModLoad: 000000009eb80000 0000000000068000 C:\Windows\System32\WINTRUST.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1566

ModLoad: 000000009ea20000 0000000000156000 C:\Windows\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009de90000 0000000000012000 C:\Windows\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000009f780000 0000000000744000 C:\Windows\System32\Shell32.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009c3c0000 0000000000794000 C:\Windows\SYSTEM32\windows.storage.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009fed0000 0000000000354000 C:\Windows\System32\combase.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009dd10000 000000000002c000 C:\Windows\SYSTEM32\Wldp.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : WLDP.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 000000009f600000 00000000000ad000 C:\Windows\System32\SHCORE.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 00000000a0af0000 0000000000055000 C:\Windows\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.19041.1526 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.19041.1526

ModLoad: 00000000603a0000 00000000002cd000 C:\Windows\system32\nvcuda.dll (30.0.15.1123) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
    File Version          : 30.0.15.1123
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.6.58 driver
    Product Version       : 30.0.15.1123

ModLoad: 00000000f3ad0000 00000000016a0000 C:\Windows\system32\DriverStore\FileRepository\nvaci.inf_amd64_a87281c4bd1a1369\nvcuda64.dll (30.0.15.1123) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
    File Version          : 30.0.15.1123
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.6.58 driver
    Product Version       : 30.0.15.1123

ModLoad: 0000000095a00000 000000000000a000 C:\Windows\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000959c0000 0000000000031000 C:\Windows\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptnet.pdb
    File Version          : 10.0.19041.906 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.906

ModLoad: 000000008f8f0000 0000000000148000 C:\Windows\SYSTEM32\drvstore.dll (6.2.19041.1566) (-exported- Symbols Loaded)
    Linked PDB Filename   : drvstore.pdb
    File Version          : 10.0.19041.1566 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1566

ModLoad: 000000009dc60000 000000000000c000 C:\Windows\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000008ebb0000 0000000000767000 C:\Windows\SYSTEM32\nvapi64.dll (30.0.15.1123) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
    File Version          : 30.0.15.1123
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA Windows drivers
    Product Version       : 30.0.15.1123

ModLoad: 000000009e990000 0000000000082000 C:\Windows\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.19041.1415 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1415



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3618, Write: 308, Other 844

- I/O Transfers Counters -
Read: 6691868, Write: 6043, Other 28990

- Paged Pool Usage -
QuotaPagedPoolUsage: 3465040, QuotaPeakPagedPoolUsage: 3465040
QuotaNonPagedPoolUsage: 36304, QuotaPeakNonPagedPoolUsage: 36368

- Virtual Memory Usage -
VirtualSize: -1513844736, PeakVirtualSize: -1023528960

- Pagefile Usage -
PagefileUsage: -1513844736, PeakPagefileUsage: -1513844736

- Working Set Size -
WorkingSetSize: 1133568000, PeakWorkingSetSize: 1133572096, PageFaultCount: 376784

*** Dump of thread ID 6284 (state: Ready): ***

- Information -
Status: Base Priority: Above Normal, Priority: Highest, , Kernel Time: 22500000.000000, User Time: 51718752.000000, Wait Time: 39449476.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFB9E6A4F69

- Registers -
rax=00000000a4b25b38 rbx=000000006ff24910 rcx=000000007df28200 rdx=00000000a0bb5b00 rsi=00000000a4b263a0 rdi=0000000019930520
r8=00000000a4b25b60 r9=00000000a4b25b88 r10=0000000000000004 r11=00000000a4b25b98 r12=000000005d936aa0 r13=00000000a4b265b0
r14=000000000254dbb0 r15=0000000000000000 rip=000000009e6a4f69 rsp=00000000a4b26240 rbp=00000000a4b26530
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
a4b26310 6fe3a884 00000040 0000004f 5d936e00 000005a7 KERNELBASE!RaiseException+0x0 
a4b26370 6fbe1ca6 00000080 2cb50000 7dcc25d8 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b26420 6fbe593b 00000000 00000030 00000020 00000008 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b26570 6fbe1a78 00000000 00000000 a4b268a0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b26640 6fbe1cfc a4b268a0 00000000 00000080 a4b26b70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b26700 6fbec4dd a4b26b70 a4b26b70 a4b26b70 a4b26b70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b269f0 6fbdb2fe a4b27008 a4b26b70 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b3f790 6fe1c6f8 2e72cd50 00000000 00000000 2e740fb0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b3f7d0 9ec07034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a4b3f800 a0be2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
a4b3f880 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 12968 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 39449476.000000

- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000a52ff9f0 rsi=0000000000000000 rdi=0000000000000064
r8=00000000a52ff9c8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000a0c2d3f4 rsp=00000000a52ff9c8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
a52ff9c0 9e6b962e a52ffa88 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0 
a52ffa60 6fbfa07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0 
a52ffa90 9ec07034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
a52ffac0 a0be2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
a52ffb40 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 9404 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 39449472.000000

- Registers -
rax=000000000000005b rbx=000000000000000a rcx=000000000000000a rdx=00000000a53ffb50 rsi=0000000000000000 rdi=000000000000000a
r8=00000000a53ff7f8 r9=000000002e8fe3b0 r10=0000000000000000 r11=00000000a53ffac0 r12=0000000000000064 r13=00000000a53ffb50
r14=00000000a53ff850 r15=0000000000000000 rip=00000000a0c2d8c4 rsp=00000000a53ff7f8 rbp=000000002e8fe3b0
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
a53ff7f0 9e6bcb20 00000000 a4c25000 a0cfa4d0 a4c25000 ntdll!ZwWaitForMultipleObjects+0x0 
a53ffae0 9e6bca1e 2cb637e0 006e0069 0000000a 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
a53ffb20 f3cad384 2e900d50 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0 
a53ffd60 f3d556fd 00000000 2e8fe3b0 00000000 00000000 nvcuda64!+0x0 
a53ffe10 f3cacd53 32caacd0 e84e8445 2ef00b60 00000000 nvcuda64!cuProfilerStop+0x0 
a53ffe40 f3fe4064 2ef00b60 00000000 00000000 00000000 nvcuda64!+0x0 
a53ffe70 9ec07034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
a53ffea0 a0be2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
a53fff20 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 10192 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 39441552.000000

- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002b8 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002b8
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=000000002e92d6b0 r13=0000000000000000
r14=00000000000002b8 r15=0000000000000000 rip=00000000a0c2cdf4 rsp=00000000a54ffb48 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
a54ffb40 9e691a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0 
a54ffbe0 f3ed596e 2ef00fb0 00000000 00000000 000002b8 KERNELBASE!WaitForSingleObjectEx+0x0 
a54ffe20 f3cacd53 00000000 2ef00fb0 f3ad0000 2e964440 nvcuda64!cuProfilerStop+0x0 
a54ffe50 f3fe4064 2ef00fb0 00000000 00000000 00000000 nvcuda64!+0x0 
a54ffe80 9ec07034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
a54ffeb0 a0be2651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
a54fff30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)