Task 12199003
| Name | ParityModified-1645996661-14266-0_0 |
| Workunit | 9380079 |
| Created | 27 Feb 2022, 21:17:44 UTC |
| Sent | 28 Feb 2022, 20:07:33 UTC |
| Report deadline | 8 Mar 2022, 20:07:33 UTC |
| Received | 14 Mar 2022, 18:58:36 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | -529697949 (0xE06D7363) Unknown error code |
| Computer ID | 7737 |
| Run time | 2 hours 3 min 53 sec |
| CPU time | 1 hours 41 min 38 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 1,508.72 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 1.54 GB |
| Peak swap size | 3.44 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 3765269347 (0xe06d7363)</message>
<stderr_txt>
main:478] : INFO : Batch Size: 128
[2022-03-14 14:38:43 main:479] : INFO : Learning Rate: 0.01
[2022-03-14 14:38:43 main:480] : INFO : Patience: 10
[2022-03-14 14:38:43 main:481] : INFO : Hidden Width: 12
[2022-03-14 14:38:43 main:482] : INFO : # Recurrent Layers: 4
[2022-03-14 14:38:44 main:483] : INFO : # Backend Layers: 4
[2022-03-14 14:38:44 main:484] : INFO : # Threads: 1
[2022-03-14 14:38:44 main:486] : INFO : Preparing Dataset
[2022-03-14 14:38:44 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-03-14 14:38:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-03-14 14:38:48 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-03-14 14:38:48 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-03-14 14:38:49 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-03-14 14:38:49 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-03-14 14:38:49 main:494] : INFO : Creating Model
[2022-03-14 14:38:49 main:507] : INFO : Preparing config file
[2022-03-14 14:38:49 main:511] : INFO : Found checkpoint, attempting to load...
[2022-03-14 14:38:49 main:512] : INFO : Loading config
[2022-03-14 14:38:49 main:514] : INFO : Loading state
[2022-03-14 14:38:51 main:559] : INFO : Loading DataLoader into Memory
[2022-03-14 14:38:51 main:562] : INFO : Starting Training
[2022-03-14 14:38:55 main:574] : INFO : Epoch 1708 | loss: nan | val_loss: nan | Time: 3883.76 ms
[2022-03-14 14:38:59 main:574] : INFO : Epoch 1709 | loss: nan | val_loss: nan | Time: 3376.97 ms
[2022-03-14 14:39:02 main:574] : INFO : Epoch 1710 | loss: nan | val_loss: nan | Time: 3364.01 ms
[2022-03-14 14:39:06 main:574] : INFO : Epoch 1711 | loss: nan | val_loss: nan | Time: 3474.74 ms
[2022-03-14 14:39:09 main:574] : INFO : Epoch 1712 | loss: nan | val_loss: nan | Time: 3295.14 ms
[2022-03-14 14:39:13 main:574] : INFO : Epoch 1713 | loss: nan | val_loss: nan | Time: 3254.42 ms
[2022-03-14 14:39:17 main:574] : INFO : Epoch 1714 | loss: nan | val_loss: nan | Time: 3633.1 ms
[2022-03-14 14:39:20 main:574] : INFO : Epoch 1715 | loss: nan | val_loss: nan | Time: 3284.57 ms
[2022-03-14 14:39:23 main:574] : INFO : Epoch 1716 | loss: nan | val_loss: nan | Time: 3293.18 ms
[2022-03-14 14:39:27 main:574] : INFO : Epoch 1717 | loss: nan | val_loss: nan | Time: 3517.03 ms
[2022-03-14 14:39:31 main:574] : INFO : Epoch 1718 | loss: nan | val_loss: nan | Time: 3331.52 ms
[2022-03-14 14:39:34 main:574] : INFO : Epoch 1719 | loss: nan | val_loss: nan | Time: 3258.56 ms
[2022-03-14 14:39:37 main:574] : INFO : Epoch 1720 | loss: nan | val_loss: nan | Time: 3289.07 ms
[2022-03-14 14:39:41 main:574] : INFO : Epoch 1721 | loss: nan | val_loss: nan | Time: 3678.19 ms
[2022-03-14 14:39:45 main:574] : INFO : Epoch 1722 | loss: nan | val_loss: nan | Time: 3365.6 ms
[2022-03-14 14:39:48 main:574] : INFO : Epoch 1723 | loss: nan | val_loss: nan | Time: 3311.62 ms
[2022-03-14 14:39:51 main:574] : INFO : Epoch 1724 | loss: nan | val_loss: nan | Time: 3356.6 ms
[2022-03-14 14:39:55 main:574] : INFO : Epoch 1725 | loss: nan | val_loss: nan | Time: 3357.7 ms
[2022-03-14 14:39:58 main:574] : INFO : Epoch 1726 | loss: nan | val_loss: nan | Time: 3265.87 ms
[2022-03-14 14:40:02 main:574] : INFO : Epoch 1727 | loss: nan | val_loss: nan | Time: 3334.32 ms
[2022-03-14 14:40:05 main:574] : INFO : Epoch 1728 | loss: nan | val_loss: nan | Time: 3390.75 ms
[2022-03-14 14:40:09 main:574] : INFO : Epoch 1729 | loss: nan | val_loss: nan | Time: 3249.03 ms
[2022-03-14 14:40:12 main:574] : INFO : Epoch 1730 | loss: nan | val_loss: nan | Time: 3359.87 ms
[2022-03-14 14:40:16 main:574] : INFO : Epoch 1731 | loss: nan | val_loss: nan | Time: 3410.59 ms
[2022-03-14 14:40:19 main:574] : INFO : Epoch 1732 | loss: nan | val_loss: nan | Time: 3296.79 ms
[2022-03-14 14:40:22 main:574] : INFO : Epoch 1733 | loss: nan | val_loss: nan | Time: 3203.85 ms
[2022-03-14 14:40:26 main:574] : INFO : Epoch 1734 | loss: nan | val_loss: nan | Time: 3577.61 ms
[2022-03-14 14:40:30 main:574] : INFO : Epoch 1735 | loss: nan | val_loss: nan | Time: 3304.49 ms
[2022-03-14 14:40:33 main:574] : INFO : Epoch 1736 | loss: nan | val_loss: nan | Time: 3214.96 ms
[2022-03-14 14:40:36 main:574] : INFO : Epoch 1737 | loss: nan | val_loss: nan | Time: 3243.05 ms
[2022-03-14 14:40:40 main:574] : INFO : Epoch 1738 | loss: nan | val_loss: nan | Time: 3467.19 ms
[2022-03-14 14:40:43 main:574] : INFO : Epoch 1739 | loss: nan | val_loss: nan | Time: 3185.87 ms
[2022-03-14 14:40:46 main:574] : INFO : Epoch 1740 | loss: nan | val_loss: nan | Time: 3156.92 ms
[2022-03-14 14:40:50 main:574] : INFO : Epoch 1741 | loss: nan | val_loss: nan | Time: 3498.03 ms
[2022-03-14 14:40:53 main:574] : INFO : Epoch 1742 | loss: nan | val_loss: nan | Time: 3287.52 ms
[2022-03-14 14:40:57 main:574] : INFO : Epoch 1743 | loss: nan | val_loss: nan | Time: 3356.79 ms
[2022-03-14 14:41:00 main:574] : INFO : Epoch 1744 | loss: nan | val_loss: nan | Time: 3444.39 ms
[2022-03-14 14:41:04 main:574] : INFO : Epoch 1745 | loss: nan | val_loss: nan | Time: 3575.99 ms
[2022-03-14 14:41:08 main:574] : INFO : Epoch 1746 | loss: nan | val_loss: nan | Time: 3363.94 ms
[2022-03-14 14:41:11 main:574] : INFO : Epoch 1747 | loss: nan | val_loss: nan | Time: 3327.96 ms
[2022-03-14 14:41:15 main:574] : INFO : Epoch 1748 | loss: nan | val_loss: nan | Time: 3343.19 ms
[2022-03-14 14:41:18 main:574] : INFO : Epoch 1749 | loss: nan | val_loss: nan | Time: 3376.02 ms
[2022-03-14 14:41:22 main:574] : INFO : Epoch 1750 | loss: nan | val_loss: nan | Time: 3214.99 ms
[2022-03-14 14:41:25 main:574] : INFO : Epoch 1751 | loss: nan | val_loss: nan | Time: 3418.81 ms
[2022-03-14 14:41:28 main:574] : INFO : Epoch 1752 | loss: nan | val_loss: nan | Time: 3212.82 ms
[2022-03-14 14:41:32 main:574] : INFO : Epoch 1753 | loss: nan | val_loss: nan | Time: 3146.16 ms
[2022-03-14 14:41:35 main:574] : INFO : Epoch 1754 | loss: nan | val_loss: nan | Time: 3472.65 ms
[2022-03-14 14:41:39 main:574] : INFO : Epoch 1755 | loss: nan | val_loss: nan | Time: 3663.94 ms
[2022-03-14 14:41:42 main:574] : INFO : Epoch 1756 | loss: nan | val_loss: nan | Time: 3162.4 ms
[2022-03-14 14:41:46 main:574] : INFO : Epoch 1757 | loss: nan | val_loss: nan | Time: 3377.96 ms
[2022-03-14 14:41:49 main:574] : INFO : Epoch 1758 | loss: nan | val_loss: nan | Time: 3555.06 ms
[2022-03-14 14:41:53 main:574] : INFO : Epoch 1759 | loss: nan | val_loss: nan | Time: 3247.52 ms
[2022-03-14 14:41:56 main:574] : INFO : Epoch 1760 | loss: nan | val_loss: nan | Time: 3238.88 ms
[2022-03-14 14:42:00 main:574] : INFO : Epoch 1761 | loss: nan | val_loss: nan | Time: 3421.34 ms
[2022-03-14 14:42:03 main:574] : INFO : Epoch 1762 | loss: nan | val_loss: nan | Time: 3408.16 ms
[2022-03-14 14:42:07 main:574] : INFO : Epoch 1763 | loss: nan | val_loss: nan | Time: 3169.76 ms
[2022-03-14 14:42:10 main:574] : INFO : Epoch 1764 | loss: nan | val_loss: nan | Time: 3200.27 ms
[2022-03-14 14:42:14 main:574] : INFO : Epoch 1765 | loss: nan | val_loss: nan | Time: 3410.08 ms
[2022-03-14 14:42:17 main:574] : INFO : Epoch 1766 | loss: nan | val_loss: nan | Time: 3398.67 ms
[2022-03-14 14:42:20 main:574] : INFO : Epoch 1767 | loss: nan | val_loss: nan | Time: 3237.13 ms
[2022-03-14 14:42:24 main:574] : INFO : Epoch 1768 | loss: nan | val_loss: nan | Time: 3333.95 ms
[2022-03-14 14:42:27 main:574] : INFO : Epoch 1769 | loss: nan | val_loss: nan | Time: 3296.45 ms
[2022-03-14 14:42:31 main:574] : INFO : Epoch 1770 | loss: nan | val_loss: nan | Time: 3205.44 ms
[2022-03-14 14:42:34 main:574] : INFO : Epoch 1771 | loss: nan | val_loss: nan | Time: 3342.91 ms
[2022-03-14 14:42:38 main:574] : INFO : Epoch 1772 | loss: nan | val_loss: nan | Time: 3547.41 ms
[2022-03-14 14:42:41 main:574] : INFO : Epoch 1773 | loss: nan | val_loss: nan | Time: 3348.52 ms
[2022-03-14 14:42:45 main:574] : INFO : Epoch 1774 | loss: nan | val_loss: nan | Time: 3333.27 ms
[2022-03-14 14:42:48 main:574] : INFO : Epoch 1775 | loss: nan | val_loss: nan | Time: 3387.63 ms
[2022-03-14 14:42:52 main:574] : INFO : Epoch 1776 | loss: nan | val_loss: nan | Time: 3264.5 ms
[2022-03-14 14:42:55 main:574] : INFO : Epoch 1777 | loss: nan | val_loss: nan | Time: 3339.22 ms
[2022-03-14 14:42:59 main:574] : INFO : Epoch 1778 | loss: nan | val_loss: nan | Time: 3459.21 ms
[2022-03-14 14:43:02 main:574] : INFO : Epoch 1779 | loss: nan | val_loss: nan | Time: 3291.68 ms
[2022-03-14 14:43:05 main:574] : INFO : Epoch 1780 | loss: nan | val_loss: nan | Time: 3171.45 ms
[2022-03-14 14:43:09 main:574] : INFO : Epoch 1781 | loss: nan | val_loss: nan | Time: 3182.64 ms
[2022-03-14 14:43:13 main:574] : INFO : Epoch 1782 | loss: nan | val_loss: nan | Time: 3501.89 ms
[2022-03-14 14:43:16 main:574] : INFO : Epoch 1783 | loss: nan | val_loss: nan | Time: 3356.67 ms
[2022-03-14 14:43:20 main:574] : INFO : Epoch 1784 | loss: nan | val_loss: nan | Time: 3517.53 ms
[2022-03-14 14:43:23 main:574] : INFO : Epoch 1785 | loss: nan | val_loss: nan | Time: 3441.46 ms
[2022-03-14 14:43:27 main:574] : INFO : Epoch 1786 | loss: nan | val_loss: nan | Time: 3351.17 ms
[2022-03-14 14:43:30 main:574] : INFO : Epoch 1787 | loss: nan | val_loss: nan | Time: 3218.52 ms
[2022-03-14 14:43:34 main:574] : INFO : Epoch 1788 | loss: nan | val_loss: nan | Time: 3402.75 ms
[2022-03-14 14:43:37 main:574] : INFO : Epoch 1789 | loss: nan | val_loss: nan | Time: 3541.24 ms
[2022-03-14 14:43:41 main:574] : INFO : Epoch 1790 | loss: nan | val_loss: nan | Time: 3200.6 ms
[2022-03-14 14:43:44 main:574] : INFO : Epoch 1791 | loss: nan | val_loss: nan | Time: 3405.1 ms
[2022-03-14 14:43:48 main:574] : INFO : Epoch 1792 | loss: nan | val_loss: nan | Time: 3318.24 ms
[2022-03-14 14:43:51 main:574] : INFO : Epoch 1793 | loss: nan | val_loss: nan | Time: 3217.52 ms
[2022-03-14 14:43:54 main:574] : INFO : Epoch 1794 | loss: nan | val_loss: nan | Time: 3357.43 ms
[2022-03-14 14:43:58 main:574] : INFO : Epoch 1795 | loss: nan | val_loss: nan | Time: 3312.57 ms
[2022-03-14 14:44:01 main:574] : INFO : Epoch 1796 | loss: nan | val_loss: nan | Time: 3275.61 ms
[2022-03-14 14:44:05 main:574] : INFO : Epoch 1797 | loss: nan | val_loss: nan | Time: 3233.85 ms
[2022-03-14 14:44:08 main:574] : INFO : Epoch 1798 | loss: nan | val_loss: nan | Time: 3334.13 ms
[2022-03-14 14:44:11 main:574] : INFO : Epoch 1799 | loss: nan | val_loss: nan | Time: 3379.21 ms
[2022-03-14 14:44:15 main:574] : INFO : Epoch 1800 | loss: nan | val_loss: nan | Time: 3317.16 ms
[2022-03-14 14:44:18 main:574] : INFO : Epoch 1801 | loss: nan | val_loss: nan | Time: 3162.76 ms
[2022-03-14 14:44:22 main:574] : INFO : Epoch 1802 | loss: nan | val_loss: nan | Time: 3424.19 ms
[2022-03-14 14:44:25 main:574] : INFO : Epoch 1803 | loss: nan | val_loss: nan | Time: 3425.85 ms
[2022-03-14 14:44:29 main:574] : INFO : Epoch 1804 | loss: nan | val_loss: nan | Time: 3394.99 ms
[2022-03-14 14:44:32 main:574] : INFO : Epoch 1805 | loss: nan | val_loss: nan | Time: 3418.59 ms
[2022-03-14 14:44:36 main:574] : INFO : Epoch 1806 | loss: nan | val_loss: nan | Time: 3308.76 ms
[2022-03-14 14:44:39 main:574] : INFO : Epoch 1807 | loss: nan | val_loss: nan | Time: 3180.99 ms
[2022-03-14 14:44:43 main:574] : INFO : Epoch 1808 | loss: nan | val_loss: nan | Time: 3480.62 ms
[2022-03-14 14:44:46 main:574] : INFO : Epoch 1809 | loss: nan | val_loss: nan | Time: 3298.24 ms
[2022-03-14 14:44:50 main:574] : INFO : Epoch 1810 | loss: nan | val_loss: nan | Time: 3308.41 ms
[2022-03-14 14:44:53 main:574] : INFO : Epoch 1811 | loss: nan | val_loss: nan | Time: 3309.37 ms
[2022-03-14 14:44:56 main:574] : INFO : Epoch 1812 | loss: nan | val_loss: nan | Time: 3277.59 ms
[2022-03-14 14:45:00 main:574] : INFO : Epoch 1813 | loss: nan | val_loss: nan | Time: 3621.02 ms
[2022-03-14 14:45:03 main:574] : INFO : Epoch 1814 | loss: nan | val_loss: nan | Time: 3200.5 ms
[2022-03-14 14:45:07 main:574] : INFO : Epoch 1815 | loss: nan | val_loss: nan | Time: 3393.33 ms
[2022-03-14 14:45:11 main:574] : INFO : Epoch 1816 | loss: nan | val_loss: nan | Time: 3328.04 ms
[2022-03-14 14:45:14 main:574] : INFO : Epoch 1817 | loss: nan | val_loss: nan | Time: 3199.68 ms
[2022-03-14 14:45:18 main:574] : INFO : Epoch 1818 | loss: nan | val_loss: nan | Time: 3481.16 ms
[2022-03-14 14:45:21 main:574] : INFO : Epoch 1819 | loss: nan | val_loss: nan | Time: 3312.18 ms
[2022-03-14 14:45:24 main:574] : INFO : Epoch 1820 | loss: nan | val_loss: nan | Time: 3223.41 ms
[2022-03-14 14:45:28 main:574] : INFO : Epoch 1821 | loss: nan | val_loss: nan | Time: 3252.79 ms
[2022-03-14 14:45:31 main:574] : INFO : Epoch 1822 | loss: nan | val_loss: nan | Time: 3641.95 ms
[2022-03-14 14:45:35 main:574] : INFO : Epoch 1823 | loss: nan | val_loss: nan | Time: 3488.26 ms
[2022-03-14 14:45:39 main:574] : INFO : Epoch 1824 | loss: nan | val_loss: nan | Time: 3500.03 ms
[2022-03-14 14:45:42 main:574] : INFO : Epoch 1825 | loss: nan | val_loss: nan | Time: 3315.62 ms
[2022-03-14 14:45:46 main:574] : INFO : Epoch 1826 | loss: nan | val_loss: nan | Time: 3364.83 ms
[2022-03-14 14:45:49 main:574] : INFO : Epoch 1827 | loss: nan | val_loss: nan | Time: 3224.4 ms
[2022-03-14 14:45:52 main:574] : INFO : Epoch 1828 | loss: nan | val_loss: nan | Time: 3273.3 ms
[2022-03-14 14:45:56 main:574] : INFO : Epoch 1829 | loss: nan | val_loss: nan | Time: 3537.64 ms
[2022-03-14 14:45:59 main:574] : INFO : Epoch 1830 | loss: nan | val_loss: nan | Time: 3247.32 ms
[2022-03-14 14:46:03 main:574] : INFO : Epoch 1831 | loss: nan | val_loss: nan | Time: 3505.17 ms
[2022-03-14 14:46:07 main:574] : INFO : Epoch 1832 | loss: nan | val_loss: nan | Time: 3418.37 ms
[2022-03-14 14:46:10 main:574] : INFO : Epoch 1833 | loss: nan | val_loss: nan | Time: 3440.73 ms
[2022-03-14 14:46:14 main:574] : INFO : Epoch 1834 | loss: nan | val_loss: nan | Time: 3229.73 ms
[2022-03-14 14:46:17 main:574] : INFO : Epoch 1835 | loss: nan | val_loss: nan | Time: 3319.94 ms
[2022-03-14 14:46:21 main:574] : INFO : Epoch 1836 | loss: nan | val_loss: nan | Time: 3484.43 ms
[2022-03-14 14:46:24 main:574] : INFO : Epoch 1837 | loss: nan | val_loss: nan | Time: 3172.51 ms
[2022-03-14 14:46:27 main:574] : INFO : Epoch 1838 | loss: nan | val_loss: nan | Time: 3248.23 ms
[2022-03-14 14:46:31 main:574] : INFO : Epoch 1839 | loss: nan | val_loss: nan | Time: 3313.34 ms
[2022-03-14 14:46:34 main:574] : INFO : Epoch 1840 | loss: nan | val_loss: nan | Time: 3302.46 ms
[2022-03-14 14:46:38 main:574] : INFO : Epoch 1841 | loss: nan | val_loss: nan | Time: 3225.4 ms
[2022-03-14 14:46:41 main:574] : INFO : Epoch 1842 | loss: nan | val_loss: nan | Time: 3411.64 ms
[2022-03-14 14:46:45 main:574] : INFO : Epoch 1843 | loss: nan | val_loss: nan | Time: 3356.52 ms
[2022-03-14 14:46:48 main:574] : INFO : Epoch 1844 | loss: nan | val_loss: nan | Time: 3201.26 ms
[2022-03-14 14:46:51 main:574] : INFO : Epoch 1845 | loss: nan | val_loss: nan | Time: 3428.99 ms
[2022-03-14 14:46:55 main:574] : INFO : Epoch 1846 | loss: nan | val_loss: nan | Time: 3397.07 ms
[2022-03-14 14:46:58 main:574] : INFO : Epoch 1847 | loss: nan | val_loss: nan | Time: 3315.14 ms
[2022-03-14 14:47:02 main:574] : INFO : Epoch 1848 | loss: nan | val_loss: nan | Time: 3237.07 ms
[2022-03-14 14:47:05 main:574] : INFO : Epoch 1849 | loss: nan | val_loss: nan | Time: 3366.83 ms
[2022-03-14 14:47:09 main:574] : INFO : Epoch 1850 | loss: nan | val_loss: nan | Time: 3295.02 ms
[2022-03-14 14:47:12 main:574] : INFO : Epoch 1851 | loss: nan | val_loss: nan | Time: 3171.95 ms
[2022-03-14 14:47:16 main:574] : INFO : Epoch 1852 | loss: nan | val_loss: nan | Time: 3384.13 ms
[2022-03-14 14:47:19 main:574] : INFO : Epoch 1853 | loss: nan | val_loss: nan | Time: 3536.73 ms
[2022-03-14 14:47:23 main:574] : INFO : Epoch 1854 | loss: nan | val_loss: nan | Time: 3173.51 ms
[2022-03-14 14:47:26 main:574] : INFO : Epoch 1855 | loss: nan | val_loss: nan | Time: 3337.9 ms
[2022-03-14 14:47:30 main:574] : INFO : Epoch 1856 | loss: nan | val_loss: nan | Time: 3518.54 ms
[2022-03-14 14:47:33 main:574] : INFO : Epoch 1857 | loss: nan | val_loss: nan | Time: 3208.68 ms
[2022-03-14 14:47:36 main:574] : INFO : Epoch 1858 | loss: nan | val_loss: nan | Time: 3155.36 ms
[2022-03-14 14:47:40 main:574] : INFO : Epoch 1859 | loss: nan | val_loss: nan | Time: 3547.22 ms
[2022-03-14 14:47:43 main:574] : INFO : Epoch 1860 | loss: nan | val_loss: nan | Time: 3339.7 ms
[2022-03-14 14:47:47 main:574] : INFO : Epoch 1861 | loss: nan | val_loss: nan | Time: 3243.61 ms
[2022-03-14 14:47:50 main:574] : INFO : Epoch 1862 | loss: nan | val_loss: nan | Time: 3356.65 ms
[2022-03-14 14:47:54 main:574] : INFO : Epoch 1863 | loss: nan | val_loss: nan | Time: 3356.34 ms
[2022-03-14 14:47:57 main:574] : INFO : Epoch 1864 | loss: nan | val_loss: nan | Time: 3261.17 ms
[2022-03-14 14:48:01 main:574] : INFO : Epoch 1865 | loss: nan | val_loss: nan | Time: 3382.25 ms
[2022-03-14 14:48:04 main:574] : INFO : Epoch 1866 | loss: nan | val_loss: nan | Time: 3431.4 ms
[2022-03-14 14:48:08 main:574] : INFO : Epoch 1867 | loss: nan | val_loss: nan | Time: 3333.31 ms
[2022-03-14 14:48:11 main:574] : INFO : Epoch 1868 | loss: nan | val_loss: nan | Time: 3284.25 ms
[2022-03-14 14:48:15 main:574] : INFO : Epoch 1869 | loss: nan | val_loss: nan | Time: 3519.58 ms
[2022-03-14 14:48:18 main:574] : INFO : Epoch 1870 | loss: nan | val_loss: nan | Time: 3300.66 ms
[2022-03-14 14:48:21 main:574] : INFO : Epoch 1871 | loss: nan | val_loss: nan | Time: 3271.52 ms
[2022-03-14 14:48:25 main:574] : INFO : Epoch 1872 | loss: nan | val_loss: nan | Time: 3388.19 ms
[2022-03-14 14:48:29 main:574] : INFO : Epoch 1873 | loss: nan | val_loss: nan | Time: 3397.83 ms
[2022-03-14 14:48:32 main:574] : INFO : Epoch 1874 | loss: nan | val_loss: nan | Time: 3318.45 ms
[2022-03-14 14:48:35 main:574] : INFO : Epoch 1875 | loss: nan | val_loss: nan | Time: 3293.36 ms
[2022-03-14 14:48:39 main:574] : INFO : Epoch 1876 | loss: nan | val_loss: nan | Time: 3359.22 ms
[2022-03-14 14:48:42 main:574] : INFO : Epoch 1877 | loss: nan | val_loss: nan | Time: 3440.05 ms
[2022-03-14 14:48:46 main:574] : INFO : Epoch 1878 | loss: nan | val_loss: nan | Time: 3318.88 ms
[2022-03-14 14:48:49 main:574] : INFO : Epoch 1879 | loss: nan | val_loss: nan | Time: 3334.79 ms
[2022-03-14 14:48:53 main:574] : INFO : Epoch 1880 | loss: nan | val_loss: nan | Time: 3621.26 ms
[2022-03-14 14:48:57 main:574] : INFO : Epoch 1881 | loss: nan | val_loss: nan | Time: 3273.07 ms
[2022-03-14 14:49:00 main:574] : INFO : Epoch 1882 | loss: nan | val_loss: nan | Time: 3160.05 ms
[2022-03-14 14:49:03 main:574] : INFO : Epoch 1883 | loss: nan | val_loss: nan | Time: 3289.06 ms
[2022-03-14 14:49:07 main:574] : INFO : Epoch 1884 | loss: nan | val_loss: nan | Time: 3375.76 ms
[2022-03-14 14:49:10 main:574] : INFO : Epoch 1885 | loss: nan | val_loss: nan | Time: 3510.46 ms
[2022-03-14 14:49:14 main:574] : INFO : Epoch 1886 | loss: nan | val_loss: nan | Time: 3343.42 ms
[2022-03-14 14:49:17 main:574] : INFO : Epoch 1887 | loss: nan | val_loss: nan | Time: 3350.6 ms
[2022-03-14 14:49:21 main:574] : INFO : Epoch 1888 | loss: nan | val_loss: nan | Time: 3273.52 ms
[2022-03-14 14:49:24 main:574] : INFO : Epoch 1889 | loss: nan | val_loss: nan | Time: 3272.73 ms
[2022-03-14 14:49:28 main:574] : INFO : Epoch 1890 | loss: nan | val_loss: nan | Time: 3363.1 ms
[2022-03-14 14:49:31 main:574] : INFO : Epoch 1891 | loss: nan | val_loss: nan | Time: 3309.68 ms
[2022-03-14 14:49:34 main:574] : INFO : Epoch 1892 | loss: nan | val_loss: nan | Time: 3307.56 ms
[2022-03-14 14:49:38 main:574] : INFO : Epoch 1893 | loss: nan | val_loss: nan | Time: 3427.38 ms
[2022-03-14 14:49:41 main:574] : INFO : Epoch 1894 | loss: nan | val_loss: nan | Time: 3438.56 ms
[2022-03-14 14:49:45 main:574] : INFO : Epoch 1895 | loss: nan | val_loss: nan | Time: 3204.78 ms
[2022-03-14 14:49:48 main:574] : INFO : Epoch 1896 | loss: nan | val_loss: nan | Time: 3390.12 ms
[2022-03-14 14:49:52 main:574] : INFO : Epoch 1897 | loss: nan | val_loss: nan | Time: 3459.52 ms
[2022-03-14 14:49:55 main:574] : INFO : Epoch 1898 | loss: nan | val_loss: nan | Time: 3271.39 ms
[2022-03-14 14:49:59 main:574] : INFO : Epoch 1899 | loss: nan | val_loss: nan | Time: 3689.08 ms
[2022-03-14 14:50:03 main:574] : INFO : Epoch 1900 | loss: nan | val_loss: nan | Time: 3515.51 ms
[2022-03-14 14:50:06 main:574] : INFO : Epoch 1901 | loss: nan | val_loss: nan | Time: 3407.46 ms
[2022-03-14 14:50:10 main:574] : INFO : Epoch 1902 | loss: nan | val_loss: nan | Time: 3319.2 ms
[2022-03-14 14:50:13 main:574] : INFO : Epoch 1903 | loss: nan | val_loss: nan | Time: 3301.18 ms
[2022-03-14 14:50:17 main:574] : INFO : Epoch 1904 | loss: nan | val_loss: nan | Time: 3492.52 ms
[2022-03-14 14:50:20 main:574] : INFO : Epoch 1905 | loss: nan | val_loss: nan | Time: 3246.79 ms
[2022-03-14 14:50:24 main:574] : INFO : Epoch 1906 | loss: nan | val_loss: nan | Time: 3212.29 ms
[2022-03-14 14:50:27 main:574] : INFO : Epoch 1907 | loss: nan | val_loss: nan | Time: 3228.24 ms
[2022-03-14 14:50:31 main:574] : INFO : Epoch 1908 | loss: nan | val_loss: nan | Time: 3457.52 ms
[2022-03-14 14:50:34 main:574] : INFO : Epoch 1909 | loss: nan | val_loss: nan | Time: 3232 ms
[2022-03-14 14:50:38 main:574] : INFO : Epoch 1910 | loss: nan | val_loss: nan | Time: 3743.15 ms
[2022-03-14 14:50:42 main:574] : INFO : Epoch 1911 | loss: nan | val_loss: nan | Time: 3438.09 ms
[2022-03-14 14:50:45 main:574] : INFO : Epoch 1912 | loss: nan | val_loss: nan | Time: 3200.55 ms
[2022-03-14 14:50:48 main:574] : INFO : Epoch 1913 | loss: nan | val_loss: nan | Time: 3371.04 ms
[2022-03-14 14:50:52 main:574] : INFO : Epoch 1914 | loss: nan | val_loss: nan | Time: 3346.2 ms
[2022-03-14 14:50:55 main:574] : INFO : Epoch 1915 | loss: nan | val_loss: nan | Time: 3291.84 ms
[2022-03-14 14:50:59 main:574] : INFO : Epoch 1916 | loss: nan | val_loss: nan | Time: 3179.1 ms
[2022-03-14 14:51:02 main:574] : INFO : Epoch 1917 | loss: nan | val_loss: nan | Time: 3442.75 ms
[2022-03-14 14:51:06 main:574] : INFO : Epoch 1918 | loss: nan | val_loss: nan | Time: 3438.98 ms
[2022-03-14 14:51:09 main:574] : INFO : Epoch 1919 | loss: nan | val_loss: nan | Time: 3103.86 ms
[2022-03-14 14:51:13 main:574] : INFO : Epoch 1920 | loss: nan | val_loss: nan | Time: 3385.31 ms
[2022-03-14 14:51:16 main:574] : INFO : Epoch 1921 | loss: nan | val_loss: nan | Time: 3481.28 ms
[2022-03-14 14:51:19 main:574] : INFO : Epoch 1922 | loss: nan | val_loss: nan | Time: 3188.79 ms
[2022-03-14 14:51:23 main:574] : INFO : Epoch 1923 | loss: nan | val_loss: nan | Time: 3152.99 ms
[2022-03-14 14:51:26 main:574] : INFO : Epoch 1924 | loss: nan | val_loss: nan | Time: 3553.64 ms
[2022-03-14 14:51:30 main:574] : INFO : Epoch 1925 | loss: nan | val_loss: nan | Time: 3332.83 ms
[2022-03-14 14:51:33 main:574] : INFO : Epoch 1926 | loss: nan | val_loss: nan | Time: 3207.83 ms
[2022-03-14 14:51:37 main:574] : INFO : Epoch 1927 | loss: nan | val_loss: nan | Time: 3305.86 ms
[2022-03-14 14:51:40 main:574] : INFO : Epoch 1928 | loss: nan | val_loss: nan | Time: 3314.84 ms
[2022-03-14 14:51:43 main:574] : INFO : Epoch 1929 | loss: nan | val_loss: nan | Time: 3259.31 ms
[2022-03-14 14:51:47 main:574] : INFO : Epoch 1930 | loss: nan | val_loss: nan | Time: 3304.66 ms
[2022-03-14 14:51:50 main:574] : INFO : Epoch 1931 | loss: nan | val_loss: nan | Time: 3450.43 ms
[2022-03-14 14:51:54 main:574] : INFO : Epoch 1932 | loss: nan | val_loss: nan | Time: 3483.13 ms
[2022-03-14 14:51:57 main:574] : INFO : Epoch 1933 | loss: nan | val_loss: nan | Time: 3266.81 ms
[2022-03-14 14:52:01 main:574] : INFO : Epoch 1934 | loss: nan | val_loss: nan | Time: 3415.94 ms
[2022-03-14 14:52:04 main:574] : INFO : Epoch 1935 | loss: nan | val_loss: nan | Time: 3418.3 ms
[2022-03-14 14:52:08 main:574] : INFO : Epoch 1936 | loss: nan | val_loss: nan | Time: 3359.78 ms
[2022-03-14 14:52:11 main:574] : INFO : Epoch 1937 | loss: nan | val_loss: nan | Time: 3275.83 ms
[2022-03-14 14:52:15 main:574] : INFO : Epoch 1938 | loss: nan | val_loss: nan | Time: 3489.44 ms
[2022-03-14 14:52:18 main:574] : INFO : Epoch 1939 | loss: nan | val_loss: nan | Time: 3202.26 ms
[2022-03-14 14:52:22 main:574] : INFO : Epoch 1940 | loss: nan | val_loss: nan | Time: 3283.67 ms
[2022-03-14 14:52:25 main:574] : INFO : Epoch 1941 | loss: nan | val_loss: nan | Time: 3417.18 ms
[2022-03-14 14:52:29 main:574] : INFO : Epoch 1942 | loss: nan | val_loss: nan | Time: 3241.83 ms
[2022-03-14 14:52:32 main:574] : INFO : Epoch 1943 | loss: nan | val_loss: nan | Time: 3275.86 ms
[2022-03-14 14:52:36 main:574] : INFO : Epoch 1944 | loss: nan | val_loss: nan | Time: 3430.21 ms
[2022-03-14 14:52:39 main:574] : INFO : Epoch 1945 | loss: nan | val_loss: nan | Time: 3421.56 ms
[2022-03-14 14:52:43 main:574] : INFO : Epoch 1946 | loss: nan | val_loss: nan | Time: 3279.35 ms
[2022-03-14 14:52:46 main:574] : INFO : Epoch 1947 | loss: nan | val_loss: nan | Time: 3364.02 ms
[2022-03-14 14:52:50 main:574] : INFO : Epoch 1948 | loss: nan | val_loss: nan | Time: 3383.87 ms
[2022-03-14 14:52:53 main:574] : INFO : Epoch 1949 | loss: nan | val_loss: nan | Time: 3375.19 ms
[2022-03-14 14:52:57 main:574] : INFO : Epoch 1950 | loss: nan | val_loss: nan | Time: 3312.31 ms
[2022-03-14 14:53:00 main:574] : INFO : Epoch 1951 | loss: nan | val_loss: nan | Time: 3434.41 ms
[2022-03-14 14:53:04 main:574] : INFO : Epoch 1952 | loss: nan | val_loss: nan | Time: 3366.22 ms
[2022-03-14 14:53:07 main:574] : INFO : Epoch 1953 | loss: nan | val_loss: nan | Time: 3382.15 ms
[2022-03-14 14:53:11 main:574] : INFO : Epoch 1954 | loss: nan | val_loss: nan | Time: 3303.08 ms
[2022-03-14 14:53:14 main:574] : INFO : Epoch 1955 | loss: nan | val_loss: nan | Time: 3397.87 ms
[2022-03-14 14:53:17 main:574] : INFO : Epoch 1956 | loss: nan | val_loss: nan | Time: 3259.37 ms
[2022-03-14 14:53:21 main:574] : INFO : Epoch 1957 | loss: nan | val_loss: nan | Time: 3254.04 ms
[2022-03-14 14:53:24 main:574] : INFO : Epoch 1958 | loss: nan | val_loss: nan | Time: 3497.79 ms
[2022-03-14 14:53:28 main:574] : INFO : Epoch 1959 | loss: nan | val_loss: nan | Time: 3157.54 ms
[2022-03-14 14:53:31 main:574] : INFO : Epoch 1960 | loss: nan | val_loss: nan | Time: 3217.95 ms
[2022-03-14 14:53:35 main:574] : INFO : Epoch 1961 | loss: nan | val_loss: nan | Time: 3453.88 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce GTX 960M)
[2022-03-14 14:56:36 main:435] : INFO : Set logging level to 1
[2022-03-14 14:56:36 main:441] : INFO : Running in BOINC Client mode
[2022-03-14 14:56:36 main:444] : INFO : Resolving all filenames
[2022-03-14 14:56:36 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-03-14 14:56:37 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1)
[2022-03-14 14:56:37 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-03-14 14:56:37 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 0)
[2022-03-14 14:56:37 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-03-14 14:56:37 main:472] : INFO : Dataset filename: dataset.hdf5
[2022-03-14 14:56:37 main:474] : INFO : Configuration:
[2022-03-14 14:56:38 main:475] : INFO : Model type: GRU
[2022-03-14 14:56:38 main:476] : INFO : Validation Loss Threshold: 0.0001
[2022-03-14 14:56:38 main:477] : INFO : Max Epochs: 2048
[2022-03-14 14:56:38 main:478] : INFO : Batch Size: 128
[2022-03-14 14:56:38 main:479] : INFO : Learning Rate: 0.01
[2022-03-14 14:56:38 main:480] : INFO : Patience: 10
[2022-03-14 14:56:38 main:481] : INFO : Hidden Width: 12
[2022-03-14 14:56:39 main:482] : INFO : # Recurrent Layers: 4
[2022-03-14 14:56:39 main:483] : INFO : # Backend Layers: 4
[2022-03-14 14:56:39 main:484] : INFO : # Threads: 1
[2022-03-14 14:56:39 main:486] : INFO : Preparing Dataset
[2022-03-14 14:56:39 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-03-14 14:56:40 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
[2022-03-14 14:56:45 load:106] : INFO : Successfully loaded dataset of 2048 examples into memory.
[2022-03-14 14:56:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xv from dataset.hdf5 into memory
[2022-03-14 14:56:45 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yv from dataset.hdf5 into memory
[2022-03-14 14:56:45 load:106] : INFO : Successfully loaded dataset of 512 examples into memory.
[2022-03-14 14:56:45 main:494] : INFO : Creating Model
[2022-03-14 14:56:45 main:507] : INFO : Preparing config file
[2022-03-14 14:56:46 main:511] : INFO : Found checkpoint, attempting to load...
[2022-03-14 14:56:46 main:512] : INFO : Loading config
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFCA7AE4F69
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 03/14/22 14:56:46
Install Directory : C:\Program Files\BOINC\
Data Directory : F:\BOINC_Data
Project Symstore :
LoadLibraryA( F:\BOINC_Data\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( F:\BOINC_Data\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( F:\BOINC_Data\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( F:\BOINC_Data\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: F:\BOINC_Data\slots\2;F:\BOINC_Data\projects\www.mlcathome.org_mlcathome
ModLoad: 000000002f2e0000 00000000003df000 F:\BOINC_Data\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000aa130000 00000000001f5000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1466
ModLoad: 00000000a9380000 00000000000be000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1503 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1503
ModLoad: 00000000a7ab0000 00000000002c8000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1503 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1503
ModLoad: 00000000a89a0000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a78e0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.1526) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1526 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1526
ModLoad: 00000000a9e20000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 00000000a79a0000 000000000010d000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1466
ModLoad: 00000000a7840000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000a7d80000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000a83f0000 00000000000ae000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000978f0000 0000000000058000 F:\BOINC_Data\slots\2\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000a8770000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 00000000a9ce0000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a82c0000 0000000000125000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000196d0000 000000000abe7000 F:\BOINC_Data\slots\2\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000093980000 0000000000091000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000967e0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 0000000093a60000 0000000000019000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 0000000093550000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.27.29112.0) (-exported- Symbols Loaded)
Linked PDB Filename : D:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.27.29112.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.27.29112.0
ModLoad: 00000000ef8e0000 0000000024b49000 F:\BOINC_Data\slots\2\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000005f3b0000 000000000020d000 F:\BOINC_Data\slots\2\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 000000005a220000 00000000002db000 F:\BOINC_Data\slots\2\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 000000008d890000 0000000000045000 F:\BOINC_Data\slots\2\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000036570000 00000000030a5000 F:\BOINC_Data\slots\2\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000330b0000 00000000034b2000 F:\BOINC_Data\slots\2\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000a36b0000 0000000000010000 F:\BOINC_Data\slots\2\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 0000000015970000 0000000003d5b000 F:\BOINC_Data\slots\2\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000cb390000 000000000919f000 F:\BOINC_Data\slots\2\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 00000000d4530000 000000001b3a4000 F:\BOINC_Data\slots\2\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 0000000090190000 0000000000040000 F:\BOINC_Data\slots\2\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000c94c0000 0000000001eca000 F:\BOINC_Data\slots\2\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000a9c00000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000a5730000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000a8b40000 0000000000472000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a80c0000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 00000000a8090000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a75e0000 0000000000034000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 00000000a8110000 0000000000069000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1503 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1503
ModLoad: 00000000a7e80000 0000000000156000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a7420000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000a94b0000 0000000000744000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.964 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.964
ModLoad: 00000000a5930000 0000000000794000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.1503) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a8fc0000 0000000000355000 C:\WINDOWS\System32\combase.dll (6.2.19041.1348) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 00000000a7290000 000000000002e000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a8210000 00000000000ad000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.1387) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000a9320000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1
ModLoad: 00000000509b0000 00000000002b9000 C:\WINDOWS\system32\nvcuda.dll (30.0.14.9729) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.14.9729
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.5.125 driver
Product Version : 30.0.14.9729
ModLoad: 00000000c7e40000 0000000001679000 C:\WINDOWS\system32\DriverStore\FileRepository\nvmii.inf_amd64_65c95878dc88aeb5\nvcuda64.dll (30.0.14.9729) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.14.9729
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.5.125 driver
Product Version : 30.0.14.9729
ModLoad: 00000000a1c90000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000a1c30000 0000000000031000 C:\WINDOWS\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 000000009f670000 000000000014a000 C:\WINDOWS\SYSTEM32\drvstore.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 00000000a71e0000 000000000000c000 C:\WINDOWS\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 000000009d900000 0000000000762000 C:\WINDOWS\SYSTEM32\nvapi64.dll (30.0.14.9729) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r495\r497_25\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.14.9729
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.14.9729
ModLoad: 00000000a7910000 0000000000082000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 3628, Write: 308, Other 775
- I/O Transfers Counters -
Read: 6703642, Write: 6051, Other 29160
- Paged Pool Usage -
QuotaPagedPoolUsage: 3464576, QuotaPeakPagedPoolUsage: 3464624
QuotaNonPagedPoolUsage: 37120, QuotaPeakNonPagedPoolUsage: 37120
- Virtual Memory Usage -
VirtualSize: -1502801920, PeakVirtualSize: -1003192320
- Pagefile Usage -
PagefileUsage: -1502801920, PeakPagefileUsage: -1502760960
- Working Set Size -
WorkingSetSize: 1139154944, PeakWorkingSetSize: 1139159040, PageFaultCount: 376641
*** Dump of thread ID 68224 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 10156250.000000, User Time: 30156250.000000, Wait Time: 165901120.000000
- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFCA7AE4F69
- Registers -
rax=000000001f268b93 rbx=000000002f644910 rcx=0000000082975c50 rdx=0000000000000001 rsi=0000000082976590 rdi=0000000019930520
r8=0000000082975ce8 r9=0000000000000000 r10=0000000000000000 r11=000000000dd84700 r12=000000001a4b3260 r13=00000000829767a0
r14=00000000340e1af0 r15=0000000000000000 rip=00000000a7ae4f69 rsp=0000000082976430 rbp=0000000082976720
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206
- Callstack -
ChildEBP RetAddr Args to Child
82976500 2f55a884 00000040 0000004f 1a4b2c00 0000076c KERNELBASE!RaiseException+0x0
82976560 2f301ca6 00000008 5f3b0000 2a79bcf8 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
82976610 2f30593b 00000000 00000030 00000020 00000008 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
82976760 2f301a78 00000000 00000000 82976a90 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
82976830 2f301cfc 82976a90 00000000 00000080 82976d60 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
829768f0 2f30c4dd 82976d60 82976d60 82976d60 82976d60 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
82976be0 2f2fb2fe 829771f8 82976d60 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8298f980 2f53c6f8 60e14d00 00000000 00000000 60dbea70 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8298f9c0 a9397034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
8298f9f0 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
8298fa70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 31856 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 165900448.000000
- Registers -
rax=00000000000001d3 rbx=000000005f3b5690 rcx=0000000000000070 rdx=000000005f3b5310 rsi=000000005f3b3490 rdi=000000005f3b5690
r8=00000000c966f000 r9=0000000082cff1c0 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000aa1d07c4 rsp=0000000082cff6f8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
82cff6f0 aa182dc7 00000000 00000000 5f3b3490 5f3b5690 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
82cff9f0 a9397034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
82cffa20 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
82cffaa0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 31424 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 165900448.000000
- Registers -
rax=00000000000001d3 rbx=000000005f3b5b10 rcx=0000000000000070 rdx=000000005f3b5790 rsi=000000005f3b3490 rdi=000000005f3b5b10
r8=0000000000000008 r9=0000000000000040 r10=00000000000005c0 r11=0000000000000100 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000aa1d07c4 rsp=0000000082dffbe8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
82dffbe0 aa182dc7 00000000 00000000 5f3b3490 5f3b5b10 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
82dffee0 a9397034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
82dfff10 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
82dfff90 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 15412 (state: Waiting): ***
- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 165900448.000000
- Registers -
rax=00000000000001d3 rbx=000000005f3b5f40 rcx=0000000000000070 rdx=000000005f3b5bc0 rsi=000000005f3b3490 rdi=000000005f3b5f40
r8=00000000a1c95008 r9=0000000082eff670 r10=00000000a1c900f0 r11=0000000000000001 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000aa1d07c4 rsp=0000000082effbc8 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
82effbc0 aa182dc7 00000000 00000000 5f3b3490 5f3b5f40 ntdll!ZwWaitForWorkViaWorkerFactory+0x0
82effec0 a9397034 00000000 00000000 00000000 00000000 ntdll!TpReleaseCleanupGroupMembers+0x0
82effef0 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
82efff70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 58432 (state: Waiting): ***
- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 165901120.000000
- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000830ff690 rsi=0000000000000000 rdi=0000000000000064
r8=00000000830ff668 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000aa1cd3f4 rsp=00000000830ff668 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
830ff660 a7af962e 830ff728 00000000 00000000 00000000 ntdll!NtDelayExecution+0x0
830ff700 2f31a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0
830ff730 a9397034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
830ff760 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
830ff7e0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 70920 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 165901120.000000
- Registers -
rax=000000000000005b rbx=000000000000000a rcx=000000000000000a rdx=00000000831ff4f0 rsi=0000000000000000 rdi=000000000000000a
r8=00000000831ff198 r9=0000000060f8afb0 r10=0000000000000000 r11=0000000000000246 r12=0000000000000064 r13=00000000831ff4f0
r14=00000000831ff1f0 r15=0000000000000000 rip=00000000aa1cd8c4 rsp=00000000831ff198 rbp=0000000060f8afb0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
831ff190 a7afcb20 00000000 82bef000 aa29a4d0 82bef000 ntdll!ZwWaitForMultipleObjects+0x0
831ff480 a7afca1e 5f3c2930 004e0049 0000000a 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
831ff4c0 c8015be4 60f8c0f0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
831ff700 c80bd15d 00000000 60f8afb0 00000000 00000000 nvcuda64!+0x0
831ff7b0 c80155b3 7375a4bb 705a8f73 6164a4d0 00000000 nvcuda64!cuProfilerStop+0x0
831ff7e0 c8344264 6164a4d0 00000000 00000000 00000000 nvcuda64!+0x0
831ff810 a9397034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
831ff840 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
831ff8c0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 2604 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 165900848.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002cc rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002cc
r8=00000000832ffbf8 r9=0000000000000000 r10=0000000000000000 r11=0000000000000246 r12=0000000060f5da30 r13=0000000000000000
r14=00000000000002cc r15=0000000000000000 rip=00000000aa1ccdf4 rsp=00000000832ff978 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
832ff970 a7ad1a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
832ffa10 c82395ee 6164aef0 00000000 00000000 000002cc KERNELBASE!WaitForSingleObjectEx+0x0
832ffc50 c80155b3 00000000 6164aef0 c7e40000 60fa6520 nvcuda64!cuProfilerStop+0x0
832ffc80 c8344264 6164aef0 00000000 00000000 00000000 nvcuda64!+0x0
832ffcb0 a9397034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
832ffce0 aa182651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
832ffd60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)