Task 10581671
| Name | ParityModified-1638636549-23339-2-0_0 |
| Workunit | 7913445 |
| Created | 8 Jan 2022, 23:44:53 UTC |
| Sent | 25 Jan 2022, 17:06:40 UTC |
| Report deadline | 2 Feb 2022, 17:06:40 UTC |
| Received | 30 Jan 2022, 13:13:09 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED |
| Computer ID | 11997 |
| Run time | 6 hours 32 min 9 sec |
| CPU time | 5 hours 50 min 6 sec |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 10,511.00 GFLOPS |
| Application version | Machine Learning Dataset Generator (GPU) v9.75 (cuda10200) windows_x86_64 |
| Peak working set size | 2.91 GB |
| Peak swap size | 4.71 GB |
| Peak disk usage | 1.54 GB |
Stderr output
<core_client_version>7.16.20</core_client_version>
<![CDATA[
<message>
exceeded elapsed time limit 23528.83 (4000000.00G/170.00G)</message>
<stderr_txt>
0:48:51 main:574] : INFO : Epoch 1714 | loss: 0.0311315 | val_loss: 0.0311704 | Time: 3513.64 ms
[2022-01-28 20:48:55 main:574] : INFO : Epoch 1715 | loss: 0.0311282 | val_loss: 0.031177 | Time: 3546.68 ms
[2022-01-28 20:48:58 main:574] : INFO : Epoch 1716 | loss: 0.0311279 | val_loss: 0.0311764 | Time: 3275.98 ms
[2022-01-28 20:49:02 main:574] : INFO : Epoch 1717 | loss: 0.0311288 | val_loss: 0.0311732 | Time: 3502.48 ms
[2022-01-28 20:49:05 main:574] : INFO : Epoch 1718 | loss: 0.0311284 | val_loss: 0.0311709 | Time: 3635.71 ms
[2022-01-28 20:49:09 main:574] : INFO : Epoch 1719 | loss: 0.0311287 | val_loss: 0.0311771 | Time: 3434.04 ms
[2022-01-28 20:49:12 main:574] : INFO : Epoch 1720 | loss: 0.0311303 | val_loss: 0.0311806 | Time: 3358.87 ms
[2022-01-28 20:49:16 main:574] : INFO : Epoch 1721 | loss: 0.031131 | val_loss: 0.0311759 | Time: 3472.98 ms
[2022-01-28 20:49:19 main:574] : INFO : Epoch 1722 | loss: 0.0311328 | val_loss: 0.0311689 | Time: 3589.55 ms
[2022-01-28 20:49:23 main:574] : INFO : Epoch 1723 | loss: 0.0311312 | val_loss: 0.0311744 | Time: 3595.38 ms
[2022-01-28 20:49:26 main:574] : INFO : Epoch 1724 | loss: 0.0311307 | val_loss: 0.0311752 | Time: 3465.93 ms
[2022-01-28 20:49:30 main:574] : INFO : Epoch 1725 | loss: 0.0311282 | val_loss: 0.0311818 | Time: 3552.82 ms
[2022-01-28 20:49:33 main:574] : INFO : Epoch 1726 | loss: 0.0311311 | val_loss: 0.0311773 | Time: 3269.95 ms
[2022-01-28 20:49:36 main:574] : INFO : Epoch 1727 | loss: 0.0311328 | val_loss: 0.0311755 | Time: 3466.03 ms
[2022-01-28 20:49:40 main:574] : INFO : Epoch 1728 | loss: 0.0311286 | val_loss: 0.0311713 | Time: 3389.08 ms
[2022-01-28 20:49:43 main:574] : INFO : Epoch 1729 | loss: 0.0311284 | val_loss: 0.031172 | Time: 3272.07 ms
[2022-01-28 20:49:47 main:574] : INFO : Epoch 1730 | loss: 0.0311274 | val_loss: 0.0311749 | Time: 3490.4 ms
[2022-01-28 20:49:50 main:574] : INFO : Epoch 1731 | loss: 0.031129 | val_loss: 0.0311707 | Time: 3458.16 ms
[2022-01-28 20:49:54 main:574] : INFO : Epoch 1732 | loss: 0.0311298 | val_loss: 0.0311759 | Time: 3572.93 ms
[2022-01-28 20:49:57 main:574] : INFO : Epoch 1733 | loss: 0.0311344 | val_loss: 0.0311824 | Time: 3407.96 ms
[2022-01-28 20:50:01 main:574] : INFO : Epoch 1734 | loss: 0.0311373 | val_loss: 0.0311812 | Time: 3590.37 ms
[2022-01-28 20:50:04 main:574] : INFO : Epoch 1735 | loss: 0.0311396 | val_loss: 0.0311807 | Time: 3537.66 ms
[2022-01-28 20:50:08 main:574] : INFO : Epoch 1736 | loss: 0.0311392 | val_loss: 0.0311707 | Time: 3391.48 ms
[2022-01-28 20:50:11 main:574] : INFO : Epoch 1737 | loss: 0.0311385 | val_loss: 0.0311717 | Time: 3348.2 ms
[2022-01-28 20:50:14 main:574] : INFO : Epoch 1738 | loss: 0.0311339 | val_loss: 0.0311802 | Time: 3414.82 ms
[2022-01-28 20:50:18 main:574] : INFO : Epoch 1739 | loss: 0.0311318 | val_loss: 0.0311714 | Time: 3386.57 ms
[2022-01-28 20:50:21 main:574] : INFO : Epoch 1740 | loss: 0.0311331 | val_loss: 0.0311741 | Time: 3313.73 ms
[2022-01-28 20:50:25 main:574] : INFO : Epoch 1741 | loss: 0.0311331 | val_loss: 0.031177 | Time: 3587.42 ms
[2022-01-28 20:50:28 main:574] : INFO : Epoch 1742 | loss: 0.0311337 | val_loss: 0.0311779 | Time: 3530.16 ms
[2022-01-28 20:50:31 main:574] : INFO : Epoch 1743 | loss: 0.0311344 | val_loss: 0.0311796 | Time: 3245.51 ms
[2022-01-28 20:50:35 main:574] : INFO : Epoch 1744 | loss: 0.0311336 | val_loss: 0.0311804 | Time: 3566.4 ms
[2022-01-28 20:50:38 main:574] : INFO : Epoch 1745 | loss: 0.0311399 | val_loss: 0.0311691 | Time: 3367.18 ms
[2022-01-28 20:50:42 main:574] : INFO : Epoch 1746 | loss: 0.0311408 | val_loss: 0.0311698 | Time: 3619.47 ms
[2022-01-28 20:50:45 main:574] : INFO : Epoch 1747 | loss: 0.0311378 | val_loss: 0.0311716 | Time: 3376.19 ms
[2022-01-28 20:50:49 main:574] : INFO : Epoch 1748 | loss: 0.0311358 | val_loss: 0.0311779 | Time: 3376.3 ms
[2022-01-28 20:50:52 main:574] : INFO : Epoch 1749 | loss: 0.031133 | val_loss: 0.0311734 | Time: 3628.7 ms
[2022-01-28 20:50:56 main:574] : INFO : Epoch 1750 | loss: 0.0311319 | val_loss: 0.031175 | Time: 3382.63 ms
[2022-01-28 20:50:59 main:574] : INFO : Epoch 1751 | loss: 0.0311316 | val_loss: 0.0311706 | Time: 3420.47 ms
[2022-01-28 20:51:03 main:574] : INFO : Epoch 1752 | loss: 0.0311303 | val_loss: 0.0311741 | Time: 3539.66 ms
[2022-01-28 20:51:06 main:574] : INFO : Epoch 1753 | loss: 0.0311297 | val_loss: 0.0311785 | Time: 3453.12 ms
[2022-01-28 20:51:10 main:574] : INFO : Epoch 1754 | loss: 0.0311317 | val_loss: 0.0311808 | Time: 3402.65 ms
[2022-01-28 20:51:13 main:574] : INFO : Epoch 1755 | loss: 0.031129 | val_loss: 0.0311729 | Time: 3315.48 ms
[2022-01-28 20:51:16 main:574] : INFO : Epoch 1756 | loss: 0.0311298 | val_loss: 0.031178 | Time: 3436.6 ms
[2022-01-28 20:51:20 main:574] : INFO : Epoch 1757 | loss: 0.0311288 | val_loss: 0.0311765 | Time: 3410.12 ms
[2022-01-28 20:51:23 main:574] : INFO : Epoch 1758 | loss: 0.0311289 | val_loss: 0.0311739 | Time: 3342.71 ms
[2022-01-28 20:51:27 main:574] : INFO : Epoch 1759 | loss: 0.0311287 | val_loss: 0.0311703 | Time: 3523.76 ms
[2022-01-28 20:51:30 main:574] : INFO : Epoch 1760 | loss: 0.0311286 | val_loss: 0.0311747 | Time: 3425.54 ms
[2022-01-28 20:51:34 main:574] : INFO : Epoch 1761 | loss: 0.0311278 | val_loss: 0.0311744 | Time: 3533.61 ms
[2022-01-28 20:51:37 main:574] : INFO : Epoch 1762 | loss: 0.0311314 | val_loss: 0.0311737 | Time: 3722.43 ms
[2022-01-28 20:51:41 main:574] : INFO : Epoch 1763 | loss: 0.0311348 | val_loss: 0.0311779 | Time: 3848.55 ms
[2022-01-28 20:51:45 main:574] : INFO : Epoch 1764 | loss: 0.0311333 | val_loss: 0.0311765 | Time: 3412.13 ms
[2022-01-28 20:51:48 main:574] : INFO : Epoch 1765 | loss: 0.0311281 | val_loss: 0.0311742 | Time: 3377.39 ms
[2022-01-28 20:51:52 main:574] : INFO : Epoch 1766 | loss: 0.0311268 | val_loss: 0.0311753 | Time: 3448.13 ms
[2022-01-28 20:51:55 main:574] : INFO : Epoch 1767 | loss: 0.0311271 | val_loss: 0.0311731 | Time: 3146.05 ms
[2022-01-28 20:51:58 main:574] : INFO : Epoch 1768 | loss: 0.0311261 | val_loss: 0.0311736 | Time: 3343.37 ms
[2022-01-28 20:52:01 main:574] : INFO : Epoch 1769 | loss: 0.0311263 | val_loss: 0.0311776 | Time: 3454.43 ms
[2022-01-28 20:52:05 main:574] : INFO : Epoch 1770 | loss: 0.031125 | val_loss: 0.031173 | Time: 3579.37 ms
[2022-01-28 20:52:08 main:574] : INFO : Epoch 1771 | loss: 0.0311237 | val_loss: 0.0311768 | Time: 3339.49 ms
[2022-01-28 20:52:12 main:574] : INFO : Epoch 1772 | loss: 0.0311247 | val_loss: 0.0311726 | Time: 3479.68 ms
[2022-01-28 20:52:15 main:574] : INFO : Epoch 1773 | loss: 0.0311247 | val_loss: 0.0311745 | Time: 3405.45 ms
[2022-01-28 20:52:19 main:574] : INFO : Epoch 1774 | loss: 0.0311257 | val_loss: 0.0311752 | Time: 3367.72 ms
[2022-01-28 20:52:22 main:574] : INFO : Epoch 1775 | loss: 0.0311231 | val_loss: 0.0311785 | Time: 3192.44 ms
[2022-01-28 20:52:25 main:574] : INFO : Epoch 1776 | loss: 0.0311237 | val_loss: 0.0311782 | Time: 3438.52 ms
[2022-01-28 20:52:29 main:574] : INFO : Epoch 1777 | loss: 0.0311253 | val_loss: 0.0311738 | Time: 3321.24 ms
[2022-01-28 20:52:32 main:574] : INFO : Epoch 1778 | loss: 0.0311299 | val_loss: 0.0311724 | Time: 3530.92 ms
[2022-01-28 20:52:35 main:574] : INFO : Epoch 1779 | loss: 0.0311316 | val_loss: 0.0311825 | Time: 3205.57 ms
[2022-01-28 20:52:39 main:574] : INFO : Epoch 1780 | loss: 0.0311312 | val_loss: 0.031168 | Time: 3413.18 ms
[2022-01-28 20:52:42 main:574] : INFO : Epoch 1781 | loss: 0.031129 | val_loss: 0.0311776 | Time: 3444.07 ms
[2022-01-28 20:52:46 main:574] : INFO : Epoch 1782 | loss: 0.0311279 | val_loss: 0.0311777 | Time: 3654.66 ms
[2022-01-28 20:52:49 main:574] : INFO : Epoch 1783 | loss: 0.0311299 | val_loss: 0.0311747 | Time: 3431.02 ms
[2022-01-28 20:52:53 main:574] : INFO : Epoch 1784 | loss: 0.0311296 | val_loss: 0.0311763 | Time: 3451.08 ms
[2022-01-28 20:52:57 main:574] : INFO : Epoch 1785 | loss: 0.0311267 | val_loss: 0.0311833 | Time: 3723.82 ms
[2022-01-28 20:53:00 main:574] : INFO : Epoch 1786 | loss: 0.0311267 | val_loss: 0.0311723 | Time: 3441.08 ms
[2022-01-28 20:53:03 main:574] : INFO : Epoch 1787 | loss: 0.031142 | val_loss: 0.0311971 | Time: 3432.47 ms
[2022-01-28 20:53:07 main:574] : INFO : Epoch 1788 | loss: 0.03114 | val_loss: 0.0311747 | Time: 3322.22 ms
[2022-01-28 20:53:11 main:574] : INFO : Epoch 1789 | loss: 0.0311371 | val_loss: 0.0311786 | Time: 3960.02 ms
[2022-01-28 20:53:14 main:574] : INFO : Epoch 1790 | loss: 0.031128 | val_loss: 0.0311773 | Time: 3598.45 ms
[2022-01-28 20:53:18 main:574] : INFO : Epoch 1791 | loss: 0.0311306 | val_loss: 0.0311929 | Time: 3677.82 ms
[2022-01-28 20:53:22 main:574] : INFO : Epoch 1792 | loss: 0.0311312 | val_loss: 0.0311795 | Time: 3587.6 ms
[2022-01-28 20:53:25 main:574] : INFO : Epoch 1793 | loss: 0.0311311 | val_loss: 0.0311732 | Time: 3667.18 ms
[2022-01-28 20:53:29 main:574] : INFO : Epoch 1794 | loss: 0.0311312 | val_loss: 0.0311757 | Time: 3950.5 ms
[2022-01-28 20:53:33 main:574] : INFO : Epoch 1795 | loss: 0.0311307 | val_loss: 0.0311749 | Time: 3824.42 ms
[2022-01-28 20:53:37 main:574] : INFO : Epoch 1796 | loss: 0.0311317 | val_loss: 0.0311755 | Time: 3644.07 ms
[2022-01-28 20:53:40 main:574] : INFO : Epoch 1797 | loss: 0.0311303 | val_loss: 0.0311716 | Time: 3552.33 ms
[2022-01-28 20:53:44 main:574] : INFO : Epoch 1798 | loss: 0.0311339 | val_loss: 0.0311706 | Time: 3676.14 ms
[2022-01-28 20:53:48 main:574] : INFO : Epoch 1799 | loss: 0.0311348 | val_loss: 0.0311788 | Time: 3828.21 ms
[2022-01-28 20:53:51 main:574] : INFO : Epoch 1800 | loss: 0.03114 | val_loss: 0.0311798 | Time: 3596.32 ms
[2022-01-28 20:53:55 main:574] : INFO : Epoch 1801 | loss: 0.0311517 | val_loss: 0.0311704 | Time: 3493.43 ms
[2022-01-28 20:53:58 main:574] : INFO : Epoch 1802 | loss: 0.0311408 | val_loss: 0.0311663 | Time: 3322.34 ms
[2022-01-28 20:54:02 main:574] : INFO : Epoch 1803 | loss: 0.0311343 | val_loss: 0.0311705 | Time: 3566.82 ms
[2022-01-28 20:54:05 main:574] : INFO : Epoch 1804 | loss: 0.0311306 | val_loss: 0.0311742 | Time: 3495.09 ms
[2022-01-28 20:54:09 main:574] : INFO : Epoch 1805 | loss: 0.031134 | val_loss: 0.031166 | Time: 3766.6 ms
[2022-01-28 20:54:12 main:574] : INFO : Epoch 1806 | loss: 0.0311352 | val_loss: 0.0311671 | Time: 3349.66 ms
[2022-01-28 20:54:16 main:574] : INFO : Epoch 1807 | loss: 0.0311438 | val_loss: 0.0311722 | Time: 3636.6 ms
[2022-01-28 20:54:20 main:574] : INFO : Epoch 1808 | loss: 0.031143 | val_loss: 0.0311756 | Time: 3758.19 ms
[2022-01-28 20:54:24 main:574] : INFO : Epoch 1809 | loss: 0.0311417 | val_loss: 0.0311721 | Time: 3897.4 ms
[2022-01-28 20:54:27 main:574] : INFO : Epoch 1810 | loss: 0.031144 | val_loss: 0.0311684 | Time: 3532.45 ms
[2022-01-28 20:54:31 main:574] : INFO : Epoch 1811 | loss: 0.0311411 | val_loss: 0.0311658 | Time: 3629.88 ms
[2022-01-28 20:54:35 main:574] : INFO : Epoch 1812 | loss: 0.0311402 | val_loss: 0.0311698 | Time: 3698.33 ms
[2022-01-28 20:54:38 main:574] : INFO : Epoch 1813 | loss: 0.0311402 | val_loss: 0.0311639 | Time: 3514.35 ms
[2022-01-28 20:54:42 main:574] : INFO : Epoch 1814 | loss: 0.0311394 | val_loss: 0.0311698 | Time: 3651.27 ms
[2022-01-28 20:54:45 main:574] : INFO : Epoch 1815 | loss: 0.0311373 | val_loss: 0.031167 | Time: 3549.02 ms
[2022-01-28 20:54:49 main:574] : INFO : Epoch 1816 | loss: 0.0311346 | val_loss: 0.0311671 | Time: 3467.74 ms
[2022-01-28 20:54:52 main:574] : INFO : Epoch 1817 | loss: 0.0311365 | val_loss: 0.0311651 | Time: 3601.98 ms
[2022-01-28 20:54:56 main:574] : INFO : Epoch 1818 | loss: 0.0311406 | val_loss: 0.0311669 | Time: 3679.23 ms
[2022-01-28 20:55:00 main:574] : INFO : Epoch 1819 | loss: 0.0311411 | val_loss: 0.0311657 | Time: 3488.78 ms
[2022-01-28 20:55:03 main:574] : INFO : Epoch 1820 | loss: 0.0311427 | val_loss: 0.0311729 | Time: 3624.58 ms
[2022-01-28 20:55:07 main:574] : INFO : Epoch 1821 | loss: 0.0311418 | val_loss: 0.0311693 | Time: 3748.67 ms
[2022-01-28 20:55:11 main:574] : INFO : Epoch 1822 | loss: 0.0311391 | val_loss: 0.0311697 | Time: 3592.69 ms
[2022-01-28 20:55:14 main:574] : INFO : Epoch 1823 | loss: 0.0311355 | val_loss: 0.0311694 | Time: 3726.23 ms
[2022-01-28 20:55:18 main:574] : INFO : Epoch 1824 | loss: 0.0311372 | val_loss: 0.0311679 | Time: 3748.22 ms
[2022-01-28 20:55:22 main:574] : INFO : Epoch 1825 | loss: 0.0311384 | val_loss: 0.0311694 | Time: 3788.02 ms
[2022-01-28 20:55:25 main:574] : INFO : Epoch 1826 | loss: 0.031138 | val_loss: 0.0311692 | Time: 3653.25 ms
[2022-01-28 20:55:29 main:574] : INFO : Epoch 1827 | loss: 0.0311364 | val_loss: 0.0311678 | Time: 3631.28 ms
[2022-01-28 20:55:33 main:574] : INFO : Epoch 1828 | loss: 0.0311395 | val_loss: 0.031167 | Time: 3924.6 ms
[2022-01-28 20:55:37 main:574] : INFO : Epoch 1829 | loss: 0.0311371 | val_loss: 0.0311704 | Time: 3778.07 ms
[2022-01-28 20:55:41 main:574] : INFO : Epoch 1830 | loss: 0.0311364 | val_loss: 0.0311676 | Time: 4060.36 ms
[2022-01-28 20:55:45 main:574] : INFO : Epoch 1831 | loss: 0.0311385 | val_loss: 0.0311683 | Time: 3569.68 ms
[2022-01-28 20:55:48 main:574] : INFO : Epoch 1832 | loss: 0.0311426 | val_loss: 0.0311649 | Time: 3710.39 ms
[2022-01-28 20:55:52 main:574] : INFO : Epoch 1833 | loss: 0.0311427 | val_loss: 0.0311675 | Time: 3801.62 ms
[2022-01-28 20:55:56 main:574] : INFO : Epoch 1834 | loss: 0.0311392 | val_loss: 0.0311666 | Time: 3711.32 ms
[2022-01-28 20:56:00 main:574] : INFO : Epoch 1835 | loss: 0.0311381 | val_loss: 0.0311694 | Time: 3861.67 ms
[2022-01-28 20:56:03 main:574] : INFO : Epoch 1836 | loss: 0.0311375 | val_loss: 0.0311667 | Time: 3639.84 ms
[2022-01-28 20:56:07 main:574] : INFO : Epoch 1837 | loss: 0.0311338 | val_loss: 0.0311672 | Time: 3520.99 ms
[2022-01-28 20:56:10 main:574] : INFO : Epoch 1838 | loss: 0.0311348 | val_loss: 0.0311663 | Time: 3553.26 ms
[2022-01-28 20:56:14 main:574] : INFO : Epoch 1839 | loss: 0.031133 | val_loss: 0.031173 | Time: 3698.62 ms
[2022-01-28 20:56:18 main:574] : INFO : Epoch 1840 | loss: 0.0311305 | val_loss: 0.0311701 | Time: 3545.3 ms
[2022-01-28 20:56:22 main:574] : INFO : Epoch 1841 | loss: 0.0311322 | val_loss: 0.0311794 | Time: 3905.59 ms
[2022-01-28 20:56:25 main:574] : INFO : Epoch 1842 | loss: 0.0311361 | val_loss: 0.0311758 | Time: 3741.88 ms
[2022-01-28 20:56:29 main:574] : INFO : Epoch 1843 | loss: 0.0311386 | val_loss: 0.0311836 | Time: 3621.94 ms
[2022-01-28 20:56:33 main:574] : INFO : Epoch 1844 | loss: 0.0311386 | val_loss: 0.0311683 | Time: 3615.97 ms
[2022-01-28 20:56:36 main:574] : INFO : Epoch 1845 | loss: 0.0311357 | val_loss: 0.0311669 | Time: 3758.6 ms
[2022-01-28 20:56:40 main:574] : INFO : Epoch 1846 | loss: 0.0311322 | val_loss: 0.0311678 | Time: 3491.93 ms
[2022-01-28 20:56:43 main:574] : INFO : Epoch 1847 | loss: 0.031132 | val_loss: 0.0311726 | Time: 3301.94 ms
[2022-01-28 20:56:47 main:574] : INFO : Epoch 1848 | loss: 0.0311314 | val_loss: 0.0311669 | Time: 3482.33 ms
[2022-01-28 20:56:51 main:574] : INFO : Epoch 1849 | loss: 0.0311299 | val_loss: 0.0311756 | Time: 4041.42 ms
[2022-01-28 20:56:54 main:574] : INFO : Epoch 1850 | loss: 0.0311338 | val_loss: 0.031164 | Time: 3668.6 ms
[2022-01-28 20:56:58 main:574] : INFO : Epoch 1851 | loss: 0.0311343 | val_loss: 0.0311656 | Time: 3740.57 ms
[2022-01-28 20:57:02 main:574] : INFO : Epoch 1852 | loss: 0.0311315 | val_loss: 0.0311648 | Time: 3699.28 ms
[2022-01-28 20:57:06 main:574] : INFO : Epoch 1853 | loss: 0.031132 | val_loss: 0.0311638 | Time: 3728.74 ms
[2022-01-28 20:57:09 main:574] : INFO : Epoch 1854 | loss: 0.0311295 | val_loss: 0.0311652 | Time: 3911.24 ms
[2022-01-28 20:57:13 main:574] : INFO : Epoch 1855 | loss: 0.0311279 | val_loss: 0.0311696 | Time: 3624.19 ms
[2022-01-28 20:57:17 main:574] : INFO : Epoch 1856 | loss: 0.0311295 | val_loss: 0.0311625 | Time: 4288.68 ms
[2022-01-28 20:57:21 main:574] : INFO : Epoch 1857 | loss: 0.0311282 | val_loss: 0.0311754 | Time: 3925.91 ms
[2022-01-28 20:57:25 main:574] : INFO : Epoch 1858 | loss: 0.0311283 | val_loss: 0.031167 | Time: 3679.99 ms
[2022-01-28 20:57:29 main:574] : INFO : Epoch 1859 | loss: 0.0311265 | val_loss: 0.0311637 | Time: 3907.86 ms
[2022-01-28 20:57:33 main:574] : INFO : Epoch 1860 | loss: 0.0311254 | val_loss: 0.0311739 | Time: 3965.67 ms
[2022-01-28 20:57:37 main:574] : INFO : Epoch 1861 | loss: 0.0311288 | val_loss: 0.0311737 | Time: 3964.44 ms
[2022-01-28 20:57:41 main:574] : INFO : Epoch 1862 | loss: 0.0311277 | val_loss: 0.0311818 | Time: 3916.41 ms
[2022-01-28 20:57:45 main:574] : INFO : Epoch 1863 | loss: 0.0311299 | val_loss: 0.0311712 | Time: 3872.77 ms
[2022-01-28 20:57:49 main:574] : INFO : Epoch 1864 | loss: 0.0311296 | val_loss: 0.0311743 | Time: 4152.87 ms
[2022-01-28 20:57:53 main:574] : INFO : Epoch 1865 | loss: 0.031127 | val_loss: 0.0311706 | Time: 3996.4 ms
[2022-01-28 20:57:57 main:574] : INFO : Epoch 1866 | loss: 0.0311233 | val_loss: 0.0311723 | Time: 4109.97 ms
[2022-01-28 20:58:01 main:574] : INFO : Epoch 1867 | loss: 0.0311254 | val_loss: 0.031171 | Time: 3956.76 ms
[2022-01-28 20:58:05 main:574] : INFO : Epoch 1868 | loss: 0.0311414 | val_loss: 0.0311692 | Time: 3707.71 ms
[2022-01-28 20:58:09 main:574] : INFO : Epoch 1869 | loss: 0.0311387 | val_loss: 0.0311729 | Time: 3849.39 ms
[2022-01-28 20:58:12 main:574] : INFO : Epoch 1870 | loss: 0.0311353 | val_loss: 0.0311721 | Time: 3647.8 ms
[2022-01-28 20:58:26 main:574] : INFO : Epoch 1871 | loss: 0.0311334 | val_loss: 0.0311742 | Time: 14215.4 ms
[2022-01-28 20:58:30 main:574] : INFO : Epoch 1872 | loss: 0.0311315 | val_loss: 0.0311749 | Time: 3648.59 ms
[2022-01-28 20:58:34 main:574] : INFO : Epoch 1873 | loss: 0.0311343 | val_loss: 0.0311688 | Time: 3628.12 ms
[2022-01-28 20:58:37 main:574] : INFO : Epoch 1874 | loss: 0.0311336 | val_loss: 0.0311751 | Time: 3254.95 ms
[2022-01-28 20:58:40 main:574] : INFO : Epoch 1875 | loss: 0.0311321 | val_loss: 0.0311774 | Time: 3273.64 ms
[2022-01-28 20:58:44 main:574] : INFO : Epoch 1876 | loss: 0.0311292 | val_loss: 0.0311729 | Time: 3382.9 ms
[2022-01-28 20:58:47 main:574] : INFO : Epoch 1877 | loss: 0.0311283 | val_loss: 0.0311768 | Time: 3313.34 ms
[2022-01-28 20:58:50 main:574] : INFO : Epoch 1878 | loss: 0.0311278 | val_loss: 0.0311737 | Time: 3344.91 ms
[2022-01-28 20:58:54 main:574] : INFO : Epoch 1879 | loss: 0.0311278 | val_loss: 0.0311699 | Time: 3351.04 ms
[2022-01-28 20:58:57 main:574] : INFO : Epoch 1880 | loss: 0.0311251 | val_loss: 0.031178 | Time: 3458.58 ms
[2022-01-28 20:59:00 main:574] : INFO : Epoch 1881 | loss: 0.0311245 | val_loss: 0.031177 | Time: 3217.27 ms
[2022-01-28 20:59:04 main:574] : INFO : Epoch 1882 | loss: 0.0311257 | val_loss: 0.0311736 | Time: 3503.47 ms
[2022-01-28 20:59:07 main:574] : INFO : Epoch 1883 | loss: 0.031125 | val_loss: 0.0311736 | Time: 3338.25 ms
[2022-01-28 20:59:11 main:574] : INFO : Epoch 1884 | loss: 0.031126 | val_loss: 0.031172 | Time: 3491.49 ms
[2022-01-28 20:59:14 main:574] : INFO : Epoch 1885 | loss: 0.0311279 | val_loss: 0.0311736 | Time: 3305.38 ms
[2022-01-28 20:59:17 main:574] : INFO : Epoch 1886 | loss: 0.0311402 | val_loss: 0.0311744 | Time: 3457.53 ms
[2022-01-28 20:59:21 main:574] : INFO : Epoch 1887 | loss: 0.0311416 | val_loss: 0.0311706 | Time: 3390.56 ms
[2022-01-28 20:59:24 main:574] : INFO : Epoch 1888 | loss: 0.0311361 | val_loss: 0.031171 | Time: 3568.42 ms
[2022-01-28 20:59:28 main:574] : INFO : Epoch 1889 | loss: 0.0311317 | val_loss: 0.0311786 | Time: 3225.41 ms
[2022-01-28 20:59:31 main:574] : INFO : Epoch 1890 | loss: 0.0311331 | val_loss: 0.0311743 | Time: 3416.48 ms
[2022-01-28 20:59:34 main:574] : INFO : Epoch 1891 | loss: 0.0311321 | val_loss: 0.0311685 | Time: 3289.86 ms
[2022-01-28 20:59:38 main:574] : INFO : Epoch 1892 | loss: 0.0311283 | val_loss: 0.0311696 | Time: 3443.28 ms
[2022-01-28 20:59:41 main:574] : INFO : Epoch 1893 | loss: 0.0311278 | val_loss: 0.0311751 | Time: 3544.91 ms
[2022-01-28 20:59:45 main:574] : INFO : Epoch 1894 | loss: 0.0311263 | val_loss: 0.0311872 | Time: 3223.74 ms
[2022-01-28 20:59:48 main:574] : INFO : Epoch 1895 | loss: 0.0311406 | val_loss: 0.0311738 | Time: 3272.91 ms
[2022-01-28 20:59:51 main:574] : INFO : Epoch 1896 | loss: 0.031138 | val_loss: 0.0311744 | Time: 3362.97 ms
[2022-01-28 20:59:55 main:574] : INFO : Epoch 1897 | loss: 0.0311328 | val_loss: 0.0311728 | Time: 3438.54 ms
[2022-01-28 20:59:58 main:574] : INFO : Epoch 1898 | loss: 0.0311297 | val_loss: 0.0311757 | Time: 3332.61 ms
[2022-01-28 21:00:01 main:574] : INFO : Epoch 1899 | loss: 0.0311292 | val_loss: 0.0311719 | Time: 3445.28 ms
[2022-01-28 21:00:05 main:574] : INFO : Epoch 1900 | loss: 0.0311267 | val_loss: 0.0311795 | Time: 3287.59 ms
[2022-01-28 21:00:08 main:574] : INFO : Epoch 1901 | loss: 0.0311275 | val_loss: 0.0311715 | Time: 3601.26 ms
[2022-01-28 21:00:12 main:574] : INFO : Epoch 1902 | loss: 0.0311338 | val_loss: 0.0311742 | Time: 3421.66 ms
[2022-01-28 21:00:15 main:574] : INFO : Epoch 1903 | loss: 0.031127 | val_loss: 0.0311764 | Time: 3310.79 ms
[2022-01-28 21:00:18 main:574] : INFO : Epoch 1904 | loss: 0.0311286 | val_loss: 0.0311807 | Time: 3292.07 ms
[2022-01-28 21:00:22 main:574] : INFO : Epoch 1905 | loss: 0.0311254 | val_loss: 0.0311787 | Time: 3319.19 ms
[2022-01-28 21:00:25 main:574] : INFO : Epoch 1906 | loss: 0.0311287 | val_loss: 0.031174 | Time: 3624.39 ms
[2022-01-28 21:00:29 main:574] : INFO : Epoch 1907 | loss: 0.0311261 | val_loss: 0.0311731 | Time: 3245.41 ms
[2022-01-28 21:00:32 main:574] : INFO : Epoch 1908 | loss: 0.0311248 | val_loss: 0.0311775 | Time: 3420.45 ms
[2022-01-28 21:00:35 main:574] : INFO : Epoch 1909 | loss: 0.0311237 | val_loss: 0.0311719 | Time: 3300.12 ms
[2022-01-28 21:00:39 main:574] : INFO : Epoch 1910 | loss: 0.0311232 | val_loss: 0.0311711 | Time: 3324.53 ms
[2022-01-28 21:00:42 main:574] : INFO : Epoch 1911 | loss: 0.0311255 | val_loss: 0.0311777 | Time: 3254.06 ms
[2022-01-28 21:00:45 main:574] : INFO : Epoch 1912 | loss: 0.0311216 | val_loss: 0.0311797 | Time: 3377.81 ms
[2022-01-28 21:00:49 main:574] : INFO : Epoch 1913 | loss: 0.0311233 | val_loss: 0.0311792 | Time: 3347.7 ms
[2022-01-28 21:00:52 main:574] : INFO : Epoch 1914 | loss: 0.0311243 | val_loss: 0.0311731 | Time: 3538.53 ms
[2022-01-28 21:00:56 main:574] : INFO : Epoch 1915 | loss: 0.0311272 | val_loss: 0.0311781 | Time: 3440.2 ms
[2022-01-28 21:00:59 main:574] : INFO : Epoch 1916 | loss: 0.0311304 | val_loss: 0.0311701 | Time: 3223.22 ms
[2022-01-28 21:01:02 main:574] : INFO : Epoch 1917 | loss: 0.0311285 | val_loss: 0.0311785 | Time: 3359.46 ms
[2022-01-28 21:01:06 main:574] : INFO : Epoch 1918 | loss: 0.0311249 | val_loss: 0.0311731 | Time: 3298.38 ms
[2022-01-28 21:01:09 main:574] : INFO : Epoch 1919 | loss: 0.0311264 | val_loss: 0.0311785 | Time: 3518.26 ms
[2022-01-28 21:01:12 main:574] : INFO : Epoch 1920 | loss: 0.0311273 | val_loss: 0.0311723 | Time: 3246.31 ms
[2022-01-28 21:01:16 main:574] : INFO : Epoch 1921 | loss: 0.0311247 | val_loss: 0.0311728 | Time: 3451.55 ms
[2022-01-28 21:01:19 main:574] : INFO : Epoch 1922 | loss: 0.0311271 | val_loss: 0.0311748 | Time: 3411.94 ms
[2022-01-28 21:01:23 main:574] : INFO : Epoch 1923 | loss: 0.0311273 | val_loss: 0.0311767 | Time: 3434.98 ms
[2022-01-28 21:01:26 main:574] : INFO : Epoch 1924 | loss: 0.0311278 | val_loss: 0.0311825 | Time: 3329.47 ms
[2022-01-28 21:01:30 main:574] : INFO : Epoch 1925 | loss: 0.0311289 | val_loss: 0.0311763 | Time: 3373.14 ms
[2022-01-28 21:01:33 main:574] : INFO : Epoch 1926 | loss: 0.0311305 | val_loss: 0.031176 | Time: 3462.91 ms
[2022-01-28 21:01:36 main:574] : INFO : Epoch 1927 | loss: 0.0311329 | val_loss: 0.0311788 | Time: 3483.96 ms
[2022-01-28 21:01:40 main:574] : INFO : Epoch 1928 | loss: 0.0311336 | val_loss: 0.0311692 | Time: 3300.53 ms
[2022-01-28 21:01:43 main:574] : INFO : Epoch 1929 | loss: 0.0311294 | val_loss: 0.0311706 | Time: 3261.99 ms
[2022-01-28 21:01:47 main:574] : INFO : Epoch 1930 | loss: 0.0311281 | val_loss: 0.0311722 | Time: 3361.26 ms
[2022-01-28 21:01:50 main:574] : INFO : Epoch 1931 | loss: 0.031132 | val_loss: 0.0311775 | Time: 3232.83 ms
[2022-01-28 21:01:53 main:574] : INFO : Epoch 1932 | loss: 0.0311363 | val_loss: 0.031173 | Time: 3368.35 ms
[2022-01-28 21:01:56 main:574] : INFO : Epoch 1933 | loss: 0.0311328 | val_loss: 0.0311754 | Time: 3201.34 ms
[2022-01-28 21:02:00 main:574] : INFO : Epoch 1934 | loss: 0.0311362 | val_loss: 0.0311885 | Time: 3368.83 ms
[2022-01-28 21:02:03 main:574] : INFO : Epoch 1935 | loss: 0.0311356 | val_loss: 0.0311709 | Time: 3366.94 ms
[2022-01-28 21:02:07 main:574] : INFO : Epoch 1936 | loss: 0.031133 | val_loss: 0.0311854 | Time: 3516.55 ms
[2022-01-28 21:02:10 main:574] : INFO : Epoch 1937 | loss: 0.0311377 | val_loss: 0.0311731 | Time: 3292.98 ms
[2022-01-28 21:02:14 main:574] : INFO : Epoch 1938 | loss: 0.0311358 | val_loss: 0.031185 | Time: 3487.52 ms
[2022-01-28 21:02:17 main:574] : INFO : Epoch 1939 | loss: 0.031137 | val_loss: 0.0311797 | Time: 3429.36 ms
[2022-01-28 21:02:20 main:574] : INFO : Epoch 1940 | loss: 0.0311306 | val_loss: 0.0311721 | Time: 3425.14 ms
[2022-01-28 21:02:24 main:574] : INFO : Epoch 1941 | loss: 0.0311275 | val_loss: 0.0311778 | Time: 3361.62 ms
[2022-01-28 21:02:27 main:574] : INFO : Epoch 1942 | loss: 0.0311285 | val_loss: 0.031178 | Time: 3314.38 ms
[2022-01-28 21:02:30 main:574] : INFO : Epoch 1943 | loss: 0.0311267 | val_loss: 0.0311807 | Time: 3430.54 ms
[2022-01-28 21:02:34 main:574] : INFO : Epoch 1944 | loss: 0.03113 | val_loss: 0.0311757 | Time: 3465.73 ms
[2022-01-28 21:02:37 main:574] : INFO : Epoch 1945 | loss: 0.0311323 | val_loss: 0.0311733 | Time: 3426.39 ms
[2022-01-28 21:02:41 main:574] : INFO : Epoch 1946 | loss: 0.0311306 | val_loss: 0.0311772 | Time: 3295.93 ms
[2022-01-28 21:02:44 main:574] : INFO : Epoch 1947 | loss: 0.0311335 | val_loss: 0.0311753 | Time: 3358.99 ms
[2022-01-28 21:02:47 main:574] : INFO : Epoch 1948 | loss: 0.031131 | val_loss: 0.0311728 | Time: 3250.06 ms
[2022-01-28 21:02:51 main:574] : INFO : Epoch 1949 | loss: 0.0311301 | val_loss: 0.0311752 | Time: 3465.39 ms
[2022-01-28 21:02:54 main:574] : INFO : Epoch 1950 | loss: 0.0311336 | val_loss: 0.0311688 | Time: 3431.16 ms
[2022-01-28 21:02:58 main:574] : INFO : Epoch 1951 | loss: 0.0311312 | val_loss: 0.0311727 | Time: 3525.59 ms
[2022-01-28 21:03:01 main:574] : INFO : Epoch 1952 | loss: 0.0311311 | val_loss: 0.031172 | Time: 3372.21 ms
[2022-01-28 21:03:05 main:574] : INFO : Epoch 1953 | loss: 0.0311341 | val_loss: 0.0311727 | Time: 3477.06 ms
[2022-01-28 21:03:08 main:574] : INFO : Epoch 1954 | loss: 0.0311315 | val_loss: 0.0311769 | Time: 3296.74 ms
[2022-01-28 21:03:12 main:574] : INFO : Epoch 1955 | loss: 0.0311302 | val_loss: 0.0311708 | Time: 3603.51 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: NVIDIA GeForce RTX 3080 Laptop GPU)
[2022-01-30 13:50:34 main:435] : INFO : Set logging level to 1
[2022-01-30 13:50:34 main:441] : INFO : Running in BOINC Client mode
[2022-01-30 13:50:34 main:444] : INFO : Resolving all filenames
[2022-01-30 13:50:34 main:452] : INFO : Resolved: dataset.hdf5 => dataset.hdf5 (exists = 1)
[2022-01-30 13:50:34 main:452] : INFO : Resolved: model.cfg => model.cfg (exists = 1)
[2022-01-30 13:50:34 main:452] : INFO : Resolved: model-final.pt => model-final.pt (exists = 0)
[2022-01-30 13:50:34 main:452] : INFO : Resolved: model-input.pt => model-input.pt (exists = 1)
[2022-01-30 13:50:34 main:452] : INFO : Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2022-01-30 13:50:34 main:472] : INFO : Dataset filename: dataset.hdf5
[2022-01-30 13:50:34 main:474] : INFO : Configuration:
[2022-01-30 13:50:34 main:475] : INFO : Model type: GRU
[2022-01-30 13:50:34 main:476] : INFO : Validation Loss Threshold: 0.0001
[2022-01-30 13:50:34 main:477] : INFO : Max Epochs: 2048
[2022-01-30 13:50:34 main:478] : INFO : Batch Size: 128
[2022-01-30 13:50:34 main:479] : INFO : Learning Rate: 0.01
[2022-01-30 13:50:34 main:480] : INFO : Patience: 10
[2022-01-30 13:50:34 main:481] : INFO : Hidden Width: 12
[2022-01-30 13:50:34 main:482] : INFO : # Recurrent Layers: 4
[2022-01-30 13:50:34 main:483] : INFO : # Backend Layers: 4
[2022-01-30 13:50:34 main:484] : INFO : # Threads: 1
[2022-01-30 13:50:34 main:486] : INFO : Preparing Dataset
[2022-01-30 13:50:34 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Xt from dataset.hdf5 into memory
[2022-01-30 13:50:35 load_hdf5_ds_into_tensor:28] : INFO : Loading Dataset /Yt from dataset.hdf5 into memory
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFDCF4090D2
Engaging BOINC Windows Runtime Debugger...
********************
BOINC Windows Runtime Debugger Version 7.17.0
Dump Timestamp : 01/30/22 14:11:39
Install Directory : C:\Program Files\BOINC\
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\0;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome
ModLoad: 00000000daee0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000d1bd0000 00000000001f5000 C:\Windows\SYSTEM32\ntdll.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : ntdll.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d0240000 00000000000be000 C:\Windows\System32\KERNEL32.DLL (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : kernel32.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000cf340000 00000000002c8000 C:\Windows\System32\KERNELBASE.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : kernelbase.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d0090000 00000000001a1000 C:\Windows\System32\USER32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : user32.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000cf860000 0000000000022000 C:\Windows\System32\win32u.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : win32u.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1466
ModLoad: 00000000cfdb0000 000000000002b000 C:\Windows\System32\GDI32.dll (6.2.19041.1202) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32.pdb
File Version : 10.0.19041.1202 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1202
ModLoad: 00000000cfa40000 000000000010d000 C:\Windows\System32\gdi32full.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : gdi32full.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1466
ModLoad: 00000000cf7c0000 000000000009d000 C:\Windows\System32\msvcp_win.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : msvcp_win.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000cf610000 0000000000100000 C:\Windows\System32\ucrtbase.dll (6.2.19041.789) (-exported- Symbols Loaded)
Linked PDB Filename : ucrtbase.pdb
File Version : 10.0.19041.789 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.789
ModLoad: 00000000cfeb0000 00000000000ae000 C:\Windows\System32\ADVAPI32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d1490000 000000000009e000 C:\Windows\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.19041.546
ModLoad: 00000000d15a0000 000000000009c000 C:\Windows\System32\sechost.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 10.0.19041.320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.320
ModLoad: 00000000cff60000 0000000000125000 C:\Windows\System32\RPCRT4.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : rpcrt4.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000c3780000 0000000000058000 C:\ProgramData\BOINC\slots\0\c10.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000af0d0000 000000000008d000 C:\Windows\SYSTEM32\MSVCP140.dll (14.29.30133.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
File Version : 14.29.30133.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30133.0
ModLoad: 00000000cd0d0000 00000000001e4000 C:\Windows\SYSTEM32\dbghelp.dll (6.2.19041.867) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 10.0.19041.867 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.867
ModLoad: 00000000b1ac0000 000000000001b000 C:\Windows\SYSTEM32\VCRUNTIME140.dll (14.29.30133.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
File Version : 14.29.30133.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30133.0
ModLoad: 00000000af080000 000000000000c000 C:\Windows\SYSTEM32\VCRUNTIME140_1.dll (14.29.30133.0) (-exported- Symbols Loaded)
Linked PDB Filename : d:\a01\_work\2\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
File Version : 14.29.30133.0 built by: vcwrkspc
Company Name : Microsoft Corporation
Product Name : Microsoft® Visual Studio®
Product Version : 14.29.30133.0
ModLoad: 00000000f6340000 000000000abe7000 C:\ProgramData\BOINC\slots\0\torch_cpu.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 0000000000f30000 0000000024b49000 C:\ProgramData\BOINC\slots\0\torch_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000689a0000 000000000020d000 C:\ProgramData\BOINC\slots\0\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
Linked PDB Filename : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
File Version : 20200205
Company Name : Intel Corporation
Product Name : Intel(R) OpenMP* Runtime Library
Product Version : 5.0
ModLoad: 0000000042060000 00000000002db000 C:\ProgramData\BOINC\slots\0\fbgemm.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000bbc20000 0000000000045000 C:\ProgramData\BOINC\slots\0\c10_cuda.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000f2e80000 00000000034b2000 C:\ProgramData\BOINC\slots\0\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA RAND Library
Product Version : 6,14,11,1012
ModLoad: 00000000cb790000 0000000000010000 C:\ProgramData\BOINC\slots\0\nvToolsExt64_1.dll (-exported- Symbols Loaded)
Linked PDB Filename : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb
ModLoad: 00000000efdd0000 00000000030a5000 C:\ProgramData\BOINC\slots\0\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1031
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA SPARSE BLAS Library
Product Version : 6,14,11,1031
ModLoad: 00000000d4a20000 000000001b3a4000 C:\ProgramData\BOINC\slots\0\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,10020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 10.2.86 CUDNN Library
Product Version : 6,14,11,10020
ModLoad: 00000000a3c80000 0000000000040000 C:\ProgramData\BOINC\slots\0\asmjit.dll (-exported- Symbols Loaded)
Linked PDB Filename :
ModLoad: 00000000c9a50000 0000000003d5b000 C:\ProgramData\BOINC\slots\0\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000c08b0000 000000000919f000 C:\ProgramData\BOINC\slots\0\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1012
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA FFT Library
Product Version : 6,14,11,1012
ModLoad: 00000000be9e0000 0000000001eca000 C:\ProgramData\BOINC\slots\0\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,1022
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA BLAS Library
Product Version : 6,14,11,1022
ModLoad: 00000000cfc10000 0000000000030000 C:\Windows\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : imm32.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000cdb60000 0000000000012000 C:\Windows\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : Kernel.Appcore.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d16a0000 0000000000472000 C:\Windows\System32\Setupapi.dll (6.2.19041.1237) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000cf9f0000 000000000004e000 C:\Windows\System32\cfgmgr32.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 00000000cfbe0000 0000000000027000 C:\Windows\System32\bcrypt.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : bcrypt.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000cf0a0000 0000000000034000 C:\Windows\SYSTEM32\DEVOBJ.dll (6.2.19041.1151) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 10.0.19041.1151 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1151
ModLoad: 00000000cf2e0000 0000000000060000 C:\Windows\System32\WINTRUST.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1466
ModLoad: 00000000cf890000 0000000000156000 C:\Windows\System32\CRYPT32.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000ceec0000 0000000000012000 C:\Windows\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000d04a0000 0000000000744000 C:\Windows\System32\Shell32.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000cd3b0000 0000000000795000 C:\Windows\SYSTEM32\windows.storage.dll (6.2.19041.1466) (-exported- Symbols Loaded)
Linked PDB Filename : Windows.Storage.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d0c20000 0000000000355000 C:\Windows\System32\combase.dll (6.2.19041.1348) (-exported- Symbols Loaded)
Linked PDB Filename : combase.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000ced30000 000000000002e000 C:\Windows\SYSTEM32\Wldp.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : WLDP.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d1060000 00000000000ad000 C:\Windows\System32\SHCORE.dll (6.2.19041.1387) (-exported- Symbols Loaded)
Linked PDB Filename : shcore.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 00000000d1540000 0000000000055000 C:\Windows\System32\shlwapi.dll (6.2.19041.1023) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 10.0.19041.1466 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Betriebssystem Microsoft® Windows®
Product Version : 10.0.19041.1466
ModLoad: 000000003e690000 00000000002cd000 C:\Windows\system32\nvcuda.dll (30.0.15.1123) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
File Version : 30.0.15.1123
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.58 driver
Product Version : 30.0.15.1123
ModLoad: 00000000bd340000 00000000016a0000 C:\Windows\system32\DriverStore\FileRepository\nvmii.inf_amd64_b43217aab37e3e24\nvcuda64.dll (30.0.15.1123) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
File Version : 30.0.15.1123
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 11.6.58 driver
Product Version : 30.0.15.1123
ModLoad: 00000000c7680000 000000000000a000 C:\Windows\SYSTEM32\VERSION.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000c6e30000 0000000000031000 C:\Windows\SYSTEM32\cryptnet.dll (6.2.19041.906) (-exported- Symbols Loaded)
Linked PDB Filename : cryptnet.pdb
File Version : 10.0.19041.906 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.906
ModLoad: 00000000c3d50000 000000000014a000 C:\Windows\SYSTEM32\drvstore.dll (6.2.19041.1320) (-exported- Symbols Loaded)
Linked PDB Filename : drvstore.pdb
File Version : 10.0.19041.1320 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1320
ModLoad: 00000000ceca0000 000000000000c000 C:\Windows\SYSTEM32\cryptbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 10.0.19041.546 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.546
ModLoad: 00000000c2ca0000 0000000000767000 C:\Windows\SYSTEM32\nvapi64.dll (30.0.15.1123) (-exported- Symbols Loaded)
Linked PDB Filename : C:\dvs\p4\build\sw\rel\gpu_drv\r510\r511_04\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
File Version : 30.0.15.1123
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 30.0.15.1123
ModLoad: 0000000036250000 00000000008b3000 C:\Windows\System32\DriverStore\FileRepository\nvmii.inf_amd64_b43217aab37e3e24\nvptxJitCompiler64.dll (30.0.15.1123) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 30.0.15.1123
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows PTX JIT Compiler
Product Version : 30.0.15.1123
ModLoad: 00000000cfb50000 0000000000082000 C:\Windows\System32\bcryptPrimitives.dll (6.2.19041.1415) (-exported- Symbols Loaded)
Linked PDB Filename : bcryptprimitives.pdb
File Version : 10.0.19041.1415 (WinBuild.160101.0800)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 10.0.19041.1415
*** Dump of the Process Statistics: ***
- I/O Operations Counters -
Read: 2752, Write: 1197, Other 5964
- I/O Transfers Counters -
Read: 6664893, Write: 828411339, Other 27366
- Paged Pool Usage -
QuotaPagedPoolUsage: 3445288, QuotaPeakPagedPoolUsage: 3445288
QuotaNonPagedPoolUsage: 39432, QuotaPeakNonPagedPoolUsage: 52352
- Virtual Memory Usage -
VirtualSize: -1134342144, PeakVirtualSize: 66387968
- Pagefile Usage -
PagefileUsage: -1134342144, PeakPagefileUsage: -683339776
- Working Set Size -
WorkingSetSize: 1512710144, PeakWorkingSetSize: 1966149632, PageFaultCount: 12839706
*** Dump of thread ID 33984 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 15422553.000000
- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFDCF4090D2
- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=00000000db198c40 rdx=00000000f64fe500 rsi=0000000000000000 rdi=0000000000000000
r8=00000000f64fe500 r9=00000000db198c30 r10=0000000000000fff r11=0000000000000ff0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000cf4090d2 rsp=00000000f64fe4d8 rbp=0000000000000000
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f64fe4d0 daf19100 00000001 f64fe500 f64fe500 db198c30 KERNELBASE!DebugBreak+0x0
f64fe910 daf19ce4 39400000 f64fec74 00000000 000003a4 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f64ff870 daf1a084 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f64ff8a0 d0257034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f64ff8d0 d1c22651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f64ff950 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 38832 (state: Running): ***
- Information -
Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 180624992.000000, User Time: 11949531136.000000, Wait Time: 15422553.000000
- Registers -
rax=0000000000000003 rbx=0000000026a9b428 rcx=000000000000001b rdx=000000000000007b rsi=00000000269b0920 rdi=000000000000007a
r8=0000000000000003 r9=0000000026a9b408 r10=0000000000000084 r11=00000000d4020078 r12=0000000000000000 r13=00000000233f3d00
r14=0000000000000001 r15=0000000023461410 rip=00000000362d91c5 rsp=00000000f5d22868 rbp=0000000023458be8
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000206
- Callstack -
ChildEBP RetAddr Args to Child
f5d22860 365e6eca f5d22ad0 23458be8 269b0920 0000007a nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d228e0 365e4212 23458be8 00000000 23465148 00000000 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d22960 365dd462 0000018f 233f3d00 23458be8 23465148 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d22c60 365dd5f8 233f3d00 26929400 233f3d00 00000001 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d22ed0 36320959 233f3d00 233f3d00 233f6c30 26921d98 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d22f00 36320cd2 f5d230f0 36826e98 233f3d00 f5d230f0 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d22fa0 36320dc3 0000008a 23c2f0f0 00000086 f5d230f0 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d230a0 362f7c99 00000000 233f98c8 233f3d00 f5d232d0 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d23160 36477b12 00c48060 00000007 00c48060 65d30008 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d23190 36251ff9 00c48060 6fb3fea0 3625c310 6fb3fea0 nvptxJitCompiler64!nvLinkerGetLinkedCubinSize+0x0
f5d231c0 362581c1 6a6a3668 00000000 00c48060 f5d232d0 nvptxJitCompiler64!+0x0
f5d23420 36254a12 16248c88 f5d235f0 67980158 67980158 nvptxJitCompiler64!__cuda_CallJitEntryPoint+0x0
f5d237b0 36256c92 67c828e8 00000005 12244068 00000005 nvptxJitCompiler64!__cuda_CallJitEntryPoint+0x0
f5d23840 3625fa60 6fb3fea0 f5d23950 00000000 00000005 nvptxJitCompiler64!__cuda_CallJitEntryPoint+0x0
f5d23a10 3625282b 00000000 00000000 00000000 00000000 nvptxJitCompiler64!__cuda_CallJitEntryPoint+0x0
f5d24960 bd7c4afa 00000001 00000005 f5d249c0 f5d24990 nvptxJitCompiler64!__cuda_CallJitEntryPoint+0x0
f5d24bd0 bd7c4da0 6e0a5680 6bd558c8 00000056 35962025 nvcuda64!cuProfilerStop+0x0
f5d24dc0 bd4cc791 f5d25040 00000000 f5d25040 00000002 nvcuda64!cuProfilerStop+0x0
f5d24e80 bd4ccb6e 35b59090 00000000 f5d25040 02fc0f33 nvcuda64!+0x0
f5d24f40 bd4cce20 f5d251c0 f5d25201 f5d25040 f5d25029 nvcuda64!+0x0
f5d25000 bd4ee6c2 00000000 00000001 181d7e80 0000007f nvcuda64!+0x0
f5d250f0 bd3e7781 00000000 25a64858 000097b0 f5d252a0 nvcuda64!+0x0
f5d25160 bd3e78e7 7c7fbdd0 25a64858 00000000 eb0000eb nvcuda64!+0x0
f5d25250 bd3e8461 00000000 00000000 00000000 f5d252e1 nvcuda64!+0x0
f5d25330 00f38333 00000000 00000000 25a64858 181d80f8 nvcuda64!+0x0
f5d253b0 00f3fd19 36be1140 36be5248 00000000 00f3fd00 torch_cuda!+0x0
f5d253e0 00f3722b 36be5238 36be5988 36ad0101 36b98920 torch_cuda!+0x0
f5d25420 00f3a2d9 36bbf3e8 36bbf3e8 00000000 36bbf3e0 torch_cuda!+0x0
f5d254a0 00f39ef1 00000001 00000001 00000001 36bd48c0 torch_cuda!+0x0
f5d254e0 00f403d3 00000000 36ad0860 00000000 f5d254c0 torch_cuda!+0x0
f5d25510 00f323c4 00000000 358fe060 433d0000 00000001 torch_cuda!+0x0
f5d25560 00f32381 36ad0860 f5d25671 00000001 3e9c6040 torch_cuda!+0x0
f5d255a0 00f35ab0 f5d25710 00000000 04800000 cf39a65b torch_cuda!+0x0
f5d256c0 0309471e 00000001 00000001 f5d25d10 00000001 torch_cuda!+0x0
f5d258d0 fba4f058 11f75b9b f5d279a0 00c616c0 f5d25d10 torch_cuda!at::cuda::CUDAEvent::block+0x0
f5d25ad0 fba522d2 f5d25d10 f5d25c08 f5d26940 f5d26940 torch_cpu!at::native::conv_tbc_backward+0x0
f5d26030 fba51eff 00000000 f5d26940 14000000 fb92bfac torch_cpu!at::native::copy_+0x0
f5d26090 fbee4a0d 00c60101 00c60640 ffffffff fb910231 torch_cpu!at::native::copy_+0x0
f5d262b0 fd8115e6 10000000 00000000 fee2ac10 00000000 torch_cpu!at::Tensor::copy_+0x0
f5d266f0 fbee4a0d 00c60640 d1bfb3c7 f5d26778 00c60640 torch_cpu!torch::autograd::VariableType::allCUDATypes+0x0
f5d26910 fbbaa38f f5d26ad0 c379a8c0 33c4c2d0 f5d26940 torch_cpu!at::Tensor::copy_+0x0
f5d269c0 fbba9c19 f5d26f68 f5d279a0 f5d26cd0 f5d26f68 torch_cpu!at::native::to_dense_backward+0x0
f5d26bd0 fbece363 f122baad 00000000 00000002 00000002 torch_cpu!at::native::to+0x0
f5d26c10 fd136573 f5d20000 35900b40 f5d26c50 0000657a torch_cpu!at::zeros_out+0x0
f5d26c50 fbd102f2 f5d20000 feb824d6 11f7609b f5d279a0 torch_cpu!torch::autograd::GraphRoot::apply+0x0
f5d26c90 fbf01222 f5d20000 00000000 f5d26d18 f5d26da0 torch_cpu!at::native::mkldnn_sigmoid_+0x0
f5d26ed0 daefcd74 f5d27930 f5d27930 f5d27930 00000000 torch_cpu!at::Tensor::to+0x0
f5d270e0 daefaf96 f5d279c0 f5d27260 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f5d3fe80 db13c6f8 356c8250 00000000 00000000 350e4c60 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f5d3fec0 d0257034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0
f5d3fef0 d1c22651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f5d3ff70 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 25624 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15422549.000000
- Registers -
rax=000000000000005b rbx=000000000000000d rcx=000000000000000d rdx=00000000f65ff9d0 rsi=0000000000000000 rdi=000000000000000d
r8=00000000f65ff678 r9=000000003589eaa0 r10=0000000000000000 r11=00000000f65ff940 r12=0000000000000064 r13=00000000f65ff9d0
r14=00000000f65ff6d0 r15=0000000000000000 rip=00000000d1c6d8c4 rsp=00000000f65ff678 rbp=000000003589eaa0
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f65ff670 cf38cb20 00000000 f5f0f000 d1d3a4d0 f5f0f000 ntdll!ZwWaitForMultipleObjects+0x0
f65ff960 cf38ca1e 33c53110 006e0069 0000000d 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0
f65ff9a0 bd51d384 358a1ec0 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0
f65ffbe0 bd5c56fd 00000000 3589eaa0 00000000 00000000 nvcuda64!+0x0
f65ffc90 bd51cd53 1f8caac0 33db324b 35eb7530 00000000 nvcuda64!cuProfilerStop+0x0
f65ffcc0 bd854064 35eb7530 00000000 00000000 00000000 nvcuda64!+0x0
f65ffcf0 d0257034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
f65ffd20 d1c22651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f65ffda0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Dump of thread ID 10780 (state: Waiting): ***
- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 15341791.000000
- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=0000000000000294 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000294
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000035872d60 r13=0000000000000000
r14=0000000000000294 r15=0000000000000000 rip=00000000d1c6cdf4 rsp=00000000f66ffb48 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246
- Callstack -
ChildEBP RetAddr Args to Child
f66ffb40 cf361a5e 00000000 00001f80 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
f66ffbe0 bd74596e 35eb7560 00000000 00000000 00000294 KERNELBASE!WaitForSingleObjectEx+0x0
f66ffe20 bd51cd53 00000000 35eb7560 bd340000 35889790 nvcuda64!cuProfilerStop+0x0
f66ffe50 bd854064 35eb7560 00000000 00000000 00000000 nvcuda64!+0x0
f66ffe80 d0257034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0
f66ffeb0 d1c22651 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0
f66fff30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0
*** Debug Message Dump ****
*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0
Exiting...
</stderr_txt>
]]>
©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)