Task 3519758

Name ParityModified-1606530440-31170-8_5
Workunit 1695353
Created 3 Jan 2021, 9:27:12 UTC
Sent 3 Jan 2021, 10:49:59 UTC
Report deadline 10 Jan 2021, 10:49:59 UTC
Received 3 Jan 2021, 18:08:58 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -529697949 (0xE06D7363) Unknown error code
Computer ID 1625
Run time 16 min 53 sec
CPU time 14 min 11 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 3,197.66 GFLOPS
Application version Machine Learning Dataset Generator (GPU) v9.75 (cuda10200)
windows_x86_64
Peak working set size 2.10 GB
Peak swap size 4.00 GB
Peak disk usage 1.53 GB

Stderr output

<core_client_version>7.14.3</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -529697949 (0xe06d7363)</message>
<stderr_txt>
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1650)
[2021-01-04 01:33:05	                main:435]	:	INFO	:	Set logging level to 1
[2021-01-04 01:33:05	                main:441]	:	INFO	:	Running in BOINC Client mode
[2021-01-04 01:33:05	                main:444]	:	INFO	:	Resolving all filenames
[2021-01-04 01:33:05	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 (exists = 1)
[2021-01-04 01:33:05	                main:452]	:	INFO	:	Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8_5_r1552687190_1 (exists = 0)
[2021-01-04 01:33:05	                main:452]	:	INFO	:	Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8_5_r1552687190_0 (exists = 0)
[2021-01-04 01:33:05	                main:452]	:	INFO	:	Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8 (exists = 1)
[2021-01-04 01:33:05	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 0)
[2021-01-04 01:33:05	                main:472]	:	INFO	:	Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5
[2021-01-04 01:33:05	                main:474]	:	INFO	:	Configuration: 
[2021-01-04 01:33:05	                main:475]	:	INFO	:	    Model type: GRU
[2021-01-04 01:33:05	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2021-01-04 01:33:05	                main:477]	:	INFO	:	    Max Epochs: 1024
[2021-01-04 01:33:05	                main:478]	:	INFO	:	    Batch Size: 128
[2021-01-04 01:33:05	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2021-01-04 01:33:06	                main:480]	:	INFO	:	    Patience: 10
[2021-01-04 01:33:06	                main:481]	:	INFO	:	    Hidden Width: 12
[2021-01-04 01:33:06	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2021-01-04 01:33:06	                main:483]	:	INFO	:	    # Backend Layers: 4
[2021-01-04 01:33:06	                main:484]	:	INFO	:	    # Threads: 1
[2021-01-04 01:33:06	                main:486]	:	INFO	:	Preparing Dataset
[2021-01-04 01:33:06	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:33:07	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:33:16	                load:106]	:	INFO	:	Successfully loaded dataset of 2048 examples into memory.
[2021-01-04 01:33:16	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:33:16	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:33:17	                load:106]	:	INFO	:	Successfully loaded dataset of 512 examples into memory.
[2021-01-04 01:33:17	                main:494]	:	INFO	:	Creating Model
[2021-01-04 01:33:17	                main:507]	:	INFO	:	Preparing config file
[2021-01-04 01:33:17	                main:519]	:	INFO	:	Creating new config file
[2021-01-04 01:33:17	                main:538]	:	INFO	:	This is a continuation WU, loading previous network
[2021-01-04 01:33:24	                main:559]	:	INFO	:	Loading DataLoader into Memory
[2021-01-04 01:33:24	                main:562]	:	INFO	:	Starting Training
[2021-01-04 01:33:30	                main:574]	:	INFO	:	Epoch 1 | loss: nan | val_loss: nan | Time: 5909.04 ms
[2021-01-04 01:33:34	                main:574]	:	INFO	:	Epoch 2 | loss: nan | val_loss: nan | Time: 4618.4 ms
[2021-01-04 01:33:39	                main:574]	:	INFO	:	Epoch 3 | loss: nan | val_loss: nan | Time: 4745.81 ms
[2021-01-04 01:33:54	                main:574]	:	INFO	:	Epoch 4 | loss: nan | val_loss: nan | Time: 14756.3 ms
[2021-01-04 01:33:59	                main:574]	:	INFO	:	Epoch 5 | loss: nan | val_loss: nan | Time: 4824.92 ms
[2021-01-04 01:34:04	                main:574]	:	INFO	:	Epoch 6 | loss: nan | val_loss: nan | Time: 4712.74 ms
[2021-01-04 01:34:08	                main:574]	:	INFO	:	Epoch 7 | loss: nan | val_loss: nan | Time: 4539.34 ms
[2021-01-04 01:34:13	                main:574]	:	INFO	:	Epoch 8 | loss: nan | val_loss: nan | Time: 4680.19 ms
[2021-01-04 01:34:18	                main:574]	:	INFO	:	Epoch 9 | loss: nan | val_loss: nan | Time: 4589.73 ms
[2021-01-04 01:34:23	                main:574]	:	INFO	:	Epoch 10 | loss: nan | val_loss: nan | Time: 5231.47 ms
[2021-01-04 01:34:28	                main:574]	:	INFO	:	Epoch 11 | loss: nan | val_loss: nan | Time: 5030.45 ms
[2021-01-04 01:34:33	                main:574]	:	INFO	:	Epoch 12 | loss: nan | val_loss: nan | Time: 4543.69 ms
[2021-01-04 01:34:38	                main:574]	:	INFO	:	Epoch 13 | loss: nan | val_loss: nan | Time: 4730.68 ms
[2021-01-04 01:34:42	                main:574]	:	INFO	:	Epoch 14 | loss: nan | val_loss: nan | Time: 4711.36 ms
[2021-01-04 01:34:47	                main:574]	:	INFO	:	Epoch 15 | loss: nan | val_loss: nan | Time: 4656.04 ms
[2021-01-04 01:34:52	                main:574]	:	INFO	:	Epoch 16 | loss: nan | val_loss: nan | Time: 4933.17 ms
[2021-01-04 01:34:57	                main:574]	:	INFO	:	Epoch 17 | loss: nan | val_loss: nan | Time: 4712 ms
[2021-01-04 01:35:01	                main:574]	:	INFO	:	Epoch 18 | loss: nan | val_loss: nan | Time: 4707.98 ms
[2021-01-04 01:35:06	                main:574]	:	INFO	:	Epoch 19 | loss: nan | val_loss: nan | Time: 4708.11 ms
[2021-01-04 01:35:11	                main:574]	:	INFO	:	Epoch 20 | loss: nan | val_loss: nan | Time: 4838.11 ms
[2021-01-04 01:35:16	                main:574]	:	INFO	:	Epoch 21 | loss: nan | val_loss: nan | Time: 4406.52 ms
[2021-01-04 01:35:21	                main:574]	:	INFO	:	Epoch 22 | loss: nan | val_loss: nan | Time: 4783.5 ms
[2021-01-04 01:35:25	                main:574]	:	INFO	:	Epoch 23 | loss: nan | val_loss: nan | Time: 4451.41 ms
[2021-01-04 01:35:30	                main:574]	:	INFO	:	Epoch 24 | loss: nan | val_loss: nan | Time: 4481.28 ms
[2021-01-04 01:35:34	                main:574]	:	INFO	:	Epoch 25 | loss: nan | val_loss: nan | Time: 4861.33 ms
[2021-01-04 01:35:39	                main:574]	:	INFO	:	Epoch 26 | loss: nan | val_loss: nan | Time: 4745.68 ms
[2021-01-04 01:35:44	                main:574]	:	INFO	:	Epoch 27 | loss: nan | val_loss: nan | Time: 4718 ms
[2021-01-04 01:35:49	                main:574]	:	INFO	:	Epoch 28 | loss: nan | val_loss: nan | Time: 5064.71 ms
[2021-01-04 01:35:54	                main:574]	:	INFO	:	Epoch 29 | loss: nan | val_loss: nan | Time: 4938.81 ms
[2021-01-04 01:35:59	                main:574]	:	INFO	:	Epoch 30 | loss: nan | val_loss: nan | Time: 4992.4 ms
[2021-01-04 01:36:04	                main:574]	:	INFO	:	Epoch 31 | loss: nan | val_loss: nan | Time: 4890.46 ms
[2021-01-04 01:36:08	                main:574]	:	INFO	:	Epoch 32 | loss: nan | val_loss: nan | Time: 4435 ms
[2021-01-04 01:36:13	                main:574]	:	INFO	:	Epoch 33 | loss: nan | val_loss: nan | Time: 4354.07 ms
[2021-01-04 01:36:18	                main:574]	:	INFO	:	Epoch 34 | loss: nan | val_loss: nan | Time: 4899.12 ms
[2021-01-04 01:36:23	                main:574]	:	INFO	:	Epoch 35 | loss: nan | val_loss: nan | Time: 5298.6 ms
[2021-01-04 01:36:27	                main:574]	:	INFO	:	Epoch 36 | loss: nan | val_loss: nan | Time: 4056.95 ms
[2021-01-04 01:36:32	                main:574]	:	INFO	:	Epoch 37 | loss: nan | val_loss: nan | Time: 4925.88 ms
[2021-01-04 01:36:37	                main:574]	:	INFO	:	Epoch 38 | loss: nan | val_loss: nan | Time: 4592.4 ms
[2021-01-04 01:36:42	                main:574]	:	INFO	:	Epoch 39 | loss: nan | val_loss: nan | Time: 4817.94 ms
[2021-01-04 01:36:47	                main:574]	:	INFO	:	Epoch 40 | loss: nan | val_loss: nan | Time: 4750.37 ms
[2021-01-04 01:36:52	                main:574]	:	INFO	:	Epoch 41 | loss: nan | val_loss: nan | Time: 5799.25 ms
[2021-01-04 01:36:57	                main:574]	:	INFO	:	Epoch 42 | loss: nan | val_loss: nan | Time: 4905.94 ms
[2021-01-04 01:37:02	                main:574]	:	INFO	:	Epoch 43 | loss: nan | val_loss: nan | Time: 5020.51 ms
[2021-01-04 01:37:08	                main:574]	:	INFO	:	Epoch 44 | loss: nan | val_loss: nan | Time: 5393.68 ms
[2021-01-04 01:37:13	                main:574]	:	INFO	:	Epoch 45 | loss: nan | val_loss: nan | Time: 4647.6 ms
[2021-01-04 01:37:17	                main:574]	:	INFO	:	Epoch 46 | loss: nan | val_loss: nan | Time: 4355.53 ms
[2021-01-04 01:37:22	                main:574]	:	INFO	:	Epoch 47 | loss: nan | val_loss: nan | Time: 5183.94 ms
[2021-01-04 01:37:27	                main:574]	:	INFO	:	Epoch 48 | loss: nan | val_loss: nan | Time: 4700.25 ms
[2021-01-04 01:37:32	                main:574]	:	INFO	:	Epoch 49 | loss: nan | val_loss: nan | Time: 5003.09 ms
[2021-01-04 01:37:37	                main:574]	:	INFO	:	Epoch 50 | loss: nan | val_loss: nan | Time: 4652.73 ms
[2021-01-04 01:37:52	                main:574]	:	INFO	:	Epoch 51 | loss: nan | val_loss: nan | Time: 15032.6 ms
[2021-01-04 01:37:57	                main:574]	:	INFO	:	Epoch 52 | loss: nan | val_loss: nan | Time: 4716.92 ms
[2021-01-04 01:38:01	                main:574]	:	INFO	:	Epoch 53 | loss: nan | val_loss: nan | Time: 4634.47 ms
[2021-01-04 01:38:06	                main:574]	:	INFO	:	Epoch 54 | loss: nan | val_loss: nan | Time: 4631.34 ms
[2021-01-04 01:38:11	                main:574]	:	INFO	:	Epoch 55 | loss: nan | val_loss: nan | Time: 4880.55 ms
[2021-01-04 01:38:16	                main:574]	:	INFO	:	Epoch 56 | loss: nan | val_loss: nan | Time: 5028.38 ms
[2021-01-04 01:38:21	                main:574]	:	INFO	:	Epoch 57 | loss: nan | val_loss: nan | Time: 4339.61 ms
[2021-01-04 01:38:25	                main:574]	:	INFO	:	Epoch 58 | loss: nan | val_loss: nan | Time: 4545.88 ms
[2021-01-04 01:38:30	                main:574]	:	INFO	:	Epoch 59 | loss: nan | val_loss: nan | Time: 4681.41 ms
[2021-01-04 01:38:35	                main:574]	:	INFO	:	Epoch 60 | loss: nan | val_loss: nan | Time: 4863.9 ms
[2021-01-04 01:38:40	                main:574]	:	INFO	:	Epoch 61 | loss: nan | val_loss: nan | Time: 4590.38 ms
[2021-01-04 01:38:45	                main:574]	:	INFO	:	Epoch 62 | loss: nan | val_loss: nan | Time: 4936.92 ms
[2021-01-04 01:38:50	                main:574]	:	INFO	:	Epoch 63 | loss: nan | val_loss: nan | Time: 5816.96 ms
[2021-01-04 01:39:05	                main:574]	:	INFO	:	Epoch 64 | loss: nan | val_loss: nan | Time: 15073.8 ms
[2021-01-04 01:39:10	                main:574]	:	INFO	:	Epoch 65 | loss: nan | val_loss: nan | Time: 4057.52 ms
[2021-01-04 01:39:14	                main:574]	:	INFO	:	Epoch 66 | loss: nan | val_loss: nan | Time: 4662.71 ms
[2021-01-04 01:39:19	                main:574]	:	INFO	:	Epoch 67 | loss: nan | val_loss: nan | Time: 4875.62 ms
[2021-01-04 01:39:24	                main:574]	:	INFO	:	Epoch 68 | loss: nan | val_loss: nan | Time: 4714.12 ms
[2021-01-04 01:39:29	                main:574]	:	INFO	:	Epoch 69 | loss: nan | val_loss: nan | Time: 4851.19 ms
[2021-01-04 01:39:34	                main:574]	:	INFO	:	Epoch 70 | loss: nan | val_loss: nan | Time: 5269.89 ms
[2021-01-04 01:39:39	                main:574]	:	INFO	:	Epoch 71 | loss: nan | val_loss: nan | Time: 4989.92 ms
[2021-01-04 01:39:44	                main:574]	:	INFO	:	Epoch 72 | loss: nan | val_loss: nan | Time: 4696.33 ms
[2021-01-04 01:39:49	                main:574]	:	INFO	:	Epoch 73 | loss: nan | val_loss: nan | Time: 4534.22 ms
[2021-01-04 01:39:53	                main:574]	:	INFO	:	Epoch 74 | loss: nan | val_loss: nan | Time: 4766.1 ms
[2021-01-04 01:39:58	                main:574]	:	INFO	:	Epoch 75 | loss: nan | val_loss: nan | Time: 4074.3 ms
[2021-01-04 01:40:03	                main:574]	:	INFO	:	Epoch 76 | loss: nan | val_loss: nan | Time: 5480.36 ms
[2021-01-04 01:40:07	                main:574]	:	INFO	:	Epoch 77 | loss: nan | val_loss: nan | Time: 4344.05 ms
[2021-01-04 01:40:12	                main:574]	:	INFO	:	Epoch 78 | loss: nan | val_loss: nan | Time: 4672.04 ms
[2021-01-04 01:40:18	                main:574]	:	INFO	:	Epoch 79 | loss: nan | val_loss: nan | Time: 5300.21 ms
[2021-01-04 01:40:23	                main:574]	:	INFO	:	Epoch 80 | loss: nan | val_loss: nan | Time: 4907.69 ms
[2021-01-04 01:40:27	                main:574]	:	INFO	:	Epoch 81 | loss: nan | val_loss: nan | Time: 4459.04 ms
[2021-01-04 01:40:32	                main:574]	:	INFO	:	Epoch 82 | loss: nan | val_loss: nan | Time: 4853.72 ms
[2021-01-04 01:40:37	                main:574]	:	INFO	:	Epoch 83 | loss: nan | val_loss: nan | Time: 4898.48 ms
[2021-01-04 01:40:42	                main:574]	:	INFO	:	Epoch 84 | loss: nan | val_loss: nan | Time: 4890.87 ms
[2021-01-04 01:40:47	                main:574]	:	INFO	:	Epoch 85 | loss: nan | val_loss: nan | Time: 4846.51 ms
[2021-01-04 01:40:52	                main:574]	:	INFO	:	Epoch 86 | loss: nan | val_loss: nan | Time: 4802.96 ms
[2021-01-04 01:41:06	                main:574]	:	INFO	:	Epoch 87 | loss: nan | val_loss: nan | Time: 14832.6 ms
[2021-01-04 01:41:11	                main:574]	:	INFO	:	Epoch 88 | loss: nan | val_loss: nan | Time: 4639.46 ms
[2021-01-04 01:41:16	                main:574]	:	INFO	:	Epoch 89 | loss: nan | val_loss: nan | Time: 4493.66 ms
[2021-01-04 01:41:20	                main:574]	:	INFO	:	Epoch 90 | loss: nan | val_loss: nan | Time: 4526.34 ms
[2021-01-04 01:41:25	                main:574]	:	INFO	:	Epoch 91 | loss: nan | val_loss: nan | Time: 4645.71 ms
[2021-01-04 01:41:30	                main:574]	:	INFO	:	Epoch 92 | loss: nan | val_loss: nan | Time: 4691.8 ms
[2021-01-04 01:41:35	                main:574]	:	INFO	:	Epoch 93 | loss: nan | val_loss: nan | Time: 4932.66 ms
[2021-01-04 01:41:40	                main:574]	:	INFO	:	Epoch 94 | loss: nan | val_loss: nan | Time: 4620.93 ms
[2021-01-04 01:41:44	                main:574]	:	INFO	:	Epoch 95 | loss: nan | val_loss: nan | Time: 4722.67 ms
[2021-01-04 01:41:49	                main:574]	:	INFO	:	Epoch 96 | loss: nan | val_loss: nan | Time: 4986.83 ms
[2021-01-04 01:41:54	                main:574]	:	INFO	:	Epoch 97 | loss: nan | val_loss: nan | Time: 4882.77 ms
[2021-01-04 01:41:59	                main:574]	:	INFO	:	Epoch 98 | loss: nan | val_loss: nan | Time: 5003.27 ms
[2021-01-04 01:42:04	                main:574]	:	INFO	:	Epoch 99 | loss: nan | val_loss: nan | Time: 4956.1 ms
[2021-01-04 01:42:09	                main:574]	:	INFO	:	Epoch 100 | loss: nan | val_loss: nan | Time: 4585.31 ms
[2021-01-04 01:42:14	                main:574]	:	INFO	:	Epoch 101 | loss: nan | val_loss: nan | Time: 4638.78 ms
[2021-01-04 01:42:18	                main:574]	:	INFO	:	Epoch 102 | loss: nan | val_loss: nan | Time: 4766.33 ms
[2021-01-04 01:42:23	                main:574]	:	INFO	:	Epoch 103 | loss: nan | val_loss: nan | Time: 4897.33 ms
[2021-01-04 01:42:28	                main:574]	:	INFO	:	Epoch 104 | loss: nan | val_loss: nan | Time: 4567.93 ms
[2021-01-04 01:42:33	                main:574]	:	INFO	:	Epoch 105 | loss: nan | val_loss: nan | Time: 4647.72 ms
[2021-01-04 01:42:38	                main:574]	:	INFO	:	Epoch 106 | loss: nan | val_loss: nan | Time: 4948.05 ms
[2021-01-04 01:42:42	                main:574]	:	INFO	:	Epoch 107 | loss: nan | val_loss: nan | Time: 4620.36 ms
[2021-01-04 01:42:48	                main:574]	:	INFO	:	Epoch 108 | loss: nan | val_loss: nan | Time: 5111.18 ms
[2021-01-04 01:42:52	                main:574]	:	INFO	:	Epoch 109 | loss: nan | val_loss: nan | Time: 4539.92 ms
[2021-01-04 01:42:57	                main:574]	:	INFO	:	Epoch 110 | loss: nan | val_loss: nan | Time: 4712.77 ms
[2021-01-04 01:43:02	                main:574]	:	INFO	:	Epoch 111 | loss: nan | val_loss: nan | Time: 5103.04 ms
[2021-01-04 01:43:07	                main:574]	:	INFO	:	Epoch 112 | loss: nan | val_loss: nan | Time: 4824.1 ms
[2021-01-04 01:43:12	                main:574]	:	INFO	:	Epoch 113 | loss: nan | val_loss: nan | Time: 4875.91 ms
[2021-01-04 01:43:17	                main:574]	:	INFO	:	Epoch 114 | loss: nan | val_loss: nan | Time: 4804.44 ms
[2021-01-04 01:43:22	                main:574]	:	INFO	:	Epoch 115 | loss: nan | val_loss: nan | Time: 5001.58 ms
[2021-01-04 01:43:26	                main:574]	:	INFO	:	Epoch 116 | loss: nan | val_loss: nan | Time: 4598.1 ms
[2021-01-04 01:43:31	                main:574]	:	INFO	:	Epoch 117 | loss: nan | val_loss: nan | Time: 4607.38 ms
[2021-01-04 01:43:36	                main:574]	:	INFO	:	Epoch 118 | loss: nan | val_loss: nan | Time: 4843.25 ms
[2021-01-04 01:43:40	                main:574]	:	INFO	:	Epoch 119 | loss: nan | val_loss: nan | Time: 4448.41 ms
[2021-01-04 01:43:46	                main:574]	:	INFO	:	Epoch 120 | loss: nan | val_loss: nan | Time: 5172.72 ms
[2021-01-04 01:43:51	                main:574]	:	INFO	:	Epoch 121 | loss: nan | val_loss: nan | Time: 4855.6 ms
[2021-01-04 01:43:55	                main:574]	:	INFO	:	Epoch 122 | loss: nan | val_loss: nan | Time: 4591.13 ms
[2021-01-04 01:44:01	                main:574]	:	INFO	:	Epoch 123 | loss: nan | val_loss: nan | Time: 5301.72 ms
[2021-01-04 01:44:06	                main:574]	:	INFO	:	Epoch 124 | loss: nan | val_loss: nan | Time: 4994.53 ms
[2021-01-04 01:44:11	                main:574]	:	INFO	:	Epoch 125 | loss: nan | val_loss: nan | Time: 4846.77 ms
[2021-01-04 01:44:15	                main:574]	:	INFO	:	Epoch 126 | loss: nan | val_loss: nan | Time: 4813.79 ms
[2021-01-04 01:44:20	                main:574]	:	INFO	:	Epoch 127 | loss: nan | val_loss: nan | Time: 4431.32 ms
[2021-01-04 01:44:25	                main:574]	:	INFO	:	Epoch 128 | loss: nan | val_loss: nan | Time: 5024.18 ms
[2021-01-04 01:44:30	                main:574]	:	INFO	:	Epoch 129 | loss: nan | val_loss: nan | Time: 4808.16 ms
[2021-01-04 01:44:35	                main:574]	:	INFO	:	Epoch 130 | loss: nan | val_loss: nan | Time: 4673.75 ms
[2021-01-04 01:44:39	                main:574]	:	INFO	:	Epoch 131 | loss: nan | val_loss: nan | Time: 4753.56 ms
[2021-01-04 01:44:44	                main:574]	:	INFO	:	Epoch 132 | loss: nan | val_loss: nan | Time: 4699.28 ms
[2021-01-04 01:44:49	                main:574]	:	INFO	:	Epoch 133 | loss: nan | val_loss: nan | Time: 4812.29 ms
[2021-01-04 01:44:54	                main:574]	:	INFO	:	Epoch 134 | loss: nan | val_loss: nan | Time: 5091.24 ms
[2021-01-04 01:44:59	                main:574]	:	INFO	:	Epoch 135 | loss: nan | val_loss: nan | Time: 4579.74 ms
[2021-01-04 01:45:03	                main:574]	:	INFO	:	Epoch 136 | loss: nan | val_loss: nan | Time: 4403.43 ms
[2021-01-04 01:45:07	                main:574]	:	INFO	:	Epoch 137 | loss: nan | val_loss: nan | Time: 4148.26 ms
[2021-01-04 01:45:12	                main:574]	:	INFO	:	Epoch 138 | loss: nan | val_loss: nan | Time: 4488.47 ms
[2021-01-04 01:45:17	                main:574]	:	INFO	:	Epoch 139 | loss: nan | val_loss: nan | Time: 4759.89 ms
[2021-01-04 01:45:22	                main:574]	:	INFO	:	Epoch 140 | loss: nan | val_loss: nan | Time: 4865.56 ms
[2021-01-04 01:45:26	                main:574]	:	INFO	:	Epoch 141 | loss: nan | val_loss: nan | Time: 4172.7 ms
[2021-01-04 01:45:30	                main:574]	:	INFO	:	Epoch 142 | loss: nan | val_loss: nan | Time: 4467.47 ms
[2021-01-04 01:45:35	                main:574]	:	INFO	:	Epoch 143 | loss: nan | val_loss: nan | Time: 4436.86 ms
[2021-01-04 01:45:39	                main:574]	:	INFO	:	Epoch 144 | loss: nan | val_loss: nan | Time: 4397.71 ms
[2021-01-04 01:45:44	                main:574]	:	INFO	:	Epoch 145 | loss: nan | val_loss: nan | Time: 4774.54 ms
[2021-01-04 01:45:48	                main:574]	:	INFO	:	Epoch 146 | loss: nan | val_loss: nan | Time: 4205.64 ms
[2021-01-04 01:45:53	                main:574]	:	INFO	:	Epoch 147 | loss: nan | val_loss: nan | Time: 4670.69 ms
[2021-01-04 01:46:08	                main:574]	:	INFO	:	Epoch 148 | loss: nan | val_loss: nan | Time: 14511.4 ms
[2021-01-04 01:46:13	                main:574]	:	INFO	:	Epoch 149 | loss: nan | val_loss: nan | Time: 4875.23 ms
[2021-01-04 01:46:17	                main:574]	:	INFO	:	Epoch 150 | loss: nan | val_loss: nan | Time: 4380.6 ms
[2021-01-04 01:46:22	                main:574]	:	INFO	:	Epoch 151 | loss: nan | val_loss: nan | Time: 4692.09 ms
[2021-01-04 01:46:27	                main:574]	:	INFO	:	Epoch 152 | loss: nan | val_loss: nan | Time: 4816.63 ms
[2021-01-04 01:46:31	                main:574]	:	INFO	:	Epoch 153 | loss: nan | val_loss: nan | Time: 4467.99 ms
[2021-01-04 01:46:36	                main:574]	:	INFO	:	Epoch 154 | loss: nan | val_loss: nan | Time: 4738.02 ms
[2021-01-04 01:46:41	                main:574]	:	INFO	:	Epoch 155 | loss: nan | val_loss: nan | Time: 4800.03 ms
[2021-01-04 01:46:45	                main:574]	:	INFO	:	Epoch 156 | loss: nan | val_loss: nan | Time: 4343.87 ms
[2021-01-04 01:46:50	                main:574]	:	INFO	:	Epoch 157 | loss: nan | val_loss: nan | Time: 4592.68 ms
[2021-01-04 01:46:55	                main:574]	:	INFO	:	Epoch 158 | loss: nan | val_loss: nan | Time: 4564.02 ms
[2021-01-04 01:46:59	                main:574]	:	INFO	:	Epoch 159 | loss: nan | val_loss: nan | Time: 4619.36 ms
[2021-01-04 01:47:04	                main:574]	:	INFO	:	Epoch 160 | loss: nan | val_loss: nan | Time: 4981.87 ms
[2021-01-04 01:47:09	                main:574]	:	INFO	:	Epoch 161 | loss: nan | val_loss: nan | Time: 5040.64 ms
[2021-01-04 01:47:14	                main:574]	:	INFO	:	Epoch 162 | loss: nan | val_loss: nan | Time: 4939.98 ms
[2021-01-04 01:47:19	                main:574]	:	INFO	:	Epoch 163 | loss: nan | val_loss: nan | Time: 4711.97 ms
[2021-01-04 01:47:24	                main:574]	:	INFO	:	Epoch 164 | loss: nan | val_loss: nan | Time: 4653.33 ms
[2021-01-04 01:47:29	                main:574]	:	INFO	:	Epoch 165 | loss: nan | val_loss: nan | Time: 5051.24 ms
[2021-01-04 01:47:34	                main:574]	:	INFO	:	Epoch 166 | loss: nan | val_loss: nan | Time: 4753.38 ms
[2021-01-04 01:47:38	                main:574]	:	INFO	:	Epoch 167 | loss: nan | val_loss: nan | Time: 4264.42 ms
[2021-01-04 01:47:43	                main:574]	:	INFO	:	Epoch 168 | loss: nan | val_loss: nan | Time: 4856.48 ms
[2021-01-04 01:47:48	                main:574]	:	INFO	:	Epoch 169 | loss: nan | val_loss: nan | Time: 4417.85 ms
[2021-01-04 01:47:52	                main:574]	:	INFO	:	Epoch 170 | loss: nan | val_loss: nan | Time: 4781.93 ms
[2021-01-04 01:48:08	                main:574]	:	INFO	:	Epoch 171 | loss: nan | val_loss: nan | Time: 15098.6 ms
[2021-01-04 01:48:12	                main:574]	:	INFO	:	Epoch 172 | loss: nan | val_loss: nan | Time: 4864.46 ms
[2021-01-04 01:48:17	                main:574]	:	INFO	:	Epoch 173 | loss: nan | val_loss: nan | Time: 4884.08 ms
[2021-01-04 01:48:22	                main:574]	:	INFO	:	Epoch 174 | loss: nan | val_loss: nan | Time: 4516.94 ms
[2021-01-04 01:48:27	                main:574]	:	INFO	:	Epoch 175 | loss: nan | val_loss: nan | Time: 4845.01 ms
[2021-01-04 01:48:32	                main:574]	:	INFO	:	Epoch 176 | loss: nan | val_loss: nan | Time: 5001.52 ms
[2021-01-04 01:48:37	                main:574]	:	INFO	:	Epoch 177 | loss: nan | val_loss: nan | Time: 4641.32 ms
[2021-01-04 01:48:42	                main:574]	:	INFO	:	Epoch 178 | loss: nan | val_loss: nan | Time: 4992.87 ms
[2021-01-04 01:48:46	                main:574]	:	INFO	:	Epoch 179 | loss: nan | val_loss: nan | Time: 4745.91 ms
[2021-01-04 01:48:52	                main:574]	:	INFO	:	Epoch 180 | loss: nan | val_loss: nan | Time: 5488.19 ms
[2021-01-04 01:48:57	                main:574]	:	INFO	:	Epoch 181 | loss: nan | val_loss: nan | Time: 5389.8 ms
[2021-01-04 01:49:02	                main:574]	:	INFO	:	Epoch 182 | loss: nan | val_loss: nan | Time: 4599.78 ms
[2021-01-04 01:49:07	                main:574]	:	INFO	:	Epoch 183 | loss: nan | val_loss: nan | Time: 4755.38 ms
[2021-01-04 01:49:12	                main:574]	:	INFO	:	Epoch 184 | loss: nan | val_loss: nan | Time: 4946.42 ms
[2021-01-04 01:49:16	                main:574]	:	INFO	:	Epoch 185 | loss: nan | val_loss: nan | Time: 4710.99 ms
[2021-01-04 01:49:21	                main:574]	:	INFO	:	Epoch 186 | loss: nan | val_loss: nan | Time: 4910.72 ms
[2021-01-04 01:49:26	                main:574]	:	INFO	:	Epoch 187 | loss: nan | val_loss: nan | Time: 4486.1 ms
[2021-01-04 01:49:31	                main:574]	:	INFO	:	Epoch 188 | loss: nan | val_loss: nan | Time: 5174.03 ms
[2021-01-04 01:49:36	                main:574]	:	INFO	:	Epoch 189 | loss: nan | val_loss: nan | Time: 4413.07 ms
[2021-01-04 01:49:41	                main:574]	:	INFO	:	Epoch 190 | loss: nan | val_loss: nan | Time: 5319.08 ms
[2021-01-04 01:49:46	                main:574]	:	INFO	:	Epoch 191 | loss: nan | val_loss: nan | Time: 5114.42 ms
[2021-01-04 01:49:51	                main:574]	:	INFO	:	Epoch 192 | loss: nan | val_loss: nan | Time: 4618.1 ms
[2021-01-04 01:50:05	                main:574]	:	INFO	:	Epoch 193 | loss: nan | val_loss: nan | Time: 14344.8 ms
[2021-01-04 01:50:10	                main:574]	:	INFO	:	Epoch 194 | loss: nan | val_loss: nan | Time: 4556.79 ms
[2021-01-04 01:50:15	                main:574]	:	INFO	:	Epoch 195 | loss: nan | val_loss: nan | Time: 4434.66 ms
[2021-01-04 01:50:20	                main:574]	:	INFO	:	Epoch 196 | loss: nan | val_loss: nan | Time: 5244.77 ms
[2021-01-04 01:50:25	                main:574]	:	INFO	:	Epoch 197 | loss: nan | val_loss: nan | Time: 5191.57 ms
[2021-01-04 01:50:30	                main:574]	:	INFO	:	Epoch 198 | loss: nan | val_loss: nan | Time: 4801.97 ms
[2021-01-04 01:50:35	                main:574]	:	INFO	:	Epoch 199 | loss: nan | val_loss: nan | Time: 4589.38 ms
[2021-01-04 01:50:39	                main:574]	:	INFO	:	Epoch 200 | loss: nan | val_loss: nan | Time: 4758.26 ms
[2021-01-04 01:50:44	                main:574]	:	INFO	:	Epoch 201 | loss: nan | val_loss: nan | Time: 4872.61 ms
[2021-01-04 01:50:50	                main:574]	:	INFO	:	Epoch 202 | loss: nan | val_loss: nan | Time: 5358.75 ms
[2021-01-04 01:51:06	                main:574]	:	INFO	:	Epoch 203 | loss: nan | val_loss: nan | Time: 16446.6 ms
Machine Learning Dataset Generator v9.75 (Windows/x64) (libTorch: release/1.6 GPU: GeForce GTX 1650)
[2021-01-04 01:54:14	                main:435]	:	INFO	:	Set logging level to 1
[2021-01-04 01:54:14	                main:441]	:	INFO	:	Running in BOINC Client mode
[2021-01-04 01:54:14	                main:444]	:	INFO	:	Resolving all filenames
[2021-01-04 01:54:14	                main:452]	:	INFO	:	Resolved: dataset.hdf5 => ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 (exists = 1)
[2021-01-04 01:54:14	                main:452]	:	INFO	:	Resolved: model.cfg => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8_5_r1552687190_1 (exists = 1)
[2021-01-04 01:54:14	                main:452]	:	INFO	:	Resolved: model-final.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8_5_r1552687190_0 (exists = 0)
[2021-01-04 01:54:14	                main:452]	:	INFO	:	Resolved: model-input.pt => ../../projects/www.mlcathome.org_mlcathome/ParityModified-1606530440-31170-8 (exists = 1)
[2021-01-04 01:54:14	                main:452]	:	INFO	:	Resolved: snapshot.pt => snapshot.pt (exists = 1)
[2021-01-04 01:54:14	                main:472]	:	INFO	:	Dataset filename: ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5
[2021-01-04 01:54:14	                main:474]	:	INFO	:	Configuration: 
[2021-01-04 01:54:14	                main:475]	:	INFO	:	    Model type: GRU
[2021-01-04 01:54:15	                main:476]	:	INFO	:	    Validation Loss Threshold: 0.0001
[2021-01-04 01:54:15	                main:477]	:	INFO	:	    Max Epochs: 1024
[2021-01-04 01:54:15	                main:478]	:	INFO	:	    Batch Size: 128
[2021-01-04 01:54:15	                main:479]	:	INFO	:	    Learning Rate: 0.01
[2021-01-04 01:54:15	                main:480]	:	INFO	:	    Patience: 10
[2021-01-04 01:54:15	                main:481]	:	INFO	:	    Hidden Width: 12
[2021-01-04 01:54:15	                main:482]	:	INFO	:	    # Recurrent Layers: 4
[2021-01-04 01:54:15	                main:483]	:	INFO	:	    # Backend Layers: 4
[2021-01-04 01:54:15	                main:484]	:	INFO	:	    # Threads: 1
[2021-01-04 01:54:15	                main:486]	:	INFO	:	Preparing Dataset
[2021-01-04 01:54:15	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:54:16	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yt from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:54:24	                load:106]	:	INFO	:	Successfully loaded dataset of 2048 examples into memory.
[2021-01-04 01:54:24	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Xv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:54:24	load_hdf5_ds_into_tensor:28]	:	INFO	:	Loading Dataset /Yv from ../../projects/www.mlcathome.org_mlcathome/ParityModified-train-val-dataset.hdf5 into memory
[2021-01-04 01:54:24	                load:106]	:	INFO	:	Successfully loaded dataset of 512 examples into memory.
[2021-01-04 01:54:24	                main:494]	:	INFO	:	Creating Model
[2021-01-04 01:54:24	                main:507]	:	INFO	:	Preparing config file
[2021-01-04 01:54:25	                main:511]	:	INFO	:	Found checkpoint, attempting to load... 
[2021-01-04 01:54:25	                main:512]	:	INFO	:	Loading config


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFF0A54D759

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.17.0


Dump Timestamp    : 01/04/21 01:54:25
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\12;C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome


ModLoad: 00000000783d0000 00000000003df000 C:\ProgramData\BOINC\projects\www.mlcathome.org_mlcathome\mlds-gpu_9.75_windows-x86_64__cuda10200.exe (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000000c970000 00000000001f6000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 000000000aa80000 00000000000bd000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 000000000a520000 00000000002c9000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 000000000c450000 00000000001a0000 C:\WINDOWS\System32\USER32.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000a9a0000 0000000000022000 C:\WINDOWS\System32\win32u.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 000000000c180000 000000000002a000 C:\WINDOWS\System32\GDI32.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.19041.685 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.685

ModLoad: 000000000a890000 0000000000109000 C:\WINDOWS\System32\gdi32full.dll (6.2.19041.685) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.19041.685 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.685

ModLoad: 000000000a7f0000 000000000009d000 C:\WINDOWS\System32\msvcp_win.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 0000000001f50000 0000000000058000 C:\ProgramData\BOINC\slots\12\c10.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000000a0b0000 0000000000100000 C:\WINDOWS\System32\ucrtbase.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000bc50000 00000000000ac000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.19041.610) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000bba0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.19041.546

ModLoad: 000000000b690000 000000000009c000 C:\WINDOWS\System32\sechost.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000c220000 000000000012b000 C:\WINDOWS\System32\RPCRT4.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000007db0000 00000000001e4000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.19041.488) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.19041.488 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.488

ModLoad: 0000000078960000 000000000abe7000 C:\ProgramData\BOINC\slots\12\torch_cpu.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000043a0000 0000000000017000 C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll (14.24.28127.4) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\1\s\\binaries\amd64ret\bin\amd64\\vcruntime140.amd64.pdb
    File Version          : 14.24.28127.4 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.24.28127.4

ModLoad: 0000000005490000 000000000000c000 C:\WINDOWS\SYSTEM32\VCRUNTIME140_1.dll (14.24.28127.4) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\1\s\\binaries\amd64ret\bin\amd64\\vcruntime140_1.amd64.pdb
    File Version          : 14.24.28127.4 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.24.28127.4

ModLoad: 00000000d7f10000 00000000002db000 C:\ProgramData\BOINC\slots\12\fbgemm.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e0d40000 0000000000099000 C:\WINDOWS\SYSTEM32\MSVCP140.dll (14.24.28127.4) (-exported- Symbols Loaded)
    Linked PDB Filename   : d:\agent\_work\1\s\\binaries\amd64ret\bin\amd64\\msvcp140.amd64.pdb
    File Version          : 14.24.28127.4 built by: vcwrkspc
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Visual Studio&#174;
    Product Version       : 14.24.28127.4

ModLoad: 00000000dfca0000 000000000020d000 C:\ProgramData\BOINC\slots\12\libiomp5md.dll (5.0.2020.205) (-exported- Symbols Loaded)
    Linked PDB Filename   : O:\promo\20200205\tmp\win_32e-rtl_int_5_nor_dyn.rel.c0.s0.t1..h1.w1-FXILAB103\libiomp5md.pdb
    File Version          : 20200205
    Company Name          : Intel Corporation
    Product Name          : Intel(R) OpenMP* Runtime Library
    Product Version       : 5.0

ModLoad: 00000000017a0000 0000000000040000 C:\ProgramData\BOINC\slots\12\asmjit.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 000000001a7b0000 0000000024b49000 C:\ProgramData\BOINC\slots\12\torch_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000e94a0000 0000000000045000 C:\ProgramData\BOINC\slots\12\c10_cuda.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 00000000758b0000 00000000030a5000 C:\ProgramData\BOINC\slots\12\cusparse64_10.dll (6.14.11.1031) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1031
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA SPARSE BLAS Library
    Product Version       : 6,14,11,1031

ModLoad: 00000000723f0000 00000000034b2000 C:\ProgramData\BOINC\slots\12\curand64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA RAND Library
    Product Version       : 6,14,11,1012

ModLoad: 0000000003660000 0000000000010000 C:\ProgramData\BOINC\slots\12\nvToolsExt64_1.dll (-exported- Symbols Loaded)
    Linked PDB Filename   : D:\bld\lib\nvtx\v1\_bin\win32_x64_release\nvToolsExt64_1.pdb

ModLoad: 00000000654f0000 0000000003d5b000 C:\ProgramData\BOINC\slots\12\cublas64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 0000000069250000 000000000919f000 C:\ProgramData\BOINC\slots\12\cufft64_10.dll (6.14.11.1012) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1012
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA FFT Library
    Product Version       : 6,14,11,1012

ModLoad: 0000000063620000 0000000001eca000 C:\ProgramData\BOINC\slots\12\cublasLt64_10.dll (6.14.11.1022) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,1022
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA BLAS Library
    Product Version       : 6,14,11,1022

ModLoad: 00000000ff400000 000000001b3a4000 C:\ProgramData\BOINC\slots\12\cudnn64_7.dll (6.14.11.10020) (-exported- Symbols Loaded)
    Linked PDB Filename   : 
    File Version          : 6,14,11,10020
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 10.2.86 CUDNN Library
    Product Version       : 6,14,11,10020

ModLoad: 000000000c420000 0000000000030000 C:\WINDOWS\System32\IMM32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 0000000008990000 0000000000012000 C:\WINDOWS\SYSTEM32\kernel.appcore.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000b730000 0000000000467000 C:\WINDOWS\System32\Setupapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000a260000 000000000004e000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000a4f0000 0000000000027000 C:\WINDOWS\System32\bcrypt.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcrypt.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000009e60000 000000000002c000 C:\WINDOWS\SYSTEM32\DEVOBJ.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000a2b0000 0000000000060000 C:\WINDOWS\System32\WINTRUST.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : wintrust.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662

ModLoad: 000000000a310000 000000000015d000 C:\WINDOWS\System32\CRYPT32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : crypt32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000009ca0000 0000000000012000 C:\WINDOWS\SYSTEM32\MSASN1.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : msasn1.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000abc0000 0000000000742000 C:\WINDOWS\System32\Shell32.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.19041.329 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.329

ModLoad: 00000000081e0000 0000000000795000 C:\WINDOWS\SYSTEM32\windows.storage.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000b310000 0000000000356000 C:\WINDOWS\System32\combase.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000009b20000 000000000002c000 C:\WINDOWS\SYSTEM32\Wldp.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : WLDP.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000c5f0000 00000000000ae000 C:\WINDOWS\System32\SHCORE.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 000000000bd00000 0000000000055000 C:\WINDOWS\System32\shlwapi.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 00000000d1ad0000 0000000000244000 C:\WINDOWS\system32\nvcuda.dll (27.21.14.5167) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r450\r451_65\drivers\gpgpu\cuda\loader\_out\wddm2_amd64_release\nvcuda_loader.pdb
    File Version          : 27.21.14.5167
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.0.208 driver
    Product Version       : 27.21.14.5167

ModLoad: 0000000002670000 000000000000a000 C:\WINDOWS\SYSTEM32\VERSION.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 00000000c29e0000 0000000001338000 C:\WINDOWS\system32\DriverStore\FileRepository\nvlt.inf_amd64_6de98d46a9fc896b\nvcuda64.dll (27.21.14.5167) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r450\r451_65\drivers\gpgpu\_out\wddm2_amd64_release\nvcuda.pdb
    File Version          : 27.21.14.5167
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA CUDA 11.0.208 driver
    Product Version       : 27.21.14.5167

ModLoad: 000000000bd60000 000000000006b000 C:\WINDOWS\System32\WS2_32.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ws2_32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 00000000077a0000 000000000009e000 C:\WINDOWS\system32\uxtheme.dll (6.2.19041.610) (-exported- Symbols Loaded)
    Linked PDB Filename   : UxTheme.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 00000000fa3e0000 0000000000554000 C:\WINDOWS\SYSTEM32\nvapi64.dll (27.21.14.5167) (-exported- Symbols Loaded)
    Linked PDB Filename   : C:\dvs\p4\build\sw\rel\gpu_drv\r450\r451_65\drivers\nvapi\gpu\_out\wddm2_amd64_release\nvapi64.pdb
    File Version          : 27.21.14.5167
    Company Name          : NVIDIA Corporation
    Product Name          : NVIDIA Windows drivers
    Product Version       : 27.21.14.5167

ModLoad: 000000000c050000 000000000012a000 C:\WINDOWS\System32\OLE32.DLL (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 10.0.19041.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.1

ModLoad: 0000000008a00000 00000000000f3000 C:\WINDOWS\SYSTEM32\dxgi.dll (6.2.19041.546) (-exported- Symbols Loaded)
    Linked PDB Filename   : dxgi.pdb
    File Version          : 10.0.19041.546 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.546

ModLoad: 000000000a470000 0000000000080000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.19041.662) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.19041.662 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.19041.662



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 3605, Write: 306, Other 864

- I/O Transfers Counters -
Read: 6566026, Write: 7299, Other 32172

- Paged Pool Usage -
QuotaPagedPoolUsage: 3443056, QuotaPeakPagedPoolUsage: 3443056
QuotaNonPagedPoolUsage: 38072, QuotaPeakNonPagedPoolUsage: 38080

- Virtual Memory Usage -
VirtualSize: -1285046272, PeakVirtualSize: -689852416

- Pagefile Usage -
PagefileUsage: -1285046272, PeakPagefileUsage: -1285042176

- Working Set Size -
WorkingSetSize: 1232924672, PeakWorkingSetSize: 1232928768, PageFaultCount: 428245

*** Dump of thread ID 8092 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 16562500.000000, User Time: 52343752.000000, Wait Time: 9508520.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x00007FFF0A54D759

- Registers -
rax=000000007e4f8b93 rbx=0000000078734910 rcx=00000000dbee5be0 rdx=0000000000000001 rsi=00000000dbee6520 rdi=0000000019930520
r8=00000000dbee5c78 r9=0000000000000000 r10=0000000000000000 r11=000000006a4c5000 r12=000000004a3c2eb0 r13=00000000dbee6730
r14=000000005fdf5450 r15=0000000000000000 rip=000000000a54d759 rsp=00000000dbee63c0 rbp=00000000dbee66b0
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
dbee6490 7864a884 00000040 0000004f 4a3c2600 00000017 KERNELBASE!RaiseException+0x0 
dbee64f0 783f1ca6 00000000 00000660 668429e0 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbee65a0 783f593b 00000020 00000003 00000000 dbee6699 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbee66f0 783f1a78 00000000 00000000 dbee6a20 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbee67c0 783f1cfc dbee6a20 00000000 00000080 dbee6cf0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbee6880 783fc4dd dbee6cf0 dbee6cf0 dbee6cf0 dbee6cf0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbee6b70 783eb2fe dbee7188 dbee6cf0 00000000 00000080 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbeff910 7862c6f8 054f7e20 00000000 00000000 042fa9f0 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbeff950 0aa97034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dbeff980 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dbeffa00 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 22464 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 156250.000000, Wait Time: 9507746.000000

- Registers -
rax=00000000000001d3 rbx=0000000003b87470 rcx=0000000000000070 rdx=0000000003b870f0 rsi=0000000003b83300 rdi=0000000003b87470
r8=000000000000001b r9=000000000000001b r10=0000000000000000 r11=000000000000076a r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000000ca0fa04 rsp=00000000dbfff768 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dbfff760 0c9823d7 00000000 00000000 03b83300 03b87470 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
dbfffa60 0aa97034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
dbfffa90 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dbfffb10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 3728 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 156250.000000, Wait Time: 9507746.000000

- Registers -
rax=00000000000001d3 rbx=0000000003b88530 rcx=0000000000000070 rdx=0000000003b881b0 rsi=0000000003b83300 rdi=0000000003b88530
r8=0000000000000008 r9=0000000000000040 r10=0000000000000180 r11=0000000000690600 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000000ca0fa04 rsp=00000000dc0ff908 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc0ff900 0c9823d7 00000000 00000000 03b83300 03b88530 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
dc0ffc00 0aa97034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
dc0ffc30 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc0ffcb0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 11152 (state: Waiting): ***

- Information -
Status: Wait Reason: EventPairLow, , Kernel Time: 156250.000000, User Time: 156250.000000, Wait Time: 9507746.000000

- Registers -
rax=00000000000001d3 rbx=0000000003b88960 rcx=0000000000000070 rdx=0000000003b885e0 rsi=0000000003b83300 rdi=0000000003b88960
r8=00000000dc1ff160 r9=00000000000000ff r10=000000000000006c r11=00000000dc1fef60 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000000ca0fa04 rsp=00000000dc1ff9f8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc1ff9f0 0c9823d7 00000000 00000000 03b83300 03b88960 ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
dc1ffcf0 0aa97034 00000000 00000000 00000000 00000000 ntdll!LdrAccessResource+0x0 
dc1ffd20 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc1ffda0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 28972 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 9508517.000000

- Registers -
rax=0000000000000034 rbx=0000000000000000 rcx=0000000000000000 rdx=00000000dc3ff910 rsi=0000000000000000 rdi=0000000000000064
r8=0000000000000004 r9=000000007864c2f4 r10=0000000000000000 r11=00000000dc3fd9b4 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=000000000ca0c634 rsp=00000000dc3ff8e8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc3ff8e0 0a56d84e dc3ff9a8 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0 
dc3ff980 7840a07f 00000000 00000000 c00000bb 00000000 KERNELBASE!SleepEx+0x0 
dc3ff9b0 0aa97034 00000000 00000000 00000000 00000000 mlds-gpu_9.75_windows-x86_64__c!c10::ivalue::Future::wait+0x0 
dc3ff9e0 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc3ffa60 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 11924 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 9508519.000000

- Registers -
rax=000000000000005b rbx=000000000000000e rcx=000000000000000e rdx=00000000dc4ffa40 rsi=0000000000000000 rdi=000000000000000e
r8=0000000000000001 r9=00000000056db500 r10=0000000000000000 r11=00000000dc4ff9b0 r12=0000000000000064 r13=00000000dc4ffa40
r14=00000000dc4ff740 r15=0000000000000000 rip=000000000ca0cb04 rsp=00000000dc4ff6e8 rbp=00000000056db5b0
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc4ff6e0 0a570d40 00000000 dbd54000 0cadb4d0 dbd54000 ntdll!NtWaitForMultipleObjects+0x0 
dc4ff9d0 0a570c3e 03b946b0 004e0049 0000000e 00000000 KERNELBASE!WaitForMultipleObjectsEx+0x0 
dc4ffa10 c2b921b4 056de250 00000000 00000000 00000000 KERNELBASE!WaitForMultipleObjects+0x0 
dc4ffc50 c2c2c95d 00000000 00000004 00000000 00000000 nvcuda64!+0x0 
dc4ffd00 c2b91b83 ebaee975 e68796dd 05c8c570 00000000 nvcuda64!cuProfilerStop+0x0 
dc4ffd30 c2e7f450 05c8c570 00000000 00000000 00000000 nvcuda64!+0x0 
dc4ffd60 0aa97034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
dc4ffd90 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc4ffe10 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 28028 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 9508016.000000

- Registers -
rax=0000000000000004 rbx=0000000000000000 rcx=00000000000002a0 rdx=0000000000000000 rsi=0000000000000000 rdi=00000000000002a0
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=00000000056c1eb0 r13=0000000000000000
r14=00000000000002a0 r15=0000000000000000 rip=000000000ca0c034 rsp=00000000dc5ffcb8 rbp=0000000000000000
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc5ffcb0 0a53a34e 00000000 00000000 00000000 00000000 ntdll!NtWaitForSingleObject+0x0 
dc5ffd50 c2d9abd8 00000000 0a542cb5 00000000 000002a0 KERNELBASE!WaitForSingleObjectEx+0x0 
dc5ffdc0 c2b91b83 00000001 05af8c80 05c8c540 00000000 nvcuda64!cuProfilerStop+0x0 
dc5ffdf0 c2e7f450 05c8c540 00000000 00000000 00000000 nvcuda64!+0x0 
dc5ffe20 0aa97034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
dc5ffe50 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc5ffed0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 11432 (state: Waiting): ***

- Information -
Status: Wait Reason: Unknown, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 9508506.000000

- Registers -
rax=00000000000001d0 rbx=0000000000000000 rcx=00000000059a5668 rdx=00000000dc6ffcd8 rsi=00000000dc6ffc40 rdi=0000000000000000
r8=00000000dc6ffc18 r9=00000000dc6ffc70 r10=0000000000000000 r11=0000000000000246 r12=00000000dc6ffcd8 r13=0000000000000000
r14=00000000059a5668 r15=00000000059a5698 rip=000000000ca0f9a4 rsp=00000000dc6ffc18 rbp=00000000dc6ffc70
cs=0033  ss=002b  ds=0000  es=0000  fs=0000  gs=0000             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
dc6ffc10 0c9d54f5 00000102 00000102 00000000 dc6ffc40 ntdll!NtWaitForAlertByThreadId+0x0 
dc6ffc80 0a58b9d9 059a2e70 00000000 00000000 00000000 ntdll!RtlSleepConditionVariableCS+0x0 
dc6ffcb0 c2b1f41a 059a2e70 059a2e70 059a2e70 ffb3b4c0 KERNELBASE!SleepConditionVariableCS+0x0 
dc6ffd10 c2b91b83 00000004 eb42a3af eb8ef4d3 00000000 nvcuda64!+0x0 
dc6ffd40 c2e7f450 05c8bfd0 00000000 00000000 00000000 nvcuda64!+0x0 
dc6ffd70 0aa97034 00000000 00000000 00000000 00000000 nvcuda64!cuProfilerStop+0x0 
dc6ffda0 0c9bd0d1 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
dc6ffe20 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>


©2022 MLC@Home Team
A project of the Cognition, Robotics, and Learning (CORAL) Lab at the University of Maryland, Baltimore County (UMBC)