I read in a news article, that INT8 commands are supported.
Do know that a single RTX 3090 can push around 250-300 Tops on that data.
They're very well optimized for INT8, and full or half precision.
If you ever wish to include GPUs in data crunching, the RTX 3000 series will do in a day what would take normal PCs months or even years!

Defintely interesting to think about. Wouldn't the multitude of precisions that the RTX2000/3000 series Tensors cores have to offer, such as the TF32, FP16-, int8 und INT4, potentially be of interest for future app implementations? Don't know how feasible it would be or whether it is at all, but certainly Tensor cores seem to target the very use case that MLC is interested in at the moment.

The CUDA-X AI ecosystem seems to offer a vast library with many modules targeted at specific AI/ML use cases such as DL-train and DL-inference. Or are we already exploiting this performance potential of these precision types with the cudNN library and the other relevant libraries of the Cuda-x ecosystem (cuBLAS, cuFFT)?

And somehow I can't figure out what version of cudNN MLC deployed on my system, but as far as I can tell, it seems to work with version 7 while the latest cudNN version is version 8 that was released on 11/25/20. From what I read on here, it seems to offer further potential to improve performance. NVIDIA cuDNN 8