These terms usually confuse the beginners. Let's explain them in a simple and clear way.
CUDA vs cuDNN:
NVIDIA CUDA Deep Neural Network (cuDNN) is a GPU-accelerated library of primitives for deep neural networks.
The cuDNN library is a library optimized for CUDA containing GPU implementations. Think of cuDNN as a library for Deep Learning using CUDA and CUDA as a way to talk to the GPU.
In summary, CUDA provides the programming interface and runtime environment for general-purpose computing on a GPU, while cuDNN is a library that specifically optimizes deep learning operations on a GPU.
tiny-cuda-nn:
tiny-cuda-nn comes witha PyTorch extension that allows using the fast MLPs and input encodings from within a Python context. These bindings can be significantly faster than full Python implementations; in particular for the multi-resolution hash encoding.