CUDA Tile C++ is a novel C++ DSL and compiler that automatically parallelizes array operations across GPU threads while presenting a single logical thread to the programmer. In CUDA Tile C++, parallelism is expressed through SIMD-style operations on arrays of data called "tiles" while memory accesses are written using structured memory abstractions. The compiler analyzes the tile and memory operations to generate an equivalent multi-threaded program that leverages the advanced hardware capabilities of modern GPUs. This allows the user to write simple programs that exploit GPU parallelism without needing to manage low level hardware concurrency.
As GPU architectures evolve to meet the demands of high performance computing, the traditional multi-threaded GPU programming model becomes increasingly unwieldy. To achieve optimal performance, the GPU programmer must manage a growing variety of hardware features including shared memory, Tensor Memory Accelerators, and Tensor Cores. Traditional CUDA code is deeply coupled to the underlying hardware resulting in complex and non-portable algorithms. The CUDA Tile C++ compiler removes the need for coordinating these low level hardware details, freeing the user to reason about their algorithm rather than asynchronous code.
The tile programming model introduces three key abstractions:
- cuda::tiles::tile - a multi-dimensional array with value semantics whose operations are parallelized across the GPU
- cuda::tiles::tensor_span - a view into a multi-dimensional in-memory array with an interface similar to std::mdspan
- Tile Views - adapters that present a uniform "chunking" of a tensor_span which gives the compiler visibility into memory access patterns
In this talk, we will explore how these abstractions allow NVIDIA's Tile C++ compiler to schedule the program across hardware threads all while preserving the illusion of a single-threaded abstract machine to the user.
