HotCSE Seminar: Vijay Thakkar

Name: School of CSE Ph.D. Student Vijay Thakkar

Date: Wednesday, Feb. 12, 2025, at 12:00 p.m.

Location: TBD (Google Maps link)

Lunch provided!

Title: CUTLASS 3.x: A Microkernel Abstraction for GPU Linear Algebra

Abstract: We have created a micro-kernel abstraction for GPUs robust enough to uniformly represent the tensor core and data movement operations from NVIDIA GPU architectures spanning Maxwell all the way to Blackwell. In this talk, we describe the novel two level microkernel abstraction that allows for this generality. Spatial microkernels described by CuTe layouts and layout algebra allow us to uniformly represent GPU architecture specific operations regardless of the threads and data they operate upon. CUTLASS 3.x's temporal microkernels allow for a hierarchical organization of synchronization and programmer managed on-chip memory abstracting away architecture specific synchronization. Both of these together result in a robust programming model that is performant, extensible, and simply a joy to use.

Bio: Vijay is a senior architect in the fast kernels team where he has worked on CUTLASS 3.0 project since its inception as one of its leads. For the past three years he has focused on the development of Blackwell kernels via CuTe and CUTLASS's programming model and PTX ISA which just released as CUTLASS 3.8 and CUDA 12.8 respectively. He broadly collaborates with the GPU architecture, compiler, and programming model teams on software/hardware codesign for tensor cores and other DL features of datacenter GPUs. At Georgia Tech, he is registered as a part time Ph.D. student in Rich Vuduc's HPC garage lab, where he hopes to defend his Ph.D. some day.

About HotCSE

HotCSE is an academic seminar series to bring Ph.D. students in Computational Science and Engineering together to discuss interesting topics. The topics consist of high-performance computing, machine learning, data analysis, simulation, computational sustainability, medical informatics, etc.

The talks have always been enjoyable and have ranged from quite informal to formal conference style talks. Either chalks or slides can be used to help people understand your talk. It is also a great forum to practice conference talks and bounce around new ideas.

Currently the talks are sponsored by the School of Computational Science and Engineering. The goal of CSE is slightly broader than that of these talks - we want to bring more people outside CSE to discuss their related work here.