Learnings For TPU Greatness: A Series

TPUs: a series introduced

I’ve become fascinated with TPUs.

From the business use case, the foresight by Google, the technical implementation, to the software stack. Everything.

I’m going to write a blog series on Tensor Processing Units. I want my endeavor to ultimately end up at the place of total understanding when staring at ML specific JAX or Pallas code.

Ultimately, I want to do the following:

• write code that could run on a TPU (a kernel)
• analyze system performance of those kernels
• debug kernel performance at the most basic level

I’ll try to cover from soup to nuts everything that I learn about TPUs, the code that runs on them, the systems that interact with them, and any thing else I may encounter.

let’s go!

btw, i’ll update this post with the posts of the series so they are easier to find.