Good introduction! Building pytorch-lite using python and numpy is the way to go.

Free book: https://zekcrates.quarto.pub/deep-learning-library/

Ml by hand : https://github.com/workofart/ml-by-hand

Micrograd: https://github.com/karpathy/micrograd