Karpathy micrograd
creating karpathy's micrograd from scratch
February 8,2025
Went through karpathy’s incredible micrograd tutorial.
His explanation and the code was incredibly intuitive.
Autograd libraries build a network graph of a given equation with each operation being a node and then apply chain rule to find the local gradient. This is useful because it enables you to calculate back propagation easily as well as locally.
The network graph then undergoes topological sort to make it easy to go from front to back.
It only takes 100 lines of code to implement autograd from scratch