to learn
My to-do list of learning.
These are the things that I am interested in learning.
- Nested Learning — ~30–45 min -[x] Jeff Dean on performance — ~2–3 h
- Non zero sum games — ~10–15 h (whole site, ~40 short posts)
courses and competitions
ml (~85–145 h)
- Karpathy Zero to Hero — ~40–60 h coding along (~16 h watch-only)
- RL on wordle — ~1 h read; ~6–10 h to reproduce
- RL on 2048 — ~3–5 h (mostly training runs)
- nanochat — ~15–30 h (read code + run speedrun + tinker)
- RL in Huggingface — ~25–40 h (deep RL course, ~11 units)
cuda (~110–200 h)
- leetgpu — ~30–60 h (50+ problems)
- tensara — ~20–40 h (open-ended perf tuning)
- [programming massively parallel processors] — ~60–100 h (580 pp + exercises)
cuda misc (~35–55 h)
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance — ~2–3 h
- Outperforming cuBLAS on H100: a Worklog — ~1.5–2.5 h
- Defeating Nondeterminism in LLM Inference: — ~1–1.5 h
- Making Deep Learning go Brrrr From First Principles: — ~1–1.5 h
- Transformer Inference Arithmetic: — ~2–3 h (with the end exercises)
- Domain specific architectures for AI inference — ~1.5–2.5 h
- A postmortem of three recent issues: — ~45 min
- [How To Scale Your Model:] (http://jax-ml.github.io/scaling-book/ ) — ~7–10 h read; ~25–40 h with exercises
- [The Ultra-Scale Playbook:] (http://huggingface.co/spaces/nanotron/ultrascale-playbook ) — ~15–25 h
- The Case for Co-Designing Model Architectures with Hardware: — ~1.5–2.5 h
shaders (~16–32 h)
- The book of Shaders — ~10–20 h (with exercises)
- making software — ~6–12 h (early access, still growing)
others (~800–1,200 h)
- Forking chrome to render in the terminal — ~30–60 min (read)
- Wordle-RK Training Language Model to play wordle — see ml above
- Build your own c compiler — ~120–200 h (792 pp, full implementation)
- Beej’s guide to networking — ~8–15 h
- Mamba the easy way — ~30–60 min
- Build your own redis — ~40–60 h
- Distributed system from maelstorm — ~30–50 h
- Gossip glomers, fly.io — ~20–40 h
- Advanced compilers: self-guided online course (cornell) — ~60–100 h
- pintos: a simple operating system — ~150–250 h (full-semester course, built for teams)
- let’s write a browser — ~3–6 h
- write your own git — ~10–15 h
- Explained from first principles — ~25–40 h (3 articles, ~250k words)
- Programmer’s introduction to Mathematics — ~60–100 h (with exercises)
- How does a database work, database internals — ~10–20 h (unfinished tutorial)
- makefiles — ~2–4 h
- Embedded system — ~60–100 h (labs assume TM4C board)
- Ray Tracing — ~50–80 h (all 3 books)
- Nand2Tetris — ~100–150 h (both parts)
- Stanford crypto — ~50–80 h
- associated book — ~70–100 h on its own (818 pp)
- GDPR ✅
books to read (~140–200 h)
- How to Scale Your Model — ~7–10 h
- Harry Potter and the Methods of Rationality — ~35–45 h (~660k words)
- Algorithms for decision making — ~40–60 h (~700 pp, dense)
- Linux insides — ~22–30 h (~69 chapters)
- Google SRE — ~25–35 h (SRE book + workbook)
- compiler books — ~8–12 h (~250 pp)
- Git from the ground up — ~1.5–2.5 h
- post quantum cryptography — ~3–4 h (72 pp)
blogs to read (~25–30 h)
- Larry Sanger on baby reading — ~2.5–3 h (links a 140-page essay)
- How to win friends and influence people unabridged — ~4.5–6 h (it’s the full book)
- Understanding Apache Fluss — ~45 min
- Peter Thiel’s CS183 — ~6–8 h (19 essays)
- interview with shang Yi Chiang, R&D head at TSMC — ~1 h (49 pp transcript)
- how to do Distributed locking — ~20 min
- There’s no getting around you’re building a distributed system — ~25 min
- Getting real about distributed system reliability — ~15 min
- Coding for SSDs — ~2–3 h (6 parts)
- http crash course nobody asked for — ~1.5–2.5 h
- lessons from running codepen — ~1 h (58 min podcast)
- Everything you ever wanted to know about terminals — ~35 min
- Rqlite design — ~15 min (link 404s now; docs moved to rqlite.io)
- Taxonomy of Moats — ~30 min
- Leverage Points — ~40 min
- All Markets Are Not Created Equal: 10 Factors To Consider When Evaluating Digital Marketplaces — ~15 min
- What is amazon — ~1 h (~12k words)
- Bill gates interview 1992 PDC 1992 — ~1 h (57 min video)
- https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript — ~30 min talk
- Adverserial collaboration - Daniel Kahneman — ~25–30 min
to build (~100–200 h)
- RL environment for agents — ~40–80 h
- LMDB rust — ~60–120 h
privacy (~3–5 h)
- Science, Privacy, and Freedom: Issues and Proposals for the 1970’s. Part I–The Current Impact of Surveillance on Privacy, Alan F. Westin Columbia Law Review, Vol. 66, No. 6 (Jun., 1966), pp. 1003-1050 (48 pages) https://www.jstor.org/stable/1120997 — ~3–5 h
games: (~30–60 min)
https://flexboxfroggy.com/ — ~30–60 min (24 levels)
projects to go through (~110–200 h)
- etcd — ~30–50 h (raft/mvcc/lease/watch, architecture-level)
- cockroachDB — ~80–150 h (architecture-level only; multi-million LOC)
courses and others
dolby webcast
browsers (~80–120 h)
- https://browser.engineering/ — ~80–120 h (16 chapters + exercises)
a quote i like
I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. ― Frank Herbert, Dune