The Curious Engineer

All blogs / AI / ML

AI by Hand ✍️

byhand.ai

Math, Algorithms, Architectures, by hand. Click to read AI by Hand ✍️, by Prof. Tom Yeh, a Substack publication with tens of thousands of subscribers.

AI / MLMath

’23’24’25’26
Posts
346 from 15 Dec 2023 to 5 Oct 2026
Posting
about daily (218 in the last year)
Kind
personal blog
Feed
https://www.byhand.ai/feed

2026 200

  1. Jev to Kernels by Hand ~ Dr. Dandekar, PhD. MIT
  2. RNNs by Hand, in Excel
  3. Small Language Model
  4. Library sheets rnn loss
  5. Autoregressive
  6. Library sheets rnn tokens
  7. Seq2Seq
  8. Library sheets rnn depth
  9. Outputs
  10. Library sheets rnn size
  11. Recurrence
  12. AI by Hand ✍️ Yantra Jnana Award
  13. Counting AI by Hand ✍️ ~ Recordings
  14. Loop Engineering (coming soon, Sep 3, 2026)
  15. Agent Tokens (Jul 14, 2026)
  16. Transformer Ops (Jun 23, 2026)
  17. Kimi 3 ~ New Seminar Recording
  18. Context Problems
  19. Cost Forecast
  20. Quadratic Cost
  21. Context Tax
  22. Retrieval Cost
  23. Stateless API
  24. Verbatim Tail
  25. The Compaction Threshold
  26. Compacted Retrieval
  27. The Compaction Bill
  28. Compaction
  29. Truncation
  30. Window Sizing
  31. Search Rounds
  32. Retrieval per Turn
  33. Retrieval Footprint
  34. Turn Budget
  35. Remaining Space
  36. Growth per Turn
  37. Window Occupancy
  38. The Context Window
  39. Full Fine-Tuning vs LoRA
  40. New Series! Multi-Layer Perceptron (MLP)
  41. Library models mlp heads
  42. Two-Tower MLP
  43. Library models mlp gan
  44. Shrink then Expand
  45. Expand then Shrink
  46. Autoencoder
  47. Decoder
  48. Encoder
  49. Binary Classifier
  50. Library models mlp deep
  51. Library models mlp wide
  52. Perceptron
  53. Deep Learning by Hand ✍️ 2026
  54. Autoencoder
  55. Library videos deeplearning 2026 svm
  56. Library videos deeplearning 2026 mlp
  57. Neural Network
  58. Linear Algebra
  59. Frontier AI Software Engineering Seminar #7 ~ Richard Hsu, Nuro + previous recordings
  60. Qwen 3.6 (May 21, 2026)
  61. Calculating AI by Hand ✍️
  62. Seminar Series 2026
  63. Seminar Series 2025
  64. Matrix Multiplication
  65. Library
  66. Frontier AI Drawings
  67. Token Problems
  68. Usage Audit
  69. Rate Cards
  70. Output Bound
  71. Two Speeds
  72. Token Allowance
  73. Budget Ceiling
  74. Fixed Overhead
  75. Standing Instructions
  76. Usage Forecast
  77. Token Mix
  78. Words to Cents
  79. Asymmetric Pricing
  80. The Billing Line
  81. Prompt and Completion
  82. Token Pricing
  83. Headroom
  84. Token Ratio
  85. Non-Word Tokens
  86. Subword Tokens
  87. Tokenization
  88. Attention
  89. Grouped-Query Attention
  90. Multi-Query Attention
  91. Fused QKV (Multi-Head)
  92. Multi-Head Attention
  93. Self Attention (Shared KV)
  94. Cross Attention
  95. Self Attention
  96. Attention Computation
  97. QKV Projection
  98. New Seminar Series: Frontier in AI Software Engineering
  99. Gemma 4 (May 7, 2026)
  100. Activation chapter, complete: 12 interactive lessons
  101. Library math activation swiglu
  102. Library math activation glu
  103. Softplus
  104. Log-Sum-Exp
  105. Library math activation gelu
  106. Library math activation silu
  107. Library math activation elu
  108. Library math activation tanh
  109. Activation
  110. Sigmoid
  111. Softmax
  112. Leaky ReLU
  113. Library math activation relu
  114. Seminar next week ~ Google's Gemma 4
  115. Single vs Multi-Head Attention
  116. Self Attention vs Cross Attention
  117. Fine-Tuning
  118. Library models fine tuning lora intro
  119. Adapter Layers
  120. Feature Extraction + Head
  121. Linear Probe
  122. Freezing Layers
  123. Full Fine-Tuning
  124. Pretrain vs Fine-Tune
  125. Weight Update
  126. The Library is Live! ~ check your access
  127. Two New Interactive Workbooks ~ Matmul & Linear Layer
  128. My Deep Learning Math Workbook — Now Interactive
  129. AI by Hand Library ~ Attention, MHA, MQA, GQA
  130. Self Attention Flow ~ New Release!
  131. Matmul Flow ~ New Visualization Tool
  132. SwiGLU: The Activation Function Behind Frontier AI
  133. AI Isn't Human
  134. PPO → DPO → GRPO → Rubrics (Mar 2, 2026)
  135. Entropy
  136. GAN ~ draw by hand
  137. PPO Clipped Policy Loss Algorithm
  138. Essential AI Math #16 to #20
  139. OpenClaw Seminar (Feb 23, 2026)
  140. GLU (Gated Linear Unit)
  141. Tanh
  142. GELU (Gaussian Error Linear Unit)
  143. Swish (SiLU)
  144. ELU (Exponential Linear Unit)
  145. Essential AI Math #11 to #15
  146. CNN ~ draw by hand ✍️
  147. KL Divergence
  148. Binary Cross Entropy Loss
  149. L2 Loss
  150. Transformer: Six Levels of Understanding (Feb 13, 2026)
  151. L2 Norm
  152. Cross Entropy Loss
  153. Chapter 1 - Foundation
  154. First Seven AI by Hand Seminars - Roundup
  155. Debouncing in OpenClaw - A Study Guide
  156. First 10 Essential AI Math Blueprints
  157. RMS Normalization
  158. Layer Normalization
  159. Batch Normalization
  160. OpenClaw
  161. Sinusoidal Positional Encoding
  162. Leaky ReLU
  163. Relu
  164. Double Sum
  165. Negative Log-Likelihood
  166. Meta Superintelligence Labs vs Facebook AI Research (Jan 30, 2026)
  167. Building Multi-Agents from Paper 📝 by Hand ✍️
  168. Sigmoid
  169. Softmax
  170. 9 AI Eval Formulas You Must Know (Jan 24, 2026)
  171. Google Ironwood TPU: From Bits to HBM
  172. The Architect: New Open Source Tool from the AI by Hand Lab
  173. Google Ironwood TPU: From Bits to HBM (Jan 19, 2026)
  174. Module 4: Tool Use
  175. Module 3: Vector Database
  176. Module 2: LLM to RAG to Agents
  177. Tiled Algorithm
  178. Linear Equations
  179. Inverse
  180. Transpose
  181. Chapter 12 chain
  182. Distributive
  183. Associative
  184. Not Commutative
  185. Combine Columns
  186. Combine Rows
  187. Chapter 6 shift
  188. Chapter 5 scale
  189. Identity
  190. Chapter 3 shape
  191. Complexity
  192. Calculate
  193. Module 1: Foundation
  194. Attention (Jan 15, 2026)
  195. Coding on Paper with Tivadar Danka
  196. How Small Models Learn Tool Use (Jan 11, 2026)
  197. Generative AI (Jan 11, 2026)
  198. Manifold-Constrained Hyper Connections (mHC) from DeepSeek (Jan 9, 2026)
  199. AI by Hand ✍️ Academy
  200. My 2026 resolution: a weekly AI seminar — for everyone

2025 70

  1. Gated Attention ~ NeurIPS 2025 Best Paper (Dec 18, 2025)
  2. Agentic AI Problem Set
  3. Mistral 3
  4. Mistral 3 Live! (12/9)
  5. Can AI Find its Own Poetic Voice?
  6. Sharding
  7. AI by Hand ✍️ Academy
  8. Deep Learning Math Workbook
  9. MCP Workbook
  10. RLHF PPO Blueprint
  11. AI by Hand ✍️ Academy
  12. AI by Hand ✍️ 2026 Desk Calendar
  13. ResNet, ViT, Qwen3 Blueprint (Google Bonsai)
  14. Agent Hallucination (Nov 12, 2025)
  15. Einsum: Outer Product to GQA
  16. LLM vs RAG vs Agent Workbook
  17. LinearLayer+LoRA: PyTorch, BLAS, CUDA
  18. GPU by Hand ✍️ Together (Nov 2, 2025)
  19. DeepSeek OCR
  20. DeepSeek Attention (DSA)
  21. GPU by Hand ✍️ Together
  22. Linear Regression Workbook
  23. Qwen 3
  24. AI book I love in October
  25. Finished the workbook? Claim your official AI by Hand ✍️ certificate
  26. RMS, Group, Layer, Batch Norm, Tensor Parallelism
  27. RoPE vs PE in QKV Self-Attention
  28. MLP Parallelism: Data, Context, Row, Column, Pipeline
  29. Paper Club, A2A Lecture, Intro to AI Agents Course
  30. Inference Batching, Request-vs-Token Level
  31. Next Tuesday: New AI by Hand ✍️ Mini Course by Mohsena Begins
  32. EmbeddingGemma, MRL, InfoNCE, Embed vs. Decode
  33. Intro to Gen AI Mini Course
  34. KV Cache, Prefill, Decode
  35. 18th AI by Hand ✍️ Workshops
  36. QLoRA, DoRA, BitFit, NF4 vs INT4
  37. This Week’s AI by Hand ✍️ Events
  38. 17th AI by Hand ✍️ Workshops
  39. Intro to Gen AI Mini Course
  40. LoRA, Fine-Tune, Pre-Train
  41. MXFP4, FP4, FP8
  42. Scaling LLMs by Hand ✍️ in JAX (Aug 13, 2025)
  43. 16th AI by Hand ✍️ Workshops
  44. MHA, MQA, GQA, MoE-A: More Attention!
  45. New GPT-OSS Trick to Ignore Tokens
  46. NN to Transformer by Hand ✍️
  47. Expert Choice Mixture of Experts (MoE)
  48. Special Lecture: From Neural Networks to Transformers
  49. 🔥 New: Frontier Subscription
  50. Batch Normalization by Hand ✍️
  51. Adding Two Arrays in GPU by Hand ✍️
  52. Context Engineering by Hand ✍️
  53. Matmul on GPU/TPU by Hand ✍️
  54. MCP by hand ✍️
  55. Transformer by hand ✍️
  56. CS Isn’t the Top Major Anymore. Guess What Is.
  57. AI by Hand ✍️ Workshop
  58. Math Workshop, Certificates
  59. LLaMA 1 → 2 → 3 → 4 (May 14, 2025)
  60. Deep Learning Math Workshop
  61. India Edition, ResNet, Fundamentals vs Tools
  62. Gradient, CNN, Evolution of Deep Learning
  63. GAN, C++, Love to Learn
  64. Wide, ML Interviews, Llama 4 Live
  65. Llama 4, Keep Up, Totoro
  66. Introduction to Agentic AI
  67. Draw DeepSeek (Feb 20, 2025)
  68. Computer Vision Lectures
  69. DeepSeek Lecture - Followup
  70. DeepSeek by Hand ✍️ in Excel

2024 73

  1. Xlstm
  2. Multihead Attention
  3. Temperature
  4. AI by Hand 2025 Calendar
  5. "Full-Stack" Transformer
  6. Building a Vector Database in Excel
  7. Livestream Today!
  8. 10K Subscribers! Live Stream (10/31)
  9. Beginner's Guide to Multi-Agents
  10. AlphaFold
  11. Geoff Hinton - Evolution of Deep Learning by Hand ✍️
  12. Query Rewriting RAG
  13. Backpropagation
  14. Beginner's Guide to Graph RAG
  15. BitNet by Hand ✍️
  16. OpenAI Strawberry 🍓
  17. Recurrent Neural Network
  18. Self Attention - Spreadsheet
  19. Beginner's Guide to RAG Evaluation
  20. Multi-Layer Perceptron (MLP) - Spreadsheet
  21. Vector Database - Spreadsheet
  22. Transformer - Spreadsheet
  23. Discrete Fourier Transform by Hand ✍️
  24. Superposition by Hand ✍️
  25. LSTM by Hand ✍️
  26. Beginner's Guide to RAG Agents
  27. Sparse Auto Encoder (SAE) by Hand ✍️
  28. Beginner's Guide to RAG
  29. Are Emergent Abilities of Large Language Models a Mirage?
  30. xLSTM: Extended Long Short-Term Memory
  31. Variational Auto Encoder (VAE) by Hand ✍️
  32. Build a Vector Computer with Superlinked
  33. A Note on LoRA
  34. LoRA: Low-Rank Adaptation of Large Language Models
  35. KAN: Kolmogorov-Arnold Networks
  36. Better & Faster Large Language Models via Multi-token Prediction
  37. SVM, Self Attention, Linear Layer, Workbook Videos
  38. W12 - Gradient
  39. LLaMA 3 Live Panel
  40. LLaMA 3, Sora, Artificial Neuron, RLHF, Transformer
  41. U-Net, Transformer, Activation, Backpropagation, MLP
  42. W11 - Softmax
  43. W10 - Wide
  44. W9 - Deep
  45. W8 - Hidden Layer
  46. W7 - Connection
  47. W6 - Batch
  48. Beginner's Guide to Vector Databases
  49. W5 - Artificial Neuron
  50. W4 - Activation
  51. W3 - Linear Layer
  52. W2 - Matrix Multiplication
  53. W1 - Dot Product
  54. Deep RNN by Hand ✍️
  55. SVM by Hand ✍️
  56. U-Net by Hand ✍️
  57. Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
  58. Switch Transformer by Hand ✍️
  59. Sora's Diffusion Transformer (DiT) by Hand ✍️
  60. Graph Convolutional Network (GCN) by Hand ✍️
  61. Residual Network by Hand ✍️
  62. CLIP by Hand ✍️
  63. Vector Database by Hand ✍️
  64. Autoencoder by Hand ✍️
  65. Dropout by Hand ✍️
  66. Self Attention by Hand ✍️
  67. Generative Adversarial Network (GAN) by Hand ✍️
  68. Batch Normalization by Hand ✍️
  69. Transformer by Hand ✍️
  70. Backpropagation by Hand ✍️
  71. Multi Layer Perceptron (MLP) by Hand ✍️
  72. Sampling a Sentence by Hand ✍️
  73. Matrix Multiplication by Hand ✍️

2023 3

  1. Mamba's S6 by Hand ✍️
  2. Recurrent Neural Network (RNN) by Hand ✍️
  3. Mixture of Experts (MoE) by Hand ✍️