Skip to content

Neural Networks

PART 4 — Neural Networks

Under the hood, a neural network is just massive for loops performing Matrix Multiplication (MatMul).

  • Equation: y = Activation(W * x + b) where W is a massive matrix of weights, and x is your input vector.
  • Parallelism: CPUs are terrible at doing 10 million multiplications simultaneously. GPUs excel at this because they have thousands of ALUs (Arithmetic Logic Units).

SDE Takeaway: Inference is purely linear algebra. The bottleneck in AI serving is usually getting the matrix W from the GPU's memory (VRAM) into the GPU's compute cores fast enough.