Multi Head Attention

PART 6 — Multi-Head Attention

Instead of running one large Attention operation, we split it into smaller, parallel operations called "Heads".

SDE Takeaway: Think of this as sharding a database query. Head 1 looks for grammar, Head 2 looks for sentiment, Head 3 looks for subject-verb agreement. They run concurrently on the GPU and their outputs are concatenated. This allows the model to capture different sub-spaces of meaning simultaneously.

---