Skip to content

Llms Large Language Models

PART 8 — LLMs (Large Language Models)

  • LLM: A massive Transformer model trained on internet-scale data.
  • Autoregressive generation: The model generates the next word, appends it to the input, and then runs the whole process again to guess the word after that.
  • Prompt: The input text.
  • Completion: The output text.
  • Temperature: A setting controlling randomness. Temperature 0 = deterministic (always picks highest probability word). High temperature = creative/random.

The Loop: 

"The cat sat on the"  → LLM → "mat"
"The cat sat on the mat" → LLM → "."

Why does an HLD engineer care?

Generation is strictly sequential. You cannot parallelize the generation of the 5th token and the 10th token because the 10th token depends on the 5th. This autoregressive bottleneck is the primary challenge in LLM system design.