Llms Large Language Models

PART 8 — LLMs (Large Language Models)

An LLM is just a scaled-up Decoder-only Transformer.

The Algorithm (Autoregressive Generation):

output_tokens = []
while True:
    input_array = prompt_tokens + output_tokens
    probabilities = model.forward(input_array)
    next_token = sample(probabilities, temperature)

    if next_token == STOP_TOKEN:
        break

    output_tokens.append(next_token)

SDE Takeaway: Text generation is a while loop. Because of this, it is fundamentally a synchronous, blocking operation that cannot be easily parallelized across time.

---