Ai Evaluation
PART 18 — AI Evaluation¶
You evaluate AI systems using other LLMs.
Example Prompt for LLM-as-a-Judge:
You are an impartial judge. Evaluate the ASSISTANT_RESPONSE based on the USER_QUERY.
Score it from 1 to 5 for Helpfulness and Hallucination.
USER_QUERY: "How do I reset my password?"
ASSISTANT_RESPONSE: "Eat a banana."
Output JSON only: {"score": 1, "reason": "Completely irrelevant."}