Skip to content

Verifying AI output

~8 min

In short

An assistant's answer is a claim, not a result. You check it the same way you check anything else: run it, test it against inputs where you know the answer, read the official documentation, and confirm that every name and link exists. This page gives four checks and shows each one on a wrong answer that is easy to produce. The examples were run on Python 3.14.7 in the environment that wrote this page.

Check 1: run it on inputs you chose

Do not trust code because it looks tidy. Choose two or three inputs where you already know the answer, including one edge case such as an empty list, and run the code on them.

>>> "abc".reverse()
AttributeError: 'str' object has no attribute 'reverse'

Strings have no reverse method. A list does (my_list.reverse() changes the list in place and returns None). An assistant that mixes the two will write code that looks right and crashes at runtime. The fix is to read the error, then check the method in the docs.

Check 2: write a test with the answer you expect

A test states the expected result before you look at the code's output. If you let the assistant write the test, it may copy the code's behaviour instead of the correct behaviour.

>>> got = sorted([3, 1, 2])
>>> got
[1, 2, 3]
>>> got == [3, 1, 2]
False

If you had written assert sorted([3, 1, 2]) == [3, 1, 2] because that felt right, the test would fail, and the failure would tell you the expectation was wrong. A useful habit is to write the expected value on paper first, then run the code. For a real project, use a test framework such as pytest, documented at https://docs.pytest.org/.

Check 3: check the name in the official documentation

Assistants sometimes give functions that exist only in some versions of a library, or that never existed. Check two things: that the name exists, and in which version it was added.

>>> import itertools, math
>>> hasattr(itertools, "batched"), hasattr(math, "fma")
(True, True)

On this interpreter both names exist. Other versions may not have them. The Python docs at https://docs.python.org/3/library/itertools.html give version notes for functions that were added in later releases. Read the note for the version you run, and check your version with python3 --version. A name you cannot find in the docs is a warning sign, not proof that it does not exist, so check the docs and then test it.

Code from an assistant may import a package you have never heard of. A package name can be invented. The package-hallucination study found invented names in a large share of generated package recommendations (arXiv:2406.10279). Before you install anything, check that the name is on the registry and that it is the package you meant.

The PyPI registry has a JSON endpoint for each project. A check from the environment in which this page was written:

requests                  200   (project exists)
fastapi                   200   (project exists)
zzqx-nonexistent-pkg-7731 404   (no project with this name)

A 404 means the name is not on PyPI, so the assistant invented it or made a typo. A 200 does not mean the package is the one you want. Read its project page, check the maintainer, and compare it with the official docs of the library you are trying to use.

Links in a generated answer need the same check. A URL may look plausible and return 404. Run curl -sI <url> and look at the status code before you cite it.

Reading long answers and long context

Models can miss information placed in the middle of a long input. A study of long-context models found that performance is highest when the relevant information is at the start or end of the input and drops when it is in the middle (arXiv:2307.03172). When you paste a long file or log and ask a question, put the part you care about near the top, or quote it in your question, and check the answer against the source lines.

A short routine

  1. Write down what you expect the code to do, with two inputs.
  2. Run the code on those inputs and on one edge case.
  3. Check every function, method, and package name against the official docs.
  4. Check every link with curl -sI.
  5. Read the code line by line before you commit it, as described in using AI to learn.

Try this

Ask an assistant to write a function that reverses a string and returns the result. Run its output on "abc" and on an empty string. Then search the Python docs for the string methods and find the correct way to reverse a string. Write down the difference between the assistant's answer and the docs. Keep the note with your weekly check.

Sources