A test written by the model that wrote the code is not a second opinion

A green suite from the same model answers a smaller question

The workflow is ordinary now. An agent reads a ticket, writes a function, writes tests for the function, runs them, fixes whatever fails and reports that everything passes. The report is true. The suite is green.

What the report leaves out is what the green is evidence of. The tests ran the code, the code did not crash, and the assertions matched. Every one of those facts is real. None of them is a second opinion on whether the function does what the ticket asked, because one reading of the ticket produced both sides of the comparison.

This is not a claim that AI-written tests are poor. One study mined 13,866 test oracles from 135 Java projects, all written after the models’ training cutoffs so none of them could have been memorised. On the projects the authors put through mutation analysis, the oracles the models generated reached an average mutation score of 43%, against 45% for the ones the developers wrote ¹. That is close to human work.

Good tests can still be the wrong kind of evidence. The useful question is not whether a test is well written. It is whether the test would fail if the code were wrong in the way this code is most likely to be wrong. A pass is strong evidence only when a wrong implementation would usually fail it.

The assertion is the only part of a test that can agree with a bug

A test has three parts: an input, an observation of what the code did, and an oracle that decides whether that observation was right. Inputs are cheap. A model can generate thousands of them, including the empty list, the negative number and the value one past the boundary. The oracle is the expensive part, and it is the only part that can share a mistake with the code.

01// Ticket: tax each line, add them up, round the total once.02function invoiceTotal(lines: { net: number; rate: number }[]): number {03  return lines.reduce(04    (sum, { net, rate }) => sum + Math.round(net * rate * 100) / 100,05    0,06  );07}08​09test("three small lines", () => {10  const line = { net: 0.1, rate: 1.075 };11  expect(invoiceTotal([line, line, line])).toBeCloseTo(0.33);12});

Nothing about this test is sloppy. It exercises the loop, the tax and the rounding. It would still pass for as long as the function keeps the bug, because the number on line 11 was computed with the function’s own rule. Coverage says the rounding ran. It cannot say the rounding happens in the right place.

This failure is older than language models. Classic white-box test generators choose inputs by reading the code and record whatever the code returns as the expected value, so a fault in the implementation becomes an assertion in the suite. When 54 graduate students were asked to sort generated tests into fault-encoding and correct ones, the median participant misclassified 33% of the fault-encoding tests and 25% of the correct ones ². A test that defends a bug does not look like one.

Language models were supposed to improve on this, because they can read intent from names, comments and the ticket rather than only from behaviour. They partly do. They also tend, like the generators before them, to write oracles that capture what the program actually does rather than what it was expected to do ³.

Showing the model the code changes the tests it writes

The cleanest measurement holds everything else fixed and swaps correct code for incorrect code in the prompt. Across 11 models, tests written from a HumanEval task description plus the correct code were accurate 80.4% of the time on average. With the same description and incorrect code, average accuracy fell to 57.1% ⁴. The authors’ warning is the whole post in one line: used on relatively immature code, the approach “will be more likely to ‘bake in’ errors.”

A later study separated the two things that happen. It counted misguided tests, which pass on the buggy code and fail on the fixed version, and effective tests, which do the opposite. Across 11 models on real bugs from Defects4J, prompting with the buggy code produced 137.69 misguided tests per model on average, against 16.46 from the fixed code. Effective tests fell from 304.08 to 104.15 ⁵. The code in the prompt did two jobs at once. It added tests that defend the bug, and it removed the tests that would have exposed it.

The agentic version of the workflow has been measured directly too. In one arm, a model wrote the code and then wrote the tests in the same session with the full conversation history. In the other, tests came from a fresh interaction holding only the task description. On the faulty implementations, the first arm detected about 14% of faults and the second 25% ⁶.

It gets worse when the tests drive the next revision of the code. Studying models that debug their own output against tests they wrote themselves, one group found that this post-execution loop “struggles with the test bias introduced by self-generated tests, which can lead to misleading feedback” ⁷. A loop like that converges on a program and a suite that agree with each other, which is not the same as converging on the ticket.

A summary of the buggy code is still the buggy code

The obvious fix is to hide the code and describe it instead. The misguidance study tried exactly that: replace the code in the prompt with a specification docstring that a model writes by reading the buggy code. Misguided tests dropped from 137.69 to 113.00, and effective tests rose from 104.15 to 186.77 ⁵.

Read those two numbers separately. Bug-finding recovered a good part of what it had lost. Bug-defending barely moved: 113.00 misguided tests is still nearly seven times the 16.46 that came from the fixed code. A description written by reading a bug still carries the bug, only more quietly.

The same study also put the docstring and the code in the prompt together. Compared with the docstring alone, that produced 33.08 more misguided tests and 64.85 fewer effective ones on average. The description did not help by adding information. It helped because the implementation was gone.

So a specification protects a test on two conditions. It must not have been derived from the implementation, and the implementation must be out of view while the tests are written. A summary of the code fails the first condition. A spec pasted next to the code fails the second.

A different model is not an independent witness

The next obvious fix is to ask a different model. It helps less than it sounds, and the evidence for that predates language models by forty years.

In 1986, Knight and Leveson had 27 versions of one program written independently from the same specification at two universities, then ran each against one million tests. The programs were individually very reliable, but the number of tests on which more than one of them failed was substantially more than independence would predict ⁸. Separate authors did not produce separate mistakes.

The experiment was repeated this year with AI coding agents in place of students. Forty-eight agent-written implementations passed the acceptance screen. Over the million-input campaign they produced 429 coincident failures where an independence model predicted 115.36, and the paper’s answer to its own question is flat: “Cross-language and cross-agent program generation do not provide enough diversity to make failure correlation disappear” ⁹. The largest cluster is a shared misreading, not a shared typo. Many agents computed the circumcircle where the specification needed the smallest enclosing circle.

The same paper has the other half of the lesson. Majority voting over three versions still cut the mean failure count from 387.44 to 130.99. Correlated evidence is worth having. It is just not independent, and counting it as independent overstates the confidence it buys.

Put the two findings together and model identity turns out to be the weaker variable. A different model reading the buggy implementation is anchored to the bug. The same model in a fresh session, holding only the ticket, is not. If you can only afford one of the two, the studies above point to the fresh session.

A model grading its own translation has the same problem

Localization runs the same loop under a different name. A model translates a string, then scores the translation, or critiques it and rewrites it. That second pass is a test written by the author of the code.

It behaves like one. A study of six models on translation, constrained generation and maths found self-bias, the tendency to favour one’s own output, in every model it examined. Self-refinement made the output more fluent and more understandable, and it amplified the bias. What reduced it was external feedback with accurate assessment, along with larger models ¹⁰. A separate study defines self-preference as an evaluator scoring its own outputs higher than others’ “while human annotators consider them of equal quality”, and finds the strength of that preference tracks, linearly, the model’s ability to recognise its own text ¹¹.

That is an uncomfortable pair of results for anyone using a model to check a model’s translation. Fluency is exactly what a self-review improves, and the self-reviewer is exactly the judge most likely to mistake its own fluency for correctness.

Some of the checks a translation needs already have an oracle that shares nothing with the model. Whether a message still parses, whether every placeholder survived and whether the plural branches are all there are questions a parser answers without having read the model’s reasoning at all. We have made the case for putting those checks in a gate in the post on constrained localization. The point this post adds is where the gate has to sit: outside the model’s loop. Asking the model that wrote the string whether its placeholders are intact is asking it to grade its own reading.

The checks no parser can make, such as whether a short label names the right action on the right screen, need a different path in. Often that is a person who holds the fact the model was never given, which is the routing question human in the loop is not a review queue is about.

Independence is a list of things the two paths do not share

Independence is not a property of who typed the test. The clearest working definition comes from fields where an undetected defect is expensive. NASA’s independent verification and validation program requires technical independence, meaning the work is done independently of the developer. It requires managerial independence, meaning a separate organisation that decides for itself what to analyse, which methods to use and on what schedule. And it requires financial independence, meaning a budget the development organisation does not control ¹².

Two prompts called “developer” and “tester” inside one agent loop share all three. The loop decides what gets tested, when testing stops, and whether the result is good enough to report. Renaming a role does not move any of those decisions somewhere else.

The more useful way to think about it is as a list of things the code path and the test path can share, each of which lets a particular kind of mistake through.

Keep the tests, and count them as what they are

Keep the tests. Same-model tests catch crashes, type errors, missed inputs and regressions, and after the code has been reviewed they are a perfectly good way to stop it changing by accident. The mistake is not writing them. The mistake is counting them as corroboration.

For every behaviour that matters, make sure at least one path runs from the requirement to an assertion without passing through the implementation. In practice that means a handful of habits:

• Write the cases that pin a requirement before the code exists, and freeze them. Boundaries, forbidden behaviour and error conditions come first, because those are where a misreading hides.
• When tests are generated after the code, generate them in a fresh context that holds the ticket and not the implementation. A summary of the implementation does not count.
• Take expected values from somewhere the code did not produce: a worked example in the ticket, a reference implementation, a known identity, an invariant that must hold for any input.
• Use a method that does not trust the author. Mutation testing asks whether the suite can tell the code apart from slightly wrong versions of it. Property tests and fuzzing choose inputs nobody thought to write down.
• Label what a suite is. A thousand tests written in the same session as the code are a regression net. They are not a thousand confirmations, and a report that says “verified” should not be built on them alone.

The rule underneath all of it fits in a sentence. Never let the implementation be the source of truth for the tests you cite as evidence that the implementation is correct.

Confidence does not come from how many checks agree. It comes from how hard it would be for them to be wrong in the same way.

References

1. 1.Molinelli, Di Grazia, Martin-Lopez, Ernst and Pezzè, 2025 Do LLMs Generate Useful Test Oracles? An Empirical Study with an Unbiased Dataset ASE 2025
2. 2.Honfi and Micskei, 2017 Classifying the Correctness of Generated White-Box Tests: An Exploratory Study arXiv:1706.02217, Software Quality Journal 2019
3. 3.Konstantinou, Degiovanni and Papadakis, 2024 Do LLMs generate test oracles that capture the actual or the expected program behaviour? arXiv:2410.21136
4. 4.Huang, Zhang, Harman, Du and Cui, 2024 Measuring the Influence of Incorrect Code on Test Generation arXiv:2409.09464
5. 5.Zhao, Zhou and Cohen, 2026 Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests ISSTA 2026, arXiv:2607.22883
6. 6.Konstantinou, Tambon and Papadakis, 2026 On the risk of coding before testing: An empirical study on LLM-based test generation workflow arXiv:2607.05139
7. 7.Chen et al., 2025 Revisit Self-Debugging with Self-Generated Tests for Code Generation ACL 2025
8. 8.Knight and Leveson, 1986 An Experimental Evaluation of the Assumption of Independence in Multiversion Programming IEEE Transactions on Software Engineering 12(1)
9. 9.Ron, Baudry and Monperrus, 2026 N-Version Programming with Coding Agents arXiv:2606.20158
10. 10.Xu, Zhu, Zhao, Pan, Li and Wang, 2024 Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement ACL 2024
11. 11.Panickssery, Bowman and Feng, 2024 LLM Evaluators Recognize and Favor Their Own Generations NeurIPS 2024
12. 12.NASA IV&V Overview NASA Independent Verification and Validation Program