How translation memory can become slow poison

A translation memory is supposed to give one answer

We have written before about what a translation memory should retrieve on: the string, the meaning, or both. That question is about ranking. This one is about what happens after the ranking is done, when the memory has found the segment you asked for and has more than one answer for it.

A translation memory exists to answer one question: have we translated something like this before. The useful version of that question is stricter than it sounds. Ask it about the exact same source string, byte for byte, submitted twice, and a memory that is doing its job returns the same target both times. That is the entire point of storing it in the first place.

It does not always happen. A source segment can sit in a memory paired with two, three, or more different approved target segments, none of them flagged as wrong, because none of them necessarily are. A different translator worked the string at a different time, in a context the tool cannot see, and made a different, defensible choice. The memory now holds a contradiction it has no way to notice on its own.

The rate that held across four independently studied memories

The size of the problem was measured directly, not estimated. A 2011 study took four translation memories from two commercial companies, two in English to German and two in English to Japanese, ranging from 18,799 to 301,583 translation units, and built a typology for exactly this question: when the same source segment repeats inside a memory, does it come back with the same target every time¹?

Four categories fall out of that question. Inconsistent source translated as inconsistent target. Inconsistent source translated as consistent target, which is a human correctly overriding a formatting difference. Consistent source translated as consistent target, the outcome a memory exists to produce. And the one this post is about: a source segment that repeats identically, paired with a target that does not.

Read the table by what does not move. TM C is sixteen times the size of TM B and has the lowest rate of the four. TM D is close to the same size as TM C and has the highest. Two companies, two language pairs, a sixteen-fold range in size, and every single memory landed inside a five-point band. Nothing here looks like decay. A memory that got less reliable as it aged would show the rate climbing with size, or with time in production, and it does not.

What holds steady instead is closer to a per-unit event rate. Every time a source segment gets translated for the second, third, or eleventh time, there is something like a one-in-twenty chance the new instance disagrees with the ones already stored. That rate does not go up as the memory grows. The count does, because count is rate times opportunities, and a bigger memory has more segments getting reused. TM D’s sample alone contains 1,713 of them.

Where the second answer actually comes from

Microsoft’s own globalization documentation names the two ordinary events that produce this, and neither one is an accident or a vendor failing to do their job².

The first is combination: multiple vendors working the same content across multiple projects, or a company merger combining two organizations’ translation assets, merged through the TMX exchange format because the two memories were never in the same tool to begin with. Nothing about the merge is wrong. It is two valid histories landing in one database, and the database has no native way to tell you they disagree.

The second is drift: a product changes its own terminology over time, for reasons that have nothing to do with translation at all, and the memory does not update itself to match. The documentation states this plainly: a company’s terminology database is routinely out of sync with its own translation memory, which means the memory keeps offering the old term as a valid match long after the product stopped using it.

en   Click an empty part of the drawing area.​de   Klicken Sie auf der freien Zeichenfläche.de   Klicken Sie auf einen freien Bereich der Zeichenfläche.de   Klicken Sie auf einen freien Bereich der Zeichenfläche.de   Klicken Sie auf einen beliebigen freien Bereich auf der Zeichenfläche.​      one source string, three distinct approved German translations,      all four instances still live in the same memory

The framing Microsoft uses for the fix is worth sitting with, because it is not a translation-industry metaphor borrowed for this post. It is their own words: “TMs are an asset that requires maintenance and investment, just like your product’s code” ². Treat a memory as something created once and left alone, and it gets less useful over time in exactly the way an unrefactored codebase does, for the same underlying reason: nobody goes back and reconciles the entries that started disagreeing with each other.

Not every disagreement is a defect

The tempting fix is automatic: find every source segment with more than one target, keep one, delete the rest. Microsoft’s own maintenance guidance stops short of that, and specifically warns against it: combine entries with multiple translations for the same source segment, but only if the source is used in the same context². A second translation is not automatically the wrong one. Sometimes it is the correct response to a context the first translator never saw.

The 2011 study caught this happening in both directions. In one Japanese memory, a single English word, “Selecting,” came back with eleven different translations, most of them adding detail the source never stated: selection of an element, selection of a callout element, selection of a polygon, and so on. Some of that added detail earns its place, because the interface element being selected really does differ. The study’s own conclusion is careful not to call this a defect outright, noting only that the first translation may have contained detail that was not appropriate for the segments that reused it¹.

The same study also found something with no legitimate reading at all: a translator’s inline comment explaining a word choice, saved directly into the memory alongside the segment, and then propagated to later reuses of that entry as if it were part of the approved translation. Nothing about the tooling flagged that as different in kind from the deliberate variation above. Both are “a second target for a source that repeated,” and only one of them is noise.

Which is the actual argument against automatic deduplication. Deciding which target is current requires knowing whether the context changed, and context is exactly the information a source-segment match throws away. A tool that resolves the contradiction by picking one has a coin’s worth of information to work with.

How a human-facing CAT tool copes with it today

Production CAT tools do not try to resolve the contradiction automatically. They make sure a person sees it. The mechanism is the leverage penalty: a configured deduction, applied per translation memory, per user, or per role, that lowers the reported match percentage before it reaches the translator³. If a memory returns an 80% match and carries a five-point penalty, the translator sees 75%, not 80%.

The number itself is arbitrary. What it buys is real: a segment that might be one of several conflicting answers gets pushed below the threshold that would apply it automatically, and the translator has to look at it and decide. The penalty is not a fix for the contradiction. It is a guarantee that a human, not the match score, gets to make the call.

That guarantee has a hidden dependency. It works because the consumer of the match is a person capable of rejecting it. Take that person out of the loop and the penalty stops doing anything at all, because a lower number is still just a number, and nothing downstream of the retrieval step has ever been asked to treat a number as a reason to say no.

What changes when a model reads the match instead of a person

Which is now an ordinary thing to do. Supplying translation memory matches as exemplars in a prompt, with no change to how the model was trained, is an established technique rather than a shortcut⁵. The match reaches the model as context, and a model conditioned on retrieved context is trained to treat it as informative, not as a claim that might be false.

There is a measured cost to that trust. When a retrieval-augmented translation model is fed matches from a memory that does not fit the text being translated, quality does not merely fail to improve. It drops below what the same model produces with no memory at all⁴.

A human translator looking at a 75% match with a penalty on it can still say no. A model reading the same match as prompt context has no equivalent gesture, because nothing in the pipeline asked it to have one. This is the same distinction we have made before between a prompt and a constraint: a prompt is a request, and a request can be declined or ignored inconsistently. A translation memory contradiction that a leverage penalty was enough to catch in a human workflow is invisible again the moment the same match is handed to a model as text.

The fix is resolution, not pruning

None of this is solved by pruning old entries. The data argues against it directly: TM C, the oldest and largest memory in the study, had the lowest rate of contradiction of the four, not the highest. Age is not the mechanism. Reuse by more than one person, across more than one moment of context, is the mechanism, and that keeps happening for as long as a memory keeps being useful.

The fix Microsoft’s own guidance describes is resolution, not deletion²: for a source segment with more than one live target, decide which one is current, mark it, and keep the others queryable rather than silently discoverable as if they were equally valid. That decision needs the same thing the deduplication problem needed and did not have: context and provenance recorded on the unit itself, not reconstructed after the fact from which one happens to be newest.

We have made a version of this argument before about a memory’s own machine-translation output feeding back into itself: a memory that cannot answer “did a person approve this, and under what conditions” for a given segment does not have a resolvable contradiction, it has an unresolvable one, because there is nothing left to arbitrate between. The same bookkeeping that a postfill loop needs to stay honest is the bookkeeping a merged or drifted memory needs to ever get untangled.

The number worth tracking

Match rate, memory size, and TM age are the numbers most dashboards already show, and none of them caught this. The number that does is narrow and specific: of the source segments that repeat in your memory, what fraction currently resolve to more than one live target. The 2011 study found that number sitting between one in twenty and one in sixteen in every memory it looked at, without anyone having merged a vendor’s TM or shipped a rename in the meantime. Ordinary reuse produces it on its own.

A translation memory does not get safer as it gets bigger. It accumulates a larger, quieter set of segments that no longer agree with themselves, at a rate that has nothing to do with how long the memory has existed. Nothing about its size, its age, or its average match score reports that number to you. Finding it means asking the question directly, and the moment a match starts reaching a model instead of a person, asking it stops being optional.

References

1. 1.Moorkens, 2011 Translation Memories guarantee consistency: Truth or fiction? Translating and the Computer 2011
2. 2.Microsoft Learn, Globalization documentation Maintain translation memories learn.microsoft.com, Globalization
3. 3.memoQ documentation Edit TM settings docs.memoq.com
4. 4.Hoang, Sachan, Mathur, Thompson and Federico, 2022 Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestions arXiv:2210.05059
5. 5.Reheman, Zhou, Luo, Yang, Xiao and Zhu, 2023 Prompting Neural Machine Translation with Translation Memories arXiv:2301.05380