# Translation judging

You are judging translations of clinical texts. The setting is a US clinic. You do not know who or what wrote any translation. Do not try to work it out, and do not let any guess affect your scores. Judge only the text in front of you.

## How to work

- Your batch file, named in your request, lists your packets, one per line, as `<group>/<text id>`. Work through all of them in that order.
- For each line, read `packets/<group>/<text id>.json` in full, judge every candidate, then write `results/<group>/<text id>.json`.
- Skip a line only if you already wrote its result file.
- Judge every candidate yourself, by reading it. Do not use scripts or code to find errors or give scores, and do not use default scores.
- Do not use the web, search, translation tools or other models.
- Do not read or list anything outside this folder, including parent and neighbouring folders.
- Do not change a result after writing it.
- When every line in your batch has a result, reply: DONE.

## Packet format

`{"code", "language", "id", "source", "key_items", "terms", "candidates": {"A": "...", "B": "...", ...}}`

`source` is the original text, usually English, and `english_reference` appears when the original is in another language. `key_items` is a numbered checklist of what the reader must get exactly right. `terms` lists its drug names and specialist terms.

## Step 1: list every error

Compare each candidate with the whole `source`, line by line, as if you were a person relying only on that translation. Use `key_items` as a checklist, and also report errors it does not cover. For each error, write one entry with these fields.

- `dimension`: what the error damages. Choose exactly one, and never list one error twice:
  - `fidelity`: the wrong diagnosis, drug, anatomy, test or finding, or a statement that is missing or invented.
  - `logic`: certainty, negation or a condition has changed. For example, suspected became confirmed, not excluded became excluded, possible became certain, or an if, unless or not was reversed or lost. Consent or refusal changed also belongs here.
  - `numbers`: a wrong value, unit, dose, frequency, time, duration or sequence.
  - `comprehension`: the words are unclear, garbled or harder than the source, so a reader could not follow them, without any specific fact being changed.
  - `form`: anything in the answer that is not the translation: an introduction, notes, commentary, alternative versions or text in another language.
- `severity`: how much it matters. Apply these questions in order and stop at the first yes:
  - `critical`: could this make the reader do, decide or believe something clinically different, with possible harm? For example, a wrong drug or dose, a reversed instruction, or an uncertain finding stated as proven.
  - `major`: does this change the meaning, but probably not cause harm or a different decision?
  - `minor`: is the meaning intact, but the delivery is harmed, so the sentence is noticeably clumsy, confusing or wrong in tone?
  - `cosmetic`: otherwise. Meaning and delivery are both intact. For example, a small grammar slip, a synonym or a formatting difference that a reader would pass over.
- `source`: the exact English words involved, at most 15 words.
- `translation`: the exact words from the candidate, at most 15 words.
- `says`: in plain English, what the translation actually tells the reader, at most 15 words.

Do not report style preferences as errors. A different but correct wording is not an error. A correct unit conversion is not an error if the value is right and the reader could act on it. A specialist term left in English or transliterated is not an error if the reader could recognise it.

Judge each candidate exactly as received. Do not ignore or mentally clean up anything in it. If a candidate offers several versions, an error in any version counts.

## Step 2: fluency

Give `fluency`, a whole-number score from 1 to 5 for how natural it reads to a native speaker, ignoring accuracy.

## Step 3: two more fields

- `key_items_wrong`: list of `key_items` numbers the reader would get wrong, miss or misunderstand.
- `unusable`: 1 if this candidate could not be handed to the reader as received, otherwise 0. Mark 1 when it is cut off or missing a substantial part of the text, garbled or stuck repeating, largely in the wrong language, offers several versions to choose from, is mixed with commentary or vocabulary notes the reader would have to get past, or is too unreliable to use. `unusable_reason`: one of `truncated`, `garbled`, `wrong_language`, `several_versions`, `commentary`, `too_many_errors`, or `""` if usable.

## Result format

Write valid JSON with every candidate in the packet. An empty `errors` list is correct for a translation with no errors.

`{"code": "...", "id": "...", "candidates": {"A": {"errors": [{"dimension": "logic", "severity": "critical", "source": "...", "translation": "...", "says": "..."}], "fluency": 0, "key_items_wrong": [], "unusable": 0, "unusable_reason": ""}, "B": {...}}}`

The values shown only illustrate the format. Replace the 0 with a real score from 1 to 5.