LLM as a Judge: When 90% Agreement Means Nothing

TL;DR: LLM as a judge means having one model score another model's output. Every guide explains how to build one. Almost none explains how to find out whether the one you built is any good, and the number the industry quotes for that, 80 to 90 percent agreement with human labels, cannot be interpreted without knowing your label distribution. On a set where 90 percent of outputs are already fine, a judge that approves everything scores 90 percent and has a Cohen's kappa of exactly zero. This is the piece to get right, because it is the number you will be asked to defend.

Key Takeaways:

• Raw agreement between a judge and a human counts accidental agreement as success. A 2026 evaluation of 21 judges across nine providers found the gap between exact match agreement and Cohen's kappa was 33 to 41 percentage points on MT-Bench, universally across the cohort
• The same set of verdicts can be reported as 55.1 percent or 89.9 percent accuracy depending on bookkeeping choices alone, without a single judgement changing
• Of the five top ranking guides for this term that I read in full, none names a chance corrected agreement statistic. One of them recommends shipping at a 75 to 90 percent raw match, which is the range the 2026 work shows is inflated
• The failure modes that break judges in production are not the famous biases. They are a sampling temperature that makes the grader change its mind, and a malformed response being coerced into a score of zero
• We ran our own suite five times over. Not one of the five test cases returned the same score twice, three moved by 15 points, and the headline average could have been reported as anything from 88 to 100

Table of Contents

• The Question the Guides Do Not Answer
• What LLM as a Judge Actually Is
• When 90% Agreement Means Nothing
• How to Validate an LLM Judge
• LLM as a Judge Without Ground Truth
• The Failure Modes That Are Not Bias
• We Graded Our Own Grader
• Where the Judge Actually Runs
• When Not to Use a Judge at All
• LLM as a Judge Best Practices
• Frequently Asked Questions

The Question the Guides Do Not Answer

Someone deployed an agent that answers customer questions. It has been running for six weeks. In a meeting, a person who does not write code asks whether it is doing a good job, and how we know.

The honest answer at that moment was that we had a number. An LLM was reading each reply and scoring it out of a hundred, the average was in the high eighties, and the average had been in the high eighties since the second week. What we did not have was any way to say whether that number was measuring the assistant or measuring the scorer.

That gap is the subject of this article. It is not a hard problem, it is just a step that almost nobody writes down. Search for this topic and you will find a dozen excellent guides on how to construct a judge: pick a model, write a rubric, decide between scoring one output and comparing two. They are good guides. But the question in that meeting was not how to build a judge. It was how to know whether to believe one.

This piece is written for the person who has to answer that question to somebody else. There is arithmetic in it, but no machine learning background is assumed, and the one piece of statistics involved takes a paragraph to explain and a spreadsheet to compute.

What this article does not cover: benchmark design, training reward models, and the research literature on judge architectures. Those are real subjects with better sources than me. This is about the operational question of trusting a score.

What LLM as a Judge Actually Is

Definition: LLM as a judge is an evaluation method in which one language model scores another model's output against a rubric or a reference answer, replacing a human reader. It exists because human review does not scale and string matching cannot grade a paragraph.

The encyclopedia entry files it under language model based evaluation. The method comes in a few shapes, and the shape you pick changes what your validation has to prove.

Most teams start with reference based scoring because it fits a test suite, and then discover that the interesting question happens in production, where there is no reference. That transition is covered further down.

The mechanics are not the hard part. A survey on LLM as a judge catalogues how judges are built and where their known weaknesses sit, and the Hugging Face cookbook walks through a working implementation. If you want to build one, those are better starting points than another explainer.

The hard part is the next sentence, and it is the one that gets skipped.

When 90% Agreement Means Nothing

Every guide that mentions validation says the same thing: label a sample by hand, run the judge, and see how often they agree. That is correct advice. The problem is the metric it stops at.

Here is why the raw percentage cannot be trusted, in one worked example with round numbers.

The judge that knows nothing and scores 90 percent

You have 100 support replies. You read all of them and decide that 90 are acceptable and 10 are not. This is a normal distribution for a working system, because a system where half the outputs are bad does not stay in production.

Now imagine the laziest possible judge. It approves everything. It does not read the input. It returns acceptable, always.

This judge agrees with you on 90 of 100 items. Ninety percent agreement, from a judge that is a constant. If your validation stops at the percentage, this judge passes, and every reasonable person in the room would be right to believe you had measured something.

Cohen's kappa exists to remove exactly this. It asks how much of the agreement is more than you would expect if both raters were guessing with the same overall habits. The formula is one line, where po is the agreement you observed and pe is the agreement you would expect by chance:

kappa = (po - pe) / (1 - pe)
 
po = 0.90
pe = (0.90 x 1.00) + (0.10 x 0.00) = 0.90
 
kappa = (0.90 - 0.90) / (1 - 0.90) = 0

Zero. Exactly no better than chance, which is the correct verdict on a judge that approves everything.

Now change one thing. Suppose the judge is genuinely doing some work: it flags 15 items, catching 8 of your 10 real failures and wrongly flagging 7 good ones.

po = (83 + 8) / 100 = 0.91
pe = (0.85 x 0.90) + (0.15 x 0.10) = 0.78
 
kappa = (0.91 - 0.78) / (1 - 0.78) = 0.59

Key principle: One percentage point of raw agreement separates a judge that is worthless from a judge that is moderately useful. Ninety percent and 91 percent look identical on a slide. Their kappa values are 0 and 0.59.

This is not a theoretical concern, and it is not small. The largest systematic evaluation of the method published so far, Reliability without Validity by Norman, Rivera and Hughes in June 2026, tested 21 judges from nine providers across three benchmarks, over 118 runs and roughly 541,000 individual judgements. Their first finding is that "kappa deflation between exact match and Cohen's kappa is universal", measured at 33 to 41 percentage points on MT-Bench, and that this held across the entire cohort including the April 2026 frontier models. A judge you were told agrees with humans 85 percent of the time may be sitting in the forties once chance is removed.

It gets worse before it gets better. In Agreement Metrics for LLM-as-Judge Evaluation, Rao and Callison-Burch show that the bookkeeping choices alone, which cases you keep, how you handle refusals and invalid outputs, how you pool scores across criteria, move a reported accuracy from 0.551 to 0.899 "without altering a single verdict". The same judge, the same answers, the same humans, and a number that swings by 35 points depending on what you decided to count.

What the 2026 evidence says

It is worth knowing why raw agreement became the default, because it was not a mistake so much as an inheritance. The method's founding work compared model verdicts to human preferences on chat responses and reported the match rate, which was the natural thing to report when the question was whether the idea worked at all. The tooling that grew around it kept the metric and dropped the question. What changed in 2026 is that the cohort is now large enough to show what the metric was hiding.

So when a vendor page tells you that properly calibrated judges reach over 80 percent agreement with human evaluators, that sentence is not false. It is unusable. It describes someone else's task, someone else's label distribution, and someone else's counting rules.

This is the gap in the published guidance, and I checked rather than assumed. On 2 September 2026 I searched this term, took the guides ranking at positions 2, 5, 11, 14 and 19, and read each one end to end looking for a single thing: does it name a chance corrected agreement statistic anywhere. Evidently AI, Confident AI, DeepEval, Arize and Braintrust. Five of five, no. Four of the five do tell you to validate against human labels, which is the right instruction, and all four stop at a raw percentage. One of them puts a number on it: ship when the judge and the human match on 75 to 90 percent of a golden dataset. That is precisely the range the 2026 work shows overstates discriminative ability by 33 to 41 points.

Credit where it is due: Confident AI publishes an alignment rate measured on its own annotated customer data rather than borrowing someone else's benchmark, which is more than the other four do. It is still a raw percentage.

How to Validate an LLM Judge

Here is the whole procedure. LLM as a judge validation is a seven step loop, and none of the steps requires a data science team.

1. Write down what you are actually measuring. Not good or bad. One property, phrased so that two people would label the same output identically. "Does this reply answer the question that was asked" is a criterion. "Is this reply good" is a mood.

2. Label a sample by hand before you write the judge. A few hundred items, drawn from real traffic rather than examples you made up. Do the labelling first. A rubric written after you have seen the judge's answers has been fitted to the judge. While you are labelling, write down the distribution, because you will need it in step four.

3. Run the judge over the same sample. Same items, no adjustments. Keep the explanations as well as the scores. The explanations are where you discover the judge has been applying a criterion you never wrote.

4. Report a chance corrected statistic, not a percentage. Cohen's kappa for one human and one judge on categorical labels. Compute it next to the raw agreement and look at the gap between them. A spreadsheet does this. There is no library requirement.

5. Read every disagreement. Not a sample of them, all of them. In my experience about half turn out to be the judge being wrong and the other half turn out to be the criterion being ambiguous. The second half is the more valuable finding, because it is a defect in your rubric that would otherwise have been permanent.

6. Re-run the judge twice on the same input. Two separate calls, identical prompt. If the two scores differ, your evaluation has a noise floor, and any difference smaller than that spread is not a real difference. This one takes two minutes and almost nobody does it.

7. Put the deterministic checks in front. Anything a schema or a regular expression can decide should never reach a model.

Notable fact: Step six is the cheapest step in the list and the one most likely to surprise you. It is also the step that catches the defect described in the self audit further down, in our own product.

The related work here is worth reading if you want to go deeper. Judge's Verdict, from October 2025, assesses 54 models on how well they replicate human judgement and makes Cohen's kappa the primary metric rather than an afterthought, which at the time was unusual enough to be the paper's contribution.

LLM as a Judge Without Ground Truth

Everything above assumes you have a reference answer. In production you usually do not. Nobody writes the ideal reply to a support ticket before the ticket arrives.

LLM as a judge without ground truth works by changing the question. Instead of asking whether the output matches a reference, you ask whether it satisfies a property that can be checked from the output and its input alone:

• Groundedness. Is every factual claim in this answer supported by the retrieved context that was passed in? This is the workhorse for anything built on retrieval, and it needs no reference because the context is the reference.
• Instruction adherence. Did the answer do what the system prompt told it to do, in the format it was told to use?
• Prohibited commitments. Does the reply promise a refund, a delivery date, or a discount that the business has not authorised? This one is usually better handled by guardrails before it ships, and by a judge after, as a measurement rather than a block.
• Question answered. Does the response address the question that was asked, as opposed to a nearby question the model preferred?

The validation loop does not change. You still need human labels, because you still have to prove the judge can detect the property. What changes is that you build that labelled set once, deliberately, rather than expecting one to exist for live traffic. A few hundred production items labelled by hand for one property is a weekend of work and it is the entire foundation of every quality number you will publish afterwards.

Definition: A referenceless judge scores an output using only the output and the context that produced it, with no known good answer to compare against. It is the standard shape for evaluation in production, and it shifts the burden from writing expected answers to writing precise criteria.

The Failure Modes That Are Not Bias

Every guide lists the same three biases: position bias, verbosity bias, self preference bias. They are real. The 2026 cohort study found position bias above 0.10 in two production deployed judges, while verbosity bias came out small, under 0.011, across the models they tested under a single pairwise rubric. So the famous list is not evenly weighted, and one item on it turns out to be mostly folklore in that setting.

But in a shipped system, the scores that go wrong usually go wrong for reasons that appear on no bias list.

It helps to separate two things that both travel under the name llm as a judge reliability. One is whether the judge repeats itself, which is a property of the configuration. The other is whether it is right, which is a property of the rubric. They come apart, and the 2026 cohort study has a name for it: a consistency and bias paradox, in which two production deployed judges showed test and retest reliability above 0.95 while carrying position bias above 0.10. Perfectly repeatable, and repeatably wrong.

The grader is sampling. If your judge call runs at any temperature above zero, it is generating its verdict rather than deriving it. The same input scores 82 on Tuesday and 76 on Thursday, the weekly average moves, and someone builds a theory about what changed in the product. Nothing changed in the product. High repeatability is not correctness either way. A judge can be beautifully consistent about the wrong answer, and a judge that samples cannot even manage that.

A malformed response becomes a score. Judges are usually asked to return JSON. Sometimes they return JSON wrapped in prose, or truncated at the token limit, or with the score as a string. The parser fails, and the code has to decide what to do. If it returns zero, that zero lands in your results table next to genuine failures, and it is indistinguishable from them. Your average drops and the cause is a parsing bug.

The rubric drifts and nothing tells you. Someone edits the criterion to clarify it. Scores shift by four points. The old scores and the new scores are now different measurements sharing a column in the same table, and no version marker separates them.

The judge is grading its own homework. If the same model produces the answer and scores it, self preference bias is not a risk, it is the architecture. Sometimes that trade is worth making. It should be a decision, not a default you did not know about.

We Graded Our Own Grader

Heym ships an evaluation feature with three scoring methods, and the third one is a judge. Since this article is about auditing judges, the honest thing is to audit ours, in public, with file paths. Everything below is from backend/app/services/eval_service.py unless stated, read on 2 September 2026.

What we got right. Two of our three scoring methods are deterministic. exact_match and contains are string comparisons that cost nothing and never drift, and they are offered before the judge in the interface. Our judge rubric is also anchored in five explicit score bands rather than an open request for a number out of a hundred, which is the practice the literature actually supports.

Running step six on ourselves

Then we ran step six on ourselves. Five customer support test cases, each scored five times, same suite, same system prompt, same model, LLM-as-Judge scoring at the default temperature of 0.7. Twenty five scored results. Here is every score.

Not one of the five test cases returned the same score five times. Three of them moved by 15 points. The suite average across all 25 results is 92.8, which the interface reports as 93 percent.

Now the number that matters to the person in the meeting. If you had run this suite once, the way almost everyone does, the average you would have reported depends on which five results you happened to get. Taking the worst available result for each case gives a suite average of 88. Taking the best available gives 100. There are thirteen distinct suite averages reachable from one run per test case, and all thirteen would have been reported as the quality of the same unchanged system.

Key principle: A twelve point swing in the headline number, with nothing about the product changing. Before you compare this week's score to last week's, you have to know how wide that band is, and the only way to know is to run the suite more than once.

Being precise about what this measures. At temperature 0.7 in a combined call, both the answer and the grade are sampled, so the variation above is the two of them together rather than judge noise in isolation. That distinction matters for diagnosis and not at all for the decision, because the number you report moves either way. The clean signal inside the data points the same direction: on the four occasions the model happened to reproduce the reference answer word for word, the judge scored it 100 every time, so the grader is consistent when the input truly is.

And the cost, since no guide on this topic will give you one. Those 25 scored items consumed 19,364 tokens, an average of 775 per scored item, at an average latency of 790 milliseconds. That is the real unit price of an LLM as a judge run, on a small suite, from a real execution.

Two more things our defaults get wrong

The temperature default is the finding above. It is 0.7 in the API schema and in the panel the user sees, and for non reasoning models it goes straight into the judge call. Reasoning models take a different branch that omits temperature and sets a low reasoning effort instead, which is the safer path by accident rather than by design. Nothing in the interface warns you that the number you are about to read has a band around it.

A parse failure scores zero. If the judge's response cannot be parsed as JSON, the parser returns an empty answer and the string "0". That zero enters the results table looking exactly like a genuine quality failure.

From building Heym: We ran this experiment because the article's own checklist asked a question we had never asked our own product. The spread was wider than anyone on the team expected, and it had been sitting in the run history the whole time.

None of that makes the feature useless. A judge with a measured band around it is far more informative than reading twenty replies by hand and forming an impression. It just means the number travels with the band attached, which is the whole point of this article. If you want the same audit applied to your own setup, the seven steps above are the audit, and step six took us one afternoon.

Where the Judge Actually Runs

An llm as a judge workflow is not one model call. It is a small pipeline, and most of it should not involve a model at all.

What is Heym: Heym is an open source, self hosted AI workflow automation platform with a visual editor. Workflows are built from nodes on a canvas, and it ships an evaluation feature with three scoring methods, two deterministic and one LLM judge.

In Heym the judge is not something you assemble. Evaluation is its own tab, documented at Evals, where you create a suite, add test cases with an input and an expected output, pick a scoring method, and run it across one or more models. The three scoring methods are Exact Match, Contains, and LLM-as-Judge, which is the deterministic-first ordering from step one of the diagram expressed as a dropdown. LLM-as-Judge takes an optional separate credential and model, so the model doing the grading does not have to be the model being graded.

What the tab does not do is the boundary routing in steps three and four. That part belongs in a workflow, where a Condition node on the score sends the confident cases through and the uncertain band to human in the loop, which pauses the run and waits for a person rather than guessing. Scores and traces land in execution history either way, so the drift question becomes a query rather than a memory.

The wider system view, suites and test cases and running the same prompt across several models, is covered in the AI agent evaluation guide. This article is about one method inside that system. If you are setting evaluation up from scratch, start there and come back here when you have a score you need to defend.

There is a related pattern worth knowing about. In adversarial code review we run a reviewer agent and a challenger agent with opposing incentives, and the challenger's job is to remove findings rather than add them. That is the same idea as a judge, arranged so that the two roles cannot share a blind spot. It is more expensive and it is the right shape when the cost of a wrong verdict is high. The evaluator and optimizer pairing in agentic design patterns is the general form.

When Not to Use a Judge at All

LLM as a judge in production is a tool with a narrow correct use, and the most common mistake is reaching for it when nothing needed reading.

Do not use a judge when the property is decidable by code. Valid JSON, a required field present, a number within a range, an identifier matching, a forbidden word absent. These are checks, not judgements. They are free, instant, and unarguable.

Do not use a judge as a safety control. A score computed after the fact does not stop anything. If the output must not contain a customer's card number, that is a guardrail at the boundary, not a grade in a report.

Do not use a judge when the decision is expensive and the volume is small. If you are shipping forty responses a day and a wrong one costs a customer, read the forty. Automated evaluation is a scaling tool. At small volume, scaling is not the problem you have.

Do not use a judge to settle a question of taste that the organisation has not settled first. If two people on the team disagree about whether a reply is too formal, no model will resolve that, and the rubric you write will simply encode whichever of them wrote it.

On llm as a judge vs human evaluation, the framing as a contest is the error. Human evaluation is the measuring stick that tells you whether the judge works. The judge is what lets you apply that stick to volumes a person cannot read. Remove the humans entirely and you have an unvalidated instrument reporting numbers nobody can check, which is worse than no number at all, because a number gets quoted.

LLM as a Judge Best Practices

The llm as a judge best practices worth the name are short, and most of them are about arithmetic and defaults rather than prompting.

1. Set temperature to zero for the judge call. A grader that samples cannot reproduce itself. This is one line of configuration and it removes an entire class of phantom trend.
2. Report kappa next to the raw agreement, always. Two numbers, not one. The gap between them is the interesting part.
3. Write the rubric before you look at the model's output. Otherwise you are fitting the criterion to the judge.
4. Anchor the scale. Define what 100, 80, 50 and 0 mean in words. An unanchored request for a score out of a hundred is an invitation to invent one. Our own llm as a judge prompt uses five explicit bands for exactly this reason.
5. Treat a parse failure as an error, not a zero. They belong in different columns.
6. Version the rubric with the results. A score without the criterion that produced it is not comparable to anything.
7. Use a different model to grade than to answer, or accept in writing that you have chosen self evaluation and why.
8. Re-validate whenever the rubric, the judge model, or the traffic changes. All three move, usually without anyone announcing it.
9. Keep the explanations. The scores tell you the level. The explanations tell you whether the level means anything.

Notable fact: Of these nine, the two that most often go unimplemented are the first and the second, and both take under an hour. The remaining seven are the ones that need a habit.

A rubric you can paste

This is the shape that survives step five. Replace the criterion and the anchors, keep the structure, and keep the explanation field, because the explanations are what let you audit the scores later.

You are scoring one support reply against one criterion. Do not score anything else.
 
CRITERION: Every factual claim in the reply is supported by the CONTEXT below.
Claims about policy, timing, or money count. Pleasantries do not.
 
CONTEXT:
{context}
 
REPLY:
{reply}
 
Score:
- 2 = every claim is supported by CONTEXT
- 1 = one unsupported claim, and it is not about policy, timing or money
- 0 = any unsupported claim about policy, timing or money, or more than one of any kind
 
Return only: {"score": <0|1|2>, "unsupported_claims": ["<quote>"], "explanation": "<one sentence>"}

Two deliberate choices in there. The scale is three points rather than a hundred, because a judge asked for a number out of a hundred invents precision it does not have, and because a three point scale is one a human can apply consistently enough to be the reference. And the judge is asked to quote the claims it objected to, which turns every disagreement into something you can check in seconds rather than a number you have to argue with.

For step four, the arithmetic fits in a spreadsheet. With counts in cells A1 to A4 in the order both-pass, judge-pass-human-fail, judge-fail-human-pass, both-fail:

po  = (A1+A4)/SUM(A1:A4)
pe  = ((A1+A2)/SUM(A1:A4))*((A1+A3)/SUM(A1:A4)) + ((A3+A4)/SUM(A1:A4))*((A2+A4)/SUM(A1:A4))
kappa = (po-pe)/(1-pe)

Frequently Asked Questions

What is LLM as a judge?

LLM as a judge is an evaluation method where one language model scores another model's output against a rubric or a reference answer, instead of a human reading every response. It exists because human review does not scale and string matching cannot grade an open ended answer. The score it returns is a measurement, and like any measurement it is only useful once you know its error rate on your own data.

How do you know if your LLM judge is right?

You label a sample by hand, run the judge over the same sample, and compare. The part almost everyone skips is the last step: report a chance corrected agreement statistic rather than the raw percentage. Raw agreement counts the times you and the judge happened to say the same thing, including the times you both said it for no reason. On a set where most outputs are fine, a judge that approves everything scores very high and knows nothing.

How accurate are LLM judges compared to humans?

The commonly quoted figure is 80 to 90 percent agreement with human evaluators, and on its own that figure cannot be interpreted. A 2026 evaluation of 21 judges across nine providers found that the gap between exact match agreement and Cohen's kappa was 33 to 41 percentage points on MT-Bench, and that it was universal across the cohort. A judge advertised at 85 percent agreement can be sitting at a kappa near zero on a skewed dataset.

Can you use LLM as a judge without ground truth?

Yes, and most production evaluation is exactly that case, because labelled answers rarely exist at runtime. You switch from asking whether the answer matches a reference to asking whether it satisfies a property you can define, such as being grounded in the retrieved context or containing no commitment the business cannot keep. Validation still requires human labels, but only on a sample you build once, not on live traffic.

How much does LLM as a judge cost?

It scales linearly with the volume you grade and with the size of the judge model. For a concrete figure from a real run: 25 scored items on a small customer support suite consumed 19,364 tokens, an average of 775 tokens per scored item, at 790 milliseconds average latency. The cost that surprises people is not that one. It is the human sample, because validating a judge properly means a person labelling a few hundred items and relabelling them whenever the rubric changes.

When should you not use an LLM judge?

Whenever the property you care about can be checked by code. Valid JSON, a required field, a number inside a range, a forbidden phrase, an exact identifier: all of these are string and schema checks that cost nothing, never drift, and cannot be argued with. Use a judge only for the part of quality that genuinely needs a reader, and put the deterministic checks in front of it.

The Number You Can Defend

The meeting question was whether the assistant was doing a good job and how we knew. The answer that works is not a higher score. It is a smaller, duller sentence: on a sample of three hundred replies that a person labelled in June, this judge agrees with that person at a kappa of 0.61, it re-scores the same input identically at temperature zero, and here are the twelve cases where it was wrong.

That sentence survives being challenged. The average out of a hundred does not, because the first person who asks what it is measuring will get an answer about the scorer rather than the system.

Both of the 2026 papers end in the same place, which is worth noticing. Norman and colleagues distil their findings into what they call a Minimum Viable Validation Protocol; Rao and Callison-Burch end with a reporting checklist meant to make agreement claims reconstructible. Two independent groups, working on different questions, both concluded that the missing artefact is a protocol rather than a better judge. My read is that within a year or two, publishing a bare agreement percentage will look the way publishing a benchmark score without the test set looks now, and the teams that will find that transition cheap are the ones already recording their label distribution today.

If you are running agents and reporting on them, the seven step loop above is a day of work, once, and it converts a number you are quoting into a number you can defend. If you want to build the pipeline rather than the spreadsheet, the evaluation guide covers suites and test cases, and the Heym documentation covers the nodes. Heym is open source and self hosted, so the judge, the rubric and the labels all stay on your own infrastructure, which matters more than usual when the labels are real customer conversations.

Sources

• Norman, J. D., Rivera, M. U., Hughes, D. A., arXiv, June 2026. Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias
• Rao, D., Callison-Burch, C., arXiv, May 2026. Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why
• arXiv, October 2025. Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
• Gu et al., arXiv, 2025. A Survey on LLM-as-a-Judge
• Hugging Face, 2026. Using LLM-as-a-judge for automated evaluation
• Wikipedia, 2026. LLM-as-a-Judge