Trusted sources

Every drawing in the arena has to be what it says it is: one model’s own answer to one of our prompts. This is where the drawings come from and how each one is checked.

Our own runs

We send each model the same request, word for word, with its default settings, and keep the first answer. No retries, no choosing the best of several, no edits. Models from Anthropic, OpenAI, Google and xAI are asked through those companies’ own APIs; models from other labs are asked through OpenRouter. Every request and answer is logged with the provider’s response ID.

Community submissions

Anyone can submit a drawing. Before it joins the arena we check that it answers the prompt it was entered for, that it’s consistent with what the named model produces, and that it’s the model’s unedited output. Drawings with scripts or anything loaded from outside the file are refused automatically.

If a problem turns up later, we take the drawing out. The votes it had already received stay in the record.

Contributors

Loading contributors.

What the ranking is built on

The rating method isn’t ours. It follows published work, which you can check our numbers against:

• arena-rank, LM Arena’s open-source Bradley-Terry ranking, which our fit matches on scale and tie handling.
• LMSYS on moving from online Elo to Bradley-Terry, and why the order of votes shouldn’t matter.
• Hunter, “MM algorithms for generalized Bradley-Terry models” (Annals of Statistics, 2004), the fitting procedure we use.

How we apply it is on the method page.