Methodology
Each model gets one rating, built from head-to-head comparisons inside every benchmark it appears on. Every model starts at a score of 1500, and the gap between two ratings predicts how often one outscores the other.
1. Collect each benchmark's scores
We pulled every benchmark's public results as of November 17, 2026, and took one headline score per model from each. The table at the end of this section lists the score used for each benchmark. We matched the same model across benchmarks by hand, using name and version only. We leave out entries without a version or with an ambiguous name. A policy wrapped in a separate planner is not merged with the bare policy, and neither is a model trained on a benchmark's own data under a new name, such as π0.5-MolmoSpaces. Under this framework, a model fine-tuned separately for two benchmarks still counts as one model. A separate checkpoint trained for each task of one benchmark is different. We list those checkpoints as their own entry, marked per-task, so they do not stand in for the model they fine-tune. We discuss both choices in the limitations section.
Where a benchmark publishes success against time, we use its success at the longest time limit it reports, so a rating reflects whether a model finishes a task rather than how fast. Only PhAIL publishes this. Its DROID headline counts how much of each task is done within 240 s, and its success curves run to 480 s. We add each model's gain between the two on PhAIL's own curve to its headline score. Only GPT-6 Astra ran past 240 s, and it rises from 55.9 to 68.5, from fifth of seven to first. The other six had nearly stopped improving, gaining at most 1.7 points between 180 and 240 s. The noise on PhAIL's 480 s scores keeps the width of its published 240 s range.
RoboArena and RoboColosseum's open track publish ratings rather than success rates. We use each rating as the score and its published uncertainty as the noise, and count a RoboArena entry only once it has at least 20 evaluations. RoboColosseum rates each robot separately, so each robot is its own table, and its SO-101 table counts 0.71 times as much as the others because its ranking held up in only 59% of the benchmark's site splits, against about 85% on the other robots. When a benchmark re-runs a model, the new result replaces the old one. When a source cannot be reached, the index keeps the last data it retrieved.
2. Turn each benchmark into head-to-head matchups
Within a benchmark, every pair of models it tested is one matchup. Instead of a plain win or loss, each matchup records how likely it is that one model really scored higher, given how noisy that benchmark is.
qAB is the probability assigned to A outscoring B, sA and sB are their benchmark scores, and σA and σB are the assumed standard deviations of their score noise. Φ is the standard normal cumulative distribution function. Noise describes how much a score could move if the benchmark were run again. On RoboDojo's real robots, π0.5 scores 22.9 and X-VLA 7.6, each with a noise of 2.5 points. The gap is about four times the combined noise, so π0.5 counts as better with near certainty. GR00T N1.7 at 5.9 against π0 at 5.8 counts as 0.51, almost a tie.
Treating the two scores' noise as independent ignores that models tested on the same tasks tend to succeed and fail together. On a shared task set that usually overstates the noise in their difference, which makes matchups softer rather than more decisive, though other patterns of dependence can work the other way. For most benchmarks the noise is the sampling error plus a fixed 1 to 3 points, so a tiny gap between two precisely measured scores does not become a near-certain win. Where an entry has no published interval, RoboLab-120's noise is set to 2 points, and where MolmoSpaces gives no episode count we assume 1,000 episodes. Task-type scores use the same approach, with the sampling error computed from the attempts behind each average.
3. Weight each benchmark
In our index, each benchmark has a benchmark weight, which is how independent its evaluators are multiplied by how repeatable its scores are. We judge and score independence by rubric, and we take repeatability from published repeat runs where they exist and from attempts per task where they do not. Each matchup is scaled by the size of its benchmark to ensure a model’s result counts for roughly the same amount regardless of how many models are included in a benchmark's leaderboard. Benchmarks that publish several separate tables, such as MolmoSpaces' tasks or Poke & Wiggle's separate tables for robot models and agents, split their benchmark weight across those tables, and a model that appears in only one table gets only that table's share.
We set both factors with a fixed rubric. The evaluator factor is 1.0 when the benchmark's own team or an independent operator runs or verifies every entry. It is 0.9 when an independent party runs every entry but the evaluation has a known limit, such as volunteer labs running the comparisons, scoring without blinding, or an operator that fine-tunes the models it tests. It is 0.85 when a competing model's authors help run the benchmark, 0.8 when the operator runs only part of the evaluation, results mix operator runs and submissions, or the operator enters its own model, 0.75 when submitters run their own models and the operator also builds models, 0.7 for results posted in a benchmark's README and 0.4 for numbers self-reported in papers.
The repeatability factor comes from published reliability measurements for RoboChallenge (0.70, its repeat-run correlation) and RoboArena (0.90, its split-half agreement). For the other benchmarks it follows from how many attempts each runs per task and how they are scored, from 0.95 for the largest evaluations down to 0.55 for DataCenterBench, which scores three tries on each of ten tasks with LLM judges. The table below gives both factors for every benchmark. Because these judgments were subjective, we also tested robustness to several different weighting schemes, and found broad agreement between them. Section 9 reports those results in detail.
Simulation results get two further adjustments, adopted on November 11 from our sim-to-real study. In the overall and task-type rankings every simulation benchmark counts half as much as it otherwise would, because simulators share errors and are not independent votes. Each simulator is also scaled by an agreement factor, from 0.88 to 1.17 today, that reflects how often it picks the same winner as other teams' real robots. The simulation ranking uses the agreement factors but not the halving. In the task-type rankings, paper-reported results keep a factor of 1. The sim-to-real agreement tables below list both.
4. Fit one rating per model
Our model uses two ratings to predict a matchup's outcome:
pAB is the predicted probability that A outscores B, and RA and RB are their fitted ratings.
We choose the set of ratings whose predictions best match all the weighted matchup outcomes at once. This is a weighted, regularized Bradley–Terry model fitted to the probabilistic matchups from step 2. Fitting Bradley–Terry ratings to results pooled across many benchmarks has precedent in machine learning, from comparing algorithms across data sets to ranking agents across Kaggle competitions.
Models that never met are placed through the models they share, with shared models acting as bridges between benchmarks. A small pull toward 1500 keeps a model with little evidence from landing at an extreme. The pull works like a normal prior centered on 1500 with a standard deviation of about 550 rating points. We fit the overall, simulation, real-robot and four task-type rankings separately, so a rating only compares models within one setting.
5. Read a rating
A gap between two ratings gives the predicted chance that the higher-rated model outscores the other.
The table above follows from the formula in step 4. We checked these predictions against unseen results, refitting the ratings 23 times, leaving out one benchmark each time, and predicting the order of every pair of models that benchmark tested with different scores, with 1,896 pairs in all. We find the fitted model's predictions strongly correlate with real outcomes. When the model gave the higher-rated model a 90 to 100% chance, that model outscored the other 97% of the time. In the 70 to 80% range it did so 83% of the time, meaning predictions in this band understate the winning model's relative strength. These checks only cover benchmarks like the ones in the index. Relative performance may vary on tasks that are out of distribution.
6. Measure uncertainty
We re-ran the whole fit 300 times. Each re-run draws benchmarks at random with replacement and nudges every score by its noise. A benchmark drawn more than once still counts once, so each re-run keeps about two thirds of the benchmarks and leaves out the rest. Related simulators are drawn together as one unit, the LIBERO family (LIBERO-Pro2.0, LIBERO-Plus and paper-reported results) and the two DROID-Franka simulators (RoboLab-120 and MolmoSpaces), so a blind spot they share cannot count twice. A model's rank range is where its rank falls in 90% of those re-runs, from the 5th to the 95th percentile, rounded down. In each re-run a model is ranked only against the models that re-run still includes. The leaderboard sorts by rating. Sorting by worst plausible rank instead uses the bottom of that range, which penalizes models with uncertain rankings. A model's range comes from the re-runs that include its benchmarks.
A model with only one benchmark in a leaderboard shows no range, since no re-run can test it against a second. When a re-run drops a benchmark that links two groups of models, the groups are placed relative to each other only by the pull toward 1500. In 2,000 separate draws, that happens in under 1% of re-runs overall, 2% in simulation and 3% of real-robot re-runs. Twelve real-robot benchmarks are in the final set. RoboDojo-RC and RoboColosseum's open track test coding agents and robot models on the same robots, and seven of RoboColosseum's models also appear on other benchmarks in the index. Real-robot rank ranges are therefore less reliable than the others.
7. Measure disagreement
For each benchmark in a view, we place the model by percentile among the models we matched there, from 100 for first to 0 for last. Where a benchmark has several tables in the view, its percentile is the average across them. We calculate spread as the sample standard deviation of those percentiles. The leaderboard shows it as ± N under each rating. It measures how much benchmarks disagree about a model, not the uncertainty in its rating, which the rank range covers. Dream-1 finishes first on RoboDojo-RC and in the 28th percentile on RoboColiseum, for a spread of 51. Claude Fable 5.2 finishes in the 91st percentile or higher on all four of its benchmarks, for a spread of 4. A model with one benchmark in the view has no spread. Because percentiles depend on which other models each benchmark tested, spread reflects differences in the level of competition as well as disagreement between benchmarks.
8. Decide who gets a rank
Every model's comparisons inform the full set of ratings, but only some models receive a rank. By default, the overall ranking requires at least one real-robot and one simulation result, the simulation ranking requires two simulated benchmarks, the real-robot ranking includes every model with a real result and the task types follow the overall rule while also requiring two benchmarks within the task type. We hold back entries whose underlying model or agent harness is undisclosed, or that were scored only after using several times the time other entries on the same benchmark were given, until their developers document them. AA-Robotics scores for GPT-6.2 Astra, GPT-6.2 Sol, Claude Fable 5.2, Gemini 4 Hydrogen and Gemini 4 Carbon, published on November 17 as a chart with no harness, effort setting or task scores, are held this way until Artificial Analysis publishes the runs. We also hold back entries that a benchmark itself marks as not following its rules, such as RoboDojo agent-track entries marked with an asterisk. An entry the benchmark also scores at the time limit the others had is not held back; step 1 explains which limit we use. Held entries stay in each benchmark's table but do not enter the fit. We make one exception. Claude Fable 5.2's RoboDojo simulation score comes from Anthropic's own runs in a harness it has not published, and RoboDojo has not listed it. We count it like RoboDojo's own entries and tag it as self-reported in the leaderboard.
9. Check robustness
We refit the overall ranking under alternatives to our input values for benchmark weights, the weight given to simulation, noise and the prior. The table compares every alternative with the published top five, Claude Fable 5.2, SLA-Preview, Dream-1, GPT-6 Astra and GPT-6.1 Sol.
Claude Fable 5.2's first place holds by a wide margin in every alternative, including leaving out RoboDojo-RC, Poke & Wiggle or its self-reported RoboDojo simulation score, and leaving out the whole RoboDojo family. Places two to nine depend more on these choices. SLA-Preview's second place rests on RoboDojo's real-robot track, its only real-robot result, which it leads by 3.4 points, and it stays second or third in 90% of the random reweightings between ½ and 2. Dream-1's place rests on the higher weight for real-robot results. It leads RoboDojo-RC, its only real-robot result, and finishes 24th of 30 on RoboColiseum, its only simulation result, so it leaves the top five when simulation counts three quarters as much as a real result or more. RoboDojo's real-robot track gives SLA-Preview, InternW0-Δ, MWA-Phoenix and OpenWAM-α their only real-robot result, and RoboColiseum gives Dream-1 its only simulation result. PhAIL gives FLUX 3 Action and Cosmos3-Nano-Policy their only real-robot result.
Benchmark weights and noise
Benchmark weight is the evaluator factor multiplied by the repeatability factor, both listed in step 3. The last column is the benchmark's weight in the overall ranking, after the halving and agreement factor for simulators. Giving every benchmark equal weight, with simulation at full weight and no agreement factors, keeps four of the top five, with InternW0-Δ in place of Dream-1, and rank correlation 0.97 overall and 0.90 or higher in every other leaderboard. Section 9 reports wider changes to the weights.
Sim-to-real agreement
Our sim-to-real study found that simulators from different teams often share the same mistakes, and that a simulator picks the same winner as another team's real robot about as often as two real-robot benchmarks from different teams agree. We publish these rates with every update, recomputed on the index's current data. A benchmark decides a matchup when its score gap would survive a rerun at least 90% of the time under the noise in step 2, and two benchmarks agree when both decide a matchup and pick the same winner. We never compare two benchmarks run by the same team, and we split paper-reported results by suite.
The agreement factor pulls each simulator's agreement toward the pooled rate, as if it had 71 more matchups at that rate, then rescales it so a simulator at the pooled rate gets 1 and one at chance gets 0, within limits of 0.25 and 1.5. Simulators with fewer than 10 decided matchups keep a factor of 1. The differences between simulators are still within noise, so the factors stay close to 1 for now and will spread as shared real-robot results accumulate. Listing per-task checkpoints separately raised overall agreement from 75% to 79% at the time.