page = f'''<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>Characterizing the Sim-to-Real Gap | Playground research</title><meta name="description" content="{esc(DESC)}">
<link rel="canonical" href="https://www.withplayground.ai/blog/sim-to-real-gap">
<meta name="robots" content="index, follow, max-image-preview:large">
<meta property="og:type" content="article"><meta property="og:site_name" content="Playground">
<meta property="og:title" content="Characterizing the Sim-to-Real Gap"><meta property="og:description" content="{esc(DESC)}">
<meta property="og:url" content="https://www.withplayground.ai/blog/sim-to-real-gap">
<meta property="article:published_time" content="2026-11-10">
<meta name="twitter:card" content="summary_large_image"><meta name="twitter:title" content="Characterizing the Sim-to-Real Gap"><meta name="twitter:description" content="{esc(DESC)}">
<meta name="theme-color" content="#fbf7f1">
<link rel="stylesheet" href="styles.css"><link rel="stylesheet" href="post.css?v=14"><script src="post.js?v=14" defer></script>
</head>
<body class="has-toc">
<a class="skip" href="#intro">Skip to the paper</a>
<header class="nav playground-header"><a href="/" class="nav__home" aria-label="Playground home"><span class="logo">{LOGO}<span class="logo__word">playground</span></span></a><nav class="blog-nav" aria-label="Main navigation"><a class="blog-nav-link blog-nav-link--plain" href="/#robot-intelligence-index">Leaderboard</a><a class="blog-nav-link" href="/blog" aria-current="location">Blog</a><a href="/#waitlist" class="btn btn--dark btn--small"><span class="cta-long">Request an evaluation</span><span class="cta-short">Request an eval</span></a></nav></header>
<aside class="article-toc"><details open><summary>Contents</summary><nav aria-label="Contents">
<a href="#top" class="toc-level-0">Abstract</a>
<a href="#intro" class="toc-level-0">1. Introduction</a>
<a href="#data" class="toc-level-0">2. Data and method</a>
<a href="#matched" class="toc-level-0">3. Matched simulators track real rankings</a>
<a href="#setups" class="toc-level-0">4. Rankings vary across benchmarks</a>
<a href="#aggregate" class="toc-level-0">5. Aggregation stabilizes rankings</a>
<a href="#real" class="toc-level-0">6. Where real robots matter</a>
<a href="#agents" class="toc-level-0">7. LLM agents on real robots</a>
<a href="#index" class="toc-level-0">8. Implications for our index methodology</a>
<a href="#conclusion" class="toc-level-0">9. Conclusion</a>
<a href="#refs" class="toc-level-0">References</a>
<a href="#appendix" class="toc-level-0">Appendix</a>
</nav></details></aside>
<main id="top">

<section class="hero wrap">
<h1>Characterizing the<br>Sim-to-Real Gap</h1>
<div class="byline"><span>Playground research</span><time datetime="2026-11-10">November 10, 2026</time><span>Analysis</span></div>
<div class="opening-copy prose">
<p class="lead lead-abstract">We measure the sim-to-real gap across the {N['benchmarks']} public benchmarks behind the <a href="/blog/robot-intelligence-index">Robot Intelligence Index</a>: {cm(N['scores'])} published scores for {N['policies']} policies, and {N['paired_results']} paired simulated and real results from seven validation studies. We find that rankings from simulators matched to a real setup agree closely with the real-robot ranking, and that substantial ranking variation occurs across benchmark setups (environment, embodiment, assets) in both simulated and real-world evals.</p>
<div class="x-shell abstract-table"><div class="x-scroll"><table class="x">
<thead><tr><th>Measure</th><th class="num">Result</th><th class="num">Compared with</th></tr></thead>
<tbody>
<tr class="grp"><th colspan="3">Same setup</th></tr>
<tr><th>Mirrored simulator vs. its real setup<span class="sub">Rank correlation, builder-run studies</span></th><td class="num"><span class="v">{f2(TW['spearman'])}</span></td><td class="num ref"><span class="v">{f2(TW['rerun_spearman'])}</span><span class="sub">Modeled repeat of the real test</span></td></tr>
<tr class="grp"><th colspan="3">Across setups</th></tr>
<tr><th>Simulator vs. another team’s real robot<span class="sub">Same winner, {sr['n_matchups']} matchups</span></th><td class="num"><span class="v">{pct(sr['agree'])}</span><span class="sub">90% CI {p0(sr['lo'])}–{p0(sr['hi'])}%</span></td><td class="num ref"><span class="v">{pct(rr['agree'])}</span><span class="sub">Two real benchmarks from different teams, {rr['n_matchups']} matchups, 90% CI {p0(rr['lo'])}–{p0(rr['hi'])}%</span></td></tr>
<tr class="grp"><th colspan="3">Aggregation</th></tr>
<tr><th>Two halves of the index<span class="sub">Rank correlation</span></th><td class="num"><span class="v">{f2(SH['split_half_all'][0])}</span></td><td class="num ref"><span class="v">{f2(SH['single_pair_median'])}</span><span class="sub">Two single benchmarks</span></td></tr>
<tr><th>Simulation vs. real-robot rating<span class="sub">{IX_2['n']} policies on two or more benchmarks of each kind</span></th><td class="num"><span class="v">{f2(IX_2['sim_real'][0])}</span></td><td class="num ref"><span class="v">{f2(IX_1['sim_real'][0])}</span><span class="sub">{IX_1['n']} policies on one benchmark of a kind</span></td></tr>
<tr><th>The same {IX_2['n']} policies, one benchmark of each kind<span class="sub">Our refit, median of 200 draws</span></th><td class="num"><span class="v">{f2(COVT['thin_median'])}</span></td><td class="num ref"><span class="v">{f2(COVT['full'])}</span><span class="sub">Full coverage, our refit</span></td></tr>
<tr class="grp"><th colspan="3">New model families</th></tr>
<tr><th>World-action models (WAMs) vs. VLAs<span class="sub">Real-robot matchups won</span></th><td class="num"><span class="v">{FAM['WAM', 'VLA', 'real']['won']} of {FAM['WAM', 'VLA', 'real']['n']}</span></td><td class="num ref"><span class="v">{pct(FAM['WAM', 'VLA', 'real']['pred'])}</span><span class="sub">Simulation forecast</span></td></tr>
</tbody></table></div></div>
<p class="lead lead-abstract">In mirrored studies, we find that simulated and real rankings agree as closely as a repeat of the real-robot evaluation would. When setups vary, agreement falls for simulation and real robots alike: simulation evals pick the same winner as a separate team’s real-robot evals at approximately the same rate as two real-robot benchmarks from separate teams.</p>
<p class="lead lead-abstract">Aggregating benchmarks stabilizes rankings. Rankings built from two disjoint halves of our index correlate at {f2(SH['split_half_all'][0])}, against {f2(SH['single_pair_median'])} for two single benchmarks from different teams. Holding the policies fixed shows the effect of coverage: for the {WORD.get(COVT['n'], COVT['n']).lower()} best-covered policies, simulation-only and real-only ratings from our refit of the index correlate at {f2(COVT['full'])}, and rebuilding each rating from one benchmark of each kind lowers that to {f2(COVT['thin_median'])}.</p>
<p class="lead lead-abstract">Real robot evals remain key for benchmarking success rates, identifying shared errors in simulation, and for evaluating new model families. WAMs, for example, win every one of their real-robot matchups against VLAs, far more often than simulation forecasts. Real-robot testing is also unevenly spread: LLM agents make up {pct(LA["real"])} of real-robot entries against {pct(LA["sim"])} of simulation entries, mostly through two small benchmarks that test only agents, and no benchmark running {LA["big"]} or more trials per model tests one.</p>
</div>
</section>

<div class="article-shell wrap"><article class="prose">
<section id="intro">
<h2>1. Introduction</h2>
<p>Separating two policies 10 points apart takes hundreds of real trials each (<a href="#ref-step">STEP 2025</a>; <a href="#ref-kress">Kress-Gazit et al. 2024</a>), and nearly every new policy reports simulation scores instead. Whether those scores rank policies the way real robots would is the evaluation form of the reality gap (<a href="#ref-jakobi">Jakobi et al. 1995</a>; <a href="#ref-koos">Koos et al. 2013</a>).</p>
<p>Prior work tests one simulator at a time. Each study validates a simulator against its builders’ own robot, usually on four to nine policies, and reports correlations near 0.9 (<a href="#ref-kadian">Kadian et al. 2020</a>; <a href="#ref-simpler">Li et al. 2024</a>; <a href="#ref-polaris">PolaRiS 2025</a>; <a href="#ref-genie">Genie Sim 3.0 2026</a>). Two questions remain open. How well do simulators predict real robots run by other teams, on the policies the field actually uses? And how reliable is the real reference itself?</p>
<p>We answer both with the data behind the Robot Intelligence Index, which collects every public robot leaderboard active in 2026. We compare benchmarks on the matchups both decide confidently, and group each comparison by whether the setups match (a simulator that mirrors a real setup, or one team’s simulator and robots running the same tasks and policies) and by whether each side is simulated or real. Figure 1 states our results. When the setup matches, simulation predicts real-robot results closely. When it does not, simulators and other teams’ real-robot benchmarks miss at similar rates, picking the wrong winner in roughly a quarter to a third of matchups.</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 1. How often a result predicts the winner on a real robot</span></div><div class="legend-key"><span><i style="background:#4f82e0"></i>Simulation predicting real</span><span><i style="background:#7f828d"></i>Reference</span></div></div>
{fig1}
{CM(1)}</div>
<p class="figure-note">Share of matchups, decided by both sources, in which the predictor picks the real benchmark’s winner. 90% intervals from resampling policies.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>The result changes what a leaderboard should measure. If a policy’s rank depends on its setup, no single benchmark, simulated or real, measures how good the policy is. Our index aggregates comparisons across setups for this reason. The sections below show that simulation ranks correctly when the setup matches (Section 3), that setups differ enough to reorder policies (Section 4), that aggregation recovers a stable ranking (Section 5), where real robots remain necessary (Section 6), and how real-robot testing is spread across model families (Section 7).</p>
</section>

<section id="data">
<h2>2. Data and method</h2>
<p>Our data is the index snapshot of {DATE_LONG} (Table 1), plus every published study that scores the same policies on the same tasks in simulation and on a real robot.</p>
</section>
</article></div>

<section class="wrap combined-results narrow-table">
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Data</th><th class="num">Count</th></tr></thead>
<tbody>{data_rows}</tbody></table></div></div>
<p class="figure-note"><strong>Table 1.</strong> Robot Intelligence Index snapshot, {DATE_LONG}, and the published validation studies.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>Our unit of comparison is the <em>matchup</em>: two policies scored on the same benchmark. A benchmark <em>decides</em> a matchup when its score gap would survive a rerun at least 90% of the time, under the index’s noise model. Two benchmarks agree on a matchup when both decide it and pick the same winner. We never compare two benchmarks run by the same team, except inside RoboDojo, where we hold the team fixed on purpose. For paired results we report the rank correlation of simulated and real scores within each task. <a href="#appendix">The appendix</a> gives details and robustness checks.</p>
</section>

<section id="matched">
<h2>3. Matched simulators closely track real rankings</h2>
<p>We audited all {N['sources_audited']} sources in the index, including papers, appendices and code, for the same policies scored in simulation and on a matching real setup. Three simulators mirror a specific real setup: SimplerEnv, PolaRiS and Genie Sim 3.0. Together they provide {TW['cells']} paired results over {TW['units']} tasks and {TW['policies']} policies.</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 2. Mirrored simulators track real rankings</span></div><div class="legend-key"><span><i style="background:#4f82e0"></i>Mirrored simulator</span><span><i style="background:#7f828d"></i>Comparison</span></div></div>
{fig2}
{CM(3)}</div>
<p class="figure-note">Mean Spearman correlation of simulated and real scores within each task, with 90% intervals.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>In the mirrored studies, mean within-task rank correlation is {f2(TW['spearman'])}, close to the {f2(TW['rerun_spearman'])} expected for a repeat evaluation under our trial-count noise model. All {TW['n_calls']} matchups confidently decided by both simulation and reality agree. This supports simulation as a relative evaluation tool in these matched settings, especially for clearly separated policies. It does not establish equivalent reliability for close comparisons or unseen model families: the rerun reference is modeled, and the mirrored studies were conducted by their builders on limited policy sets.</p>
<p>Two measured references complement the modeled one. A second real arm inside RoboDojo agrees with a held-out arm in {pct(X['rd_arm']['agree'])} of decided matchups, while the one study that rebuilt a real lab elsewhere, with four policies and five trials per cell, reproduced the original ranking at only {f2(RP['spearman'])} (<a href="#ref-replica">VLA-Replica 2026</a>). How closely the simulator matches the real setup is the factor these studies vary directly. On the same real tasks, unmatched simulators reach {f2(CO['spearman'])}, and simulators that share only task names reach {f2(SN['spearman'])}.</p>
<p>Simulation also tracks real results closely inside one team’s protocol. RoboDojo trains {X['rd_policies']} policies with one recipe and evaluates them in simulation and on three real arms. Its simulator predicts a held-out arm in {pct(X['rd_sim']['agree'])} of decided matchups, against {pct(X['rd_arm']['agree'])} for another real arm.</p>
<p>Within these studies, what the policy sees appears to drive the match. PolaRiS’s photoreal replicas correlate with real results at only {POLARIS_NO_COFT:.2f} before a brief fine-tuning step in simulation, and its builders trace the error to small visual differences. SimplerEnv rises from {f2(DT['SimplerEnv | Google Robot, variant aggregation']['spearman'])} to {f2(DT['SimplerEnv | Google Robot, visual matching']['spearman'])} when its visuals match the real scene.</p>
</section>

<section id="setups">
<h2>4. Rankings vary across benchmarks</h2>
<p>Most simulators mirror no real setup, and most real benchmarks share none with each other. Across teams, agreement falls for both kinds of evidence (Figure 1). A public simulator agrees with another team’s real robot on {pct(sr['agree'])} of {sr['n_matchups']} decided matchups. Two real-robot benchmarks from different teams agree on {pct(rr['agree'])} of {rr['n_matchups']}. On the matchups where both are available, a simulator and another team’s real benchmark predict a target real benchmark at similar rates ({pct(m['sim'])} and {pct(m['real'])}). Similar is not the same as equivalent. The difference is {sg(m['diff'])} points with a 90% interval of −{p0(abs(m['lo']))} to {sg(m['hi'])}, which cannot exclude simulation being up to {p0(abs(m['lo']))} points worse. Pinning the difference to within 10 points would take roughly {EQN} shared matchups, against {m['cases']} today. The only earlier pairwise figure, from Habitat navigation, sits close to ours at about 75% (<a href="#ref-kadian">Kadian et al. 2020</a>).</p>
<p>The disagreements are not noise. Score noise accounts for {X['noise_only']*100:.0f} points of the sim–real disagreement, and in {pct(dis['share99'])} of the matchups where simulation and reality pick opposite winners, the simulator was at least 99% sure. We isolate the noise share by computing, for each of the {sr['n_matchups']} sim–real matchups, the probability that a rerun of either benchmark would reverse its call under the index’s noise model, and averaging. They reflect systematic differences between benchmarks. Coarse features of a benchmark pair do not account for them: pairs that share a robot, a training recipe or a scoring type agree no more often than pairs that do not (<a href="#app-h">Appendix H</a>).</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 3. One policy, many rankings</span></div><div class="legend-key"><span><i style="background:#4f82e0"></i>Simulation</span><span><i style="background:#f28b74"></i>Real robot</span></div></div>
{chart0}
{CM(0)}</div>
<p class="figure-note">Each policy’s finish on each benchmark, from last to first. Policies with four or more benchmarks, ordered by index rating.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>The effect on rankings is large. Among the {SW['n_3plus']} policies tested on three or more benchmarks, the median gap between best and worst finish is {SW['median_swing_3plus']:.0f} percentile points. Of the {SW['top3_somewhere']} policies that place top three somewhere, {SW['top3_and_bottom_half']} place in the bottom half elsewhere. π0.5 finishes first on ArmnetBench and last on PhAIL.</p>
<p class="finding">Cross-setup disagreement is substantial in both simulated and real evaluation. A result from another real lab is therefore not automatically a more reliable guide to a target benchmark than a simulation result. These comparisons do not isolate the contribution of physics fidelity: benchmarks also differ in tasks, embodiments, policy training, and evaluation protocols. Our evidence supports treating benchmark context as a major source of ranking variation, without identifying which differences cause it.</p>
</section>

<section id="aggregate">
<h2>5. Aggregating benchmarks stabilizes rankings</h2>
<p>If performance depends on setup, the quantity worth measuring is performance across setups. This is what motivates the Robot Intelligence Index. We view the index as a first step towards aggregate performance measurement. It compares policies only within each benchmark, where the setup is shared, and fits one rating per policy to all comparisons across {N['benchmarks']} benchmarks with a weighted Bradley–Terry model (<a href="#ref-bt">Bradley and Terry 1952</a>). Two tests compare the aggregate with single benchmarks (Figure 4), and a third holds the policies fixed.</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 4. Aggregated rankings agree more than single benchmarks</span></div><div class="legend-key"><span><i style="background:#4f82e0"></i>Index</span><span><i style="background:#7f828d"></i>Comparison</span></div></div>
{fig4}
{CM(4)}</div>
<p class="figure-note">Spearman rank correlations. Top: two rankings built from separate benchmarks. Bottom: the index fitted on simulation only and on real robots only, for the {IX_ALL['n']} policies in its default ranking.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>Ratings built from disjoint halves of the benchmark teams have a median correlation of {f2(SH['split_half_all'][0])}, compared with {f2(SH['single_pair_median'])} between individual benchmarks from different teams. Simulation-only and real-only ratings correlate at {f2(IX_2['sim_real'][0])} among {WORD.get(IX_2['n'], IX_2['n']).lower()} policies evaluated on at least two benchmarks of each kind, versus {f2(IX_1['sim_real'][0])} among {IX_1['n']} policies with thinner coverage. These patterns are consistent with aggregation improving ranking stability, as in machine-learning benchmarking, where an aggregate reference ranks models more stably than any single benchmark (<a href="#ref-perlitz">Perlitz et al. 2024</a>). However, the coverage groups contain different policies and their uncertainty intervals overlap, so that comparison alone does not show that coverage produces the improvement. Holding the policies fixed does. For the same {WORD.get(COVT['n'], COVT['n']).lower()} well-covered policies, rebuilding each rating from one randomly drawn benchmark of each kind lowers the correlation from {f2(COVT['full'])} in our refit to a median of {f2(COVT['thin_median'])} (90% of draws between {f2(COVT['thin_lo'])} and {f2(COVT['thin_hi'])}). The results support aggregation as a promising measurement strategy whose reliability should be tracked as coverage expands.</p>
</section>

<section id="real">
<h2>6. Where real robots remain necessary</h2>
<p>Aggregation averages over differences between benchmarks. Four things still need real-robot results.</p>
<p><strong>Success rates.</strong> Mirrored simulators track the order more closely than the level. On average they underestimate real success rates by {TW['level']:.0f} points (90% CI {TW['ci']['level'][0]:.0f}–{TW['ci']['level'][1]:.0f}), as earlier manipulation twins do (<a href="#ref-simpler">Li et al. 2024</a>). This average measures directional bias, not absolute error: larger errors on individual tasks can offset one another. Deployment decisions need real trials.</p>
<p><strong>Errors simulators share.</strong> Simulators from different teams agree with each other on {pct(ss['agree'])} of decided matchups, far more often than they agree with real robots, and part of that agreement is shared error. On the {SE['matchups']} real-robot matchups two or more simulators decide, a simulator picks the wrong winner {pct(SE['p_wrong'])} of the time, and {pct(SE['p_wrong_given_other_wrong'])} of the time when another simulator is already wrong. {WORD.get(se_fav['X-VLA'] + se_fav['MolmoAct2'], se_fav['X-VLA'] + se_fav['MolmoAct2'])} of the {len(SEROWS)} shared mistakes over-rate X-VLA or MolmoAct2, the leaders of the saturated LIBERO, SimplerEnv and CALVIN suites (<a href="#ref-liberopro">LIBERO-PRO 2025</a>; <a href="#ref-liberoplus">LIBERO-Plus 2025</a>). Adding simulators does not average this error out.</p>
<p><strong>New model families.</strong> Our simulation-based rating forecast underestimates WAMs’ pairwise wins against VLAs on the available real benchmarks (Figure 5). WAMs win all {FAM['WAM', 'VLA', 'real']['n']} observed matchups, compared with a forecast of {pct(FAM['WAM', 'VLA', 'real']['pred'])}. On other teams’ simulators, the WAMs tested on real robots also beat their forecast, by {p0(S38['sim_tested']['gap'])} points, while WAMs never tested on real robots match theirs ({sg(S38['sim_untested']['gap'])}). Compared like for like, the discrepancy is {p0(S38['both_diff']['diff'])} points larger on real robots (90% CI {sg(S38['both_diff']['ci'][0])} to {sg(S38['both_diff']['ci'][1])}). The forecast understates these WAMs everywhere, and most on real robots. The WAM gap persists when we compare only with VLAs that have equally thin simulation coverage, compare only with VLAs released after the first WAM, or drop any one WAM or benchmark (<a href="#app-d">Appendix D</a>). LLM agents beat their forecast against VLAs by similar margins in simulation and on real robots ({sg(LVS['gap'])} and {sg(LV['gap'])} points), so for them the error lies in the forecast, not the transfer. In success rates, WAMs lead the same VLAs by {WG['real']['mean']:.0f} points on real robots and {WG['sim']['mean']:.0f} points in simulation (<a href="#app-h">Appendix H</a>). This is evidence of a family-associated forecast error in our sample. These checks narrow the competing explanations, but they do not establish whether the cause is simulation fidelity, coverage, or differences in the evaluated tasks and checkpoints.</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 5. Simulation-based forecasts understate WAMs</span></div><div class="legend-key"><span><i class="ring"></i>Simulation forecast</span><span><i style="background:#4f82e0"></i>Observed in sim</span><span><i style="background:#f28b74"></i>Observed on real robots</span></div></div>
{fig5}
{CM(5)}</div>
<p class="figure-note">WAMs’ win rate against VLAs, forecast by the index’s rating fitted on other teams’ simulators. Values are observed minus forecast, in points.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>For WAMs, evals in sim often identify the winner, while the aggregate forecast understates WAMs’ frequency of pairwise wins. Individually they pick the real winner in {PT['wam'][0]} of {PT['wam'][1]} of these matchups, against {PT['vla_same_sims'][0]} of {PT['vla_same_sims'][1]} VLA-against-VLA matchups on the same simulators. The miss is largest where a model has one outside simulator: Dream-1, rated only by RoboColiseum, was forecast at {pct(COVW['Dream-1']['pred'])} and won every real matchup. A matched simulator does better. RoboDojo’s own simulator ranks its two WAMs first and second, as its real arms do, and decides all {RDW['VLA-WAM']['n']} of their matchups against VLAs correctly. Real results also fix the index: with them, WAMs outrate {pct(A25_W['index'])} of co-rated policies, as on real robots ({pct(A25_W['real'])}); simulation alone puts them at {pct(A25_W['sim'])}.</p>
<p><strong>A specific robot.</strong> No aggregate predicts the quirks of one lab’s robot. On a held-out real benchmark, the index and a single other benchmark pick the winner equally often ({pct(AG['composite, all'][0])} and {pct(AG['single benchmark (average)'][0])}). To know how a policy performs on a particular robot, tests with that specific setup are needed. RoboColosseum takes a first step in this direction by rating policies separately on each robot it runs: a Franka, a two-arm YAM and an SO-101.</p>
<p class="finding">Matched to a real setup, simulation tracks relative rankings closely. Real robot results set success rates, catch the errors simulators share, and test model families simulators were not built for.</p>
</section>

<section id="agents">
<h2>7. LLM agents are over-represented on small real-robot benchmarks</h2>
<p>Real-robot tests are scarce, so which policies receive them shapes what real results can show. The index covers {A37['policies']['Agent']} LLM agents: general-purpose language models, such as GPT-6 Astra and Claude Opus 5.5, that control a robot through a software harness. By count, agents receive more than their share of real-robot testing. They are {pct(LA['pol'])} of the policies with a result and {pct(LA['sim'])} of simulation entries, but {pct(LA['real'])} of real-robot entries. Of the {LA['cov']['Agent'][1]} agents, {LA['cov']['Agent'][0]} have a real-robot result, against {LA['cov']['VLA'][0]} of {LA['cov']['VLA'][1]} VLAs and {LA['cov']['WAM'][0]} of {LA['cov']['WAM'][1]} WAMs.</p>
<p>The surplus reflects which benchmarks test agents, not how they are tested: each benchmark runs all its policies for the same number of trials. Agents appear on {len(A37['real_with_agents'])} of the {A37['n_real']} real-robot benchmarks (Figure 6). Two of them, AA-Robotics and DataCenterBench, test only agents, at {LA['trials_aa']} and {LA['trials_dc']} trials per model, the fewest of any real-robot benchmark. Without these two, agents are {pct(LA['ex'])} of real-robot entries, below their share of policies. Two evaluators, Robocurve and Artificial Analysis, supply {LA['two_ops']} of agents’ {AE['real']['Agent']} real-robot entries, and no benchmark running {LA['big']} or more trials per model tests an agent.</p>
</section>
</article></div>

<section class="wrap combined-results">
<div class="chart-box"><div class="chart-head"><div class="chart-l"><span class="chart-title">Figure 6. LLM agents are tested mostly on the smallest real-robot benchmarks</span></div><div class="legend-key"><span><i class="uc-key ag"></i>LLM agent</span><span><i class="uc-key"></i>Robot policy (VLA, WAM or baseline)</span></div></div>
{fig6}
{CM(6)}</div>
<p class="figure-note">Each dot is one policy on one real-robot benchmark. Benchmarks are ordered by real trials per model. RoboArena and RoboColosseum’s open track rate policies by pairwise comparison instead.</p>
</section>

<div class="article-shell wrap"><article class="prose">
<section>
<p>The concentration thins the evidence behind agents’ real-robot results in two ways. Their entries carry {pct(LA['trials'])} of the real trials behind the index, so each result is less certain, and the index’s noise model weights it less. And the comparison that places agents against robot policies on the same robot rests on two benchmarks, RoboDojo-RC and RoboColosseum’s open track, with {LA['meet']} head-to-head pairs between them. Our findings do not depend on agents: excluding every matchup that involves one, simulation agrees with another team’s real robot on {pct(LA['sr_no'])} of decided matchups, the same rate as with them. Nor does agents’ standing depend on the agent-only benchmarks: refitting the index without them keeps {WORD[LA['top10_no']].lower()} agents in its top ten (<a href="#app-f">Appendix F</a>). Real-robot coverage is thinnest for the family real-robot forecasts miss most: only {LA['cov']['WAM'][0]} of {LA['cov']['WAM'][1]} WAMs have a real-robot result (Section 6).</p>
<p class="finding">LLM agents are over-represented on real robots by count, not by evidence. Most of their real-robot results come from two small benchmarks that test only agents, and the large real-robot benchmarks test none. Until those benchmarks include agents, how LLM agents compare with robot policies on real hardware rests on two benchmarks.</p>
</section>

<section id="index">
<h2>8. Implications for our index methodology</h2>
<p>The index already compares policies only within benchmarks and aggregates across benchmarks, the design these results favor. Based on this analysis, we’ll be rolling out four changes to our index methodology in the coming weeks:</p>
<ul>
<li><strong>Treat simulators as related evidence.</strong> Ten simulators that share a blind spot are not ten independent votes. The overall rating will weight real-robot results more, and rank ranges will resample related simulators together.</li>
<li><strong>Weight each simulator by how well it tracks held-out real results</strong> (<a href="#app-c">Appendix C</a>) once shared real matchups can tell simulators apart (<a href="#app-g">Appendix G</a>), alongside evaluator independence and repeatability.</li>
<li><strong>List per-task checkpoints separately,</strong> such as the models some developers fine-tuned for each RoboChallenge task.</li>
<li><strong>Publish the index’s sim-to-real agreement</strong> with every update, by model family, starting with the numbers here.</li>
</ul>
<p>For benchmark builders our evidence suggests that evaluation requires measurement of the same models across many different environments. Our results also imply that having more simulation episodes does not help resolve the sim2real disagreement, given that most of the sim2real misses were already high-confidence (<a href="#app-e">Appendix E</a>).</p>
</section>

<section id="conclusion">
<h2>9. Conclusion</h2>
<p>In the studies we analyzed, matched simulators often reproduce real-policy rankings closely. Across teams and setups, both simulated and real benchmarks disagree, making a single benchmark an incomplete measure of policy performance. Aggregating benchmarks yields encouraging ranking consistency, although coverage and shared errors limit what that agreement establishes. Real-robot evaluation remains necessary to validate absolute success rates and detect failures of simulation-based forecasts, particularly for new model families. The practical goal is to measure when simulation predicts real outcomes reliably and where additional real evidence changes the assessment.</p>
</section>

<section id="refs" class="refs">
<h2>References</h2>
<ol>
<li id="ref-bt">Bradley, R. A., and Terry, M. E. (1952). Rank analysis of incomplete block designs. <i>Biometrika</i> 39.</li>
<li id="ref-genie">Genie Sim 3.0 (2026). Simulation benchmark behind AgiBot’s RoboColiseum. <a href="https://arxiv.org/abs/2601.02078">arXiv:2601.02078</a>.</li>
<li id="ref-jakobi">Jakobi, N., Husbands, P., and Harvey, I. (1995). Noise and the reality gap: The use of simulation in evolutionary robotics. <i>European Conference on Artificial Life</i>.</li>
<li id="ref-kadian">Kadian, A., et al. (2020). Sim2Real predictivity: Does evaluation in simulation predict real-world performance? <i>IEEE Robotics and Automation Letters</i>. <a href="https://arxiv.org/abs/1912.06321">arXiv:1912.06321</a>.</li>
<li id="ref-koos">Koos, S., Mouret, J.-B., and Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. <i>IEEE Transactions on Evolutionary Computation</i>. <a href="https://arxiv.org/abs/1307.1870">arXiv:1307.1870</a>.</li>
<li id="ref-kress">Kress-Gazit, H., et al. (2024). Robot learning as an empirical science: Best practices for policy evaluation. <a href="https://arxiv.org/abs/2409.09491">arXiv:2409.09491</a>.</li>
<li id="ref-simpler">Li, X., et al. (2024). Evaluating real-world robot manipulation policies in simulation (SimplerEnv). <a href="https://arxiv.org/abs/2405.05941">arXiv:2405.05941</a>.</li>
<li id="ref-liberopro">LIBERO-PRO (2025). Perturbation tests of the LIBERO suite. <a href="https://arxiv.org/abs/2510.03827">arXiv:2510.03827</a>.</li>
<li id="ref-liberoplus">LIBERO-Plus (2025). Camera, layout and initial-state perturbations of LIBERO. <a href="https://arxiv.org/abs/2510.13626">arXiv:2510.13626</a>.</li>
<li id="ref-perlitz">Perlitz, Y., et al. (2024). BenchBench: benchmark agreement testing for language models. <a href="https://arxiv.org/abs/2407.13696">arXiv:2407.13696</a>.</li>
<li id="ref-playground">Playground (2026). Introducing the Robot Intelligence Index. <a href="/blog/robot-intelligence-index">withplayground.ai</a>.</li>
<li id="ref-polaris">PolaRiS (2025). Real-to-sim evaluation with Gaussian-splat replicas of real DROID scenes. <a href="https://arxiv.org/abs/2512.16881">arXiv:2512.16881</a>.</li>
<li id="ref-step">STEP (2025). Sequential testing for robot policy evaluation. <a href="https://arxiv.org/abs/2503.10966">arXiv:2503.10966</a>.</li>
<li id="ref-replica">VLA-Replica (2026). A real-robot benchmark rebuilt in a second lab. <a href="https://arxiv.org/abs/2605.20774">arXiv:2605.20774</a>.</li>
</ol>
</section>

<section id="appendix" class="methods"><details><summary><h2>Appendix: method, robustness and limitations</h2></summary><div>
<h3 id="app-a">A. Definitions</h3>
<p>A benchmark decides a matchup when Φ(gap / noise) ≥ 0.9, using the index’s published noise for each score. Each pair of policies counts once within a comparison. Intervals come from 2,000–4,000 bootstrap resamples of policies. Ratings use our refit of the index’s weighted Bradley–Terry model, which reproduces the published ratings at 0.998. We split the paper-reported results into LIBERO, SimplerEnv WidowX, SimplerEnv Google and CALVIN, and PhAIL into its two tracks, which gives {N['leaderboards']} leaderboards from {N['benchmarks']} benchmarks. The finishes in Figure 3 come from each benchmark’s published ranks.</p>
<p><em>Split-half test.</em> We randomly divide the benchmark teams in two, fit the rating on each half and correlate the halves; {f2(SH['split_half_all'][0])} is the median of {SH['split_half_all'][3]} splits. The held-out test refits the rating without each real benchmark’s team and compares its winners with single benchmarks on the same matchups.</p>
<h3 id="app-b">B. Same-setup sources</h3>
<p>Paired results come from published tables, parsed by our own scripts and checked against each paper’s reported correlations: SimplerEnv Tables IV–V (leaf tasks), PolaRiS’s per-scene data file, Genie Sim 3.0 Table I, MolmoBot Tables 6–7, RoboLab-120 v2 Table V (removed by its authors in v3 without comment), RoboDojo’s per-task leaderboard and VLA-Replica. We average Spearman correlations within tasks; the rerun reference redraws each real score from its trial count. PolaRiS is scored after its brief sim fine-tuning; the two twins without any reach {f2(DTN['spearman'])}, and dropping any one source leaves 0.89–0.94.</p>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Source<span class="sub">simulation · real</span></th><th>Same setup?</th><th>What exists</th><th>Fine-tuned in sim?</th><th class="num">Used</th></tr></thead>
<tbody>{''.join(audit_rows)}</tbody></table></div></div>
<h3 id="app-c">C. Each simulator against real robots</h3>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Simulation benchmark</th><th class="num">Agreement with real robots</th><th>90% interval</th><th class="num">Rank correlation with real rating</th></tr></thead>
<tbody>{''.join(bench_rows)}</tbody></table></div></div>
<p>Against real-robot results from other teams. Too few shared policies to judge: {few_txt} matchups. RoboTwin 2.0 has the highest agreement among simulators with enough matchups, but the differences between simulators are within noise (<a href="#app-g">Appendix G</a>). Paper-reported LIBERO and SimplerEnv Google show no positive link to other teams’ robots, although SimplerEnv ranks policies well against the setups it mirrors.</p>
<h3 id="app-d">D. Model-family forecast</h3>
<p>Families follow the index’s model type. For each real benchmark, the forecast is the index rating refitted on eight team-run simulators (RoboDojo sim, RoboTwin 2.0, RoboLab-120, MolmoSpaces, RoboCasa365, RoboColiseum, EBench and Embodied First Exam), leaving out every benchmark from the target’s team. Rating gaps convert to win probabilities with the Elo curve; observed win probabilities are Φ(gap / noise). The real-robot excess of {sg(WV['gap'])} points holds when we drop any one WAM ({sg(WV['lomo_min'])} to {sg(WV['lomo_max'])}) or benchmark ({sg(min(LOBO))} to {sg(max(LOBO))}), weaken the rating prior ({sg(CW['real'])}), add paper-reported suites ({sg(CSTD['real'])}), restrict to VLAs with equally thin simulation coverage ({sg(COV['gap_low_cov_vla'])}), or restrict to VLAs released after the first WAM ({sg(RC['recent_vla'])}). LLM coding agents beat the forecast against VLAs by similar margins in simulation and on real robots ({sg(LVS['gap'])} and {sg(LV['gap'])} points), which places their error in the forecast rather than the transfer. The like-for-like comparison in Section 6 uses the {WORD[len(S38['both_diff']['wams'])].lower()} WAMs with forecast pairs in both domains ({', '.join(S38['both_diff']['wams'])}). Developers choose which checkpoints reach some real benchmarks. On RoboDojo, where the benchmark team chooses and trains every policy, WAMs win all {S38['robodojo']['n']} matchups against VLAs, against a forecast of {pct(S38['robodojo']['pred'])}; on benchmarks that run developer-supplied checkpoints they win all {S38['developer']['n']}, against {pct(S38['developer']['pred'])}. Removing every LLM agent from every benchmark and refitting leaves the real-robot excess at {sg(S38['real_no_agents']['gap'])} points. {WORD.get(len(RD_DROP), len(RD_DROP))} policies on RoboDojo ({', '.join(RD_DROP)}) have no outside simulation results and drop out of its forecast.</p>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Simulator</th><th class="num">WAM vs. VLA</th><th class="num">VLA vs. VLA</th></tr></thead>
<tbody>{''.join(dir_rows)}</tbody></table></div></div>
<p>Share of real-robot matchups a real benchmark decides in which the simulator’s score gap picks the real winner.</p>
<h3 id="app-e">E. Design comparisons</h3>
<p>Versions of the same benchmark on the same policies and real matchups, changing one design choice at a time. These comparisons are observational, not randomized.</p>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Change</th><th>Comparison</th><th>Result</th><th>Evidence</th></tr></thead>
<tbody>{iv_rows}</tbody></table></div></div>
<h3 id="app-f">F. Robustness</h3>
<div class="x-shell"><div class="x-scroll"><table class="x"><thead><tr><th>Setting</th><th class="num">Sim ↔ real agreement</th></tr></thead><tbody>{rob_rows}</tbody></table></div></div>
<p>Without a confidence threshold, so that close calls count, sim ↔ real agreement is {pct(X['all_matchups']['sr'])} over {X['all_matchups']['sr_n']} matchups and real ↔ real {pct(X['all_matchups']['rr'])} over {X['all_matchups']['rr_n']} (difference {sg(X['all_matchups']['diff'])} points, 90% CI {sg(X['all_matchups']['lo'])} to {sg(X['all_matchups']['hi'])}). Leaving out one benchmark at a time keeps sim ↔ real agreement between {pct(X['lobo']['lo'])} and {pct(X['lobo']['hi'])}.</p>
<p><em>LLM agents.</em> Excluding every matchup that involves an LLM agent leaves sim ↔ real agreement at {pct(LA['sr_no'])} ({LA['sr_no_n']} matchups) and sim ↔ sim at {pct(LA['ss_no'])}. Real ↔ real agreement falls from {pct(LA['rr_all'])} to {pct(LA['rr_no'])} over {LA['rr_no_n']} matchups (90% CI {p0(LA['rr_lo'])}–{p0(LA['rr_hi'])}%), within its wide interval. Refitting the index without the three benchmarks that test only agents (AA-Robotics, DataCenterBench and the simulated Embodied First Exam) leaves {WORD[LA['top10_no']].lower()} agents in the top ten, as in the full refit. Trial shares exclude the two pairwise-rated benchmarks. Agent entries are counted per benchmark; the figure counts RoboColosseum’s two tracks and PhAIL’s two task sets separately.</p>
<h3 id="app-g">G. Ranking engines and single benchmarks</h3>
<p>The data cannot rank physics engines. Pooled by engine, simulators pick another team’s real winner {pct(GE['Isaac']['agree'])} of the time on Isaac Sim and Isaac Lab ({GE['Isaac']['benchmarks']} benchmarks, {hu(GE['Isaac']['n'])} matchups), {pct(GE['SAPIEN']['agree'])} on SAPIEN ({GE['SAPIEN']['benchmarks']}, {hu(GE['SAPIEN']['n'])}) and {pct(GE['MuJoCo/robosuite']['agree'])} on MuJoCo ({GE['MuJoCo/robosuite']['benchmarks']}, {hu(GE['MuJoCo/robosuite']['n'])}). A permutation test finds no difference (p = {fp(G['engine_p'])}). The spread within an engine exceeds the spread between engines: RoboTwin 2.0 and paper-reported SimplerEnv Google both run on SAPIEN and agree {pct(G['sim_bench']['robotwin']['agree'])} and {pct(G['sim_bench']['simpler_g']['agree'])} of the time. Who runs a benchmark and how it is built matter more than its physics engine. Engines follow each benchmark’s documentation; MolmoSpaces assets also load in other simulators, and its benchmark runs in MuJoCo.</p>
<p>Nor can the data rank single simulators by how well they predict real robots. Each simulator decides {hu(G['sim_n_range'][0])} to {hu(G['sim_n_range'][1])} matchups against other teams’ robots. Telling 95% agreement from 70% with 80% power takes about {hu(G['need_95_70'])} matchups per benchmark once shared policies are accounted for, and telling 85% from 70% takes about {hu(G['need_85_70'])}. Differences across the {len(G['sim_bench'])} simulators fall short of significance (p = {fp(G['sim_bench_p'])}) under a permutation test that, ignoring shared policies, already overstates them. Direct comparison fails for the same reason: only {G['h2h_disc5']} of {len(G['h2h'])} simulator pairs disagree with each other on five or more of the matchups they share.</p>
<p>A weaker question has an answer: which benchmarks agree with the rest of the index. We refit the index without each benchmark’s team and score the benchmark’s decided matchups against the refit, using every matchup it decides rather than only those a real robot also decides. Expected agreement comes from the refit’s rating gap, so a benchmark that compares only distant policies earns no credit for it. RoboTwin 2.0 agrees {p0(abs(GA['robotwin']['excess']))} points more often than expected over {GA['robotwin']['n']} matchups; RoboChallenge agrees {p0(abs(GA['robochallenge']['excess']))} points less often over {GA['robochallenge']['n']}. MolmoSpaces and RoboCasa365 also clear expectation on fewer matchups. Embodied First Exam and DataCenterBench clear it on {GA['efe']['n']} matchups each between coding agents rated far apart, which we do not count as evidence. Every other benchmark, real or simulated, is indistinguishable from the rest.</p>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Benchmark</th><th class="num">Decided matchups</th><th class="num">Agreement with the index</th><th class="num">Points above expected</th><th>90% interval</th></tr></thead>
<tbody>{''.join(g_rows)}</tbody></table></div></div>
<p>Agreement with an index refitted without the benchmark’s team, on matchups the benchmark decides and the refit rates. Expected agreement is a logistic function of the refit’s rating gap, fitted on all benchmarks together; intervals resample policies. Agreement with the index is not agreement with real robots. The rest of the index is mostly simulation, and a benchmark that shares the simulators’ blind spots scores well here. RoboChallenge differs from the index in design as well as outcome.</p>
<h3 id="app-h">H. Checks added after review</h3>
<p><em>Coverage, same policies.</em> For the {COVT['n']} policies with two or more simulation and two or more real benchmarks, we refit the simulation-only and real-only ratings 200 times, each time keeping one randomly drawn benchmark of each kind for these policies (every other policy keeps all its results). The correlation falls from {f2(COVT['full'])} to a median of {f2(COVT['thin_median'])}, and {pct(COVT['share_below'])} of draws fall below the full-coverage value.</p>
<p><em>Benchmark-pair features.</em> Across all {sum(v['n_yes'] + v['n_no'] for v in list(RC6['features'].values())[:1])} decided cross-team matchups, simulated and real, agreement does not rise when the two benchmarks share a robot, both train every policy with one recipe, or score the same way. In a joint logistic fit only the pair’s kind matters: simulator–real and real–real pairs agree less than simulator–simulator pairs. The features are coarse, and the analysis shows association only.</p>
<div class="x-shell"><div class="x-scroll"><table class="x">
<thead><tr><th>Benchmark pair</th><th class="num">Agree when true</th><th class="num">Agree when false</th><th class="num">Difference, points</th></tr></thead>
<tbody>{feat_rows}</tbody></table></div></div>
<p><em>WAM success rates.</em> On benchmarks scored in percent, for the {WG['real']['n']} WAM–VLA pairs also observed in simulation, WAMs score {WG['real']['mean']:.0f} points higher on real robots (90% CI {WG['real']['lo']:.0f}–{WG['real']['hi']:.0f}) and {WG['sim']['mean']:.0f} points higher in simulation ({WG['sim']['lo']:.0f}–{WG['sim']['hi']:.0f}). The real-robot figure rests on {WORD.get(WG['real']['benches'], WG['real']['benches']).lower()} benchmarks.</p>
<p><em>Close calls in forecasts.</em> Hiding each policy’s real results and forecasting it from simulation alone, the index picks the real winner in {pct(NEAR['acc'])} of the {NEAR['n']} matchups where the two policies are rated within 150 points of each other.</p>
<h3 id="app-i">I. Limitations</h3>
<ul>
<li>Real-to-real agreement rests on {rr['n_matchups']} matchups (90% CI {p0(rr['lo'])}–{p0(rr['hi'])}%). The matched comparison on shared matchups and the RoboDojo test carry the setup result alongside it.</li>
<li>The coverage comparison in Figure 4 rests on {IX_2['n']} well-covered policies, and its intervals overlap ({f2(IX_2['sim_real'][1])}–{f2(IX_2['sim_real'][2])} against {f2(IX_1['sim_real'][1])}–{f2(IX_1['sim_real'][2])}).</li>
<li>Counting only decided matchups favors large gaps. Twins never confidently pick the wrong winner; close calls between strong policies still need many real trials, and simulation-based forecasts for closely rated policies are little better than a coin flip (<a href="#app-h">Appendix H</a>).</li>
<li>Each twin was validated by its own builders on three to six policies per task, mostly older policies in SimplerEnv and checkpoints of a single model in Genie Sim 3.0. None of the paired datasets includes a WAM; for WAMs, same-setup evidence is RoboDojo alone, whose simulator calls all {RDW['VLA-WAM']['n']} matchups between its two WAMs and VLAs correctly. An independent rerun of one mirrored twin, on policies its builders did not test, is the most direct next test.</li>
<li>The model-family result rests on {NW} WAMs and {FAM['WAM', 'VLA', 'real']['n']} real matchups; its size is uncertain ({sg(WV['lo'])} to {sg(WV['hi'])} points). Developers choose where to submit on some real benchmarks. RoboDojo trains every policy itself, and WAMs lead there too.</li>
<li>Several DROID benchmarks share policies and reference data, which makes their comparisons less independent than their count suggests. Coding agents’ results mix harness and model. Agreement between benchmarks is not ground truth.</li>
</ul>
</div></details></section>
</article></div>

<section class="evidence wrap" id="downloads"><div><span class="mono">REPRODUCE THE ANALYSIS</span><h2>Data and code.</h2><p>Every number on this page is generated from these files.</p></div><div class="download-list">
<a href="data/results.json" download><span>All results</span><span class="mono">JSON ↗</span></a>
<a href="data/sim-real-matchups.csv" download><span>Every sim ↔ real matchup</span><span class="mono">CSV ↗</span></a>
<a href="data/model-family-matchups.csv" download><span>Every cross-family matchup and its forecast</span><span class="mono">CSV ↗</span></a>
<a href="data/same-setup-pairs.csv" download><span>Every same-setup sim and real result</span><span class="mono">CSV ↗</span></a>
<a href="data/benchmark-audit.json" download><span>Benchmark setup audit</span><span class="mono">JSON ↗</span></a>
<a href="data/index-data-{DATE_ISO}.json" download><span>Index data snapshot</span><span class="mono">JSON ↗</span></a>
<a href="data/analysis-code.zip" download><span>Analysis code</span><span class="mono">ZIP ↗</span></a>
</div></section>

<div class="article-shell wrap"><article class="prose"><section>
<p class="feedback">Send feedback or corrections to <a href="mailto:contact@withplayground.ai">contact@withplayground.ai</a>.</p>
</section></article></div>
</main>
<footer class="footer wrap"><span class="brand">playground / research</span><p>November 10, 2026<br>All numbers come from public leaderboards and papers collected for the Robot Intelligence Index.</p><a href="#top">Back to top ↑</a></footer>
</body></html>
'''
