Blender MCP · weighted verifier review

Blender tasks.
Explicit verifier.

Ten long-horizon Blender MCP tasks scored from 0 to 1. Each case presents a concise parent rubric first, with every atomic check available on demand.

10Tasks reviewed
0.459Mean reward
118Parent terms
360Atomic checks
1Invalid inputs
396Rollout turns
333Tool calls
Verifier V3 in plain language

Every valid task receives partial credit from 0 to 1.

Each task divides one reward point across its active categories. The page groups repetitive atomic checks into a smaller set of parent criteria for readability; expanding a parent reveals the original checks and exact allocated credit.

Scene semanticsRequired roles, hierarchy, materials, and relative spatial relationships.
Actual behaviorObject transforms are sampled across instructed animation phases, not inferred from action metadata.
DeliverablesThe requested Blend, GLB, JSON reports, and named render set each contribute partial credit.
Verifier re-renderThe submitted Blend is reopened at a task-relevant frame. A blank result loses this rule's points only.
Asset integrityThe GLB receives credit for parsing, import round-trip, animation, and retained semantic content.
FORMULA: reward = sum(category weight × category pass fraction) / sum(active category weights). INVALID_INPUT remains N/A because a broken task package is not a rollout outcome. Grouping changes presentation only, never scoring. Exact numeric tolerances and raw measurements remain hidden.
CASE 01

Neon Last Mile

WEIGHTED VERIFIER REWARD0.920/ 1.000
RULES RECEIVING CREDIT32/34completed
Evaluation input
# Neon Last Mile

Create a rainy neon alley at night. A compact autonomous courier drives from the
alley entrance, steers around two construction barriers, and stops beneath a DELIVERY
sign. Its cargo hatch opens at the destination.

The scene must include a wet road, modular facades on both sides, visible utility
pipes, neon signs, the courier, two barriers, a route curve, and hero/side/top cameras.
Animate frames 1-180: travel during 1-140 and hatch opening during 145-175.

Deliver `output/final.blend`, `output/final.glb`, four renders, and
`output/validation.json`. Use only Blender built-ins and procedural geometry.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

31 rollout turns 31 tool calls 4 tool errors 0.920 reward / 1.000
Category reward breakdown0.920 / 1.000
Scene structure18/18100.0% · 0.250/0.250 · 25% weightSpatial relationships1/1100.0% · 0.200/0.200 · 20% weightAnimation behavior4/580.0% · 0.200/0.250 · 25% weightDeliverable integrity5/5100.0% · 0.150/0.150 · 15% weightRendered evidence4/580.0% · 0.120/0.150 · 15% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure25% of total · 0.250 / 0.250
ResultVerifier checkEarned / max
CREDITRequired scene entitiesTask contract0.056 / 0.056
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired scene object exists: ROAD WetTask contract0.014 / 0.014
CREDITRequired scene object exists: COURIER BodyTask contract0.014 / 0.014
CREDITRequired scene object exists: COURIER HatchTask contract0.014 / 0.014
CREDITRequired scene object exists: ROUTE CourierTask contract0.014 / 0.014
CREDITSemantic organization and coverageTask contract0.097 / 0.097
View 7 atomic checks · 7/7 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired semantic collection exists: ENVIRONMENTTask contract0.014 / 0.014
CREDITRequired semantic collection exists: COURIERTask contract0.014 / 0.014
CREDITRequired semantic collection exists: LIGHTINGTask contract0.014 / 0.014
CREDITScene contains the required amount of FACADE * geometryTask contract0.014 / 0.014
CREDITScene contains the required amount of SIGN Neon * geometryTask contract0.014 / 0.014
CREDITScene contains the required amount of PIPE * geometryTask contract0.014 / 0.014
CREDITScene contains the required amount of BARRIER * geometryTask contract0.014 / 0.014
CREDITScene metadata and camera setupTask contract0.028 / 0.028
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe scene contains the required camera coverageTask contract0.014 / 0.014
CREDITRequired scene metadata is recorded: showcase case idTask contract0.014 / 0.014
CREDITObject types, hierarchy, and materialsBehavior rubric0.069 / 0.069
View 5 atomic checks · 5/5 receiving credit
ResultAtomic verifier checkEarned / max
CREDITObject has the required Blender type: ROAD WetBehavior rubric0.014 / 0.014
CREDITObject has the required Blender type: ROUTE CourierBehavior rubric0.014 / 0.014
CREDITRequired parent-child relationship is correct for COURIER HatchBehavior rubric0.014 / 0.014
CREDITMaterials are assigned to ROAD Wet,COURIER BodyBehavior rubric0.014 / 0.014
CREDITMaterials are assigned to SIGN Neon *Behavior rubric0.014 / 0.014
Spatial relationships20% of total · 0.200 / 0.200
ResultVerifier checkEarned / max
CREDITKey spatial outcomesBehavior rubric0.200 / 0.200
Animation behavior25% of total · 0.200 / 0.250
ResultVerifier checkEarned / max
CREDITTimeline coverageTask contract0.050 / 0.050
PARTIALRequested animation sequenceBehavior rubric0.150 / 0.200
View 4 atomic checks · 3/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe courier visibly travels through the delivery routeBehavior rubric0.050 / 0.050
CREDITThe courier settles at the delivery destinationBehavior rubric0.050 / 0.050
CREDITThe courier hatch opens after arrival rather than before itBehavior rubric0.050 / 0.050
NO CREDITAll four courier wheels visibly rotate during travelBehavior rubric0.000 / 0.050
Deliverable integrity15% of total · 0.150 / 0.150
ResultVerifier checkEarned / max
CREDITExternal asset resolutionDelivery contract0.030 / 0.030
CREDITRequired deliverable filesDelivery contract0.060 / 0.060
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired deliverable exists: final.blendDelivery contract0.030 / 0.030
CREDITRequired deliverable exists: final.glbDelivery contract0.030 / 0.030
CREDITExported asset integrityDelivery contract0.060 / 0.060
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe exported GLB retains the required scene content and animationDelivery contract0.030 / 0.030
CREDITThe exported GLB can be parsed and imported back successfullyDelivery contract0.030 / 0.030
Rendered evidence15% of total · 0.120 / 0.150
ResultVerifier checkEarned / max
PARTIALRequired rendered view setDelivery contract0.090 / 0.120
View 4 atomic checks · 3/4 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.030
CREDITRequired render is present and non-degenerate: hero.pngDelivery contract0.030 / 0.030
CREDITRequired render is present and non-degenerate: side.pngDelivery contract0.030 / 0.030
CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.030 / 0.030
CREDITIndependent verifier renderDelivery contract0.030 / 0.030
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute1 TOOL ERROR
T4blender_execute
T5blender_execute1 TOOL ERROR
T6blender_execute
T7blender_execute
T8blender_execute1 TOOL ERROR
T9blender_execute
T10blender_execute
T11blender_execute1 TOOL ERROR
T12blender_execute
T13blender_execute
T14blender_execute
T15blender_screenshot
T16blender_execute
T17blender_screenshot
T18blender_execute
T19blender_execute
T20blender_screenshot
T21blender_execute
T22blender_execute
T23blender_execute
T24blender_execute
T25blender_execute
T26blender_execute
T27blender_execute
T28blender_screenshot
T29blender_execute
T30blender_screenshot
T31finish
Task structural contract
{
  "case_id": "01-neon-last-mile",
  "required_collections": ["ENVIRONMENT", "COURIER", "LIGHTING"],
  "required_objects": ["ROAD_Wet", "COURIER_Body", "COURIER_Hatch", "ROUTE_Courier"],
  "object_counts": [
    {"pattern": "FACADE_*", "min": 6},
    {"pattern": "SIGN_Neon_*", "min": 8},
    {"pattern": "PIPE_*", "min": 6},
    {"pattern": "BARRIER_*", "min": 2, "max": 2}
  ],
  "animated_objects": [{"pattern": "COURIER_*", "min": 2}],
  "min_cameras": 3,
  "min_frame_end": 180,
  "scene_properties": {"showcase_case_id": "01-neon-last-mile"}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 02

Orbital Repair Bay

WEIGHTED VERIFIER REWARD0.000/ 1.000
RULES RECEIVING CREDIT0/35blocked
Evaluation input
# Orbital Repair Bay

Build a cinematic orbital maintenance bay around a damaged communications satellite.
Two articulated repair arms coordinate a repair: one stabilizes the satellite while
the other removes a damaged panel and installs a replacement. Include Earth as a
curved background, bay trusses, work lights, warning beacons, and drifting particles.

Animate frames 1-240 with a readable approach, grasp, panel swap, and inspection
sequence. All arm joints must have explicit hierarchy or constraints. Deliver an
editable Blend, GLB, hero/wide/top renders, and validation JSON.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal observed viewport
No inspectable GLBThe evaluated rollout did not produce a final 3D asset.
48 rollout turns 48 tool calls 4 tool errors 0.000 reward / 1.000
Category reward breakdown0.000 / 1.000
Scene structure0/10.0% · 0.000/0.300 · 30% weightSpatial relationships0/10.0% · 0.000/0.200 · 20% weightAnimation behavior0/10.0% · 0.000/0.250 · 25% weightDeliverable integrity0/30.0% · 0.000/0.150 · 15% weightRendered evidence0/50.0% · 0.000/0.100 · 10% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure30% of total · 0.000 / 0.300
ResultVerifier checkEarned / max
NOT EVALUABLERequired scene entitiesTask contract0.000 / 0.071
View 4 atomic checks · 0/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired scene object exists: SATELLITE CoreTask contract0.000 / 0.018
NOT EVALUABLERequired scene object exists: PANEL DamagedTask contract0.000 / 0.018
NOT EVALUABLERequired scene object exists: PANEL ReplacementTask contract0.000 / 0.018
NOT EVALUABLERequired scene object exists: EARTH BackdropTask contract0.000 / 0.018
NOT EVALUABLESemantic organization and coverageTask contract0.000 / 0.141
View 8 atomic checks · 0/8 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired semantic collection exists: BAYTask contract0.000 / 0.018
NOT EVALUABLERequired semantic collection exists: SATELLITETask contract0.000 / 0.018
NOT EVALUABLERequired semantic collection exists: ARM ATask contract0.000 / 0.018
NOT EVALUABLERequired semantic collection exists: ARM BTask contract0.000 / 0.018
NOT EVALUABLEScene contains the required amount of ARM A Joint * geometryTask contract0.000 / 0.018
NOT EVALUABLEScene contains the required amount of ARM B Joint * geometryTask contract0.000 / 0.018
NOT EVALUABLEScene contains the required amount of TRUSS * geometryTask contract0.000 / 0.018
NOT EVALUABLEScene contains the required amount of BEACON * geometryTask contract0.000 / 0.018
NOT EVALUABLEScene metadata and camera setupTask contract0.000 / 0.035
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe scene contains the required camera coverageTask contract0.000 / 0.018
NOT EVALUABLERequired scene metadata is recorded: showcase case idTask contract0.000 / 0.018
NOT EVALUABLEObject types, hierarchy, and materialsBehavior rubric0.000 / 0.053
View 3 atomic checks · 0/3 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEArticulated hierarchy is complete: ARM A Mount->ARM A Joint 0->ARM A Joint 1->ARM A Joint 2->ARM A Joint 3->ARM A Joint 4Behavior rubric0.000 / 0.018
NOT EVALUABLEArticulated hierarchy is complete: ARM B Mount->ARM B Joint 0->ARM B Joint 1->ARM B Joint 2->ARM B Joint 3->ARM B Joint 4Behavior rubric0.000 / 0.018
NOT EVALUABLEMaterials are assigned to required scene objectsBehavior rubric0.000 / 0.018
Spatial relationships20% of total · 0.000 / 0.200
ResultVerifier checkEarned / max
NOT EVALUABLEKey spatial outcomesBehavior rubric0.000 / 0.200
View 3 atomic checks · 0/3 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEObject scale and dimensions are task-appropriate: EARTH BackdropBehavior rubric0.000 / 0.067
NOT EVALUABLEThe replacement panel finishes at the intended mountBehavior rubric0.000 / 0.067
NOT EVALUABLEThe damaged panel finishes clear of the mountBehavior rubric0.000 / 0.067
Animation behavior25% of total · 0.000 / 0.250
ResultVerifier checkEarned / max
NOT EVALUABLETimeline coverageTask contract0.000 / 0.050
NOT EVALUABLERequested animation sequenceBehavior rubric0.000 / 0.200
View 4 atomic checks · 0/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERepair arm A articulates across the repair sequenceBehavior rubric0.000 / 0.050
NOT EVALUABLERepair arm B articulates across the repair sequenceBehavior rubric0.000 / 0.050
NOT EVALUABLEThe damaged satellite panel moves away from its mountBehavior rubric0.000 / 0.050
NOT EVALUABLEThe replacement panel moves into the original mountBehavior rubric0.000 / 0.050
Deliverable integrity15% of total · 0.000 / 0.150
ResultVerifier checkEarned / max
NOT EVALUABLEExternal asset resolutionDelivery contract0.000 / 0.030
NO CREDITRequired deliverable filesDelivery contract0.000 / 0.060
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITRequired deliverable exists: final.blendDelivery contract0.000 / 0.030
NO CREDITRequired deliverable exists: final.glbDelivery contract0.000 / 0.030
NOT EVALUABLEExported asset integrityDelivery contract0.000 / 0.060
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe exported GLB retains the required scene content and animationDelivery contract0.000 / 0.030
NOT EVALUABLEThe exported GLB can be parsed and imported back successfullyDelivery contract0.000 / 0.030
Rendered evidence10% of total · 0.000 / 0.100
ResultVerifier checkEarned / max
NOT EVALUABLERequired rendered view setDelivery contract0.000 / 0.080
View 4 atomic checks · 0/4 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.020
NO CREDITRequired render is present and non-degenerate: hero.pngDelivery contract0.000 / 0.020
NOT EVALUABLERequired render is present and non-degenerate: side.pngDelivery contract0.000 / 0.020
NO CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.000 / 0.020
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.020
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute1 TOOL ERROR
T4blender_execute
T5blender_execute1 TOOL ERROR
T6blender_execute
T7blender_execute
T8blender_execute
T9blender_screenshot
T10blender_execute
T11blender_execute1 TOOL ERROR
T12blender_execute
T13blender_execute
T14blender_screenshot
T15blender_execute
T16blender_execute
T17blender_screenshot
T18blender_execute
T19blender_execute
T20blender_execute
T21blender_screenshot
T22blender_execute1 TOOL ERROR
T23blender_execute
T24blender_screenshot
T25blender_execute
T26blender_screenshot
T27blender_execute
T28blender_screenshot
T29blender_execute
T30blender_screenshot
T31blender_execute
T32blender_screenshot
T33blender_execute
T34blender_screenshot
T35blender_execute
T36blender_screenshot
T37blender_execute
T38blender_screenshot
T39blender_execute
T40blender_screenshot
T41blender_execute
T42blender_screenshot
T43blender_execute
T44blender_screenshot
T45blender_execute
T46blender_execute
T47blender_screenshot
T48blender_execute
Task structural contract
{
  "case_id": "02-orbital-repair-bay",
  "required_collections": ["BAY", "SATELLITE", "ARM_A", "ARM_B"],
  "required_objects": ["SATELLITE_Core", "PANEL_Damaged", "PANEL_Replacement", "EARTH_Backdrop"],
  "object_counts": [
    {"pattern": "ARM_A_Joint_*", "min": 4},
    {"pattern": "ARM_B_Joint_*", "min": 4},
    {"pattern": "TRUSS_*", "min": 6},
    {"pattern": "BEACON_*", "min": 4}
  ],
  "animated_objects": [{"pattern": "ARM_*", "min": 4}, {"pattern": "PANEL_*", "min": 1}],
  "min_cameras": 3,
  "min_frame_end": 240,
  "scene_properties": {"showcase_case_id": "02-orbital-repair-bay"}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 03

Kinetic Watch

WEIGHTED VERIFIER REWARD0.000/ 1.000
RULES RECEIVING CREDIT0/37blocked
Evaluation input
# Kinetic Watch

Create a premium skeleton watch with a readable case, crystal, dial ring, hands,
crown, movement plates, balance wheel, and at least eight gears. Produce an exploded
animation that separates the assembly along its depth axis and returns every part to
its original transform without intersection.

Frames 1-60 show the assembled watch, 61-150 explode it, 151-240 hold and orbit the
camera, and 241-330 reassemble it. The balance wheel and gear train continue moving.
Deliver macro hero, exploded, side, and top renders plus Blend, GLB, and validation.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutNo visual evidence was produced
No inspectable GLBThe evaluated rollout did not produce a final 3D asset.
48 rollout turns 11 tool calls 1 tool errors 0.000 reward / 1.000
Category reward breakdown0.000 / 1.000
Scene structure0/10.0% · 0.000/0.250 · 25% weightSpatial relationships0/10.0% · 0.000/0.150 · 15% weightAnimation behavior0/10.0% · 0.000/0.300 · 30% weightDeliverable integrity0/30.0% · 0.000/0.150 · 15% weightRendered evidence0/60.0% · 0.000/0.150 · 15% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure25% of total · 0.000 / 0.294
ResultVerifier checkEarned / max
NOT EVALUABLERequired scene entitiesTask contract0.000 / 0.088
View 6 atomic checks · 0/6 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired scene object exists: WATCH CaseTask contract0.000 / 0.015
NOT EVALUABLERequired scene object exists: WATCH CrystalTask contract0.000 / 0.015
NOT EVALUABLERequired scene object exists: WATCH CrownTask contract0.000 / 0.015
NOT EVALUABLERequired scene object exists: Balance WheelTask contract0.000 / 0.015
NOT EVALUABLERequired scene object exists: Hand HourTask contract0.000 / 0.015
NOT EVALUABLERequired scene object exists: Hand MinuteTask contract0.000 / 0.015
NOT EVALUABLESemantic organization and coverageTask contract0.000 / 0.103
View 7 atomic checks · 0/7 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired semantic collection exists: CASETask contract0.000 / 0.015
NOT EVALUABLERequired semantic collection exists: MOVEMENTTask contract0.000 / 0.015
NOT EVALUABLERequired semantic collection exists: HANDSTask contract0.000 / 0.015
NOT EVALUABLERequired semantic collection exists: CAMERASTask contract0.000 / 0.015
NOT EVALUABLEScene contains the required amount of Gear * geometryTask contract0.000 / 0.015
NOT EVALUABLEScene contains the required amount of Screw * geometryTask contract0.000 / 0.015
NOT EVALUABLEScene contains the required amount of Movement Plate * geometryTask contract0.000 / 0.015
NOT EVALUABLEScene metadata and camera setupTask contract0.000 / 0.029
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe scene contains the required camera coverageTask contract0.000 / 0.015
NOT EVALUABLERequired scene metadata is recorded: showcase case idTask contract0.000 / 0.015
NOT EVALUABLEObject types, hierarchy, and materialsBehavior rubric0.000 / 0.074
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEObject has the required Blender type: WATCH CaseBehavior rubric0.000 / 0.015
NOT EVALUABLEObject has the required Blender type: WATCH CrystalBehavior rubric0.000 / 0.015
NOT EVALUABLEObject has the required Blender type: Balance WheelBehavior rubric0.000 / 0.015
NOT EVALUABLEMaterials are assigned to Gear *Behavior rubric0.000 / 0.015
NOT EVALUABLEMaterials are assigned to required scene objectsBehavior rubric0.000 / 0.015
Animation behavior30% of total · 0.000 / 0.353
ResultVerifier checkEarned / max
NOT EVALUABLETimeline coverageTask contract0.000 / 0.059
NOT EVALUABLERequested animation sequenceBehavior rubric0.000 / 0.294
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe watch casing components separate and reassembleBehavior rubric0.000 / 0.059
NOT EVALUABLEThe watch movement separates and reassemblesBehavior rubric0.000 / 0.059
NOT EVALUABLEThe gear train remains animated through the sequenceBehavior rubric0.000 / 0.059
NOT EVALUABLEThe balance wheel continues oscillatingBehavior rubric0.000 / 0.059
NOT EVALUABLEThe presentation camera performs the requested orbitBehavior rubric0.000 / 0.059
Deliverable integrity15% of total · 0.000 / 0.176
ResultVerifier checkEarned / max
NOT EVALUABLEExternal asset resolutionDelivery contract0.000 / 0.035
NO CREDITRequired deliverable filesDelivery contract0.000 / 0.071
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITRequired deliverable exists: final.blendDelivery contract0.000 / 0.035
NO CREDITRequired deliverable exists: final.glbDelivery contract0.000 / 0.035
NOT EVALUABLEExported asset integrityDelivery contract0.000 / 0.071
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe exported GLB retains the required scene content and animationDelivery contract0.000 / 0.035
NOT EVALUABLEThe exported GLB can be parsed and imported back successfullyDelivery contract0.000 / 0.035
Rendered evidence15% of total · 0.000 / 0.176
ResultVerifier checkEarned / max
NO CREDITRequired rendered view setDelivery contract0.000 / 0.147
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.029
NO CREDITRequired render is present and non-degenerate: hero.pngDelivery contract0.000 / 0.029
NO CREDITRequired render is present and non-degenerate: exploded.pngDelivery contract0.000 / 0.029
NO CREDITRequired render is present and non-degenerate: side.pngDelivery contract0.000 / 0.029
NO CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.000 / 0.029
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.029
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute
T4blender_execute
T5blender_execute
T6blender_execute
T7blender_execute
T8blender_execute
T9blender_execute
T10blender_execute1 TOOL ERROR
T11assistant turn
T12assistant turn
T13assistant turn
T14blender_execute
T15assistant turn
T16assistant turn
T17assistant turn
T18assistant turn
T19assistant turn
T20assistant turn
T21assistant turn
T22assistant turn
T23assistant turn
T24assistant turn
T25assistant turn
T26assistant turn
T27assistant turn
T28assistant turn
T29assistant turn
T30assistant turn
T31assistant turn
T32assistant turn
T33assistant turn
T34assistant turn
T35assistant turn
T36assistant turn
T37assistant turn
T38assistant turn
T39assistant turn
T40assistant turn
T41assistant turn
T42assistant turn
T43assistant turn
T44assistant turn
T45assistant turn
T46assistant turn
T47assistant turn
T48assistant turn
Task structural contract
{
  "case_id": "03-kinetic-watch",
  "required_collections": ["CASE", "MOVEMENT", "HANDS", "CAMERAS"],
  "required_objects": ["WATCH_Case", "WATCH_Crystal", "WATCH_Crown", "Balance_Wheel", "Hand_Hour", "Hand_Minute"],
  "object_counts": [
    {"pattern": "Gear_*", "min": 8},
    {"pattern": "Screw_*", "min": 6},
    {"pattern": "Movement_Plate_*", "min": 2}
  ],
  "animated_objects": [{"pattern": "Gear_*", "min": 4}, {"pattern": "WATCH_*", "min": 2}],
  "min_cameras": 3,
  "min_frame_end": 330,
  "scene_properties": {"showcase_case_id": "03-kinetic-watch"}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 04

Deployable Rescue Drone

WEIGHTED VERIFIER REWARD0.000/ 1.000
RULES RECEIVING CREDIT0/41blocked
Evaluation input
# Deployable Rescue Drone

Design a compact quadcopter carried as a rectangular backpack module. Four folded arms
deploy symmetrically, rotor blades unfold, landing feet extend, and a winch lowers a
medical package onto a marked landing zone in a storm-damaged street.

Animate frames 1-220: arm deployment, rotor startup, short flight, hover, winch drop,
and stable package placement. Show folded and deployed dimensions with measurement
objects. Deliver Blend, GLB, four views, and validation JSON.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal observed viewport
No inspectable GLBThe evaluated rollout did not produce a final 3D asset.
48 rollout turns 46 tool calls 11 tool errors 0.000 reward / 1.000
Category reward breakdown0.000 / 1.000
Scene structure0/10.0% · 0.000/0.250 · 25% weightSpatial relationships0/10.0% · 0.000/0.200 · 20% weightAnimation behavior0/10.0% · 0.000/0.300 · 30% weightDeliverable integrity0/30.0% · 0.000/0.150 · 15% weightRendered evidence0/60.0% · 0.000/0.100 · 10% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure25% of total · 0.000 / 0.250
ResultVerifier checkEarned / max
NOT EVALUABLERequired scene entitiesTask contract0.000 / 0.045
View 4 atomic checks · 0/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired scene object exists: DRONE CoreTask contract0.000 / 0.011
NOT EVALUABLERequired scene object exists: WINCH CableTask contract0.000 / 0.011
NOT EVALUABLERequired scene object exists: PAYLOAD MedicalTask contract0.000 / 0.011
NOT EVALUABLERequired scene object exists: LANDING TargetTask contract0.000 / 0.011
NOT EVALUABLESemantic organization and coverageTask contract0.000 / 0.080
View 7 atomic checks · 0/7 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired semantic collection exists: DRONETask contract0.000 / 0.011
NOT EVALUABLERequired semantic collection exists: PAYLOADTask contract0.000 / 0.011
NOT EVALUABLERequired semantic collection exists: RESCUE SITETask contract0.000 / 0.011
NOT EVALUABLEScene contains the required amount of ARM * geometryTask contract0.000 / 0.011
NOT EVALUABLEScene contains the required amount of ROTOR * geometryTask contract0.000 / 0.011
NOT EVALUABLEScene contains the required amount of FOOT * geometryTask contract0.000 / 0.011
NOT EVALUABLEScene contains the required amount of RUBBLE * geometryTask contract0.000 / 0.011
NOT EVALUABLEScene metadata and camera setupTask contract0.000 / 0.023
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe scene contains the required camera coverageTask contract0.000 / 0.011
NOT EVALUABLERequired scene metadata is recorded: showcase case idTask contract0.000 / 0.011
NOT EVALUABLEObject types, hierarchy, and materialsBehavior rubric0.000 / 0.102
View 9 atomic checks · 0/9 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired parent-child relationship is correct for ARM 1Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ARM 2Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ARM 3Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ARM 4Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ROTOR 1Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ROTOR 2Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ROTOR 3Behavior rubric0.000 / 0.011
NOT EVALUABLERequired parent-child relationship is correct for ROTOR 4Behavior rubric0.000 / 0.011
NOT EVALUABLEMaterials are assigned to required scene objectsBehavior rubric0.000 / 0.011
Spatial relationships20% of total · 0.000 / 0.200
ResultVerifier checkEarned / max
NOT EVALUABLEKey spatial outcomesBehavior rubric0.000 / 0.200
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe payload begins attached to or directly below the droneBehavior rubric0.000 / 0.100
NOT EVALUABLEThe payload finishes on the designated landing targetBehavior rubric0.000 / 0.100
Animation behavior30% of total · 0.000 / 0.300
ResultVerifier checkEarned / max
NOT EVALUABLETimeline coverageTask contract0.000 / 0.050
NOT EVALUABLERequested animation sequenceBehavior rubric0.000 / 0.250
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEAll four drone arms deployBehavior rubric0.000 / 0.050
NOT EVALUABLEAll four drone rotors spinBehavior rubric0.000 / 0.050
NOT EVALUABLEThe drone follows a visible flight pathBehavior rubric0.000 / 0.050
NOT EVALUABLEThe rescue payload is lowered from the droneBehavior rubric0.000 / 0.050
NOT EVALUABLEThe payload is stable at the end of the sequenceBehavior rubric0.000 / 0.050
Deliverable integrity15% of total · 0.000 / 0.150
ResultVerifier checkEarned / max
NOT EVALUABLEExternal asset resolutionDelivery contract0.000 / 0.030
NO CREDITRequired deliverable filesDelivery contract0.000 / 0.060
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITRequired deliverable exists: final.blendDelivery contract0.000 / 0.030
NO CREDITRequired deliverable exists: final.glbDelivery contract0.000 / 0.030
NOT EVALUABLEExported asset integrityDelivery contract0.000 / 0.060
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe exported GLB retains the required scene content and animationDelivery contract0.000 / 0.030
NOT EVALUABLEThe exported GLB can be parsed and imported back successfullyDelivery contract0.000 / 0.030
Rendered evidence10% of total · 0.000 / 0.100
ResultVerifier checkEarned / max
NOT EVALUABLERequired rendered view setDelivery contract0.000 / 0.083
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.017
NO CREDITRequired render is present and non-degenerate: hero.pngDelivery contract0.000 / 0.017
NO CREDITRequired render is present and non-degenerate: side.pngDelivery contract0.000 / 0.017
NO CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.000 / 0.017
NOT EVALUABLERequired render is present and non-degenerate: detail.pngDelivery contract0.000 / 0.017
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.017
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute1 TOOL ERROR
T3blender_execute1 TOOL ERROR
T4blender_execute
T5blender_execute
T6blender_execute1 TOOL ERROR
T7blender_execute
T8blender_execute
T9blender_execute1 TOOL ERROR
T10blender_execute
T11blender_execute
T12blender_execute1 TOOL ERROR
T13blender_execute
T14blender_execute
T15blender_execute
T16blender_execute1 TOOL ERROR
T17blender_execute
T18blender_execute
T19blender_execute
T20blender_execute
T21blender_execute
T22blender_execute
T23blender_execute1 TOOL ERROR
T24blender_execute
T25blender_execute
T26blender_execute1 TOOL ERROR
T27blender_execute
T28blender_screenshot
T29blender_execute
T30blender_execute
T31assistant turn
T32blender_execute
T33blender_execute1 TOOL ERROR
T34blender_execute1 TOOL ERROR
T35blender_execute
T36blender_screenshot
T37blender_execute1 TOOL ERROR
T38blender_execute
T39blender_screenshot
T40blender_execute
T41blender_screenshot
T42blender_execute
T43blender_screenshot
T44blender_execute
T45blender_execute
T46blender_screenshot
T47blender_execute
T48assistant turn
Task structural contract
{
  "case_id": "04-rescue-drone",
  "required_collections": ["DRONE", "PAYLOAD", "RESCUE_SITE"],
  "required_objects": ["DRONE_Core", "WINCH_Cable", "PAYLOAD_Medical", "LANDING_Target"],
  "object_counts": [
    {"pattern": "ARM_*", "min": 4, "max": 4},
    {"pattern": "ROTOR_*", "min": 4, "max": 4},
    {"pattern": "FOOT_*", "min": 4},
    {"pattern": "RUBBLE_*", "min": 8}
  ],
  "animated_objects": [{"pattern": "ARM_*", "min": 4}, {"pattern": "ROTOR_*", "min": 4}, {"pattern": "PAYLOAD_*", "min": 1}],
  "min_cameras": 3,
  "min_frame_end": 220,
  "scene_properties": {"showcase_case_id": "04-rescue-drone"}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 05

48 sqm, Three Lives

WEIGHTED VERIFIER REWARD0.000/ 1.000
RULES RECEIVING CREDIT0/37blocked
Evaluation input
# 48 sqm, Three Lives

Build one 8 m by 6 m apartment shell and furnish three mutually exclusive versions:
SOLO for a remote developer, FAMILY for two adults and one child, and ACCESSIBLE for
an older resident using a walker. The shell, doors, windows, cameras, and lighting must
remain identical so the three layouts can be compared directly.

Each layout needs sleeping, cooking, dining, working, storage, and bathroom functions.
Keep a continuous entrance-to-bed circulation path and add named clearance proxy
objects: 0.8 m minimum for SOLO/FAMILY and 1.0 m for ACCESSIBLE. Deliver three matched
hero renders, one top comparison, Blend, GLB, and validation.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal observed viewport
No inspectable GLBThe evaluated rollout did not produce a final 3D asset.
48 rollout turns 24 tool calls 3 tool errors 0.000 reward / 1.000
Category reward breakdown0.000 / 1.000
Scene structure0/10.0% · 0.000/0.400 · 40% weightSpatial relationships0/10.0% · 0.000/0.150 · 15% weightDeliverable integrity0/30.0% · 0.000/0.200 · 20% weightRendered evidence0/60.0% · 0.000/0.250 · 25% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure40% of total · 0.000 / 0.400
ResultVerifier checkEarned / max
NOT EVALUABLERequired scene entitiesTask contract0.000 / 0.064
View 4 atomic checks · 0/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired scene object exists: SHELL Floor 8x6Task contract0.000 / 0.016
NOT EVALUABLERequired scene object exists: DOOR EntranceTask contract0.000 / 0.016
NOT EVALUABLERequired scene object exists: CAM HeroTask contract0.000 / 0.016
NOT EVALUABLERequired scene object exists: CAM TopTask contract0.000 / 0.016
NOT EVALUABLESemantic organization and coverageTask and delivery contract0.000 / 0.288
View 18 atomic checks · 0/18 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLERequired semantic collection exists: SHELL SHAREDTask contract0.000 / 0.016
NOT EVALUABLERequired semantic collection exists: LAYOUT SOLOTask contract0.000 / 0.016
NOT EVALUABLERequired semantic collection exists: LAYOUT FAMILYTask contract0.000 / 0.016
NOT EVALUABLERequired semantic collection exists: LAYOUT ACCESSIBLETask contract0.000 / 0.016
NOT EVALUABLEScene contains the required amount of SOLO Furniture * geometryTask contract0.000 / 0.016
NOT EVALUABLEScene contains the required amount of FAMILY Furniture * geometryTask contract0.000 / 0.016
NOT EVALUABLEScene contains the required amount of ACCESSIBLE Furniture * geometryTask contract0.000 / 0.016
NOT EVALUABLEScene contains the required amount of CLEARANCE * geometryTask contract0.000 / 0.016
NOT EVALUABLELAYOUT SOLO contains the required SOLO Furniture * membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT FAMILY contains the required FAMILY Furniture * membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT ACCESSIBLE contains the required ACCESSIBLE Furniture * membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT SOLO contains the required *Bed*,*Kitchen*,*Dining*,*Desk*,*Shelf*,*Bath* membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT FAMILY contains the required *BedParents*,*BedChild*,*Kitchen*,*Dining*,*Desk*,*Bath* membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT ACCESSIBLE contains the required *Bed*,*Kitchen*,*Dining*,*Work*,*Storage*,*Bath*,*Walker* membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT SOLO contains the required CLEARANCE SOLO *800* membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT FAMILY contains the required CLEARANCE FAMILY *800* membersBehavior rubric0.000 / 0.016
NOT EVALUABLELAYOUT ACCESSIBLE contains the required CLEARANCE ACCESSIBLE *1000*,CLEARANCE ACCESSIBLE *1500* membersBehavior rubric0.000 / 0.016
NOT EVALUABLECollection disjoint:layout solo,layout family,layout accessibleBehavior rubric0.000 / 0.016
NOT EVALUABLEScene metadata and camera setupTask contract0.000 / 0.048
View 3 atomic checks · 0/3 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe scene contains the required camera coverageTask contract0.000 / 0.016
NOT EVALUABLERequired scene metadata is recorded: showcase case idTask contract0.000 / 0.016
NOT EVALUABLERequired scene metadata is recorded: floor area sqmTask contract0.000 / 0.016
Spatial relationships15% of total · 0.000 / 0.150
ResultVerifier checkEarned / max
NOT EVALUABLEKey spatial outcomesBehavior rubric0.000 / 0.150
Animation behavior0% of total · 0.000 / 0.000
ResultVerifier checkEarned / max
INFORMATIONALTimeline coverageTask contract0.000 / 0.000
Deliverable integrity20% of total · 0.000 / 0.200
ResultVerifier checkEarned / max
NOT EVALUABLEExternal asset resolutionDelivery contract0.000 / 0.040
NO CREDITRequired deliverable filesDelivery contract0.000 / 0.080
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITRequired deliverable exists: final.blendDelivery contract0.000 / 0.040
NO CREDITRequired deliverable exists: final.glbDelivery contract0.000 / 0.040
NOT EVALUABLEExported asset integrityDelivery contract0.000 / 0.080
View 2 atomic checks · 0/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT EVALUABLEThe exported GLB retains the required scene content and animationDelivery contract0.000 / 0.040
NOT EVALUABLEThe exported GLB can be parsed and imported back successfullyDelivery contract0.000 / 0.040
Rendered evidence25% of total · 0.000 / 0.250
ResultVerifier checkEarned / max
NO CREDITRequired rendered view setDelivery contract0.000 / 0.208
View 5 atomic checks · 0/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.042
NO CREDITRequired render is present and non-degenerate: hero_solo.pngDelivery contract0.000 / 0.042
NO CREDITRequired render is present and non-degenerate: hero_family.pngDelivery contract0.000 / 0.042
NO CREDITRequired render is present and non-degenerate: hero_accessible.pngDelivery contract0.000 / 0.042
NO CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.000 / 0.042
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.042
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute
T4blender_execute1 TOOL ERROR
T5blender_execute
T6blender_execute
T7blender_execute1 TOOL ERROR
T8blender_execute1 TOOL ERROR
T9blender_execute
T10blender_execute
T11assistant turn
T12blender_execute
T13blender_screenshot
T14blender_execute
T15blender_screenshot
T16blender_execute
T17blender_screenshot
T18blender_execute
T19blender_execute
T20blender_execute
T21blender_execute
T22blender_screenshot
T23blender_execute
T24blender_screenshot
T25assistant turn
T26assistant turn
T27assistant turn
T28blender_execute
T29assistant turn
T30assistant turn
T31assistant turn
T32assistant turn
T33assistant turn
T34assistant turn
T35assistant turn
T36assistant turn
T37assistant turn
T38assistant turn
T39assistant turn
T40assistant turn
T41assistant turn
T42assistant turn
T43assistant turn
T44assistant turn
T45assistant turn
T46assistant turn
T47assistant turn
T48assistant turn
Task structural contract
{
  "case_id": "05-micro-apartment",
  "required_collections": ["SHELL_SHARED", "LAYOUT_SOLO", "LAYOUT_FAMILY", "LAYOUT_ACCESSIBLE"],
  "required_objects": ["SHELL_Floor_8x6", "DOOR_Entrance", "CAM_Hero", "CAM_Top"],
  "object_counts": [
    {"pattern": "SOLO_Furniture_*", "min": 8},
    {"pattern": "FAMILY_Furniture_*", "min": 10},
    {"pattern": "ACCESSIBLE_Furniture_*", "min": 8},
    {"pattern": "CLEARANCE_*", "min": 3}
  ],
  "min_cameras": 2,
  "min_frame_end": 1,
  "scene_properties": {"showcase_case_id": "05-micro-apartment", "floor_area_sqm": 48}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 06

Autonomous Warehouse: The Last Box

WEIGHTED VERIFIER REWARD0.794/ 1.000
RULES RECEIVING CREDIT34/42completed
Evaluation input
# Autonomous Warehouse: The Last Box

Create a compact fulfillment cell with three rack aisles. An AMR starts at charging,
navigates around a temporary blocked aisle, receives a box from a pick station, and
delivers it to a palletizing robot arm. The arm places the box on a pallet and both
machines return to a safe idle state.

Animate frames 1-300. Include a visible planned route, blocked zone, safety perimeter,
fork sensors, pick station, pallet, and status lights. No moving machine may cross the
safety perimeter while the other is active. Deliver Blend, GLB, four views, and a
validation report.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

40 rollout turns 40 tool calls 2 tool errors 0.794 reward / 1.000
Category reward breakdown0.794 / 1.000
Scene structure17/1989.5% · 0.224/0.250 · 25% weightSpatial relationships3/475.0% · 0.188/0.250 · 25% weightAnimation behavior6/6100.0% · 0.250/0.250 · 25% weightDeliverable integrity5/683.3% · 0.083/0.100 · 10% weightRendered evidence3/650.0% · 0.050/0.100 · 10% weightStructured reports0/10.0% · 0.000/0.050 · 5% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure25% of total · 0.224 / 0.250
ResultVerifier checkEarned / max
CREDITRequired scene entitiesTask contract0.079 / 0.079
View 6 atomic checks · 6/6 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired scene object exists: AMR BodyTask contract0.013 / 0.013
CREDITRequired scene object exists: ROUTE AMRTask contract0.013 / 0.013
CREDITRequired scene object exists: ZONE BlockedTask contract0.013 / 0.013
CREDITRequired scene object exists: ZONE SafetyTask contract0.013 / 0.013
CREDITRequired scene object exists: BOX LastTask contract0.013 / 0.013
CREDITRequired scene object exists: PALLET TargetTask contract0.013 / 0.013
CREDITSemantic organization and coverageTask contract0.105 / 0.105
View 8 atomic checks · 8/8 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired semantic collection exists: WAREHOUSETask contract0.013 / 0.013
CREDITRequired semantic collection exists: AMRTask contract0.013 / 0.013
CREDITRequired semantic collection exists: ROBOT ARMTask contract0.013 / 0.013
CREDITRequired semantic collection exists: SAFETYTask contract0.013 / 0.013
CREDITScene contains the required amount of RACK * geometryTask contract0.013 / 0.013
CREDITScene contains the required amount of ARM Joint * geometryTask contract0.013 / 0.013
CREDITScene contains the required amount of ROUTE Waypoint * geometryTask contract0.013 / 0.013
CREDITScene contains the required amount of STATUS Light * geometryTask contract0.013 / 0.013
CREDITScene metadata and camera setupTask contract0.026 / 0.026
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe scene contains the required camera coverageTask contract0.013 / 0.013
CREDITRequired scene metadata is recorded: showcase case idTask contract0.013 / 0.013
PARTIALObject types, hierarchy, and materialsBehavior rubric0.013 / 0.039
View 3 atomic checks · 1/3 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITRequired parent-child relationship is correct for AMR ForkBehavior rubric0.000 / 0.013
NO CREDITArticulated hierarchy is complete: ARM Joint 1->ARM Joint 2->ARM Joint 3->ARM Joint 4->ARM Joint 5Behavior rubric0.000 / 0.013
CREDITMaterials are assigned to required scene objectsBehavior rubric0.013 / 0.013
Spatial relationships25% of total · 0.188 / 0.250
ResultVerifier checkEarned / max
PARTIALKey spatial outcomesBehavior rubric0.188 / 0.250
View 4 atomic checks · 3/4 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe mobile robot starts at the charging dockBehavior rubric0.000 / 0.062
CREDITThe mobile robot avoids the blocked zoneBehavior rubric0.062 / 0.062
CREDITThe mobile robot remains outside the arm safety zone during arm motionBehavior rubric0.062 / 0.062
CREDITThe box finishes on the destination palletBehavior rubric0.062 / 0.062
Animation behavior25% of total · 0.250 / 0.250
ResultVerifier checkEarned / max
CREDITTimeline coverageTask contract0.042 / 0.042
CREDITRequested animation sequenceBehavior rubric0.208 / 0.208
View 5 atomic checks · 5/5 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe mobile robot traverses the warehouse cellBehavior rubric0.042 / 0.042
CREDITThe forklift mechanism performs the required lift motionBehavior rubric0.042 / 0.042
CREDITThe box is visibly transferred through the work cellBehavior rubric0.042 / 0.042
CREDITThe robot arm articulates during the transferBehavior rubric0.042 / 0.042
CREDITThe mobile robot reaches the required final idle stateBehavior rubric0.042 / 0.042
Deliverable integrity10% of total · 0.083 / 0.100
ResultVerifier checkEarned / max
CREDITExternal asset resolutionDelivery contract0.017 / 0.017
PARTIALRequired deliverable filesDelivery contract0.033 / 0.050
View 3 atomic checks · 2/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired deliverable exists: final.blendDelivery contract0.017 / 0.017
CREDITRequired deliverable exists: final.glbDelivery contract0.017 / 0.017
NO CREDITRequired deliverable exists: validation_report.jsonDelivery contract0.000 / 0.017
CREDITExported asset integrityDelivery contract0.033 / 0.033
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe exported GLB retains the required scene content and animationDelivery contract0.017 / 0.017
CREDITThe exported GLB can be parsed and imported back successfullyDelivery contract0.017 / 0.017
Rendered evidence10% of total · 0.050 / 0.100
ResultVerifier checkEarned / max
PARTIALRequired rendered view setDelivery contract0.050 / 0.083
View 5 atomic checks · 3/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.017
CREDITRequired render is present and non-degenerate: hero.pngDelivery contract0.017 / 0.017
CREDITRequired render is present and non-degenerate: side.pngDelivery contract0.017 / 0.017
CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.017 / 0.017
NO CREDITRequired render is present and non-degenerate: cell.pngDelivery contract0.000 / 0.017
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.017
Structured reports5% of total · 0.000 / 0.050
ResultVerifier checkEarned / max
NO CREDITStructured report validityDelivery contract0.000 / 0.050
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute
T4blender_execute
T5blender_execute
T6blender_execute1 TOOL ERROR
T7blender_execute
T8blender_screenshot
T9blender_execute
T10blender_execute1 TOOL ERROR
T11blender_execute
T12blender_execute
T13blender_screenshot
T14blender_execute
T15blender_execute
T16blender_screenshot
T17blender_execute
T18blender_screenshot
T19blender_execute
T20blender_execute
T21blender_execute
T22blender_screenshot
T23blender_execute
T24blender_execute
T25blender_screenshot
T26blender_execute
T27blender_screenshot
T28blender_execute
T29blender_execute
T30blender_screenshot
T31blender_execute
T32blender_screenshot
T33blender_execute
T34blender_screenshot
T35blender_execute
T36blender_screenshot
T37blender_execute
T38blender_execute
T39blender_screenshot
T40finish
Task structural contract
{
  "case_id": "06-autonomous-warehouse",
  "required_collections": ["WAREHOUSE", "AMR", "ROBOT_ARM", "SAFETY"],
  "required_objects": ["AMR_Body", "ROUTE_AMR", "ZONE_Blocked", "ZONE_Safety", "BOX_Last", "PALLET_Target"],
  "object_counts": [
    {"pattern": "RACK_*", "min": 9},
    {"pattern": "ARM_Joint_*", "min": 4},
    {"pattern": "ROUTE_Waypoint_*", "min": 6},
    {"pattern": "STATUS_Light_*", "min": 3}
  ],
  "animated_objects": [{"pattern": "AMR_*", "min": 1}, {"pattern": "ARM_*", "min": 3}, {"pattern": "BOX_*", "min": 1}],
  "min_cameras": 3,
  "min_frame_end": 300,
  "scene_properties": {"showcase_case_id": "06-autonomous-warehouse"}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 07

Boss Room: 15 FPS to 60 FPS

WEIGHTED VERIFIER REWARD0.805/ 1.000
RULES RECEIVING CREDIT26/30completed
Evaluation input
# Boss Room: 15 FPS to 60 FPS

Create a ruined cathedral boss arena and preserve two comparable collections:
BASELINE_HIGH and DELIVERY_OPTIMIZED. The optimized version must retain the hero-camera
silhouette and material identity while using instancing for repeated architecture,
three LOD levels for hero assets, simple collision proxies, and consolidated materials.

The final report must record triangle counts for both versions, optimized-to-baseline
ratio, unique mesh count, material count, and GLB size. Target at most 35% of baseline
triangles and no more than 24 optimized materials. Deliver matched before/after renders,
top view, Blend, optimized GLB, and validation JSON.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

22 rollout turns 22 tool calls 2 tool errors 0.805 reward / 1.000
Category reward breakdown0.805 / 1.000
Scene structure17/1894.4% · 0.447/0.474 · 45% weightAnimation behavior1/1100.0% · 0.000/0.000 · 0% weightDeliverable integrity6/6100.0% · 0.263/0.263 · 25% weightRendered evidence3/560.0% · 0.095/0.158 · 15% weightStructured reports0/10.0% · 0.000/0.105 · 10% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure45% of total · 0.447 / 0.474
ResultVerifier checkEarned / max
CREDITRequired scene entitiesTask contract0.079 / 0.079
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired scene object exists: ARENA FloorTask contract0.026 / 0.026
CREDITRequired scene object exists: BOSS AltarTask contract0.026 / 0.026
CREDITRequired scene object exists: CAM ComparisonTask contract0.026 / 0.026
CREDITSemantic organization and coverageTask contract0.211 / 0.211
View 8 atomic checks · 8/8 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired semantic collection exists: BASELINE HIGHTask contract0.026 / 0.026
CREDITRequired semantic collection exists: DELIVERY OPTIMIZEDTask contract0.026 / 0.026
CREDITRequired semantic collection exists: COLLISIONTask contract0.026 / 0.026
CREDITScene contains the required amount of OPT Column * geometryTask contract0.026 / 0.026
CREDITScene contains the required amount of LOD0 * geometryTask contract0.026 / 0.026
CREDITScene contains the required amount of LOD1 * geometryTask contract0.026 / 0.026
CREDITScene contains the required amount of LOD2 * geometryTask contract0.026 / 0.026
CREDITScene contains the required amount of UCX * geometryTask contract0.026 / 0.026
CREDITScene metadata and camera setupTask contract0.079 / 0.079
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe scene contains the required camera coverageTask contract0.026 / 0.026
CREDITRequired scene metadata is recorded: showcase case idTask contract0.026 / 0.026
CREDITRequired scene metadata is recorded: optimization report presentTask contract0.026 / 0.026
PARTIALObject types, hierarchy, and materialsBehavior rubric0.079 / 0.105
View 4 atomic checks · 3/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired objects share the intended reusable mesh: OPT Column *Behavior rubric0.026 / 0.026
CREDITGeometry complexity ratio meets the LOD requirement for DELIVERY OPTIMIZED/BASELINE HIGHBehavior rubric0.026 / 0.026
CREDITMaterial count stays within the required budget for DELIVERY OPTIMIZEDBehavior rubric0.026 / 0.026
NO CREDITSuccessive LOD meshes decrease in geometric complexityBehavior rubric0.000 / 0.026
Animation behavior0% of total · 0.000 / 0.000
ResultVerifier checkEarned / max
INFORMATIONALTimeline coverageTask contract0.000 / 0.000
Deliverable integrity25% of total · 0.263 / 0.263
ResultVerifier checkEarned / max
CREDITExternal asset resolutionDelivery contract0.044 / 0.044
CREDITRequired deliverable filesDelivery contract0.132 / 0.132
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired deliverable exists: final.blendDelivery contract0.044 / 0.044
CREDITRequired deliverable exists: final.glbDelivery contract0.044 / 0.044
CREDITRequired deliverable exists: optimization_report.jsonDelivery contract0.044 / 0.044
CREDITExported asset integrityDelivery contract0.088 / 0.088
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe exported GLB retains the required scene content and animationDelivery contract0.044 / 0.044
CREDITThe exported GLB can be parsed and imported back successfullyDelivery contract0.044 / 0.044
Rendered evidence15% of total · 0.095 / 0.158
ResultVerifier checkEarned / max
PARTIALRequired rendered view setDelivery contract0.063 / 0.126
View 4 atomic checks · 2/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe requested set of rendered views is completeDelivery contract0.032 / 0.032
NO CREDITRequired render is present and non-degenerate: before.pngDelivery contract0.000 / 0.032
NO CREDITRequired render is present and non-degenerate: after.pngDelivery contract0.000 / 0.032
CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.032 / 0.032
CREDITIndependent verifier renderDelivery contract0.032 / 0.032
Structured reports10% of total · 0.000 / 0.105
ResultVerifier checkEarned / max
NO CREDITStructured report validityDelivery contract0.000 / 0.105
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute1 TOOL ERROR
T3blender_execute
T4blender_execute1 TOOL ERROR
T5blender_execute
T6blender_execute
T7blender_execute
T8blender_execute
T9blender_execute
T10blender_execute
T11blender_execute
T12blender_screenshot
T13blender_execute
T14blender_screenshot
T15blender_execute
T16blender_screenshot
T17blender_execute
T18blender_screenshot
T19blender_execute
T20blender_execute
T21blender_screenshot
T22finish
Task structural contract
{
  "case_id": "07-boss-room-optimization",
  "required_collections": ["BASELINE_HIGH", "DELIVERY_OPTIMIZED", "COLLISION"],
  "required_objects": ["ARENA_Floor", "BOSS_Altar", "CAM_Comparison"],
  "object_counts": [
    {"pattern": "OPT_Column_*", "min": 12},
    {"pattern": "LOD0_*", "min": 3},
    {"pattern": "LOD1_*", "min": 3},
    {"pattern": "LOD2_*", "min": 3},
    {"pattern": "UCX_*", "min": 8}
  ],
  "min_cameras": 3,
  "min_frame_end": 1,
  "scene_properties": {"showcase_case_id": "07-boss-room-optimization", "optimization_report_present": true}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 08

Save the Friday Build

WEIGHTED VERIFIER REWARDN/AINVALID INPUT
RULES RECEIVING CREDITN/Acompleted
Evaluation input
# Save the Friday Build

Repair the supplied broken product-launch scene without rebuilding it from scratch.
The scene contains twelve seeded defects spanning missing material links, inverted
normals, wrong scale, broken parenting, an offset camera target, disabled lights,
intersections, duplicate objects, a detached mechanical joint, invalid frame range,
missing export collection membership, and an incorrect output color transform.

First create `output/diagnosis.json` listing each suspected defect and affected object.
Then repair the scene with minimal edits, preserving unaffected object transforms and
names. Deliver repaired Blend and GLB, four renders, `output/change_log.json`, and
validation. Do not delete the `FAULT_MANIFEST` text block.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

30 rollout turns 30 tool calls 3 tool errors N/A reward / 1.000
Category reward breakdownN/A
Scene structureN/Anot scored · 25% weightAnimation behaviorN/Anot scored · 15% weightDeliverable integrityN/Anot scored · 20% weightRendered evidenceN/Anot scored · 10% weightStructured reportsN/Anot scored · 20% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Input validityvalidation gate · N/A
ResultVerifier checkEarned / max
INVALIDCanonical task inputInput contractgate
View 2 atomic checks · 0/2 checks valid
ResultAtomic verifier checkEarned / max
NO CREDITRequired source asset is present: broken_scene.blendInput contractgate
NO CREDITThe canonical task input is present and readableInput contractgate
Scene structure25% of total · 0.278 / 0.278
ResultVerifier checkEarned / max
NOT SCOREDRequired scene entitiesTask contractN/A / 0.088
View 6 atomic checks · 6/6 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDRequired scene object exists: PRODUCT CoreTask contractN/A / 0.015
NOT SCOREDRequired scene object exists: PRODUCT JointTask contractN/A / 0.015
NOT SCOREDRequired scene object exists: CAM HeroTask contractN/A / 0.015
NOT SCOREDRequired scene object exists: TARGET HeroTask contractN/A / 0.015
NOT SCOREDRequired scene object exists: LIGHT KeyTask contractN/A / 0.015
NOT SCOREDRequired scene object exists: LIGHT RimTask contractN/A / 0.015
NOT SCOREDSemantic organization and coverageTask and delivery contractN/A / 0.102
View 7 atomic checks · 7/7 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDRequired semantic collection exists: PRODUCTTask contractN/A / 0.015
NOT SCOREDRequired semantic collection exists: STAGETask contractN/A / 0.015
NOT SCOREDRequired semantic collection exists: EXPORTTask contractN/A / 0.015
NOT SCOREDScene contains the required amount of FAULT FIXED * geometryTask contractN/A / 0.015
NOT SCOREDScene contains the required amount of PRODUCT * geometryTask contractN/A / 0.015
NOT SCOREDScene contains the required amount of STAGE * geometryTask contractN/A / 0.015
NOT SCOREDPRODUCT * objects belong to EXPORTBehavior rubricN/A / 0.015
NOT SCOREDScene metadata and camera setupTask contractN/A / 0.058
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDThe scene contains the required camera coverageTask contractN/A / 0.015
NOT SCOREDRequired scene metadata is recorded: showcase case idTask contractN/A / 0.015
NOT SCOREDRequired scene metadata is recorded: diagnosis presentTask contractN/A / 0.015
NOT SCOREDRequired scene metadata is recorded: change log presentTask contractN/A / 0.015
NOT SCOREDObject types, hierarchy, and materialsBehavior rubricN/A / 0.029
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDRequired parent-child relationship is correct for PRODUCT JointBehavior rubricN/A / 0.015
NOT SCOREDMaterials are assigned to required scene objectsBehavior rubricN/A / 0.015
Animation behavior15% of total · 0.167 / 0.167
ResultVerifier checkEarned / max
NOT SCOREDTimeline coverageTask contractN/A / 0.083
NOT SCOREDRequested animation sequenceBehavior rubricN/A / 0.083
Deliverable integrity20% of total · 0.222 / 0.222
ResultVerifier checkEarned / max
NOT SCOREDExternal asset resolutionDelivery contractN/A / 0.032
NOT SCOREDRequired deliverable filesDelivery contractN/A / 0.127
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDRequired deliverable exists: final.blendDelivery contractN/A / 0.032
NOT SCOREDRequired deliverable exists: final.glbDelivery contractN/A / 0.032
NOT SCOREDRequired deliverable exists: diagnosis.jsonDelivery contractN/A / 0.032
NOT SCOREDRequired deliverable exists: change_log.jsonDelivery contractN/A / 0.032
NOT SCOREDExported asset integrityDelivery contractN/A / 0.063
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDThe exported GLB retains the required scene content and animationDelivery contractN/A / 0.032
NOT SCOREDThe exported GLB can be parsed and imported back successfullyDelivery contractN/A / 0.032
Rendered evidence10% of total · 0.089 / 0.111
ResultVerifier checkEarned / max
NOT SCOREDRequired rendered view setDelivery contractN/A / 0.089
View 4 atomic checks · 3/4 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDThe requested set of rendered views is completeDelivery contractN/A / 0.022
NOT SCOREDRequired render is present and non-degenerate: hero.pngDelivery contractN/A / 0.022
NOT SCOREDRequired render is present and non-degenerate: side.pngDelivery contractN/A / 0.022
NOT SCOREDRequired render is present and non-degenerate: top.pngDelivery contractN/A / 0.022
NOT SCOREDIndependent verifier renderDelivery contractN/A / 0.022
Structured reports20% of total · 0.222 / 0.222
ResultVerifier checkEarned / max
NOT SCOREDStructured report validityDelivery contractN/A / 0.222
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
NOT SCOREDStructured report is present and satisfies its schema: diagnosis.jsonDelivery contractN/A / 0.111
NOT SCOREDStructured report is present and satisfies its schema: change_log.jsonDelivery contractN/A / 0.111
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute
T4blender_execute
T5blender_execute
T6blender_execute
T7blender_execute
T8blender_execute
T9blender_execute1 TOOL ERROR
T10blender_execute
T11blender_execute1 TOOL ERROR
T12blender_execute
T13blender_execute
T14blender_execute
T15blender_execute
T16blender_execute
T17blender_screenshot
T18blender_execute
T19blender_execute
T20blender_screenshot
T21blender_execute1 TOOL ERROR
T22blender_execute
T23blender_execute
T24blender_execute
T25blender_execute
T26blender_execute
T27blender_execute
T28blender_execute
T29blender_screenshot
T30finish
Task structural contract
{
  "case_id": "08-friday-scene-rescue",
  "required_collections": ["PRODUCT", "STAGE", "EXPORT"],
  "required_objects": ["PRODUCT_Core", "PRODUCT_Joint", "CAM_Hero", "TARGET_Hero", "LIGHT_Key", "LIGHT_Rim"],
  "object_counts": [
    {"pattern": "FAULT_FIXED_*", "min": 12, "max": 12},
    {"pattern": "PRODUCT_*", "min": 6},
    {"pattern": "STAGE_*", "min": 4}
  ],
  "animated_objects": [{"pattern": "PRODUCT_Joint", "min": 1}],
  "min_cameras": 3,
  "min_frame_end": 120,
  "scene_properties": {"showcase_case_id": "08-friday-scene-rescue", "diagnosis_present": true, "change_log_present": true}
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 09

Guardian from Fragments

WEIGHTED VERIFIER REWARD0.794/ 1.000
RULES RECEIVING CREDIT29/35completed
Evaluation input
# Guardian from Fragments

Create an archaeological reconstruction showcase for a damaged stone guardian statue.
Preserve two comparable collections: SOURCE_FRAGMENTS contains the discovered scan-like
pieces in their catalogued positions, while RECONSTRUCTED contains the assembled
interpretation. Keep uncertain or invented surfaces visibly distinct from confirmed
stone without weakening the hero silhouette.

Animate frames 1-240 as at least eight fragments move from an exploded survey layout
into the reconstructed guardian. Include alignment pins, reference planes, gap markers,
scale bars, and a color-coded uncertainty map. Write `output/reconstruction_report.json`
with fragment counts, confirmed-versus-inferred surface estimates, unresolved gaps, and
the reconstruction assumptions. Deliver matched source/reconstructed renders, exploded
and top views, Blend, GLB, and validation JSON.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

43 rollout turns 43 tool calls 5 tool errors 0.794 reward / 1.000
Category reward breakdown0.794 / 1.000
Scene structure18/1994.7% · 0.368/0.389 · 35% weightAnimation behavior3/3100.0% · 0.222/0.222 · 20% weightDeliverable integrity6/6100.0% · 0.167/0.167 · 15% weightRendered evidence2/633.3% · 0.037/0.111 · 10% weightStructured reports0/10.0% · 0.000/0.111 · 10% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure35% of total · 0.368 / 0.389
ResultVerifier checkEarned / max
CREDITRequired scene entitiesTask contract0.082 / 0.082
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired scene object exists: GUARDIAN CoreTask contract0.020 / 0.020
CREDITRequired scene object exists: RECON GuardianTask contract0.020 / 0.020
CREDITRequired scene object exists: MAP UncertaintyTask contract0.020 / 0.020
CREDITRequired scene object exists: CAM ComparisonTask contract0.020 / 0.020
PARTIALSemantic organization and coverageTask and delivery contract0.205 / 0.225
View 11 atomic checks · 10/11 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired semantic collection exists: SOURCE FRAGMENTSTask contract0.020 / 0.020
CREDITRequired semantic collection exists: RECONSTRUCTEDTask contract0.020 / 0.020
CREDITRequired semantic collection exists: DOCUMENTATIONTask contract0.020 / 0.020
CREDITScene contains the required amount of FRAG * geometryTask contract0.020 / 0.020
CREDITScene contains the required amount of ALIGN Pin * geometryTask contract0.020 / 0.020
CREDITScene contains the required amount of GAP Uncertain * geometryTask contract0.020 / 0.020
CREDITScene contains the required amount of REF Plane * geometryTask contract0.020 / 0.020
CREDITSOURCE FRAGMENTS contains the required * membersBehavior rubric0.020 / 0.020
CREDITRECONSTRUCTED contains the required * membersBehavior rubric0.020 / 0.020
NO CREDITDOCUMENTATION contains the required GAP Uncertain * membersBehavior rubric0.000 / 0.020
CREDITDOCUMENTATION contains the required REF Plane * membersBehavior rubric0.020 / 0.020
CREDITScene metadata and camera setupTask contract0.061 / 0.061
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe scene contains the required camera coverageTask contract0.020 / 0.020
CREDITRequired scene metadata is recorded: showcase case idTask contract0.020 / 0.020
CREDITRequired scene metadata is recorded: reconstruction report presentTask contract0.020 / 0.020
CREDITObject types, hierarchy, and materialsBehavior rubric0.020 / 0.020
Animation behavior20% of total · 0.222 / 0.222
ResultVerifier checkEarned / max
CREDITTimeline coverageTask contract0.074 / 0.074
CREDITRequested animation sequenceBehavior rubric0.148 / 0.148
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe reconstructed fragments assemble into the target formBehavior rubric0.074 / 0.074
CREDITThe reconstructed fragments remain stable at the endBehavior rubric0.074 / 0.074
Deliverable integrity15% of total · 0.167 / 0.167
ResultVerifier checkEarned / max
CREDITExternal asset resolutionDelivery contract0.028 / 0.028
CREDITRequired deliverable filesDelivery contract0.083 / 0.083
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired deliverable exists: final.blendDelivery contract0.028 / 0.028
CREDITRequired deliverable exists: final.glbDelivery contract0.028 / 0.028
CREDITRequired deliverable exists: reconstruction_report.jsonDelivery contract0.028 / 0.028
CREDITExported asset integrityDelivery contract0.056 / 0.056
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe exported GLB retains the required scene content and animationDelivery contract0.028 / 0.028
CREDITThe exported GLB can be parsed and imported back successfullyDelivery contract0.028 / 0.028
Rendered evidence10% of total · 0.037 / 0.111
ResultVerifier checkEarned / max
PARTIALRequired rendered view setDelivery contract0.019 / 0.093
View 5 atomic checks · 1/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.019
NO CREDITRequired render is present and non-degenerate: source.pngDelivery contract0.000 / 0.019
NO CREDITRequired render is present and non-degenerate: reconstructed.pngDelivery contract0.000 / 0.019
NO CREDITRequired render is present and non-degenerate: exploded.pngDelivery contract0.000 / 0.019
CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.019 / 0.019
CREDITIndependent verifier renderDelivery contract0.019 / 0.019
Structured reports10% of total · 0.000 / 0.111
ResultVerifier checkEarned / max
NO CREDITStructured report validityDelivery contract0.000 / 0.111
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute
T4blender_execute
T5blender_execute1 TOOL ERROR
T6blender_execute
T7blender_execute1 TOOL ERROR
T8blender_execute
T9blender_execute
T10blender_execute
T11blender_screenshot
T12blender_execute
T13blender_execute
T14blender_execute
T15blender_execute
T16blender_execute
T17blender_screenshot
T18blender_execute
T19blender_screenshot
T20blender_execute1 TOOL ERROR
T21blender_execute
T22blender_execute
T23blender_execute
T24blender_execute
T25blender_screenshot
T26blender_execute
T27blender_screenshot
T28blender_execute
T29blender_execute
T30blender_execute
T31blender_screenshot
T32blender_execute1 TOOL ERROR
T33blender_execute
T34blender_execute1 TOOL ERROR
T35blender_execute
T36blender_execute
T37blender_execute
T38blender_execute
T39blender_execute
T40blender_execute
T41blender_screenshot
T42blender_execute
T43finish
Task structural contract
{
  "case_id": "09-guardian-reconstruction",
  "required_collections": [
    "SOURCE_FRAGMENTS",
    "RECONSTRUCTED",
    "DOCUMENTATION"
  ],
  "required_objects": [
    "GUARDIAN_Core",
    "RECON_Guardian",
    "MAP_Uncertainty",
    "CAM_Comparison"
  ],
  "object_counts": [
    {
      "pattern": "FRAG_*",
      "min": 16
    },
    {
      "pattern": "ALIGN_Pin_*",
      "min": 6
    },
    {
      "pattern": "GAP_Uncertain_*",
      "min": 4
    },
    {
      "pattern": "REF_Plane_*",
      "min": 3
    }
  ],
  "animated_objects": [
    {
      "pattern": "FRAG_*",
      "min": 8
    }
  ],
  "min_cameras": 3,
  "min_frame_end": 240,
  "scene_properties": {
    "showcase_case_id": "09-guardian-reconstruction",
    "reconstruction_report_present": true
  }
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.
CASE 10

Stormwall: 90-Second Response

WEIGHTED VERIFIER REWARD0.815/ 1.000
RULES RECEIVING CREDIT29/34completed
Evaluation input
# Stormwall: 90-Second Response

Create a compact canal-district digital twin that responds to a fast storm surge.
Show a normal public waterfront, rising water, sensor alerts, evacuation routing, and
a modular flood barrier deploying before the surge reaches the protected plaza. The
scene must communicate both the civic environment and the control-system logic.

Animate frames 1-300. Include at least twelve independently named barrier modules,
water-level sensors, warning beacons, protected and flood-risk zones, evacuation
waypoints, and a visible control station. Barrier modules must deploy in a coordinated
sequence and the protected zone must remain dry after full deployment. Write
`output/emergency_plan.json` with the alert threshold, deployment sequence, evacuation
route, sampled safety checks, and any remaining risk. Deliver calm/storm hero renders,
side and top views, Blend, animated GLB, and validation JSON.

Text-only task · no source image supplied

Evaluated rollout evidence
Evaluated rolloutFinal render
LOADING 3D

Evaluated output asset · drag to orbit · scroll to zoom

38 rollout turns 38 tool calls 5 tool errors 0.815 reward / 1.000
Category reward breakdown0.815 / 1.000
Scene structure16/16100.0% · 0.278/0.278 · 25% weightAnimation behavior5/5100.0% · 0.333/0.333 · 30% weightDeliverable integrity6/6100.0% · 0.167/0.167 · 15% weightRendered evidence2/633.3% · 0.037/0.111 · 10% weightStructured reports0/10.0% · 0.000/0.111 · 10% weight
Verifier rubric

Scannable criteria, complete checks

Parent criteria are shown first. Expand a grouped criterion to inspect every atomic check and its allocated credit. Exact numeric tolerances and internal observed values remain hidden.

Scene structure25% of total · 0.278 / 0.278
ResultVerifier checkEarned / max
CREDITRequired scene entitiesTask contract0.069 / 0.069
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired scene object exists: WATER SurgeTask contract0.017 / 0.017
CREDITRequired scene object exists: BARRIER ControlTask contract0.017 / 0.017
CREDITRequired scene object exists: ZONE ProtectedTask contract0.017 / 0.017
CREDITRequired scene object exists: ROUTE EvacuationTask contract0.017 / 0.017
CREDITSemantic organization and coverageTask contract0.139 / 0.139
View 8 atomic checks · 8/8 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired semantic collection exists: CITYTask contract0.017 / 0.017
CREDITRequired semantic collection exists: WATERTask contract0.017 / 0.017
CREDITRequired semantic collection exists: BARRIER SYSTEMTask contract0.017 / 0.017
CREDITRequired semantic collection exists: EVACUATIONTask contract0.017 / 0.017
CREDITScene contains the required amount of BARRIER Module * geometryTask contract0.017 / 0.017
CREDITScene contains the required amount of SENSOR Water * geometryTask contract0.017 / 0.017
CREDITScene contains the required amount of BUILDING * geometryTask contract0.017 / 0.017
CREDITScene contains the required amount of ROUTE Waypoint * geometryTask contract0.017 / 0.017
CREDITScene metadata and camera setupTask contract0.052 / 0.052
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe scene contains the required camera coverageTask contract0.017 / 0.017
CREDITRequired scene metadata is recorded: showcase case idTask contract0.017 / 0.017
CREDITRequired scene metadata is recorded: emergency plan presentTask contract0.017 / 0.017
CREDITObject types, hierarchy, and materialsBehavior rubric0.017 / 0.017
Animation behavior30% of total · 0.333 / 0.333
ResultVerifier checkEarned / max
CREDITTimeline coverageTask contract0.067 / 0.067
CREDITRequested animation sequenceBehavior rubric0.267 / 0.267
View 4 atomic checks · 4/4 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe barriers visibly rise from their initial stateBehavior rubric0.067 / 0.067
CREDITThe barriers deploy in the requested sequenceBehavior rubric0.067 / 0.067
CREDITThe stormwall visibly changes into its surge-response stateBehavior rubric0.067 / 0.067
CREDITThe barriers remain deployed at the endBehavior rubric0.067 / 0.067
Deliverable integrity15% of total · 0.167 / 0.167
ResultVerifier checkEarned / max
CREDITExternal asset resolutionDelivery contract0.028 / 0.028
CREDITRequired deliverable filesDelivery contract0.083 / 0.083
View 3 atomic checks · 3/3 receiving credit
ResultAtomic verifier checkEarned / max
CREDITRequired deliverable exists: final.blendDelivery contract0.028 / 0.028
CREDITRequired deliverable exists: final.glbDelivery contract0.028 / 0.028
CREDITRequired deliverable exists: emergency_plan.jsonDelivery contract0.028 / 0.028
CREDITExported asset integrityDelivery contract0.056 / 0.056
View 2 atomic checks · 2/2 receiving credit
ResultAtomic verifier checkEarned / max
CREDITThe exported GLB retains the required scene content and animationDelivery contract0.028 / 0.028
CREDITThe exported GLB can be parsed and imported back successfullyDelivery contract0.028 / 0.028
Rendered evidence10% of total · 0.037 / 0.111
ResultVerifier checkEarned / max
PARTIALRequired rendered view setDelivery contract0.037 / 0.093
View 5 atomic checks · 2/5 receiving credit
ResultAtomic verifier checkEarned / max
NO CREDITThe requested set of rendered views is completeDelivery contract0.000 / 0.019
NO CREDITRequired render is present and non-degenerate: calm.pngDelivery contract0.000 / 0.019
NO CREDITRequired render is present and non-degenerate: storm.pngDelivery contract0.000 / 0.019
CREDITRequired render is present and non-degenerate: side.pngDelivery contract0.019 / 0.019
CREDITRequired render is present and non-degenerate: top.pngDelivery contract0.019 / 0.019
NO CREDITIndependent verifier renderDelivery contract0.000 / 0.019
Structured reports10% of total · 0.000 / 0.111
ResultVerifier checkEarned / max
NO CREDITStructured report validityDelivery contract0.000 / 0.111
Rollout trajectory timeline
T1blender_scene_info
T2blender_execute
T3blender_execute1 TOOL ERROR
T4blender_execute
T5blender_execute1 TOOL ERROR
T6blender_execute
T7blender_execute1 TOOL ERROR
T8blender_execute
T9blender_execute
T10blender_execute
T11blender_execute1 TOOL ERROR
T12blender_execute
T13blender_execute
T14blender_execute
T15blender_screenshot
T16blender_execute
T17blender_execute
T18blender_screenshot
T19blender_execute
T20blender_screenshot
T21blender_execute
T22blender_screenshot
T23blender_execute
T24blender_screenshot
T25blender_execute
T26blender_screenshot
T27blender_execute
T28blender_screenshot
T29blender_execute1 TOOL ERROR
T30blender_execute
T31blender_screenshot
T32blender_execute
T33blender_execute
T34blender_screenshot
T35blender_execute
T36blender_screenshot
T37blender_execute
T38finish
Task structural contract
{
  "case_id": "10-stormwall-response",
  "required_collections": [
    "CITY",
    "WATER",
    "BARRIER_SYSTEM",
    "EVACUATION"
  ],
  "required_objects": [
    "WATER_Surge",
    "BARRIER_Control",
    "ZONE_Protected",
    "ROUTE_Evacuation"
  ],
  "object_counts": [
    {
      "pattern": "BARRIER_Module_*",
      "min": 12
    },
    {
      "pattern": "SENSOR_Water_*",
      "min": 6
    },
    {
      "pattern": "BUILDING_*",
      "min": 6
    },
    {
      "pattern": "ROUTE_Waypoint_*",
      "min": 8
    }
  ],
  "animated_objects": [
    {
      "pattern": "BARRIER_Module_*",
      "min": 8
    },
    {
      "pattern": "WATER_*",
      "min": 1
    }
  ],
  "min_cameras": 3,
  "min_frame_end": 300,
  "scene_properties": {
    "showcase_case_id": "10-stormwall-response",
    "emergency_plan_present": true
  }
}
Sanitized rollout trajectory JSON
Available in the downloadable offline audit bundle.