THE USEFUL PART

  • Write down the exact task and conditions the video demonstrates.
  • Count complete attempts, including resets and interventions.
  • Request one ordinary change that reflects the place you will use the robot.

Start with a sentence you can check

A robot carries a cup across a spotless room. That is an interesting event, but it leaves a buyer with several possible claims. Did it navigate the room, identify the cup, grasp an unfamiliar object, or repeat a prepared sequence? Pause the video and write one observable sentence: the robot moved this cup from this table to that tray under these conditions. Keep the claim as small as the evidence.

This is the discipline behind useful evaluation. NIST's response robot program develops repeatable test methods so users can measure specific capabilities and compare results. Its ground robot methods concern response operations, not a certification for household companions. The transferable lesson is to define the task and conditions before judging the performance. The following viewing exercise is our editorial method, not a NIST test.

Find the missing denominator

One successful attempt establishes that a sequence happened at least once. To estimate how often it works, you need the attempts around it. Ask for a continuous recording of the next ten attempts, or the seller's existing trial log with its selection method explained. Ten is a manageable request, not a statistically sufficient guarantee. The useful change is that failures become visible rather than disappearing into the edit.

Agree on success first. For the cup task, is spilling a failure? Does placement count if the cup ends up beside the tray? Record completions, incomplete attempts and human interventions separately. If the seller reports a success percentage, ask what was counted, what was excluded and how many trials produced it. A percentage without those details is difficult to compare with your own requirements.

Keep the whole cycle on screen

Request the time before and after the impressive movement. A robot may need a person to position the object, choose a mode, clear a fault or return it to the starting point. These steps are part of the workflow you will operate. Record time from the real starting state to a useful finished state, then record the reset separately.

Watch the camera frame as carefully as the robot. Is the operator visible? Is the object handoff visible? Are there cuts, speed changes or a second computer outside the frame? These are questions about what the clip documents, not evidence of dishonesty. A helpful presenter can identify preparation, supervision and editing without diminishing the engineering achievement.

Change one ordinary thing

Choose a variation from your intended environment. Move the tray a little, use a different approved object, alter the starting position or change the lighting within the product's stated limits. Ask the seller to select a safe variation and explain whether it needs new setup. Do not improvise obstacles around moving machinery or demand a stunt outside its operating envelope.

Changing one condition at a time makes the result easier to interpret. If the robot needs a fresh map, mark that as setup. If it needs new code, mark that as development. If it succeeds immediately, record the changed condition alongside the outcome. You are building a description of where the capability holds, rather than awarding a universal intelligence score.

Leave with a useful next request

Finish the review with three lines: what the evidence shows, what remains unknown and what evidence would answer your purchase question. For example: it repeatedly moves a known container on this bench; performance after a layout change is unknown; request a supervised demonstration with a different tray position. This is more actionable than calling the robot impressive or disappointing.

Save the product version, demonstration date and relevant conditions with your notes. A later software release may change the result, so keep observations tied to the version shown. If the remaining unknown concerns the task you actually need, treat the next demonstration as part of the buying process. A beautiful clip can earn attention. A well specified repeat earns a place on the shortlist.

TAKE THESE QUESTIONS WITH YOU

Six questions for a robot demonstration

  1. What exact task, conditions, robot version and demonstration date does this clip establish?
  2. What counted as success, and how many attempts were included or excluded?
  3. Which human interventions were needed during the attempts?
  4. How long did the complete task take, including preparation and reset?
  5. Which cuts, speed changes or off-screen controls need explaining?
  6. What changes when one ordinary condition varies within the product's stated limits?
Based on published sources, with our analysis. Product imagery is credited to its source; editorial illustrations are labeled. Read our editorial approach.