HN
Hacker News • 20일 전
로봇 팔을 제어하는 GPT-6 Astra
IMP 8/10
핵심 요약
OpenAI의 GPT-6 Astra를 로봇 팔(YAM 암)에 적용해 Claude Fable 5/5.1과 동일한 두 가지 조작 과제로 비교한 벤치마크 결과다. 블록을 그릇에 넣는 과제에서 Astra는 20회 중 19회 성공(95%)하며 Fable 5.1(8회)을 크게 앞섰고, 회당 비용도 약 절반($0.94 vs $2.12), 소요 시간도 2.5분으로 훨씬 빨랐다. 반면 퍼즐 조각 끼우기 과제에서는 두 모델 모두 20회 중 2회에 그치며 마지막 삽입 단계에서 같은 지점에 막히는 한계를 보였다.
번역된 본문
GPT-6 Astra의 로봇 조작 실험 (2026년 9월 4일)
본 글은 Claude Fable 5와 Fable 5.1 비교에 이은 후속 실험이다. OpenAI의 GPT-6 Astra에 동일한 YAM 로봇 팔과 동일한 Inspect Robots 에이전트 정책을 적용하고, 같은 두 가지 과제를 수행시켰다:
- "테이블 위의 빨간 블록을 집어 그릇 안에 놓아라."
- "중앙의 손잡이를 잡고 둥근 파란 퍼즐 조각을 집어, 보드의 해당하는 원형 홈에 끼워 넣어라."
그릇 과제에서 Astra는 20회 중 19회 블록을 넣는 데 성공했으며, Fable 5.1은 8회, Fable 5는 1회에 그쳤다. 회당 소요 시간은 2.5분으로 Fable 5.1의 6.8분보다 짧았고, 추정 비용도 회당 $0.94로 $2.12보다 저렴했다.
퍼즐 과제는 다른 양상이다: Astra는 20회 중 2회 삽입에 성공했으며, Fable 5.1도 2회 성공했다. Astra는 홈까지는 도달하지만 Fable과 마찬가지로 마지막 단계에서 멈추며, 회당 비용은 $1.36으로 $2.18보다 저렴했다.
블록을 그릇에 넣기: 각 모델의 최고 완주 기록(최고 단계, 그 다음 최단 시간 기준)을 동일한 배속으로 재생한 것이다. 타이머는 사고 일시정지 시간을 제외한 실제 경과 시간을 나타낸다. Astra는 그릇 과제를 훨씬 더 자주 완료하며, 회당 비용도 약 절반이다.
- 그릇 과제: 완료율 2.4배, 비용 2.3배 저렴
- 퍼즐 과제: 완료율 동일, 비용 1.6배 저렴
추정 회당 비용(USD, 정가). 큰 점은 조건별 평균, 흐린 점은 개별 시도(완료 시 100, 미완료 시 0)를 각 비용으로 표시한 것이다.
채점 방식
모든 시도는 도달한 최고 단계를 기준으로 사람 채점자가 점수를 매겼으므로, 실패한 실행도 얼마나 진행됐는지 기록된다. 채점 기준은 Fable 보고서와 동일하다.
- 0: 목적 있는 접근 없음
- 1: 물체와 접촉
- 2: 물체를 테이블에서 완전히 들어올림
- 3: 목표 지점 위에 배치
- 4: 최종 위치에 놓음
Astra는 블록을 거의 매번 넣는 데 성공하지만, 퍼즐에서는 Fable과 같은 지점에서 막힌다. 각 모델이 단계별 도달한 시도 비율(행별 n은 해당 칸의 시도 수).
결과 요약
| 과제 | 모델 | 평균 단계 | 완료 | 완료율 | 출력 토큰/실행 | 추정 비용/실행 | 시간(분)/실행 |
|---|---|---|---|---|---|---|---|
| 블록→그릇 | Fable 5 | 1.30 | 1/20 | 5% | 19.2k | $2.69 | 8.2 |
| 블록→그릇 | Fable 5.1 | 2.40 | 8/20 | 40% | 12.9k | $2.12 | 6.8 |
| 블록→그릇 | GPT-6 Astra | 3.95 | 19/20 | 95% | 2.1k | $0.94 | 2.5 |
| 퍼즐→홈 | Fable 5 | 1.50 | 0/20 | 0% | 16.3k | $2.63 | 7.9 |
| 퍼즐→홈 | Fable 5.1 | 2.35 | 2/20 | 10% | 10.5k | $2.18 | 5.9 |
| 퍼즐→홈 | GPT-6 Astra | 2.00 | 2/20 | 10% | 2.7k | $1.36 | 3.4 |
전체 실행 기록
집계된 전체 시도는 총 120회다. 각 실행에는 점수, 소요 시간(분), 출력 토큰 수와 함께 기록(transcript)·영상(video)·재실행(rerun) 링크가 제공된다. 예시로 그릇 과제의 GPT-6 Astra 실행들은 대부분 1.73.2분, 1,5003,000개 출력 토큰으로 점수 4(완전 완료)를 기록했다.
원문 보기 (영어)
GPT‑6 Astra on robotic manipulation September 4, 2026 A follow‑up to our comparison of Claude Fable 5 and Fable 5.1 . We gave OpenAI's GPT‑6 Astra control of the same YAM arms under the same Inspect Robots agent policy, on the same two tasks: “Pick up the red block from the table and place it inside the bowl.” “Pick up the round blue puzzle piece by the knob at its center and place it into the matching circular groove in the board.” On the bowl task Astra placed the block in 19 of 20 trials, against Fable 5.1's 8 of 20 and Fable 5 in 1 of 20, in 2.5 minutes per trial to Fable 5.1's 6.8, at an estimated $0.94 per run to $2.12. The puzzle task is a different story: Astra completed the insertion 2 times in 20 against Fable 5.1's 2 in 20. It reaches the groove and stalls at the same final step Fable does, at $1.36 per run to $2.18. Block into bowl: the best completed run of each model (highest stage, then shortest), each played in its own time at the same speed‑up. Timers show real elapsed time with thinking pauses removed. Astra completes the bowl task far more often, at about half the cost per run 2026-09-04T18:50:02.475437 image/svg+xml Matplotlib v3.11.1, https://matplotlib.org/ $1 $2 $3 0 20 40 60 80 100 completion rate (%) Fable 5 Fable 5.1 GPT-6 Astra 2.4× higher completion rate 2.3× cheaper Block into bowl $1 $2 $3 Fable 5 Fable 5.1 GPT-6 Astra same completion rate 1.6× cheaper Puzzle piece into groove estimated cost per run (USD, list price) Large dots are condition means; faint dots are individual trials (100 if completed, 0 otherwise) at their own cost. Scoring Every trial was scored by a human grader on the highest stage it reached, so a run that fails still records how far it got. The rubric is unchanged from the Fable report. 0 No purposeful approach 1 Made contact with the object 2 Lifted the object clear of the table 3 Positioned it above the deposit point 4 Placed it in its final position Astra places the block almost every time; on the puzzle it stalls where Fable does 2026-09-04T18:50:02.444943 image/svg+xml Matplotlib v3.11.1, https://matplotlib.org/ Fable 5 (n=20) Fable 5.1 (n=20) GPT-6 Astra (n=20) 2 13 3 2 6 2 2 8 19 Block into bowl 0 20 40 60 80 100 share of trials (%) Fable 5 (n=20) Fable 5.1 (n=20) GPT-6 Astra (n=20) 2 11 2 5 4 4 9 2 3 6 8 2 Puzzle piece into groove 0 no approach 1 contact 2 lifted 3 positioned 4 placed Share of trials per model reaching each stage; n per row is the number of trials in that cell. Results Task Model Mean stage Completions Rate Output tokens/run Est. cost/run Minutes/run Block into bowl Fable 5 1.30 1 / 20 5% 19.2k $2.69 8.2 Block into bowl Fable 5.1 2.40 8 / 20 40% 12.9k $2.12 6.8 Block into bowl GPT-6 Astra 3.95 19 / 20 95% 2.1k $0.94 2.5 Puzzle into groove Fable 5 1.50 0 / 20 0% 16.3k $2.63 7.9 Puzzle into groove Fable 5.1 2.35 2 / 20 10% 10.5k $2.18 5.9 Puzzle into groove GPT-6 Astra 2.00 2 / 20 10% 2.7k $1.36 3.4 All runs Every counted trial, 120 in total. Task Model Score Time (min) Output tokens Transcript Video Rerun Block into bowl GPT-6 Astra 4 1.7 1,533 view watch download Block into bowl GPT-6 Astra 4 1.7 1,734 view watch download Block into bowl GPT-6 Astra 4 1.7 1,573 view watch download Block into bowl GPT-6 Astra 4 1.8 1,669 view watch download Block into bowl GPT-6 Astra 4 1.9 1,570 view watch download Block into bowl GPT-6 Astra 4 2.1 1,491 view watch download Block into bowl GPT-6 Astra 4 2.1 2,153 view watch download Block into bowl GPT-6 Astra 4 2.1 1,782 view watch download Block into bowl GPT-6 Astra 4 2.2 2,034 view watch download Block into bowl GPT-6 Astra 4 2.5 2,441 view watch download Block into bowl GPT-6 Astra 4 2.5 2,625 view watch download Block into bowl GPT-6 Astra 4 2.5 1,697 view watch download Block into bowl GPT-6 Astra 4 2.8 2,434 view watch download Block into bowl GPT-6 Astra 4 2.9 2,966 view watch download Block into bowl GPT-6 Astra 4 3.2 1,627 view watch download Block into bowl GPT-6 Astra 4 3.3 2,768 view watch download Block into bowl GPT-6 Astra 4 3.3 2,777 view watch download Block into bowl GPT-6 Astra 4 3.4 2,185 view watch download Block into bowl GPT-6 Astra 4 3.5 1,979 view watch download Block into bowl GPT-6 Astra 3 3.5 2,428 view watch download Block into bowl Fable 5 4 5.2 13,120 view watch download Block into bowl Fable 5 3 7.1 14,048 view watch download Block into bowl Fable 5 2 5.0 10,663 view watch download Block into bowl Fable 5 2 8.7 17,859 view watch download Block into bowl Fable 5 2 9.1 15,064 view watch download Block into bowl Fable 5 1 4.7 9,096 view watch download Block into bowl Fable 5 1 5.8 13,800 view watch download Block into bowl Fable 5 1 5.8 9,954 view watch download Block into bowl Fable 5 1 7.1 16,666 view watch download Block into bowl Fable 5 1 7.2 18,165 view watch download Block into bowl Fable 5 1 7.2 14,719 view watch download Block into bowl Fable 5 1 8.4 24,975 view watch download Block into bowl Fable 5 1 8.7 17,436 view watch download Block into bowl Fable 5 1 8.8 21,388 view watch download Block into bowl Fable 5 1 9.6 30,566 view watch download Block into bowl Fable 5 1 10.8 28,784 view watch download Block into bowl Fable 5 1 11.7 27,810 view watch download Block into bowl Fable 5 1 14.8 35,211 view watch download Block into bowl Fable 5 0 8.7 17,764 view watch download Block into bowl Fable 5 0 9.7 27,582 view watch download Block into bowl Fable 5.1 4 4.2 9,124 view watch download Block into bowl Fable 5.1 4 4.8 9,779 view watch download Block into bowl Fable 5.1 4 5.0 11,359 view watch download Block into bowl Fable 5.1 4 5.1 8,526 view watch download Block into bowl Fable 5.1 4 5.6 10,998 view watch download Block into bowl Fable 5.1 4 6.5 7,031 view watch download Block into bowl Fable 5.1 4 6.7 15,901 view watch download Block into bowl Fable 5.1 4 7.5 8,964 view watch download Block into bowl Fable 5.1 3 5.7 14,477 view watch download Block into bowl Fable 5.1 3 8.6 20,110 view watch download Block into bowl Fable 5.1 2 7.3 20,022 view watch download Block into bowl Fable 5.1 2 18.9 14,526 view watch download Block into bowl Fable 5.1 1 5.7 13,265 view watch download Block into bowl Fable 5.1 1 5.9 18,073 view watch download Block into bowl Fable 5.1 1 6.0 13,070 view watch download Block into bowl Fable 5.1 1 6.6 11,506 view watch download Block into bowl Fable 5.1 1 7.1 11,404 view watch download Block into bowl Fable 5.1 1 8.5 17,760 view watch download Block into bowl Fable 5.1 0 4.5 8,979 view watch download Block into bowl Fable 5.1 0 5.2 13,394 view watch download Puzzle into groove GPT-6 Astra 4 2.8 2,229 view watch download Puzzle into groove GPT-6 Astra 4 3.3 1,967 view watch download Puzzle into groove GPT-6 Astra 3 2.3 2,581 view watch download Puzzle into groove GPT-6 Astra 3 2.6 2,805 view watch download Puzzle into groove GPT-6 Astra 3 2.6 2,470 view watch download Puzzle into groove GPT-6 Astra 3 2.9 2,978 view watch download Puzzle into groove GPT-6 Astra 3 3.0 2,937 view watch download Puzzle into groove GPT-6 Astra 3 3.1 2,932 view watch download Puzzle into groove GPT-6 Astra 3 3.6 2,538 view watch download Puzzle into groove GPT-6 Astra 3 3.6 3,417 view watch download Puzzle into groove GPT-6 Astra 2 3.9 3,142 view watch download Puzzle into groove GPT-6 Astra 1 2.2 2,458 view watch download Puzzle into groove GPT-6 Astra 1 2.8 2,464 view watch download Puzzle into groove GPT-6 Astra 1 3.0 2,117 view watch download Puzzle into groove GPT-6 Astra 1 3.1 2,901 view watch download Puzzle into groove GPT-6 Astra 1 3.9 3,041 view watch download Puzzle into groove GPT-6 Astra 1 10.2 2,491 view watch download Puzzle into groove GPT-6 Astra 0 2.6 3,025 view watch download Puzzle into groove GPT-6 Astra 0 2.8 2,916 view watch download Puzzle into groove GPT-6 Astra 0 3.3 3,161 view watch download Puzzle into groove Fable 5 3 5.4 12,599 view watch download Puzzle into groove Fable 5 3 6.6 11,044 view watch download Puzzle into groove Fab
관련 소식