에이전트 과학을 위한 역 루브릭 최적화 테스트베드

“It is important to draw wisdom from many different places. If you take it from only one place, it becomes rigid and stale.” — Uncle Iroh At Fulcrum Research, we study the performance and behavior of long-horizon agents. Although each task setting has its own specific structure, we believe it’s possible to find general principles of agent performance across settings, each contributing to a nascent agent science. In this post, we motivate the difficulty of finding suitable settings for agent science and propose inverse rubric optimization (IRO) settings, in which an agent has to optimize the preferences of a blackbox judge it has variable access to. We observe these tasks induce rich behavior and smooth scaling. We find that frontier models effectively iterate and improve with more judge access but by default do not maximally use the resources provided to them. Notably, Fable 5 outperforms all models given smaller amounts of labels, but does not improve at the largest budget and plateaus around the level of Opus 4.6. We open source code here . Testbeds for agent science Studying the behavior of agents is challenging due to the variance and cost of long horizon tasks. Trajectory-level variance is often notoriously high due to the many non-deterministic choices made in a run, making it hard to estimate the impact of various methods. Often this variance is precisely high in the tasks with large action spaces. But those tasks are those that induce the exact complex behaviors we intend to study and intervene on in our experiments. The challenge is then to find settings that require general kinds of capability and benefit from a broad range of strategies, like resource utilization, exploration, hypothesis testing, etc., while being smooth enough for research. To remedy this, we look at toy settings that remain challenging, rich and smooth. Inverse rubric optimization In an IRO task, the agent being evaluated has the goal of learning the preferences of a black-box judge model, parametrized by some judging rubric. It submits a policy for generation, e.g. a prompt, a scaffold, etc. which is then used to generate domain samples which are evaluated by the judge. It has to learn and explore the judge preferences by studying its scores and submitting new attempts. In IRO tasks we vary the budget of judge labels that the optimizer can sample. The dynamics of how the optimizer interacts and samples that resource, whether it tests hypotheses, whether it properly considers sampling noise, etc. constitute the main interest of the setting. In this post, we study a specific set of IRO tasks specified by various black-box LLM poetry judges. We generated a set of poem topics, as well as various poetry rubrics that grade poems along various stylistic and textual features, guided by the style of various poets. The optimizer agent then iterates on a prompt that is used by a generator model to generate poems which are scored by the judge. Its goal is to hillclimb the judge’s mean score on the generated poems. Each of our rubrics grades a set of features additively with partial credit, so that partial stylistic discoveries move the score (see Appendix A.1 for the full rubric of our Milton judge). 1 In the poetry setting, the optimizer agent is given a submit_train_batch(generation_prompt, num_labels) tool that takes in a set of guidelines for poem generation, and can generate num_labels poems using train set topics that are then scored and returned to the optimizer. Each run has a label budget that it can use, and at some point submits a final prompt that is evaluated on the eval set. It can call the tool as many times as it wants until its budget has been consumed. We observed that different judges define different optimization landscapes, in terms of the range of scores and the difficulty of the learning problem they pose to the optimizer (see Appendix A.3 for per-judge learning curves). To normalize the difference in scale, we normalize by the gap between the performance of a policy that sees the hidden rubric and a policy that has no guidance. This gap provides a natural scale to benchmark optimizer progress against, although the rubric-visible score is not necessarily an upper bound 2 . We define the normalized score of the original mean judge score S as follows: \[\hat{S} = \frac{S - \text{blind mean}}{\text{rubric-visible mean} - \text{blind mean}}\] Results Performance is smooth in label budget and use In our preliminary experiments, we ran optimizer agents against 5 different poetry rubrics, 3 seeds each and varied the amount of access to judge labels. We used Opus 4.6, GPT-5.5, Fable 5, or Haiku 4.5 as the optimizer, Opus 4.6 as the judge, and always used Haiku 4.5 as the poem generator. Unless a caption says otherwise, plots average over these 5 judges × 3 seeds per optimizer and error bars are standard errors over runs: We can also track the average train performance scores for the smaller batches an agent submits along the trajectory to see how it progresses over time: Models employ rich strategies to learn the judge In our qualitative analysis of the trajectories, we find that models are able to iterate, test out hypotheses about the judge, and notice patterns that allow them to make progress. Models inspect per-sample scores, analyze low and high scoring samples to make hypotheses about plausible features, and run experiments where they isolate characteristics of good prompts. Fable and Opus seem more systematic than GPT-5.5 in terms of exploring a broad range of hypotheses and stylistic variations and isolating the effect of their interventions. As a concrete example, let’s look at a Fable 5 run against the Milton judge (the hardest of our five judges, see Appendix A.3 ) at budget 1000. The run does systematic testing, statistical feature checks and ablations and closes 91% of the gap to the rubric-visible policy. The trajectory decomposes into six phases — select a phase in the figure to follow what the agent was doing: This is one of the best agent runs on Milton, in which the optimizer discovered almost all the features of the hidden rubric ( Appendix A.1 ). In some runs which we’ve excluded from our results, Fable made various attempts to reward-hack the blackbox judge through the generator. We document this in Appendix A.2 . Models use a smaller and smaller proportion of labels as budget increases However, models do not seem to maximally leverage the resources provided to them. We measure proportion of budget used vs total budget: The models seem to have a sense of the “natural” effort scale they put into the task, which is not necessarily strictly increasing with the amount of budget they have. Fable in particular never uses more than 1000 labels when given a budget of 10000, which explains its plateau at 10000, whereas Opus 4.6 keeps improving from 1000 to 10000. Models have different approaches to using their resource We measure how agents split their label spend across submit_train_batch calls within a trajectory, and averaging batch size by call index in the trajectory, on Budget 1000 runs: GPT-5.5 is quickly biased towards larger batch sizes, whereas Anthropic models seem to more gradually increase batch size throughout. What’s next We will soon release a second post presenting elicitation results in this setting. We have found simple and general interventions that allow us to scale agent performance to near saturation on our current settings by increasing the optimizer’s propensity to iterate on the task and use all its labels. We have also been studying a broader range of settings and are excited about doing generalization experiments on our method findings. Appendix Citation Information Please cite this post as: Girit, Uzay, Eleni Shor, Kaivalya Hariharan, and Rowan Huang. “Inverse Rubric Optimization: A testbed for agent science.” Fulcrum Research, June 2026. https://fulcrum.inc/2026/06/09/inverse-rubric-optimization.html Or in BibTeX: @misc { g