BioBasicsBench: AI That Understands Experimental Data—How Far Are We From a Reliable Lab Assistant?

In a laboratory, an image may not be clear enough, a record may contain contradictions, and results may not necessarily support the initial hypothesis. For AI to become a useful research assistant, it must determine which conclusions hold up based on this incomplete evidence. On November 6, 2026, OpenAI and the enzyme design company Ligo launched BioBasicsBench, introducing 100 routine wet lab analysis tasks as a new evaluation dataset. This provides a specific lens through which to observe how models handle experimental data that has not yet been organized into a research paper. (Official Research Statement)

Wet lab experiments typically involve conducting hands-on experiments on biological samples. In this context, the AI is provided with materials generated by the experiment, including images, lab reports, and raw instrument data; its task is to use this information to interpret the results, identify problems, or point out gaps in the evidence, rather than to directly operate the experimental equipment. (Explanation of Assignment Formats)

How should a score of 66.7% be interpreted?

Under the xhigh inference settings, GPT-6 Astra achieved an average of 66.7% of the total possible score, while GPT-5.6 Sol achieved 43.8%. Scoring is based on the specific criteria for each task, and partial credit may be awarded; each model was allowed five attempts per task. Therefore, this figure cannot be interpreted as an experiment success rate, nor does it imply that the models independently completed approximately two-thirds of the experiments. Rather, it measures the extent to which the responses met requirements regarding conclusions, measurements, reasoning, or lack of evidence. (Test Methods and Results)

While partial scoring is useful for tracking changes in performance, it also raises deployment issues: a model might score well on a particular question but still provide an incorrect recommendation. For a lab, the consequences of these two scenarios differ significantly. When purchasing or evaluating such systems, in addition to looking at the overall score, it is worth asking which errors could influence subsequent decisions, whether they are easily detectable, and whether the model can pause when evidence is insufficient.

Scientific judgments must stand up to counterevidence.

The value of raw data lies in the fact that it forces the model to grapple with conflicts between on-site evidence and existing knowledge. Even among the failure cases listed by OpenAI, there are instances where models allowed conventional assumptions to override counterevidence. (The Limits of Capability in Research) This suggests that there is still a gap between providing answers that appear professional and explain technical terms, and ensuring that the reasoning process is reliable.

From a practical standpoint, a helpful analysis should distinguish between observations, interpretations, and points requiring confirmation. Readers need to know exactly what the model has observed, which elements stem from background knowledge, and on which assumptions the conclusions rely. If an answer conflates these levels, it becomes difficult for researchers to quickly pinpoint the problem. Conversely, even if the model does not provide a definitive conclusion at the moment, simply pointing out precisely what kind of evidence is missing can help avoid fruitless discussions.

These are also dimensions worth adding to scientific AI evaluations: when encountering conflicting information, will the model proactively reduce the strength of its conclusions; if the order of the presented material is changed, will its judgments remain consistent; and when the same fact is repeatedly emphasized, will it overshadow more critical counterexamples? These are recommendations for future evaluations, not capabilities that have already been demonstrated by the results of this study.

Incorporate Review into the Workflow

For teams wishing to try out AI analysis, a prudent starting point is to use it for support tasks with clearly defined scope and verifiable results. You can conduct a comparative evaluation using existing, expert-validated materials: first, have the system organize the evidence and list points of uncertainty, and then have staff decide which content should be included in the official record. When selecting samples, be sure to include ambiguous cases and instances of failure, and avoid choosing only samples with clear results and straightforward answers.

The review process also requires clearly defined subjects. Simply having personnel confirm the results one final time is unlikely to compensate for evidence that has already been lost earlier in the process. Ideally, the system should link key decisions to their corresponding locations in the source materials, retain version histories and revision logs, and distinguish between recommendations generated by the model and conclusions confirmed by personnel. Only then can we identify where improvements are needed when errors are discovered, rather than leaving responsibility buried in a smooth-sounding summary.

At the same time, professional review takes time. If every observation generated by the model must be rechecked, the apparent speed of generation may not necessarily translate into an overall increase in efficiency. A trial evaluation should document the time spent on review, the cost of correcting errors, and any significant omissions in order to determine whether it truly reduces the burden on researchers.

Evaluations should also account for changes to the system itself. Model upgrades, adjustments to input formats, or the addition of new analytical tools can all alter existing performance. For assistants that have been integrated into daily workflows, teams need to maintain a fixed set of regression samples to verify that changes have not introduced new critical errors. Testing only once at the time of procurement is insufficient to support long-term reliance on the same set of results.

The next step is to see how reliability is implemented.

The significance of BioBasicsBench lies in grounding the discussion of AI’s research capabilities in the day-to-day handling of evidence. In my view, the product goal that lab assistants should prioritize in the near term is to make it easier to review evidence and to identify uncertainties at an earlier stage. Only when researchers can clearly see when the system is helpful and when it should step back will performance improvements have a chance to translate into trustworthy collaboration.