CompBioBench v1

Experiment totals

• Coverage: each row combines the published 100-item replicates for that model and condition. Astra has one replicate; the other groups have three. Adopted reruns replace the original run's usage; superseded attempts are not added. Total tokens are input plus output; reasoning output is part of output and is not charged twice.
• Astra: reported token usage is included; API-equivalent cost is not estimated.
• OpenAI costs: API-equivalent estimates using the documented pricing schedule, not invoice totals. Fast processing rates per 1M uncached input / cached input / output tokens were GPT-5.5 $12.50 / $1.25 / $75; Sol $10 / $1 / $60 before its August 21 price reduction and $8 / $0.80 / $40 afterward; and Luna $0.40 / $0.04 / $2.40 for these runs. The adopted GPT-5.5 standard-service rerun uses $5 / $0.50 / $30.
• DeepSeek-v4-pro-0813 costs: the rate before August 16 at 16:00 UTC was $0.435 / $0.003625 / $0.87. The published peak rate is $1.32 / $0.044 / $3.96 and the off-peak rate is $0.66 / $0.022 / $1.98. Peak hours are 01:00-04:00 and 06:00-10:00 UTC, Monday-Friday.
• Two DeepSeek-v4-pro-0813 totals: “Estimated run-time cost” uses the documented historical schedule and each selected run's interval; “Same tokens at published DeepSeek rates” reprices the same usage under the peak/off-peak schedule documented August 27, 2026.
• Timing estimate: providers report usage per run rather than per minute, so tokens are distributed evenly across the few runs that crossed a price boundary.
• Execution comparison: Galaxy / code compares total tokens for matched question/replicate pairs of the same model. Pairs with missing usage on either side are excluded from both sides of this ratio; missing usage is not imputed. There are 299 matched pairs for GPT-5.5, and 300 for DeepSeek-v4-pro-0813, Sol and Luna. The other columns retain all reported usage. Astra has no Galaxy counterpart.

Prices checked August 27, 2026: OpenAI Fast mode · OpenAI models · DeepSeek pricing · DeepSeek price change.