bix-18-q1 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

In a P. aeruginosa swarming analysis, what is the mean circularity measurement for the genotype showing the largest mean area?

Accepted answer: (0.07,0.08)
Domain: general

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 2.42M total	3/3 replicate 1 replicate 2 replicate 3 141.8K total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 3.19M total	3/3 replicate 1 replicate 2 replicate 3 200.4K total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 6.51M total	3/3 replicate 1 replicate 2 replicate 3 213.9K total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 4.10M total	3/3 replicate 1 replicate 2 replicate 3 246.0K total
DeepSeek V4 Pro via Claude Codesuperseded	3/3 replicate 1 replicate 2 replicate 3 3.10M total	3/3 replicate 1 replicate 2 replicate 3 329.5K total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer0.076

Workflow

Csv To TabularDatamash summarycolony morphology summary

Total tokens: 647.1K
Output: 4.9K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

pass

Answer0.076

Workflow

Select trailing rowsGroup and aggregatecolony morphology summary

Total tokens: 1.10M
Output: 4.6K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

pass

Answer0.076

Workflow

Csv To TabularDatamash summarycolony morphology summary

Total tokens: 679.3K
Output: 6.1K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer0.076

Workflow

Csv To TabularGroup and aggregateShow Tail1colony morphology summary

Total tokens: 1.52M
Output: 7.0K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer0.076

Workflow

Datamash summaryCsv To Tabularcolony morphology summary

Total tokens: 740.5K
Output: 5.5K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer0.076

Workflow

Group and aggregateRemove headercolony morphology summary

Total tokens: 920.4K
Output: 5.5K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer0.076

Workflow

Csv To TabularDatamash summarycolony morphology summary

Total tokens: 1.43M
Output: 13.1K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer0.076

Workflow

Group and aggregateFilter Tabularcolony morphology summary

Total tokens: 1.72M
Output: 17.5K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer0.076

Workflow

Csv To TabularDatamash summarycolony morphology summary

Total tokens: 3.37M
Output: 19.4K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer0.076

Workflow

Datamash summary

Total tokens: 712.1K
Output: 8.0K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer0.076

Workflow

Csv To TabularGroup and aggregateRemove header

Total tokens: 1.81M
Output: 15.1K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer0.076

Workflow

Group and aggregateRemove headerSort rowsShow Beginning1Select columnscolony morphology summary

Total tokens: 1.58M
Output: 12.2K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

Answer0.076

Workflow

Group and aggregateRemove headercolony morphology summary

Total tokens: 614.9K
Output: 8.9K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer0.076

Workflow

Group and aggregateDatamash summary

Total tokens: 1.02M
Output: 10.6K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer0.076

Workflow

Group and aggregateDatamash summary

Total tokens: 1.46M
Output: 17.8K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer0.076

Workflow

Local analysiscolony morphology summary

Total tokens: 45.0K
Output: 697

Run trace

Codex GPT-5.5

replicate 2

pass

Answer0.076

Workflow

Local analysiscolony morphology summary

Total tokens: 48.6K
Output: 641

Run trace

Codex GPT-5.5

replicate 3

pass

Answer0.076

Workflow

Local analysiscolony morphology summary

Total tokens: 48.2K
Output: 535

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer0.076

Workflow

Local analysis

Total tokens: 71.8K
Output: 513

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer0.076

Workflow

Local analysis

Total tokens: 71.6K
Output: 555

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer0.076

Workflow

Local analysis

Total tokens: 57.0K
Output: 560

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer0.076

Workflow

Local analysis

Total tokens: 80.5K
Output: 930

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer0.076

Workflow

Local analysis

Total tokens: 65.8K
Output: 899

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer0.076

Workflow

Local analysis

Total tokens: 67.6K
Output: 1.3K

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer0.076

Workflow

Local analysiscolony morphology summarymean summary

Total tokens: 74.8K
Output: 1.6K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer0.076

Workflow

Local analysiscolony morphology summarymean summary

Total tokens: 77.9K
Output: 3.5K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer0.076

Workflow

Local analysismean summarycolony morphology summary

Total tokens: 93.3K
Output: 2.1K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

Answer0.076

Workflow

Local analysiscolony morphology summary

Total tokens: 117.8K
Output: 1.1K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer0.076

Workflow

Local analysiscolony morphology summary

Total tokens: 93.7K
Output: 1.1K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer0.076

Workflow

Local analysismean summarycolony morphology summary

Total tokens: 118.0K
Output: 1.3K

Run trace

bix-18-q1 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

Answer0.076

Total tokens: 361.0K
Input: 358.1K
Cached input: 338.8K
Output: 2.9K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

Answer0.076

Total tokens: 560.6K
Input: 556.9K
Cached input: 514.6K
Output: 3.7K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

Answer0.076

Total tokens: 612.1K
Input: 608.0K
Cached input: 575.4K
Output: 4.1K

Run trace Final answer Galaxy history

Open-ended code averaged 0.07M tokens; previous website Galaxy averaged 1.06M; the new Galaxy runs averaged 0.51M, or 7.65× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs