bix-18-q3 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

What is the approximate percent reduction in mean colony area for the ΔlasR mutant compared to wildtype?

Accepted answer: (69,72)
Domain: general

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 4.11M total	3/3 replicate 1 replicate 2 replicate 3 187.7K total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 3.04M total	3/3 replicate 1 replicate 2 replicate 3 245.7K total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 11.11M total	3/3 replicate 1 replicate 2 replicate 3 240.0K total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 3.88M total	3/3 replicate 1 replicate 2 replicate 3 371.2K total
DeepSeek V4 Pro via Claude Codesuperseded	3/3 replicate 1 replicate 2 replicate 3 3.35M total	3/3 replicate 1 replicate 2 replicate 3 352.5K total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer71%

Workflow

Datamash summaryCsv To TabularMatch rowsSummary statistics

Total tokens: 2.07M
Output: 8.1K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

pass

Answer71%

Workflow

Csv To TabularDatamash summarycolony morphology summary

Total tokens: 1.20M
Output: 6.7K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

pass

Answer71.0%

Workflow

Csv To TabularGroup and aggregateRemove header

Total tokens: 834.3K
Output: 6.2K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer71%

Workflow

Csv To TabularDatamash summaryMatch rowsSummary statisticscolony morphology summarymean summary

Total tokens: 1.60M
Output: 9.9K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer71%

Workflow

Datamash summarycolony morphology summarymean summary

Total tokens: 720.5K
Output: 5.7K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer71%

Workflow

Csv To TabularGroup and aggregatecolony morphology summary

Total tokens: 724.5K
Output: 4.4K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer71%

Workflow

Filter TabularRemove headerGroup and aggregateDatamash TransposeAdd derived columncolony morphology summary

Total tokens: 3.13M
Output: 20.2K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer71.0%

Workflow

Csv To TabularFilter TabularDatamash summarycolony morphology summary

Total tokens: 3.97M
Output: 18.5K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer71%

Workflow

Csv To TabularDatamash summarySelect columnsmore stepsSummary statisticscolony morphology summary

Total tokens: 4.02M
Output: 24.8K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer71%

Workflow

Datamash summaryFilter rowsSummary statisticscolony morphology summary

Total tokens: 1.85M
Output: 19.2K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer71%

Workflow

Datamash summary

Total tokens: 320.3K
Output: 3.2K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer71%

Workflow

Group and aggregateDatamash summarycolony morphology summary

Total tokens: 1.71M
Output: 13.0K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

Answer71%

Workflow

Group and aggregatecolony morphology summary

Total tokens: 1.34M
Output: 11.3K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer71%

Workflow

Group and aggregateRemove headercolony morphology summary

Total tokens: 1.12M
Output: 8.3K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer71%

Workflow

Datamash summarycolony morphology summary

Total tokens: 897.3K
Output: 8.9K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 62.9K
Output: 706

Run trace

Codex GPT-5.5

replicate 2

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 46.6K
Output: 604

Run trace

Codex GPT-5.5

replicate 3

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 78.1K
Output: 625

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer71%

Workflow

Local analysis

Total tokens: 72.0K
Output: 681

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer71%

Workflow

Local analysis

Total tokens: 86.8K
Output: 687

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer71%

Workflow

Local analysis

Total tokens: 86.8K
Output: 693

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer71%

Workflow

Local analysis

Total tokens: 80.7K
Output: 1.3K

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer71%

Workflow

Local analysis

Total tokens: 93.3K
Output: 1.1K

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer71%

Workflow

Local analysis

Total tokens: 66.0K
Output: 983

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 76.8K
Output: 1.8K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 121.4K
Output: 2.6K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer71%

Workflow

Local analysismean summary

Total tokens: 173.0K
Output: 2.9K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

Answer70.99%

Workflow

Local analysis

Total tokens: 117.3K
Output: 1.1K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer71%

Workflow

Local analysis

Total tokens: 117.6K
Output: 1.2K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer71%

Workflow

Local analysis

Total tokens: 117.6K
Output: 1.2K

Run trace

bix-18-q3 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

Answer71.0%

Total tokens: 489.2K
Input: 486.1K
Cached input: 449.5K
Output: 3.0K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

Answer71%

Total tokens: 319.8K
Input: 316.4K
Cached input: 293.6K
Output: 3.4K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

Answer71%

Total tokens: 413.2K
Input: 409.4K
Cached input: 388.0K
Output: 3.7K

Run trace Final answer Galaxy history

Open-ended code averaged 0.08M tokens; previous website Galaxy averaged 1.01M; the new Galaxy runs averaged 0.41M, or 4.97× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs