bix-47-q3 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

Which gene has the most non-reference variants in the oldest male carrier?

Accepted answer: NOTCH1
Domain: variant callset

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 5.49M total	3/3 replicate 1 replicate 2 replicate 3 1.09M total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 7.07M total	3/3 replicate 1 replicate 2 replicate 3 1.17M total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 17.99M total	3/3 replicate 1 replicate 2 replicate 3 1.46M total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 33.96M total	3/3 replicate 1 replicate 2 replicate 3 2.26M total
DeepSeek V4 Pro via Claude Codesuperseded	3/3 replicate 1 replicate 2 replicate 3 7.34M total	3/3 replicate 1 replicate 2 replicate 3 1.90M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

passUDT-only

AnswerNOTCH1

Workflow

Galaxy UDT

Total tokens: 2.46M
Output: 17.9K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

passUDT-only

AnswerNOTCH1

Workflow

Galaxy UDT

Total tokens: 1.71M
Output: 17.7K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

passUDT-only

AnswerNOTCH1

Workflow

Galaxy UDT

Total tokens: 1.31M
Output: 19.4K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularMatch rowsTp Sort Header ToolGroup and aggregateCount rows

Total tokens: 2.30M
Output: 10.1K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularMatch rowsSort rowsTp Head ToolCount rows

Total tokens: 3.22M
Output: 10.6K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

passUDT-only

AnswerNOTCH1

Workflow

Galaxy UDT

Total tokens: 1.54M
Output: 15.9K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

passUDT + standard tools

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularMatch rowsTp Sort Header ToolDatamash summaryGalaxy UDT

Total tokens: 4.03M
Output: 27.5K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularTp Sort Header ToolJoin datasetsDatamash summary

Total tokens: 6.98M
Output: 24.1K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularCount rowsJoin datasets

Total tokens: 6.97M
Output: 31.2K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

AnswerNOTCH1

Workflow

Galaxy via BioBlend

Total tokens: 4.51M
Output: 34.6K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

AnswerNOTCH1

Workflow

Galaxy via BioBlendpysam

Total tokens: 19.97M
Output: 70.3K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

AnswerNOTCH1

Workflow

Galaxy via BioBlend

Total tokens: 9.48M
Output: 39.1K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularGroup and aggregate

Total tokens: 2.20M
Output: 20.6K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter TabularDatamash summary

Total tokens: 2.18M
Output: 18.4K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

AnswerNOTCH1

Workflow

Excel to tabularFilter rowsGroup and aggregateCount rows

Total tokens: 2.96M
Output: 27.0K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 273.1K
Output: 4.2K

Run trace

Codex GPT-5.5

replicate 2

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 344.3K
Output: 5.9K

Run trace

Codex GPT-5.5

replicate 3

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 474.1K
Output: 5.2K

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 306.3K
Output: 3.7K

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 476.9K
Output: 3.4K

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 382.2K
Output: 4.4K

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 351.1K
Output: 5.8K

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 518.5K
Output: 8.2K

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 588.1K
Output: 8.0K

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 1.05M
Output: 18.7K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 618.0K
Output: 11.5K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 592.3K
Output: 13.2K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 594.4K
Output: 7.7K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 723.3K
Output: 10.2K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

AnswerNOTCH1

Workflow

Local analysis

Total tokens: 584.6K
Output: 6.1K

Run trace

bix-47-q3 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

AnswerNOTCH1

Total tokens: 2.00M
Input: 1.98M
Cached input: 1.93M
Output: 11.1K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

AnswerNOTCH1

Total tokens: 1.53M
Input: 1.51M
Cached input: 1.45M
Output: 13.5K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

AnswerNOTCH1

Total tokens: 1.27M
Input: 1.27M
Cached input: 1.22M
Output: 7.8K

Run trace Final answer Galaxy history

Open-ended code averaged 0.39M tokens; previous website Galaxy averaged 2.36M; the new Galaxy runs averaged 1.60M, or 4.11× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs