BixBench current item

bix-34-q5

Question

What is the ratio of median mean patristic distances between fungi and animals?

Accepted answer
1.95
Domain
phylogenetics

Comparison

Pass counts and token totals across the same three replicates.

AgentGalaxy-API codeOpen-ended code
Codex GPT-5.53/3
replicate 1 replicate 2 replicate 3
2.64M total		3/3
replicate 1 replicate 2 replicate 3
724.2K total
Codex GPT-5.6 Sol3/3
replicate 1 replicate 2 replicate 3
16.52M total		3/3
replicate 1 replicate 2 replicate 3
728.5K total
Codex GPT-5.6 Luna3/3
replicate 1 replicate 2 replicate 3
19.98M total		3/3
replicate 1 replicate 2 replicate 3
2.50M total
DeepSeek V4 Pro via Codex3/3
replicate 1 replicate 2 replicate 3
10.49M total		3/3
replicate 1 replicate 2 replicate 3
5.17M total
DeepSeek V4 Pro via Claude Codesuperseded2/3
replicate 1 replicate 2 replicate 3
16.65M total		0/3
replicate 1 replicate 2 replicate 3
6.18M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates
Codex GPT-5.5

replicate 1

passUDT-only
Answer1.9474821091918069
Workflow
Galaxy UDTBiopythonNewick analysisMAFFTmedian summary
Total tokens
903.4K
Output
6.7K
Galaxy historyRun trace
Codex GPT-5.5

replicate 2

passUDT-only
Answer1.9474821091918069
Workflow
Galaxy UDTBiopythonNewick analysismedian summary
Total tokens
969.9K
Output
9.2K
Galaxy historyRun trace
Codex GPT-5.5

replicate 3

passUDT-only
Answer1.9474821091918069
Workflow
Galaxy UDTPhyKITBiopythonNewick analysismedian summary
Total tokens
770.3K
Output
7.0K
Galaxy historyRun trace

Codex GPT-5.6 Sol

3 primary replicates
Codex GPT-5.6 Sol

replicate 1

pass
Answer1.9475011462631819
Workflow
UnzipPhykit Tree BasedNewick analysismedian summary
Total tokens
14.57M
Output
25.0K
Galaxy historyRun trace
Codex GPT-5.6 Sol

replicate 2

passUDT-only
Answer1.9474821091918069
Workflow
Galaxy UDTBiopythonmedian summary
Total tokens
754.1K
Output
9.4K
Galaxy historyRun trace
Codex GPT-5.6 Sol

replicate 3

passUDT-only
Answer1.9474821091918071
Workflow
Galaxy UDTBiopythonMAFFT
Total tokens
1.20M
Output
9.3K
Galaxy historyRun trace

Codex GPT-5.6 Luna

3 primary replicates
Codex GPT-5.6 Luna

replicate 1

passUDT + standard tools
Answer1.9475011462631819
Workflow
Phykit Tree BasedGalaxy UDTBUSCOMAFFTNewick analysismedian summary
Total tokens
5.43M
Output
25.9K
Galaxy historyRun trace
Codex GPT-5.6 Luna

replicate 2

passUDT-only
Answer1.9474821091918069
Workflow
Galaxy UDTBiopythonNewick analysismean summarymedian summary
Total tokens
7.63M
Output
18.8K
Galaxy historyRun trace
Codex GPT-5.6 Luna

replicate 3

passUDT + standard tools
Answer1.9474821091918070760703711426358512838795938936646738382911981436127296859207774
Workflow
UnzipPhykit Tree BasedGalaxy UDTmore stepsSummary statisticsNewick analysis
Total tokens
6.91M
Output
29.2K
Galaxy historyRun trace

DeepSeek V4 Pro via Codex

3 primary replicates
DeepSeek V4 Pro via Codex

replicate 1

passUDT-only
Answer1.95
Workflow
Galaxy UDTPhyKITMAFFTmedian summary
Total tokens
3.01M
Output
24.5K
Galaxy historyRun trace
DeepSeek V4 Pro via Codex

replicate 2

passUDT + standard tools
Answer1.95
Workflow
Galaxy UDTDatamash summaryPhyKITMAFFT
Total tokens
4.10M
Output
31.4K
Galaxy historyRun trace
DeepSeek V4 Pro via Codex

replicate 3

passUDT + standard tools
Answer1.95
Workflow
Galaxy UDTDatamash summaryMAFFTIQ-TREEPhyKITBiopython
Total tokens
3.38M
Output
32.8K
Galaxy historyRun trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates
DeepSeek V4 Pro via Claude Code

replicate 1

passUDT-only
Answer1.9474
Workflow
Galaxy UDTPhyKITNewick analysismedian summary
Total tokens
2.12M
Output
23.7K
Galaxy historyRun trace
DeepSeek V4 Pro via Claude Code

replicate 2

passUDT-only
Answer1.94748
Workflow
Galaxy via BioBlendBUSCONewick analysisMAFFTmedian summarymean summary
Total tokens
2.63M
Output
27.7K
Galaxy historyRun trace
DeepSeek V4 Pro via Claude Code

replicate 3

failUDT + standard tools

Why it missed

Execution failureThis run spent 101 turns trying to complete the Galaxy route and never wrote an answer. The complete route summarizes all 100 animal and 249 fungal trees and returns about 1.95.
Answernot produced
Workflow
UnzipBUSCOMAFFTPhyKITIQ-TREENewick analysis
Total tokens
11.89M
Output
96.6K
Galaxy historyRun trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates
Codex GPT-5.5

replicate 1

pass
Answer1.94748210919181
Workflow
Local analysisBUSCOMAFFTNewick analysisIQ-TREEmedian summary
Total tokens
227.1K
Output
6.1K
Run trace
Codex GPT-5.5

replicate 2

pass
Answer1.94748210919181
Workflow
Local analysisBUSCOIQ-TREEMAFFTNewick analysismean summary
Total tokens
200.9K
Output
6.4K
Run trace
Codex GPT-5.5

replicate 3

pass
Answer1.94748210919181
Workflow
Local analysisBUSCONewick analysismedian summarymean summaryBiopython
Total tokens
296.2K
Output
7.5K
Run trace

Codex GPT-5.6 Sol

3 primary replicates
Codex GPT-5.6 Sol

replicate 1

pass
Answer1.947482109191807
Workflow
Local analysis
Total tokens
257.4K
Output
3.5K
Run trace
Codex GPT-5.6 Sol

replicate 2

pass
Answer1.947482109191807
Workflow
Local analysis
Total tokens
232.2K
Output
4.9K
Run trace
Codex GPT-5.6 Sol

replicate 3

pass
Answer1.947482109191807
Workflow
Local analysis
Total tokens
238.9K
Output
6.0K
Run trace

Codex GPT-5.6 Luna

3 primary replicates
Codex GPT-5.6 Luna

replicate 1

pass
Answer1.947482109191807
Workflow
Local analysis
Total tokens
901.7K
Output
11.7K
Run trace
Codex GPT-5.6 Luna

replicate 2

pass
Answer1.947482109191807
Workflow
Local analysis
Total tokens
610.2K
Output
18.2K
Run trace
Codex GPT-5.6 Luna

replicate 3

pass
Answer1.9474821091918071
Workflow
Local analysis
Total tokens
984.9K
Output
14.7K
Run trace

DeepSeek V4 Pro via Codex

3 primary replicates
DeepSeek V4 Pro via Codex

replicate 1

pass
Answer1.95
Workflow
Local analysisBUSCOIQ-TREEMAFFTSciPyBiopython
Total tokens
994.4K
Output
18.0K
Run trace
DeepSeek V4 Pro via Codex

replicate 2

pass
Answer1.95
Workflow
Local analysisBUSCONewick analysismedian summarymean summaryBiopython
Total tokens
1.28M
Output
14.4K
Run trace
DeepSeek V4 Pro via Codex

replicate 3

pass
Answer1.95
Workflow
Local analysisBUSCOMAFFTIQ-TREESciPyscikit-learn
Total tokens
2.90M
Output
23.7K
Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates
DeepSeek V4 Pro via Claude Code

replicate 1

fail

Why it missed

Tree population changedThis run selected 96 shared orthogroups and rebuilt each as a combined animal-plus-fungal tree. That is a different tree population from independently summarizing the supplied animal and fungal trees and produced 1.11.
Answer1.112544928253171
Workflow
Local analysisBUSCOMAFFTFastTreeNewick analysisSciPy
Total tokens
3.68M
Output
56.2K
Run trace
DeepSeek V4 Pro via Claude Code

replicate 2

fail

Why it missed

Tree population changedThis run used only the 96 orthogroups shared by both archives, excluding the remaining supplied trees. The restricted medians produced 1.70 instead of about 1.95.
Answer1.703604
Workflow
Local analysisBUSCOMAFFTBiopythonNewick analysismean summary
Total tokens
1.84M
Output
21.9K
Run trace
DeepSeek V4 Pro via Claude Code

replicate 3

fail

Why it missed

Wrong distance sourceThis run summarized IQ-TREE .mldist model-distance matrices. The question asks for patristic distances along the supplied .treefile trees, and the substitution produced 1.83 instead of about 1.95.
Answer1.8302633986
Workflow
Local analysisBUSCOMAFFTmean summarymedian summary
Total tokens
657.6K
Output
10.4K
Run trace

bix-34-q5 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Open-ended code averaged 0.24M tokens; previous website Galaxy averaged 5.51M; the new Galaxy runs averaged 0.75M, or 3.10× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs