bix-12-q4 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

What is the Mann-Whitney U statistic when comparing parsimony informative site percentages between animals and fungi?

Accepted answer: 6948.0
Domain: phylogenetics

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 3.59M total	3/3 replicate 1 replicate 2 replicate 3 1.29M total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 21.00M total	3/3 replicate 1 replicate 2 replicate 3 1.82M total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 27.49M total	2/3 replicate 1 replicate 2 replicate 3 4.16M total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 12.51M total	3/3 replicate 1 replicate 2 replicate 3 3.04M total
DeepSeek V4 Pro via Claude Codesuperseded	1/3 replicate 1 replicate 2 replicate 3 9.17M total	0/3 replicate 1 replicate 2 replicate 3 3.67M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

passUDT-only

Answer6948

Workflow

Galaxy UDTMAFFTSciPyMann-Whitney U

Total tokens: 928.7K
Output: 8.4K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

passUDT-only

Answer6948

Workflow

Galaxy UDTPhyKITSciPyMAFFT

Total tokens: 1.21M
Output: 11.9K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsPhyKITmore stepsscikit-learnMAFFT

Total tokens: 1.45M
Output: 14.9K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsPhyKITMAFFT

Total tokens: 1.26M
Output: 12.2K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer6948

Workflow

UnzipPhykit Alignment BasedCollapse DatasetNonparametric Rank Tests

Total tokens: 17.10M
Output: 28.9K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsMAFFTPhyKIT

Total tokens: 2.64M
Output: 12.7K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

passUDT + standard tools

Answer6948

Workflow

UnzipPhykit Alignment BasedGalaxy UDTmore stepsSciPyMann-Whitney U

Total tokens: 14.33M
Output: 38.9K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

passUDT + standard tools

Answer6948

Workflow

UnzipPhykit Alignment BasedGalaxy UDTNonparametric Rank TestsSciPyMAFFT

Total tokens: 4.70M
Output: 23.2K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

passUDT + standard tools

Answer6948

Workflow

UnzipPhykit Alignment BasedGalaxy UDTNonparametric Rank TestsMAFFTBUSCO

Total tokens: 8.45M
Output: 33.2K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsMAFFTPhyKIT

Total tokens: 3.00M
Output: 28.6K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsMAFFTPhyKIT

Total tokens: 5.56M
Output: 24.8K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

passUDT + standard tools

Answer6948

Workflow

Galaxy UDTNonparametric Rank TestsPhyKITMAFFT

Total tokens: 3.95M
Output: 23.0K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

failUDT + standard tools

Why it missed

Scientific implementation mismatchThis custom UDT produced a parsimony-informative-site distribution that differed from the gap-aware implementation, yielding U=6160 instead of 6948.

Answer6160

Workflow

Phykit Alignment BasedGalaxy UDTNonparametric Rank TestsMAFFT

Total tokens: 4.16M
Output: 34.5K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

failUDT + standard tools

Why it missed

Scientific implementation mismatchThis custom UDT produced a different parsimony-informative-site distribution before the test, yielding U=7048.5 instead of 6948.

Answer7048.5

Workflow

Galaxy UDTNonparametric Rank TestsBiopythonPhyKITMAFFT

Total tokens: 2.03M
Output: 21.7K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

passUDT + standard tools

Answer6948

Workflow

UnzipPhykit Alignment BasedGalaxy UDTmore stepsBiopythonMAFFT

Total tokens: 2.98M
Output: 35.7K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer6948

Workflow

Local analysisMAFFTSciPymean summarymedian summary

Total tokens: 359.2K
Output: 9.5K

Run trace

Codex GPT-5.5

replicate 2

pass

Answer6948.0

Workflow

Local analysisBUSCOMAFFTSciPyBiopythonMann-Whitney U

Total tokens: 338.3K
Output: 7.2K

Run trace

Codex GPT-5.5

replicate 3

pass

Answer6948.0

Workflow

Local analysisBUSCOMAFFTSciPyBiopython

Total tokens: 596.1K
Output: 8.2K

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer6948

Workflow

Local analysis

Total tokens: 713.8K
Output: 8.1K

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer6948

Workflow

Local analysis

Total tokens: 579.5K
Output: 7.8K

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer6948

Workflow

Local analysis

Total tokens: 528.6K
Output: 7.2K

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

fail

Why it missed

Alignment population was restrictedThis run required every alignment to contain all four taxa, reducing the test to 22 animal and 211 fungal observations and producing U=1162. The benchmark comparison uses all 241 animal and 255 fungal alignments and gives U=6948.

Answer1162.0

Workflow

Local analysis

Total tokens: 1.67M
Output: 32.3K

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer6948

Workflow

Local analysis

Total tokens: 1.25M
Output: 24.1K

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer6948

Workflow

Local analysis

Total tokens: 1.24M
Output: 25.7K

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer6948.0

Workflow

Local analysisBUSCOMAFFTBiopythonSciPyPhyKIT

Total tokens: 789.3K
Output: 5.3K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer6948.0

Workflow

Local analysisBUSCOMAFFTSciPy

Total tokens: 822.9K
Output: 6.5K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer6948

Workflow

Local analysisBUSCOMAFFTSciPyMann-Whitney UPhyKIT

Total tokens: 1.42M
Output: 19.7K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

fail

Why it missed

Data scope changedThis run restricted both groups to the 241 orthogroups shared by the two archives instead of comparing all 241 animal and 255 fungal alignments. That changed the tested distributions and produced U=6350.

Answer6350

Workflow

Local analysisBUSCOMAFFTSciPyMann-Whitney U

Total tokens: 2.02M
Output: 27.7K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

fail

Why it missed

Data scope and character handlingThis run restricted both groups to 241 shared orthogroups and also counted gaps as a character state. Those choices changed the distributions and produced U=5604.

Answer5604

Workflow

Local analysisBUSCOMAFFTSciPyMann-Whitney U

Total tokens: 1.03M
Output: 16.6K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

fail

Why it missed

Answer6350

Workflow

Local analysisMAFFTSciPyMann-Whitney Umean summarymedian summary

Total tokens: 617.1K
Output: 17.2K

Run trace

bix-12-q4 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

Answer6948.0

Total tokens: 2.96M
Input: 2.94M
Cached input: 2.83M
Output: 14.3K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

Answer6948

Total tokens: 4.54M
Input: 4.53M
Cached input: 4.41M
Output: 14.3K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

Answer6948

Total tokens: 2.20M
Input: 2.19M
Cached input: 2.11M
Output: 13.6K

Run trace Final answer Galaxy history

Open-ended code averaged 0.61M tokens; previous website Galaxy averaged 7.00M; the new Galaxy runs averaged 3.23M, or 5.32× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs