bix-22-q4 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

What is the Pearson correlation coefficient between gene length and mean expression (across samples) for expressed protein-coding genes (total counts ≥ 10) in CD14 immune cells?

Accepted answer: (0.015,0.025)
Domain: general

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 2.33M total	3/3 replicate 1 replicate 2 replicate 3 391.2K total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 6.15M total	3/3 replicate 1 replicate 2 replicate 3 545.1K total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 13.40M total	3/3 replicate 1 replicate 2 replicate 3 701.7K total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 20.23M total	3/3 replicate 1 replicate 2 replicate 3 1.75M total
DeepSeek V4 Pro via Claude Codesuperseded	3/3 replicate 1 replicate 2 replicate 3 21.38M total	3/3 replicate 1 replicate 2 replicate 3 1.46M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

passUDT-only

Answer0.015815830205872596

Workflow

Galaxy UDTPearson correlationmean summary

Total tokens: 714.9K
Output: 8.0K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

passUDT-only

Answer1.5815830205872596e-02

Workflow

Galaxy UDTPearson correlationSciPyDESeq2

Total tokens: 804.8K
Output: 11.3K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

passUDT-only

Answer0.015815830205872596

Workflow

Galaxy UDTPearson correlationSciPymean summary

Total tokens: 815.3K
Output: 6.9K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer0.015815830205872593

Workflow

Convert Characters1Filter TabularSelect columnsmore stepsMatch rowsPearson correlation

Total tokens: 4.36M
Output: 13.5K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

passUDT-only

Answer0.015815830205872593

Workflow

Galaxy UDTPearson correlationSciPy

Total tokens: 983.7K
Output: 7.0K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

passUDT-only

Answer0.015815830205872596

Workflow

Galaxy UDTPearson correlationSciPymean summary

Total tokens: 806.0K
Output: 11.0K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

passUDT-only

Answer0.015815830205872596

Workflow

Galaxy UDTPearson correlationSciPy

Total tokens: 2.89M
Output: 20.9K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

passUDT + standard tools

Answer0.015815830205872593

Workflow

Galaxy UDTPearson correlationRegexcolumn1mean summary

Total tokens: 8.05M
Output: 28.5K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

passUDT-only

Answer0.015815830205872593

Workflow

Galaxy UDTPearson correlation

Total tokens: 2.45M
Output: 17.5K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer0.015815830205872593

Workflow

Select columnsAdd derived columnFilter rowsSort rowsTp Easyjoin ToolPearson correlation

Total tokens: 6.35M
Output: 31.4K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer0.021413157986867361

Workflow

Filter rowsSelect columnsConvert Characters1more stepsRemove headerPearson correlation

Total tokens: 9.37M
Output: 32.7K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

passUDT + standard tools

Answer0.0158158302

Workflow

Csv To TabularTable ComputeGalaxy UDTPearson correlation

Total tokens: 4.51M
Output: 32.2K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

passUDT + standard tools

Answer0.0158158302

Workflow

Datamash TransposeJoin datasetsFilter rowsmore stepsGalaxy UDTPearson correlation

Total tokens: 5.78M
Output: 44.1K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer0.015802

Workflow

Filter TabularTable ComputeJoin datasetsPearson correlation

Total tokens: 7.85M
Output: 59.4K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer0.015816

Workflow

Select columnsTable ComputeMerge columnsFilter TabularJoin datasetsPearson correlation

Total tokens: 7.75M
Output: 67.0K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer0.0158158302058721

Workflow

Local analysis

Total tokens: 150.4K
Output: 3.1K

Run trace

Codex GPT-5.5

replicate 2

pass

Answer0.0158158302058725

Workflow

Local analysisPearson correlation

Total tokens: 109.9K
Output: 2.8K

Run trace

Codex GPT-5.5

replicate 3

pass

Answer0.015815830205872121

Workflow

Local analysis

Total tokens: 130.8K
Output: 3.9K

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer0.0158158302058726

Workflow

Local analysis

Total tokens: 157.2K
Output: 3.5K

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer0.015815830205872593

Workflow

Local analysis

Total tokens: 190.4K
Output: 4.4K

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer0.0158158302058726

Workflow

Local analysis

Total tokens: 197.5K
Output: 5.1K

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer0.0158158302058726

Workflow

Local analysis

Total tokens: 240.6K
Output: 5.3K

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer0.015815830205872593

Workflow

Local analysis

Total tokens: 157.8K
Output: 5.1K

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer0.015815830205872594

Workflow

Local analysis

Total tokens: 303.3K
Output: 8.1K

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer0.0158158302058726

Workflow

Local analysisSciPyPearson correlation

Total tokens: 676.8K
Output: 11.9K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer0.015815830205872593

Workflow

Local analysisSpearman correlationSciPy

Total tokens: 402.6K
Output: 8.6K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer0.01581583020587259

Workflow

Local analysisSciPyscikit-learnmean summarySpearman correlation

Total tokens: 666.1K
Output: 11.5K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

pass

Answer0.01581583020587259

Workflow

Local analysisSciPymean summaryPearson correlation

Total tokens: 441.0K
Output: 6.3K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer0.01581583020587259

Workflow

Local analysisSciPymean summarymedian summarySpearman correlation

Total tokens: 670.7K
Output: 6.1K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer0.01581583020587259

Workflow

Local analysisSciPyPearson correlationmean summary

Total tokens: 352.2K
Output: 4.4K

Run trace

bix-22-q4 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

Answer0.015815830205872596

Total tokens: 895.2K
Input: 883.5K
Cached input: 841.9K
Output: 11.7K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

Answer0.015815830205872593

Total tokens: 1.79M
Input: 1.78M
Cached input: 1.71M
Output: 9.4K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

Answer0.015815830205872596

Total tokens: 825.9K
Input: 818.6K
Cached input: 779.4K
Output: 7.3K

Run trace Final answer Galaxy history

Open-ended code averaged 0.18M tokens; previous website Galaxy averaged 2.05M; the new Galaxy runs averaged 1.17M, or 6.45× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs