bix-16-q4 | Current BixBench results

New token-optimization experiment

Three additional GPT-5.6 Sol Galaxy replicates are available separately. Previous results remain unchanged.

View the October 2026 batch and its run records

Question

Using a Spearman Rank correlation test and Benjamini-Hochberg multi-test correction, what percentage of genes show a statistically significant correlation between expression and essentiality, in either direction?

Accepted answer: (20,25)
Domain: crispr dependency

Comparison

Pass counts and token totals across the same three replicates.

Agent	Galaxy-API code	Open-ended code
Codex GPT-5.5	3/3 replicate 1 replicate 2 replicate 3 1.89M total	3/3 replicate 1 replicate 2 replicate 3 1.35M total
Codex GPT-5.6 Sol	3/3 replicate 1 replicate 2 replicate 3 2.95M total	3/3 replicate 1 replicate 2 replicate 3 852.2K total
Codex GPT-5.6 Luna	3/3 replicate 1 replicate 2 replicate 3 6.07M total	3/3 replicate 1 replicate 2 replicate 3 2.94M total
DeepSeek V4 Pro via Codex	3/3 replicate 1 replicate 2 replicate 3 5.78M total	2/3 replicate 1 replicate 2 replicate 3 2.15M total
DeepSeek V4 Pro via Claude Codesuperseded	3/3 replicate 1 replicate 2 replicate 3 3.58M total	2/3 replicate 1 replicate 2 replicate 3 2.42M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 495.3K
Output: 4.3K

Galaxy history Run trace

Codex GPT-5.5

replicate 2

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 799.7K
Output: 4.7K

Galaxy history Run trace

Codex GPT-5.5

replicate 3

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 595.4K
Output: 5.4K

Galaxy history Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 1.57M
Output: 5.3K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 734.9K
Output: 4.6K

Galaxy history Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 640.5K
Output: 4.2K

Galaxy history Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 3.07M
Output: 19.8K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 1.46M
Output: 9.7K

Galaxy history Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 1.53M
Output: 10.7K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer21.803575469563249%

Workflow

Galaxy via BioBlendSpearman correlation

Total tokens: 1.30M
Output: 13.4K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 2

pass

Answer21.803575469563249%

Workflow

Galaxy historySpearman correlation

Total tokens: 2.30M
Output: 12.2K

Galaxy history Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer21.803575469563249%

Workflow

Galaxy via BioBlendSpearman correlation

Total tokens: 2.19M
Output: 15.2K

Galaxy history Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

passUDT-only

Answer21.806765%

Workflow

Galaxy UDTSpearman correlationSciPyscikit-learn

Total tokens: 1.53M
Output: 29.7K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 1.03M
Output: 8.4K

Galaxy history Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer21.803575469563249%

Workflow

Spearman correlation

Total tokens: 1.02M
Output: 11.3K

Galaxy history Run trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates

Codex GPT-5.5

replicate 1

pass

Answer21.803575469563249%

Workflow

Local analysisSciPystatsmodelsSpearman correlation

Total tokens: 465.9K
Output: 8.7K

Run trace

Codex GPT-5.5

replicate 2

pass

Answer21.8035754696%

Workflow

Local analysisSciPystatsmodelsSpearman correlation

Total tokens: 465.5K
Output: 6.2K

Run trace

Codex GPT-5.5

replicate 3

pass

Answer21.8035754695632%

Workflow

Local analysisSciPystatsmodelsSpearman correlation

Total tokens: 414.9K
Output: 11.0K

Run trace

Codex GPT-5.6 Sol

3 primary replicates

Codex GPT-5.6 Sol

replicate 1

pass

Answer21.669278996865206%

Workflow

Local analysis

Total tokens: 339.3K
Output: 7.1K

Run trace

Codex GPT-5.6 Sol

replicate 2

pass

Answer21.69%

Workflow

Local analysis

Total tokens: 237.0K
Output: 6.8K

Run trace

Codex GPT-5.6 Sol

replicate 3

pass

Answer21.80357546956325%

Workflow

Local analysis

Total tokens: 276.0K
Output: 6.9K

Run trace

Codex GPT-5.6 Luna

3 primary replicates

Codex GPT-5.6 Luna

replicate 1

pass

Answer21.80357546956325%

Workflow

Local analysis

Total tokens: 697.5K
Output: 16.6K

Run trace

Codex GPT-5.6 Luna

replicate 2

pass

Answer21.80357546956325%

Workflow

Local analysis

Total tokens: 1.11M
Output: 21.2K

Run trace

Codex GPT-5.6 Luna

replicate 3

pass

Answer21.80357546956325%

Workflow

Local analysis

Total tokens: 1.13M
Output: 13.9K

Run trace

DeepSeek V4 Pro via Codex

3 primary replicates

DeepSeek V4 Pro via Codex

replicate 1

pass

Answer21.80357546956325%

Workflow

Local analysisSciPystatsmodelsscikit-learnmean summarySpearman correlation

Total tokens: 829.3K
Output: 19.7K

Run trace

DeepSeek V4 Pro via Codex

replicate 2

fail

Evaluation status

Manual review pendingThe evaluator marked this answer incorrect. The answer, workflow summary, usage, and available run records are linked, but a manual root-cause audit has not yet been added.

Answer6.885041864675266%

Workflow

Local analysisSciPystatsmodelsmedian summarySpearman correlation

Total tokens: 650.6K
Output: 23.3K

Run trace

DeepSeek V4 Pro via Codex

replicate 3

pass

Answer21.80357546956325%

Workflow

Local analysisSciPystatsmodelsscikit-learnSpearman correlationmedian summary

Total tokens: 672.9K
Output: 17.6K

Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates

DeepSeek V4 Pro via Claude Code

replicate 1

fail

Why it missed

Statistical implementation errorThis run tested the same 17,676 genes but its p-value or Benjamini-Hochberg implementation retained only 103 genes, or 0.58%. The verified SciPy and statsmodels calculation retains 3,854 genes, or 21.80%.

Answer0.582711%

Workflow

Local analysisSciPystatsmodels

Total tokens: 707.6K
Output: 12.5K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 2

pass

Answer21.8036%

Workflow

Local analysisSciPystatsmodelsmean summarySpearman correlation

Total tokens: 1.10M
Output: 19.6K

Run trace

DeepSeek V4 Pro via Claude Code

replicate 3

pass

Answer21.8036%

Workflow

Local analysisSciPystatsmodels

Total tokens: 614.4K
Output: 8.3K

Run trace

bix-16-q4 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Galaxy-API code · October 2026

replicate 1

Correct

Answer21.803575469563249%

Total tokens: 617.3K
Input: 612.7K
Cached input: 574.2K
Output: 4.7K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 2

Correct

Answer21.803575469563249%

Total tokens: 347.4K
Input: 344.2K
Cached input: 323.3K
Output: 3.1K

Run trace Final answer Galaxy history

Galaxy-API code · October 2026

replicate 3

Correct

Answer21.803575469563249%

Total tokens: 561.8K
Input: 557.4K
Cached input: 526.8K
Output: 4.5K

Run trace Final answer Galaxy history

Open-ended code averaged 0.28M tokens; previous website Galaxy averaged 0.98M; the new Galaxy runs averaged 0.51M, or 1.79× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs