BixBench current item

bix-51-q2

Question

What is the Akaike Information Criterion (AIC) for the logistic regression model that uses BMI as the sole predictor of treatment response (efficacy PR)?

Accepted answer
(111.80,111.82)
Domain
model fitting

Comparison

Pass counts and token totals across the same three replicates.

AgentGalaxy-API codeOpen-ended code
Codex GPT-5.53/3
replicate 1 replicate 2 replicate 3
3.27M total		3/3
replicate 1 replicate 2 replicate 3
513.3K total
Codex GPT-5.6 Sol3/3
replicate 1 replicate 2 replicate 3
3.81M total		3/3
replicate 1 replicate 2 replicate 3
565.1K total
Codex GPT-5.6 Luna3/3
replicate 1 replicate 2 replicate 3
13.23M total		3/3
replicate 1 replicate 2 replicate 3
874.0K total
DeepSeek V4 Pro via Codex3/3
replicate 1 replicate 2 replicate 3
8.02M total		3/3
replicate 1 replicate 2 replicate 3
958.1K total
DeepSeek V4 Pro via Claude Codesuperseded3/3
replicate 1 replicate 2 replicate 3
9.50M total		3/3
replicate 1 replicate 2 replicate 3
1.62M total

Execution condition

Galaxy-API code with skills

Codex GPT-5.5

3 primary replicates
Codex GPT-5.5

replicate 1

passUDT + standard tools
Answer111.81245992528
Workflow
Excel to tabularGalaxy UDTstatsmodelsAIC model selectiongeneralized linear model
Total tokens
1.56M
Output
9.2K
Galaxy historyRun trace
Codex GPT-5.5

replicate 2

passUDT + standard tools
Answer111.8124599252796
Workflow
Galaxy UDTExcel to tabularAIC model selectionstatsmodelsgeneralized linear model
Total tokens
1.24M
Output
15.6K
Galaxy historyRun trace
Codex GPT-5.5

replicate 3

passUDT + standard tools
Answer111.8124599252796
Workflow
Excel to tabularGalaxy UDTAIC model selectionmore stepsgeneralized linear modellogistic regression
Total tokens
464.0K
Output
7.0K
Galaxy historyRun trace

Codex GPT-5.6 Sol

3 primary replicates
Codex GPT-5.6 Sol

replicate 1

passUDT + standard tools
Answer111.8124599252796
Workflow
Excel to tabularGalaxy UDTAIC model selectionstatsmodelsgeneralized linear modellogistic regression
Total tokens
850.9K
Output
7.1K
Galaxy historyRun trace
Codex GPT-5.6 Sol

replicate 2

passUDT + standard tools
Answer111.81245992528
Workflow
Excel to tabularGalaxy UDTAIC model selectionstatsmodelsgeneralized linear modellogistic regression
Total tokens
1.94M
Output
11.4K
Galaxy historyRun trace
Codex GPT-5.6 Sol

replicate 3

passUDT + standard tools
Answer111.8124599252796
Workflow
Excel to tabularGalaxy UDTAIC model selectionstatsmodelsgeneralized linear modellogistic regression
Total tokens
1.02M
Output
7.3K
Galaxy historyRun trace

Codex GPT-5.6 Luna

3 primary replicates
Codex GPT-5.6 Luna

replicate 1

passUDT + standard tools
Answer1.118124599252796e+02
Workflow
Excel to tabularGalaxy UDTAIC model selectiongeneralized linear modellogistic regression
Total tokens
3.71M
Output
14.7K
Galaxy historyRun trace
Codex GPT-5.6 Luna

replicate 2

passUDT-only
Answer111.8124599252796
Workflow
Galaxy UDTAIC model selectionstatsmodelslogistic regression
Total tokens
6.02M
Output
21.3K
Galaxy historyRun trace
Codex GPT-5.6 Luna

replicate 3

passUDT-only
Answer111.8124599252796
Workflow
Galaxy UDTAIC model selectionstatsmodelsSciPygeneralized linear model
Total tokens
3.49M
Output
22.4K
Galaxy historyRun trace

DeepSeek V4 Pro via Codex

3 primary replicates
DeepSeek V4 Pro via Codex

replicate 1

pass
Answer111.81245992528
Workflow
Galaxy via BioBlendAIC model selectionscikit-learnlogistic regressiongeneralized linear model
Total tokens
2.42M
Output
11.6K
Galaxy historyRun trace
DeepSeek V4 Pro via Codex

replicate 2

passUDT-only
Answer111.812459925280
Workflow
Galaxy via BioBlendlogistic regressionscikit-learn
Total tokens
2.53M
Output
21.5K
Galaxy historyRun trace
DeepSeek V4 Pro via Codex

replicate 3

pass
Answer111.81245992528
Workflow
Galaxy via BioBlendAIC model selectionscikit-learnlogistic regressionstatsmodelsgeneralized linear model
Total tokens
3.07M
Output
15.3K
Galaxy historyRun trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates
DeepSeek V4 Pro via Claude Code

replicate 1

passUDT + standard tools
Answer111.8125
Workflow
Sklearn Generalized LinearGalaxy UDTAIC model selectionlogistic regressionscikit-learngeneralized linear model
Total tokens
4.64M
Output
28.7K
Galaxy historyRun trace
DeepSeek V4 Pro via Claude Code

replicate 2

passUDT + standard tools
Answer111.81245992528
Workflow
Excel to tabularAIC model selectiongeneralized linear model
Total tokens
2.45M
Output
13.6K
Galaxy historyRun trace
DeepSeek V4 Pro via Claude Code

replicate 3

passUDT + standard tools
Answer111.8124599252796
Workflow
Excel to tabularGalaxy UDTAIC model selectionmore stepsscikit-learnlogistic regression
Total tokens
2.40M
Output
19.5K
Galaxy historyRun trace

Execution condition

Open-ended code with skills

Codex GPT-5.5

3 primary replicates
Codex GPT-5.5

replicate 1

pass
Answer111.81245992527957
Workflow
Local analysisstatsmodelsSciPyAIC model selectiongeneralized linear model
Total tokens
211.7K
Output
4.6K
Run trace
Codex GPT-5.5

replicate 2

pass
Answer111.81245992528
Workflow
Local analysisstatsmodelsSciPyAIC model selection
Total tokens
139.7K
Output
3.6K
Run trace
Codex GPT-5.5

replicate 3

pass
Answer111.81245992528
Workflow
Local analysisstatsmodelsSciPyAIC model selectiongeneralized linear model
Total tokens
162.0K
Output
3.8K
Run trace

Codex GPT-5.6 Sol

3 primary replicates
Codex GPT-5.6 Sol

replicate 1

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
195.9K
Output
3.9K
Run trace
Codex GPT-5.6 Sol

replicate 2

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
176.8K
Output
3.1K
Run trace
Codex GPT-5.6 Sol

replicate 3

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
192.4K
Output
3.3K
Run trace

Codex GPT-5.6 Luna

3 primary replicates
Codex GPT-5.6 Luna

replicate 1

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
278.2K
Output
6.2K
Run trace
Codex GPT-5.6 Luna

replicate 2

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
339.4K
Output
6.8K
Run trace
Codex GPT-5.6 Luna

replicate 3

pass
Answer111.8124599252796
Workflow
Local analysis
Total tokens
256.4K
Output
5.4K
Run trace

DeepSeek V4 Pro via Codex

3 primary replicates
DeepSeek V4 Pro via Codex

replicate 1

pass
Answer111.8124599252796
Workflow
Local analysisstatsmodelsscikit-learnSciPylogistic regressionAIC model selection
Total tokens
386.5K
Output
6.6K
Run trace
DeepSeek V4 Pro via Codex

replicate 2

pass
Answer111.81245992527958
Workflow
Local analysisSciPystatsmodelsscikit-learnAIC model selectiongeneralized linear model
Total tokens
382.2K
Output
9.1K
Run trace
DeepSeek V4 Pro via Codex

replicate 3

pass
Answer111.8124599252796
Workflow
Local analysisstatsmodelsscikit-learnSciPylogistic regressionAIC model selection
Total tokens
189.4K
Output
3.9K
Run trace

DeepSeek V4 Pro via Claude CodeSuperseded

3 primary replicates
DeepSeek V4 Pro via Claude Code

replicate 1

pass
Answer111.8125
Workflow
Local analysisstatsmodelsSciPygeneralized linear modelAIC model selection
Total tokens
762.9K
Output
7.2K
Run trace
DeepSeek V4 Pro via Claude Code

replicate 2

pass
Answer111.8124599252796
Workflow
Local analysisSciPystatsmodelslogistic regressionAIC model selection
Total tokens
337.2K
Output
3.3K
Run trace
DeepSeek V4 Pro via Claude Code

replicate 3

pass
Answer111.8124599252796
Workflow
Local analysisstatsmodelslogistic regressionAIC model selectionSciPy
Total tokens
518.9K
Output
5.2K
Run trace

bix-51-q2 · October 2026 · three additional replicates

GPT-5.6 Sol · token optimization

Galaxy-API code, high reasoning, Fast service. Previous runs are retained above.

Open-ended code averaged 0.19M tokens; previous website Galaxy averaged 1.27M; the new Galaxy runs averaged 0.71M, or 3.75× Open-ended code. Each mean covers three replicates of this same task with GPT-5.6 Sol.

Cached input is already included in input tokens. Full nested tool/job evidence is in each HF run's galaxy_execution_records.tar.gz, with an accompanying member index.

Optimization overview and all 150 runs