Back to results

CompBioBench item

enformer-basic-q1

Question

What is the exact count of the number of parameters in the Enformer model (https://doi.org/10.1038/s41592-021-01252-x). Specifically, report the number of trainable parameters including both the human and mouse heads. Format the number without commas.

Inferred answer
251221292Uniquely inferred from submitted scores.
Domain
Machine Learning
Question style
Tooling

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
Answer251221292
Workflow
Enformer Trainable Parameter Count
Codex GPT-5.5

Replicate 2

CorrectUDT-only
Answer251221292
Workflow
Inspect Enformer SourcesCopy Dummy ProbeCopy Dummy Probe 2Copy Dummy Probe 3Copy Dummy Probe 41 more tools
Total tokens
1.9M
Output
20.4K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
Answer251221292
Workflow
Enformer trainable parameter countEnformer trainable parameter count v2Enformer trainable parameter count v3Enformer Parameter Count V4UDT Echo Probe V11 more tools
Total tokens
2.0M
Output
18.1K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT-only
Answer251221292
Workflow
Enformer symbolic trainable parameter countEnformer TensorFlow runtime probeEnformer Sonnet runtime probeEnformer Sonnet compatible runtime probeEnformer Sonnet dependency-complete runtime probe5 more tools
Total tokens
3.1M
Output
13.8K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT-only
Answer251221292
Workflow
Enformer exact trainable parameter count
Total tokens
2.9M
Output
14.1K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT-only
Answer251221292
Workflow
Enformer exact trainable parameter audit
Total tokens
629.2K
Output
5.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT-only
Answer251221292
Workflow
Enformer Param CountEnformer Trainable Count
Total tokens
1.3M
Output
13.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT-only
Answer251221292
Workflow
TF Sonnet ProbeTF25 Sonnet ProbeTF15 Patched Sonnet ProbeTF15 Patched Sonnet Probe v2TF15 Patched Sonnet Probe v32 more tools
Total tokens
4.8M
Output
31.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT-only
Answer251221292
Workflow
Enformer Parameter CountUDT Probe SimpleUDT Probe EnvUDT Probe Env2
Total tokens
5.4M
Output
37.0K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT-only
Answer251221292
Workflow
Enformer source diagnosticEnformer trainable parameter countEnformer runtime parameter checkEnformer runtime parameter shape inventoryEnformer runtime parameter vector name inventory
Total tokens
5.7M
Output
38.1K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT-only
Answer251221292
Workflow
Enformer exact trainable parameter countEnformer runtime parameter validation
Total tokens
2.5M
Output
23.3K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT-only
Answer251221292
Workflow
Enformer source diagnosticEnformer trainable parameter count
Total tokens
2.5M
Output
27.0K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answer251221292
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
Answer251221292
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Correct
Answer251221292
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answer251221292
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Correct
Answer251221292
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 3

Correct
Answer251221292
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
Answer251221292
Workflow
Local codePython
Total tokens
4.7M
Output
27.4K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answer251221292
Workflow
Local codePython
Total tokens
4.7M
Output
26.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
Answer251221292
Workflow
Local codePython
Total tokens
8.1M
Output
52.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Incorrect
Answer251225900
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 2

Correct
Answer251221292
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Correct
Answer251221292
Workflow
Local codePython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answer251221292
Workflow
Local codePython
Total tokens
488.8K
Output
4.8K