Back to results

CompBioBench item

genomic-state-q1

Question

Consider chr11:124,738,681-124,738,772 (hg38). What is the likely purpose of this region? Choose only one. A. Active enhancer, Liver B. Active enhancer, Brain frontal lobe C. Active enhancer, ESC D. Active transcription, ubiquitous E. TSS poised or flanking, ubiquitous F. Polycomb repression, ubiquitous. Respond with a single letter.

Inferred answer
BUniquely inferred from submitted scores.
Domain
Epigenomics
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
AnswerB
Workflow
Roadmap interval chromatin state queryRoadmap interval chromatin state query legacyUpload1Ucsc Bigbedtobed
Total tokens
2.9M
Output
14.1K
Codex GPT-5.5

Replicate 2

CorrectStandard tools only
AnswerB
Workflow
Galaxy dataset inspection
Total tokens
3.1M
Output
14.5K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerB
Workflow
Roadmap interval probeRoadmap interval probe v2Bedtools Intersectbed
Total tokens
1.7M
Output
11.9K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT + standard tools
AnswerB
Workflow
Converter Archive To DirectoryGalaxy UDTparse_track_catalogParse track catalogSelect Roadmap tissue tracks7 more tools
Total tokens
12.0M
Output
28.6K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT-only
AnswerB
Workflow
EpiMap hg38 ChromHMM interval probe
Total tokens
1.2M
Output
6.5K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT-only
AnswerB
Workflow
Coordinate annotation diagnosticRoadmap chromatin state probeRoadmap chromatin state panel summaryReduce Roadmap panel summaryFinalize genomic state answer
Total tokens
2.2M
Output
11.9K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
AnswerB
Workflow
No Galaxy tool job recorded
Total tokens
24.9M
Output
75.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
AnswerB
Workflow
Bedtools Intersectbed
Total tokens
11.3M
Output
33.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectNo analysis tools recorded
AnswerB
Workflow
Ucsc Table Direct1
Total tokens
27.7M
Output
46.4K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
AnswerB
Workflow
Ucsc Table Direct1Converter Gz To UncompressedCreateintervalBedtools IntersectbedUcsc Bigbedtobed
Total tokens
13.1M
Output
24.8K
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
AnswerB
Workflow
Ucsc BigbedtobedBedtools Intersectbed
Total tokens
17.5M
Output
26.9K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
AnswerB
Workflow
Data FetchUpload1Xlsx2TsvFilter TabularTp Grep Tool1 more tools
Total tokens
12.1M
Output
27.5K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerB
Workflow
Local code
Codex GPT-5.5

Replicate 2

Correct
AnswerB
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Correct
AnswerB
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerB
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerB
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerB
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerB
Workflow
Local codePython
Total tokens
9.7M
Output
47.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Incorrect
AnswerE
Workflow
Local codePython
Total tokens
6.8M
Output
29.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerB
Workflow
Local codePythonBedtoolspysam
Total tokens
3.6M
Output
21.8K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerB
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerB
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerB
Workflow
Local codePythonBedtools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Incorrect
AnswerF
Workflow
Local codePython
Total tokens
111.2K
Output
1.2K