Back to results

CompBioBench item

overexpress-tf-q1

Question

We perturbed a primary cell culture with a cocktail of transcription factors (TFs). ATAC-seq is performed on the initial sample and 48 hours after perturbation. overexpress.tf.q1.ref.bed.gz and overexpress.tf.q1.perturb.bed.gz contain the filtered aligned fragment files (hg38) corresponding to the initial and perturbed samples, respectively. Exactly one of the TFs in the following list is part of the cocktail: FOXA1, ASCL1, TFAP2A, RUNX1, KLF4, PAX7, TCF7, REST, SPI1, GATA3. Identify which one.

Inferred answer
KLF4Uniquely inferred from submitted scores.
Domain
Epigenomics
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

IncorrectUDT-only
AnswerFOXA1
Workflow
TF fragment diagnosticTF motif enrichment smallTF motif enrichment GC blacklist
Total tokens
1.5M
Output
19.7K
Codex GPT-5.5

Replicate 2

IncorrectUDT + standard tools
AnswerSPI1
Workflow
TF motif accessibility diagnosticTF motif accessibility diagnostic v2TF motif accessibility diagnostic v3UDT static command probeUDT expression command probe3 more tools
Total tokens
7.8M
Output
36.1K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerKLF4
Workflow
TF perturbation differential fragment windowsTF perturbation differential fragment windows v2UDT render probeUDT no interpolation probeMacs2 Callpeak8 more tools
Total tokens
6.7M
Output
25.8K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
AnswerKLF4
Workflow
Macs2 CallpeakTp Sort Header ToolSort rowsTp Head ToolBedtools Slopbed3 more tools
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
AnswerKLF4
Workflow
Macs2 CallpeakWc GnuATAC differential windows q1ATAC differential windows q1 v2ATAC differential windows q1 v35 more tools
Total tokens
15.1M
Output
31.5K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
AnswerKLF4
Workflow
ATAC fragment bin diagnosticATAC fragment bin diagnostic v2Macs2 CallpeakBedtools IntersectbedTp Sort Header Tool5 more tools
Total tokens
31.2M
Output
37.8K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
AnswerKLF4
Workflow
No Galaxy tool job recorded
Total tokens
28.2M
Output
59.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
AnswerKLF4
Workflow
Bedtools GenomecoveragebedMacs2 BdgpeakcallBedtools SubtractbedBedtools GetfastabedMeme Chip5 more tools
Total tokens
25.6M
Output
69.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

IncorrectUDT + standard tools
AnswerPAX7
Workflow
Meme FimoDatamash OpsBedtools IntersectbedBedtools GroupbybedConverter Gff To Bed 013 more tools
Total tokens
35.1M
Output
93.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
AnswerKLF4
Workflow
Wc GnuMacs2 CallpeakBedtools IntersectbedBedtools GetfastabedMeme Chip
Total tokens
8.3M
Output
19.6K
Codex GPT-5.6 Luna

Replicate 2

IncorrectUDT + standard tools
AnswerPAX7
Workflow
Macs2 CallpeakBedtools GetfastabedJASPAR to MEMEUDT Render ProbeMeme Fimo3 more tools
Total tokens
18.2M
Output
55.9K
Codex GPT-5.6 Luna

Replicate 3

IncorrectUDT + standard tools
AnswerPAX7
Workflow
Macs2 CallpeakBedtools IntersectbedBedtools GetfastabedJASPAR PFM to MEME XMLMeme Fimo1 more tools
Total tokens
13.3M
Output
53.3K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerKLF4
Workflow
Local codeBedtoolsPythonSciPypysam1 more tools
Codex GPT-5.5

Replicate 2

Incorrect
AnswerPAX7
Workflow
Local codeBedtoolsRPythonSamtools
Codex GPT-5.5

Replicate 3

Correct
AnswerKLF4
Workflow
Local codeBedtoolsSamtoolsPythonSciPy1 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerKLF4
Workflow
Local codeBedtoolsSamtoolsPythonSciPy
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerKLF4
Workflow
Local codeBedtoolsRSamtoolsPython2 more tools
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerKLF4
Workflow
Local codeBedtoolsPythonSamtools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerKLF4
Workflow
Local codeBedtoolsSamtoolsPythonR2 more tools
Total tokens
21.5M
Output
86.2K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerKLF4
Workflow
Local codeBedtoolsSamtoolsRPython2 more tools
Total tokens
9.7M
Output
65.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerKLF4
Workflow
Local codePythonBedtoolsSamtoolsBowtie23 more tools
Total tokens
10.8M
Output
60.0K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerKLF4
Workflow
Local codeSamtoolsBedtoolsPythonSciPy1 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerKLF4
Workflow
Local codePythonSciPypysamSamtools1 more tools
Codex GPT-5.6 Luna

Replicate 3

Incorrect
AnswerTCF7
Workflow
Local codeBedtoolsSamtoolsPythonpysam1 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerKLF4
Workflow
Local codePythonSciPypysam
Total tokens
750.2K
Output
10.7K