Back to results

CompBioBench item

bedtools-ops-q1

Question

In the given ENCODE narrowpeak bed file ENCFF333TAT.bed.gz, convert each interval into a 500bp interval centered at the peak summit. Merge all intervals, and then exclude any that overlaps with the provided blacklist file ENCFF356LFX.bed.gz. Return the total number of bases in the final file, no commas.

Inferred answer
111423829Uniquely inferred from submitted scores.
Domain
Epigenomics
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
Answer111423829
Workflow
Summit 500bp blacklist base countSummit 500bp diagnostics
Codex GPT-5.5

Replicate 2

CorrectUDT-only
Answer111423829
Workflow
NarrowPeak Summit DiagnosticNarrowPeak 500bp Blacklist TotalBedtools
Total tokens
386.5K
Output
6.3K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
Answer111423829
Workflow
Summit centered interval diagnosticSummit centered interval diagnostic v2Summit centered interval diagnostic v3Add A Column1Select columns4 more tools
Total tokens
2.0M
Output
12.4K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT + standard tools
Answer111423829
Workflow
Add A Column1Galaxy UDTcenter_peaksCenter peaksCenter narrowPeak summits, merge, exclude blacklist, and total bases1 more tools
Codex GPT-5.6 Sol

Replicate 2

CorrectStandard tools only
Answer111423829
Workflow
Add A Column1Select columnsBedtools SortbedSummary Statistics1Bedtools Mergebed2 more tools
Total tokens
3.3M
Output
13.7K
Codex GPT-5.6 Sol

Replicate 3

CorrectStandard tools only
Answer111423829
Workflow
Summary Statistics1Add A Column1Select columnsSort rowsBedtools Mergebed2 more tools
Total tokens
8.0M
Output
9.8K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT + standard tools
Answer111423829
Workflow
NarrowPeak center merge blacklist totalBedtools MergebedBedtools IntersectbedSum BED bases
Total tokens
1.5M
Output
19.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT + standard tools
Answer111423829
Workflow
No Galaxy tool job recorded
Total tokens
21.6M
Output
65.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
Answer111423829
Workflow
Add A Column1Select columnsBedtools SortbedBedtools MergebedBedtools Subtractbed3 more tools
Total tokens
6.1M
Output
29.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
Answer111423829
Workflow
Add A Column1Bedtools SortbedBedtools MergebedSelect columnsBedtools Subtractbed
Total tokens
11.0M
Output
31.4K
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
Answer111423829
Workflow
Tp Head ToolAdd A Column1Bedtools SortbedBedtools MergebedBedtools Subtractbed1 more tools
Total tokens
2.7M
Output
19.2K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
Answer111423829
Workflow
Add A Column1Bedtools SortbedBedtools MergebedBedtools SubtractbedSummary Statistics12 more tools
Total tokens
11.7M
Output
33.6K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answer111423829
Workflow
Local codeBedtoolsPython
Codex GPT-5.5

Replicate 2

Correct
Answer111423829
Workflow
Local codeBedtoolsPython
Codex GPT-5.5

Replicate 3

Correct
Answer111423829
Workflow
Local codeBedtools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answer111423829
Workflow
Local codeBedtools
Codex GPT-5.6 Sol

Replicate 2

Correct
Answer111423829
Workflow
Local codeBedtools
Codex GPT-5.6 Sol

Replicate 3

Correct
Answer111423829
Workflow
Local codeBedtools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
Answer111424282
Workflow
Local codeBedtoolsPython
Total tokens
247.2K
Output
5.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answer111423829
Workflow
Local codeBedtoolsPython
Total tokens
168.3K
Output
7.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
Answer111423829
Workflow
Local codeBedtoolsPython
Total tokens
662.7K
Output
13.4K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
Answer111423829
Workflow
Local codeBedtools
Codex GPT-5.6 Luna

Replicate 2

Correct
Answer111423829
Workflow
Local codeBedtools
Codex GPT-5.6 Luna

Replicate 3

Correct
Answer111423829
Workflow
Local codeBedtoolsPython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answer111423829
Workflow
Local codePython
Total tokens
71.2K
Output
1.2K