Back to results

CompBioBench item

annotate-variant-regulatory-overlap-q1

Question

Determine the regulatory element that overlap the given variant’s reference span for this specific variant and regulatory resource, without liftover and reporting 1-based inclusive coordinates. Variant: GRCh38, contig chr19, position 44907187, ref allele G, alt allele A. Regulatory source: ENCODE cCRE Registry, release v4. Return the class of the regulatory element overlapping the variant and the CRE accession (e.g. EH38EXXXXX) as a comma-separated list. Expected output format: "class,accession".

Inferred answer
pELS,EH38E1957012Uniquely inferred from submitted scores.
Domain
Epigenomics
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bedtools Intersectbed
Codex GPT-5.5

Replicate 2

IncorrectUDT-only
AnswerProximal enhancer,EH38E1957012
Workflow
cCRE Registry v4 overlap query
Total tokens
361.7K
Output
4.6K
Codex GPT-5.5

Replicate 3

IncorrectUDT-only
AnswerProximal enhancer,EH38E1957012
Workflow
Query ENCODE4 cCRE overlap for one variantQuery ENCODE4 cCRE overlap for one variant v2Query ENCODE4 cCRE overlap for one variant v3
Total tokens
1.7M
Output
11.5K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bedtools IntersectbedTp Cut Tool
Codex GPT-5.6 Sol

Replicate 2

IncorrectStandard tools only
AnswerProximal enhancer,EH38E1957012
Workflow
Ucsc Bigbedtobed
Total tokens
955.3K
Output
4.6K
Codex GPT-5.6 Sol

Replicate 3

IncorrectStandard tools only
AnswerProximal enhancer,EH38E1957012
Workflow
Ucsc Bigbedtobed
Total tokens
1.8M
Output
4.8K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

IncorrectStandard tools only
AnswerProximal enhancer,EH38E1957012
Workflow
Ucsc BigbedtobedBedtools Intersectbed
Total tokens
544.5K
Output
5.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
No Galaxy tool job recorded
Total tokens
4.8M
Output
29.1K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bedtools Intersectbed
Total tokens
615.4K
Output
7.7K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bedtools Intersectbed
Total tokens
1.4M
Output
7.7K
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bedtools Intersectbed
Total tokens
834.8K
Output
7.5K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
AnswerpELS,EH38E1957012
Workflow
Bcftools QueryBedtools Intersectbed
Total tokens
2.5M
Output
14.3K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerpELS,EH38E1957012
Workflow
Local code
Codex GPT-5.5

Replicate 2

Incorrect
AnswerProximal enhancer,EH38E1957012
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Incorrect
AnswerProximal enhancer,EH38E1957012
Workflow
Local code

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerpELS,EH38E1957012
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerpELS,EH38E1957012
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerpELS,EH38E1957012
Workflow
Local code

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerpELS,EH38E1957012
Workflow
Local codePython
Total tokens
2.8M
Output
10.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Incorrect
AnswerProximal enhancer,EH38E1957012
Workflow
Local codeBedtoolsPython
Total tokens
716.1K
Output
6.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Incorrect
AnswerProximal enhancer,EH38E1957012
Workflow
Local codeBedtoolsPythonpysam
Total tokens
461.9K
Output
5.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerpELS,EH38E1957012
Workflow
Local codeBedtools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerpELS,EH38E1957012
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerpELS,EH38E1957012
Workflow
Local code

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerpELS,EH38E1957012
Workflow
Local codePython
Total tokens
71.3K
Output
681