Back to results

CompBioBench item

hic-differential-loop-q1

Question

Consider the MicroC data for H1 and HFF cells from the following paper: https://doi.org/10.1016/j.molcel.2020.03.003. In the sub-compartment containing the NANOG gene (chr12:7629950-7809597), there is a differential loop between the two samples. Report the position of the loop (on chr12) in the following format start;end, with start<end rounded to the nearest 20000 and no commas.

Inferred answer
7720000;7800000Uniquely inferred from submitted scores.
Domain
Epigenomics
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

RerunCorrectUDT-only
Answer7720000;7800000
Workflow
NANOG Loop DiagnosticNANOG Loop Final
Total tokens
1.4M
Output
15.3K
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
Answer7720000;7800000
Workflow
NANOG HiC differential diagnosticNANOG HiC differential diagnostic v2NANOG HiC differential diagnostic v3NANOG HiC differential diagnostic v4UDT smoke echo2 more tools
Total tokens
6.2M
Output
24.7K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
Answer7720000;7800000
Workflow
NANOG loop diagnosticNANOG loop diagnostic v2Filter Tabular
Total tokens
1.7M
Output
11.3K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answer7720000;7800000
Workflow
Filter TabularGrepSelect columnsSubtract Query1
Total tokens
4.1M
Output
10.5K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT-only
Answer7720000;7800000
Workflow
Micro-C chr12 region diagnosticMicro-C chr12 normalization diagnosticMicro-C chr12 raw contact diagnosticMicro-C chr12 isolated raw contact diagnosticRank chr12 Micro-C differential pixels3 more tools
Total tokens
3.6M
Output
15.2K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
Answer7720000;7800000
Workflow
NANOG published dot-call diagnosticFilter TabularNANOG differential Micro-C loop final
Total tokens
4.5M
Output
16.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
Answer7720000;7800000
Workflow
No Galaxy tool job recorded
Total tokens
11.5M
Output
28.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
Answer7720000;7800000
Workflow
Converter Gz To UncompressedFilter rowsAdd A Column1Comp1
Total tokens
8.7M
Output
30.6K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
Answer7720000;7800000
Workflow
Inspect NANOG regionInspect NANOG region v2Probe CatProbe Cat DocProbe Cat Seq8 more tools
Total tokens
13.7M
Output
53.8K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
Answer7720000;7800000
Workflow
Upload1Filter TabularTp Grep ToolSelect columnsComp11 more tools
Total tokens
16.4M
Output
38.4K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
Answer7720000;7800000
Workflow
Filter TabularMicro-C region candidatesMicro-C UDT render probeMicro-C UDT command probe
Total tokens
12.2M
Output
40.9K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
Answer7720000;7800000
Workflow
Filter TabularGrepSelect columnsComp1Add A Column1
Total tokens
6.3M
Output
24.4K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answer7720000;7800000
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
Answer7720000;7800000
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Incorrect
Answer7680000;7920000
Workflow
Local codePythonSciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answer7720000;7800000
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Correct
Answer7720000;7800000
Workflow
Local codePythonSciPy
Codex GPT-5.6 Sol

Replicate 3

Correct
Answer7720000;7800000
Workflow
Local codePythonSciPy

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
Answer7720000;7800000
Workflow
Local codePythonSciPy
Total tokens
5.1M
Output
25.3K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answer7720000;7800000
Workflow
Local codePython
Total tokens
4.8M
Output
24.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunCorrect
Answer7720000;7800000
Workflow
Open-ended code
Total tokens
5.9M
Output
44.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunCorrect
Answer7720000;7800000
Workflow
Local codePython
Total tokens
5.9M
Output
23.6K
Codex GPT-5.6 Luna

Replicate 2

Correct
Answer7720000;7800000
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Correct
Answer7720000;7800000
Workflow
Local codePythonSciPy

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answer7720000;7800000
Workflow
Local codePython
Total tokens
230.4K
Output
1.9K