Back to results

CompBioBench item

gene-info-grab-q1

Question

Consider the gene Hand1. Answer each question with the exact format specified. 1. What is its Ensembl gene ID? (format: ENSMUSGXXXXXXXXXXX, no version suffix) 2. How many amino acids does the canonical protein isoform encode? (integer) 3. Does this gene have more than 5 or fewer than 5 introns? (answer: "more" or "fewer") 4. What is the Ensembl gene ID of the nearest protein-coding gene upstream of Hand1? (format: ENSMUSGXXXXXXXXXXX, no version suffix) Respond with semi-colon separate values, no spaces, e.g. "ENSMUSGXXXXXXXXXXX;23;more;ENSMUSGXXXXXXXXXXX"

Predicted answer
ENSMUSG00000037335;216;fewer;ENSMUSG00000037331Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Genomics
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Hand1 Ensembl Lookup
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Hand1 Ensembl diagnosticHand1 Ensembl diagnostic v2UDT minimal echoUDT minimal echo v2Upload1
Total tokens
2.0M
Output
15.2K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Hand1 Ensembl LookupHand1 Ensembl Lookup v2UDT Render ProbeUDT Doc Cat ProbeFasta Compute Length2 more tools
Total tokens
4.3M
Output
24.2K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
hand1_probeHand1 Ensembl annotation auditHand1 final answer acceptance check
Total tokens
1.4M
Output
10.1K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Derive Hand1 Ensembl answer with strand-aware neighbor diagnostics
Total tokens
804.8K
Output
4.9K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT-only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Hand1 Ensembl REST canonical and neighbor analysis
Total tokens
677.8K
Output
4.6K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Gtf Filter By Attribute Values ListAgatEnsembl Gtf2Gene ListFilter rowsTp Sort Header Tool7 more tools
Total tokens
10.3M
Output
35.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Filter Tabular
Total tokens
9.2M
Output
34.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
AnnotatemyidsGtf Filter By Attribute Values ListGffreadGff Filter By AttributeFlanking Features 12 more tools
Total tokens
5.5M
Output
26.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Gff Filter By AttributeTp Sort Header ToolConverter Gff To Bed 0Count Gff FeaturesFasta Compute Length
Total tokens
7.1M
Output
22.2K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Upload1Show Beginning1GrepFlanking Features 1
Total tokens
5.6M
Output
20.8K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionStandard tools only
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Upload1Tp Grep Tool
Total tokens
5.2M
Output
18.9K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.5

Replicate 2

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.5

Replicate 3

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local codePython
Total tokens
119.8K
Output
5.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local codePython
Total tokens
7.2M
Output
27.2K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local codePython
Total tokens
1.9M
Output
12.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local code

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
AnswerENSMUSG00000037335;216;fewer;ENSMUSG00000037331
Workflow
Local codePython
Total tokens
37.9K
Output
340