Back to results

CompBioBench item

pathogenic-variant-lookup-q1

Question

Using the human SHH MANE Select transcript on hg38, consider the coding sequence within exon 1. Within this interval, identify all OMIM allelic variants with phenotypes that map to single-nucleotide missense substitutions in SHH. Retrieve the AlphaMissense pathogenicity scores for those amino-acid substitutions. Return the single variant with the highest AlphaMissense score, in chr:pos (1-based) format.

Predicted answer
chr7:155811860Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Genomics
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
Answerchr7:155811860
Workflow
SHH OMIM AlphaMissense exon1 MANESHH OMIM AlphaMissense exon1 MANE diagnosticSHH OMIM HGC detail debugSHH OMIM ClinVar AlphaMissense final
Codex GPT-5.5

Replicate 2

Matches predictionUDT + standard tools
Answerchr7:155811860
Workflow
SHH OMIM diagnosticUDT render probeUDT render probe scalar formatUDT render probe no interpolationUDT render probe busybox
Total tokens
6.5M
Output
23.1K
Codex GPT-5.5

Replicate 3

Matches predictionUDT-only
Answerchr7:155811860
Workflow
Probe SHH MANE exon1 OMIM AV intervalProbe SHH MANE exon1 OMIM AV interval v2UDT text smokeUDT constant smokeUcsc Table Direct1
Total tokens
5.8M
Output
25.0K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionStandard tools only
Answerchr7:155811860
Workflow
Select columnsGrepCsv To TabularTp Sort Header ToolSort rows
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
Answerchr7:155811860
Workflow
UCSC SHH regional source diagnosticUCSC SHH compact source diagnosticUCSC OMIM SHH source rowsNCBI OMIM ClinVar diagnosticSHH exon1 OMIM AlphaMissense final analysis
Total tokens
1.5M
Output
12.1K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT-only
Answerchr7:155811860
Workflow
SHH exon1 OMIM AlphaMissense integration
Total tokens
2.5M
Output
12.9K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionUDT + standard tools
Answerchr7:155811860
Workflow
No Galaxy tool job recorded
Total tokens
14.4M
Output
44.2K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
Answerchr7:155811860
Workflow
Datasets Download GeneUcsc Table Direct1Filter Tabular
Total tokens
13.1M
Output
52.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionUDT-only
Answerchr7:155811860
Workflow
SHH OMIM AlphaMissenseSHH OMIM AlphaMissense CombinedSHH OMIM AlphaMissense Combined v2
Total tokens
6.6M
Output
43.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT + standard tools
Answerchr7:155811860
Workflow
SHH OMIM exon-1 AlphaMissense diagnosticSHH OMIM exon-1 AlphaMissense diagnostic v2SHH UDT render probeSHH UDT static probeSHH UDT static probe auto9 more tools
Total tokens
22.8M
Output
61.5K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT + standard tools
Answerchr7:155811860
Workflow
Data FetchSHH OMIM exon 1 diagnosticUDT render probeUDT command probeUDT command probe two5 more tools
Total tokens
25.9M
Output
67.3K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT-only
Answerchr7:155811860
Workflow
SHH OMIM MANE diagnosticSHH OMIM MANE AlphaMissense ranking
Total tokens
10.8M
Output
34.5K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Matches prediction
Answerchr7:155811860
Workflow
Local codeBCFtoolsPython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answerchr7:155811860
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answerchr7:155811860
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Total tokens
1.1M
Output
16.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answerchr7:155811860
Workflow
Local codeBCFtoolsPythonpysam
Total tokens
2.1M
Output
17.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Total tokens
998.9K
Output
15.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answerchr7:155811860
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answerchr7:155811860
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answerchr7:155811860
Workflow
Local codeBCFtoolsBedtoolsPython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answerchr7:155811860
Workflow
Local codePython
Total tokens
285.3K
Output
3.2K