Back to results

CompBioBench item

splice-pred-q1

Question

Extract the human genomic DNA sequence from hg38 at chr12:120,196,699-120,201,111. This locus is on the minus strand. Run OpenSpliceAI (v0.0.5) to predict with the OSAIMANE-10000nt models, averaging predictions across all 5 checkpoints. Call donor and acceptor splice sites with score ≥ 0.9. Reconstruct exons on the gene strand by pairing each predicted acceptor with the next predicted donor in transcript order, and count the terminal exons using the interval boundaries as transcript boundaries. Report the total number of predicted exons.

Predicted answer
7Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Machine Learning
Question style
Tooling

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
Answer7
Workflow
OSAI hg38 diagnosticOSAI MANE exon countOSAI MANE exon count v2
Total tokens
1.1M
Output
12.0K
Codex GPT-5.5

Replicate 3

RerunMatches predictionUDT-only
Answer7
Workflow
OpenSpliceAI environment probeOpenSpliceAI exon count chr12 minus intervalOpenSpliceAI exon count chr12 minus interval v2OpenSpliceAI BioContainer probeOpenSpliceAI BioContainer overlay probe15 more tools
Total tokens
7.0M
Output
39.9K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1OpenSpliceAI 0.0.5 OSAIMANE exon count
Total tokens
3.6M
Output
13.2K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1Fasta Compute LengthProbe OpenSpliceAI 0.0.5 inference runtimeOpenSpliceAI 0.0.5 OSAIMANE-10000nt five-checkpoint exon reconstruction
Total tokens
4.5M
Output
17.9K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1OpenSpliceAI MANE 10000nt five-checkpoint exon reconstruction
Total tokens
2.6M
Output
11.6K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

RerunMatches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic DNAbedtools getfastaOpenSpliceAI probeOpenSpliceAI OSAI-MANE 10000nt predict
Total tokens
11.1M
Output
81.3K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1pysamProbe OpenSpliceAI Deps ModelsProbe Render DollarProbe Render Zip3 more tools
Total tokens
4.0M
Output
46.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1Bedtools GetfastabedOpenSpliceAI v0.0.5 predictOpenSpliceAI probeOpenSpliceAI probe 216 more tools
Total tokens
28.3M
Output
138.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunMatches predictionUDT + standard tools
Answer7
Workflow
Extract genomic DNA 1 → User-defined tool
Total tokens
17.7M
Output
48.4K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic Dna 1OpenSpliceAI runtime probeOpenSpliceAI v0.0.5 OSAI-MANE-10000nt five-checkpoint ensembleOpenSpliceAI v0.0.5 OSAI-MANE-10000nt five-checkpoint ensemble corrected coordi...OpenSpliceAI thresholded site exon reconstruction1 more tools
Total tokens
9.5M
Output
38.6K
Codex GPT-5.6 Luna

Replicate 3

RerunMatches predictionUDT + standard tools
Answer7
Workflow
Extract Genomic DNAOpenSpliceAI v0.0.5 runtime probeUDT template diagnosticUDT input diagnosticOpenSpliceAI v0.0.5 runtime probe fixed8 more tools
Total tokens
12.9M
Output
56.2K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Matches prediction
Answer7
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answer7
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answer7
Workflow
Local codePython
Total tokens
4.2M
Output
25.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answer7
Workflow
Local codePython
Total tokens
2.1M
Output
22.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answer7
Workflow
Local codePython
Total tokens
4.2M
Output
37.0K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answer7
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answer7
Workflow
Local codePython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answer7
Workflow
Local codePythonpysam
Total tokens
854.7K
Output
3.8K