Back to results

CompBioBench item

deleterious-mutation-q2

Question

In deleterious.mutation.q2.R1.fq.gz (exome 1×150 bp, reads from human 50Mb chunk of chr9 only), there is one gene that harbors a high-confidence nonsense SNV consistent with somatic mosaicism. The gene is highly LoF-intolerant. Report the HGNC gene symbol (uppercase) for the affected gene, and the approximate alternate allele frequency, rounded to the nearest 10%. Expected output format: POU5F1,10

Predicted answer
STXBP1,20Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Genomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
AnswerSTXBP1,20
Workflow
SamtoolsBCFtoolsBedtoolsMinimap2BWA-MEM1 more tools
Total tokens
517.1K
Output
11.0K
Codex GPT-5.5

Replicate 2

Matches predictionUDT + standard tools
AnswerSTXBP1,20
Workflow
Bwa MemSamtools FlagstatFreebayesSnpeffExtract stop gained SNVs2 more tools
Total tokens
2.5M
Output
16.4K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
AnswerSTXBP1,20
Workflow
SamtoolsBCFtoolsBedtoolsBWA-MEMMinimap28 more tools
Total tokens
2.6M
Output
13.1K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT + standard tools
AnswerSTXBP1,20
Workflow
FastqcBwa MemSamtools IdxstatsMinimap2BWA-MEM8 more tools
Total tokens
5.2M
Output
12.5K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
AnswerSTXBP1,20
Workflow
Bwa MemSamtools FlagstatFreebayesPicard AddorreplacereadgroupsSnpeff7 more tools
Total tokens
24.6M
Output
35.8K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
Bwa MemMinimap2Samtools FlagstatSamtools StatsLofreq Call3 more tools
Total tokens
13.5M
Output
15.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
Bwa MemSamtools FlagstatSamtools SortSamtools StatsSamtools Idxstats6 more tools
Total tokens
5.0M
Output
24.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
Bwa MemSamtools ViewSamtools IdxstatsBcftools MpileupBcftools Call5 more tools
Total tokens
9.1M
Output
29.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
Bwa MemSamtools IdxstatsSamtools FlagstatFreebayesPicard Addorreplacereadgroups4 more tools
Total tokens
4.9M
Output
22.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
FastqcFastq InfoBwaSamtools FlagstatSamtools Idxstats9 more tools
Total tokens
20.1M
Output
37.5K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
FastqcBwa MemSamtools FlagstatSamtools IdxstatsFreebayes6 more tools
Total tokens
9.1M
Output
30.8K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionStandard tools only
AnswerSTXBP1,20
Workflow
Bwa MemBowtie2Samtools SortSamtools FlagstatSamtools Idxstats5 more tools
Total tokens
4.5M
Output
21.4K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeMinimap2BWA-MEMSamtoolsBCFtools1 more tools
Codex GPT-5.5

Replicate 2

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools2 more tools
Codex GPT-5.5

Replicate 3

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMSamtoolsBCFtoolsMinimap23 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeSamtoolsBCFtoolsBWA-MEMMinimap22 more tools
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeSamtoolsBCFtoolsBWA-MEMMinimap2

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMMinimap2Bowtie2Samtools3 more tools
Total tokens
1.8M
Output
18.4K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools3 more tools
Total tokens
1.6M
Output
17.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMSamtoolsBCFtoolsMinimap22 more tools
Total tokens
773.7K
Output
10.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeSamtoolsBWA-MEMMinimap2BCFtools3 more tools
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeSamtoolsBWA-MEMMinimap2Bowtie24 more tools
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMBowtie2Minimap2Samtools4 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
AnswerSTXBP1,20
Workflow
Local codeBWA-MEMSamtoolsBCFtoolsMinimap22 more tools
Total tokens
340.5K
Output
5.4K