Back to results

CompBioBench item

read-proportions-q1

Question

read.proportions.q1.fa contains a synthetic genome with four chromosomes. read.proportions.q1.fq.gz contains 40,000 single-end 36 bp reads drawn from this genome. A read is generated by first picking a chromosome with probability pi =(pi_1, ..., pi_4), then sampling a position uniformly along that chromosome, with some small error rate. Estimate pi and report four integers (percentages) in FASTA contig order (chr1–chr4), each rounded to the nearest 5; if needed, adjust the last value so the total is 100. Expected output format: 15,25,25,35.

Inferred answer
50,20,20,10Uniquely inferred from submitted scores.
Domain
Genomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
Answer50,20,20,10
Workflow
Estimate read proportionsEstimate read proportions v2Estimate read proportions EMValidate read proportions variants
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Estimate read chromosome proportions by EMEstimate read chromosome proportions by EM forward check
Total tokens
2.8M
Output
20.4K
Codex GPT-5.5

Replicate 3

RerunCorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bowtie2Read proportions best-hit estimatorRead proportions EM best-hit estimatorFASTQ header origin diagnosticRead proportions EM best-hit rescue estimator
Total tokens
1.4M
Output
12.2K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answer50,20,20,10
Workflow
Bowtie2Samtools IdxstatsSamtools ViewConverter Fasta To FaiBedtools Makewindowsbed3 more tools
Total tokens
4.1M
Output
11.9K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bowtie2Bam To SamChromosome mixture EM from all Bowtie2 alignments
Total tokens
1.5M
Output
9.0K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bowtie2Samtools IdxstatsExhaustive seeded contig classifierContig proportion likelihood EMKallisto Quant
Total tokens
1.7M
Output
14.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

IncorrectStandard tools only
Answer35,25,25,15
Workflow
Bowtie2Samtools Idxstats
Total tokens
824.7K
Output
11.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bwa MemSamtools IdxstatsBwaSamtools ViewSciPy8 more tools
Total tokens
19.4M
Output
71.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bowtie2Samtools CoverageSamtools FlagstatFasta StatsSamtools View4 more tools
Total tokens
6.4M
Output
67.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Fasta StatsBowtie2Samtools IdxstatsBam To SamMinimap28 more tools
Total tokens
18.4M
Output
62.3K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
Answer50,20,20,10
Workflow
Bowtie2Samtools IdxstatsDatamash OpsAdd A Column1Bwa Mem8 more tools
Total tokens
11.5M
Output
67.2K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
Answer50,20,20,10
Workflow
Fasta Compute LengthBwa MemSamtools IdxstatsSamtools FlagstatBowtie24 more tools
Total tokens
4.3M
Output
19.8K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answer50,20,20,10
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
Answer50,20,20,10
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Correct
Answer50,20,20,10
Workflow
Local codePythonSciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2Bowtie2BWA-MEMSamtools1 more tools
Codex GPT-5.6 Sol

Replicate 2

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2Bowtie2BWA-MEMSamtools1 more tools
Codex GPT-5.6 Sol

Replicate 3

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2Bowtie2BWA-MEMPython1 more tools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
Answer50,20,20,10
Workflow
Local codePythonBowtie2BWA-MEMMinimap23 more tools
Total tokens
2.3M
Output
29.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answer50,20,20,10
Workflow
Local codePythonBowtie2BWA-MEMMinimap2
Total tokens
1.1M
Output
26.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
Answer50,20,20,10
Workflow
Local codePythonBowtie2BWA-MEMMinimap22 more tools
Total tokens
2.3M
Output
43.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
Answer50,20,20,10
Workflow
Local codePythonMinimap2Bowtie2BWA-MEM4 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2BWA-MEMBowtie2BLASTN5 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2BWA-MEMBowtie2BLASTN2 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answer50,20,20,10
Workflow
Local codeMinimap2BWA-MEMBowtie2Python
Total tokens
102.9K
Output
1.4K