Back to results

CompBioBench item

phase-chain-q1

Question

Files phase.chain.q1.R1.fq.gz and phase.chain.q1.R2.fq.gz contain exactly 1000 read pairs (2x150) sampled 50:50 from 2 synthetic haplotypes covering a 1.5kb region of hg38. Discover all biallelic SNPs and 1-bp indels in the interval, and phase by read-backed evidence only. Return the haplotypes in genomic order in the form A-G-T-I|G-A-C-C for the 2 haploytpes, using I for insertion and D for deletion. Use alphabetical ordering to decide which haplotype comes first.

Inferred answer
A-C-T-D-C-A-D-G|C-T-A-G-G-D-G-AUniquely inferred from submitted scores.
Domain
Population Genetics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemFreebayesPhase chain q1Summarize VCF q1Phase chain q1 primitive
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Galaxy dataset inspection
Total tokens
1.9M
Output
20.1K
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Phase chain q1 read-backed analysisPhase chain q1 read-backed analysis v2UDT probe q1UDT probe q1 v2
Total tokens
4.1M
Output
26.2K
Galaxy historyAgent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemFreebayesPicard AddorreplacereadgroupsSamtools PhaseGrep1 more tools
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemSamtools IdxstatsSamtools FlagstatFreebayesRead-backed phase chain
Total tokens
2.5M
Output
12.7K
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemSamtools FlagstatFreebayesSamtools
Total tokens
2.8M
Output
12.9K
Galaxy historyAgent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemSamtools IdxstatsBedtools BamtobedSamtools DepthFreebayes10 more tools
Total tokens
18.4M
Output
82.9K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

RerunIncorrectUDT + standard tools
AnswerA-C-T-D-C-I-D-G|C-T-A-I-G-D-I-A
Workflow
Map with BWA-MEMSambamba sortBAM-to-SAMbcftools mpileupbcftools call1 more tools
Total tokens
16.2M
Output
138.7K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemSamtools FlagstatSamtools IdxstatsSamtools DepthFreebayes10 more tools
Total tokens
30.6M
Output
83.4K
Galaxy historyAgent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa Mem2Phase read diagnosticBwaPhase BAM mapping summaryBedtools Getfastabed6 more tools
Total tokens
7.2M
Output
49.2K
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bowtie2Samtools IdxstatsBam To SamSamtools FlagstatSamtools Coverage8 more tools
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT + standard tools
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Bwa MemSamtools FlagstatSamtools IdxstatsSamtools CoverageBam To Sam5 more tools
Total tokens
6.3M
Output
55.4K
Galaxy historyAgent trace

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools4 more tools
Agent trace
Codex GPT-5.5

Replicate 2

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codePythonMinimap2BWA-MEMSamtools3 more tools
Agent trace
Codex GPT-5.5

Replicate 3

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codePythonBLASTNMinimap2Samtools3 more tools
Agent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeMinimap2BWA-MEMSamtoolsBCFtools4 more tools
Agent trace
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeMinimap2BWA-MEMSamtoolsBCFtools3 more tools
Agent trace
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools2 more tools
Agent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeBWA-MEMSamtoolsMinimap2BCFtools2 more tools
Total tokens
5.2M
Output
39.3K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeBWA-MEMMinimap2Bowtie2Samtools5 more tools
Total tokens
5.9M
Output
58.3K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codePythonBWA-MEMSamtoolsMinimap24 more tools
Total tokens
3.3M
Output
38.2K
Agent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Incorrect
AnswerA-C-T-D-C-A-G-C-G|C-T-A-I-G-G-C-G-A
Workflow
Local codeSamtoolsMinimap2BWA-MEMBCFtools6 more tools
Agent trace
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeMinimap2BWA-MEMBowtie2Samtools5 more tools
Agent trace
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codeBWA-MEMMinimap2SamtoolsBCFtools4 more tools
Agent trace

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerA-C-T-D-C-A-D-G|C-T-A-G-G-D-G-A
Workflow
Local codePythonBWA-MEMSamtoolsMinimap23 more tools
Total tokens
560.4K
Output
6.8K
Agent trace