Back to results

CompBioBench item

vcf-infer-build-q1

Question

You are given vcf.infer.build.q1.vcf.gz, a bgzipped VCF containing biallelic SNPs on chromosome 20 for one sample. The VCF has no rsIDs and does not specify a reference build in the header. Determine which reference genome build the VCF coordinates and REF alleles correspond to. Respond with exactly one of: hg18, hg19, hg38, T2T-CHM13.

Predicted answer
hg19Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Population Genetics
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
Answerhg19
Workflow
Infer VCF reference build q1Infer VCF reference build q1 v2
Total tokens
426.7K
Output
4.9K
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
Answerhg19
Workflow
Infer reference build from VCF REF alleles via UCSCInfer reference build from VCF REF alleles via UCSC v2
Total tokens
708.8K
Output
6.8K
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
Answerhg19
Workflow
Infer VCF Reference Build Chr20Infer VCF Reference Build Chr20 v2Infer VCF Reference Build Chr20 v3Infer VCF Reference Build Chr20 v4UDT Render Probe3 more tools
Total tokens
4.4M
Output
28.9K
Galaxy historyAgent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT + standard tools
Answerhg19
Workflow
Bcftools NormGalaxy UDTinfer_buildInfer reference buildInfer reference build test1 more tools
Total tokens
7.6M
Output
21.7K
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
Answerhg19
Workflow
Bcftools NormConverter Gz To UncompressedCompare VCF REF alleles to four human builds diagnosticCompare VCF REF alleles to four human builds diagnostic v2Full VCF REF-build concordance validation
Total tokens
4.8M
Output
17.8K
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT + standard tools
Answerhg19
Workflow
Bcftools NormConverter Gz To UncompressedAdd Line To FileReference build probeFull hg19 reference validation
Total tokens
6.4M
Output
17.4K
Galaxy historyAgent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionUDT + standard tools
Answerhg19
Workflow
Extract Genomic Dna 1Infer Build Q1
Total tokens
2.0M
Output
23.0K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
Answerhg19
Workflow
Bcftools NormBedtools GetfastabedVcf2TsvTp Replace In ColumnAdd A Column16 more tools
Total tokens
15.8M
Output
47.4K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
Answerhg19
Workflow
Bcftools NormConverter Gz To UncompressedAdd Line To FileTp Replace In LineTp Cat5 more tools
Total tokens
17.2M
Output
72.5K
Galaxy historyAgent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionStandard tools only
Answerhg19
Workflow
Bcftools NormConverter Vcf Bgzip To Tabix 0Converter Gz To UncompressedTp Text File With Recurring LinesAdd Line To File7 more tools
Total tokens
35.4M
Output
90.5K
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 2

Matches predictionStandard tools only
Answerhg19
Workflow
Bcftools NormConverter Gz To UncompressedBcftools ViewConverter Interval To Bgzip 0Converter Vcf Bgzip To Tabix 06 more tools
Total tokens
17.5M
Output
51.7K
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 3

Matches predictionStandard tools only
Answerhg19
Workflow
Bcftools NormAdd Line To FileBcftools QueryShow Tail1Show Beginning12 more tools
Total tokens
11.6M
Output
51.4K
Galaxy historyAgent trace

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answerhg19
Workflow
Local codePython
Agent trace
Codex GPT-5.5

Replicate 2

Matches prediction
Answerhg19
Workflow
Local codeBCFtoolsSamtoolsPython
Agent trace
Codex GPT-5.5

Replicate 3

Matches prediction
Answerhg19
Workflow
Local codeSamtoolsBCFtoolsPython
Agent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answerhg19
Workflow
Local codePython
Agent trace
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answerhg19
Workflow
Local codeBCFtoolsSamtools
Agent trace
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answerhg19
Workflow
Local code
Agent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answerhg19
Workflow
Local codePython
Total tokens
971.5K
Output
10.5K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answerhg19
Workflow
Local codeSamtoolsBCFtoolsPython
Total tokens
433.5K
Output
9.7K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answerhg19
Workflow
Local codePythonSamtoolspysam
Total tokens
289.2K
Output
4.5K
Agent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answerhg19
Workflow
Local codeBCFtoolsSamtools
Agent trace
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answerhg19
Workflow
Local codeSamtoolsBCFtools
Agent trace
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answerhg19
Workflow
Local codeSamtoolsBCFtoolsBedtools
Agent trace

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answerhg19
Workflow
Local codePython
Total tokens
56.1K
Output
702
Agent trace