Back to results

CompBioBench item

1000G-retrieve-genotype-q2

Question

I would like to make a file with genotypes at specific loci for all individuals from the 1000G data with 3202 individuals (20201028_3202_raw_GT_with_annot). The positions should be common SNPs- snp151Common from UCSC, hg38. Select SNPs in which the observed allele column consists of exactly two distinct bases, where both alleles are strictly one of A, C, G, or T (i.e. not multi-allelic or indel). Further filter them to be within chr1 and within the 1000 genome-wide strongest peaks of the ENCODE peak file with accession ENCFF717TCQ (based on the qValue column). Create an uncompressed tsv with columns CHROM, POS, REF, ALT, and then the 3202 identifiers in lexicographic order (include header). In this tsv, the REF and ALT should come from the 1000G VCF. Keep those where both REF and ALT in 1000G are both exactly one of A,C,G,T. Convert any phased calls to unphased. Then, in the genotype cells, include only one of "0/0", "0/1", "1/1" or "./.", reserving "./." for any that were not exactly one of the first 3. The SNPs should be sorted by coordinate. Report the total number of lines in this file along with the md5sum of this file, in the format "201;md5sum".

Predicted answer
296;864b06501a76b44b3037072e25c99725Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Population Genetics
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Make 1000G ENCODE chr1 SNP genotype TSV
Total tokens
484.3K
Output
9.7K
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Diagnose 1000G ENCODE UCSC inputsMake 1000G ENCODE genotype TSV
Total tokens
890.5K
Output
10.6K
Codex GPT-5.5

Replicate 3

RerunMatches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Diagnose 1000G ENCODE SNP candidatesDiagnose 1000G ENCODE SNP candidates reportSmoke UDT outputSmoke UDT output highmemSmoke UDT htslib output17 more tools
Total tokens
5.3M
Output
40.7K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
pysamValidate final 1000G genotype matrix
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Diagnose ENCODE UCSC and 1000G source conventionsBuild chr1 common ENCODE peak 1000G genotype matrix
Total tokens
1.5M
Output
14.3K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Validate ENCODE peak ranking and 1000G/UCSC coordinatesMake 1000G ENCODE common-SNP genotype matrix
Total tokens
12.5M
Output
17.8K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Probe HelloBuild TargetsInspect TargetsBuild Targets v2Make Genotype TSV
Total tokens
2.8M
Output
29.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionUDT-only
Answer296;864b06501a76b44b3037072e25c99725
Workflow
No Galaxy tool job recorded
Total tokens
16.0M
Output
90.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunMatches predictionUDT + standard tools
Answer296;864b06501a76b44b3037072e25c99725
Workflow
tp sort header tool → tp head tool → Filter1 → bcftools view → bcftools query list samples → User-defined tool → secure hash message digest → wc gnu
Total tokens
19.5M
Output
67.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT + standard tools
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Upload1Filter convention diagnosticUDT mechanics checkUDT static checkBcftools Query List Samples3 more tools
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT + standard tools
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Upload1Bcftools Query List SamplesBcftools ViewSource Convention DiagnosticSource Convention Diagnostic v210 more tools
Total tokens
24.0M
Output
72.5K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT + standard tools
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Upload1Genotype loci diagnosticUDT render probeUDT command probeUDT command probe two21 more tools
Total tokens
88.4M
Output
236.4K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsBedtoolsPython
Codex GPT-5.5

Replicate 2

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsSamtoolsPythonBedtools
Codex GPT-5.5

Replicate 3

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsPythonBedtools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codePythonBCFtools
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codePythonpysam
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsPython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codePythonBCFtools
Total tokens
7.2M
Output
46.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codePythonBCFtools
Total tokens
4.4M
Output
28.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsSamtoolsPythonBedtools
Total tokens
10.2M
Output
51.4K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsSamtoolsBedtools
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsPythonBedtools
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codeBCFtoolsSamtoolsPython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answer296;864b06501a76b44b3037072e25c99725
Workflow
Local codePythonBCFtoolspysam
Total tokens
1.7M
Output
10.6K