Back to results

CompBioBench item

match-genotypes-q1

Question

match.genotypes.q1.tar.gz contains plink genotype files in hg38 for 4 individuals of European ancestry (match.genotypes.1.1.*) and a shuffled metadata table (match.genotypes.1.2.csv) for the same 4 individuals. Please provide the most likely mapping from the IIDs in the plink files to the sample IDs in the metadata file. Format your answer as follows: “SAMPLE001,MIX1;SAMPLE002,MIX2;SAMPLE003,MIX3;SAMPLE004,MIX4”. IIDs should be in lexicographic order. You are also given genotypes for an additional set of European ancestry individuals (match.genotypes.1.3.*), which could be helpful. The individuals contained in match.genotypes.1.1.* are non-overlapping with match.genotypes.1.3.*

Predicted answer
SAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Population Genetics
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
match q1 probePLINKmatch q1 read extract logmatch q1 analyze abo
Codex GPT-5.5

Replicate 2

RerunMatches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Inspect match genotype archive q1ABO region diagnostic q1Summarize ABO raw genotypes q1ABO PVAR key lines q1ABO O deletion window q13 more tools
Total tokens
2.3M
Output
28.7K
Codex GPT-5.5

Replicate 3

RerunMatches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
match q1 inspectmatch q1 ABO VCFmatch q1 classify
Total tokens
978.4K
Output
9.8K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Converter Archive To DirectoryInspect matching genotype inputsInspect ABO and sex markersPLINKSummarize ABO marker extract1 more tools
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Inspect match genotype archive metadataPLINKInspect chromosome inventory and ABO marker presenceFinal IID to shuffled metadata assignment
Total tokens
3.4M
Output
12.8K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT + standard tools
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Inspect match genotype archiveConverter Archive To DirectoryPLINKFilter TabularJoin datasets4 more tools
Total tokens
24.5M
Output
59.7K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
PLINKInspect ABO raw columnsInfer ABO mappingNearest reference ABO patternsFinal ABO mapping
Total tokens
5.4M
Output
46.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
PLINKABO Classify Query
Total tokens
2.7M
Output
27.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
PLINKABO genotype matcher
Total tokens
2.5M
Output
40.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunMatches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
PGEN runtime probePGEN runtime probe v2PGEN runtime probe v3PGEN runtime probe v4Genotype input inspection16 more tools
Total tokens
33.1M
Output
101.7K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT + standard tools
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Converter Archive To DirectoryPLINKMatch template probeMatch staged probeBusybox probe4 more tools
Total tokens
59.3M
Output
153.0K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT-only
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Converter Archive To DirectoryInspect match genotype archiveInspect match genotype tarballInspect ABO regionInspect ABO variant window6 more tools
Total tokens
4.8M
Output
37.3K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKBCFtoolsPython
Codex GPT-5.5

Replicate 2

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPython
Codex GPT-5.5

Replicate 3

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPython
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKBCFtoolsPython
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPython
Total tokens
1.8M
Output
23.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePythonSciPyPLINKBCFtools
Total tokens
1.4M
Output
24.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPythonSciPy
Total tokens
1.4M
Output
15.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePythonPLINK
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKBCFtoolsPython
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePythonPLINKBCFtools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
AnswerSAMPLE001,MIX2;SAMPLE002,MIX4;SAMPLE003,MIX3;SAMPLE004,MIX1
Workflow
Local codePLINKPythonR
Total tokens
204.7K
Output
2.2K