Back to results

CompBioBench item

gwas-ancestry-q1

Question

Identify the ancestry group which was used to compute these summary statistics in the file gwas.ancestry.q1.tsv.gz. Your answer should be one of the following population codes: AFR (African), ASJ (Ashkenazi Jew), EAS (East Asian), EUR (European), HIS (Hispanic), MID (Middle Eastern), OCE (Oceania), SAS (South Asian). Respond with the 3 letter code only.

Inferred answer
HISUniquely inferred from submitted scores.
Domain
Population Genetics
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
AnswerHIS
Workflow
GWAS top p diagnosticGWAS LD ancestry comparisonGWAS clump lead lociGWAS clumped LD ancestry comparison
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
AnswerHIS
Workflow
Inspect GWAS ancestry q1Inspect GWAS ancestry q1 v2Inspect GWAS ancestry q1 v3Inspect GWAS ancestry q1 v4UDT render probe v13 more tools
Total tokens
3.2M
Output
18.5K
Galaxy historyAgent trace
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerHIS
Workflow
Inspect GWAS Ancestry Q1Inspect GWAS Ancestry Q1 v2Inspect GWAS Ancestry Q1 v3UDT Echo Probe Q1Sort rows5 more tools
Total tokens
6.1M
Output
21.9K
Galaxy historyAgent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT-only
AnswerHIS
Workflow
Probe GWAS and EnsemblProbe GWAS and Ensembl v2GWAS top hitsEnsembl population labelsLD ancestry comparison2 more tools
Total tokens
1.8M
Output
11.4K
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
AnswerHIS
Workflow
Datamash OpsGWAS p-value LD decay diagnosticTp Sort Header ToolBCFtoolsCompare GWAS p-value LD and variant selection with 1000 Genomes populations
Total tokens
2.6M
Output
18.4K
Galaxy historyAgent trace
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
AnswerHIS
Workflow
Tp Sort Header ToolGWAS ancestry diagnosticExact Yengo ancestry match
Total tokens
4.5M
Output
14.1K
Galaxy historyAgent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
AnswerHIS
Workflow
Tp Sort Header Tool
Total tokens
13.2M
Output
42.2K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
AnswerHIS
Workflow
Tp Sort Header Tool
Total tokens
9.9M
Output
29.4K
Galaxy historyAgent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunIncorrectUDT-only
AnswerAFR
Workflow
User-defined tool
Total tokens
17.6M
Output
91.3K
Galaxy historyAgent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT-only
AnswerHIS
Workflow
GWAS ancestry descriptive probeGWAS missing-variant ancestry frequency probeGWAS ancestry frequency pattern summaryGWAS ancestry LD profile probeGWAS ancestry LD subpopulation probe11 more tools
Total tokens
22.2M
Output
108.4K
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
AnswerHIS
Workflow
Wc GnuSummary Statistics1Datamash OpsTp Sort Header ToolFilter Tabular4 more tools
Total tokens
20.0M
Output
51.4K
Galaxy historyAgent trace
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
AnswerHIS
Workflow
Datamash OpsCount1Sort rowsFilter rowsTp Cut Tool1 more tools
Total tokens
10.3M
Output
30.3K
Galaxy historyAgent trace

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerHIS
Workflow
Local codeBCFtoolsPLINKPythonpysam1 more tools
Agent trace
Codex GPT-5.5

Replicate 2

Correct
AnswerHIS
Workflow
Local codeBCFtoolsPLINKPythonSciPy
Agent trace
Codex GPT-5.5

Replicate 3

Correct
AnswerHIS
Workflow
Local codeBCFtoolsPLINKPython
Agent trace

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerHIS
Workflow
Local codeBCFtoolsPLINKPythonSciPy
Agent trace
Codex GPT-5.6 Sol

Replicate 2

Incorrect
AnswerEUR
Workflow
Local codeBCFtoolsPythonPLINKpysam1 more tools
Agent trace
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerHIS
Workflow
Local codeBCFtoolsPLINKRPython
Agent trace

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

RerunCorrect
AnswerHIS
Workflow
Open-ended code
Total tokens
2.4M
Output
20.1K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 2

RerunCorrect
AnswerHIS
Workflow
Open-ended code
Total tokens
7.9M
Output
37.5K
Agent trace
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunCorrect
AnswerHIS
Workflow
Open-ended code
Total tokens
4.1M
Output
20.5K
Agent trace

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunCorrect
AnswerHIS
Workflow
Open-ended code
Total tokens
21.9M
Output
56.9K
Agent trace
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerHIS
Workflow
Local codePython
Agent trace
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerHIS
Workflow
Local codePython
Agent trace

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerHIS
Workflow
Local codePythonSciPypysam
Total tokens
534.9K
Output
5.1K
Agent trace