Codex GPT-5.5
Replicate 1
CorrectUDT + standard tools
Answer
36224396Workflow
Converter Gz To UncompressedShow Beginning1Filter rowsGWAS top P anchorsCompare top GWAS markers to GCST900166674 more tools
Question
Identify the PubMed ID of the study from which the summary stats in reverse.search.gwas.q2.gz are derived. Print only the PMID, e.g. 31510655.
An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.
Galaxy
12/12 terminal item resultsUDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.
Codex GPT-5.5
3 replicatesCodex GPT-5.5
Replicate 2
CorrectUDT + standard tools
Answer
36224396Workflow
Top GWAS Hits Q2Compare GWAS Source Q2
- Total tokens
- 3.9M
- Output
- 19.1K
Codex GPT-5.5
Replicate 3
CorrectUDT + standard tools
Answer
36224396Workflow
Inspect GWAS Q2Inspect GWAS Q2 v2Inspect GWAS Q2 v3Inspect GWAS Q2 v4UDT Render Probe Q2
- Total tokens
- 14.8M
- Output
- 46.5K
Codex GPT-5.6 Sol
3 replicatesCodex GPT-5.6 Sol
Replicate 1
CorrectUDT-only
Answer
36224396Workflow
Converter Gz To UncompressedPreview GWAS headerSummarize GWAS rows and strongest associationsStrongest nonmissing GWAS associationsRetrieve GWAS Catalog associations for lead variant13 more tools
- Total tokens
- 14.1M
- Output
- 23.2K
Codex GPT-5.6 Sol
Replicate 2
CorrectUDT-only
Answer
36224396Workflow
GWAS compressed-file fingerprint diagnosticGWAS top-hit diagnosticUpload1GWAS exact authoritative-source match
- Total tokens
- 4.1M
- Output
- 14.9K
Codex GPT-5.6 Sol
Replicate 3
CorrectUDT + standard tools
Answer
36224396Workflow
Inspect compressed GWAS summary statisticsScan GWAS association fingerprintParse authoritative GWAS Catalog associationsGrepFilter authoritative height study candidates13 more tools
- Total tokens
- 43.9M
- Output
- 67.7K
Codex + DeepSeek-v4-pro-0813
3 replicatesCodex + DeepSeek-v4-pro-0813
Replicate 1
CorrectUDT-only
Answer
36224396Workflow
Identify GWAS source
- Total tokens
- 20.6M
- Output
- 78.4K
Codex + DeepSeek-v4-pro-0813
Replicate 2
CorrectStandard tools only
Answer
36224396Workflow
Select columnsTp Sort Header ToolJoin datasetsGrep
- Total tokens
- 29.3M
- Output
- 68.3K
Codex + DeepSeek-v4-pro-0813
Replicate 3
CorrectStandard tools only
Answer
36224396Workflow
Converter Gz To UncompressedTp Easyjoin ToolRandom Lines1Tp Cut ToolDiff1 more tools
- Total tokens
- 30.7M
- Output
- 74.6K
Codex GPT-5.6 Luna
3 replicatesCodex GPT-5.6 Luna
Replicate 1
CorrectStandard tools only
Answer
36224396Workflow
Converter Gz To UncompressedShow Beginning1Tp Sort Header ToolSelect columnsUpload12 more tools
- Total tokens
- 30.0M
- Output
- 82.5K
Codex GPT-5.6 Luna
Replicate 2
IncorrectStandard tools only
Answer
33713608Workflow
Converter Gz To UncompressedTp Head ToolTp Sort Header ToolSort rowsFilter rows4 more tools
- Total tokens
- 37.5M
- Output
- 73.9K
Codex GPT-5.6 Luna
Replicate 3
CorrectStandard tools only
Answer
36224396Workflow
Converter Gz To UncompressedTp Head ToolSort rowsFilter TabularTp Grep Tool6 more tools
- Total tokens
- 84.5M
- Output
- 225.4K
Anycode
13/13 terminal item resultsCodex GPT-5.5
3 replicatesCodex GPT-5.5
Replicate 1
Incorrect
Answer
25865494Workflow
Local codePython
Codex GPT-5.5
Replicate 2
Incorrect
Answer
25865494Workflow
Local code
Codex GPT-5.5
Replicate 3
Correct
Answer
36224396Workflow
Local code
Codex GPT-5.6 Sol
3 replicatesCodex GPT-5.6 Sol
Replicate 1
Correct
Answer
36224396Workflow
Local codePythonSciPy
Codex GPT-5.6 Sol
Replicate 2
Incorrect
Answer
35399580Workflow
Local codePython
Codex GPT-5.6 Sol
Replicate 3
Correct
Answer
36224396Workflow
Local codeBCFtools
Codex + DeepSeek-v4-pro-0813
3 replicatesCodex + DeepSeek-v4-pro-0813
Replicate 1
Correct
Answer
36224396Workflow
Local codePythonBCFtools
- Total tokens
- 26.1M
- Output
- 87.3K
Codex + DeepSeek-v4-pro-0813
Replicate 2
Correct
Answer
36224396Workflow
Local codePython
- Total tokens
- 11.5M
- Output
- 41.5K
Codex + DeepSeek-v4-pro-0813
Replicate 3
Correct
Answer
36224396Workflow
Local codePythonBCFtoolspysamSciPy
- Total tokens
- 41.6M
- Output
- 133.7K
Codex GPT-5.6 Luna
3 replicatesCodex GPT-5.6 Luna
Replicate 1
RerunCorrect
Answer
36224396Workflow
Open-ended code
- Total tokens
- 23.7M
- Output
- 67.0K
Codex GPT-5.6 Luna
Replicate 2
Correct
Answer
36224396Workflow
Local codeBCFtoolsPython
Codex GPT-5.6 Luna
Replicate 3
Correct
Answer
36224396Workflow
Local codePython
Codex GPT-6 Astra
1 replicateCodex GPT-6 Astra
Replicate 1
Correct
Answer
36224396Workflow
Local codePython
- Total tokens
- 2.1M
- Output
- 8.7K