Question
Consider the PDB protein in the file protein.shape.q1.pdb. Which of the following uppercase letters does it resemble most closely: B, D, F, H, J, L, N, P, R, T, V, X, Z? Consider all possible projections and pick the best fit.
- Inferred answer
FUniquely inferred from submitted scores.
- Domain
- Structure
- Question style
- Synthetic/Augmented Data
An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.
Galaxy
12/12 terminal item results
UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.
Codex GPT-5.5
3 replicates
Codex GPT-5.5
Replicate 1
IncorrectUDT-only
AnswerT
WorkflowProtein Letter Projection
Codex GPT-5.5
Replicate 2
IncorrectUDT-only
AnswerN
WorkflowPDB Shape Diagnostic→PDB Shape Diagnostic Redirected→PDB Shape Letter Score
- Total tokens
- 455.9K
- Output
- 12.0K
Codex GPT-5.5
Replicate 3
IncorrectUDT-only
AnswerT
WorkflowProtein Letter Projection Score→Protein Letter Projection Score v2→Protein Letter Projection Score v3→UDT Static Probe→UDT Static Probe v2→1 more tools
- Total tokens
- 1.7M
- Output
- 14.6K
Codex GPT-5.6 Sol
3 replicates
Codex GPT-5.6 Sol
Replicate 1
IncorrectUDT-only
AnswerT
WorkflowProtein projection exploration→Protein axis rotation scan→Protein letter silhouette score→Protein projection montage
- Total tokens
- 1.4M
- Output
- 15.1K
Codex GPT-5.6 Sol
Replicate 2
IncorrectUDT + standard tools
AnswerT
WorkflowProtein projection diagnostic→Protein projection diagnostic v2→Protein projection diagnostic v3→Cat projection probe→Grep→3 more tools
- Total tokens
- 3.1M
- Output
- 14.0K
Codex GPT-5.6 Sol
Replicate 3
IncorrectUDT + standard tools
AnswerT
WorkflowPDB coordinate diagnostic Q1→PDB coordinate diagnostic Q1 v2→PDB coordinate diagnostic Q1 v3→PDB coordinate diagnostic Q1 v4→PDB coordinate diagnostic Q1 v5→6 more tools
- Total tokens
- 8.6M
- Output
- 18.8K
Codex + DeepSeek-v4-pro-0813
3 replicates
Codex + DeepSeek-v4-pro-0813
Replicate 1
CorrectUDT-only
AnswerF
WorkflowInspect PDB→PCA PDB→Project ASCII→Project PNG→Compare PDB→1 more tools
- Total tokens
- 2.1M
- Output
- 27.0K
Codex + DeepSeek-v4-pro-0813
Replicate 2
CorrectUDT-only
AnswerF
WorkflowSciPy→Protein shape letter match
- Total tokens
- 3.9M
- Output
- 41.5K
Codex + DeepSeek-v4-pro-0813
Replicate 3
CorrectUDT + standard tools
AnswerF
WorkflowSciPy→Probe Matplotlib→Shape ASCII Projections→Shape Connected Lines ASCII→Shape Render PGM→2 more tools
- Total tokens
- 3.4M
- Output
- 30.1K
Codex GPT-5.6 Luna
3 replicates
Codex GPT-5.6 Luna
Replicate 1
IncorrectUDT + standard tools
AnswerZ
WorkflowProtein shape diagnostic→Protein shape diagnostic v2→Protein shape diagnostic v3→UDT render probe→Api Udt Probe V1→9 more tools
- Total tokens
- 53.4M
- Output
- 107.1K
Codex GPT-5.6 Luna
Replicate 2
RerunIncorrectUDT-only
AnswerN
WorkflowPDB Shape Statistics→PDB Projection Diagnostic→PDB Letter Projection Score→PDB Letter Projection Score Refined→PDB Best Projection ASCII→4 more tools
- Total tokens
- 5.3M
- Output
- 41.9K
Codex GPT-5.6 Luna
Replicate 3
CorrectUDT-only
AnswerF
WorkflowPDB Shape Diagnostic→PDB Shape Projection Render→PDB Shape Letter Match→Galaxy UDT→PDB Specific Projection Views
- Total tokens
- 5.8M
- Output
- 43.3K