Back to results

CompBioBench item

geo-lookup-read-matrix-market-q1

Question

From the supplementary data of the GEO accession GSE242423, for the D2 scRNA-seq matrix, retrieve the counts for SOX2 for barcode AAACCCAAGTCTTCGA-1.

Predicted answer
20Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Single-cell
Question style
Retrieval

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
Answer20
Workflow
GSE242423 D2 SOX2 barcode count lookup
Total tokens
300.3K
Output
4.2K
Codex GPT-5.5

Replicate 2

Matches predictionUDT + standard tools
Answer20
Workflow
GSE242423 D2 SOX2 count lookupGSE242423 D2 SOX2 count lookup v2GSE242423 D2 SOX2 count lookup v3
Total tokens
2.2M
Output
15.2K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
Answer20
Workflow
GEO D2 scRNA Matrix DiagnosticGEO D2 scRNA Matrix Diagnostic v2Converter Gz To UncompressedNlGrep1 more tools
Total tokens
1.9M
Output
9.9K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionStandard tools only
Answer20
Workflow
NlTp Grep ToolShow Beginning1Wc Gnu
Total tokens
3.5M
Output
7.6K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
Answer20
Workflow
Upload1UnzipNlGrepGSE242423 D2 matrix diagnostic1 more tools
Total tokens
5.5M
Output
12.5K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionStandard tools only
Answer20
Workflow
NlTp Grep Tool
Total tokens
1.3M
Output
4.1K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionStandard tools only
Answer20
Workflow
Filter TabularNlTp Grep Tool
Total tokens
2.2M
Output
16.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
Answer20
Workflow
Filter Tabular
Total tokens
4.1M
Output
19.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
Answer20
Workflow
Filter Tabular
Total tokens
2.7M
Output
17.7K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionStandard tools only
Answer20
Workflow
Data FetchUpload1NlFilter TabularTp Grep Tool
Total tokens
6.0M
Output
19.4K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionStandard tools only
Answer20
Workflow
Filter Tabular
Total tokens
2.4M
Output
15.7K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionStandard tools only
Answer20
Workflow
NlTp Grep Tool
Total tokens
792.8K
Output
6.4K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answer20
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Matches prediction
Answer20
Workflow
Local code
Codex GPT-5.5

Replicate 3

Matches prediction
Answer20
Workflow
Local code

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answer20
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answer20
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answer20
Workflow
Local code

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answer20
Workflow
Local codePythonR
Total tokens
193.4K
Output
2.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answer20
Workflow
Local code
Total tokens
206.2K
Output
4.6K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answer20
Workflow
Local codePython
Total tokens
195.4K
Output
4.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answer20
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answer20
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answer20
Workflow
Local code

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answer20
Workflow
Local codePython
Total tokens
187.5K
Output
1.4K