Back to results

CompBioBench item

sample-swap-rna-q2

Question

I have a scRNA-seq pseudobulk lying around from wheat (sample.swap.rna.q2.tsv.gz). I'm not sure, but I suspect there is a sample swap. Can you check if that is the case? If there isn't, respond "None". Else respond with the names of the two cell types that have been swapped in case-sensitive lexicographic order, e.g. "cortex,endodermis".

Predicted answer
NoneMost likely answer for this question; not uniquely determined from submitted scores.
Domain
Single-cell
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
AnswerNone
Workflow
Wheat pseudobulk swap checkWheat pseudobulk swap acceptance check
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
AnswerNone
Workflow
Wheat pseudobulk swap diagnosticsInspect wheat root supplement workbookInspect wheat root mmc5 xlsxWheat marker swap comparison
Total tokens
1.6M
Output
13.7K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
AnswerNone
Workflow
Wheat swap XLSX diagnosticUDT mechanical probeXlsx2TsvUDT mechanical probe 2UDT mechanical probe 33 more tools
Total tokens
4.3M
Output
18.4K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
AnswerNone
Workflow
Diagnostic text UDTRetrieve wheat atlas repository treeRetrieve published wheat atlas markersRetrieve PMC archive manifestList PMC wheat atlas archive16 more tools
Total tokens
16.1M
Output
29.9K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
AnswerNone
Workflow
Inspect wheat Seurat RDSInspect serialized wheat Seurat slotsInspect wheat Seurat RDS in Seurat 4.3Export wheat Seurat metadataDatamash Ops3 more tools
Total tokens
10.4M
Output
30.4K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT + standard tools
AnswerNone
Workflow
Wheat pseudobulk diagnosticsUnzipXlsx2TsvInspect wheat reference DEGsScore wheat reference markers1 more tools
Total tokens
6.9M
Output
18.6K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionUDT + standard tools
AnswerNone
Workflow
Tp Sort Header ToolSample Swap Marker Check
Total tokens
6.1M
Output
38.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionUDT-only
AnswerNone
Workflow
Swap marker scoreColumn marker diagnosticsSwap search
Total tokens
5.7M
Output
40.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionUDT + standard tools
AnswerNone
Workflow
Filter TabularTp Grep ToolDatamash OpsWheat Marker Swap Score
Total tokens
22.3M
Output
69.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT + standard tools
AnswerNone
Workflow
Xlsx2TsvWheat pseudobulk swap diagnosticWheat UDT preflightSelect columnsSort rows3 more tools
Total tokens
22.5M
Output
58.0K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT + standard tools
AnswerNone
Workflow
Tp Sort Header ToolSort rowsXlsx2TsvGrepWheat scRNA-seq two-column swap diagnostic
Total tokens
11.6M
Output
29.2K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT + standard tools
AnswerNone
Workflow
Wheat profile diagnosticWheat runtime probeWheat runtime probe v2Wheat runtime probe v3Datamash Transpose9 more tools
Total tokens
39.4M
Output
80.7K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePythonR

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePythonAnnDataScanpyR1 more tools
Total tokens
26.7M
Output
98.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Total tokens
5.1M
Output
38.1K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePython
Total tokens
7.5M
Output
43.7K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePythonR
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePythonRAnnData

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Total tokens
486.5K
Output
2.7K