Back to results

CompBioBench item

sample-swap-atac-q2

Question

I have a single-cell ATAC-seq cluster counts pseudobulks lying around from human retinal-associated tissue (sample.swap.atac.q2.tsv.gz). I'm not sure, but I suspect there is a sample swap. Can you check if that is the case? If there isn't, respond "None". Else respond with the names of the two cell types that have been swapped in case-sensitive lexicographic order, e.g. "AIIamacrine,Rod".

Predicted answer
NoneMost likely answer for this question; not uniquely determined from submitted scores.
Domain
Single-cell
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
AnswerNone
Workflow
Detect scATAC pseudobulk cell-type swapValidate scATAC swap call across flanks
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
AnswerNone
Workflow
Retina ATAC profile diagnosticRetina ATAC marker swap checkRetina ATAC marker window check
Total tokens
645.4K
Output
13.3K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
AnswerNone
Workflow
ATAC pseudobulk diagnosticATAC pseudobulk diagnostic v2UDT minimal copyGrepBedtools Intersectbed1 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
AnswerNone
Workflow
Galaxy UDTswap_checkswap checkRetinal ATAC pseudobulk swap checkRetinal ATAC pseudobulk swap check corrected1 more tools
Total tokens
4.9M
Output
18.9K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
AnswerNone
Workflow
Retina pseudobulk marker accessibility diagnosticRetina pseudobulk swap validation
Total tokens
2.9M
Output
14.9K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT-only
AnswerNone
Workflow
Retina swap marker diagnosticRetina swap robust validation
Total tokens
1.2M
Output
12.2K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionStandard tools only
AnswerNone
Workflow
Tp Cut ToolBedtools IntersectbedBedops Sort BedTp Tail ToolBedtools Sortbed6 more tools
Total tokens
18.7M
Output
65.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionUDT + standard tools
AnswerNone
Workflow
Datamash OpsATAC pseudobulk statsATAC marker accessibility scoresATAC marker accessibility scores v2Network probe1 more tools
Total tokens
5.5M
Output
37.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionUDT-only
AnswerNone
Workflow
Retina ATAC sample swap checkRetina ATAC sample swap check v2
Total tokens
3.7M
Output
46.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT + standard tools
AnswerNone
Workflow
Summary Statistics1Retinal ATAC column diagnosticRetinal ATAC UDT runtime probeRetinal Atac Direct Build Probe4Select columns5 more tools
Total tokens
23.8M
Output
84.8K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT-only
AnswerNone
Workflow
ATAC swap diagnosticATAC marker locus probeATAC holdout marker assignmentATAC marker assignment validation
Total tokens
2.0M
Output
33.4K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT-only
AnswerNone
Workflow
ATAC Profile DiagnosticATAC Marker Locus ProbeATAC Build and Marker ComparisonATAC Sample Swap Check
Total tokens
3.5M
Output
32.2K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePythonSciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePythonSciPy
Total tokens
2.7M
Output
26.2K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePython
Total tokens
2.7M
Output
33.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePythonSciPy
Total tokens
9.1M
Output
40.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePythonSciPy
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
AnswerNone
Workflow
Local codePythonSciPy
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
AnswerNone
Workflow
Local codePython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
AnswerNone
Workflow
Local codePython
Total tokens
113.8K
Output
1.4K