Back to results

CompBioBench item

sample-swap-atac-q1

Question

I have bulk ATAC-seq counts lying around from axolotl organs (sample.swap.atac.q1.tsv.gz). Since the genome is large, I broke down standard AmexG_v6.0-DD chromosomes into smaller chunks (sample.swap.atac.q1.chrom.sizes). I'm not sure, but I suspect there is a sample swap. Can you check if that is the case? If there isn't, respond "None". Else respond with the names of the two cell types that have been swapped in case-sensitive lexicographic order, e.g. "Brain,GallBladder".

Inferred answer
Cloaca,StomachUniquely inferred from submitted scores.
Domain
Single-cell
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
ATAC swap diagnosticsATAC chromosome fractionsATAC marker bigWig compareATAC gallbladder pancreas marker compareATAC GEO prefix match1 more tools
Total tokens
2.7M
Output
26.1K
Codex GPT-5.5

Replicate 2

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
ATAC count table diagnosticsATAC internal correlation QCATAC source marker validationATAC source bigWig proxy comparisonATAC source bigWig proxy comparison v24 more tools
Total tokens
8.9M
Output
38.7K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerCloaca,Stomach
Workflow
ATAC swap diagnosticUDT render probe commandUDT render probe shellUDT render probe docshapeFilter Tabular2 more tools
Total tokens
5.7M
Output
27.2K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
Galaxy UDTprobeProbeProbe 2Validate axolotl annotation and chunk coordinates5 more tools
Total tokens
29.4M
Output
36.0K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
Inspect Axolotl ArchR referenceInspect Axolotl ArchR reference raw attributesExtract Axolotl authoritative accessibility referenceIdentify Axolotl ATAC sample swap
Total tokens
3.4M
Output
20.9K
Codex GPT-5.6 Sol

Replicate 3

IncorrectUDT-only
AnswerNone
Workflow
Upload1Axolotl ATAC swap diagnosticAxolotl ATAC swap evidence synthesisFinal axolotl ATAC swap answer
Total tokens
7.0M
Output
24.4K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
Sample Swap ATAC Analysis
Total tokens
17.6M
Output
76.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
Inspect RDSInspect RDS RawCompare Original FragmentsCompare Original Fragments v2Compare Original Fragments v3
Total tokens
7.4M
Output
51.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
AnswerCloaca,Stomach
Workflow
Wc GnuDatamash OpsSummary Statistics1Select columnsDatamash Transpose9 more tools
Total tokens
8.5M
Output
81.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
ATAC sample swap diagnosticNormalized ATAC sample-swap diagnosticSummarize normalized ATAC diagnosticMatch axolotl ATAC counts to public source fragmentsValidate axolotl ATAC sample swap
Total tokens
10.4M
Output
50.5K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
Axolotl ATAC sample-swap diagnosticAxolotl ATAC source metadata matchGalaxy GEO ranged BED probeAxolotl ATAC GEO source profile matchAxolotl ATAC GEO source profile match v23 more tools
Total tokens
23.2M
Output
63.8K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT-only
AnswerCloaca,Stomach
Workflow
ATAC Profile DiagnosticGTF Schema InspectionGene Accessibility ScanAxolotl Organ Marker ScoresNamed Organ Gene Discovery1 more tools
Total tokens
10.3M
Output
53.1K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerCloaca,Stomach
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
AnswerCloaca,Stomach
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Correct
AnswerCloaca,Stomach
Workflow
Local codePython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerCloaca,Stomach
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerCloaca,Stomach
Workflow
Local codePython
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerCloaca,Stomach
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonSciPy
Total tokens
5.7M
Output
56.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonRSciPy
Total tokens
13.6M
Output
46.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonBedtoolsSciPy
Total tokens
4.1M
Output
39.8K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonSciPyR
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonSciPypysam
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerCloaca,Stomach
Workflow
Local codePython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerCloaca,Stomach
Workflow
Local codePythonSciPy
Total tokens
1.0M
Output
7.2K