Back to results

CompBioBench item

differential-composition-q1

Question

differential.composition.q1.1.mtx.gz and differential.composition.q1.2.mtx.gz contain raw scRNA counts matrices derived from retinal samples of two individuals (one per file, each column is a unique barcode that passes QC). differential.composition.q1.genes.txt.gz is the list of common genes. One of the cell types is severely depleted in one of the two individuals. What is this cell type? Return only the name of the depleted cell type from this list: astrocyte, microglial cell, RPE cell, cone cell, rod cell, horizontal cell, bipolar cell, RGC, T cells, macrophage, Schwann cell, Pericyte, B cell, fibroblast, endothelial cell, muller glia cell.

Predicted answer
horizontal cellMost likely answer for this question; not uniquely determined from submitted scores.
Domain
Single-cell
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT-only
Answerhorizontal cell
Workflow
Retina marker diagnosticsRetina composition marker analysisRetina composition marker analysis v2Retina composition HVG validation
Codex GPT-5.5

Replicate 2

Matches predictionUDT-only
Answerhorizontal cell
Workflow
Retina marker diagnosticRetina depleted cell type finalRetina depleted cell type final v2
Total tokens
487.6K
Output
9.6K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
Answerhorizontal cell
Workflow
Retina marker depletion diagnosticRetina marker diagnostic v2Retina marker diagnostic v3UDT copy probe v1UDT copy probe v22 more tools
Total tokens
7.2M
Output
38.6K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionUDT-only
Answerhorizontal cell
Workflow
Retina composition diagnosticRetina composition marker analysisRetina composition focused checkRetina rare cell depth check
Total tokens
1.6M
Output
17.4K
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT + standard tools
Answerhorizontal cell
Workflow
Retina composition input and marker diagnosticRetina joint clustering and marker reportRetina MatrixMarket to preprocessed h5adScanpy Cluster Reduce DimensionScanpy Find Markers3 more tools
Total tokens
4.8M
Output
21.2K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT + standard tools
Answerhorizontal cell
Workflow
Retina candidate cell-type marker screenRetina marker depletion comparisonPrepare retinal AnnData metadataAnndata ImportAnndata Manipulate8 more tools
Total tokens
11.0M
Output
27.1K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

RerunMatches predictionUDT + standard tools
Answerhorizontal cell
Workflow
Select firstCutImport AnndataInspect AnnDataScanpy normalize11 more tools
Total tokens
26.3M
Output
87.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
Answerhorizontal cell
Workflow
Scanpy Read 10XScanpy Normalise DataScanpy Find Variable GenesScanpy Run PcaScanpy Compute Graph4 more tools
Total tokens
7.1M
Output
45.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
Answerhorizontal cell
Workflow
Tp Text File With Recurring Lines
Total tokens
24.9M
Output
60.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT-only
Answerhorizontal cell
Workflow
Retinal marker composition diagnosticRetinal marker co-detection diagnosticRetinal pseudo-bulk marker differential diagnosticRetinal marker pair composition diagnosticRetinal final core-marker composition comparison
Total tokens
1.7M
Output
33.9K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionUDT-only
Answerhorizontal cell
Workflow
Retinal marker diagnosticRetinal marker validationRetinal depletion ranking
Total tokens
2.6M
Output
30.7K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT + standard tools
Answerhorizontal cell
Workflow
Retinal marker availability diagnosticUDT representation probeTp Text File With Recurring LinesNlScipy Sparse14 more tools
Total tokens
23.3M
Output
56.2K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData
Codex GPT-5.5

Replicate 2

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData
Codex GPT-5.5

Replicate 3

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData
Codex GPT-5.6 Sol

Replicate 2

RerunMatches prediction
Answerhorizontal cell
Workflow
Open-ended code
Total tokens
634.2K
Output
7.6K
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData1 more tools
Total tokens
968.0K
Output
20.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData1 more tools
Total tokens
1.3M
Output
24.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonScanpyAnnDataSciPy
Total tokens
685.2K
Output
11.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpyAnnData
Codex GPT-5.6 Luna

Replicate 2

RerunDiffers from prediction
Answerrod cell
Workflow
Open-ended code
Total tokens
1.3M
Output
20.7K
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPy

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answerhorizontal cell
Workflow
Local codePythonSciPyScanpy
Total tokens
192.9K
Output
2.4K