Back to results

CompBioBench item

spatial-sim-q2

Question

You are given a set of single cell transcriptomic data and Visium spatial transcriptomic data (spatial.sim.tar.gz). The single cell counts matrix is found at spatial_q_sc_counts.csv, and accompanying cell type metadata is found at spatial_q_sc_metadata.csv. Spatial count data is found at spatial_q_matrix.mtx.gz, gene counts are found at spatial_q_features.tsv.gz, and are found at spot barcodes spatial_q_barcodes.tsv.gz. Finally scale factors are found at spatial_q_scalefactors_json.json, high and low resolution tissue images are found at spatial_q_tissue_hires_image.png and patial_q_tissue_hires_image.png respectively, and tissue positions are found at spatial_q_tissue_positions.csv. The spot-based data is comprised entirely of cell types found in the single cell data. Can you tell me what cell type or cell types, if any, are biased towards being in closer spatial proximity to spots containing Tumor_Core? If none, please return "None". Please list these separated by commas in alphabetical order. Expected answer format: Endothelial,Macrophage.

Inferred answer
Fibroblast_Stroma,MacrophageUniquely inferred from submitted scores.
Domain
Spatial
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial Tumor Proximity Analysis
Codex GPT-5.5

Replicate 2

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial proximity diagnosticSpatial Tumor_Core proximity analysis
Total tokens
467.9K
Output
12.1K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial diagnosticsSpatial diagnostics v2UDT render probeUnzipConverter Gz To Uncompressed12 more tools
Total tokens
11.8M
Output
25.8K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial input diagnosticValidated spatial deconvolutionTumor Core spatial proximity test
Total tokens
898.0K
Output
14.6K
Codex GPT-5.6 Sol

Replicate 2

IncorrectUDT + standard tools
AnswerMacrophage
Workflow
Spatial archive diagnosticsR runtime probeR BioContainer runtime probeAnalysisUnzip24 more tools
Total tokens
41.6M
Output
55.0K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial proximity input diagnosticSpatial NNLS deconvolution diagnosticSpatial Tumor Core proximity analysis
Total tokens
784.9K
Output
10.6K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Converter Archive To DirectorySpatial tumor proximity NNLS
Total tokens
5.7M
Output
56.2K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Galaxy UDTSpatial Tumor Proximity Deconvolution
Total tokens
4.7M
Output
45.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
AnswerFibroblast_Stroma,Macrophage
Workflow
SciPySpatial DiagnosticConverter Archive To DirectorySpatial Probe ReqSpatial Probe Large18 more tools
Total tokens
42.6M
Output
90.6K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial archive diagnosticSpatial label transfer and proximity exploration
Total tokens
983.4K
Output
20.0K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
AnswerFibroblast_Stroma,Macrophage
Workflow
Spatial archive inspectionSpatial UDT rendering probeSpatial UDT rendering probe legacySpatial UDT rendering probe formatSpatial UDT script probe17 more tools
Total tokens
17.6M
Output
60.2K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT-only
AnswerFibroblast_Stroma,Macrophage
Workflow
Converter Archive To DirectorySpatial input inspectionSpatial archive inspectionSpatial input characterizationSpatial neighborhood exploration2 more tools
Total tokens
2.4M
Output
28.0K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.5

Replicate 2

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.5

Replicate 3

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

RerunCorrect
AnswerFibroblast_Stroma,Macrophage
Workflow
Open-ended code
Total tokens
2.2M
Output
34.3K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPyScanpyAnnData
Total tokens
717.8K
Output
18.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPyScanpy
Total tokens
3.6M
Output
49.8K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerFibroblast_Stroma,Macrophage
Workflow
Local codePythonSciPy
Total tokens
372.6K
Output
2.9K