Back to results

CompBioBench item

tissue-fibroblast-q1

Question

I have a counts matrix of transcriptomic reads from a murine multi-tissue fibroblast atlas, tissue.fibroblast.q1.rds. I want to know what tissue the fibroblast corresponding to barcode Cell_11249_MCI is from, and what tissue the fibroblast corresponding to the barcode Cell_10369_WXV is. Please provide an answer in a string format with each answer separated by a comma, and select each answer from "Lymph node", "Pancreas", "Muscle", "Tendon", "Mesentery", "Omentum" , 'Adipose", "Artery", "Bone", "Heart", "Intestine", "Skin", "Lung", "Liver", or "Spleen". Expected output format example: Pancreas,Omentum.

Inferred answer
Lung,BoneUniquely inferred from submitted scores.
Domain
Single-cell
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

IncorrectUDT-only
AnswerOmentum,Bone
Workflow
Lookup fibroblast tissue from RDSInspect RDS structure for barcode metadataTarget barcode marker scoresInspect reference fibroblast RDSInspect reference fibroblast Seurat RDS2 more tools
Codex GPT-5.5

Replicate 2

IncorrectUDT-only
AnswerOmentum,Bone
Workflow
RDS tissue lookup diagnosticRDS marker diagnosticRDS source match tissueRDS source match tissue SeuratRDS source match tissue Seurat v23 more tools
Total tokens
3.5M
Output
21.0K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerLung,Bone
Workflow
Inspect fibroblast RDS metadataInspect fibroblast RDS metadata v2Inspect fibroblast RDS metadata v3UDT probe cat
Total tokens
6.6M
Output
28.6K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT-only
AnswerLung,Bone
Workflow
Inspect fibroblast RDS structureInspect fibroblast barcode conventionsInspect fibroblast target barcodesInspect authoritative fibroblast atlas
Total tokens
22.6M
Output
21.4K
Codex GPT-5.6 Sol

Replicate 2

IncorrectUDT-only
AnswerOmentum,Bone
Workflow
Inspect fibroblast atlas RDS structureInspect target fibroblast cellsUpload1Inspect authoritative steady-state Seurat objectMatch fibroblast cells to authoritative tissue labels
Total tokens
4.4M
Output
11.9K
Codex GPT-5.6 Sol

Replicate 3

IncorrectUDT-only
AnswerOmentum,Bone
Workflow
Inspect RDS structure for fibroblast atlasInspect target fibroblast barcodes and suffix groupsInspect authoritative fibroblast source atlasInspect authoritative fibroblast source atlas with SeuratExact match fibroblast tissues to authoritative atlas1 more tools
Total tokens
35.5M
Output
27.4K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

IncorrectUDT + standard tools
AnswerOmentum,Bone
Workflow
Rds To TabularSeurat DataInspect RDS StructureMatch Q1 Reference DiagnosticSeurat Export Cellbrowser3 more tools
Total tokens
10.5M
Output
42.3K
Codex + DeepSeek-v4-pro-0813

Replicate 2

IncorrectUDT + standard tools
AnswerOmentum,Bone
Workflow
Rds To TabularInspect RDSInspect MatrixInspect AttrsInspect Seurat Min3 more tools
Total tokens
15.4M
Output
40.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunCorrectUDT + standard tools
AnswerLung,Bone
Workflow
RDS to tabularSCEasy convertInspect RDSInspect ColnamesProfile Cells1 more tools
Total tokens
5.1M
Output
57.8K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunCorrectUDT + standard tools
AnswerLung,Bone
Workflow
RDS to tabularRDS structure inspectionTarget cell expression profilesExcel to TabularRDS tissue marker scoring4 more tools
Total tokens
45.6M
Output
61.0K
Codex GPT-5.6 Luna

Replicate 2

RerunIncorrectUDT-only
AnswerLung,Lymph node
Workflow
User-defined tool
Total tokens
20.9M
Output
65.8K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT + standard tools
AnswerLung,Bone
Workflow
RDS Structure ReportInspect Target CellsTarget Expression NeighborsScore Tissue Marker PanelsXlsx2Tsv3 more tools
Total tokens
20.2M
Output
64.7K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPythonSciPy
Codex GPT-5.5

Replicate 2

Incorrect
AnswerOmentum,Bone
Workflow
Local codeR
Codex GPT-5.5

Replicate 3

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPython
Codex GPT-5.6 Sol

Replicate 2

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPythonSciPyScanpy1 more tools
Codex GPT-5.6 Sol

Replicate 3

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPython
Total tokens
2.2M
Output
8.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPython
Total tokens
6.2M
Output
19.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPythonAnnDataScanpy
Total tokens
3.0M
Output
14.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Incorrect
AnswerOmentum,Bone
Workflow
Local codeRPythonSciPyAnnData1 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerLung,Bone
Workflow
Local codeRPythonSciPyAnnData1 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerLung,Bone
Workflow
Local codeRPython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerLung,Bone
Workflow
Local codeRPython
Total tokens
143.1K
Output
960