Back to results

CompBioBench item

gene-pair-ordering-fraction-q1

Question

From the dataset at 10x_pbmc68k_reduced.h5ad, consider only CD19+ B cells with percent_mito less than 0.04. Among these, compute the number of cells where expression of gene SRM exceeds that of gene MRPS21 by more than 1.0 (n_srm_higher), the number of cells where expression of gene MRPS21 exceeds that of gene SRM by more than 1.0 (n_mrps21_higher), and the number of cells where the difference in expression of the two genes is between 1 and -1 (inclusive) (n_within_range). Round all fractions to two decimal places. Return these numbers as a semicolon-separated string in the format n_srm_higher,n_mrps21_higher,n_within_range. An example of a correctly formatted string is: "50;30;20".

Inferred answer
37;20;37Uniquely inferred from submitted scores.
Domain
Single-cell
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT-only
Answer37;20;37
Workflow
PBMC SRM MRPS21 counts
Total tokens
229.1K
Output
3.3K
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
Answer37;20;37
Workflow
Inspect PBMC h5ad for SRM MRPS21 taskInspect PBMC h5ad SRM MRPS21 v2Anndata ExportScanpy FilterSelect columns2 more tools
Total tokens
5.5M
Output
19.1K
Codex GPT-5.5

Replicate 3

RerunCorrectUDT-only
Answer37;20;37
Workflow
H5AD PBMC diagnosticPBMC SRM MRPS21 counts
Total tokens
354.8K
Output
4.6K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answer37;20;37
Workflow
Anndata ExportScanpy FilterAdd A Column1Filter rowsWc Gnu1 more tools
Total tokens
4.3M
Output
7.6K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
Answer37;20;37
Workflow
Anndata InspectGrepPBMC SRM versus MRPS21 threshold countsPBMC target-gene expression source diagnostic
Total tokens
2.3M
Output
11.2K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
Answer37;20;37
Workflow
Anndata InspectInspect PBMC H5AD metadata and target expressionCompare PBMC X and raw target expressionFinal PBMC SRM MRPS21 counts
Total tokens
1.9M
Output
8.7K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
Answer37;20;37
Workflow
Scanpy FilterAnndata ExportSelect columnsAdd A Column1
Total tokens
4.7M
Output
23.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
Answer37;20;37
Workflow
Anndata InspectScanpy FilterAnndata ExportAdd A Column1Select columns1 more tools
Total tokens
8.9M
Output
26.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
Answer37;20;37
Workflow
Anndata InspectAnndata ExportScanpy FilterAdd A Column1
Total tokens
7.3M
Output
18.7K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT + standard tools
Answer37;20;37
Workflow
Anndata InspectScanpy Filterh5ad runtime probeh5ad runtime version probeh5ad runtime command probe4 more tools
Total tokens
10.5M
Output
32.3K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
Answer37;20;37
Workflow
Anndata ExportFilter TabularGalaxy UDTtest_toolTest Tool4 more tools
Total tokens
20.9M
Output
48.2K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT + standard tools
Answer37;20;37
Workflow
Scanpy FilterAnndata ExportFilter TabularPBMC Scanpy runtime probePBMC SRM versus MRPS21 difference counts
Total tokens
5.6M
Output
19.9K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answer37;20;37
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataScanpySciPy
Codex GPT-5.5

Replicate 3

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataScanpySciPy

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataSciPy
Codex GPT-5.6 Sol

Replicate 2

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataScanpy
Codex GPT-5.6 Sol

Replicate 3

Correct
Answer37;20;37
Workflow
Local codePythonAnnData

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataScanpy
Total tokens
101.4K
Output
3.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Incorrect
Answer29;23;42
Workflow
Local codePythonAnnDataScanpySciPy
Total tokens
6.4M
Output
34.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
Answer37;20;37
Workflow
Local codePythonScanpyAnnData
Total tokens
5.6M
Output
45.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataScanpySciPy
Codex GPT-5.6 Luna

Replicate 2

Correct
Answer37;20;37
Workflow
Local codePythonSciPyAnnData
Codex GPT-5.6 Luna

Replicate 3

Correct
Answer37;20;37
Workflow
Local codePythonAnnDataSciPy

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answer37;20;37
Workflow
Local codePythonAnnData
Total tokens
147.4K
Output
1.1K