Back to results

CompBioBench item

calculate-average-gene-expression-q1

Question

Using the single-cell dataset in AnnData format located at pbmc3k.h5ad, determine the average log-normalized expression for the gene S100B in all cells belonging to the group labeled 'B cells' in the 'louvain' metadata column. Use adata.X for already log-normalized expression. Use two decimal places in the answer.

Predicted answer
-0.14Most likely answer for this question; not uniquely determined from submitted scores.
Domain
Single-cell
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches predictionUDT + standard tools
Answer-0.14
Workflow
Average S100B in B cellsH5AD path render probeTXT path render probe
Codex GPT-5.5

Replicate 2

Matches predictionUDT + standard tools
Answer-0.14
Workflow
Anndata ExportS100B B cells mean
Total tokens
852.7K
Output
6.1K
Codex GPT-5.5

Replicate 3

Matches predictionUDT + standard tools
Answer-0.14
Workflow
AnnDataS100B UDT render probeAnndata ExportFilter TabularPaste12 more tools
Total tokens
4.1M
Output
16.5K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches predictionStandard tools only
Answer-0.14
Workflow
Scanpy FilterAnndata InspectDatamash Ops
Codex GPT-5.6 Sol

Replicate 2

Matches predictionUDT-only
Answer-0.14
Workflow
PBMC3K S100B input diagnosticPBMC3K mean S100B expression in B cells
Total tokens
978.4K
Output
5.1K
Codex GPT-5.6 Sol

Replicate 3

Matches predictionUDT-only
Answer-0.14
Workflow
PBMC3K S100B input diagnosticPBMC3K S100B mean in B cells
Total tokens
1.2M
Output
5.7K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches predictionStandard tools only
Answer-0.14
Workflow
No Galaxy tool job recorded
Total tokens
3.5M
Output
24.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches predictionStandard tools only
Answer-0.14
Workflow
No Galaxy tool job recorded
Total tokens
4.5M
Output
32.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches predictionStandard tools only
Answer-0.14
Workflow
Anndata InspectScanpy Filter CellsScanpy FilterAnndata ExportDatamash Ops
Total tokens
3.8M
Output
15.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches predictionUDT + standard tools
Answer-0.14
Workflow
PBMC3K S100B diagnosticPBMC3K UDT render probeAnndata ExportScanpy FilterDatamash Ops
Total tokens
11.6M
Output
35.0K
Codex GPT-5.6 Luna

Replicate 2

Matches predictionStandard tools only
Answer-0.14
Workflow
Anndata ExportFilter TabularNlSelect columnsGrep3 more tools
Total tokens
7.3M
Output
22.0K
Codex GPT-5.6 Luna

Replicate 3

Matches predictionUDT + standard tools
Answer-0.14
Workflow
Anndata InspectS100B mean in B cellsUDT input path probeUDT AnnData path probeUDT command probe6 more tools
Total tokens
8.4M
Output
33.9K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData
Codex GPT-5.5

Replicate 2

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataScanpySciPy
Codex GPT-5.5

Replicate 3

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataSciPy
Codex GPT-5.6 Sol

Replicate 2

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataSciPy
Codex GPT-5.6 Sol

Replicate 3

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataScanpy
Total tokens
79.9K
Output
1.5K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataScanpy
Total tokens
68.1K
Output
1.1K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData
Total tokens
67.6K
Output
1.2K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData
Codex GPT-5.6 Luna

Replicate 2

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData
Codex GPT-5.6 Luna

Replicate 3

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnDataScanpySciPy

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Matches prediction
Answer-0.14
Workflow
Local codePythonAnnData
Total tokens
181.9K
Output
1.2K