Back to results

CompBioBench item

characterize-response-q1

Question

characterize.response.q1.txt contains an ordered list of differentially regulated genes in one direction (decreasing abs log-fold change, all sharing the same sign) in a specific immune cell type in response to a specific condition. Which condition, cell type pair from the following options most specifically describes this gene set. Conditions: a. stimulation by lipopolysaccharide (LPS), b. exhaustion, c. stimulation by cytokine LIF, d. stimulation by cytokine IL-39, e. COVID-19, f. sepsis, g. Med12 knockout, h. systemic lupus erythematosus, i. treatment with BMS-34554, j. treatment with aldesleukin, k. aging, l. rheumatoid arthritis. Cell types: i. monocytes, ii. CD8 T cells, iii. CD4 T cells, iv. B cells, v. dendritic cells, vi. NK cells, vii. Neutrophils. Respond in the format d;ii.

Inferred answer
c;vUniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Metadata Recovery

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
Answerc;v
Workflow
Show Beginning1Gene set overlap check CUI CDC2 LIFGene set overlap check CUI CDC2 LIF v2
Codex GPT-5.5

Replicate 2

CorrectStandard tools only
Answerc;v
Workflow
Show Beginning1
Total tokens
338.2K
Output
4.6K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
Answerc;v
Workflow
Characterize Q1 MSigDB CheckUDT Path Syntax ProbeUDT Path Syntax Probe QualifiedUDT Path Syntax Probe Project ImageComp1
Total tokens
1.6M
Output
11.7K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answerc;v
Workflow
Gseapy EnrichrTp Head ToolDatamash TransposeComp1
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
Answerc;v
Workflow
Tp Head ToolShow Tail1Gseapy EnrichrInspect Immune Dictionary XLSXMatch Immune Dictionary Signature
Total tokens
4.7M
Output
14.0K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
Answerc;v
Workflow
Inspect ranked genes Q1Xlsx2TsvMatch ranked cytokine signatureCat Multi DatasetsMatch ranked cytokine signature all candidate cells1 more tools
Total tokens
16.4M
Output
20.2K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

IncorrectStandard tools only
Answera;i
Workflow
Gseapy Enrichr
Total tokens
14.0M
Output
50.1K
Codex + DeepSeek-v4-pro-0813

Replicate 2

IncorrectStandard tools only
Answera;i
Workflow
Gseapy Enrichr
Total tokens
5.3M
Output
32.0K
Codex + DeepSeek-v4-pro-0813

Replicate 3

IncorrectStandard tools only
Answera;i
Workflow
Gseapy Enrichr
Total tokens
13.8M
Output
82.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

IncorrectStandard tools only
Answera;i
Workflow
Show Beginning1Gseapy EnrichrFilter Tabular
Total tokens
10.4M
Output
49.3K
Codex GPT-5.6 Luna

Replicate 2

IncorrectStandard tools only
Answera;i
Workflow
Tp Head ToolHgv LinktogprofileGseapy EnrichrFilter Tabular
Codex GPT-5.6 Luna

Replicate 3

IncorrectStandard tools only
Answerf;iii
Workflow
Select columnsGprofiler GostGseapy EnrichrFilter Tabular
Total tokens
12.7M
Output
55.1K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answerc;v
Workflow
Local code
Codex GPT-5.5

Replicate 2

Correct
Answerc;v
Workflow
Local code
Codex GPT-5.5

Replicate 3

Correct
Answerc;v
Workflow
Local code

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answerc;v
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Correct
Answerc;v
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 3

Correct
Answerc;v
Workflow
Local codePython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
Answera;v
Workflow
Local codePythonSciPy
Total tokens
54.2M
Output
170.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Incorrect
Answera;i
Workflow
Local codePythonRSciPy
Total tokens
33.5M
Output
118.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Incorrect
Answera;v
Workflow
Local codePythonSciPy
Total tokens
30.6M
Output
120.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
Answerc;v
Workflow
Local code
Codex GPT-5.6 Luna

Replicate 2

Correct
Answerc;v
Workflow
Local codePython
Codex GPT-5.6 Luna

Replicate 3

Correct
Answerc;v
Workflow
Local codePython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answerc;v
Workflow
Local codePython
Total tokens
679.1K
Output
3.7K