Back to results

CompBioBench item

deg-simple-q2

Question

I have bulk RNA-seq fastq files from 2 human samples (deg.simple.q2.sampleA.fq.gz, deg.simple.q2.sampleB.fq.gz). There is exactly one gene that has differentially expressed isoforms in the 2 samples. Report the HGNC gene symbol (uppercase) for that gene.

Inferred answer
DEFB110Uniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Routine Analysis

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
AnswerDEFB110
Workflow
SalmonIsoform switch Salmon analysisSalmon runtime probeSalmon runtime probe compilerPython Salmon runtime probe6 more tools
Codex GPT-5.5

Replicate 2

CorrectUDT-only
AnswerDEFB110
Workflow
FASTQ diagnostic sampleRuntime tool probeIsoform usage by GENCODE transcript alignment
Total tokens
621.2K
Output
9.6K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerDEFB110
Workflow
FASTQ inspect q2FASTQ inspect q2 v2SalmonFastq StatsGrep
Total tokens
7.7M
Output
27.1K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT + standard tools
AnswerDEFB110
Workflow
FastqcSalmonRank differential isoform usageRna StarRank splice junction usage changes
Total tokens
18.9M
Output
25.4K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
AnswerDEFB110
Workflow
FastqcSalmonKallisto QuantCompare isoform usage GENCODE v50Rna Star4 more tools
Total tokens
98.8M
Output
46.7K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
AnswerDEFB110
Workflow
FastqcSalmonDatamash OpsSalmon quantification diagnosticsRank human differential isoform usage2 more tools
Total tokens
7.9M
Output
24.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectUDT + standard tools
AnswerDEFB110
Workflow
SalmonConverter Gz To UncompressedTp Sort Header ToolGrepIsoform Switch Analysis1 more tools
Total tokens
23.5M
Output
62.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT + standard tools
AnswerDEFB110
Workflow
SalmonIsoformswitchanalyzerIsoform Switch Two SampleTp Head Tool
Total tokens
8.6M
Output
40.7K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
AnswerDEFB110
Workflow
SalmonIsoformswitchanalyzerIsoform switch scoreFilter Tabular
Total tokens
25.3M
Output
71.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

IncorrectUDT + standard tools
AnswerMS4A1
Workflow
SalmonUpload1Converter Gz To UncompressedHisat2Samtools Flagstat9 more tools
Total tokens
19.9M
Output
70.8K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
AnswerDEFB110
Workflow
SalmonIsoformswitchanalyzerTwo-sample isoform usage effect-size analysisSupported two-sample isoform usage effect-size analysisCat15 more tools
Total tokens
10.9M
Output
68.2K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT + standard tools
AnswerDEFB110
Workflow
FastqcSalmonConverter Gz To UncompressedKallisto QuantGENCODE within-gene isoform usage comparison2 more tools

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerDEFB110
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsPython
Codex GPT-5.5

Replicate 3

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsPython

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerDEFB110
Workflow
Local codeMinimap2SamtoolsPython
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsPython
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsPython

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerDEFB110
Workflow
Local codePythonSamtoolsBowtie2BWA-MEM4 more tools
Total tokens
2.3M
Output
21.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2BWA-MEMSamtools2 more tools
Total tokens
3.1M
Output
22.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsBedtools1 more tools
Total tokens
3.5M
Output
24.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerDEFB110
Workflow
Local codeMinimap2Bowtie2SamtoolsPython1 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerDEFB110
Workflow
Local codePythonMinimap2BWA-MEMSamtools3 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerDEFB110
Workflow
Local codeBowtie2Minimap2SamtoolsPython

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerDEFB110
Workflow
Local codePythonMinimap2SciPypysam
Total tokens
846.9K
Output
5.0K