Back to results

CompBioBench item

contaminated-rna-q2

Question

You are given a single end RNA-seq fastq file (contaminated.rna.q2.fq.gz). This file contains mostly human reads and a small fraction of contaminant reads. Identify the contaminant species present. Report the answer as the exact scientific name of the species in all lowercase letters with a single space separating genus and species, for example "mus musculus".

Inferred answer
sus scrofaUniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
Answersus scrofa
Workflow
Kraken2Kraken taxon countsHisat2Kraken taxon counts filtered
Total tokens
1.6M
Output
13.1K
Codex GPT-5.5

Replicate 2

IncorrectStandard tools only
Answerlymphocryptovirus humangamma4
Workflow
Kraken2Krakentools Extract Kraken ReadsNcbi Blastn Wrapper
Total tokens
1.1M
Output
11.3K
Codex GPT-5.5

Replicate 3

RerunCorrectUDT + standard tools
Answersus scrofa
Workflow
Kraken2FASTQ header diagnosticRemote BLAST sampled FASTQNCBI URL BLAST sampled FASTQNCBI URL BLAST sampled FASTQ small10 more tools
Total tokens
4.3M
Output
38.3K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answersus scrofa
Workflow
Kraken2Filter TabularGrepTp Sort Header ToolSort rows1 more tools
Total tokens
36.6M
Output
51.1K
Codex GPT-5.6 Sol

Replicate 2

IncorrectStandard tools only
Answerlymphocryptovirus humangamma4
Workflow
Kraken2Filter rowsSort rowsTp Head ToolHisat21 more tools
Total tokens
1.6M
Output
7.4K
Codex GPT-5.6 Sol

Replicate 3

CorrectStandard tools only
Answersus scrofa
Workflow
Kraken2Filter TabularFilter rowsSort rowsShow Beginning14 more tools
Total tokens
52.8M
Output
32.6K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

RerunCorrectStandard tools only
Answersus scrofa
Workflow
kraken2 → hisat2 → cshl fastq to fasta → ncbi blastn wrapper → samtools flagstat
Total tokens
7.8M
Output
36.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectStandard tools only
Answersus scrofa
Workflow
Kraken2Datamash OpsGroup and aggregateCount1Sylph Profile
Total tokens
43.2M
Output
87.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
Answersus scrofa
Workflow
Kraken2
Total tokens
48.5M
Output
82.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
Answersus scrofa
Workflow
Seqtk SampleKraken2Bamtools Split MappedBedtools BamtofastqFastq To Fasta Python2 more tools
Total tokens
39.8M
Output
113.4K
Codex GPT-5.6 Luna

Replicate 2

IncorrectStandard tools only
Answerlymphocryptovirus humangamma4
Workflow
Fastq StatsSeqtk SampleKraken2Tp Grep ToolKrakentools Extract Kraken Reads
Total tokens
5.3M
Output
27.1K
Codex GPT-5.6 Luna

Replicate 3

IncorrectStandard tools only
Answerhuman gammaherpesvirus 4
Workflow
Fastq StatsFastqcKraken2
Total tokens
6.2M
Output
26.0K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Incorrect
Answermus musculus
Workflow
Local codeBLASTNMinimap2Bowtie2BWA-MEM3 more tools
Codex GPT-5.5

Replicate 2

Correct
Answersus scrofa
Workflow
Local codeMinimap2Kraken2BLASTNPython3 more tools
Codex GPT-5.5

Replicate 3

Incorrect
Answerrattus norvegicus
Workflow
Local codeBLASTNMinimap2Bowtie2Kraken2

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answersus scrofa
Workflow
Local codeMinimap2Bowtie2BLASTNKraken22 more tools
Codex GPT-5.6 Sol

Replicate 2

Correct
Answersus scrofa
Workflow
Local codeMinimap2Bowtie2Kraken2BLASTN2 more tools
Codex GPT-5.6 Sol

Replicate 3

Correct
Answersus scrofa
Workflow
Local codeMinimap2Bowtie2BLASTNKraken21 more tools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
Answerlymphocryptovirus humangamma4
Workflow
Local codeKraken2BLASTNMinimap2Samtools3 more tools
Total tokens
19.1M
Output
61.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

RerunIncorrect
Answerlymphocryptovirus humangamma4
Workflow
Open-ended code
Total tokens
4.8M
Output
36.6K
Codex + DeepSeek-v4-pro-0813

Replicate 3

RerunCorrect
Answersus scrofa
Workflow
Open-ended code
Total tokens
9.3M
Output
59.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Incorrect
Answerlymphocryptovirus humangamma4
Workflow
Local codeMinimap2Kraken2BLASTNSamtools2 more tools
Codex GPT-5.6 Luna

Replicate 2

Incorrect
Answerhuman gammaherpesvirus 4
Workflow
Local codeMinimap2BWA-MEMBowtie2Samtools3 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
Answersus scrofa
Workflow
Local codeMinimap2Bowtie2Kraken2BLASTN4 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answersus scrofa
Workflow
Local codePythonBLASTNMinimap2Samtools
Total tokens
942.6K
Output
8.0K