Back to results

CompBioBench item

contaminated-rna-q1

Question

You are given a single end RNA-seq fastq file (contaminated.rna.q1.fq.gz). This file contains mostly human reads and a small fraction of contaminant reads. Identify the contaminant species present. Report the answer as the exact scientific name of the species in all lowercase letters with a single space separating genus and species, for example "mus musculus".

Inferred answer
hydra vulgarisUniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
Answerhydra vulgaris
Workflow
Kraken2Summarize Kraken contaminant assignments
Codex GPT-5.5

Replicate 2

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2Krakentools Extract Kraken ReadsNcbi Blastn Wrapper
Total tokens
882.2K
Output
6.2K
Codex GPT-5.5

Replicate 3

RerunCorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2DatamashSort
Total tokens
1.4M
Output
11.4K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2Kraken2TaxDatamash OpsTp Sort Header ToolSelect columns6 more tools
Total tokens
66.6M
Output
61.8K
Codex GPT-5.6 Sol

Replicate 2

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2Grep
Total tokens
21.8M
Output
29.3K
Codex GPT-5.6 Sol

Replicate 3

CorrectStandard tools only
Answerhydra vulgaris
Workflow
FastqcKraken2
Total tokens
96.1M
Output
55.1K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2Krakentools Extract Kraken ReadsNcbi Blastn Wrapper
Total tokens
800.6K
Output
7.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

RerunCorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2GroupConvert Kraken
Total tokens
2.1M
Output
23.5K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Datamash OpsGroup and aggregateKraken2
Total tokens
916.4K
Output
9.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

RerunCorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2
Total tokens
3.8M
Output
15.6K
Codex GPT-5.6 Luna

Replicate 2

IncorrectStandard tools only
Answerhuman gammaherpesvirus 4
Workflow
Kraken2
Total tokens
3.2M
Output
17.7K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
Answerhydra vulgaris
Workflow
Kraken2Kraken2TaxDatamash OpsFilter TabularKrakentools Extract Kraken Reads
Total tokens
3.7M
Output
15.7K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answerhydra vulgaris
Workflow
Local codeBLASTNMinimap2Bowtie2Samtools1 more tools
Codex GPT-5.5

Replicate 2

Correct
Answerhydra vulgaris
Workflow
Local codeBLASTNKraken2Minimap2BWA-MEM2 more tools
Codex GPT-5.5

Replicate 3

Correct
Answerhydra vulgaris
Workflow
Local codeBLASTNKraken2Minimap2Bowtie22 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
Answerhydra vulgaris
Workflow
Local codeMinimap2Bowtie2SamtoolsBLASTN2 more tools
Codex GPT-5.6 Sol

Replicate 2

Correct
Answerhydra vulgaris
Workflow
Local codeBLASTNMinimap2Kraken2Bowtie2
Codex GPT-5.6 Sol

Replicate 3

Correct
Answerhydra vulgaris
Workflow
Local codeMinimap2SamtoolsKraken2BLASTN1 more tools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
Answerrattus norvegicus
Workflow
Local codeKraken2Bowtie2Minimap2BLASTN3 more tools
Total tokens
17.7M
Output
71.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answerhydra vulgaris
Workflow
Local codePythonKraken2BLASTNMinimap24 more tools
Total tokens
7.2M
Output
35.1K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
Answerhydra vulgaris
Workflow
Local codePythonKraken2Minimap2BWA-MEM3 more tools
Total tokens
5.0M
Output
32.9K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
Answerhydra vulgaris
Workflow
Local codePythonMinimap2BWA-MEMBowtie23 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
Answerhydra vulgaris
Workflow
Local codeKraken2Minimap2Bowtie2BWA-MEM3 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
Answerhydra vulgaris
Workflow
Local codeMinimap2Bowtie2BWA-MEMSamtools4 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
Answerhydra vulgaris
Workflow
Local codePythonBLASTNMinimap2Kraken2
Total tokens
677.4K
Output
6.9K