Back to results

CompBioBench item

contaminated-rna-q3

Question

You are given a single-end RNA-seq FASTQ file (contaminated.rna.q3.fq.gz). The sample is expected to be human, but it may contain reads from another organism. Determine the genus of the most likely non-human organism present, if any. Report your answer as the exact scientific genus name in all lowercase letters (for example, mus), else respond with "none".

Inferred answer
panUniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
Answerpan
Workflow
Seqtk SampleSeqtk SeqNcbi Blastn WrapperSummarize BLAST GeneraKraken25 more tools
Codex GPT-5.5

Replicate 2

CorrectUDT + standard tools
Answerpan
Workflow
Extract Kraken Pan validation readsExtract Kraken Pan validation reads v2
Total tokens
2.1M
Output
14.6K
Codex GPT-5.5

Replicate 3

RerunCorrectStandard tools only
Answerpan
Workflow
Kraken2Krakentools: Extract Kraken Reads By IDFASTQ Summary StatisticsNCBI BLAST+ blastn
Total tokens
1.4M
Output
9.5K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectStandard tools only
Answerpan
Workflow
FastqcKraken2GrepTp Sort Header ToolHisat2
Total tokens
10.0M
Output
11.1K
Codex GPT-5.6 Sol

Replicate 2

IncorrectUDT + standard tools
Answerrattus
Workflow
Kraken2Bowtie2Fastq To Fasta PythonSubset FASTA q3Hisat22 more tools
Total tokens
99.5M
Output
67.6K
Codex GPT-5.6 Sol

Replicate 3

RerunCorrectStandard tools only
Answerpan
Workflow
Kraken2SelectSort
Total tokens
26.2M
Output
34.7K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
Answerpan
Workflow
Kraken2
Total tokens
644.0K
Output
9.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

IncorrectStandard tools only
Answermycoplasma
Workflow
No Galaxy tool job recorded
Total tokens
28.7M
Output
71.4K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectStandard tools only
Answerpan
Workflow
Kraken2Krakentools Extract Kraken ReadsFastq To Fasta PythonCshl Fastq To FastaSeqtk Seq1 more tools
Total tokens
2.0M
Output
14.7K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
Answerpan
Workflow
Kraken2Kraken2TaxDatamash OpsFilter TabularTp Grep Tool
Total tokens
3.4M
Output
13.3K
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
Answerpan
Workflow
Fastq StatsKraken2Sylph ProfileKraken2TaxFilter Tabular1 more tools
Total tokens
5.7M
Output
24.7K
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
Answerpan
Workflow
Kraken2Group and aggregateCount1Kraken2TaxFilter rows
Total tokens
3.3M
Output
10.1K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
Answerpan
Workflow
Local codeKraken2BLASTNMinimap2Bowtie21 more tools
Codex GPT-5.5

Replicate 2

Correct
Answerpan
Workflow
Local codeKraken2Minimap2BLASTNBWA-MEM1 more tools
Codex GPT-5.5

Replicate 3

Correct
Answerpan
Workflow
Local codeBLASTNKraken2Minimap2Bowtie22 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Incorrect
Answermesomycoplasma
Workflow
Local codeKraken2Minimap2BLASTNBowtie22 more tools
Codex GPT-5.6 Sol

Replicate 2

Correct
Answerpan
Workflow
Local codeKraken2Minimap2BLASTNBWA-MEM2 more tools
Codex GPT-5.6 Sol

Replicate 3

Correct
Answerpan
Workflow
Local codeKraken2Minimap2BLASTNSamtools1 more tools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Incorrect
Answermesomycoplasma
Workflow
Local codeBLASTNKraken2Minimap2BWA-MEM2 more tools
Total tokens
8.8M
Output
46.9K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
Answerpan
Workflow
Local codePythonKraken2BLASTNSamtools3 more tools
Total tokens
20.2M
Output
80.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Incorrect
Answermesomycoplasma
Workflow
Local codeKraken2Minimap2Bowtie2BWA-MEM3 more tools
Total tokens
14.1M
Output
61.0K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Incorrect
Answermesomycoplasma
Workflow
Local codeMinimap2Bowtie2BLASTNKraken21 more tools
Codex GPT-5.6 Luna

Replicate 2

Incorrect
Answermesomycoplasma
Workflow
Local codeMinimap2Bowtie2BWA-MEMBLASTN2 more tools
Codex GPT-5.6 Luna

Replicate 3

Incorrect
Answermesomycoplasma
Workflow
Local codeMinimap2Bowtie2BWA-MEMBLASTN3 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Incorrect
Answermesomycoplasma
Workflow
Local codePythonBLASTNMinimap2Bowtie22 more tools
Total tokens
1.0M
Output
7.7K