Back to results

CompBioBench item

exogenous-mix-reads-q2

Question

I have FASTQ data from a perturbation experiment involving KLF4 exogenous overexpression using a Sendai virus. I want to estimate the fraction of exogenous-origin reads in exogenous.mix.reads.q2.mix.fq, given exogenous.mix.reads.q2.exo.fq (early timepoint, assumed purely exogenous) and exogenous.mix.reads.q2.endo.fq (later timepoint, predominantly endogenous). All files contain reads mapping to KLF4. I don't have details on the Sendai construct. If possible, estimate the percent of exogenous reads in exogenous.mix.reads.q2.mix.fq (by read count) to the nearest 10 and return only the integer (e.g. 50); if not possible, return NA.

Inferred answer
NAUniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

IncorrectUDT-only
Answer90
Workflow
KLF4 exogenous diagnosticKLF4 exogenous diagnostic inlineKLF4 FASTQ overlap order diagnosticKLF4 segment private kmer diagnosticKLF4 final exogenous percent
Total tokens
792.1K
Output
16.4K
Codex GPT-5.5

Replicate 2

IncorrectUDT + standard tools
Answer90
Workflow
KLF4 exogenous mixture estimateKLF4 exogenous mixture diagnostics v2KLF4 exogenous mixture diagnostics v3UDT render probe v1UDT render probe v26 more tools
Total tokens
3.6M
Output
18.0K
Codex GPT-5.5

Replicate 3

IncorrectUDT + standard tools
Answer90
Workflow
KLF4 Origin DiagnosticKLF4 Origin Diagnostic v2KLF4 Origin Diagnostic v3KLF4 Render ProbeKLF4 Render Probe 28 more tools
Total tokens
4.4M
Output
24.3K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

IncorrectUDT-only
Answer90
Workflow
KLF4 FASTQ source diagnosticKLF4 exogenous read fraction estimator
Total tokens
432.5K
Output
7.9K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT-only
AnswerNA
Workflow
KLF4 read signature diagnosticKLF4 read nesting diagnosticKLF4 exogenous fraction final result
Total tokens
1.1M
Output
9.4K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT-only
AnswerNA
Workflow
KLF4 FASTQ sequence diagnosticKLF4 FASTQ file relation
Total tokens
536.2K
Output
6.3K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

IncorrectStandard tools only
Answer90
Workflow
Fastq To TabularSelect columnsWc GnuComp1Tp Sorted Uniq2 more tools
Total tokens
16.2M
Output
31.6K
Codex + DeepSeek-v4-pro-0813

Replicate 2

CorrectUDT + standard tools
AnswerNA
Workflow
Cshl Fastq To FastaCshl Fastx CollapserKLF4 mix k-mer diagnosticProbe MinimalMeryl Count Kmers11 more tools
Total tokens
27.7M
Output
102.6K
Codex + DeepSeek-v4-pro-0813

Replicate 3

CorrectUDT + standard tools
AnswerNA
Workflow
Cshl Fastx CollapserRead overlap diagnosticFastq To TabularGroup and aggregateSummary Statistics11 more tools
Total tokens
14.6M
Output
76.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectUDT + standard tools
AnswerNA
Workflow
Fastq To TabularKLF4 origin sequence diagnosticsKLF4 origin sequence profileKLF4 origin sequence holdout testKLF4 origin overlap diagnostic3 more tools
Total tokens
3.2M
Output
29.5K
Codex GPT-5.6 Luna

Replicate 2

CorrectUDT + standard tools
AnswerNA
Workflow
Cshl Fastx CollapserFastq StatsRnaspadesKLF4 origin diagnosticKLF4 common-reference diagnostic6 more tools
Codex GPT-5.6 Luna

Replicate 3

CorrectStandard tools only
AnswerNA
Workflow
Fastq StatsFastq To TabularComp1Datasets Download GeneMinimap26 more tools
Total tokens
21.4M
Output
62.3K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerNA
Workflow
Local codePython
Codex GPT-5.5

Replicate 2

Incorrect
Answer90
Workflow
Local codePython
Codex GPT-5.5

Replicate 3

Incorrect
Answer90
Workflow
Local code

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Incorrect
Answer40
Workflow
Local code
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerNA
Workflow
Local codeMinimap2BWA-MEMBowtie2BLASTN1 more tools
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerNA
Workflow
Local codeMinimap2Bowtie2BWA-MEMBLASTN1 more tools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerNA
Workflow
Local codePythonSciPy
Total tokens
5.9M
Output
82.8K
Codex + DeepSeek-v4-pro-0813

Replicate 2

RerunCorrect
AnswerNA
Workflow
Open-ended code
Total tokens
2.0M
Output
57.9K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerNA
Workflow
Local codePythonBLASTNSamtoolsBWA-MEM3 more tools
Total tokens
2.5M
Output
52.3K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerNA
Workflow
Local codeMinimap2Bowtie2SamtoolsBLASTN1 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerNA
Workflow
Local codeMinimap2Bowtie2BWA-MEMBLASTN5 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerNA
Workflow
Local codeMinimap2BWA-MEMBowtie2Samtools5 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerNA
Workflow
Local codePython
Total tokens
82.4K
Output
934