Back to results

CompBioBench item

cryptic-exon-q1

Question

I have a bulk human RNA-seq fastq file cryptic.exon.q1.fq.gz. There is exactly one highly expressed coding gene that has a cryptic exon in it formed by two novel junctions. Report the HGNC gene symbol (uppercase) for that gene.

Inferred answer
GNG10Uniquely inferred from submitted scores.
Domain
Transcriptomics
Question style
Synthetic/Augmented Data

An answer can be uniquely inferred for 54 of 100 questions from submitted scores. Green: correct · Red: incorrect. For questions without a uniquely determined answer: Blue: matches prediction · Orange: differs from prediction. These are inferred answer checks, not official per-question grades.

Galaxy

12/12 terminal item results

UDT = user-defined tool: custom code executed by Galaxy. Route badges summarize analysis tools, excluding uploads and automatic conversions. How usage is counted.

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

CorrectUDT + standard tools
AnswerGNG10
Workflow
Hisat2Rna StarRank STAR junctionsFind cryptic exon junction pairs
Codex GPT-5.5

Replicate 2

CorrectUDT-only
AnswerGNG10
Workflow
FASTQ diagnosticFASTQ diagnostic gzip-awareMinimap2SamtoolsBedtools5 more tools
Total tokens
1.3M
Output
18.6K
Codex GPT-5.5

Replicate 3

CorrectUDT + standard tools
AnswerGNG10
Workflow
Rna StarFind cryptic exon geneFind cryptic exon gene v2Find cryptic exon gene v3Find cryptic exon gene v46 more tools
Total tokens
4.0M
Output
19.9K

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

CorrectUDT + standard tools
AnswerGNG10
Workflow
Rna StarRank protein-coding cryptic exons
Total tokens
2.7M
Output
12.1K
Codex GPT-5.6 Sol

Replicate 2

CorrectUDT + standard tools
AnswerGNG10
Workflow
FastqcRna StarRank novel splice-junction exon pairsIdentify paired novel cryptic exons with GENCODEResolve overlapping cryptic-exon gene models
Total tokens
6.7M
Output
16.7K
Codex GPT-5.6 Sol

Replicate 3

CorrectUDT + standard tools
AnswerGNG10
Workflow
Rna StarCryptic exon junction-pair scanNovel junction cryptic-exon diagnosticGTF-coordinate novel junction-pair scanFilter Tabular
Total tokens
12.2M
Output
20.5K

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

CorrectStandard tools only
AnswerGNG10
Workflow
No Galaxy tool job recorded
Total tokens
9.2M
Output
29.0K
Codex + DeepSeek-v4-pro-0813

Replicate 2

IncorrectStandard tools only
AnswerCD74
Workflow
Rna StarFilter rowsSelect columnsConverter Interval To Bedstrict 0Converter Interval To Bed 06 more tools
Total tokens
12.8M
Output
45.8K
Codex + DeepSeek-v4-pro-0813

Replicate 3

IncorrectStandard tools only
AnswerUBC
Workflow
FastqcRna StarHisat2FeaturecountsFilter Tabular1 more tools
Total tokens
23.4M
Output
73.5K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

CorrectStandard tools only
AnswerGNG10
Workflow
Hisat2Upload1Gtftobed12Rseqc Junction AnnotationSort rows4 more tools
Total tokens
15.3M
Output
38.4K
Codex GPT-5.6 Luna

Replicate 2

CorrectStandard tools only
AnswerGNG10
Workflow
FastqcHisat2FeaturecountsGtftobed12Rseqc Junction Annotation6 more tools
Total tokens
13.5M
Output
37.1K
Codex GPT-5.6 Luna

Replicate 3

CorrectUDT + standard tools
AnswerGNG10
Workflow
Fastq InfoHisat2Cryptic Junction AnalyzerUDT Render ProbeStringtie5 more tools
Total tokens
15.1M
Output
62.0K

Anycode

13/13 terminal item results

Codex GPT-5.5

3 replicates
Codex GPT-5.5

Replicate 1

Correct
AnswerGNG10
Workflow
Local codePythonMinimap2Samtools
Codex GPT-5.5

Replicate 2

Correct
AnswerGNG10
Workflow
Local codeMinimap2SamtoolsBLASTNBowtie21 more tools
Codex GPT-5.5

Replicate 3

Correct
AnswerGNG10
Workflow
Local codeSamtoolsBowtie2Minimap2BWA-MEM2 more tools

Codex GPT-5.6 Sol

3 replicates
Codex GPT-5.6 Sol

Replicate 1

Correct
AnswerGNG10
Workflow
Local codeMinimap2SamtoolsPython
Codex GPT-5.6 Sol

Replicate 2

Correct
AnswerGNG10
Workflow
Local codeMinimap2Samtools
Codex GPT-5.6 Sol

Replicate 3

Correct
AnswerGNG10
Workflow
Local codeMinimap2SamtoolsBedtools

Codex + DeepSeek-v4-pro-0813

3 replicates
Codex + DeepSeek-v4-pro-0813

Replicate 1

Correct
AnswerGNG10
Workflow
Local codePythonSamtoolsBWA-MEMBedtools1 more tools
Total tokens
20.7M
Output
62.7K
Codex + DeepSeek-v4-pro-0813

Replicate 2

Correct
AnswerGNG10
Workflow
Local codeMinimap2SamtoolsBWA-MEMBowtie23 more tools
Total tokens
6.3M
Output
46.3K
Codex + DeepSeek-v4-pro-0813

Replicate 3

Correct
AnswerGNG10
Workflow
Local codeBowtie2SamtoolsBedtoolsPython3 more tools
Total tokens
5.6M
Output
42.1K

Codex GPT-5.6 Luna

3 replicates
Codex GPT-5.6 Luna

Replicate 1

Correct
AnswerGNG10
Workflow
Local codeMinimap2BWA-MEMBowtie2Samtools3 more tools
Codex GPT-5.6 Luna

Replicate 2

Correct
AnswerGNG10
Workflow
Local codeMinimap2BWA-MEMSamtoolsBedtools5 more tools
Codex GPT-5.6 Luna

Replicate 3

Correct
AnswerGNG10
Workflow
Local codeMinimap2BWA-MEMSamtoolsBowtie23 more tools

Codex GPT-6 Astra

1 replicate
Codex GPT-6 Astra

Replicate 1

Correct
AnswerGNG10
Workflow
Local codePythonMinimap2Samtoolspysam
Total tokens
512.5K
Output
4.8K