Task 8: Small-variant differences among haploid yeast strains

What the task asks for

Call small variants jointly in four haploid yeast strains, annotate nuclear and mitochondrial effects, and list confident differences between every pair of strains.

8 scored results14 required18 runs scored

Codex GPT-5.6 Luna

6 of 6 runs scored

Galaxy-API code 0.167 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.694

All results pass0 of 3 runsruns in which every scored result passes

run 10.1251/8run 20.2502/8run 30.1251/8

Open-ended code 0.167 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.384

All results pass0 of 3 runsruns in which every scored result passes

run 10.1251/8run 20.1251/8run 30.2502/8

Codex GPT-5.6 Sol

6 of 6 runs scored

Galaxy-API code 0.917 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.909

All results pass1 of 3 runsruns in which every scored result passes

run 10.8757/8run 21.0008/8run 30.8757/8

Open-ended code 0.917 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.912

All results pass2 of 3 runsruns in which every scored result passes

run 11.0008/8run 21.0008/8run 30.7506/8

DeepSeek V4 Pro via Codex

6 of 6 runs scored

Galaxy-API code 0.833 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.918

All results pass1 of 3 runsruns in which every scored result passes

run 11.0008/8run 20.6255/8run 30.8757/8

Open-ended code 0.417 BixBench3 acc (share of results that pass), averaged over 3 of 3 runs continuous score 0.615

All results pass0 of 3 runsruns in which every scored result passes

run 10.3753/8run 20.1251/8run 30.7506/8

Three measures. Every required result gets a score from 0 to 1 by comparison with the reference. Passing requires a score of at least 0.80 and compliance with the required content, method and output instructions. BixBench3 acc is the share of results that pass. The continuous score is the plain average of all the result scores, with no pass mark. For example, run 1 of GPT-5.6 Luna with Galaxy-API code scored 0.994, 0.996, 0.728, 0.989, 0.599, 0.597, 0.264 and 0.335 on its 8 results. 1 of them pass, so BixBench3 acc = 1/8 = 0.125; the average of all 8 scores is 0.688, the continuous score. A third measure, all results pass, counts a run only when every scored result passes, which is the share of runs that delivered the whole analysis correctly. A failing result still contributes its numerical score to the continuous average. A score above 0.80 does not excuse an incorrect method, missing member or an explicitly required output format. Each card averages the scored runs of this task.

Error analysis

BixBench3 acc, Galaxy-API code vs Open-ended code: GPT-5.6 Luna 0.167 vs 0.167; GPT-5.6 Sol 0.917 vs 0.917; DeepSeek V4 Pro 0.833 vs 0.417.

What Galaxy added

DeepSeek V4 Pro passed twice the share of results in Galaxy. Its Open-ended runs added options the task does not ask for, which changed the variant calls in one run and the read filtering in another, and the third wrote its pairwise files with a script that left blank lines between records. Galaxy's trimming tool also produced the current report format, where two Open-ended runs installed older versions.

Why Galaxy did not do better

GPT-5.6 Luna failed in both conditions for the same reason: it switched off two extra genotype priors along with the requested ones, which changed genotype qualities. In two of its Galaxy runs the samples came out in reverse column order and the contrast filters chose samples by position, so four pairwise files compared the wrong strains. The Galaxy normalization form leaves indel left-alignment off by default, unlike the command-line tool, and one DeepSeek V4 Pro run kept that default. Annotation passed whenever the calls before it were right.

Run-level details (7)

All three Galaxy runs and the third Open-ended code run with GPT-5.6 Luna disabled the binomial-observation and allele-balance priors along with the requested population and HWE priors. The extra changes altered genotype qualities.
GPT-5.6 Luna's three Open-ended code runs and third Galaxy run returned short duplicate-count summaries or logs without the required library metrics table. Its first two Open-ended code runs also wrote their own QC summaries instead of complete native reports.
The first Open-ended code run with DeepSeek V4 Pro and the second with GPT-5.6 Luna merged nearby changes over a 50-base window. This changed the variant records and the downstream annotation and contrast domains.
In the first and third Galaxy runs and the third Open-ended code run with GPT-5.6 Sol, the caller also reported alleles it had seen but not called. The added alleles changed multiallelic records and their consequences.
The second Galaxy run with DeepSeek V4 Pro normalized without indel left-alignment, which the Galaxy normalization form leaves off by default. Its second Open-ended code run allowed mate overlaps as short as one base during filtering, changing trimming and downstream read counts.
The first two Galaxy runs with GPT-5.6 Luna listed strains in reverse order, S04 to S01. Their contrast filters selected samples by column position and therefore returned the wrong pair for four pairwise files. The first run also computed alignment statistics on deduplicated BAMs instead of the required proper-pair BAMs.
The third Open-ended code run with DeepSeek V4 Pro inserted blank lines between VCF records. The first Open-ended code run with GPT-5.6 Luna added a tag declared as one value but containing two values at multiallelic sites; its contrast script consumed the call set once, leaving the next five contrast files empty and reporting them as valid empty sets.

Instruction, format and method errors (8)

fastp trimmed with adapter sequences (detected or supplied) rather than overlap only: S01, S02, S03, S04.Task requirement: Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection
GPT-5.6 Luna / Galaxy-API code / run 1 → result 1+2; GPT-5.6 Luna / Galaxy-API code / run 2 → result 1+2; GPT-5.6 Luna / Galaxy-API code / run 3 → result 1+2; GPT-5.6 Luna / Open-ended code / run 3 → result 1+2
Supplementary alignments are not soft-clipped with their own MAPQ (3287 of 3287 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 1 → result 3
Supplementary alignments are not soft-clipped with their own MAPQ (bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 2 → result 3; GPT-5.6 Luna / Open-ended code / run 3 → result 3; DeepSeek V4 Pro / Galaxy-API code / run 3 → result 3; DeepSeek V4 Pro / Open-ended code / run 2 → result 3
Supplementary alignments are not soft-clipped with their own MAPQ (3287 of 3287 supplementary records hard-clipped; bwa mem without -Y) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 3 → result 3
Pairwise contrasts differ from differing nonmissing genotypes with QUAL≥30 and DP≥10/GQ≥20 in both samples of the delivered normalized calls, with all four sample columns: S01-S02, S01-S03, S01-S04, S02-S03, S02-S04, S03-S04.Task requirement: For every unordered pair, select differing nonmissing haploid genotypes with QUAL≥30 and DP≥10/GQ≥20 in both samples. / derive pairwise contrasts from its unannotated normalized representation, retaining all four sample columns S01–S04 even in pairwise or empty VCFs.
GPT-5.6 Luna / Galaxy-API code / run 3 → result 14
Supplementary alignments are not soft-clipped with their own MAPQ (3296 of 3296 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Open-ended code / run 1 → result 3
Supplementary alignments are not soft-clipped with their own MAPQ (3282 of 3282 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Open-ended code / run 2 → result 3
Supplementary alignments are not soft-clipped with their own MAPQ (3286 of 3286 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
DeepSeek V4 Pro / Open-ended code / run 3 → result 3

Every run

The task asks the agent to return a numbered list of results, each a file or a folder of files. The columns on the right give the score of each required result in each run, from 0 to 1 against the reference, under the result's number and name; the task text and the scoring table below describe each one. Green marks a passing result. Failing cells link to the reasons below, including instruction and format errors even when the numerical score is at least 0.80. Results joined by + are scored together as one. In each row, BixBench3 acc is the number of green cells divided by the number of results, and the continuous score is the average of all the result scores in the row. Tokens are input (including cached input) plus output tokens; minutes are the wall time of the run. Under Records, Trace opens the run's task text, agent event log, final message and returned files in the lab's run trace dataset on Hugging Face, and History opens the usegalaxy.org history that a Galaxy-API code run worked in.

								Score of each required result
Model	Condition	Run	BixBench3 acc share passing	Continuous score	Tokens	Minutes	Records	1+2read QC + filtered reads	3aligned	4+6proper pairs + alignment reports	5deduplicated	7joint raw	8joint normalized	10+11annotated nuclear + annotated mitochondrial	14pairwise contrasts
GPT-5.6 Luna	Galaxy-API code	1	1/8 = 0.125	0.688	7.2M	76	Trace History	0.99Method	0.99Method	0.72Member	0.98 Passed	0.59Content	0.59Content	0.26Content	0.33Member
		2	2/8 = 0.250	0.718	14.2M	90	Trace History	0.99Method	0.99Method	0.96 Passed	0.98 Passed	0.59Content	0.59Content	0.26Content	0.33Member
		3	1/8 = 0.125	0.676	18.5M	92	Trace History	0.99Method	0.99Method	0.96 Passed	0.00Member	0.60Content	0.59Content	0.26Content	0.99Method
	Open-ended code	1	1/8 = 0.125	0.227	4.8M	80	Trace	0.00Member	0.96Method	0.85 Passed	0.00Member	0.00Content	0.00Content	0.00Content	0.00Member
		2	1/8 = 0.125	0.256	9.5M	76	Trace	0.00Member	0.99Method	0.94 Passed	0.00Member	0.00Check	0.00Content	0.00Content	0.10Member
		3	2/8 = 0.250	0.670	24.0M	65	Trace	0.97Method	0.98Method	0.92 Passed	0.00Member	0.61Content	0.59Content	0.29Content	0.96 Passed
GPT-5.6 Sol	Galaxy-API code	1	7/8 = 0.875	0.872	10.6M	95	Trace History	1.00 Passed	1.00 Passed	0.85 Passed	1.00 Passed	0.88 Passed	0.88 Passed	0.38Content	0.97 Passed
		2	8/8 = 1.000	0.982	7.8M	84	Trace History	1.00 Passed	1.00 Passed	0.85 Passed	1.00 Passed	0.99 Passed	0.99 Passed	0.99 Passed	1.00 Passed
		3	7/8 = 0.875	0.872	11.7M	98	Trace History	1.00 Passed	1.00 Passed	0.85 Passed	1.00 Passed	0.88 Passed	0.88 Passed	0.38Content	0.97 Passed
	Open-ended code	1	8/8 = 1.000	0.987	5.5M	36	Trace	0.99 Passed	0.99 Passed	0.97 Passed	0.99 Passed	0.97 Passed	0.97 Passed	0.96 Passed	0.99 Passed
		2	8/8 = 1.000	0.990	7.3M	47	Trace	0.99 Passed	1.00 Passed	0.97 Passed	1.00 Passed	0.98 Passed	0.98 Passed	0.97 Passed	0.99 Passed
		3	6/8 = 0.750	0.759	6.2M	32	Trace	0.00Member	1.00 Passed	0.97 Passed	1.00 Passed	0.87 Passed	0.87 Passed	0.37Content	0.97 Passed
DeepSeek V4 Pro	Galaxy-API code	1	8/8 = 1.000	1.000	26.9M	102	Trace History	1.00 Passed	1.00 Passed	1.00 Passed	1.00 Passed	0.99 Passed	0.99 Passed	0.99 Passed	1.00 Passed
		2	5/8 = 0.625	0.759	20.5M	98	Trace History	1.00 Passed	1.00 Passed	1.00 Passed	1.00 Passed	0.99 Passed	0.29Content	0.00Content	0.77Member
		3	7/8 = 0.875	0.994	14.0M	111	Trace History	0.99 Passed	0.99Method	0.99 Passed	0.99 Passed	0.99 Passed	0.99 Passed	0.96 Passed	0.99 Passed
	Open-ended code	1	3/8 = 0.375	0.385	18.0M	41	Trace	0.00Member	0.99 Passed	0.97 Passed	0.99 Passed	0.00Check	0.00Content	0.00Content	0.10Member
		2	1/8 = 0.125	0.603	5.7M	41	Trace	0.44Member	0.95Method	0.70Member	0.00Member	0.69Content	0.67Content	0.39Content	0.96 Passed
		3	6/8 = 0.750	0.857	9.5M	51	Trace	0.99 Passed	0.99Method	0.97 Passed	0.99 Passed	0.97 Passed	0.97 Passed	0.93 Passed	0.00Member

Why results did not pass

GPT-5.6 Luna / Galaxy-API code / run 1 → result 1+2 (read QC + filtered reads)
- Method: fastp trimmed with adapter sequences (detected or supplied) rather than overlap only: S01, S02, S03, S04.Task requirement: Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection
GPT-5.6 Luna / Galaxy-API code / run 1 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (3287 of 3287 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 1 → result 4+6 (proper pairs + alignment reports)
- Content: Content score below 0.80. Value: 0.728999. Required minimum: 0.8.
  - Alignment records: 0.996874 to 0.997262 (4 checks)
  - Matching records (F1): 0.96546 to 0.99219 (4 checks)
  - Numerical agreement (CCC): 0.748621 to 0.768247 (4 checks)
  - Present, finite data cells (F1): 0.988691 to 0.997602 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0.740378
  - S02: 0.715133
  - S03: 0.72162
  - S04: 0.738864
GPT-5.6 Luna / Galaxy-API code / run 1 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0.599795. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.997496 to 0.99838 (9 checks)
  - Numerical agreement (CCC): 0.630235 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.997579 to 0.998445 (5 checks)
GPT-5.6 Luna / Galaxy-API code / run 1 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.59712. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.997566 to 0.998415 (9 checks)
  - Numerical agreement (CCC): 0.630223 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.99765 to 0.99848 (5 checks)
  - Required fields: 0.994956
GPT-5.6 Luna / Galaxy-API code / run 1 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.264002. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.986595 to 0.998519 (20 checks)
  - Numerical agreement (CCC): 0.518257 to 0.999999 (10 checks)
  - Present, finite data cells (F1): 0.986514 to 0.998586 (10 checks)
  - Required fields: 0.995305
GPT-5.6 Luna / Galaxy-API code / run 1 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0.335743. Required minimum: 0.8.
  - Matching records (F1): 0.198823 to 0.997802 (18 checks)
  - Required fields: 0.993174
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0.0107407
  - member 2: 0.00793034
  - member 3: 0.00804111
  - member 4: 0.0106753
GPT-5.6 Luna / Galaxy-API code / run 2 → result 1+2 (read QC + filtered reads)
- Method: fastp trimmed with adapter sequences (detected or supplied) rather than overlap only: S01, S02, S03, S04.Task requirement: Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection
GPT-5.6 Luna / Galaxy-API code / run 2 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 2 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0.599671. Required minimum: 0.8.
  - Category agreement (F1): 0.999924
  - Matching records (F1): 0.997496 to 0.99838 (9 checks)
  - Numerical agreement (CCC): 0.630154 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.997579 to 0.998445 (5 checks)
GPT-5.6 Luna / Galaxy-API code / run 2 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.596917. Required minimum: 0.8.
  - Category agreement (F1): 0.999791
  - Matching records (F1): 0.997566 to 0.998415 (9 checks)
  - Numerical agreement (CCC): 0.630142 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.99765 to 0.99848 (5 checks)
  - Required fields: 0.994956
GPT-5.6 Luna / Galaxy-API code / run 2 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.263726. Required minimum: 0.8.
  - Category agreement (F1): 0.999781
  - Matching records (F1): 0.986595 to 0.998519 (20 checks)
  - Numerical agreement (CCC): 0.517891 to 0.999999 (10 checks)
  - Present, finite data cells (F1): 0.986514 to 0.998586 (10 checks)
  - Required fields: 0.995305
GPT-5.6 Luna / Galaxy-API code / run 2 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0.335743. Required minimum: 0.8.
  - Matching records (F1): 0.198823 to 0.997802 (18 checks)
  - Required fields: 0.993174
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0.0107407
  - member 2: 0.00793034
  - member 3: 0.00804111
  - member 4: 0.0106753
GPT-5.6 Luna / Galaxy-API code / run 3 → result 1+2 (read QC + filtered reads)
- Method: fastp trimmed with adapter sequences (detected or supplied) rather than overlap only: S01, S02, S03, S04.Task requirement: Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection
GPT-5.6 Luna / Galaxy-API code / run 3 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (3287 of 3287 supplementary records hard-clipped; bwa mem without -Y) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Galaxy-API code / run 3 → result 5 (deduplicated)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Alignment records: 0.987527 to 0.992389 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Galaxy-API code / run 3 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0.601175. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.997566 to 0.99845 (9 checks)
  - Numerical agreement (CCC): 0.631019 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.99765 to 0.998515 (5 checks)
GPT-5.6 Luna / Galaxy-API code / run 3 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.598493. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.997636 to 0.998485 (9 checks)
  - Numerical agreement (CCC): 0.631006 to 0.999992 (5 checks)
  - Present, finite data cells (F1): 0.99772 to 0.998551 (5 checks)
  - Required fields: 0.994956
GPT-5.6 Luna / Galaxy-API code / run 3 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.260075. Required minimum: 0.8.
  - Category agreement (F1): 1
  - Matching records (F1): 0.985235 to 0.998608 (20 checks)
  - Numerical agreement (CCC): 0.519532 to 1 (10 checks)
  - Present, finite data cells (F1): 0.985145 to 0.998675 (10 checks)
  - Required fields: 0.995311
GPT-5.6 Luna / Galaxy-API code / run 3 → result 14 (pairwise contrasts)
- Method: Pairwise contrasts differ from differing nonmissing genotypes with QUAL≥30 and DP≥10/GQ≥20 in both samples of the delivered normalized calls, with all four sample columns: S01-S02, S01-S03, S01-S04, S02-S03, S02-S04, S03-S04.Task requirement: For every unordered pair, select differing nonmissing haploid genotypes with QUAL≥30 and DP≥10/GQ≥20 in both samples. / derive pairwise contrasts from its unannotated normalized representation, retaining all four sample columns S01–S04 even in pairwise or empty VCFs.
GPT-5.6 Luna / Open-ended code / run 1 → result 1+2 (read QC + filtered reads)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Matching records (F1): 0.960566 to 0.96906 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Open-ended code / run 1 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (3296 of 3296 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Open-ended code / run 1 → result 5 (deduplicated)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Alignment records: 0.954834 to 0.96007 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Open-ended code / run 1 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (11 checks)
GPT-5.6 Luna / Open-ended code / run 1 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (11 checks)
GPT-5.6 Luna / Open-ended code / run 1 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (24 checks)
GPT-5.6 Luna / Open-ended code / run 1 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Category agreement (F1): 0 (5 checks)
  - Matching records (F1): 0 (15 checks)
  - Required content missing or unreadable: 0 (5 checks)
  - Required fields: 0 (5 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0
  - member 2: 0
  - member 3: 0
  - member 4: 0
  - member 5: 0
  - member 6: 0
GPT-5.6 Luna / Open-ended code / run 2 → result 1+2 (read QC + filtered reads)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Matching records (F1): 0.997525 to 0.997974 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Open-ended code / run 2 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (3282 of 3282 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Open-ended code / run 2 → result 5 (deduplicated)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Alignment records: 0.98779 to 0.992251 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Open-ended code / run 2 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 7.49348e-05. Required minimum: 0.8.
  - Category agreement (F1): 0.998556
  - Matching records (F1): 0.510007 to 0.514429 (9 checks)
  - Numerical agreement (CCC): 0.882392 to 0.999966 (5 checks)
  - Present, finite data cells (F1): 0.51018 to 0.514564 (5 checks)
  - Required fields: 0.998091
- Check: A required check is below its minimum. Required minimum: 0.99.
  - Matching records (F1): 0.514429
GPT-5.6 Luna / Open-ended code / run 2 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 6.28834e-05. Required minimum: 0.8.
  - Category agreement (F1): 0.996536
  - Matching records (F1): 0.527582 to 0.532123 (9 checks)
  - Numerical agreement (CCC): 0.868172 to 0.99728 (5 checks)
  - Present, finite data cells (F1): 0.527707 to 0.532206 (5 checks)
  - Required fields: 0.58853
GPT-5.6 Luna / Open-ended code / run 2 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Category agreement (F1): 0.99284 to 0.996968 (2 checks)
  - Matching records (F1): 0 to 0.532141 (20 checks)
  - Numerical agreement (CCC): 0.863334 to 0.999985 (10 checks)
  - Present, finite data cells (F1): 0.52085 to 0.53225 (10 checks)
  - Required fields: 0.584786 to 0.722667 (2 checks)
GPT-5.6 Luna / Open-ended code / run 2 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0.105006. Required minimum: 0.8.
  - Category agreement (F1): 0.991331 to 1 (6 checks)
  - Matching records (F1): 0.533287 to 0.56713 (18 checks)
  - Required fields: 0.606237 to 0.695281 (6 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0.0938389
  - member 2: 0.117478
  - member 3: 0.124327
  - member 4: 0.0942984
  - member 5: 0.0932497
  - member 6: 0.106843
GPT-5.6 Luna / Open-ended code / run 3 → result 1+2 (read QC + filtered reads)
- Method: fastp trimmed with adapter sequences (detected or supplied) rather than overlap only: S01, S02, S03, S04.Task requirement: Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection
GPT-5.6 Luna / Open-ended code / run 3 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
GPT-5.6 Luna / Open-ended code / run 3 → result 5 (deduplicated)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Alignment records: 0.97625 to 0.981014 (4 checks)
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Luna / Open-ended code / run 3 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0.610705. Required minimum: 0.8.
  - Category agreement (F1): 0.999045
  - Matching records (F1): 0.994065 to 0.995192 (9 checks)
  - Numerical agreement (CCC): 0.672877 to 0.999902 (5 checks)
  - Present, finite data cells (F1): 0.994193 to 0.995304 (5 checks)
GPT-5.6 Luna / Open-ended code / run 3 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.598443. Required minimum: 0.8.
  - Category agreement (F1): 0.998341
  - Matching records (F1): 0.995536 to 0.996601 (9 checks)
  - Numerical agreement (CCC): 0.672592 to 0.999893 (5 checks)
  - Present, finite data cells (F1): 0.995643 to 0.996691 (5 checks)
  - Required fields: 0.964338
GPT-5.6 Luna / Open-ended code / run 3 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.299354. Required minimum: 0.8.
  - Category agreement (F1): 0.99826
  - Matching records (F1): 0.985235 to 0.996699 (20 checks)
  - Numerical agreement (CCC): 0.608787 to 0.999991 (10 checks)
  - Present, finite data cells (F1): 0.985145 to 0.996791 (10 checks)
  - Required fields: 0.966458 to 0.987261 (2 checks)
GPT-5.6 Sol / Galaxy-API code / run 1 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.380002. Required minimum: 0.8.
  - Matching records (F1): 0.929319 to 0.993006 (20 checks)
  - Numerical agreement (CCC): 0.998942 to 1 (3 checks)
  - Present, finite data cells (F1): 0.929865 to 0.993107 (10 checks)
GPT-5.6 Sol / Galaxy-API code / run 3 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.380388. Required minimum: 0.8.
  - Matching records (F1): 0.929319 to 0.993006 (20 checks)
  - Numerical agreement (CCC): 0.999956 to 1 (2 checks)
  - Present, finite data cells (F1): 0.929865 to 0.993107 (10 checks)
GPT-5.6 Sol / Open-ended code / run 3 → result 1+2 (read QC + filtered reads)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
GPT-5.6 Sol / Open-ended code / run 3 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.374921. Required minimum: 0.8.
  - Matching records (F1): 0.929319 to 0.993006 (20 checks)
  - Numerical agreement (CCC): 0.989576 to 0.996179 (2 checks)
  - Present, finite data cells (F1): 0.929865 to 0.993107 (10 checks)
DeepSeek V4 Pro / Galaxy-API code / run 2 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.29015. Required minimum: 0.8.
  - Matching records (F1): 0.913145 to 0.916168 (9 checks)
  - Numerical agreement (CCC): 1
  - Present, finite data cells (F1): 0.913575 to 0.91651 (5 checks)
DeepSeek V4 Pro / Galaxy-API code / run 2 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 1.70818e-05. Required minimum: 0.8.
  - Matching records (F1): 0.514286 to 0.921315 (20 checks)
  - Numerical agreement (CCC): 1 to 1 (2 checks)
  - Present, finite data cells (F1): 0.517869 to 0.921605 (10 checks)
DeepSeek V4 Pro / Galaxy-API code / run 2 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0.779958. Required minimum: 0.8.
  - Matching records (F1): 0.910657 to 0.92388 (18 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0.78747
  - member 2: 0.784133
  - member 3: 0.761855
  - member 4: 0.787582
  - member 5: 0.784044
  - member 6: 0.774664
DeepSeek V4 Pro / Galaxy-API code / run 3 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
DeepSeek V4 Pro / Open-ended code / run 1 → result 1+2 (read QC + filtered reads)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
DeepSeek V4 Pro / Open-ended code / run 1 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 7.29275e-05. Required minimum: 0.8.
  - Category agreement (F1): 0.997242
  - Matching records (F1): 0.509482 to 0.51377 (9 checks)
  - Numerical agreement (CCC): 0.88482 to 0.999987 (5 checks)
  - Present, finite data cells (F1): 0.509666 to 0.513928 (5 checks)
  - Required fields: 0.998114
- Check: A required check is below its minimum. Required minimum: 0.99.
  - Matching records (F1): 0.51377
DeepSeek V4 Pro / Open-ended code / run 1 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 6.2022e-05. Required minimum: 0.8.
  - Category agreement (F1): 0.996791
  - Matching records (F1): 0.527396 to 0.531805 (9 checks)
  - Numerical agreement (CCC): 0.870685 to 0.99716 (5 checks)
  - Present, finite data cells (F1): 0.52752 to 0.531901 (5 checks)
  - Required fields: 0.590341
DeepSeek V4 Pro / Open-ended code / run 1 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 1.24863e-09. Required minimum: 0.8.
  - Category agreement (F1): 0.990542 to 0.996939 (2 checks)
  - Matching records (F1): 0.514658 to 0.538129 (20 checks)
  - Numerical agreement (CCC): 0.865596 to 0.999987 (10 checks)
  - Present, finite data cells (F1): 0.512505 to 0.53205 (10 checks)
  - Required fields: 0.635646 to 0.736039 (2 checks)
DeepSeek V4 Pro / Open-ended code / run 1 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0.104104. Required minimum: 0.8.
  - Category agreement (F1): 0.989946 to 1 (6 checks)
  - Matching records (F1): 0.533362 to 0.567849 (18 checks)
  - Required fields: 0.606237 to 0.68118 (6 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0.0938131
  - member 2: 0.115025
  - member 3: 0.122377
  - member 4: 0.0941797
  - member 5: 0.0932856
  - member 6: 0.105946
DeepSeek V4 Pro / Open-ended code / run 2 → result 1+2 (read QC + filtered reads)
- Content: Content score below 0.80. Value: 0.446387. Required minimum: 0.8.
  - Matching records (F1): 0.950621 to 0.954492 (4 checks)
  - Numerical agreement (CCC): 0.388293 to 0.552904 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0.525602
  - S02: 0.370622
  - S03: 0.433723
  - S04: 0.455601
DeepSeek V4 Pro / Open-ended code / run 2 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
DeepSeek V4 Pro / Open-ended code / run 2 → result 4+6 (proper pairs + alignment reports)
- Content: Content score below 0.80. Value: 0.706182. Required minimum: 0.8.
  - Alignment records: 0.950348 to 0.954148 (4 checks)
  - Matching records (F1): 0.965414 to 0.989186 (4 checks)
  - Numerical agreement (CCC): 0.7547 to 0.767585 (4 checks)
  - Present, finite data cells (F1): 0.989395 to 0.997768 (4 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0.695235
  - S02: 0.699428
  - S03: 0.70897
  - S04: 0.721095
DeepSeek V4 Pro / Open-ended code / run 2 → result 5 (deduplicated)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Alignment records: 0.943321 to 0.947862 (4 checks)
  - Summary statistics within tolerance: 0 (12 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - S01: 0
  - S02: 0
  - S03: 0
  - S04: 0
DeepSeek V4 Pro / Open-ended code / run 2 → result 7 (joint raw)
- Content: Content score below 0.80. Value: 0.692602. Required minimum: 0.8.
  - Category agreement (F1): 0.997479
  - Matching records (F1): 0.993556 to 0.994326 (9 checks)
  - Numerical agreement (CCC): 0.802995 to 0.994269 (5 checks)
  - Present, finite data cells (F1): 0.993637 to 0.99439 (5 checks)
  - Required fields: 0.999971
DeepSeek V4 Pro / Open-ended code / run 2 → result 8 (joint normalized)
- Content: Content score below 0.80. Value: 0.675139. Required minimum: 0.8.
  - Category agreement (F1): 0.995994
  - Matching records (F1): 0.994571 to 0.995207 (9 checks)
  - Numerical agreement (CCC): 0.803114 to 0.994264 (5 checks)
  - Present, finite data cells (F1): 0.99463 to 0.995248 (5 checks)
  - Required fields: 0.964098
DeepSeek V4 Pro / Open-ended code / run 2 → result 10+11 (annotated nuclear + annotated mitochondrial)
- Content: Content score below 0.80. Value: 0.391241. Required minimum: 0.8.
  - Category agreement (F1): 0.993046 to 0.996463 (2 checks)
  - Matching records (F1): 0.97712 to 0.995412 (20 checks)
  - Numerical agreement (CCC): 0.800305 to 0.994458 (10 checks)
  - Present, finite data cells (F1): 0.977327 to 0.99545 (10 checks)
  - Required fields: 0.949045 to 0.975018 (2 checks)
DeepSeek V4 Pro / Open-ended code / run 3 → result 3 (aligned)
- Method: Supplementary alignments are not soft-clipped with their own MAPQ (3286 of 3286 supplementary records hard-clipped; bwa mem without -Y; bwa mem without -q) in S01, S02, S03, S04.Task requirement: Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction.
DeepSeek V4 Pro / Open-ended code / run 3 → result 14 (pairwise contrasts)
- Content: Content score below 0.80. Value: 0. Required minimum: 0.8.
  - Required content missing or unreadable: 0 (30 checks)
- Member: A required member scores below 0.80. Required minimum: 0.8.
  - member 1: 0
  - member 2: 0
  - member 3: 0
  - member 4: 0
  - member 5: 0
  - member 6: 0

Task text

The scientific part of the task, identical in both conditions. The general submission rules and the instructions for each condition are on the overview page.

Full task text

Task

Characterize small genomic differences among four haploid S. cerevisiae laboratory strains, preserving complete variant and annotation evidence. The strains are not treatment replicates. Use a haplotype-based Bayesian method for joint small-variant calling, allowing nearby linked changes to inform haploid genotypes. Compare normalized, annotated sample genotypes with explicit confidence filters, not merely presence in independently called variant lists. Distinguish nuclear and mitochondrial coding consequences.

Inputs

Four strains' eight complete paired-read files and sample/mate identities.
Ensembl R64-1-1 release-115 reference and annotation, including nuclear and mitochondrial sequences.

Key parameters and conditions

Align paired Illumina reads using maximal exact-match seeds with affine-gap Smith–Waterman extension; select coordinate-duplicate representatives by summed base qualities. Remove adapter sequence only where the two mates of a pair overlap (insert shorter than the read), without sequence-based adapter detection, and do not merge mates. Retain a pair only when both reads satisfy Q15 qualified bases with at most 40% unqualified bases, at most five ambiguous bases and minimum length 15 nt. Add no fixed-length trimming or sliding-window trimming. First retain alignment records marked as proper pairs, then remove duplicates; do not downsample or add region/depth exclusions.
Jointly genotype all four haploid samples without a population/HWE prior, with genotype qualities and minimum base/MAPQ 20. A candidate alternate allele must have count≥2 and fraction≥0.20 within at least one single sample; do not pool counts across the four samples to meet these thresholds. Retain all candidate sites (variant probability cutoff 0).
Normalize against the supplied reference and stop on REF mismatches. Preserve multiallelic and complex records without atomizing or splitting. Retain sample-level missing and low-confidence calls.
Build an effect-annotation database from the supplied genome and gene annotation. Report all consequences/transcripts in the standard 16-field ANN representation, including HIGH, MODERATE, LOW or MODIFIER impact categories, using the applicable nuclear/yeast-mitochondrial genetic code in each compartment. For every unordered pair, select differing nonmissing haploid genotypes with QUAL≥30 and DP≥10/GQ≥20 in both samples. Absence or missing genotype is not evidence of reference identity. Briefly discuss uncertain calls and comparison limitations.
Apply upstream/downstream effect-annotation windows of zero bases. Annotate the complete normalized call set; derive pairwise contrasts from its unannotated normalized representation, retaining all four sample columns S01–S04 even in pairwise or empty VCFs. Use lexicographic pair names. Soft-clip supplementary alignments and retain supplementary mapping quality rather than applying its special reduction. Report alignment statistics on the proper-pair BAM.

Required results

final_answer/01_read_qc/{sample}/
JSON
Complete native paired-read QC/processing report qc.json. Files: {sample}/qc.json.
final_answer/02_filtered_reads/{sample}/
FASTQ
Both processed mate files. Files: {sample}/forward.fastq.gz, {sample}/reverse.fastq.gz.
final_answer/03_aligned/{sample}.bam
BAM + index
Coordinate-sorted alignments before proper-pair selection.
final_answer/04_proper_pairs/{sample}.bam
BAM + index
Proper-pair selected alignments.
final_answer/05_deduplicated/{sample}/
BAM + native text metrics
deduplicated.bam and duplicates.txt with the coordinate-duplicate library metrics. Files: {sample}/deduplicated.bam, {sample}/duplicates.txt.
final_answer/06_alignment_reports/{sample}/
native text
stats.txt: complete flag, sequence, quality, insert-size and coverage statistics on the proper-pair BAM. Files: {sample}/stats.txt.
final_answer/07_joint_raw.vcf
VCF
Complete unfiltered four-sample joint calls, with haploid genotypes, DP/GQ and observed counts for REF and every ALT. Use RO/AO or equivalent allele-ordered AD counts; retain missing values.
final_answer/08_joint_normalized.vcf
VCF
Complete normalized joint call set preserving record representation.
final_answer/09_annotation_resources/
resource metadata + native annotation database
Genome/annotation identity, compartment-specific genetic codes and the effect-annotation database built from the supplied resources. Retain the native database and resource metadata.
final_answer/10_annotated_nuclear.vcf
VCF
Complete nuclear consequences and sample-level genotypes. Include complete standard ANN consequence/transcript records and impact categories.
final_answer/11_annotated_mitochondrial.vcf
VCF
Complete mitochondrial consequences and sample-level genotypes. Include complete standard ANN consequence/transcript records and impact categories.
final_answer/12_effect_reports/
HTML + TSV
For each nuclear/mitochondrial compartment, a complete native effect-summary HTML and gene-statistics TSV, named nuclear.html, mitochondrial.html, nuclear.genes.tsv and mitochondrial.genes.tsv.
final_answer/13_genotype_reports/
TSV
nuclear.tsv and mitochondrial.tsv: CHROM, POS, REF, ALT, QUAL, FILTER, GEN[*].GT, GEN[*].DP, GEN[*].GQ, GEN[*].RO, GEN[*].AO, ANN[*].ALLELE, ANN[*].EFFECT, ANN[*].IMPACT, ANN[*].GENEID, ANN[*].FEATUREID, ANN[*].HGVS_C, ANN[*].HGVS_P, ANN[*].ERRORS. Keep four-sample order; multiple values within a field are comma-separated and missing values are dot.
final_answer/14_pairwise_contrasts/{sampleA}__{sampleB}.vcf
VCF
All six pairwise contrast sets with supporting genotype/quality fields, including valid empty sets.

How this task is scored

Each scored result is compared with the reference result, which the task authors made by running the IWC workflow behind this task on usegalaxy.org with the settings in the task text. A result's score is the product of the checks listed for it, each from 0 to 1. When a result is scored per sample or other unit, each copy gets its own score and the result's score is their average; if any copy is missing or cannot be read, the result scores 0. A passing result must score at least 0.80 and meet the required member checks, method and output instructions. A failing result retains its numerical score in the continuous average. The overview page describes the checks in more detail.

No.	Required result	What the task asks for	How it is scored
1	`01_read_qc/{sample}/` JSON	Complete native paired-read QC/processing report qc.json. Files: {sample}/qc.json.	Scored together with result 2, as one result: the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) values are present in the same cells (F1) values agree (Lin's concordance correlation) Scored for each of the 4 strains and averaged.
2	`02_filtered_reads/{sample}/` FASTQ	Both processed mate files. Files: {sample}/forward.fastq.gz, {sample}/reverse.fastq.gz.	Scored together with result 1, as one result.
3	`03_aligned/{sample}.bam` BAM + index	Coordinate-sorted alignments before proper-pair selection.	alignment records agree (F1) Scored for each of the 4 strains and averaged.
4	`04_proper_pairs/{sample}.bam` BAM + index	Proper-pair selected alignments.	Scored together with result 6, as one result: the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) values are present in the same cells (F1) values agree (Lin's concordance correlation) alignment records agree (F1) Scored for each of the 4 strains and averaged.
5	`05_deduplicated/{sample}/` BAM + native text metrics	deduplicated.bam and duplicates.txt with the coordinate-duplicate library metrics. Files: {sample}/deduplicated.bam, {sample}/duplicates.txt.	the required columns or fields are present (F1) every reported statistic is within 1% of the reference value alignment records agree (F1) Scored for each of the 4 strains and averaged.
6	`06_alignment_reports/{sample}/` native text	stats.txt: complete flag, sequence, quality, insert-size and coverage statistics on the proper-pair BAM. Files: {sample}/stats.txt.	Scored together with result 4, as one result.
7	`07_joint_raw.vcf` VCF	Complete unfiltered four-sample joint calls, with haploid genotypes, DP/GQ and observed counts for REF and every ALT. Use RO/AO or equivalent allele-ordered AD counts; retain missing values.	the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) values are present in the same cells (F1) values agree (Lin's concordance correlation) category labels agree (macro F1)
8	`08_joint_normalized.vcf` VCF	Complete normalized joint call set preserving record representation.	the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) values are present in the same cells (F1) values agree (Lin's concordance correlation) category labels agree (macro F1)
9	`09_annotation_resources/` resource metadata + native annotation database	Genome/annotation identity, compartment-specific genetic codes and the effect-annotation database built from the supplied resources. Retain the native database and resource metadata.	Required, not scored.
10	`10_annotated_nuclear.vcf` VCF	Complete nuclear consequences and sample-level genotypes. Include complete standard ANN consequence/transcript records and impact categories.	Scored together with result 11, as one result: the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) values are present in the same cells (F1) values agree (Lin's concordance correlation) category labels agree (macro F1)
11	`11_annotated_mitochondrial.vcf` VCF	Complete mitochondrial consequences and sample-level genotypes. Include complete standard ANN consequence/transcript records and impact categories.	Scored together with result 10, as one result.
12	`12_effect_reports/` HTML + TSV	For each nuclear/mitochondrial compartment, a complete native effect-summary HTML and gene-statistics TSV, named nuclear.html, mitochondrial.html, nuclear.genes.tsv and mitochondrial.genes.tsv.	Required, not scored.
13	`13_genotype_reports/` TSV	nuclear.tsv and mitochondrial.tsv: CHROM, POS, REF, ALT, QUAL, FILTER, GEN[].GT, GEN[].DP, GEN[].GQ, GEN[].RO, GEN[].AO, ANN[].ALLELE, ANN[].EFFECT, ANN[].IMPACT, ANN[].GENEID, ANN[].FEATUREID, ANN[].HGVS_C, ANN[].HGVS_P, ANN[*].ERRORS. Keep four-sample order; multiple values within a field are comma-separated and missing values are dot.	Required, not scored.
14	`14_pairwise_contrasts/{sampleA}__{sampleB}.vcf` VCF	All six pairwise contrast sets with supporting genotype/quality fields, including valid empty sets.	the same rows or records as the reference (F1 of their identifiers) the required columns or fields are present (F1) category labels agree (macro F1) Scored for each of the 6 strain pairs and averaged.

Data updated 27 October 2026, 08:48 EDT.