Galaxy agent benchmark

Galaxy Agents Benchmarking

Archived Jul 6, 2026

Legacy experimentThis page preserves the Jul 6 Codex four-condition results. Current item pages are maintained separately.

Without skills

Galaxy-API code without skills 84.0% 126/150 correct
replicate 1 82.0% 41/50 replicate 2 84.0% 42/50 replicate 3 86.0% 43/50
Open-ended code without skills 81.3% 122/150 correct
replicate 1 76.0% 38/50 replicate 2 86.0% 43/50 replicate 3 82.0% 41/50

With skills

Galaxy-API code with skills 92.7% 139/150 correct
replicate 1 94.0% 47/50 replicate 2 94.0% 47/50 replicate 3 90.0% 45/50
Open-ended code with skills 90.0% 135/150 correct
replicate 1 90.0% 45/50 replicate 2 90.0% 45/50 replicate 3 90.0% 45/50

Item results

Each cell shows pass count across three independent replicates. Open an item for per-replicate answers, Hugging Face traces, and Galaxy history links.

Item Question Galaxy + skillsGalaxy no skillsOpen-ended code + skillsOpen-ended code no skills
bix-6-q4 What is the Spearman correlation coefficient between the replicate MAGeCK P-values for chronic round 1? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-11-q1 What is the difference between median treeness values for fungi versus animals? Report the difference as a decimal proportion (not percentage). 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-11-q2 What percentage of fungal genes have treeness values above 0.06? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-12-q2 What is the median percentage of parsimony informative sites across fungal gene alignments? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-12-q4 What is the Mann-Whitney U statistic when comparing parsimony informative site percentages between animals and fungi? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-12-q5 What is the maximum number of parsimony informative sites in any animal gene alignment? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-12-q6 What is the Mann-Whitney U statistic when comparing raw parsimony informative site counts between animals and fungi? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-14-q1 In the BLM mutation carrier cohort, what fraction of coding variants with a variant allele frequency (VAF) below 0.3 are annotated as synonymous? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-16-q1 In the provided data, what gene symbol has the strongest negative Spearman correlation between its expression and essentiality? 3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-16-q3 How many genes in the provided data show strong positive Spearman correlation between expression and essentiality across cell lines, with a correlation coefficient >= 0.6? 3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
bix-16-q4 Using a Spearman Rank correlation test and Benjamini-Hochberg multi-test correction, what percentage of genes show a statistically significant correlation between expression and essentiality, in either direction? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-17-q2 Among patients classified as BLM mutation carriers, what is the median number of somatic CHIP variants - defined as variants with a Variant Allele Frequency (VAF) < 0.3? Prior to analysis, exclude all variants that are... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-18-q1 In a P. aeruginosa swarming analysis, what is the mean circularity measurement for the genotype showing the largest mean area? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-18-q3 What is the approximate percent reduction in mean colony area for the ΔlasR mutant compared to wildtype? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-20-q3 Among samples with a 'Carrier' BLM mutation status, what proportion of somatic CHIP variants (VAF < 0.3) with ClinVar classifications are classified as Benign or Likely Benign? The proportion of somatic CHIP variants sh... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-22-q1 After calculating Pearson correlations between gene length and average gene expression levels for protein-coding genes, in which immune cell type (CD4, CD8, CD14, or CD19) is the correlation the weakest (lowest absolute... 3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-22-q4 What is the Pearson correlation coefficient between gene length and mean expression (across samples) for expressed protein-coding genes (total counts ≥ 10) in CD14 immune cells? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-24-q2 Using differential expression analysis (padj < 0.05, |log2FC| > 0.5) and GO Biological Process enrichment analysis, determine whether upregulation or downregulation of genes primarily drives the metabolic effects of CBD... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-26-q3 Among significantly upregulated genes in the iron-depleted glucose condition (FeMinus), how many genes contribute to the enrichment of the 'ABC transporters' term after conducting KEGG pathway enrichment analysis? Use a... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-26-q5 In P. aeruginosa KEGG pathway enrichment analysis, how many pathways are significantly enriched under iron-depleted conditions but show no significant enrichment under innate media conditions? Perform KEGG pathway enric... 0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-27-q5 After performing principal component analysis (PCA) with 100 components on the log10-transformed gene expression matrix (with pseudocount of 1, samples as rows and genes as columns), what percentage of the total varianc... 3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-28-q3 Using PhyKIT to calculate long branch scores, what is the median long branch score for the fungal gene 996662at2759? 3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-30-q3 What is the ratio of significant (p <= 0.05) differentially expressed miRNAs between patient and control groups identified following Bonferroni correction compared to Benjamini-Yekutieli correction? Provide your answer... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
1/3
rep 1rep 2rep 3
bix-31-q2 Using the batch-corrected read counts with batch as a covariate in the design formula, perform a sex-specific (M vs F) differential expression analysis (abs lfc>0.5, basemean>10 and using lfc shrinkage), what is the log... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-32-q2 Using KEGG over-representation analysis (ORA), how many pathways are significantly enriched (absolute value of lfc > 1.5) in the same direction for all three mutant strains (97, 98, and 99), relative to wildtype? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
bix-34-q2 What is the median patristic distance for the fungal gene 981902at2759? 3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-34-q5 What is the ratio of median mean patristic distances between fungi and animals? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-35-q1 Calculate the evolutionary rate for the BUSCO gene 156083at2759 using PhyKIT's evoluionary_rate function. What is the gene's evolutionary rate in animals? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-35-q2 What is the Mann-Whitney U statistic value for the comparison of evolutionary rates between animals and fungi across all genes? Use PhyKIT's evo_rate function to calculate evolutionary rates. 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-37-q1 Based on the proteomics data, what is the fold change in ENO1 protein abundance between tumor and normal samples? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-37-q4 What is the log2 fold change value (rounded to 2 decimal places) for ENO1 when comparing tumor samples to normal samples in the proteomics dataset? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-38-q1 What is the fold-change in median tree length between fungi and animals after calculating tree lengths with PhyKIT? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-41-q5 In a bacterial swarming assay compared mixed cultures of Strain 287 and Strain 98 at various ratios, which ratio produced colonies with a mean area and circularity value that is most similar to the mean area and circula... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-43-q2 What is the odds ratio for p53-mediated cell cycle gene regulation in an enrichment analysis using gseapy with Reactome_2022 using all statistically differentially expressed genes from a pydeseq2 analysis that includes... 2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-43-q4 After performing differential gene expression analysis comparing CBD/cisplatin treatment vs DMSO control (using padj < 0.05 and |log2FC| ≥ 0.5), followed by enrichment analysis using gseapy with Reactome_2022 gene sets,... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-45-q1 What is the p-value from a Mann-Whitney U test comparing RCV scores between animal and fungal orthologs using PhyKIT's rcv function? 0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-46-q4 What is the log2 fold change value rounded to 2 decimal points for the phenazine pathway gene PA14_35160 in the ΔrhlI mutant? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-47-q3 Which gene has the most non-reference variants in the oldest male carrier? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-49-q4 What is the total number of significantly differentially expressed genes (padj < 0.05) when performing DESeq2 differential gene expression analysis with apeglm LFC shrinkage, comparing ASXL1 mutation (disease) vs contro... 1/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
1/3
rep 1rep 2rep 3
bix-51-q2 What is the Akaike Information Criterion (AIC) for the logistic regression model that uses BMI as the sole predictor of treatment response (efficacy PR)? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-51-q8 In a simple logistic regression model assessing age and camrelizumab treatment response, what is the coefficient estimate (change in log-odds) for age? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-52-q2 What is the mean of per-chromosome densities (CpGs per base pair) for filtered (>90% or <10% methylation) unique age-related CpGs across all chromosomes (only including those with at least one filtered CpG) in the Jackd... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-52-q6 Which chromosome in the Jackdaw genome shows the highest density of age-related CpG sites? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-52-q7 How many individual methylation measurements (rows) are removed when filtering out measurements that do not show >90% or <10% methylation in the Zebra Finch dataset? 3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-53-q2 Perform differential expression analysis on the RNA-seq dataset (KL1-3 vs WL1-3) using criteria p < 0.05, |log2FC| > 1, baseMean > 10 with LFC shrinkage. Then repeat the analysis excluding the third replicates (KL3, WL3... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-53-q5 Using the gseapy package and WikiPathways_2019_Mouse as a reference library, what is the fraction of oxidative stress-related pathways (pathway name contains "oxidative") among the top 20 enriched pathways based on all... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-54-q7 What is the maximum colony area predicted at the optimal frequency (proportion of strain 287 in the mixtures) according to the best-fitting model among quadratic, cubic and natural spline (df=4) models? Use R for model... 1/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
bix-55-q1 How many single-copy orthologs are complete and present in all four proteomes after BUSCO analysis with the eukaryota_odb10 lineage? 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
2/3
rep 1rep 2rep 3
bix-61-q2 What is the average coverage depth across the entire reference genome for the MDR sample (SRR35233585) rounded to 4 decimal places? Map the trimmed reads with BWA-MEM (read group @RG\tID:SRR35233585\tSM:SRR35233585\tPL:... 3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
bix-61-q5 What is the Ts/Tv ratio for the MDR sample (SRR35233585) rounded to 2 decimal places? 3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3
3/3
rep 1rep 2rep 3
0/3
rep 1rep 2rep 3