EXCON (v2.1.1)

A Nextflow pipeline for gene family EXpansion and CONtraction analysis across multiple species using CAFE5.

Given a set of genome assemblies and annotations, EXCON builds orthogroups with OrthoFinder, fits and compares multiple CAFE models to identify gene families evolving at significantly different rates, and automatically selects the best-fitting model for downstream analysis. Optionally, GO enrichment analysis can be run on expanded and contracted gene families, and on genes grouped by chromosome.

It works with any set of species that have a genome (fasta) and annotation (gff) file. (minimum of 5 species ideally up to around 30). Maximum 100 species (normally).

To run the GO annotation. You must provide a database yourself with --eggnog_data_dir, else everytime you run the pipeline, it will download the DB for you. So be careful, it is ~45GB. Please run it once, and save the DB somewhere handy to point to. This is then used to check what GO terms are associated with expanded or contracted gene sets.

Overview

The general pipeline logic is as follows:

• Downloads genome and annotation files from NCBI [NCBIGENOMEDOWNLOAD], or you provide your own.
• Unzips the files, if necessary [GUNZIP]
• Standardises and filters GFF annotations [AGAT_CONVERTSPGXF2GXF].
• Extracts longest protein [AGAT_SPKEEPLONGESTISOFORM].
• Gets the protein sequences [GFFREAD].
• Renames the genes to gene name (as some will be isoform name) RENAME_FASTA.
• Finds orthologous genes across species [ORTHOFINDER_CAFE].
• Rescales species tree branch lengths for CAFE [RESCALE_TREE].
• Prepares gene count input and runs the base CAFE model [CAFE_PREP].
• Runs two additional CAFE models in parallel for model comparison [CAFE_RUN]:
  • Gamma model with k=3 rate categories (-k 3)
  • Gamma model with per-family rates (-p -k 3)
• Compares all three CAFE models using AIC and likelihood ratio tests, selects the best fitting model [CAFE_MODEL_COMPARE].
• Plots gene family expansions and contractions for the best model [CAFE_PLOT].

Optional — GO enrichment (--run_eggnog)

• Optionally downloads the eggnogmapper database [EGGNOG_DOWNLOAD].
• Optionally assigns GO terms to genes using [EGGNOGMAPPER].
• Optionally prepares GO gene lists from the best CAFE model results [CAFE_GO_PREP].
• Optionally runs GO enrichment in parallel, one job per species/node and direction (expansion/contraction) [CAFE_GO_RUN].

Optional — chromosome GO enrichment (--chromo_go --run_eggnog)

• Optionally plots GO enrichment of genes by chromosome [CHROMO_GO].
• Optionally summarizes GO enrichment by chromosome [SUMMARIZE_CHROMO_GO].

Optional — genome quality statistics (--stats)

• Optionally describes genome assembly and annotation:
  • [BUSCO_BUSCO]: Completeness of the genome compared to expected gene set.
  • [QUAST]: Assembly contiguity statistics (N50 etc).
  • [AGAT_SPSTATISTICS]: Gene, exon, and intron statistics.

Installation

Nextflow pipelines require a few prerequisites. There is further documentation on the nf-core webpage here, about how to install Nextflow.

Prerequisites

• Docker or Singularity.
• Java and openJDK >= 8 (Please Note: When installing Java versions are 1.VERSION so Java 8 is Java 1.8).
• Nextflow >= v25.10.0.
• When running nextflow with this pipeline, ideally run NXF_VER=25.10.0 beforehand, to ensure functionality on this version.

Install

To install the pipeline please use the following commands but replace VERSION with a release.

wget https://github.com/Eco-Flow/excon/archive/refs/tags/VERSION.tar.gz -O - | tar -xvf -

or

curl -L https://github.com/Eco-Flow/excon/archive/refs/tags/VERSION.tar.gz --output - | tar -xvf -

This will produce a directory in the current directory called excon-VERSION which contains the pipeline.

Inputs

Required

• --input /path/to/csv/file - A singular csv file as input in one of the two formats stated below.

This csv can take 2 forms:

• A 2 field csv where each row is a unique species name followed by a Refseq genome reference ID (NOT a Genbank reference ID) i.e. data/input_small-s3.csv. The pipeline will download the relevant genome fasta file and annotation gff3 (or gff augustus) file.
• A 3 field csv where each row is a unique species name, followed by an absolute path to a genome fasta file, followed by an absolute path to an annotation gff3 (or gff augustus) file. Input can be gzipped (.gz) or not.

Please Note: The genome has to be chromosome level not contig level.

2 fields (Name,Refseq_ID):

Drosophila_yakuba,GCF_016746365.2
Drosophila_simulans,GCF_016746395.2
Drosophila_santomea,GCF_016746245.2

3 fields (Name,genome.fna,annotation.gff):

Drosophila_yakuba,data/Drosophila_yakuba/genome.fna.gz,data/Drosophila_yakuba/genomic.gff.gz
Drosophila_simulans,data/Drosophila_simulans/genome.fna.gz,data/Drosophila_simulans/genomic.gff.gz
Drosophila_santomea,data/Drosophila_santomea/genome.fna.gz,data/Drosophila_santomea/genomic.gff.gz

Note: Genomes should be chromosome-level, not contig-level. RefSeq IDs must be used (not GenBank IDs).

Parameters

Core options

Quality statistics (optional)

CAFE gene family evolution

Note on CAFE model selection: The pipeline runs three CAFE models — a base single-λ model, a Gamma model with k=3 rate categories, and a Gamma model with per-family rate estimation. These run in parallel and are compared using AIC. The best-fitting model is automatically selected and used for all downstream GO enrichment and plotting. Model comparison results are written to results/cafe/model_comparison/cafe_model_comparison.tsv. If model scores cannot be parsed (e.g. on very small datasets), the pipeline defaults to the base model.

GO annotation with EggNOG-mapper (optional)

Note: The EggNOG database is ~45GB. If --eggnog_data_dir is not provided, the database will be downloaded automatically on each run. We strongly recommend downloading it once and reusing it: Then pass --eggnog_data_dir /path/to/eggnog_data to the pipeline.

GO enrichment analysis (optional, requires --run_eggnog)

Resource limits

Profiles

Profiles

This pipeline is designed to run in various modes that can be supplied as a comma separated list i.e. -profile profile1,profile2.

Container Profiles

Please select one of the following profiles when running the pipeline.

• docker - This profile uses the container software Docker when running the pipeline. This container software requires root permissions so is used when running on cloud infrastructure or your local machine (depending on permissions). Please Note: You must have Docker installed to use this profile.
• singularity - This profile uses the container software Singularity when running the pipeline. This container software does not require root permissions so is used when running on on-premise HPCs or you local machine (depending on permissions). Please Note: You must have Singularity installed to use this profile.
• apptainer - This profile uses the container software Apptainer when running the pipeline. This container software does not require root permissions so is used when running on on-premise HPCs or you local machine (depending on permissions). Please Note: You must have Apptainer installed to use this profile.

Optional Profiles

• local - This profile is used if you are running the pipeline on your local machine.
• aws_batch - This profile is used if you are running the pipeline on AWS utilising the AWS Batch functionality. Please Note: You must use the Docker profile with with AWS Batch.
• test_small - This profile is used if you want to test running the pipeline on your infrastructure, running from predownloaded go files. Please Note: You do not provide any input parameters if this profile is selected but you still provide a container profile.
• test_biomart - This profile is used if you want to test running the pipeline on your infrastructure, running from the biomart input. Please Note: You do not provide any input parameters if this profile is selected but you still provide a container profile.

Custom Configuration

If you want to run this pipeline on your institute's on-premise HPC or specific cloud infrastructure then please contact us and we will help you build and test a custom config file. This config file will be published to our configs repository.

Running the Pipeline

Please note: The -resume flag uses previously cached successful runs of the pipeline.

1. Example run the full test example data:

NXF_VER=25.04.8 # Is latest it is tested on
nextflow run main.nf -resume -profile docker,test_small

Settings in test_small: input = "input_small-s3.csv"

For the fastest run use: nextflow run main.nf -resume -profile docker,test_bacteria

2. To run on your own data (minimal run), cafe only.

# NXF_VER=25.04.8  
nextflow run main.nf -resume -profile docker --input data/input_small-s3.csv

3. To run with cafe and GO analysis

# NXF_VER=25.04.8
nextflow run main.nf -resume -profile docker --input data/input_small-s3.csv --chromo_go --go_type bonferoni --stats --run_eggnog --eggnog_data_dir /path/to/eggnogdb 

Output Structure

results/
├── cafe/
│   ├── base/                        # Base single-λ CAFE model outputs
│   │   ├── Out_cafe/                # CAFE5 output files (trees, counts, probabilities)
│   │   ├── hog_gene_counts.tsv      # Filtered gene count input to CAFE
│   │   └── hog_filtering_report.tsv # Filtering report (only present if retry triggered)
│   ├── gamma/                       # Gamma k=3 model outputs
│   │   └── Out_gamma/               # CAFE5 output files
│   ├── gamma_per_family/            # Gamma per-family rate model outputs
│   │   └── Out_gamma_per_family/    # CAFE5 output files
│   └── model_comparison/
│       ├── cafe_model_comparison.tsv # AIC, delta-AIC, AIC weights, LRT results
│       ├── best_model.txt            # Name of the winning model
│       └── Significant_trees.tre    # Nexus trees with significant branches (from best model)
├── cafe_plot/
│   └── cafe_plotter/                # Expansion/contraction plots for best model
├── cafe_go/                         # GO enrichment (one job per species/node x direction)
│   ├── CAFE_summary.txt             # Summary of expansions/contractions per branch
│   ├── *_TopGo_results_ALL.tab      # TopGO results per target
│   ├── TopGO_Pval_barplot_*.pdf     # Barplots per target
│   ├── Go_summary_pos.pdf           # Summary plot across all expansions
│   ├── Go_summary_neg.pdf           # Summary plot across all contractions
│   ├── Go_summary_pos_noNode.pdf    # As above, terminal branches only
│   └── Go_summary_neg_noNode.pdf
├── chromo_go/                       # [optional] GO enrichment by chromosome
│   ├── *.pdf                        # Per-chromosome GO plots
│   └── summary/                     # Summarized results across chromosomes
├── eggnogmapper/
│   └── go_files/                    # Per-species GO annotation files
├── gffread/
│   └── *.fasta                      # Protein sequences per species
├── ncbigenomedownload/
│   ├── *.fna.gz                     # Downloaded genome assemblies
│   └── *.gff.gz                     # Downloaded annotations
├── orthofinder_cafe/
│   └── ortho_cafe/                  # OrthoFinder results including species tree
├── busco/                           # [optional, --stats] BUSCO completeness results
├── agat/                            # [optional, --stats] AGAT annotation statistics
├── quast/                           # [optional, --stats] Assembly contiguity statistics
└── pipeline_info/
    ├── execution_report_*.html      # Nextflow execution report
    ├── execution_timeline_*.html    # Per-process timeline
    ├── execution_trace_*.txt        # Per-task resource usage
    ├── pipeline_dag_*.html          # Pipeline DAG diagram
    └── software_versions.yml        # Versions of all tools used

Citation

This pipeline is published on Workflowhub using the nf-core template. If you use this pipeline in you work, the following citations are essential:

excon: Wyatt, C. (2026). Gene EXpansion and CONtraction analysis pipeline. WorkflowHub. https://doi.org/10.48546/WORKFLOWHUB.WORKFLOW.2141.5

nf-core: Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen. Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.

If you used any of these tools within the pipeline, you must also cite:

CAFE: Fábio K Mendes, Dan Vanderpool, Ben Fulton, Matthew W Hahn, CAFE 5 models variation in evolutionary rates among gene families, Bioinformatics, 2020; btaa1022, https://doi.org/10.1093/bioinformatics/btaa1022

Orthofinder: Emms, D.M. and Kelly, S. (2019) OrthoFinder: phylogenetic orthology inference for comparative genomics. Genome Biology 20:238

AGAT: Dainat J. 2022. Another Gtf/Gff Analysis Toolkit (AGAT): Resolve interoperability issues and accomplish more with your annotations. Plant and Animal Genome XXIX Conference. https://github.com/NBISweden/AGAT.

eggNOG-mapper (if used): Carlos P Cantalapiedra, Ana Hernández-Plaza, Ivica Letunic, Peer Bork, Jaime Huerta-Cepas, eggNOG-mapper v2: Functional Annotation, Orthology Assignments, and Domain Prediction at the Metagenomic Scale, Molecular Biology and Evolution, Volume 38, Issue 12, December 2021, Pages 5825–5829, https://doi.org/10.1093/molbev/msab293

A full list of tools and their versions are found in the software_versions.yml in the results/pipeline_info. So ensure to look here for any additional tools you need to cite.

For the --stats module, you would need to cite BUSCO, Quast, AGAT

Contact Us

If you need any support do not hesitate to contact us at any of:

c.wyatt [at] ucl.ac.uk

ecoflow.ucl [at] gmail.com