V-pipe configuration

Type: object

The V-pipe workflow can be customized through the configuration file config.yaml or config.json or, for backward compatibility with the legacy INI-style format used in V-pipe v1.x/2.x, vpipe.config. This configuration file is a text file written using a basic structure composed of sections, properties and values. When using YAML or JSON format use these languages associative array/dictionaries in two levels for sections and properties. When using the older INI format, sections are expected in squared brackets, and properties are followed by corresponding values.

Further more, it is possible to specify additional options on the command line using Snakemake’s --configfile to pass additional YAML/JSON configuration files, and/or using Snakemake’s --config to pass sections and properties in a YAML Flow style/JSON syntax.

The order of precedence is:
command line options (--config, --configfile) >> default configuration file (config/config.yaml or config.yaml) >> legacy configuration INI (vpipe.config) >> Virus-specific base config (virus_based_config) >> default values

Example: For instance, we suggest providing as input a tabular file specifying sample unique identifiers (e.g., patient identifiers), and dates for different sequencing runs related to the same patient. The name of this file (here, samples.tsv) can be provided by specifying the section as input and the property as samples_file, as follows in the example below.

In this document, we provide a comprehensive list of all user-configurable options stratified by sections.


Example:

input:
  samples_file: samples.tsv

Type: object Default: {}

This section of the configuration provides general options that control the overall behavior of the pipeline.

Type: string Default: ""

We provide virus-specific base configuration files which contain handy defaults for, e.g., HIV and SARS-CoV-2. Check the git repository’s config subdirectory to learn about them.


Examples:

hiv
...
sars-cov-2
...

Type: enum (of string) Default: "ngshmmalign"

There are three options for mapping reads, either using ngshmmalign, BWA MEM (bwa) 1, or Bowtie 2 (bowtie) 2. To use a different aligner than the default, indicate which aligner you want to use by setting the property aligner.

Note: Some virus-specific base configuration specified in virus_base_config might change this option’s default to a more appropriate aligner for that virus, e.g., depending on its usual diversity and mutation rate.
You are still free to override that default in your configuration shall the need arise.


  1. Li, H. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. 2013. 

  2. Langmead, B. and Salzberg, S. Fast gapped-read alignment with Bowtie 2. Nature Methods. 2012. 

Must be one of:

  • "ngshmmalign"
  • "bwa"
  • "bowtie"

Example:

bowtie
...

Type: enum (of string) Default: "shorah"

There are two options available for calling single nucleotide variants, either using ShoRAH (shorah) 1 or LoFreq (lofreq) 2. ShoRAH is used by default. If you prefer to use LoFreq, then indicate so in the configuration file as in the example


  1. Zagordi, O. et al. ShoRAH: estimating the genetic diversity of a mixed sample from next-generation sequencing data. BMC Bioinformatics. 2011. 

  2. Wilm, A. et al. LoFreq: A sequence-quality aware, ultra-sensitive variant caller for uncovering cell-population heterogeneity from high-throughput sequencing datasets. Nucleic Acids Res. 2012. 

Must be one of:

  • "shorah"
  • "lofreq"

Example:

lofreq
...

Type: enum (of string) Default: "savage"

There are three options available for haplotype reconstruction, namely SAVAGE 1, HaploClique 2 or PredictHaplo 3. SAVAGE is used by default. If you wish to use HaploClique, then indicate it in the configuration file as in the example.


  1. Baaijens, J. A. et al., De novo assembly of viral quasispecies using overlap graphs. Genome Res. 2017. 

  2. Töpfer, A. et al. Viral quasispecies assembly via maximal clique finding. PLOS Computational Biology. 2014. 

  3. Prabhakaran, S. et al. HIV haplotype inference using a propagating dirichlet process mixture model. IEEE/ACM transactions on computational biology and bioinformatics 11.1. 2013. 

Must be one of:

  • "savage"
  • "haploclique"
  • "predicthaplo"

Example:

haploclique
...

Type: integer Default: 1

This option should be used to specify the default number of threads for all multi-threaded rules. That is, unless the number of threads is specified for each rule, this value is set as default.


Example:

4
...

Type: enum (of string) Default: "md5"

Sets the algorithm to be used when computing checksums for uploadable data.

Must be one of:

  • "md5"
  • "sha1"
  • "sha256"
  • "sha224"
  • "sha384"
  • "sha512"
  • "xxh64"
  • "xxh32"
  • "xxh128"

Example:

sha256
...

Type: string Default: ""

Some step of V-pipe produce temporary files such as, e.g., decompressed intermediate - i.e. files which aren’t kept long-term but are deleted after all steps that needed them have finished. By default, these files are written in the output data directory. This option, makes it is possible to write them in a different directory instead. Use this option to, e.g., leverage a faster cluster-local storage or avoid wasting backup space on a snapshotted storage. You might want to consult the documentation provided by your HPC.


Examples:

temp
...
/cluster/scratch
...

Type: object Default: {}

Properties in this section of the configuration control the input of the pipeline.

Type: string Default: "samples/"

The input file for the workflow will be searched in this directory.

V-pipe expects the input samples to be organized in a two-level directory hierarchy.

  • The first level can be, e.g., patient samples or biological replicates of an experiment.
  • The second level can be, e.g., different sampling dates or different sequencing runs of the same sample.
  • Inside that directory, the sub-directory raw_data holds the sequencing data in FASTQ format (optionally compressed with GZip).

For example:

samples
├── patient1
│   ├── 20100113
│   │   └──raw_data
│   │      ├──patient1_20100113_R1.fastq
│   │      └──patient1_20100113_R2.fastq
│   └── 20110202
│       └──raw_data
│          ├──patient1_20100202_R1.fastq
│          └──patient1_20100202_R2.fastq
└── patient2
    └── 20081130
        └──raw_data
           ├──patient2_20081130_R1.fastq.gz
           └──patient2_20081130_R2.fastq.gz

Examples:

tests/data/hiv/
...
tests/data/sars-cov-2/
...

Type: boolean Default: true

Indicate whether the input sequencing reads correspond to paired-end reads.

Paired-ended reads need to be in split files with _R1 and _R2 suffixes:

raw_data
├──patient2_20081130_R1.fastq.gz
└──patient2_20081130_R2.fastq.gz

Example:

false
...

Type: string Default: ""

V-pipe expects paired-end reads to be in files that end in _R1 and _R2 exactly right before the file extension, e.g., _R1.fastq.gz, because this is how the workflow finds and recognizes them.

But Illumina’s bcl2fastq demultiplexer might introduce additional strings, e.g., _R2_001.fast.gz or, depending on its mismatches settings, e.g., _R2_001_MM_1.fast.gz. Use this options to specify anything which should go between the _R1 and _R2 endings and the file extension.


Examples:

_001
...
_001_MM_1
...

Type: string Default: "config/samples.tsv"

File containing sample unique identifiers and dates as tab-separated values, e.g.,

patient1    20100113
patient1    20110202
patient2    20081130

Here, we have two samples from patient 1 and one sample from patient 2. By default, V-pipe searches for a file named samples.tsv, if this file does not exist, a list of samples is built by globbing datadir directory contents.

Optionally, the samples file can contain a third column specifying the read length. This is particularly useful when samples are sequenced using protocols with different read lengths.

Standardized Snakemake workflows place their tables inside the config/ subdirectory, but using this options you can specify alternate locations, e.g., the current working directory (as done in legacy V-pipe v1.x/2.x).


Example:

samples.tsv
...

Type: integer Default: 250

Default for those samples whose read length isn’t specified explicitly in the optional third column of the samples.tsv table.


Example:

'100'

Type: number Default: 0.8

Using this parameter, the user can specify the read-length threshold that should be applied during the quality trimming as a percentage (0 < trim_percent_cutoff < 1).


Example:

0.9
...

Type: string Default: "results/cohort_consensus.fasta"

Reference sequence to use for the alignment step

Note: The virus-specific base configuration specified in general => virus_base_config will most likely change this option’s default to a reference for that virus.
You are still free to override that default in your configuration shall the need arise.


Examples:

resources/hiv/HXB2.fasta
...
resources/sars-cov-2/NC_045512.2.fasta
...

Type: string Default: ""

A directory containing gff files that can be optionally used to annotate the reference genome in the visualization

Note: The virus-specific base configuration specified in general => virus_base_config will most likely change this option’s default.
You are still free to override that default in your configuration shall the need arise.


Examples:

resources/hiv/gffs/
...
resources/sars-cov-2/gffs/
...

Type: string Default: ""

An associative array providing user-friendly name to display for each annotation .gff file in the gff_directory

Note: The virus-specific base configuration specified in general => virus_base_config will most likely change this option’s default.
You are still free to override that default in your configuration shall the need arise.


Examples:

resources/hiv/metainfo.yaml
...
resources/sars-cov-2/metainfo.yaml
...

Type: string Default: ""

A table with primers to display on the visualization

Note: The virus-specific base configuration specified in general => virus_base_config will most likely change this option’s default.
You are still free to override that default in your configuration shall the need arise.


Example:

resources/sars-cov-2/primers/nCoV-2019.tsv
...

Type: string Default: ""

A FASTQ file with sequences of interest

Note: These sequences are used, together with the consensus sequence, to build a phylogenetic tree.


Example:

resources/sars-cov-2/phylogeny/selected_covid_sequences.fasta
...

Type: object Default: {}

Properties in this section of the configuration control the output of the pipeline.

Type: string Default: "results"

The workflow will write its output files into this directory. This will follow the same structure as for the input.

For each sample, V-pipe produces several output files that are located in the corresponding sample-specific directory. First, the alignment file and consensus sequences are located in the alignments and references subdirectories, respectively. Second, output files containing SNVs and viral haplotypes are located in the variants subdirectories.

Using the sample example as in the input section, the output files for the two patient samples will be located in the following subdirectories:

results
├──patient1
│  ├──20100113
│  │  ├──alignments
│  │  |  └──REF_aln.bam
│  │  ├──references
|  |  |  ├──consensus.bcftools.fasta
|  |  |  ├──ref_ambig.fasta
|  |  |  └──ref_majority.fasta
|  |  └──variants
|  |     ├──SNVs
|  |     |  └──snvs.vcf
|  |     └──global
|  |        └──contigs_stage_c.fasta
│  └──20110202
│     ├──alignments
│     |  └──REF_aln.bam
│     ├──references
|     |  ├──consensus.bcftools.fasta
|     |  ├──ref_ambig.fasta
|     |  └──ref_majority.fasta
|     └──variants
|        ├──SNVs
|        |  └──snvs.vcf
|        └──global
|           └──contigs_stage_c.fasta
└─patient2
  ├──alignments
  |  └──REF_aln.bam
  ├──references
  |  ├──consensus.bcftools.fasta
  |  ├──ref_ambig.fasta
  |  └──ref_majority.fasta
  └──variants
     ├──SNVs
     |  └──snvs.vcf
     └──global
        └──contigs_stage_c.fasta
  • Standardized Snakemake workflows place their output in a results subdirectory
  • If your prefer the output written, e.g., to the same samples/ subdirectory as the input (as used to be done in legacy V-pipe v1.x/2.x), you can use this options you can specify alternate target locations.

Example:

samples
...

Type: string Default: ""

In addition, V-pipe can optionally generate a few cohort-wide results, such as a current cohort consensus fasta file, or a TSV file containing the frequencies of all minor alleles that differ from the consensus among analyzed samples.
By default, these output files are located at the base of the output datadir, outside of the two-level per sample structure:

results
├──minority_variants.tsv
├──cohort_consensus.fasta
├──patient1
│  ├──20100113
│  │  ├──alignments

If you prefer instead, e.g., such cohort-wide results behind written in a subdirectory of the working directory at the same level as the datadirs, you can use this options you can specify alternate subdirectory relative to the datadir property. (Use .. prefix if you want instead your cohort-wide results to be in a directory at the sample level as samples/ and results/. See the example below to recreate the variants/ directory used by legacy V-pipe v1.x/2.x).


Example:

../variants
...

Type: boolean Default: false

V-pipe can produce several outputs to assess the quality of the output of its steps, e.g., checking whether a sample’s consensus sequence generated by bctfools does result in frameshifting indels and writing a report in sample’s …/references/frameshift_deletions_check.tsv. Such reports can be useful when submitting sequences to GISAID.

This option turns on such QA features.


Example:

true
...

Type: boolean Default: false

This option selects whether the SNV caller step should be executed and its output written to each sample’s …/variants/SNVs/snvs.csv.


Example:

true
...

Type: boolean Default: false

This option activates local haplotype reconstruction (only available when using ShoRAH).


Example:

true
...

Type: boolean Default: false

This option turns on global haplotype reconstruction.


Example:

true
...

Type: boolean Default: false

This option selects whether to generate HTML visualization of the SNVs in each sample’s …/visualization/index.html.


Example:

true
...

Type: boolean Default: false

This option turns on the computation of diversity measures in each sample.


Example:

true
...

Type: boolean Default: false

This option turns on dehumanization of the raw reads (i.e. removal of host’s reads) and generates the file dehuman.cram. This is useful to prepare raw reads for upload on public databases such as, e.g. ENA (European Nucleotide Archive).

This only applies to the upload and does not affect the main workflow.


Example:

true
...

Type: boolean Default: false

This option can be used for assistance in incremental upload of data. See section upload for an example.


Example:

true
...

Type: object Default: {}

The path to the different software packages can be specified using this section.

It is especially useful when dependencies are not obtained via conda such as VICUNA, and when the software packages are not in the PATH.

Note we strongly recommend to use conda environments, by adding the --use-conda flag to the V-pipe execution command, e.g. ./vpipe --use-conda. If you prefer to use your own installations, this section allows you to specify the location of the executables.


Example:

bwa: /path/to/bwa
haploclique: /path/to/haploclique

Type: string Default: "gunzip"

Type: string Default: "prinseq-lite.pl"

Type: string Default: "fastqc"

Type: string Default: "vicuna"

Due to a special license, VICUNA is not available from bioconda and must be installed from its original website.
Use this option to specify where you have installed its executable.

Type: string Default: "InDelFixer"

Type: string Default: "ConsensusFixer"

Type: string Default: "picard"

Type: string Default: "bwa"

Type: string Default: "bowtie2-build"

Type: string Default: "bowtie2"

Type: string Default: "samtools"

Type: string Default: "extract_consensus"

Type: string Default: "matcher"

Type: string Default: "frameshift_deletions_checks"

Type: string Default: "mafft"

Type: string Default: "ngshmmalign"

Type: string Default: "convert_reference"

Type: string Default: "extract_seq"

Type: string Default: "coverage_stats"

Type: string Default: "remove_gaps_msa"

Type: string Default: "aln2basecnt"

Type: string Default: "gather_coverage"

Type: string Default: "minority_freq"

Type: string Default: "extract_coverage_intervals"

Type: string Default: "shorah shotgun"

Type: string Default: "lofreq"

Type: string Default: "bcftools"

Type: string Default: "haploclique"

Type: string Default: "compute_mds"

Type: string Default: "savage"

Type: string Default: "predicthaplo"

Type: object Default: {}

Type: integer Default: 32

Type: integer Default: 60

Type: object Default: {}

Type: integer Default: 4096

Type: integer Default: 20

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/preprocessing.yaml"

Type: string Default: "-ns_max_n 4 -min_qual_mean 30 -trim_qual_left 30 -trim_qual_right 30 -trim_qual_window 10"

We use software PRINSEQ 1 for quality control. By default, we use options -ns_max_n 4 -min_qual_mean 30 -trim_qual_left 30 -trim_qual_right 30 -trim_qual_window 10, which indicates to trim reads using a sliding window with size 10 bp, and trim bases if their quality scores are less than 30. Additionally, reads are filtered out if the average quality score is below 30 and if they contain more than 4 N’s. The user can choose to overwrite the default settings or use additional parameters by using the property extra. E.g., if many reads are filtered out in this step, the user can choose to lower the quality threshold as indicated in the example.
Please do not modify PRINSEQ options -out_format, -out_good, nor -min_len. Instead of using -min_len to define threshold on the read length after trimming, use input => trim_percent_cutoff.


  1. Schmieder, R. and Edwards, R. Quality control and preprocessing of metagenomic datasets. Bioinformatics. 2011. 


Example:

-ns_max_n 4 -min_qual_mean 20 -trim_qual_left 20 -trim_qual_right 20 -trim_qual_window
  10
...

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/fastqc.yaml"

Type: integer Default: 6

Type: boolean Default: false

Type: object Default: {}

NOTE The conda environment for this rule doesn’t work properly. The package on the bioconda channel, mvicuna, is slightly different from VICUNA and it has different command-line arguments. Moreover, VICUNA and mvicuna are no longer maintained. In the future, this rule will be deprecated.

Type: integer Default: 1000

Type: integer Default: 600

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/initial_vicuna.yaml"

Type: object Default: {}

NOTE Obtaining a initial reference de novo is implemented for more than one sample.

Type: integer Default: 10000

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/initial_vicuna_msa.yaml"

Type: object Default: {}

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 1435

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/hmm_align.yaml"

Type: boolean Default: false

This option is useful for debugging purposes.


Example:

true
...

Type: string Default: ""

Pass additional options to run ngshmmalign

V-pipe uses option -R <path/to/initial_reference>, thus option -r arg is not allowed. Also, instead of passing -l via the property extra, set leave_msa_temp to True. Lastly, please do not modify options -o arg, -w arg, -t arg, and -N arg. These are already managed by V-pipe.

Type: object Default: {}

Type: integer Default: 5000

Type: integer Default: 30

Type: string Default: "{VPIPE_BASEDIR}/envs/sam2bam.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/bwa_QA.yaml"

Type: string Default: ""

Panel of diverse references against which to align reads as a QA step

Note: The virus-specific base configuration specified in general => virus_base_config will most likely change this option’s default.
You are still free to override that default in your configuration shall the need arise.


Example:

resources/hiv/5-Virus-Mix.fasta
...

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: string Default: "HXB2:6614-6812,7109-7217,7376-7478,7601-7634"

Type: object Default: {}

This rule takes all previously aligned reads by hmm_align. Therefore, resources should be allocated accordingly.

Type: integer Default: 10000

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/msa.yaml"

Type: object Default: {}

Type: integer Default: 8000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/bwa_align.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/bwa_align.yaml"

Type: string Default: ""

With property extra, users can pass additional options to run BWA MEM. For more details on BWA MEM configurable options refer to the software documentation.

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/bowtie_align.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/bowtie_align.yaml"

Type: enum (of string) Default: "--phred33"

Indicate if qualities are Phred+33 (default) or Phred+64 (--phred64).

Must be one of:

  • "--phred33"
  • "--phred64"

Example:

--phred64
...

Type: string Default: "--local --sensitive-local"

Specify Bowtie 2 presets.

Type: integer

Type: string Default: ""

Pass additional options to run Bowtie 2. V-pipe handles the input and output files, as well as the reference sequence. Thus, do not modify these options
For more details on Bowtie 2 configurable options refer to the software documentation.

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: integer Default: 50

Minimum read depth for reporting variants per locus.

Type: integer Default: 5

Read count below which ambiguous base ’n’ is reported.

Type: integer Default: 15

Minimum phred quality score for a base to be included.

Type: number Default: 0.05

Minimum frequency for an ambiguous nucleotide.

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/bcftools.yaml"

Type: integer Default: 10000

Type: integer Default: 10

Type: number Default: 0.05

Type: object Default: {}

Type: integer Default: 4096

Type: integer Default: 30

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/consseq_qa.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 30

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: string Default: ""

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 30

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: object Default: {}

Type: integer Default: 1250

Type: integer Default: 30

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: object Default: {}

Type: integer Default: 1000

Type: integer Default: 235

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: integer Default: 100

Minimum read depth for reporting variants per locus.


Example:

50
...

Type: boolean Default: false

Output a numpy array file containing frequencies of all bases, including gaps and also the most abundant base across samples.


Example:

true
...

Type: object Default: {}

Type: integer Default: 1000

Type: integer Default: 30

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: boolean Default: false

Construct intervals based on overlapping windows of the read alignment. By default, regions with high coverage are built based on the position-wise read depth.


Example:

true
...

Type: integer Default: 50

Minimum read depth. A region spanning the reference genome is returned if coverage is set to 0.


Example:

0
...

Type: boolean Default: true

Indicate whether to apply a more liberal shifting on intervals’ right-endpoint.


Example:

false
...

Type: object Default: {}

Type: integer Default: 10000

Type: integer Default: 2880

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/snv.yaml"

Type: boolean Default: true

Indicate whether to use the cohort-consensus sequence from the analyzed samples (output from minor_variants rule located in the cohort-wide output results/cohort_onsensus.fasta) or the reference sequence by setting this option to False.


Example:

false
...

Type: number Default: 0.1

Hyperparameter used for instantiating a new cluster.

Type: boolean Default: false

Ignore SNVs adjacent to indels.

Type: number Default: 0.9

Type: integer Default: 0

Omit windows with coverage less than this value.


Example:

50
...

Type: integer Default: 3

ShoRAH performs local haplotype reconstruction on windows of the read alignment. The overlap between these windows is defined by the window shifts. By default, it is set to 3, i.e., apart from flanking regions each position is covered by 3 windows.

Type: boolean Default: false

Indicate whether to move files produced in previous/interrupted runs to subdirectory named old


Example:

true
...

Type: string Default: ""

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 20

Type: string Default: "{VPIPE_BASEDIR}/envs/lofreq.yaml"

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 60

Type: string Default: "{VPIPE_BASEDIR}/envs/lofreq.yaml"

Type: boolean Default: true

Indicate whether to use the cohort-consensus sequence from the analyzed samples (output from minor_variants rule located in the cohort-wide output results/cohort_onsensus.fasta) or the reference sequence by setting this option to False.


Example:

false
...

Type: string Default: ""

Pass additional options to run lofreq call

Type: object Default: {}

Type: integer Default: 1000

Type: integer Default: 60

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: integer Default: 5

Type: object Default: {}

Type: string Default: "{VPIPE_BASEDIR}/envs/sam2bam.yaml"

Type: object Default: {}

NOTE This rule only works in Linux.

Type: integer Default: 10000

Type: integer Default: 1435

Type: integer

Type: string Default: "{VPIPE_BASEDIR}/envs/savage.yaml"

Type: integer Default: 20

Size of the batches of reads to be processed by SAVAGE. It is recommended that 500 < coverage/split < 1000.

Type: object Default: {}

Type: integer Default: 10000

Type: integer Default: 1435

Type: string Default: "{VPIPE_BASEDIR}/envs/haploclique.yaml"

Type: boolean Default: true

If set to True (default) a predefined set of parameter values is used for drawing edges between reads in the read graph.

Type: boolean Default: true

Singletons are defined as proposed haplotypes which are supported by a single read. If this property is set to True, singletons are discarded.

Type: boolean Default: true

If set to True (default) probability of the overhangs is ignored.

Type: integer Default: 3

Sets a threshold to limit the size of cliques.

Type: integer Default: 10000

Indicates the maximum number of clique to be considered in the next iteration.

Type: string Default: ""

Additional parameters to be passed to haploclique.

Warning: this won’t overwrite the other options (e.g. clique_size_limi and max_num_cliques should still be set via their own respective properties, do not pass parameters --limit_clique_size= nor --max_cliques= via this extra property).

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/smallgenomeutilities.yaml"

Type: integer Default: 0

Use to specify a region of interest.

Type: integer Default: -1

Use to specify a region of interest.


Examples:

9719
...
29836
...

Type: string Default: ""

When the ground truth is available (e.g., simulation studies), a multiple sequence alignment of types making up the population can be provided, and additional checks are performed.

Type: object Default: {}

Type: integer Default: 10000

Type: integer Default: 1435

Type: integer

Type: integer Default: 0

Type: string Default: "{VPIPE_BASEDIR}/envs/predicthaplo.yaml"

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/visualization.yaml"

Type: object Default: {}

Type: integer Default: 2000

Type: integer Default: 235

Type: string Default: "{VPIPE_BASEDIR}/envs/diversity_measures.yaml"

Type: object Default: {}

Type: string Default: "{VPIPE_BASEDIR}/envs/dehuman.yaml"

Type: integer Default: 4096

Type: integer Default: 235

Type: integer Default: 4

Type: string Default: "references/human.fa.gz"

Host’s genome used to remove reads (e.g. human genome)

Note: if this file is absent, it is possible to fetch it from a remote server, see property ref_host_url below.


Example:

/cluster/project/igenomes/Homo_sapiens/NCBI/GRCh38/Sequence/BWAIndex/genome.fa
...

Type: string Default: "http://ftp.ensembl.org/pub/current_fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz"

If the host’s genome specified in property ref_host isn’t present, fetch it from a remote server.

Note remember to set aside enough memory for the indexing rule, see section ref_bwa_index property mem.


Examples:

http://ftp.ensembl.org/pub/release-105/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz
...
https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.29_GRCh38.p14/GCA_000001405.29_GRCh38.p14_genomic.fna.gz
...

Type: boolean Default: false

Indicate whether to store the host-aligned reads in a CRAM file …/alignments/host_aln.cram.


Example:

true
...

Type: boolean Default: false

Use this option when generating dehumanized raw reads (dehuman.cram) on old samples that have already been processed in the past - a catch up.

Normally, removing host-mapping reads requires analyzing reads which were rejected by V-pipe’s main processing (as specified in section general, property aligner). But this output is considered temporary and will get deleted by Snakemake once the processing of a sample has finished. To generate dehuman.cram V-pipe would need to run the aligner again, which will both regenerate the data necessary for this output but also generate a new alignment which will trigger the whole workflow again.
Use this property catchup to only generate the input necessary for dehuman.cram, leaving untouched the alignment and everything else that has already been processed.


Example:

true
...

Type: object Default: {}

Type: integer Default: 256

Type: integer Default: 60

Type: string Default: "{VPIPE_BASEDIR}/envs/upload.yaml"

Type: object Default: {}

This section is used to assist and prepare uploads of the data, e.g. to European Nucleotide Archive. By default it calls a script that creates symlinks making it easy to identify new/updated samples between calls of V-pipe. But by using the properties script and options and adapting the environment provided in property conda, it is possible to heavily customize the actions (e.g. it is possible to upload to an SFTP server by calling sftp from a modified script). For inspiration, see the default script prepare_upload_symlinks.sh.

Type: integer Default: 256

Type: integer Default: 60

Type: string Default: "{VPIPE_BASEDIR}/envs/upload.yaml"

The default environment only provides hashing functions (xxhash, linux coreutils’ sha{nnn}sum collection, etc.) but depending on your needs you would want to provide a custom environment with additional tools (e.g. sftp, rsync, curl, lftp, custom specialized cloud uploaders, etc.)

Type: integer Default: 1

Type: boolean Default: true

Don’t dispatch the rule to the cluster for execution, run locally.


Example:

false
...

Type: enum (of string) Default: "ambig"

When preparing data for upload, specifies which consensus sequence should be uploaded.

Must be one of:

  • "ambig"
  • "majority"

Example:

majority
...

Type: boolean Default: false

Generate checksum for each individual consensus sequence (if a consensus is regenerated, it will help determine whether the new file has changed content or is virtually the same as the previous).


Example:

true
...

Type: boolean Default: false

Also include the original .fastq.gz sequencing reads files from raw_data/ in the list of files to be uploaded. See property orig_cram below for a compressed version and see output dehumanized_raw_reads and section dehuman for depleting reads from the host.


Example:

true
...

Type: boolean Default: false

Also include a compressed version of the original sequencing raw reads files from raw_data/. Similar to property orig_fastq above, but with reference-based compression.


Example:

true
...

Type: string Default: "{VPIPE_BASEDIR}/scripts/prepare_upload_symlinks.sh"

Custom script that assists and prepares uploads.

It will receive the following positional parameters:

  • : the output file that must be created by the script.
  • : a string (with no path separator slashes) that can be used as a name, uniquely identifying the sample and the date.
  • : the base directory of the sample.
  • …: a list of files to consider for upload

For an example, see the default script prepare_upload_symlinks.sh, it generates symlinks that help tracking which samples are new and/or updated between runs of V-pipe and thus should be considered for upload.

Type: string Default: ""

Named options to be passed to the script, before the positional parameters. E.g. for an extra configuration file with SFTP server information.