Modules
Nextflow scripts can include definitions (workflows, processes, and functions) from other scripts. When a script is included in this way, it is referred to as a module. Modules can be included by other modules or pipeline scripts and can even be shared across workflows.
Note
Modules were introduced in DSL2. If you are still using DSL1, see the Migrating from DSL 1 page to learn how to migrate your Nextflow pipelines to DSL2.
Module inclusion
You can include any definition from a module into a Nextflow script using the include keyword.
For example:
include { foo } from './some/module'
workflow {
data = channel.fromPath('/some/data/*.txt')
foo(data)
}
The above snippet imports a process named foo, defined in the module, into the main execution context. This way, foo can be invoked in the workflow scope.
Nextflow implicitly looks for the script file ./some/module.nf, resolving the path against the including script location.
Module includes are subject to the following rules:
Relative paths must begin with the
./prefix.Include statements are not allowed from within a workflow. They must occur at the script level.
Module directory
New in version 22.10.0.
A module can be defined as a directory with the same name as the module and with a script named main.nf. For example:
some
└── module
└── main.nf
When defined as a directory, the module must be included by specifying the module directory path:
include { foo } from './some/module'
Module directories allow the use of module scoped binaries scripts. See [Module binaries] for details.
Multiple inclusions
A Nextflow script can include any number of modules, and an include statement can import any number of definitions from a module. Multiple definitions can be included from the same module by using the syntax shown below:
include { foo; bar } from './some/module'
workflow {
data = channel.fromPath('/some/data/*.txt')
foo(data)
bar(data)
}
Module aliases
When including definition from a module, it’s possible to specify an alias with the as keyword. Aliasing allows you to avoid module name clashes, by assigning them different names in the including context. For example:
include { foo } from './some/module'
include { foo as bar } from './other/module'
workflow {
foo(some_data)
bar(other_data)
}
You can also include the same definition multiple times under different names:
include { foo; foo as bar } from './some/module'
workflow {
foo(some_data)
bar(other_data)
}
Module parameters
Deprecated since version 24.07.0-edge: As a best practice, parameters should be used in the entry workflow and passed to workflows, processes, and functions as explicit inputs.
A module can define parameters using the same syntax as a Nextflow workflow script:
params.foo = 'Hello'
params.bar = 'world!'
def sayHello() {
println "$params.foo $params.bar"
}
When including a module, the module will first use parameters from the including context. For example:
params.foo = 'Hola'
params.bar = 'Mundo'
include { sayHello } from './some/module'
workflow {
sayHello()
}
The above snippet prints:
Hola Mundo
Note
The module inherits the parameters defined before the include statement, therefore any parameters set afterwards will not be used by the module.
Tip
It is best to define all pipeline parameters before any include statements.
The addParams option can be used to pass parameters to the module without adding them to the including scope.
params.foo = 'Hola'
params.bar = 'Mundo'
include { sayHello } from './some/module' addParams(foo: 'Ciao')
workflow {
sayHello()
}
The above snippet prints:
Ciao Mundo
Alternatively, the params option can be used to pass parameters to module without adding them to the including scope, and without inheriting any parameters from the including scope.
params.foo = 'Hola'
params.bar = 'Mundo'
include { sayHello } from './some/module' params(foo: 'Ciao')
workflow {
sayHello()
}
The above snippet prints:
Ciao world!
Module templates
Template files can be stored in the templates directory alongside a module.
Project A
├── main.nf
└── modules
└── sayhello
├── sayhello.nf
└── templates
└── sayhello.sh
Template files can be invoked like regular scripts from a process in your pipeline using the template function. Variables prefixed with the dollar character ($) are interpreted as Nextflow variables when the template file is executed by Nextflow.
See Template files for more information utilizing template files.
Storing template files with the module that utilizes it encourages sharing of modules across pipelines. For example, future projects would be able to include the module from above by cloning the modules directory and including the module without needing to modify the process or template.
Beyond facilitating module sharing across pipelines, organizing templates locations allows for a well-structured project. For example, complex projects with multiple modules that rely on templates can be organized into logical groups:
baseDir
|── main.nf
|── Phase0-Modules
|── mymodules1.nf
|── mymodules2.nf
└── templates
|── P1-template.sh
|── P2-template.sh
|── Phase1-Modules
|── mymodules3.nf
|── mymodules4.nf
└── templates
|── P3-template.sh
└── P4-template.sh
└── Phase2-Modules
|── mymodules5.nf
|── mymodules6.nf
└── templates
|── P5-template.sh
└── P6-template.sh
Template files can also be stored in the project templates directory. See structure-template for more information about the project directory structure.
Module binaries
New in version 22.10.0.
Modules can define binary scripts that are locally scoped to the processes.
Binary scripts must be placed in the module directory named <module-dir>/resources/usr/bin. For example:
<module-dir>
|── main.nf
└── resources
└── usr
└── bin
└── script.py
Binary scripts can be invoked like regular commands from the locally scoped module without modifying the PATH environment variable or using an absolute path. Each script should include a shebang to specify the interpreter and inputs should be supplied as arguments. See The bin directory for more information about custom scripts in bin directories.
To use this feature, the module binaries must be enabled in your pipeline script or configuration file:
nextflow.enable.moduleBinaries = true
Note
Module binary scripts require a local or shared file system for the pipeline work directory or Wave containers when using cloud-based executors.
Scripts can also be stored in project level bin directory. See The bin directory for more information.