NYU HPC Configuration
All nf-core pipelines have been successfully configured for use on the HPC Cluster at New York University.
To use, run the pipeline with -profile nyu_hpc. This will download and launch the nyu_hpc.config which has been pre-configured with a setup suitable for the NYU HPC cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
Before running the pipeline you will need to load Nextflow using the environment module system on NYU HPC. You can do this by issuing the commands below:
## Load Nextflow modules
module purge
module load nextflow/23.04.1Config file
params {
config_profile_description = 'New York University HPC profile provided by nf-core/configs.'
config_profile_contact = 'HPC@nyu.edu'
config_profile_url = 'https://hpc.nyu.edu'
max_memory = 3000.GB
max_cpus = 96
max_time = 7.d
}
singularity.enabled = true
process {
resourceLimits = [
memory: 3000.GB,
cpus: 96,
time: 7.d
]
executor = 'slurm'
clusterOptions = '--export=NONE'
maxRetries = 3
errorStrategy = { task.attempt <= 3 ? 'retry' : 'finish' }
cache = 'lenient'
}
executor {
queueSize = 1900
submitRateLimit = '20 sec'
}
executor
slurm
contact
