slurm - Snakemake 在使用 --profile slurm 时给出 InputFunctionException
问题描述
我正在使用snakemake 创建一个管道来调用纳米孔测序数据中的甲基化。我已经使用 --dryrun 选项运行了蛇形,并且成功构建了 dag。但是当我添加选项 --profile slurm 时,我收到以下错误:
(nanopolish) [danielle.perley@talonhead2 nanopolish-CpG-calling]$ snakemake -np --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz
Building DAG of jobs...
Job counts:
count jobs
1 combine_tech_reps
1
InputFunctionException in line 32 of /home/danielle.perley/nanopolish-CpG-calling/Snakefile:
Error:
SyntaxError: invalid syntax (<string>, line 1)
Wildcards:
sample=20-001-002
Traceback:
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 115, in run_jobs
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 120, in run
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 131, in _run
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 151, in printjob
File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 137, in printjob
第 33 行是我的蛇文件中的规则 combine_tech_reps。(我只在这里展示我的蛇文件的第一部分)
from snakemake.utils import validate
import pandas as pd
import os.path
import glob
configfile: "config.yaml"
samples_df = pd.read_table(config["samples"],sep = '\t')
samples_df = samples_df.set_index("Sample")
samples = list(samples_df.index.unique())
wildcard_constraints:
sample = "|".join(samples)
def get_fast5(wildcards):
f5 = glob.glob(os.path.join(config["raw_data"],wildcards.sample,"2*","fast5_pass"))
return(f5)
localrules: all,build_index
rule all:
input:
expand("results/Methylation/{sample}_frequency.tsv",sample=samples),
expand("results/alignments/{sample}_flagstat.txt",sample=samples),
expand("resources/QC/{sample}_pycoQC.json",sample=samples),
expand("results/QC/{sample}_pycoQC.html",sample=samples),
"report/multiQC.html"
rule combine_tech_reps:
input:
fqs = lambda wildcards: glob.glob(os.path.join(config["raw_data"],"{sample}","2*","{sample}_fastq_pass.gz").format(sample=wildcards.sample))
output:
fq = os.path.join(config["raw_data"],"{sample}","{sample}_fastq_pass.gz")
shell: """
zcat {input} > {output}
"""
我在目录中有一个 slurm 配置文件:~/.config/snakemake/slurm/config.yaml
jobs: 10
cluster: "sbatch -p talon -t {resources.time} --mem={resources.mem} -c {resources.cpus} -o logs_slurm/{rule}_{wildcards} -e logs_slurm/{rule}_{wildcards}"
default-resources: [cpus=1, mem=2000, time=10:00]
use-conda: true
我真的很想在我们的 HPC 上使用这个管道,但我不确定是什么导致了这个错误。
解决方案
在这篇文章的帮助下,我能够解决我的问题:
InputFunctionException:解析时出现意外的 EOF
通过添加详细标志:
snakemake -np --verbose --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz
我可以看到snakemake的默认资源有问题:
10:00
^
更改我的 config.yaml 文件的默认资源行:
default-resources: [cpus=1, mem=2000, time=600]
删除了错误。
推荐阅读
- godot - GD 脚本。如何在 Godot 中创建自己的库并从游戏脚本中访问它
- java - 如何在 Java 开关中使用开关盒外的变量?
- php - 为什么此服务提供商的绑定不起作用?
- python - 安装kivy时出错
- firebase - Firebase 可调用函数,限制调用到 APP
- java - 如何在java中添加新记录?
- uml - 系统是否应该成为社交网络应用程序用例图中的参与者
- python - 在 Python 中使用用户的网格大小的表
- kubernetes - K8s: gets HTTP 415 for PATCH request to Kubernetes REST API server
- firebase - 如何在颤振[firebase]中获取文档快照?