conda - 一次运行两个使用相同 conda 环境的作业时出现 snakemake 错误
问题描述
我在执行 Snakemake (6.0.0) 工作流时遇到错误,因此在同一节点上同时启动两个作业,这两个作业都使用相同的 conda 环境。最小的例子如下。
几点观察:
- 在我的机构集群的一个节点上运行工作流时会出现问题,但不是在我的本地计算机上。(我正在
snakemake
使用 >1 cpu 的交互式 slurm 作业中运行;我正在使用 Miniconda,module
由我的集群系统管理员提供) - 当任务被强制串行运行时,工作流就可以顺利完成(
snakemake --use-conda -j1
)。仅当-j2
或更高(不超过 slurm 分配中可用的核心数)时才会出现问题。第一份工作似乎运行得很好,第二份工作总是在发牢骚。 - 在由snakemake创建后,我可以很好地激活有问题的conda环境(例如,在运行工作流之后,
conda activate /long_path_to_cluster_project_folder/testing/conda_test/.snakemake/conda/c4751dca
工作,我可以从内部运行R等) - 如果我运行
snakemake --use-conda -j2
,我得到的唯一错误是(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
在 shell 命令运行之下。如果我添加--verbose
,则会以蓝色和红色打印一个冗长的回溯,我已将其包含在下面。相关位似乎是:File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 505, in prefix_path return self.info["conda_prefix"] AttributeError: 'Conda' object has no attribute 'info'
- 怀疑某种竞争条件,我还尝试添加
--max-jobs-per-second=0.5
限制作业,以便它们不会同时启动,但这似乎没有效果(作业同时开始,与以前相同的错误。我没有运行 snakemake--cluster
或--profile
或任何东西;没有创建额外的 slurm 作业,只是在同一个计算节点上产生了进程) - 如果我创建两个完全不同的 Snakemake 规则并最终同时执行,则会出现同样的问题,只要这两个规则都使用相同的 conda 环境。
我对snakemake和HPC都很陌生,但这似乎介于系统/配置特定问题(因为它只发生在集群上)和一个小snakemake错误(因为snakemake似乎是造成问题的原因到我的shell脚本而不是与conda有关)。我对如何进一步排除故障或解决问题的建议感兴趣。
谢谢!
最小的例子:
├── input.txt
├── results
└── workflow
├── Snakefile
└── envs
└── env1.yaml
workflow/Snakefile
:rule all: input: 'results/output1.txt', 'results/output2.txt', 'results/output3.txt', 'results/output4.txt' rule rule1: input: 'input.txt' output: 'results/output{n}.txt' conda: 'envs/env1.yaml' shell:""" sleep 5s touch {output} """
workflow/envs/env1.yaml
:channels: - conda-forge - bioconda - defaults dependencies: - r-ggplot2
$ snakemake --use-conda -j2 -p --verbose
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 all
4 rule1
5
<< snip >>
[Fri Mar 5 21:01:33 2021]
Error in rule rule1:
jobid: 2
output: results/output2.txt
conda-env: /long_path_to_cluster_project_folder/testing/conda_test/.snakemake/conda/c4751dca
shell:
sleep 5s
touch results/output2.txt
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Full Traceback (most recent call last):
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2326, in run_wrapper
run(
File "/long_path_to_cluster_project_folder/testing/conda_test/workflow/Snakefile", line 33, in __rule_rule1
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/shell.py", line 141, in __new__
cmd = Conda(container_img).shellcmd(conda_env, cmd)
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 512, in shellcmd
activate = os.path.join(self.bin_path(), "activate")
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 508, in bin_path
return os.path.join(self.prefix_path(), "bin")
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 505, in prefix_path
return self.info["conda_prefix"]
AttributeError: 'Conda' object has no attribute 'info'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 568, in _callback
raise ex
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 554, in cached_or_run
run_func(*args)
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2357, in run_wrapper
raise RuleException(
snakemake.exceptions.RuleException: AttributeError in line 13 of /long_path_to_cluster_project_folder/testing/conda_test/workflow/Snakefile:
'Conda' object has no attribute 'info'
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2326, in run_wrapper
File "/long_path_to_cluster_project_folder/testing/conda_test/workflow/Snakefile", line 13, in __rule_rule1
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 512, in shellcmd
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 508, in bin_path
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 505, in prefix_path
RuleException:
AttributeError in line 13 of /long_path_to_cluster_project_folder/testing/conda_test/workflow/Snakefile:
'Conda' object has no attribute 'info'
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2326, in run_wrapper
File "/long_path_to_cluster_project_folder/testing/conda_test/workflow/Snakefile", line 13, in __rule_rule1
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 512, in shellcmd
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 508, in bin_path
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 505, in prefix_path
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 568, in _callback
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/concurrent/futures/thread.py", line 52, in run
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 554, in cached_or_run
File "/long_path_to_cluster_project_folder/conda_envs/snakemake/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2357, in run_wrapper
解决方案
推荐阅读
- kotlin - kotlin 上的 Gradle 任务。声明 OutputFile 值
- vue.js - 图像加载离子
- javascript - 如何在 Javascript 中使用模板文字进行连接?
- powershell - 如何安排 Powershell JOB 每 5 分钟运行一次
- javascript - 等效于 C# SHA512 的 Javascript
- c - 在 MacOS Catalina 上使用 openmp 库
- azure - Windows 2019 IIS On Premise Application Insight Monitor 状态
- typescript - 基于值的参数类型
- angular - 下拉菜单打不开
- vba - 如何在 MS Access 文本字段中维护 crlf