时间戳

首页 > 解决方案 > 我已经安装了 cudNN 但出现错误“无法获得卷积算法”

python - 我已经安装了 cudNN 但出现错误“无法获得卷积算法”

问题描述

所以，我有一台 RTX 2060 的机器，我想在上面运行 tensorflow。但是，尽管我在其上安装了 cudNN，但仍显示错误，无法获得卷积算法。

我的 Linux (Xubuntu 18.04) 机器上运行着 Tensorflow-GPU 1.13.1。我已按照网站上的说明（如下），并通过 pip tensorflow-gpu 安装。

我遵循的说明：


# Add NVIDIA package repositories
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
sudo apt install ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
sudo apt-get update

# Install NVIDIA driver
sudo apt-get install --no-install-recommends nvidia-driver-410
# Reboot. Check that GPUs are visible using the command: nvidia-smi

# Install development and runtime libraries (~4GB)
sudo apt-get install --no-install-recommends \
    cuda-10-0 \
    libcudnn7=7.4.1.5-1+cuda10.0  \
    libcudnn7-dev=7.4.1.5-1+cuda10.0
 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb sudo dpkg -i cuda-repo-ubunt

# Install TensorRT. Requires that libcudnn7 is installed above.
sudo apt-get update && \
        sudo apt-get install nvinfer-runtime-trt-repo-ubuntu1804-5.0.2-ga-cuda10.0 \
        && sudo apt-get update \
        && sudo apt-get install -y --no-install-recommends libnvinfer-dev=5.0.2-1+cuda10.0

我得到的错误：

2019-03-25 23:16:50.938950: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
2019-03-25 23:16:52.732720: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2019-03-25 23:16:52.736377: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
     [[{{node ppo/actions-and-internals/layered-network/apply/conv2d0/apply/Conv2D}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "start.py", line 54, in <module>
    main()
  File "start.py", line 51, in main
    main_loop(agent, curiousity_engine)
  File "start.py", line 23, in main_loop
    action1 = agent.act(states=get_screen())
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 148, in act
    independent=independent
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/model.py", line 1393, in act
    fetch_list = self.monitored_session.run(fetches=fetches, feed_dict=feed_dict)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 676, in run
    run_metadata=run_metadata)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1270, in run
    raise six.reraise(*original_exc_info)
  File "/home/user/.local/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1255, in run
    return self._sess.run(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1327, in run
    run_metadata=run_metadata)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1091, in run
    return self._sess.run(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
     [[node ppo/actions-and-internals/layered-network/apply/conv2d0/apply/Conv2D (defined at /home/user/.local/lib/python3.6/site-packages/tensorforce/core/networks/layer.py:1079) ]]

Caused by op 'ppo/actions-and-internals/layered-network/apply/conv2d0/apply/Conv2D', defined at:
  File "start.py", line 54, in <module>
    main()
  File "start.py", line 41, in main
    agent, user_input = agent_build()
  File "/home/user/Downloads/v2 (2)/agent.py", line 37, in agent_build
    actions_exploration = 'epsilon_decay'
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/agents/ppo_agent.py", line 155, in __init__
    entropy_regularization=entropy_regularization
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/agents/learning_agent.py", line 141, in __init__
    batching_capacity=batching_capacity
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 80, in __init__
    self.model = self.initialize_model()
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/agents/ppo_agent.py", line 183, in initialize_model
    likelihood_ratio_clipping=self.likelihood_ratio_clipping
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/pg_prob_ratio_model.py", line 88, in __init__
    gae_lambda=gae_lambda
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/pg_model.py", line 98, in __init__
    requires_deterministic=False
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/distribution_model.py", line 90, in __init__
    discount=discount
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 114, in __init__
    reward_preprocessing=reward_preprocessing
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/model.py", line 217, in __init__
    self.setup()
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/model.py", line 290, in setup
    independent=independent
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 605, in create_operations
    independent=independent
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/model.py", line 1193, in create_operations
    independent=independent
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/model.py", line 1019, in create_act_operations
    deterministic=deterministic
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/models/distribution_model.py", line 187, in tf_actions_and_internals
    return_internals=True
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/core/networks/network.py", line 253, in tf_apply
    x = layer.apply(x=x, update=update)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorforce/core/networks/layer.py", line 1079, in tf_apply
    x = tf.nn.conv2d(input=x, filter=self.filters, strides=(1, stride_h, stride_w, 1), padding=self.padding)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1026, in conv2d
    data_format=data_format, dilations=dilations, name=name)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()

UnknownError (see above for traceback): Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
     [[node ppo/actions-and-internals/layered-network/apply/conv2d0/apply/Conv2D (defined at /home/user/.local/lib/python3.6/site-packages/tensorforce/core/networks/layer.py:1079) ]]

标签： pythontensorflowkeras

解决方案

我用相同的设置遇到了同样的问题。我发现（如果我没记错的话）后面的一些命令会安装更新版本的驱动程序。匹配版本似乎非常关键。我的鼠标也停止工作，因为一些输入包被卸载。

摆弄花费了我几天的时间和无数的全新安装......最终起作用的是手动安装驱动程序、cuda 和 cudnn。这个过程远非最佳，我的最终结果并不像我想要的那样整洁，但它确实有效。

我的版本：驱动程序：410.48 Cuda：10.0 cuDNN：7.4.2（TensorRt：选择使用 cuDNN 7.4.2 的版本）

除此之外，还需要在 python tensorflow 代码中添加以下行之一：

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
tf.enable_eager_execution(config=config)

或者

config = tf.ConfigProto()
# config.gpu_options.allow_growth = True
config.gpu_options.per_process_gpu_memory_fraction = 0.1
sess = tf.Session(config=config)

推荐阅读