时间戳

首页 > 解决方案 > “无法在应该开箱即用的项目上创建 cudnn 句柄：CUDNN_STATUS_ALLOC_FAILED”

python-3.x - “无法在应该开箱即用的项目上创建 cudnn 句柄：CUDNN_STATUS_ALLOC_FAILED”

问题描述

https://github.com/zzh8829/yolov3-tf2是项目。我已经安装了我认为的所有正确版本。

谷歌告诉我这可能是一个低 VRAM 问题，但我仍在四处寻找其他原因。请帮忙。我在用 ：

Windows 10（不要说“有你的问题”我需要它）

cuDNN 7.4.6

CUDA 10.0

张量流 2.0.0

蟒蛇3.6

我有一个 gtx1660 超级 6GB VRAM 和 16GB RAM 上的 ryzen 7 2700x。几天后我将得到一个 gt1080 8gig，我将添加到第二个 PCI 插槽。

错误如下：

2019-11-30 06:31:26.167368: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll                                
2019-11-30 06:31:27.843742: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED                                      
2019-11-30 06:31:27.853725: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED                                      
Traceback (most recent call last):                                                                                                                                           
  File ".\convert.py", line 34, in <module>                                                                                                                                  
    app.run(main)                                                                                                                                                            
  File "C:\Program Files\Python36\lib\site-packages\absl\app.py", line 299, in run                                                                                           
    _run_main(main, args)                                                                                                                                                    
  File "C:\Program Files\Python36\lib\site-packages\absl\app.py", line 250, in _run_main                                                                                     
    sys.exit(main(argv))                                                                                                                                                     
  File ".\convert.py", line 25, in main                                                                                                                                      
    output = yolo(img)                                                                                                                                                       
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 891, in __call__                                                
    outputs = self.call(cast_inputs, *args, **kwargs)                                                                                                                        
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 708, in call                                                       
    convert_kwargs_to_constants=base_layer_utils.call_context().saving)                                                                                                      
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 860, in _run_internal_graph                                        
    output_tensors = layer(computed_tensors, **kwargs)                                                                                                                       
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 891, in __call__                                                
    outputs = self.call(cast_inputs, *args, **kwargs)                                                                                                                        
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 708, in call                                                       
    convert_kwargs_to_constants=base_layer_utils.call_context().saving)                                                                                                      
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 860, in _run_internal_graph                                        
    output_tensors = layer(computed_tensors, **kwargs)                                                                                                                       
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 891, in __call__                                                
    outputs = self.call(cast_inputs, *args, **kwargs)                                                                                                                        
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\keras\layers\convolutional.py", line 197, in call                                                 
    outputs = self._convolution_op(inputs, self.kernel)                                                                                                                      
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 1134, in __call__                                                            
    return self.conv_op(inp, filter)                                                                                                                                         
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 639, in __call__                                                             
    return self.call(inp, filter)                                                                                                                                            
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 238, in __call__                                                             
    name=self.name)                                                                                                                                                          
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\nn_ops.py", line 2010, in conv2d                                                              
    name=name)                                                                                                                                                               
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\gen_nn_ops.py", line 1031, in conv2d                                                          
    data_format=data_format, dilations=dilations, name=name, ctx=_ctx)                                                                                                       
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\ops\gen_nn_ops.py", line 1130, in conv2d_eager_fallback                                           
    ctx=_ctx, name=name)                                                                                                                                                     
  File "C:\Program Files\Python36\lib\site-packages\tensorflow_core\python\eager\execute.py", line 67, in quick_execute                                                      
    six.raise_from(core._status_to_exception(e.code, message), None)                                                                                                         
  File "<string>", line 3, in raise_from                                                                                                                                     
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a wa
rning log message was printed above. [Op:Conv2D]

标签： python-3.xtensorflowyolo

解决方案

这可能有几个原因。

(1)正如您所提到的，这可能是内存问题，您可以尝试通过为 GPU 分配更少的内存并查看该错误是否仍然存在来验证。您可以像这样在 TF 2.0 中执行此操作（https://github.com/tensorflow/tensorflow/issues/25138#issuecomment-484428798）：

import tensorflow as tf
tf.config.gpu.set_per_process_memory_fraction(0.75)
tf.config.gpu.set_per_process_memory_growth(True)

# your model creation, etc.
model = MyModel(...)

如果您有 > 1 个 GPU（ https://github.com/zzh8829/yolov3-tf2/blob/master/train.py#L46-L47 ），我看到您正在运行的代码会设置动态内存增长，但因为您只有有 1 个 GPU，那么它可能只是在开始时尝试分配所有内存 (>90%)。

(2)当您或其他用户同时使用 GPU 的其他 TensorFlow 或类似进程时，一些用户似乎在 Windows 上遇到过这种情况：https ://stackoverflow.com/a/53707323/10993413

(3)与往常一样，确保您的 PATH 变量是正确的。有时，如果您尝试了多次安装并且没有正确清理内容，则 PATH 可能会首先找到错误的版本并导致问题。如果在 PATH 的开头添加新路径，则应首先找到它们：https ://www.tensorflow.org/install/gpu#windows_setup

(4)正如@xenotecc 所述，您可以尝试升级到较新版本的 CUDNN，但我不确定这是否会有所帮助，因为您的配置在 TF 文档中被列为支持：https ://www.tensorflow.org/安装/源#gpu。如果这确实解决了它，它可能毕竟是 PATH 问题，因为您可能会在安装较新版本后更新 PATH。

推荐阅读