python - 当我尝试从 mlflow UI 下载工件时出现错误

Question

系统信息

操作系统平台和发行版：Windows 10 MLflow 安装：使用 pip MLflow 版本：1.20.2 版 **Python 版本：Python 3.9.7 **

问题

我保存了一个 .h5 keras 模型，当我尝试执行 mlflow.keras.load_model("run:/id_run/model") 时，我已经等了将近一个小时，但它没有完成。所以我停止了执行，我得到了下一个错误：

ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

Traceback (most recent call last):
  File "~\anaconda3\envs\python_38\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-3-277d37cc6084>", line 1, in <module>
    keras_model = mlflow.keras.load_model("runs:/483745e28a864eceb738c852cf062774/model")
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\keras.py", line 585, in load_model
    local_model_path = _download_artifact_from_uri(artifact_uri=model_uri)
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\tracking\artifact_utils.py", line 83, in _download_artifact_from_uri
    return get_artifact_repository(artifact_uri=root_uri).download_artifacts(
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\runs_artifact_repo.py", line 125, in download_artifacts
    return self.repo.download_artifacts(artifact_path, dst_path)
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\artifact_repo.py", line 180, in download_artifacts
    return download_artifact_dir(
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\artifact_repo.py", line 147, in download_artifact_dir
    download_artifact_dir(
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\artifact_repo.py", line 152, in download_artifact_dir
    download_artifact(
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\artifact_repo.py", line 129, in download_artifact
    self._download_file(
  File "~\AppData\Roaming\Python\Python38\site-packages\mlflow\store\artifact\azure_blob_artifact_repo.py", line 136, in _download_file
    container_client.download_blob(remote_full_path).readinto(file)
  File "~\anaconda3\envs\python_38\lib\site-packages\azure\storage\blob\_download.py", line 617, in readinto
    downloader.process_chunk(chunk)
  File "~\anaconda3\envs\python_38\lib\site-packages\azure\storage\blob\_download.py", line 129, in process_chunk
    chunk_data = self._download_chunk(chunk_start, chunk_end - 1)
  File "~\anaconda3\envs\python_38\lib\site-packages\azure\storage\blob\_download.py", line 211, in _download_chunk
    chunk_data = process_content(response, offset[0], offset[1], self.encryption_options)
  File "~\anaconda3\envs\python_38\lib\site-packages\azure\storage\blob\_download.py", line 52, in process_content
    content = b"".join(list(data))
  File "~\anaconda3\envs\python_38\lib\site-packages\azure\core\pipeline\transport\_requests_basic.py", line 158, in __next__
    chunk = next(self.iter_content_func)
  File "~\anaconda3\envs\python_38\lib\site-packages\requests\models.py", line 758, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "~\anaconda3\envs\python_38\lib\site-packages\urllib3\response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "~\anaconda3\envs\python_38\lib\site-packages\urllib3\response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "~\anaconda3\envs\python_38\lib\http\client.py", line 459, in read
    n = self.readinto(b)
  File "~\anaconda3\envs\python_38\lib\http\client.py", line 503, in readinto
    n = self.fp.readinto(b)
  File "~\anaconda3\envs\python_38\lib\socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "~\anaconda3\envs\python_38\lib\ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "~\anaconda3\envs\python_38\lib\ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~\anaconda3\envs\python_38\lib\site-packages\IPython\core\interactiveshell.py", line 2061, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'KeyboardInterrupt' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~\anaconda3\envs\python_38\lib\site-packages\IPython\core\ultratb.py", line 1101, in get_records
    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)
  File "~\anaconda3\envs\python_38\lib\site-packages\IPython\core\ultratb.py", line 248, in wrapped
    return f(*args, **kwargs)
  File "~\anaconda3\envs\python_38\lib\site-packages\IPython\core\ultratb.py", line 281, in _fixed_getinnerframes
    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
  File "~\anaconda3\envs\python_38\lib\inspect.py", line 1515, in getinnerframes
    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)
  File "~\anaconda3\envs\python_38\lib\inspect.py", line 1473, in getframeinfo
    filename = getsourcefile(frame) or getfile(frame)
  File "~\anaconda3\envs\python_38\lib\inspect.py", line 708, in getsourcefile
    if getattr(getmodule(object, filename), '__loader__', None) is not None:
  File "~\anaconda3\envs\python_38\lib\inspect.py", line 754, in getmodule
    os.path.realpath(f)] = module.__name__
  File "~\anaconda3\envs\python_38\lib\ntpath.py", line 647, in realpath
    path = _getfinalpathname(path)
KeyboardInterrupt

我的工件 sotrage 是一个 Azure Blob 存储，我的 MLflow 服务器在 axeternal 服务器中运行

我在 UI 中检查了模型，它就在那里，所以我尝试使用下载按钮下载它。但是正如您在图像中看到的那样，下载会停止，然后一遍又一遍地重新启动。我注意到它的重量为 355MB，因此我将模型目录压缩为具有或多或少相同重量的工件，当您尝试下载它时它会出现同样的问题。

重现此问题

只需记录一个具有相同权重的文件并尝试恢复它

标签： pythonazuretensorflowmlflow