时间戳

首页 > 解决方案 > _gdbm.error: 数据库需要恢复——在获取 api 数据时存储空间不足

python - _gdbm.error: 数据库需要恢复——在获取 api 数据时存储空间不足

问题描述

我真的不知道如何帮助自己，不熟悉这种错误，并且在 Google 环境中找不到任何东西。我最后的希望是你们中的一个，因为我不知道还能去哪里。我尝试重新安装所有库并设置一个新的 venv。对于更多的行动，我对这些事情不够信任。

触发错误的代码：

from wetterdienst import DWDObservationData

observations_daily = DWDObservationData(
    station_ids=station_ids_d,
    parameter=params_daily,
    time_resolution=TimeResolution.DAILY,
    start_date="2015-01-01",
    end_date="2020-10-10",
    tidy_data=True,
    humanize_column_names=True,
)

for df in observations_hourly.collect_data():
    name = str(df.STATION_ID.iloc[0]).strip(".0")
    df.to_csv('./data/hourly/{}.csv'.format(name))
    print('{} done'.format(name))

API 在这里找到：https ://github.com/earthobservations/wetterdienst

错误：

Traceback (most recent call last):
  File "/Users/sashakaun/PycharmProjects/wetter2.0/main.py", line 83, in <module>
    for df in observations_hourly.collect_data():
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/wetterdienst/dwd/observations/api.py", line 178, in collect_data
    df_parameter = self._collect_parameter_from_station(
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/wetterdienst/dwd/observations/api.py", line 243, in _collect_parameter_from_station
    df_period = collect_climate_observations_data(
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/wetterdienst/dwd/observations/access.py", line 82, in collect_climate_observations_data
    filenames_and_files = download_climate_observations_data_parallel(remote_files)
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/wetterdienst/dwd/observations/access.py", line 106, in download_climate_observations_data_parallel
    return list(zip(remote_files, files_in_bytes))
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 611, in result_iterator
    yield fs.pop().result()
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/wetterdienst/dwd/observations/access.py", line 124, in _download_climate_observations_data
    return BytesIO(__download_climate_observations_data(remote_file=remote_file))
  File "<decorator-gen-2>", line 2, in __download_climate_observations_data
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/cache/region.py", line 1356, in get_or_create_for_user_func
    return self.get_or_create(
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/cache/region.py", line 954, in get_or_create
    with Lock(
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/lock.py", line 185, in __enter__
    return self._enter()
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/lock.py", line 94, in _enter
    generated = self._enter_create(value, createdtime)
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/lock.py", line 178, in _enter_create
    return self.creator()
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/cache/region.py", line 920, in gen_value
    self.backend.set(key, value)
  File "/Users/sashakaun/PycharmProjects/wetter2.0/venv/lib/python3.8/site-packages/dogpile/cache/backends/file.py", line 239, in set
    dbm[key] = pickle.dumps(value, pickle.HIGHEST_PROTOCOL)
_gdbm.error: Database needs recovery

非常感谢！！

标签： pythondatabasepicklegnu

解决方案

GDBM 文件已损坏。您需要使用gdbmtool来恢复数据库。安装gdbmtool然后运行

gdbmtool FILENAME

其中 FILENAME 是 GDBM 数据库的名称。会出现提示，然后可以输入

gdbmtool> recover summary

如果数据库可以恢复，它将显示恢复结果的摘要，例如：

Recovery succeeded.
Keys recovered: 6870650, failed: 5, duplicate: 0
Buckets recovered: 64830, failed: 2

推荐阅读