python - 无法保留内存块,在 python 中导入 json 错误
问题描述
import pandas as pd
with open(r'data.json') as f:
df = pd.read_json(f, encoding='utf-8')
我收到“无法保留内存块”错误。Json 有 300 mb 大小,为在 python 中运行程序保留内存是否有任何限制?我在 PC 上有 8 GB RAM,使用 Windows 10
loading of json file into df
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\pydev_run_in_console.py", line 52, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Beorn/PycharmProjects/project_0/projekt/test.py", line 7, in <module>
df = pd.read_json(f, encoding='utf-8')
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 422, in read_json
result = json_reader.read()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 529, in read
obj = self._get_object_parser(self.data)
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 546, in _get_object_parser
obj = FrameParser(json, **kwargs).parse()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 638, in parse
self._parse_no_numpy()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 853, in _parse_no_numpy
loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Could not reserve memory block
PyDev console: starting.
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 02:47:15) [MSC v.1900 32 bit (Intel)] on win32
解决方案
因此,在阅读了大量帖子和解决方案后,我决定通过删除 uselles 数据来减小文件大小。也许你觉得这很有用。顺便提一句。我在某处读到,你需要的内存至少比你的 json 文件多 25 倍,所以在我的情况下,我需要超过 8Gb。
with open('data.json', 'r') as data_file:
data = json.load(data_file)
print(data.keys())
del data['author']
with open('datav2.json', 'w') as data_file:
data = json.dump(data, data_file)
推荐阅读
- javascript - Internet连接丢失时PWA黑屏(死机白屏)?
- python - 如何在 Django 中使用 BeautifulSoup?
- angular - 即使在 angular.json 中有正确的路径,Angular 也会显示样式 scss 错误
- javascript - 具有隐藏表单值选项的远程用户地址
- javascript - 页面可见性检测 - 不知道如何设置初始状态
- asp.net-mvc - Asp.Net MVC EF Web 应用的 Azure SQL 用户和连接字符串
- kibana - Kibana 无痛脚本字段
- java - 如何增加存储在整数列表java中的值
- arduino - 延迟防止键盘输入直到它结束
- java - 'void android.widget.Button.setOnClickListener(android.view.View$OnClickListener)' 在空对象引用 &
我试图在我的
Home
活动中定义一个按钮来打开我的Settings
活动,但我得到了错误: