python - 嵌套列表上的 ast.literal_eval 内存错误
问题描述
我正在尝试在字符串上使用 ast.literal_eval,但出现内存错误。列表是否过于嵌套或有什么问题?有替代方案吗?
MemoryError Traceback (most recent call last)
<ipython-input-138-3ea9110f7dc3> in <module>()
1 s = "[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['Xenophrys_bairdii'],'Vulpanser_spaldingi'],'Vipera_yeltoniensis'],'Vipera_minutus'],'Vanellus_vegans'],'Vanellus_bedriagai'],'Ursus_gratiosa'],'Uroplatus_flavomaculatus'],'Trionyx_leporosum'],'Tringa_aspera'],'Trachemys_sirtalis'],'Terpsihone_varius'],'Spermophilus_hassanica'],'Siniperca_decor'],'Scolopendra_marcianus'],'Rufibrenta_montela'],'Riparia_clinatus'],'Rhinolophus_tuberculosus'],'Rhacodactylus_citrsola'],'Remiz_niloticus'],'Pterinochilus_physalus'],'Procellaria_truncatus'],'Procellaria_lutris'],'Poephagus_indica'],'Platemys_albopillosum'],'Pica_totanus'],'Pica_mexicana'],'Phrynomerus_pelagicus'],'Philothamnus_tuberculosus'],'Petrocincla_acuta'],'Pelomedusa_rusticolus'],'Pagophila_metallica'],'Pachytriton_sibiricus'],'Otis_moschata'],'Otis_franckii'],'Otis_emarginatus'],'Opheodrys_clarus'],'Mylopharyngodon_diffidens'],'Mergus_carinata'],'Meles_carbonaria'],'Mabuya_carnivorus'],'Lystrophis_siebenrocki'],'Lyrurus_tinctorius'],'Lycodon_dendrophila'],'Lutra_plathyrhychos'],'Leptopelis_plumipes'],'Leptopelis_licin'],'Leiurus_grossmani'],'Lasiodora_hipposideros'],'Larus_pulchripes'],'Kinosternon_guineti'],'Kassina_canorus'],'Hysterocrates_chukar'],'Hydrochelidon_euptilura'],'Hemitheconyx_pulchripes'],'Haplopelma_arcticus'],'Hadrurus_dominus'],'Gypaetus_schneideri'],'Glareola_leucophyllata'],'Gerrhosaurus_ruficollis'],'Geochelone_filipjevi'],'Gallinago_pallasii'],'Eutamias_multifasciata'],'Eulabeia_tinnunculus'],'Eudrornias_penelope'],'Eudramias_tadorna'],'Eschrichtius_pulchra'],'Eremophila_similis'],'Equus_fluviatilis'],'Epicrates_carinatus'],'Emydura_avosetta'],'Emberiza_japonica'],'Dyscophus_rubicola'],'Dendrelaphis_griseus'],'Cypselus_ceterus'],'Cynops_rutila'],'Cygnus_rubicola'],'Ctenosaura_sphenocercus'],'Coenobita_variabilis'],'Clemmys_caudata'],'Cervus_comicus'],'Castor_prominanus'],'Casarca_holbrooki'],'Capella_aestivus'],'Buthus_nebularia'],'Bronchocela_bicoloratum'],'Branta_ferrumequinum'],'Balaenoptera_piscator'],'Athene_musculus'],'Argynnis_versicolor'],'Arenaria_javanica'],'Anthropoides_aestivus'],'Anolis_medirostris'],'Anodonta_infrafrenata'],'Ambystoma_alcinous'],'Acanthoceros_euptilura']"
----> 2 ast.literal_eval(s)
~\Anaconda3\lib\ast.py in literal_eval(node_or_string)
46 """
47 if isinstance(node_or_string, str):
---> 48 node_or_string = parse(node_or_string, mode='eval')
49 if isinstance(node_or_string, Expression):
50 node_or_string = node_or_string.body
~\Anaconda3\lib\ast.py in parse(source, filename, mode)
33 Equivalent to compile(source, filename, mode, PyCF_ONLY_AST).
34 """
---> 35 return compile(source, filename, mode, PyCF_ONLY_AST)
36
37
MemoryError:
解决方案
显然解释器在解析嵌套列表时有一些限制(在我们的示例中,但很可能也适用于其他容器)。它破坏了ast.literal_eval
和eval
(不安全),并且它们都具有相同的限制这一事实告诉我,它们都使用一些通用代码(这也可能影响其他领域)。
在我的情况下,它是92(93打破它),但由于快速Python源代码搜索没有显示任何相关内容(如常量、#define
或类似的东西),这个值可能与我的环境(机器、堆栈、操作系统, Python版本等)。
请注意,嵌套列表可以具有更高的嵌套级别(受限于[Python]:系统。getrecursionlimit () - 对于我试图创建999(或更高)级别的嵌套列表,raised RecursionError
)。
无论如何,作为替代方案,我使用了[Python]: json - JSON encoder and decoder,它工作(有一点处理)。请注意,尽管此方法适用于您的场景,但它一点也不健壮;例如:如果其中一个字符串包含引号(单引号或双引号),它将不再起作用。
代码.py:
import sys
import ast
import json
import traceback
LIST_NESTING_LIMIT = 92
def generate_nested_list(count):
if count <= 0:
return None
ret = ["0"]
for i in range(1, count):
ret = [ret, "{:d}".format(i)]
return ret
def test_parse_nested_list(count, parse_func):
nested_list_str = str(generate_nested_list(count))
try:
parsed = parse_func(nested_list_str)
except:
traceback.print_exc(limit=0)
else:
return parsed
def json_loads_wrapper(nested_list_str):
return json.loads(nested_list_str.replace("'", "\""))
def main():
for func in (ast.literal_eval, eval, json_loads_wrapper):
for i in (LIST_NESTING_LIMIT, LIST_NESTING_LIMIT + 1):
print("\nTrying to parse a {:d} level nested list using '{:s}':".format(i, func.__name__))
result = test_parse_nested_list(i, func)
if result is not None:
print(result)
print("\nOriginal list:")
s = "[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['Xenophrys_bairdii'],'Vulpanser_spaldingi'],'Vipera_yeltoniensis'],'Vipera_minutus'],'Vanellus_vegans'],'Vanellus_bedriagai'],'Ursus_gratiosa'],'Uroplatus_flavomaculatus'],'Trionyx_leporosum'],'Tringa_aspera'],'Trachemys_sirtalis'],'Terpsihone_varius'],'Spermophilus_hassanica'],'Siniperca_decor'],'Scolopendra_marcianus'],'Rufibrenta_montela'],'Riparia_clinatus'],'Rhinolophus_tuberculosus'],'Rhacodactylus_citrsola'],'Remiz_niloticus'],'Pterinochilus_physalus'],'Procellaria_truncatus'],'Procellaria_lutris'],'Poephagus_indica'],'Platemys_albopillosum'],'Pica_totanus'],'Pica_mexicana'],'Phrynomerus_pelagicus'],'Philothamnus_tuberculosus'],'Petrocincla_acuta'],'Pelomedusa_rusticolus'],'Pagophila_metallica'],'Pachytriton_sibiricus'],'Otis_moschata'],'Otis_franckii'],'Otis_emarginatus'],'Opheodrys_clarus'],'Mylopharyngodon_diffidens'],'Mergus_carinata'],'Meles_carbonaria'],'Mabuya_carnivorus'],'Lystrophis_siebenrocki'],'Lyrurus_tinctorius'],'Lycodon_dendrophila'],'Lutra_plathyrhychos'],'Leptopelis_plumipes'],'Leptopelis_licin'],'Leiurus_grossmani'],'Lasiodora_hipposideros'],'Larus_pulchripes'],'Kinosternon_guineti'],'Kassina_canorus'],'Hysterocrates_chukar'],'Hydrochelidon_euptilura'],'Hemitheconyx_pulchripes'],'Haplopelma_arcticus'],'Hadrurus_dominus'],'Gypaetus_schneideri'],'Glareola_leucophyllata'],'Gerrhosaurus_ruficollis'],'Geochelone_filipjevi'],'Gallinago_pallasii'],'Eutamias_multifasciata'],'Eulabeia_tinnunculus'],'Eudrornias_penelope'],'Eudramias_tadorna'],'Eschrichtius_pulchra'],'Eremophila_similis'],'Equus_fluviatilis'],'Epicrates_carinatus'],'Emydura_avosetta'],'Emberiza_japonica'],'Dyscophus_rubicola'],'Dendrelaphis_griseus'],'Cypselus_ceterus'],'Cynops_rutila'],'Cygnus_rubicola'],'Ctenosaura_sphenocercus'],'Coenobita_variabilis'],'Clemmys_caudata'],'Cervus_comicus'],'Castor_prominanus'],'Casarca_holbrooki'],'Capella_aestivus'],'Buthus_nebularia'],'Bronchocela_bicoloratum'],'Branta_ferrumequinum'],'Balaenoptera_piscator'],'Athene_musculus'],'Argynnis_versicolor'],'Arenaria_javanica'],'Anthropoides_aestivus'],'Anolis_medirostris'],'Anodonta_infrafrenata'],'Ambystoma_alcinous'],'Acanthoceros_euptilura']"
print(json_loads_wrapper(s))
if __name__ == "__main__":
print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
main()
输出:
(py35x64_test) e:\Work\Dev\StackOverflow\q050709371>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32 Trying to parse a 92 level nested list using 'literal_eval': [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['0'], '1'], '2'], '3'], '4'], '5'], '6'], '7'], '8'], '9'], '10'], '11'], '12'], '13'], '14'], '15'], '16'], '17'], '18'], '19'], '20'], '21'], '22'], '23'], '24'], '25'], '26'], '27'], '28'], '29'], '30'], '31'], '32'], '33'], '34'], '35'], '36'], '37'], '38'], '39'], '40'], '41'], '42'], '43'], '44'], '45'], '46'], '47'], '48'], '49'], '50'], '51'], '52'], '53'], '54'], '55'], '56'], '57'], '58'], '59'], '60'], '61'], '62'], '63'], '64'], '65'], '66'], '67'], '68'], '69'], '70'], '71'], '72'], '73'], '74'], '75'], '76'], '77'], '78'], '79'], '80'], '81'], '82'], '83'], '84'], '85'], '86'], '87'], '88'], '89'], '90'], '91'] Trying to parse a 93 level nested list using 'literal_eval': s_push: parser stack overflow Traceback (most recent call last): MemoryError Trying to parse a 92 level nested list using 'eval': [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['0'], '1'], '2'], '3'], '4'], '5'], '6'], '7'], '8'], '9'], '10'], '11'], '12'], '13'], '14'], '15'], '16'], '17'], '18'], '19'], '20'], '21'], '22'], '23'], '24'], '25'], '26'], '27'], '28'], '29'], '30'], '31'], '32'], '33'], '34'], '35'], '36'], '37'], '38'], '39'], '40'], '41'], '42'], '43'], '44'], '45'], '46'], '47'], '48'], '49'], '50'], '51'], '52'], '53'], '54'], '55'], '56'], '57'], '58'], '59'], '60'], '61'], '62'], '63'], '64'], '65'], '66'], '67'], '68'], '69'], '70'], '71'], '72'], '73'], '74'], '75'], '76'], '77'], '78'], '79'], '80'], '81'], '82'], '83'], '84'], '85'], '86'], '87'], '88'], '89'], '90'], '91'] Trying to parse a 93 level nested list using 'eval': s_push: parser stack overflow Traceback (most recent call last): MemoryError Trying to parse a 92 level nested list using 'json_loads_wrapper': [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['0'], '1'], '2'], '3'], '4'], '5'], '6'], '7'], '8'], '9'], '10'], '11'], '12'], '13'], '14'], '15'], '16'], '17'], '18'], '19'], '20'], '21'], '22'], '23'], '24'], '25'], '26'], '27'], '28'], '29'], '30'], '31'], '32'], '33'], '34'], '35'], '36'], '37'], '38'], '39'], '40'], '41'], '42'], '43'], '44'], '45'], '46'], '47'], '48'], '49'], '50'], '51'], '52'], '53'], '54'], '55'], '56'], '57'], '58'], '59'], '60'], '61'], '62'], '63'], '64'], '65'], '66'], '67'], '68'], '69'], '70'], '71'], '72'], '73'], '74'], '75'], '76'], '77'], '78'], '79'], '80'], '81'], '82'], '83'], '84'], '85'], '86'], '87'], '88'], '89'], '90'], '91'] Trying to parse a 93 level nested list using 'json_loads_wrapper': [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['0'], '1'], '2'], '3'], '4'], '5'], '6'], '7'], '8'], '9'], '10'], '11'], '12'], '13'], '14'], '15'], '16'], '17'], '18'], '19'], '20'], '21'], '22'], '23'], '24'], '25'], '26'], '27'], '28'], '29'], '30'], '31'], '32'], '33'], '34'], '35'], '36'], '37'], '38'], '39'], '40'], '41'], '42'], '43'], '44'], '45'], '46'], '47'], '48'], '49'], '50'], '51'], '52'], '53'], '54'], '55'], '56'], '57'], '58'], '59'], '60'], '61'], '62'], '63'], '64'], '65'], '66'], '67'], '68'], '69'], '70'], '71'], '72'], '73'], '74'], '75'], '76'], '77'], '78'], '79'], '80'], '81'], '82'], '83'], '84'], '85'], '86'], '87'], '88'], '89'], '90'], '91'], '92'] Original list: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[['Xenophrys_bairdii'], 'Vulpanser_spaldingi'], 'Vipera_yeltoniensis'], 'Vipera_minutus'], 'Vanellus_vegans'], 'Vanellus_bedriagai'], 'Ursus_gratiosa'], 'Uroplatus_flavomaculatus'], 'Trionyx_leporosum'], 'Tringa_aspera'], 'Trachemys_sirtalis'], 'Terpsihone_varius'], 'Spermophilus_hassanica'], 'Siniperca_decor'], 'Scolopendra_marcianus'], 'Rufibrenta_montela'], 'Riparia_clinatus'], 'Rhinolophus_tuberculosus'], 'Rhacodactylus_citrsola'], 'Remiz_niloticus'], 'Pterinochilus_physalus'], 'Procellaria_truncatus'], 'Procellaria_lutris'], 'Poephagus_indica'], 'Platemys_albopillosum'], 'Pica_totanus'], 'Pica_mexicana'], 'Phrynomerus_pelagicus'], 'Philothamnus_tuberculosus'], 'Petrocincla_acuta'], 'Pelomedusa_rusticolus'], 'Pagophila_metallica'], 'Pachytriton_sibiricus'], 'Otis_moschata'], 'Otis_franckii'], 'Otis_emarginatus'], 'Opheodrys_clarus'], 'Mylopharyngodon_diffidens'], 'Mergus_carinata'], 'Meles_carbonaria'], 'Mabuya_carnivorus'], 'Lystrophis_siebenrocki'], 'Lyrurus_tinctorius'], 'Lycodon_dendrophila'], 'Lutra_plathyrhychos'], 'Leptopelis_plumipes'], 'Leptopelis_licin'], 'Leiurus_grossmani'], 'Lasiodora_hipposideros'], 'Larus_pulchripes'], 'Kinosternon_guineti'], 'Kassina_canorus'], 'Hysterocrates_chukar'], 'Hydrochelidon_euptilura'], 'Hemitheconyx_pulchripes'], 'Haplopelma_arcticus'], 'Hadrurus_dominus'], 'Gypaetus_schneideri'], 'Glareola_leucophyllata'], 'Gerrhosaurus_ruficollis'], 'Geochelone_filipjevi'], 'Gallinago_pallasii'], 'Eutamias_multifasciata'], 'Eulabeia_tinnunculus'], 'Eudrornias_penelope'], 'Eudramias_tadorna'], 'Eschrichtius_pulchra'], 'Eremophila_similis'], 'Equus_fluviatilis'], 'Epicrates_carinatus'], 'Emydura_avosetta'], 'Emberiza_japonica'], 'Dyscophus_rubicola'], 'Dendrelaphis_griseus'], 'Cypselus_ceterus'], 'Cynops_rutila'], 'Cygnus_rubicola'], 'Ctenosaura_sphenocercus'], 'Coenobita_variabilis'], 'Clemmys_caudata'], 'Cervus_comicus'], 'Castor_prominanus'], 'Casarca_holbrooki'], 'Capella_aestivus'], 'Buthus_nebularia'], 'Bronchocela_bicoloratum'], 'Branta_ferrumequinum'], 'Balaenoptera_piscator'], 'Athene_musculus'], 'Argynnis_versicolor'], 'Arenaria_javanica'], 'Anthropoides_aestivus'], 'Anolis_medirostris'], 'Anodonta_infrafrenata'], 'Ambystoma_alcinous'], 'Acanthoceros_euptilura']
推荐阅读
- azure - SSIS 目录:无法编译包中包含的脚本
- python - 如何用另一个字符替换字符串特定位置的字符
- python - 如何摆脱这个 python 正则表达式匹配对象中的 \t ?
- performance - LSTM评估指标MAE解释
- mule-component - Mule 4 中的 VM 消费和 VM 侦听器之间的区别?
- php - 使用字符串名称调用静态类的方法
- npm - 无法使用 npm 安装时间模块
- python - Python 3.x - 根据另一个列表对一个列表进行排序,然后返回排序后的两个列表
- javascript - page1 中的 xmlhttp.open 调用 page2 导致 page1 的 ID
- pointers - std::cin>> char* 出错,试图找到类似的问题并使用不同的代码,没有人有这个错误