首页 > 解决方案 > 在 python 上使用多处理库时出错,脚本死亡

问题描述

我是 python 和编码的新手,(我知道我的代码可能看起来很难看)。

我有这个小脚本,它将解析来自某些设备的数据。

库存功能可以正常工作,使用 for 循环,因此一次只能处理一项。

我添加了“主要”功能,如果我传递一个非常短的列表,其中包含一些项目,它也可以正常工作。我真的不明白发生了什么:/

如何使脚本同时处理多个项目?有什么建议吗?

但是如果添加更多设备,所有脚本都会被破坏并且解释器

UnboundLocalError:分配前引用的局部变量“net_connect”

lista_dev=["dev1","dev2","dev3","dev4"]

def inventory(router):

        """send commands into network devices and save the collected data as a python dictionary"""
      try:
        net_connect=jump(router,username,password)
    except:
        pass
     facts=(net_connect.send_command("sho ver", use_textfsm=True))[0]
     serial=facts.get('serial')[0]
     config_register=facts.get('config_register') 
     model=facts.get('hardware')[0]
     host=net_connect.send_command("show run | in hostname").split()[1]
     print(host)
     bootflash_content=(net_connect.send_command("dir | in bin", use_textfsm=True))
     current_binaries=[i.get("name") for i in bootflash_content]
     net_connect.send_command("terminal lengh 0")    
     space=(net_connect.send_command("dir", use_textfsm=True))
     free_space=int(space[-1].get("total_free"))
     running_image=facts.get('running_image')
     cpld=(net_connect.send_command("show platform diag | in CPLD").splitlines())[0].split(",")[0].split(" ")[-1]       
      diccionario=({'device': host , 'model': model, 'current_version': running_image,'serial': serial ,
                                'config_reg': config_register,'space_availale_in_bytes': free_space,'clpd_version': cpld,
                                'current_binaries': current_binaries })

 output_q.put(diccionario)

def main():
"""process the list using a Queue to read multiple devices at the time"""
    start_time = datetime.now()
    output_q = Queue(maxsize=5)

    procs = []
    for a_device in lista_dev:
        my_proc = Process(target=device_inventory, args=(a_device, output_q))
        my_proc.start()
        procs.append(my_proc)

    # Make sure all processes have finished
    for a_proc in procs:
        a_proc.join()

    while not output_q.empty():
        #print(output_q.get())
        with open (json_file , "a", encoding='utf-8') as file:
            file.write(output_q.get()+'\n')


main()



Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "inventory.py", line 38, in device_inventory
    if net_connect is not None:
UnboundLocalError: local variable 'net_connect' referenced before assignment```












标签: pythonmultithreadingqueue

解决方案


我不知道我的原始代码出了什么问题,但是使用库 concurrent.futures 比使用队列更有效。

这个链接帮助我弄清楚。

https://beckernick.github.io/faster-web-scraping-python/#fromHistory


推荐阅读