首页 > 解决方案 > 我在做什么错以及如何使用 python 的 asyncio 调试内存泄漏?

问题描述

我有这样的代码:

async def worker(self):
    while True:
        proxies = get_proxies()
        for proxy in proxies:
            await self.add_proxy_to_queue(proxy)

async def add_proxy_to_queue(self, proxy):
    while self.proxies_semaphore.locked():
        await asyncio.sleep(0.001)

    asyncio.ensure_future(self.process_proxy(proxy))

async def process_proxy(self, proxy):
    async with self.proxies_semaphore:
        # send network request and wait for the resposne with timeout

我已经改变了它:

async def worker(self):
    while True:
        proxies = get_proxies()
        for proxy in proxies:
            await self.add_proxy_to_queue(proxy)

async def add_proxy_to_queue(self, proxy):
    async with self.proxies_semaphore:  # <-- HERE ARE THE CHANGES
        asyncio.ensure_future(self.process_proxy(proxy))

async def process_proxy(self, proxy):
    async with self.proxies_semaphore:
        # send network request and wait for the resposne with timeout

它仍然在泄漏,但速度要慢得多。

我做错了什么以及如何调试异步代码,如何查看事件循环并查看实际存在哪些任务?

标签: pythonmemory-leakspython-asyncio

解决方案


推荐阅读