时间戳

首页 > 解决方案 > 泳池系统错误

python - 泳池系统错误

问题描述

我正在尝试将池系统添加到我的应用程序中，但我认为我做错了什么。

我的代码：

import requests, threading, platform, os
from multiprocessing.dummy import Pool


if platform.system() == "Linux":
    clear = lambda: os.system('clear')
    clear()
if platform.system() == "Windows":
    clear = lambda: os.system('cls')
    clear()
        
            
def check(data):
    try:
        r = requests.get(data)
        if 'working ' in r.text:
            with open("working.txt", mode="a") as urlsvuln:
                print("Working : " + r.url)
                urlsvuln.write(r.url + "\n")
                         
        else:
            print("Not working")
    except Exception as e:
        print(e)      


with open("texte.txt", mode="r") as mf:
    lines = mf.read().splitlines() 
    for line in lines:
        data = line + "'" 
        print(data)
try:    
    pp = Pool(10)
    pr = pp.map(check, data)
except Exception as e:
    print(e)

所以这段代码可以正常工作，但我可以看到我是否有一个包含 3 行的文本文件来查看 url 是否正常工作，并且我看到我的应用程序正在执行 10 次请求，并且在我的文本中写入 10 倍。

标签： pythonpython-requestspool

解决方案

我认为这将实现您正在尝试做的事情

import requests, platform, os
from multiprocessing.dummy import Pool
from io import StringIO

if platform.system() == "Linux":
    clear = lambda: os.system('clear')
    clear()
if platform.system() == "Windows":
    clear = lambda: os.system('cls')
    clear()


def check(data):
    try:
        r = requests.get(data, timeout=2)
        if r.text:
            out.write(f"Working {data}\n")
        else:
            out.write(f"Not working {data}\n")
    except Exception as e:
        out.write(f"Not working {data}\n")

urls = StringIO("https://www.google.com\nhttps://www.yahoo.com\nhttps://www.nonexistent.com")
# Note I am using in-memory file here but you can read the urls from file here
data = urls.read().splitlines()

try:
    with open("working.txt", mode="a") as out:
        pp = Pool(10)
        pr = pp.map(check, data)
except Exception as e:
    print(e)

虽然multiprocessing.dummy.Pool在这里完成了工作，但我强烈建议ThreadPoolExecutor在这里使用以使其无阻塞。

线程池执行器

推荐阅读