python - 多线程 python 发布请求

Question

我需要多线程方面的帮助。

我没有这方面的经验，我不知道它有多准确。我对 api 提出了一个 post 请求，并每 1 秒得到一个响应。我在网上看到可以一次发送多个请求并获得多个回复（如果我错了，请见谅）

这是一个帖子请求：

url = 'https://open-api.trovo.live/openplatform/channels/id'
payload = {"channel_id":100379073}
headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
r = requests.post(url, data=payload, headers=headers)

最大 QPS 是 20，我想在一秒钟内发送 15 个请求，如果有人可以告诉我这是否可以做到？

我想做这样的事情：

url = 'https://open-api.trovo.live/openplatform/channels/id'
users = ['1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234']
headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
for users in users:
    payload = {"channel_id":users}
    r = requests.post(url, data=payload, headers=headers)
    

标签： pythonmultithreadingapipython-requests

Answer 1

对于 python 中的 GET 或 POST 异步请求，您应该使用concurrent.futures库。这是一个例子

max_workers是一个取决于您的计算机的参数

import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import time

# ['1234', '1234', ...] 20 times
users = ['1234'] * 20

def make_request(user):
    url = 'https://open-api.trovo.live/openplatform/channels/id'
    headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
    payload = {"channel_id": user}
    return requests.post(url, data=payload, headers=headers)


# standard request
start = time.time()    
for user in users:
    make_request(user)
print(time.time() - start, 's')

# Output: 
# 4.139220714569092s



# asynchronous
start = time.time()  
processes = []
with ThreadPoolExecutor(max_workers=10) as executor:
    for user in users:
        processes.append(executor.submit(make_request, user))
        
for task in as_completed(processes):
    print(task.result())
print(time.time() - start, 's')

# Output:
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# 0.499767541885376s