python - aiohttp - 如何绕过 Duplicate Content-Length Header 错误并读取响应标头？

Question

问题

我正在抓取一个网站，当发生错误时，服务器会返回一个带有重复Content-Lenth标头的 HTTP 响应（如服务器片段中的那个）。在此响应中，重要数据存储在标头中X-error-message，并且X-error-code

如何重现？

服务器片段

import socket
import time

s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('localhost', 5000))
s.listen(5)

while True:
    new_socket, _ = s.accept()
    data = b''

    while not data.endswith(b'\r\n\r\n'):
        data += new_socket.recv(0)

    new_socket.sendall(
        b'HTTP/1.1 499 UNKNOWN\r\n'
        b'Content-Length: 0\r\n'
        b'Content-Length: 0\r\n'
        b'X-error-message: MESSAGE ERROR\r\n'
        b'X-error-code: 23\r\n\r\n'
    )
    time.sleep(1)

客户端片段

import aiohttp
import asyncio

async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get('http://localhost:5000/') as resp:
            print(resp.status)
            print(await resp.text())

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    result = loop.run_until_complete(main())

错误

Traceback (most recent call last):
  File "client.py", line 16, in <module>
    result = loop.run_until_complete(main())
  File "/Users/x/miniconda3/envs/aiohttp-debug/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "client.py", line 7, in main
    async with session.get('http://localhost:5000/') as resp:
  File "/Users/x/miniconda3/envs/aiohttp-debug/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/Users/x/miniconda3/envs/aiohttp-debug/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/Users/x/miniconda3/envs/aiohttp-debug/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 900, in start
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Duplicate Content-Length', url=URL('http://localhost:5000/')

问题

我应该如何在响应中抓取这些标头？有没有办法绕过Content-Length标头？

标签： pythonhttpaiohttp