java - 在 HTTP 标头可用时读取它们

Question

我目前需要在发送 HTTP 标头时读取它们。我已经尝试过新的 Java 11 HTTP 客户端（java.net.http.HttpClient），但我没有运气拦截标头，因为在开始发送内容之前它们不可用（因此所有标头都立即返回） .

我需要此功能来处理进度/状态更新，因为服务器会发送多个带有操作进度的标头，然后在准备好后正常发送响应。这不是我的设计（响应是从 ClickHouse 发送的），所以我只需要能够找到一个可以读取它的 HTTP 客户端。

curl 命令行工作得很好：

curl -vsS "http://localhost:8123?send_progress_in_http_headers=1&wait_end_of_query=1" -d "select count() from numbers(100000000000) format JSON;"
*   Trying 127.0.0.1:8123...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8123 (#0)
> POST /?send_progress_in_http_headers=1&wait_end_of_query=1 HTTP/1.1
> Host: localhost:8123
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 54
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 54 out of 54 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sat, 20 Feb 2021 11:35:14 GMT
< Connection: Keep-Alive
< Content-Type: application/json; charset=UTF-8
< X-ClickHouse-Server-Display-Name: 78b40ec2d4a2
< Transfer-Encoding: chunked
< X-ClickHouse-Query-Id: 19bd4955-09ad-47f2-a64f-93d926383f55
< X-ClickHouse-Format: JSON
< X-ClickHouse-Timezone: UTC
< Keep-Alive: timeout=3
< X-ClickHouse-Progress: {"read_rows":"100794368","read_bytes":"806354944","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"249954304","read_bytes":"1999634432","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"414515200","read_bytes":"3316121600","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"558759936","read_bytes":"4470079488","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"722206720","read_bytes":"5777653760","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"901185536","read_bytes":"7209484288","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"1110573056","read_bytes":"8884584448","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"1323302912","read_bytes":"10586423296","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"1534722048","read_bytes":"12277776384","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"1729953792","read_bytes":"13839630336","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"1943535616","read_bytes":"15548284928","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"2165768192","read_bytes":"17326145536","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"2382692352","read_bytes":"19061538816","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"2597519360","read_bytes":"20780154880","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"2807758848","read_bytes":"22462070784","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"3021733888","read_bytes":"24173871104","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"3234922496","read_bytes":"25879379968","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"3457875968","read_bytes":"27663007744","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}
< X-ClickHouse-Progress: {"read_rows":"3663790080","read_bytes":"29310320640","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000000000"}

我可以从客户端的调试日志中看到数据是增量读取和解析的，但是无法访问我知道的信息：

...

2021-02-21 10:31:03,070 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [747ms] SelectorAttachment Registering jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent@7e37cce4 for 0 (false)
2021-02-21 10:31:03,072 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [749ms] SocketTube(1) read bytes: 490
2021-02-21 10:31:03,072 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [750ms] Http1AsyncReceiver(SocketTube(1)) Putting 490 bytes into the queue
2021-02-21 10:31:03,073 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [750ms] SocketTube(1) resuming read event
2021-02-21 10:31:03,073 DEBUG [HttpClient-1-Worker-0] [jdk.internal.httpclient.debug:286] - [HttpClient-1-Worker-0] [750ms] Http1AsyncReceiver(SocketTube(1)) Got 490 bytes for delegate jdk.internal.net.http.Http1Response$HeadersReader@2bcbc994
2021-02-21 10:31:03,073 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [750ms] SelectorAttachment Registering jdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent@7e37cce4 for 1 (false)
2021-02-21 10:31:03,073 DEBUG [HttpClient-1-Worker-0] [jdk.internal.httpclient.debug:286] - [HttpClient-1-Worker-0] [750ms] Http1AsyncReceiver(SocketTube(1)) downstream subscription demand is 1
2021-02-21 10:31:03,073 DEBUG [HttpClient-1-SelectorManager] [jdk.internal.httpclient.debug:286] - [HttpClient-1-SelectorManager] [750ms] SocketTube(1) leaving read() loop after onNext:  Reading: [ops=1, demand=0, stopped=false], Writing: [ops=0, demand=1]
....

关于如何在信息可用时提取信息的任何想法？似乎有多个异步 HTTP 客户端，但它们没有标头回调。

编辑：我在这里分享了我的项目：https ://github.com/ethlo/crackshack

有问题的问题必须在这里处理：https ://github.com/ethlo/clackshack/blob/main/src/main/java/com/ethlo/clackshack/Java11Client.java

该项目包含一个使用简单 HTTP 服务器的测试，以 500 毫秒延迟响应 HTTP 标头，以避免依赖 ClickHouse 进行复制。

标签： javahttp-headersclickhousejava-http-client