时间戳

首页 > 解决方案 > nginx proxy_pass 无法从输出流中获得完整的响应数据

java - nginx proxy_pass 无法从输出流中获得完整的响应数据

问题描述

后端端点使用 k8s-client 使用以下代码查看 POD 日志。

    @RequestMapping(value = "/{podName}/log", method = RequestMethod.GET)
    public void queryLog(HttpServletResponse response, @PathVariable(name = "podName", required = true) String podName,
                         @RequestParam(name = "container", required = true) String container,
                         @RequestParam(name = "tailLines", required = false) Integer tailLines,
                         @RequestParam(name = "timestamps", required = false) boolean timestamps,
                         @RequestParam(name = "follow", required = false) boolean follow) {
        try {
            PodLogs logs = new PodLogs();
            InputStream is = logs.streamNamespacedPodLog(namespace, podName, container, null, tailLines, timestamps, follow);
            copy(is, response.getOutputStream());

        } catch (Exception ex) {
            logger.error("exception", ex);
        }
    }

    public static void copy(InputStream in, OutputStream out) throws IOException {

        try {
            byte[] buffer = new byte[4096];
            int bytesRead;
            while ((bytesRead = in.read(buffer)) != -1) {
                String str = new String(buffer, 0, bytesRead);
                out.write(buffer, 0, bytesRead);
                out.flush();
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            if (in != null) {
                in.close();
            }
            if (out != null) {
                out.close();
            }
        }

    }

前方：

<html>
    <head>
        <title>Test</title>
        <script type="text/javascript">
        
        var xhr = null;
        
         function setConnected(connected) {
                
                document.getElementById('connect').disabled = connected;
                document.getElementById('disconnect').disabled = !connected;
                document.getElementById('response').innerHTML = '';
            }
          
         function connect() {
        
          xhr = new XMLHttpRequest()

          xhr.open('GET', `http://proxy.k8s.io/backend/api/pod-0/log?container=pod&tailLines=500&timestamps=true&follow=true`, true)

          xhr.onreadystatechange = () => {
            setConnected(true);
            if (xhr.readyState >= 3 && xhr.status === 200) {
               console.log("###")
               console.log(xhr.responseText)
               document.getElementById('response').innerHTML = document.getElementById('response').innerHTML + xhr.responseText;
            }
          }

          xhr.send()
        }
         function disconnect() {
                
                if(xhr != null) {
                    xhr.abort();
                }
                
                setConnected(false);
                console.log("Disconnected");
            }
        </script>
        
    </head>
    
    <body  onload="disconnect()">
     <div>
        <button id="connect" onclick="connect();">Connect</button>
        <button id="disconnect" disabled="disabled" onclick="disconnect();">Disconnect</button>
        </div>
    <div id="response"></div>
    </body>
</html>

我可以获得完整的日志并在没有 nginx的情况下观看新生成的日志。但是具有以下配置的 nginx 无法从后端获取完整的日志。

upstream backend {
        server backend:8080;
        keepalive_timeout 60s;
        keepalive 60;
        keepalive_requests 100;
}

server {
    listen       80;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    location /backend/api/ {
        proxy_pass http://backend/;
    }
}

预期的日志样本：

2021-08-02T02:51:04.857054753Z  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
2021-08-02T02:51:04.857057468Z  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
2021-08-02T02:51:04.857060467Z  at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
2021-08-02T02:51:04.857063232Z  at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

我得到的示例日志：

2021-08-02T02:51:04.857054753Z  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
2021-08-02T02:51:04.857057468Z  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
2021-08-02T02:51:04.857060467Z  at org.springframework.web.filter.HiddenHt

看起来最后一个分块数据丢失了。当我用 访问它时tailLines=200，一些数据仍然丢失。并且当生成新的日志时，没有新的数据可以显示在首页。

标签： javanginxkubernetes

解决方案

根本原因是不正确的proxy_http_version。默认情况下，使用 1.0 版本。应使用 1.1 版进行保活连接。

文档说：

语法：proxy_http_version 1.0 | 1.1；
默认值：
proxy_http_version 1.0；
上下文：http、server、location
该指令出现在 1.1.4 版本中。

设置代理的 HTTP 协议版本。默认情况下，使用 1.0 版。建议将 1.1 版与 keepalive 连接和 NTLM 身份验证一起使用。

如何修复：在位置块下
添加。proxy_http_version 1.1;

    location /backend/api/ {
        proxy_http_version    1.1;
        proxy_pass http://backend/;
    }

推荐阅读