时间戳

首页 > 解决方案 > Spring Webflux 性能测试抛出 PoolAcquirePendingLimitException

spring - Spring Webflux 性能测试抛出 PoolAcquirePendingLimitException

问题描述

我正在尝试学习 Spring webflux。我编写了以下代码来测试反应式编程的性能。这是我的一项服务的控制器：

@RestController
public class PlayerController {

    @Autowired
    private PlayerRepository playerRepository;

    @GetMapping("/players/{id}")
    public Mono<Player> getPlayerById(@PathVariable int id) {
        return playerRepository.findById(id);
    }

型号类：

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Player {

    @Id
    private Integer id;
    private String name;
    private Integer age;
}

存储库：

@Repository
public interface PlayerRepository extends ReactiveCrudRepository<Player, Integer> {}

以下是调用上述服务然后进行数据库调用然后检查它们是否返回相同数据的客户端。客户端的存储库和模型是相同的。

@RestController
public class PlayerController {

    @Autowired
    private PlayerRepository playerRepository;

    @Autowired
    private WebClient webClient;

    @GetMapping("/players/{id}")
    public Mono<Response> getPlayerById(@PathVariable int id) {
        LocalDateTime start = LocalDateTime.now();
        Mono<Player> playerExternal = webClient.get().uri("/players/{id}", id).retrieve().bodyToMono(Player.class);
        Mono<Player> playerDB = playerRepository.findById(2);


        Mono<Response> result = playerExternal.zipWith(playerDB, (ext, db) -> {
            Response response = new Response();
            response.setFlag(ext.getId() == db.getId());
            LocalDateTime end = LocalDateTime.now();
            response.setTimeTaken(end.from(start).until(end, ChronoUnit.MILLIS) + " ms");

            return response;

        });

        return result;
    }

pom.xml

    <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.0</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>reactive-spring</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>reactive-spring</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-r2dbc</artifactId>
        </dependency>
        <dependency>
            <groupId>dev.miku</groupId>
            <artifactId>r2dbc-mysql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

客户端运行在8080端口，调用服务运行在9090端口。我使用的是spring boot 2.5.0。我也在使用 jmeter 来模拟 10000 个并发请求。在执行 jmeter 测试计划时，我收到以下异常，

org.springframework.web.reactive.function.client.WebClientRequestException: Pending acquire queue has reached its maximum size of 1000; nested exception is reactor.netty.internal.shaded.reactor.pool.PoolAcquirePendingLimitException: Pending acquire queue has reached its maximum size of 1000
    at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141) ~[spring-webflux-5.3.7.jar:5.3.7]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    |_ checkpoint ⇢ Request to GET http://localhost:9090/players/1 [DefaultWebClient]
    |_ checkpoint ⇢ Handler com.example.demo.controller.PlayerController#getPlayerById(int) [DispatcherHandler]
    |_ checkpoint ⇢ HTTP GET "/players/1" [ExceptionHandlingWebHandler]
Stack trace:
        at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141) ~[spring-webflux-5.3.7.jar:5.3.7]
        at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.Mono.subscribe(Mono.java:4150) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:221) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93) ~[reactor-core-3.4.6.jar:3.4.6]
        at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204) ~[reactor-core-3.4.6.jar:3.4.6]

如果我需要提供完整的堆栈跟踪，请告诉我。如果我在做任何我不应该做的事情，请告诉我。我也可以分享 jmeter 测试结果。我看到很多请求都失败了，其中大多数都需要很长时间（范围在 1 秒到 18 秒之间）才能响应。

标签： springspring-bootspring-webfluxspring-webclientreactor-netty

解决方案

默认情况下WebClient使用连接池运行。池的默认设置是最多 500 个连接和最多 1000 个待处理请求。您已经JMeter并尝试模拟 10000，但您没有指定如何分配负载。您可能需要增加最大挂起请求数。看看这个文档和这个文档。

如果要配置，WebClient则需要：

@Bean
public ReactorResourceFactory resourceFactory() {
    ConnectionProvider provider =
            ConnectionProvider.builder("test")
                              .maxConnections(500)
                              // Set custom max pending requests
                              .pendingAcquireMaxCount(...)
                              .build();
    ReactorResourceFactory factory = new ReactorResourceFactory();
    factory.setUseGlobalResources(false); 
    factory.setConnectionProvider(provider);
    return factory;
}

@Bean
public WebClient webClient() {

    Function<HttpClient, HttpClient> mapper = client -> {
        // Further customizations...
    };

    ClientHttpConnector connector =
            new ReactorClientHttpConnector(resourceFactory(), mapper); 

    return WebClient.builder().clientConnector(connector).build(); 
}

推荐阅读