hibernate - 用于第二个 Hibernate 缓存的 Hazelcast 实例无法在 Kubernetes 中稳定连接

问题描述

我正在使用部署在 Kubernetes（GKE）中的嵌入式 Hazelcast 在 Spring Boot 应用程序中启用二级缓存 Hibernate。Spingboot 应用程序使用 spanner-jdbc ( https://github.com/olavloite/spanner-jdbc ) 连接到 Spanner。我将 1 个服务 (replicas=2) 部署到 k8s，但是位于 2 个 pod 中的 hazelcast 实例无法稳定连接（它在服务器启动时工作，但之后似乎断开连接，请参阅下面的日志）。这会导致 2 级缓存无法在 2 个 pod 之间共享。任何人都可以帮助这种情况？谢谢你。

<groupId>com.hazelcast.poc</groupId>
<artifactId>hazelcast-poc</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>hazelcast-poc</name>
<description>Demo project for Spring Boot</description>

<properties>
    <java.version>1.8</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>nl.topicus</groupId>
        <artifactId>spanner-jdbc</artifactId>
        <version>1.0.8</version>
    </dependency>

    <dependency>
        <groupId>nl.topicus</groupId>
        <artifactId>spanner-hibernate</artifactId>
        <version>0.7</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-rest</artifactId>
    </dependency>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast</artifactId>
        <version>3.11</version>
    </dependency>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-hibernate5</artifactId>
        <version>1.3.0</version>
    </dependency>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-kubernetes</artifactId>
        <version>1.3.1</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.8.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest-all</artifactId>
        <version>1.3</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.uuid</groupId>
        <artifactId>java-uuid-generator</artifactId>
        <version>3.1.5</version>
    </dependency>
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-java8</artifactId>
        <version>${hibernate.version}</version>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <version>2.0.1.RELEASE</version>
            <executions>
                <execution>
                    <goals>
                        <goal>repackage</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
                <encoding>UTF-8</encoding>
            </configuration>
        </plugin>
    </plugins>
</build>

Springboot 应用程序.java：

public static void main(String[] args) throws IOException {
    CloudSpannerDriver.setLogLevel(CloudSpannerDriver.DEBUG);
    Writer writer = new FileWriter("jdbc.log", false);
    DriverManager.setLogWriter(new PrintWriter(writer));SpringApplication.run(Application.class, args);
}

@Bean
public Config hazelCastConfig(){
    Config config = new Config();
    config
            .addMapConfig(
                    new MapConfig()
                            .setName("employees")
                            .setMaxSizeConfig(new MaxSizeConfig(200, MaxSizeConfig.MaxSizePolicy.FREE_HEAP_SIZE))
                            .setEvictionPolicy(EvictionPolicy.LRU)
                            .setTimeToLiveSeconds(20000));
    JoinConfig joinConfig = config.getNetworkConfig().getJoin();
    joinConfig.getMulticastConfig().setEnabled(false);
    joinConfig.getKubernetesConfig().setEnabled(true).setProperty("namespace", "default");

    return config;
}

}

部署.yaml：

apiVersion: apps/v1
kind: Deployment
metadata:
 name: hazelcast-embedded-0
spec:
  replicas: 2
  selector:
  matchLabels:
  app: hazelcast-embedded-0
  template:
   metadata:
    labels:
      app: hazelcast-embedded-0
    spec:
    volumes:
      - name: hazelcast-key
        secret:
        secretName: hazelcast-key
  containers:
    - name: hazelcast-embedded
      image: gcr.io/testproject-221915/hazelcast- poc@sha256:5a103d7e2e56c46e9e42617232e472891bda5d121139fe7e41057596bd71bf00
      imagePullPolicy: Always
      volumeMounts:
        - name: hazelcast-key
          mountPath: /var/secrets/google
      env:
      - name: GOOGLE_APPLICATION_CREDENTIALS
        value: /var/secrets/google/key.json
      ports:
       - containerPort: 5701
       - containerPort: 8080


  apiVersion: v1
  kind: Service
  metadata:
  name: hazelcast-embedded-0
  spec:
   type: LoadBalancer
   selector:
    app: hazelcast-embedded-0
   ports:
   - name: hazelcast
    port: 5701
   - name: app
     port: 8080

扳手数据源文件：

@Bean
public LocalContainerEntityManagerFactoryBean spannerEntityManager() {
    LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
    em.setDataSource(spannerDataSource());
    em.setPersistenceUnitName("SpannerPU");
    HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
    em.setJpaVendorAdapter(vendorAdapter);
    HashMap<String, Object> properties = new HashMap<>();
    properties.put("hibernate.dialect", "nl.topicus.hibernate.dialect.CloudSpannerDialect");
    properties.put("hibernate.temp.use_jdbc_metadata_defaults", false);
    properties.put("hibernate.show_sql", showSql);
    properties.put("hibernate.format_sql", formatSql);
    properties.put("hibernate.cache.use_query_cache", true);
    properties.put("hibernate.cache.use_second_level_cache", true);
    properties.put("hibernate.cache.region.factory_class", "com.hazelcast.hibernate.HazelcastCacheRegionFactory");
    if("development".equals(profileActive)) {
        properties.put("hibernate.generate_statistics", statistic);
        properties.put("org.hibernate.stat", hibernateStat);            
    }        
    em.setJpaPropertyMap(properties);

    return em;
}

应用程序属性

spanner.datasource.className=nl.topicus.jdbc.CloudSpannerDriver
spanner.datasource.url=jdbc:cloudspanner://;Project=testproject- 
221915;Instance=test-hazelcast;Database=hazelcast
spanner.hibernate.dialect=
 nl.topicus.hibernate.dialect.CloudSpannerDialect
spanner.hibernate.show-sql=true
spanner.hibernate.format_sql=true
spanner.datasource.hikari.maximum-pool-size=2 #depend on 
 microservice
spanner.datasource.hikari.minimum-idle=2
spanner.datasource.hikari.connection-timeout=60000

spring.profiles.active=dev
hibernate.generate_statistics=true
org.hibernate.stat=DEBUG

日志_pod1 日志 _pod2

如您所见，第二个日志只有 1 个成员，而第一个日志有 2 个成员。我通过向服务发出休息请求然后分发到每个 Pod 进行测试，测试结果有时是缓存在两个 Pod 之间共享，有时不是。

我预计第二个 Hibernate 缓存应该在服务中的 2 个 pod 之间共享

标签： hibernatespring-bootkuberneteshazelcast

hibernate - 用于第二个 Hibernate 缓存的 Hazelcast 实例无法在 Kubernetes 中稳定连接

问题描述

解决方案

推荐阅读