mongodb - 使用 bitnami Helm 图表连接到托管在 Kubernetes 内的 MongoDB 副本集的异常

Question

如果我们尝试连接到托管在 kubernetes 环境中的 MongoDB 副本集，我们会遇到一些问题（超时和非主异常）。我们使用了 bitnami helm chart 8.2.1。如果我们连接到 MongoDB 4.2.3 的本地副本集安装，一切正常。所以我们假设问题一定出在 kubernetes 安装的配置上。如果我尝试将 MongoDB Compass 连接到 kubernetes 托管实例，它可以工作，但在 HOSTS 区域中没有可见的其他节点。连接到本地安装，所有节点都是可见的。

使用本地安装不是一种选择，因为我们希望我们的每个开发人员都有自己的数据库（当然还有相同的配置！）。

我们使用以下脚本安装了副本集：

helm repo add bitnami https://charts.bitnami.com/bitnami

# changing architecture or replica count, you must delete the persistance volume claims, otherwise mongodb starts with the previous configuration.
$values = @"
architecture: replicaset
replicaCount: 2
replicaSetName: testRS

auth:
  enabled: true
  rootPassword: admin

"@;

<# if I add these part to the above values variable, no connection could be established. Neither from MongoDB Compass nor from C# client
externalAccess:
  enabled: true
  service:
    type: LoadBalancer
    port: 27017
  autoDiscovery:
    enabled: true

serviceAccount:
  create: true

rbac:
  create: true
#>



if ( (helm list --namespace="infrastructure" --filter="mongodb" -q) -eq "mongodb") {
  $values | helm upgrade mongodb bitnami/mongodb --values - --namespace infrastructure --recreate-pods --version 8.2.1
}
else {
  $values | helm install mongodb bitnami/mongodb --values - --namespace infrastructure --version 8.2.1
}



helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
$values = @"
tcp:
  # MongoDb
  27017: "infrastructure/mongodb-headless:27017"

"@;


if ( (helm list --namespace="default" --filter="ingress-nginx" -q) -eq "ingress-nginx") {
  $values | helm upgrade ingress-nginx ingress-nginx/ingress-nginx --values - --namespace default
}
else {
  $values | helm install ingress-nginx ingress-nginx/ingress-nginx --values - --namespace default
}

dotnet core 示例应用程序：

using System;
using System.Threading.Tasks;
using MongoDB.Driver;

namespace MongoDBReplicaSet {
    class Program {
        static async Task Main(string[] args) {
            // Connecting to an on-premise installation (1 primary, 2 secondary) worked without any problems.
            //var connectionString = "mongodb://root:admin@on-prem:27080/?authSource=admin&replicaSet=testRS&readPreference=primary";

            var connectionString =
                "mongodb://root:admin@localhost:27017/?authSource=admin&readPreference=primary";

            await SimpleTask(connectionString);
            await Transaction(connectionString);
            Console.WriteLine("done.");
        }


        static async Task SimpleTask(string connectionString) {
            var client = new MongoClient(connectionString);

            var database = client.GetDatabase("TestDB");
            var collection = database.GetCollection<TestData>(nameof(TestData));
            /*
             * Can not connect to the kubernetes hosted replica set. MongoDB Compass and CLI can connect with the same connection string,
             * but in Compass, you don't see the other nodes under the HOSTS label
             *
             * System.TimeoutException: 'A timeout occured after 30000ms selecting a server using CompositeServerSelector{
             * Selectors = WritableServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }.
             * Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "ReplicaSet",
             * State : "Connected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/localhost:27017" }",
             * EndPoint: "Unspecified/localhost:27017", ReasonChanged: "Heartbeat", State: "Connected", ServerVersion: 4.2.8,
             * TopologyVersion: , Type: "ReplicaSetSecondary", WireVersionRange: "[0, 8]" ...
             *
             *
             * if eliminating the replicaset parameter from the connection string, the exception is different:
             * MongoDB.Driver.MongoNotPrimaryException: 'Server returned not master error.', but sometimes it worked.
             * It seems the client connecting by random to the primary and the other time to the secondary?
             */
            await collection.InsertOneAsync(new TestData(Guid.NewGuid().ToString(), "Test1", new DateTimeOffset(DateTime.Now)));
            Console.WriteLine("done simple task.");
        }


        static async Task Transaction(string connectionString) {
            var client = new MongoClient(connectionString);

            var database = client.GetDatabase("TestDB");

            // the same exception.

            var collection = database.GetCollection<TestData>(nameof(TestData));
            using (var session = await client.StartSessionAsync()) {
                session.StartTransaction();
                try {
                    await collection.InsertOneAsync(new TestData(Guid.NewGuid().ToString(), "Test2", new DateTimeOffset(DateTime.Now)));

                    await session.CommitTransactionAsync();
                }
                catch (Exception ex) {
                    await session.AbortTransactionAsync();
                }
            }

            Console.WriteLine("done transaction.");
        }
    }

    public class TestData {
        public string Id { get; set; }
        public string Name { get; set; }
        public DateTimeOffset CreatedAt { get; private set; }

        public TestData(string id, string name, DateTimeOffset createdAt) {
            Id = id;
            Name = name;
            CreatedAt = createdAt;
        }
    }
}

有没有人在 Kubernetes 内部使用 MongoDB Replica Set 并从外部毫无例外地访问它的解决方案？

我会非常感谢帮助我们。

标签： mongodbkubernetesbitnami