diff --git a/charts/mongodb/DESIGN.md b/charts/mongodb/DESIGN.md index 705b97646..85947cd62 100644 --- a/charts/mongodb/DESIGN.md +++ b/charts/mongodb/DESIGN.md @@ -18,7 +18,10 @@ line behavior, and official image lifecycle. Standalone and replica set modes use StatefulSets so pod identity and storage are stable. Replica set mode adds a Helm hook Job that initializes the replica set -after members are reachable. Sharded mode deploys config servers and shard +after members are reachable. When enabled, the arbiter runs in a separate +single-replica StatefulSet with stable DNS and ephemeral local storage. The +post-install/post-upgrade hook registers it as `arbiterOnly` and reconciles its +presence without routing client traffic to it. Sharded mode deploys config servers and shard members as StatefulSets, mongos as a Deployment, and a hook Job that initializes the cluster and registers shards. @@ -44,6 +47,11 @@ validated with client connection strings and election behavior. Sharded clusters require shard key design, balancer monitoring, and capacity planning outside the chart. +The chart never removes data-bearing replica set members automatically. An +operator must use MongoDB's member-removal procedure before lowering +`replicaSet.members`. Arbiter enablement is narrower: the hook safely reconciles +the single known arbiter hostname on install and upgrade. + The backup CronJob performs logical `mongodump` archives to S3-compatible object storage. It is suitable for scheduled full backups and restore drills, but it is not a point-in-time recovery system or a substitute for storage snapshots where diff --git a/charts/mongodb/README.md b/charts/mongodb/README.md index 0c5dc2f5f..f85d2b533 100644 --- a/charts/mongodb/README.md +++ b/charts/mongodb/README.md @@ -49,6 +49,7 @@ Read before choosing an architecture: - **Official `mongo` image** — no vendor lock-in, standard paths (`/data/db`) - **Auto keyFile generation** — replica set internal auth handled automatically +- **Optional arbiter** — one lightweight voting member for cost-constrained replica sets - **Helm hook Jobs** — idempotent `rs.initiate()` and shard registration - **Prometheus exporter** — optional `percona/mongodb_exporter` sidecar + ServiceMonitor - **Built-in S3 backups** — scheduled `mongodump` archive upload for standalone, replica set, and sharded topologies @@ -121,6 +122,22 @@ compatible with `--configsvr` or `--shardsvr`. Production sharded authentication requires an operator-managed bootstrap flow outside this chart's automatic root user initialization path. +### Replica Set with Arbiter + +```yaml +architecture: replicaset +replicaSet: + members: 2 +arbiter: + enabled: true +``` + +This topology creates two persistent data-bearing members and one arbiter with +ephemeral local storage. MongoDB recommends three data-bearing members when +capacity permits. Use an arbiter only when the cost of a third data copy is the +deciding constraint, and review the +[official arbiter considerations](https://www.mongodb.com/docs/manual/core/replica-set-arbiter/). + ## Parameters ### Global @@ -150,6 +167,8 @@ user initialization path. |-----------|-------------|---------| | `replicaSet.name` | Replica set name | `rs0` | | `replicaSet.members` | Number of data-bearing members | `3` | +| `arbiter.enabled` | Add one non-data-bearing voting member | `false` | +| `arbiter.resources` | Arbiter container resources | `{}` | ### Sharded Cluster Parameters @@ -241,6 +260,7 @@ user initialization path. |-------------|-----------| | standalone | Secret, 2x Service (headless + client), StatefulSet | | replicaset | 2x Secret (auth + keyfile), 2x Service, StatefulSet, Job (rs-init) | +| replicaset + arbiter | 2x Secret, 3x Service, 2x StatefulSet, Job (rs-init) | | sharded | 2x Secret, 4x Service, 3x StatefulSet (config + shards), Deployment (mongos), Job (init) | ## Examples @@ -249,6 +269,7 @@ See the [`examples/`](examples/) directory: - [`standalone-simple.yaml`](examples/standalone-simple.yaml) — Minimal standalone instance - [`replicaset-production.yaml`](examples/replicaset-production.yaml) — Production RS with monitoring, init scripts, and anti-affinity +- [`replicaset-arbiter.yaml`](examples/replicaset-arbiter.yaml) — Two data members with one voting arbiter - [`sharded-cluster.yaml`](examples/sharded-cluster.yaml) — Full sharded cluster ## Architecture Guides @@ -270,6 +291,21 @@ Keep replica set keyFiles and root credentials stable across `helm upgrade`; those values are initialized by MongoDB and should be rotated with MongoDB administrative commands instead of changing chart values. +Enabling or disabling `arbiter.enabled` on an existing replica set reconciles +the arbiter membership through the post-upgrade hook. Increasing +`replicaSet.members` adds missing data-bearing members. Reducing that value does +not remove members from the MongoDB configuration; remove members safely with +MongoDB administrative procedures before scaling down the StatefulSet. + +MongoDB requires a global default write concern before a reconfiguration that +would change its implicit value. When arbiter reconciliation encounters an +implicit default, the hook preserves the currently effective value (`w: 1` or +`majority`) by promoting it to a global default. Existing operator-managed +global defaults are not changed. The same preservation runs before adding a +missing data-bearing member because scale-up can also change the implicit +default. MongoDB 5.0 and newer do not allow a global default write concern to be +unset after it is established. + ## Security Scan Security Scan: Kubescape local scan against `MITRE,NSA,SOC2` reports a 72.73% resource summary score. diff --git a/charts/mongodb/ci/arbiter-values.yaml b/charts/mongodb/ci/arbiter-values.yaml new file mode 100644 index 000000000..08b314ab7 --- /dev/null +++ b/charts/mongodb/ci/arbiter-values.yaml @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: Apache-2.0 +architecture: replicaset + +auth: + enabled: true + rootPassword: arbiter-ci-password + replicaSetKey: 0123456789abcdef0123456789abcdef + +replicaSet: + name: rs0 + members: 2 + +arbiter: + enabled: true + resources: + requests: + cpu: 25m + memory: 64Mi + limits: + cpu: 100m + memory: 512Mi + +persistence: + enabled: false + +resources: + requests: + cpu: 50m + memory: 128Mi + limits: + cpu: 250m + memory: 1Gi diff --git a/charts/mongodb/docs/replicaset.md b/charts/mongodb/docs/replicaset.md index 1a6021e52..efbbc85b7 100644 --- a/charts/mongodb/docs/replicaset.md +++ b/charts/mongodb/docs/replicaset.md @@ -15,6 +15,7 @@ Common cases: - multiple data-bearing members - automatic `rs.initiate()` bootstrap via Helm hook job +- an optional non-data-bearing arbiter for an odd number of election votes - internal member authentication with key file - a standard MongoDB replica set topology - optional metrics and `ServiceMonitor` @@ -42,6 +43,7 @@ secondaries replicating from it, and elections on failure. ## Common risks - running only 2 members and expecting safe elections +- treating an arbiter as a replacement for data redundancy - forgetting the replica set connection string in clients - scheduling all members in the same node or zone - ignoring backup and restore testing because failover exists @@ -62,6 +64,8 @@ secondaries replicating from it, and elections on failure. | `architecture` | Must be `replicaset` | | `replicaSet.name` | Replica set name used by members and clients | | `replicaSet.members` | Number of data-bearing members | +| `arbiter.enabled` | Add one non-data-bearing voting member | +| `arbiter.resources` | Resource settings for the arbiter container | | `auth.replicaSetKey` | Internal auth key when not using existing secret | | `auth.existingKeySecret` | Existing secret for key file | | `persistence.*` | Storage settings for the members | @@ -93,6 +97,43 @@ metrics: enabled: true ``` +## Arbiter topology + +MongoDB recommends three data-bearing members. When capacity constraints make a +third data copy impractical, the chart can deploy two data-bearing members and +one arbiter: + +```yaml +architecture: replicaset + +replicaSet: + members: 2 + +arbiter: + enabled: true +``` + +The arbiter uses the official MongoDB image, the same internal keyFile, stable +StatefulSet DNS, and ephemeral local storage. It votes in elections but does not +store the application data set and cannot become primary. The chart permits one +arbiter with an even number of 2 to 6 data-bearing members. Review MongoDB's +[arbiter limitations](https://www.mongodb.com/docs/manual/core/replica-set-arbiter/), +especially majority write behavior and reduced fault tolerance. + +The post-upgrade hook adds or removes the arbiter when `arbiter.enabled` +changes. It can add missing data-bearing members, but deliberately does not +remove them. Follow MongoDB's member-removal procedure before reducing +`replicaSet.members`. + +Adding or removing an arbiter can change MongoDB's implicit default write +concern. Before that reconfiguration, the hook preserves the currently effective +implicit value by promoting it to a global default. If the operator already set +a global default, the hook leaves it unchanged. This preservation also runs +before `rs.add()` when scale-up adds a missing data-bearing member. MongoDB 5.0 +and newer do not allow the global default write concern to be unset afterward. Review +[`setDefaultRWConcern`](https://www.mongodb.com/docs/manual/reference/command/setDefaultRWConcern/) +before changing `arbiter.enabled` on an existing deployment. + ## When to move to another architecture - move back to `standalone` only for non-critical simplified environments diff --git a/charts/mongodb/examples/replicaset-arbiter.yaml b/charts/mongodb/examples/replicaset-arbiter.yaml new file mode 100644 index 000000000..9af80b9ad --- /dev/null +++ b/charts/mongodb/examples/replicaset-arbiter.yaml @@ -0,0 +1,28 @@ +# SPDX-License-Identifier: Apache-2.0 +# Two data-bearing members plus one non-data-bearing voting arbiter. +# Prefer three data-bearing members when capacity permits. + +architecture: replicaset + +auth: + enabled: true + existingSecret: mongodb-root-credentials + existingKeySecret: mongodb-replica-key + +replicaSet: + name: rs0 + members: 2 + +arbiter: + enabled: true + resources: + requests: + cpu: 50m + memory: 128Mi + limits: + cpu: 250m + memory: 512Mi + +persistence: + enabled: true + size: 50Gi diff --git a/charts/mongodb/templates/NOTES.txt b/charts/mongodb/templates/NOTES.txt index e06e5eb9e..e1e15349c 100644 --- a/charts/mongodb/templates/NOTES.txt +++ b/charts/mongodb/templates/NOTES.txt @@ -12,6 +12,7 @@ Persistence: {{ .Values.persistence.enabled }} Metrics: {{ .Values.metrics.enabled }} Backup: {{ .Values.backup.enabled }} + Arbiter: {{ .Values.arbiter.enabled }} External Secrets: {{ .Values.externalSecrets.enabled }} {{- if include "mongodb.isStandalone" . }} @@ -25,7 +26,7 @@ 2. Connect ---------- - Replica Set: {{ .Values.replicaSet.name }} ({{ include "mongodb.replicaCount" . }} members) + Replica Set: {{ .Values.replicaSet.name }} ({{ include "mongodb.replicaCount" . }} data-bearing members{{ if .Values.arbiter.enabled }} + 1 arbiter{{ end }}) Connection string: mongodb://{{ if .Values.auth.enabled }}{{ .Values.auth.rootUser }}:$(kubectl get secret -n {{ $namespace }} {{ include "mongodb.secretName" . }} -o jsonpath='{.data.mongodb-root-password}' | base64 -d)@{{ end }}{{ range $i := until (int (include "mongodb.replicaCount" .)) }}{{ if $i }},{{ end }}{{ $fullname }}-{{ $i }}.{{ $fullname }}-headless.{{ $namespace }}.svc.cluster.local:{{ $.Values.port }}{{ end }}/admin?replicaSet={{ .Values.replicaSet.name }} @@ -71,6 +72,11 @@ Check logs: kubectl logs -n {{ $namespace }} -l app.kubernetes.io/instance={{ .Release.Name }} --tail=200 +{{- if and (include "mongodb.isReplicaSet" .) .Values.arbiter.enabled }} + + Verify arbiter membership: + kubectl exec -n {{ $namespace }} {{ $fullname }}-0 -- mongosh --quiet{{ if .Values.auth.enabled }} --username {{ .Values.auth.rootUser }} --password "$(kubectl get secret -n {{ $namespace }} {{ include "mongodb.secretName" . }} -o jsonpath='{.data.mongodb-root-password}' | base64 -d)" --authenticationDatabase admin{{ end }} --eval 'rs.conf().members.map(({host, arbiterOnly}) => ({host, arbiterOnly}))' +{{- end }} 5. Metrics ---------- diff --git a/charts/mongodb/templates/_helpers.tpl b/charts/mongodb/templates/_helpers.tpl index 3097b6397..e56a4fa3f 100644 --- a/charts/mongodb/templates/_helpers.tpl +++ b/charts/mongodb/templates/_helpers.tpl @@ -13,6 +13,12 @@ Validate cross-field settings that JSON Schema cannot express. {{- if and .Values.persistence.existingClaim (ne .Values.architecture "standalone") -}} {{- fail "persistence.existingClaim is supported only when architecture is standalone" -}} {{- end -}} +{{- if and .Values.arbiter.enabled (ne .Values.architecture "replicaset") -}} +{{- fail "arbiter.enabled is supported only when architecture is replicaset" -}} +{{- end -}} +{{- if and .Values.arbiter.enabled (or (lt (int .Values.replicaSet.members) 2) (gt (int .Values.replicaSet.members) 6) (ne (mod (int .Values.replicaSet.members) 2) 0)) -}} +{{- fail "arbiter.enabled requires an even replicaSet.members value between 2 and 6" -}} +{{- end -}} {{- end -}} {{/* @@ -78,6 +84,20 @@ Headless service name (for StatefulSet DNS). {{- printf "%s-headless" (include "mongodb.fullname" .) -}} {{- end -}} +{{/* +Arbiter StatefulSet name. +*/}} +{{- define "mongodb.arbiterName" -}} +{{- printf "%s-arbiter" (include "mongodb.fullname" . | trunc 46 | trimSuffix "-") -}} +{{- end -}} + +{{/* +Arbiter headless service name. +*/}} +{{- define "mongodb.arbiterHeadlessServiceName" -}} +{{- printf "%s-headless" (include "mongodb.arbiterName" .) -}} +{{- end -}} + {{/* Auth secret name. */}} diff --git a/charts/mongodb/templates/arbiter-service.yaml b/charts/mongodb/templates/arbiter-service.yaml new file mode 100644 index 000000000..825f6378f --- /dev/null +++ b/charts/mongodb/templates/arbiter-service.yaml @@ -0,0 +1,21 @@ +{{/* SPDX-License-Identifier: Apache-2.0 */}} +{{- if and (include "mongodb.isReplicaSet" .) .Values.arbiter.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "mongodb.arbiterHeadlessServiceName" . }} + labels: + {{- include "mongodb.labels" . | nindent 4 }} + app.kubernetes.io/component: arbiter +spec: + type: ClusterIP + clusterIP: None + publishNotReadyAddresses: true + ports: + - name: mongodb + port: {{ .Values.service.port }} + targetPort: mongodb + selector: + {{- include "mongodb.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: arbiter +{{- end }} diff --git a/charts/mongodb/templates/arbiter-statefulset.yaml b/charts/mongodb/templates/arbiter-statefulset.yaml new file mode 100644 index 000000000..bc5b464de --- /dev/null +++ b/charts/mongodb/templates/arbiter-statefulset.yaml @@ -0,0 +1,143 @@ +{{/* SPDX-License-Identifier: Apache-2.0 */}} +{{- if and (include "mongodb.isReplicaSet" .) .Values.arbiter.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ include "mongodb.arbiterName" . }} + labels: + {{- include "mongodb.labels" . | nindent 4 }} + app.kubernetes.io/component: arbiter + {{- with .Values.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + serviceName: {{ include "mongodb.arbiterHeadlessServiceName" . }} + replicas: 1 + podManagementPolicy: OrderedReady + updateStrategy: + type: RollingUpdate + selector: + matchLabels: + {{- include "mongodb.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: arbiter + template: + metadata: + labels: + {{- include "mongodb.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: arbiter + {{- with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.podAnnotations }} + annotations: + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ include "mongodb.serviceAccountName" . }} + terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }} + {{- with .Values.podSecurityContext }} + securityContext: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.topologySpreadConstraints }} + topologySpreadConstraints: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.priorityClassName }} + priorityClassName: {{ . }} + {{- end }} + {{- if include "mongodb.needsKeyFile" . }} + initContainers: + - name: init-keyfile + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: + - bash + - -c + - | + cp /etc/mongodb/keyfile-readonly/replica-set-key /etc/mongodb/keyfile/replica-set-key + chmod 400 /etc/mongodb/keyfile/replica-set-key + chown 999:999 /etc/mongodb/keyfile/replica-set-key + securityContext: + runAsUser: 0 + runAsNonRoot: false + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + add: + - CHOWN + volumeMounts: + - name: keyfile-readonly + mountPath: /etc/mongodb/keyfile-readonly + readOnly: true + - name: keyfile + mountPath: /etc/mongodb/keyfile + {{- end }} + containers: + - name: mongod + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + {{- include "mongodb.mongodArgs" . | nindent 12 }} + ports: + - name: mongodb + containerPort: {{ .Values.port }} + protocol: TCP + {{- with .Values.livenessProbe }} + livenessProbe: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.readinessProbe }} + readinessProbe: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.startupProbe }} + startupProbe: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.arbiter.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.securityContext }} + securityContext: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: data + mountPath: /data/db + {{- if include "mongodb.needsKeyFile" . }} + - name: keyfile + mountPath: /etc/mongodb/keyfile + readOnly: true + {{- end }} + volumes: + - name: data + emptyDir: {} + {{- if include "mongodb.needsKeyFile" . }} + - name: keyfile-readonly + secret: + secretName: {{ include "mongodb.keySecretName" . }} + defaultMode: 0400 + - name: keyfile + emptyDir: {} + {{- end }} +{{- end }} diff --git a/charts/mongodb/templates/job-rs-init.yaml b/charts/mongodb/templates/job-rs-init.yaml index 2e151a645..abd439f3d 100644 --- a/charts/mongodb/templates/job-rs-init.yaml +++ b/charts/mongodb/templates/job-rs-init.yaml @@ -8,7 +8,7 @@ metadata: {{- include "mongodb.labels" . | nindent 4 }} app.kubernetes.io/component: rs-init annotations: - "helm.sh/hook": post-install + "helm.sh/hook": post-install,post-upgrade "helm.sh/hook-weight": "5" "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation spec: @@ -50,9 +50,12 @@ spec: FULLNAME="{{ include "mongodb.fullname" . }}" HEADLESS="{{ include "mongodb.headlessServiceName" . }}" + ARBITER_NAME="{{ include "mongodb.arbiterName" . }}" + ARBITER_HEADLESS="{{ include "mongodb.arbiterHeadlessServiceName" . }}" NAMESPACE="{{ .Release.Namespace }}" RS_NAME="{{ .Values.replicaSet.name }}" MEMBERS={{ include "mongodb.replicaCount" . }} + ARBITER_ENABLED="{{ .Values.arbiter.enabled }}" PORT={{ .Values.port }} {{- if .Values.auth.enabled }} @@ -61,34 +64,90 @@ spec: AUTH_ARGS="" {{- end }} - PRIMARY="${FULLNAME}-0.${HEADLESS}.${NAMESPACE}.svc.cluster.local:${PORT}" + SEED="${FULLNAME}-0.${HEADLESS}.${NAMESPACE}.svc.cluster.local:${PORT}" + ARBITER="${ARBITER_NAME}-0.${ARBITER_HEADLESS}.${NAMESPACE}.svc.cluster.local:${PORT}" - echo "Waiting for primary to be ready..." - until mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval "db.adminCommand('ping')" 2>/dev/null; do + echo "Waiting for the seed member to be ready..." + until mongosh --host "$SEED" $AUTH_ARGS --quiet --eval "db.adminCommand('ping')" 2>/dev/null; do echo " Waiting for ${FULLNAME}-0..." sleep 5 done echo "Checking if replica set is already initialized..." - RS_STATUS=$(mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " + RS_STATUS=$(mongosh --host "$SEED" $AUTH_ARGS --quiet --eval " try { rs.status().ok } catch(e) { 0 } " 2>/dev/null || echo "0") if [ "$RS_STATUS" = "1" ]; then - echo "Replica set already initialized. Checking members..." + echo "Replica set already initialized. Reconciling members..." - CURRENT_MEMBERS=$(mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " - rs.status().members.length - " 2>/dev/null) + PRIMARY="" + for attempt in $(seq 1 60); do + PRIMARY=$(mongosh --host "$SEED" $AUTH_ARGS --quiet --eval "db.hello().primary || ''" 2>/dev/null || true) + if [ -n "$PRIMARY" ]; then + break + fi + sleep 2 + done + if [ -z "$PRIMARY" ]; then + echo "ERROR: Replica set has no elected primary for reconciliation." >&2 + exit 1 + fi - echo "Current members: $CURRENT_MEMBERS, expected: $MEMBERS" + preserve_implicit_write_concern() { + mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " + const current = db.adminCommand({ getDefaultRWConcern: 1 }); + if (current.defaultWriteConcernSource === 'implicit') { + const votingMembers = rs.conf().members.filter((member) => member.votes !== 0); + const arbiters = votingMembers.filter((member) => member.arbiterOnly === true).length; + const dataBearing = votingMembers.length - arbiters; + const votingMajority = Math.floor(votingMembers.length / 2) + 1; + const writeConcern = arbiters > 0 && dataBearing <= votingMajority ? 1 : 'majority'; + const result = db.adminCommand({ + setDefaultRWConcern: 1, + defaultWriteConcern: { w: writeConcern }, + writeConcern: { w: 'majority' } + }); + if (result.ok !== 1) { + throw new Error('failed to preserve the implicit default write concern'); + } + print('Promoted implicit default write concern to global: ' + writeConcern); + } else { + print('Keeping operator-managed global default write concern.'); + } + " + } - if [ "$CURRENT_MEMBERS" -lt "$MEMBERS" ]; then - for i in $(seq "$CURRENT_MEMBERS" $((MEMBERS - 1))); do - MEMBER="${FULLNAME}-${i}.${HEADLESS}.${NAMESPACE}.svc.cluster.local:${PORT}" - echo "Adding member: $MEMBER" + for i in $(seq 0 $((MEMBERS - 1))); do + MEMBER="${FULLNAME}-${i}.${HEADLESS}.${NAMESPACE}.svc.cluster.local:${PORT}" + MEMBER_EXISTS=$(mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " + rs.conf().members.some((member) => member.host === '${MEMBER}') + " 2>/dev/null) + if [ "$MEMBER_EXISTS" != "true" ]; then + echo "Adding data-bearing member: $MEMBER" + preserve_implicit_write_concern mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval "rs.add('${MEMBER}')" + fi + done + + ARBITER_EXISTS=$(mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " + rs.conf().members.some((member) => member.host === '${ARBITER}' && member.arbiterOnly === true) + " 2>/dev/null) + + if [ "$ARBITER_ENABLED" = "true" ]; then + until mongosh --host "$ARBITER" --quiet --eval "db.adminCommand('ping')" 2>/dev/null; do + echo " Waiting for arbiter..." + sleep 5 done + if [ "$ARBITER_EXISTS" != "true" ]; then + echo "Adding arbiter: $ARBITER" + preserve_implicit_write_concern + mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval "rs.addArb('${ARBITER}')" + fi + elif [ "$ARBITER_EXISTS" = "true" ]; then + echo "Removing disabled arbiter: $ARBITER" + preserve_implicit_write_concern + mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval "rs.remove('${ARBITER}')" fi echo "Replica set is configured." @@ -105,7 +164,15 @@ spec: MEMBERS_JSON="${MEMBERS_JSON}{_id:${i},host:'${MEMBER}'}" done - mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " + if [ "$ARBITER_ENABLED" = "true" ]; then + until mongosh --host "$ARBITER" --quiet --eval "db.adminCommand('ping')" 2>/dev/null; do + echo " Waiting for arbiter..." + sleep 5 + done + MEMBERS_JSON="${MEMBERS_JSON},{_id:${MEMBERS},host:'${ARBITER}',arbiterOnly:true}" + fi + + mongosh --host "$SEED" $AUTH_ARGS --quiet --eval " rs.initiate({ _id: '${RS_NAME}', members: [${MEMBERS_JSON}] @@ -114,8 +181,8 @@ spec: echo "Waiting for replica set to elect primary..." for attempt in $(seq 1 60); do - IS_PRIMARY=$(mongosh --host "$PRIMARY" $AUTH_ARGS --quiet --eval " - rs.isMaster().ismaster + IS_PRIMARY=$(mongosh --host "$SEED" $AUTH_ARGS --quiet --eval " + rs.status().members.some((member) => member.stateStr === 'PRIMARY') " 2>/dev/null || echo "false") if [ "$IS_PRIMARY" = "true" ]; then echo "Primary elected. Replica set ready." @@ -124,6 +191,6 @@ spec: sleep 2 done - echo "WARNING: Primary election timed out but rs.initiate succeeded." - exit 0 + echo "ERROR: Primary election timed out after replica set initialization." >&2 + exit 1 {{- end }} diff --git a/charts/mongodb/tests/arbiter_test.yaml b/charts/mongodb/tests/arbiter_test.yaml new file mode 100644 index 000000000..735f66378 --- /dev/null +++ b/charts/mongodb/tests/arbiter_test.yaml @@ -0,0 +1,158 @@ +# SPDX-License-Identifier: Apache-2.0 +suite: Arbiter +templates: + - templates/arbiter-service.yaml + - templates/arbiter-statefulset.yaml + - templates/job-rs-init.yaml +release: + name: test + namespace: default +tests: + - it: does not render arbiter resources by default + asserts: + - hasDocuments: + count: 0 + template: templates/arbiter-service.yaml + - hasDocuments: + count: 0 + template: templates/arbiter-statefulset.yaml + + - it: renders a dedicated arbiter headless service + template: templates/arbiter-service.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + asserts: + - isKind: + of: Service + - equal: + path: metadata.name + value: test-mongodb-arbiter-headless + - equal: + path: spec.clusterIP + value: None + - equal: + path: spec.selector["app.kubernetes.io/component"] + value: arbiter + + - it: renders one lightweight arbiter with stable DNS and ephemeral data + template: templates/arbiter-statefulset.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + asserts: + - isKind: + of: StatefulSet + - equal: + path: metadata.name + value: test-mongodb-arbiter + - equal: + path: spec.serviceName + value: test-mongodb-arbiter-headless + - equal: + path: spec.replicas + value: 1 + - equal: + path: spec.template.metadata.labels["app.kubernetes.io/component"] + value: arbiter + - equal: + path: spec.template.spec.containers[0].image + value: docker.io/library/mongo:8.3.8 + - contains: + path: spec.template.spec.containers[0].args + content: --replSet + - contains: + path: spec.template.spec.volumes + content: + name: data + emptyDir: {} + + - it: registers and reconciles the arbiter in the replica set + template: templates/job-rs-init.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + asserts: + - matchRegex: + path: metadata.annotations["helm.sh/hook"] + pattern: post-install,post-upgrade + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: ARBITER_ENABLED="true" + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: rs\.addArb + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: arbiterOnly:true + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: rs\.remove + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: member\.stateStr === 'PRIMARY' + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: db\.hello\(\)\.primary + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: getDefaultRWConcern + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: setDefaultRWConcern + + - it: preserves write concern before scaling a PSA topology from two to four data members + template: templates/job-rs-init.yaml + set: + architecture: replicaset + replicaSet.members: 4 + arbiter.enabled: true + asserts: + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: 'MEMBERS=4' + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: 'echo "Adding data-bearing member: \$MEMBER"\s+preserve_implicit_write_concern' + + - it: bounds the arbiter StatefulSet name for a maximum-length fullname override + template: templates/arbiter-statefulset.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + fullnameOverride: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + asserts: + - matchRegex: + path: metadata.name + pattern: '^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$' + + - it: bounds the arbiter Service name for a maximum-length fullname override + template: templates/arbiter-service.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + fullnameOverride: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + asserts: + - matchRegex: + path: metadata.name + pattern: '^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$' + + - it: reuses the bounded arbiter name in the reconciliation hook + template: templates/job-rs-init.yaml + set: + architecture: replicaset + replicaSet.members: 2 + arbiter.enabled: true + fullnameOverride: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + asserts: + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: 'ARBITER_NAME="[a-z0-9-]{1,63}"' + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: 'ARBITER="\$\{ARBITER_NAME\}-0\.' diff --git a/charts/mongodb/tests/validate_test.yaml b/charts/mongodb/tests/validate_test.yaml index 8145f6fa4..badd11349 100644 --- a/charts/mongodb/tests/validate_test.yaml +++ b/charts/mongodb/tests/validate_test.yaml @@ -10,3 +10,38 @@ tests: asserts: - failedTemplate: errorMessage: persistence.existingClaim is supported only when architecture is standalone + + - it: rejects arbiter outside replicaset architecture + set: + architecture: standalone + arbiter.enabled: true + asserts: + - failedTemplate: + errorMessage: arbiter.enabled is supported only when architecture is replicaset + + - it: rejects arbiter with fewer than two data-bearing members + set: + architecture: replicaset + replicaSet.members: 1 + arbiter.enabled: true + asserts: + - failedTemplate: + errorMessage: arbiter.enabled requires an even replicaSet.members value between 2 and 6 + + - it: rejects arbiter with an odd number of data-bearing members + set: + architecture: replicaset + replicaSet.members: 3 + arbiter.enabled: true + asserts: + - failedTemplate: + errorMessage: arbiter.enabled requires an even replicaSet.members value between 2 and 6 + + - it: rejects arbiter when the replica set would exceed seven voting members + set: + architecture: replicaset + replicaSet.members: 8 + arbiter.enabled: true + asserts: + - failedTemplate: + errorMessage: arbiter.enabled requires an even replicaSet.members value between 2 and 6 diff --git a/charts/mongodb/values.schema.json b/charts/mongodb/values.schema.json index 567cf11da..fdef8cc59 100644 --- a/charts/mongodb/values.schema.json +++ b/charts/mongodb/values.schema.json @@ -145,6 +145,22 @@ } } }, + "arbiter": { + "type": "object", + "description": "Optional non-data-bearing voting member for replica set deployments", + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean", + "description": "Add one arbiter when architecture=replicaset", + "default": false + }, + "resources": { + "type": "object", + "description": "Resources for the arbiter mongod container" + } + } + }, "sharded": { "type": "object", "description": "Sharded cluster configuration (architecture=sharded)", diff --git a/charts/mongodb/values.yaml b/charts/mongodb/values.yaml index 71e083dfb..daaaa3545 100644 --- a/charts/mongodb/values.yaml +++ b/charts/mongodb/values.yaml @@ -65,6 +65,16 @@ replicaSet: # -- Number of data-bearing members (architecture=replicaset) members: 3 +# ============================================================================= +# Arbiter +# ============================================================================= + +arbiter: + # -- Add one non-data-bearing voting member (architecture=replicaset) + enabled: false + # -- Resources for the arbiter mongod container + resources: {} + # ============================================================================= # Sharded Cluster (architecture=sharded) # =============================================================================