2026/04/08 01:37:38 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2026-04-08 01:37:38.688473 I | rookcmd: starting Rook v1.15.9 with arguments '/usr/local/bin/rook ceph operator'
2026-04-08 01:37:38.688494 I | rookcmd: flag values: --enable-machine-disruption-budget=false, --help=false, --kubeconfig=, --log-level=INFO
2026-04-08 01:37:38.688499 I | cephcmd: starting Rook-Ceph operator
2026-04-08 01:37:38.786255 I | cephcmd: base ceph version inside the rook operator image is "ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable)"
2026-04-08 01:37:38.801333 I | op-k8sutil: operator setting "CSI_GRPC_TIMEOUT_SECONDS" = "150"
2026-04-08 01:37:38.801365 I | op-k8sutil: operator setting "CSI_RBD_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n    limits:\n      memory: 1Gi\n- name : csi-omap-generator\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-08 01:37:38.801373 I | op-k8sutil: operator setting "ROOK_CSI_CEPH_IMAGE" = "quay.io/cephcsi/cephcsi:v3.12.3"
2026-04-08 01:37:38.801379 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_CEPHFS" = "false"
2026-04-08 01:37:38.801383 I | op-k8sutil: operator setting "ROOK_CSI_SNAPSHOTTER_IMAGE" = "registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1"
2026-04-08 01:37:38.801386 I | op-k8sutil: operator setting "CSI_ENABLE_CSIADDONS" = "false"
2026-04-08 01:37:38.801390 I | op-k8sutil: operator setting "CSI_ENABLE_HOST_NETWORK" = "true"
2026-04-08 01:37:38.801393 I | op-k8sutil: operator setting "CSI_FORCE_CEPHFS_KERNEL_CLIENT" = "true"
2026-04-08 01:37:38.801397 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_RBD" = "false"
2026-04-08 01:37:38.801401 I | op-k8sutil: operator setting "ROOK_CSI_PROVISIONER_IMAGE" = "registry.k8s.io/sig-storage/csi-provisioner:v5.0.1"
2026-04-08 01:37:38.801404 I | op-k8sutil: operator setting "CSI_ENABLE_VOLUME_GROUP_SNAPSHOT" = "true"
2026-04-08 01:37:38.801412 I | op-k8sutil: operator setting "CSI_NFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-04-08 01:37:38.801416 I | op-k8sutil: operator setting "CSI_RBD_ATTACH_REQUIRED" = "true"
2026-04-08 01:37:38.801419 I | op-k8sutil: operator setting "CSI_NFS_ATTACH_REQUIRED" = "true"
2026-04-08 01:37:38.801424 I | op-k8sutil: operator setting "ROOK_CSI_ENABLE_NFS" = "false"
2026-04-08 01:37:38.801429 I | op-k8sutil: operator setting "ROOK_CSI_REGISTRAR_IMAGE" = "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.11.1"
2026-04-08 01:37:38.801432 I | op-k8sutil: operator setting "ROOK_LOG_LEVEL" = "INFO"
2026-04-08 01:37:38.801436 I | op-k8sutil: operator setting "ROOK_OBC_WATCH_OPERATOR_NAMESPACE" = "true"
2026-04-08 01:37:38.801446 I | op-k8sutil: operator setting "CSI_CEPHFS_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-08 01:37:38.801451 I | op-k8sutil: operator setting "CSI_ENABLE_NFS_SNAPSHOTTER" = "true"
2026-04-08 01:37:38.801454 I | op-k8sutil: operator setting "CSI_ENABLE_OMAP_GENERATOR" = "false"
2026-04-08 01:37:38.801458 I | op-k8sutil: operator setting "ROOK_CEPH_ALLOW_LOOP_DEVICES" = "false"
2026-04-08 01:37:38.801462 I | op-k8sutil: operator setting "ROOK_CSIADDONS_IMAGE" = "quay.io/csiaddons/k8s-sidecar:v0.9.1"
2026-04-08 01:37:38.801465 I | op-k8sutil: operator setting "ROOK_CSI_DISABLE_DRIVER" = "false"
2026-04-08 01:37:38.801477 I | op-k8sutil: operator setting "CSI_CEPHFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-resizer\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-snapshotter\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-cephfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-08 01:37:38.801481 I | op-k8sutil: operator setting "CSI_PLUGIN_ENABLE_SELINUX_HOST_MOUNT" = "false"
2026-04-08 01:37:38.801485 I | op-k8sutil: operator setting "CSI_RBD_FSGROUPPOLICY" = "File"
2026-04-08 01:37:38.801488 I | op-k8sutil: operator setting "CSI_CEPHFS_ATTACH_REQUIRED" = "true"
2026-04-08 01:37:38.801495 I | op-k8sutil: operator setting "CSI_ENABLE_RBD_SNAPSHOTTER" = "true"
2026-04-08 01:37:38.801505 I | op-k8sutil: operator setting "CSI_PROVISIONER_PRIORITY_CLASSNAME" = "system-cluster-critical"
2026-04-08 01:37:38.801510 I | op-k8sutil: operator setting "CSI_ENABLE_METADATA" = "false"
2026-04-08 01:37:38.801514 I | op-k8sutil: operator setting "CSI_PROVISIONER_REPLICAS" = "2"
2026-04-08 01:37:38.801522 I | op-k8sutil: operator setting "CSI_RBD_PLUGIN_RESOURCE" = "- name : driver-registrar\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n- name : csi-rbdplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : liveness-prometheus\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 50m\n    limits:\n      memory: 256Mi\n"
2026-04-08 01:37:38.801527 I | op-k8sutil: operator setting "ROOK_CSI_ATTACHER_IMAGE" = "registry.k8s.io/sig-storage/csi-attacher:v4.6.1"
2026-04-08 01:37:38.801531 I | op-k8sutil: operator setting "ROOK_CSI_IMAGE_PULL_POLICY" = "IfNotPresent"
2026-04-08 01:37:38.801540 I | op-k8sutil: operator setting "CSI_DISABLE_HOLDER_PODS" = "true"
2026-04-08 01:37:38.801544 I | op-k8sutil: operator setting "CSI_ENABLE_CEPHFS_SNAPSHOTTER" = "true"
2026-04-08 01:37:38.801548 I | op-k8sutil: operator setting "CSI_ENABLE_ENCRYPTION" = "false"
2026-04-08 01:37:38.801552 I | op-k8sutil: operator setting "CSI_NFS_FSGROUPPOLICY" = "File"
2026-04-08 01:37:38.801566 I | op-k8sutil: operator setting "CSI_NFS_PROVISIONER_RESOURCE" = "- name : csi-provisioner\n  resource:\n    requests:\n      memory: 128Mi\n      cpu: 100m\n    limits:\n      memory: 256Mi\n- name : csi-nfsplugin\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n- name : csi-attacher\n  resource:\n    requests:\n      memory: 512Mi\n      cpu: 250m\n    limits:\n      memory: 1Gi\n"
2026-04-08 01:37:38.801573 I | op-k8sutil: operator setting "ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS" = "15"
2026-04-08 01:37:38.801578 I | op-k8sutil: operator setting "ROOK_CSI_RESIZER_IMAGE" = "registry.k8s.io/sig-storage/csi-resizer:v1.11.1"
2026-04-08 01:37:38.801581 I | op-k8sutil: operator setting "ROOK_ENABLE_DISCOVERY_DAEMON" = "false"
2026-04-08 01:37:38.801585 I | op-k8sutil: operator setting "CSI_CEPHFS_FSGROUPPOLICY" = "File"
2026-04-08 01:37:38.801589 I | op-k8sutil: operator setting "CSI_ENABLE_TOPOLOGY" = "false"
2026-04-08 01:37:38.801593 I | op-k8sutil: operator setting "CSI_PLUGIN_PRIORITY_CLASSNAME" = "system-node-critical"
2026-04-08 01:37:38.801598 I | operator: watching all namespaces for Ceph CRs
2026-04-08 01:37:38.801675 I | operator: setting up schemes
2026-04-08 01:37:38.803836 I | operator: setting up the controller-runtime manager
2026-04-08 01:37:38.804161 I | ceph-cluster-controller: successfully started
2026-04-08 01:37:38.804433 I | ceph-cluster-controller: enabling hotplug orchestration
2026-04-08 01:37:38.804453 I | ceph-nodedaemon-controller: successfully started
2026-04-08 01:37:38.804464 I | ceph-block-pool-controller: successfully started
2026-04-08 01:37:38.804475 I | ceph-object-store-user-controller: successfully started
2026-04-08 01:37:38.804489 I | ceph-object-realm-controller: successfully started
2026-04-08 01:37:38.804502 I | ceph-object-zonegroup-controller: successfully started
2026-04-08 01:37:38.804509 I | ceph-object-zone-controller: successfully started
2026-04-08 01:37:38.804581 I | ceph-object-controller: successfully started
2026-04-08 01:37:38.804611 I | ceph-file-controller: successfully started
2026-04-08 01:37:38.804630 I | ceph-nfs-controller: successfully started
2026-04-08 01:37:38.804655 I | ceph-rbd-mirror-controller: successfully started
2026-04-08 01:37:38.804678 I | ceph-client-controller: successfully started
2026-04-08 01:37:38.804694 I | ceph-filesystem-mirror-controller: successfully started
2026-04-08 01:37:38.804715 I | operator: rook-ceph-operator-config-controller successfully started
2026-04-08 01:37:38.804725 I | ceph-csi: rook-ceph-operator-csi-controller successfully started
2026-04-08 01:37:38.804830 I | op-bucket-prov: rook-ceph-operator-bucket-controller successfully started
2026-04-08 01:37:38.804847 I | ceph-bucket-topic: successfully started
2026-04-08 01:37:38.804855 I | ceph-bucket-notification: successfully started
2026-04-08 01:37:38.804864 I | ceph-bucket-notification: successfully started
2026-04-08 01:37:38.804870 I | ceph-fs-subvolumegroup-controller: successfully started
2026-04-08 01:37:38.804913 I | blockpool-rados-namespace-controller: successfully started
2026-04-08 01:37:38.804959 I | ceph-cosi-controller: successfully started
2026-04-08 01:37:38.804974 I | operator: starting the controller-runtime manager
2026-04-08 01:37:39.253903 I | operator: rook-ceph-operator-config-controller done reconciling
2026-04-08 01:37:39.261534 I | ceph-csi: successfully created csi config map "rook-ceph-csi-config"
2026-04-08 01:37:39.263857 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-04-08 01:37:39.265981 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-04-08 01:37:39.271844 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-04-08 01:37:39.271870 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-04-08 01:37:39.274059 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-04-08 01:37:39.279870 I | ceph-csi: successfully removed CSI CephFS driver
2026-04-08 01:37:39.279894 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-04-08 01:37:39.281770 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-04-08 01:37:39.287439 I | ceph-csi: successfully removed CSI NFS driver
2026-04-08 01:38:01.128022 I | ceph-spec: adding finalizer "cephcluster.ceph.rook.io" on "ceph"
2026-04-08 01:38:01.132186 I | clusterdisruption-controller: deleted all legacy node drain canary pods
2026-04-08 01:38:01.134534 I | ceph-spec: adding finalizer "cephobjectstore.ceph.rook.io" on "ceph"
2026-04-08 01:38:01.141523 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-08 01:38:01.141565 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-08 01:38:01.141725 I | op-bucket-prov: ceph bucket provisioner launched watching for provisioner "openstack.ceph.rook.io/bucket"
2026-04-08 01:38:01.142248 I | op-bucket-prov: successfully reconciled bucket provisioner
I0408 01:38:01.142376       1 manager.go:135] "msg"="starting provisioner" "logger"="objectbucket.io/provisioner-manager" "name"="openstack.ceph.rook.io/bucket"
2026-04-08 01:38:01.151467 I | ceph-cluster-controller: reconciling ceph cluster in namespace "openstack"
2026-04-08 01:38:01.175823 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-08 01:38:01.175848 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-08 01:38:01.175865 I | ceph-spec: found the cluster info to connect to the external cluster. will use "client.admin" to check health and monitor status. mons=map[instance:0xc001d70f00]
2026-04-08 01:38:01.175879 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-04-08 01:38:02.130219 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-08 01:38:02.130252 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-08 01:38:02.539153 I | op-k8sutil: removing daemonset csi-rbdplugin if it exists
2026-04-08 01:38:02.541898 I | op-k8sutil: removing deployment csi-rbdplugin-provisioner if it exists
2026-04-08 01:38:02.733875 I | ceph-csi: successfully removed CSI Ceph RBD driver
2026-04-08 01:38:02.733910 I | op-k8sutil: removing daemonset csi-cephfsplugin if it exists
2026-04-08 01:38:02.736480 I | op-k8sutil: removing deployment csi-cephfsplugin-provisioner if it exists
2026-04-08 01:38:02.933827 I | ceph-csi: successfully removed CSI CephFS driver
2026-04-08 01:38:02.933875 I | op-k8sutil: removing daemonset csi-nfsplugin if it exists
2026-04-08 01:38:02.937132 I | op-k8sutil: removing deployment csi-nfsplugin-provisioner if it exists
2026-04-08 01:38:03.133740 I | ceph-csi: successfully removed CSI NFS driver
2026-04-08 01:38:18.407545 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-04-08 01:38:18.407568 I | ceph-cluster-controller: validating ceph version from provided image
2026-04-08 01:38:18.414432 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-08 01:38:18.414561 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-08 01:38:18.417480 I | cephclient: writing config file /var/lib/rook/openstack/openstack.config
2026-04-08 01:38:18.417565 I | cephclient: generated admin config in /var/lib/rook/openstack
2026-04-08 01:38:18.899625 E | cephver: external cluster ceph version is a major version higher "18.2.7-0 reef" than the local cluster "0.0.0-0 <unknown version>", consider upgrading
2026-04-08 01:38:19.374457 I | ceph-cluster-controller: cluster "openstack": version "18.2.7-0 reef" detected for image "harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7"
2026-04-08 01:38:19.912030 I | ceph-cluster-controller: creating "rook-config-override" configmap
2026-04-08 01:38:19.919530 I | ceph-cluster-controller: creating "rook-ceph-config" secret
2026-04-08 01:38:19.941647 I | ceph-cluster-controller: external cluster identity established
2026-04-08 01:38:19.941670 I | cephclient: getting or creating ceph auth key "client.csi-rbd-provisioner"
2026-04-08 01:38:20.860769 I | cephclient: getting or creating ceph auth key "client.csi-rbd-node"
2026-04-08 01:38:21.551624 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-provisioner"
2026-04-08 01:38:22.206449 I | clusterdisruption-controller: all PGs are active+clean. Restoring default OSD pdb settings
2026-04-08 01:38:22.206575 I | clusterdisruption-controller: creating the default pdb "rook-ceph-osd" with maxUnavailable=1 for all osd
2026-04-08 01:38:22.214931 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:38:22.531773 I | cephclient: getting or creating ceph auth key "client.csi-cephfs-node"
2026-04-08 01:38:23.208065 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:38:23.600734 I | ceph-csi: created kubernetes csi secrets for cluster "openstack"
2026-04-08 01:38:23.612291 I | ceph-cluster-controller: successfully updated csi config map
2026-04-08 01:38:23.612334 I | cephclient: getting or creating ceph auth key "client.crash"
2026-04-08 01:38:24.198231 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:38:24.459564 I | ceph-nodedaemon-controller: created kubernetes crash collector secret for cluster "openstack"
2026-04-08 01:38:24.462382 I | ceph-cluster-controller: enabling ceph mon monitoring goroutine for cluster "openstack"
2026-04-08 01:38:24.462487 I | ceph-cluster-controller: ceph status check interval is 1m0s
2026-04-08 01:38:24.462531 I | ceph-cluster-controller: enabling ceph status monitoring goroutine for cluster "openstack"
2026-04-08 01:38:31.245477 I | ceph-spec: parsing mon endpoints: instance=10.96.240.200:6789
2026-04-08 01:38:31.245502 I | ceph-spec: updating obsolete maxMonID 0 to actual value 76846025234
2026-04-08 01:38:31.245515 I | ceph-spec: detecting the ceph image version for image harbor.atmosphere.dev/quay.io/ceph/ceph:v18.2.7...
2026-04-08 01:38:33.602802 I | ceph-spec: detected ceph image version: "18.2.7-0 reef"
2026-04-08 01:38:34.839667 I | ceph-object-controller: reconciling object store deployments
2026-04-08 01:38:34.859106 I | ceph-object-controller: ceph object store gateway service running at 10.100.21.217
2026-04-08 01:38:34.859197 I | ceph-object-controller: reconciling object store pools
2026-04-08 01:38:37.363520 I | cephclient: reconciling replicated pool ceph.rgw.control succeeded
2026-04-08 01:38:37.363548 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.control"
2026-04-08 01:38:42.113993 I | cephclient: reconciling replicated pool ceph.rgw.meta succeeded
2026-04-08 01:38:42.114031 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.meta"
2026-04-08 01:38:46.499981 I | cephclient: reconciling replicated pool ceph.rgw.log succeeded
2026-04-08 01:38:46.500010 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.log"
2026-04-08 01:38:50.563448 I | cephclient: reconciling replicated pool ceph.rgw.buckets.index succeeded
2026-04-08 01:38:50.563474 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.index"
2026-04-08 01:38:52.716467 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:25} {StateName:unknown Count:8}]"
2026-04-08 01:38:52.721354 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:38:53.902549 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:27} {StateName:unknown Count:6}]"
2026-04-08 01:38:53.907152 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:38:55.643238 I | cephclient: reconciling replicated pool ceph.rgw.buckets.non-ec succeeded
2026-04-08 01:38:55.643263 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.buckets.non-ec"
2026-04-08 01:38:59.704937 I | cephclient: reconciling replicated pool ceph.rgw.otp succeeded
2026-04-08 01:38:59.704974 I | cephclient: setting pool property "pg_num_min" to "8" on pool "ceph.rgw.otp"
2026-04-08 01:39:03.777863 I | cephclient: reconciling replicated pool .rgw.root succeeded
2026-04-08 01:39:03.777909 I | cephclient: setting pool property "pg_num_min" to "8" on pool ".rgw.root"
2026-04-08 01:39:07.854863 I | cephclient: reconciling replicated pool ceph.rgw.buckets.data succeeded
2026-04-08 01:39:07.854909 I | ceph-object-controller: configuring object store "ceph"
2026-04-08 01:39:08.436014 I | ceph-object-controller: Object store "ceph": realm=ceph, zonegroup=ceph, zone=ceph
2026-04-08 01:39:08.598614 I | ceph-object-controller: committing changes to RGW configuration period for CephObjectStore "openstack/ceph"
2026-04-08 01:39:08.959919 I | ceph-object-controller: configuration for object-store ceph is complete
2026-04-08 01:39:08.963084 I | ceph-object-controller: creating object store "ceph" in namespace "openstack"
2026-04-08 01:39:08.966102 I | cephclient: getting or creating ceph auth key "client.rgw.ceph.a"
2026-04-08 01:39:09.443806 I | ceph-object-controller: setting rgw config flags
2026-04-08 01:39:09.443860 I | ceph-object-controller: Configuring authentication with keystone
2026-04-08 01:39:09.443888 I | op-config: setting option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:10.189537 I | op-config: successfully set option "rgw_enable_usage_log" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:10.189571 I | op-config: setting option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:10.997401 I | op-config: successfully set option "rgw_zone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:10.997430 I | op-config: setting option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:11.404144 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:39:11.631995 I | op-config: successfully set option "rgw_keystone_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:11.632031 I | op-config: setting option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.004358 I | op-config: successfully set option "rgw_keystone_accepted_roles" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.004398 I | op-config: setting option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.380385 I | op-config: successfully set option "rgw_keystone_implicit_tenants" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.380423 I | op-config: setting option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.776242 I | op-config: successfully set option "rgw_keystone_admin_project" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:12.776279 I | op-config: setting option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.157240 I | op-config: successfully set option "rgw_run_sync_thread" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.157292 I | op-config: setting option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.535907 I | op-config: successfully set option "rgw_log_nonexistent_bucket" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.535937 I | op-config: setting option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.966917 I | op-config: successfully set option "rgw_keystone_api_version" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:13.966947 I | op-config: setting option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:14.349704 I | op-config: successfully set option "rgw_keystone_admin_domain" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:14.349747 I | op-config: setting option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:14.772825 I | op-config: successfully set option "rgw_keystone_admin_password" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:14.772863 I | op-config: setting option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.146976 I | op-config: successfully set option "rgw_s3_auth_use_keystone" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.147020 I | op-config: setting option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.533143 I | op-config: successfully set option "rgw_log_object_name_utc" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.533181 I | op-config: setting option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.916037 I | op-config: successfully set option "rgw_zonegroup" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:15.916074 I | op-config: setting option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:16.288737 I | op-config: successfully set option "rgw_keystone_admin_user" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:16.288777 I | op-config: setting option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:16.672104 I | op-config: successfully set option "rgw_swift_account_in_url" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:16.672157 I | op-config: setting option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:17.043347 I | op-config: successfully set option "rgw_swift_versioning_enabled" (user "client.rgw.ceph.a") to the mon configuration database
2026-04-08 01:39:17.043677 I | ceph-object-controller: object store "ceph" deployment "rook-ceph-rgw-ceph-a" created
2026-04-08 01:39:17.099755 I | ceph-object-controller: enabling rgw dashboard
2026-04-08 01:39:18.829127 I | ceph-object-controller: created object store "ceph" in namespace "openstack"
2026-04-08 01:39:18.829840 I | ceph-object-controller: setting the dashboard api secret key
2026-04-08 01:39:19.615232 I | ceph-object-controller: done setting the dashboard api secret key
2026-04-08 01:39:19.772152 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:39:23.189021 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:39:24.373444 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:39:53.650275 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:39:54.843136 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:39:57.322055 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:40:24.107484 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:107} {StateName:peering Count:2} {StateName:clean+premerge+peered Count:1}]"
2026-04-08 01:40:24.112819 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:40:25.337007 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:107} {StateName:peering Count:2} {StateName:clean+premerge+peered Count:1}]"
2026-04-08 01:40:25.341331 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:40:43.233512 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:40:54.573735 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:105} {StateName:activating Count:1}]"
2026-04-08 01:40:54.577790 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:40:55.790460 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:105} {StateName:activating Count:1}]"
2026-04-08 01:40:55.798033 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:41:25.040922 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:110} {StateName:peering Count:1}]"
2026-04-08 01:41:25.047189 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:41:26.281986 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:110} {StateName:peering Count:1}]"
2026-04-08 01:41:26.285782 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:41:29.238897 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:41:55.538204 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:105} {StateName:activating Count:2}]"
2026-04-08 01:41:55.542510 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:41:56.755833 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:105} {StateName:activating Count:2}]"
2026-04-08 01:41:56.760367 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:42:15.173678 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:42:26.054528 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:42:27.222879 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:42:56.528685 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:123} {StateName:activating Count:1}]"
2026-04-08 01:42:56.534433 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:42:57.732243 I | clusterdisruption-controller: all "host" failure domains: []. osd is down in failure domain: "". active node drains: false. pg health: "cluster is not fully clean. PGs: [{StateName:active+clean Count:123} {StateName:activating Count:1}]"
2026-04-08 01:42:57.736435 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:43:01.111826 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:43:27.019608 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:43:28.225144 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:43:47.077708 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:43:57.507209 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:43:59.374264 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:44:27.984597 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:44:29.826957 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:44:33.036652 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:44:58.437460 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:45:00.294062 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:45:19.005728 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:45:28.901109 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:45:30.756172 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:45:59.374326 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:46:01.232089 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:46:04.941589 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:46:29.835414 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:46:31.686179 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:46:50.867585 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:47:00.294442 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:47:02.158273 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:47:30.804780 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:47:32.635134 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:47:36.832736 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:48:01.272672 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:48:03.097006 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:48:22.774921 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:48:31.743243 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:48:33.594632 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:49:02.233074 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:49:04.076912 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:49:08.758151 W | op-mon: external cluster mon count is 1, consider adding new monitors.
2026-04-08 01:49:32.702176 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0
2026-04-08 01:49:34.538320 I | clusterdisruption-controller: reconciling osd pdb reconciler as the allowed disruptions in default pdb is 0