I0418 03:30:21.982112 1 options.go:220] external host was not specified, using 199.204.45.78 I0418 03:30:21.982884 1 server.go:148] Version: v1.28.13 I0418 03:30:21.982970 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0418 03:30:22.617866 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0418 03:30:22.628948 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0418 03:30:22.628972 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0418 03:30:22.629184 1 instance.go:298] Using reconciler: lease I0418 03:30:22.680339 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0418 03:30:22.680369 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0418 03:30:22.944620 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0418 03:30:22.944840 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0418 03:30:23.420436 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0418 03:30:23.438458 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0418 03:30:23.438554 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.438573 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.439048 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0418 03:30:23.439082 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0418 03:30:23.439894 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0418 03:30:23.440578 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0418 03:30:23.440615 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0418 03:30:23.440631 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0418 03:30:23.441879 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0418 03:30:23.441917 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0418 03:30:23.442854 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0418 03:30:23.442891 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.442906 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.443449 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0418 03:30:23.443482 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.443530 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0418 03:30:23.444024 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0418 03:30:23.445581 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0418 03:30:23.445618 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.445635 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.446096 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0418 03:30:23.446136 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.446151 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.446897 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0418 03:30:23.446935 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0418 03:30:23.448605 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0418 03:30:23.448655 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.448671 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.449116 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0418 03:30:23.449154 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.449169 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.451166 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0418 03:30:23.451207 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.451226 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.452334 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0418 03:30:23.453363 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0418 03:30:23.453401 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.453419 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.456657 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0418 03:30:23.456698 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0418 03:30:23.456715 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0418 03:30:23.457464 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0418 03:30:23.457503 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0418 03:30:23.457521 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0418 03:30:23.458233 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0418 03:30:23.458271 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0418 03:30:23.505907 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0418 03:30:23.505948 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0418 03:30:23.897343 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0418 03:30:23.897342 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0418 03:30:23.897563 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0418 03:30:23.897707 1 secure_serving.go:213] Serving securely on [::]:6443 I0418 03:30:23.897800 1 aggregator.go:164] waiting for initial CRD sync... I0418 03:30:23.897812 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0418 03:30:23.897896 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0418 03:30:23.897783 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0418 03:30:23.898078 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0418 03:30:23.898093 1 system_namespaces_controller.go:67] Starting system namespaces controller I0418 03:30:23.898188 1 controller.go:78] Starting OpenAPI AggregationController I0418 03:30:23.898244 1 available_controller.go:423] Starting AvailableConditionController I0418 03:30:23.898272 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0418 03:30:23.898290 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0418 03:30:23.898297 1 controller.go:80] Starting OpenAPI V3 AggregationController I0418 03:30:23.898299 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0418 03:30:23.898335 1 controller.go:116] Starting legacy_token_tracking_controller I0418 03:30:23.898400 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0418 03:30:23.898540 1 gc_controller.go:78] Starting apiserver lease garbage collector I0418 03:30:23.898559 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0418 03:30:23.898584 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0418 03:30:23.898601 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0418 03:30:23.898659 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0418 03:30:23.898740 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0418 03:30:23.898782 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0418 03:30:23.898809 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0418 03:30:23.898881 1 controller.go:134] Starting OpenAPI controller I0418 03:30:23.898940 1 controller.go:85] Starting OpenAPI V3 controller I0418 03:30:23.898985 1 naming_controller.go:291] Starting NamingConditionController I0418 03:30:23.899020 1 establishing_controller.go:76] Starting EstablishingController I0418 03:30:23.899047 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0418 03:30:23.899077 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0418 03:30:23.899107 1 crd_finalizer.go:266] Starting CRDFinalizer I0418 03:30:23.898087 1 gc_controller.go:78] Starting apiserver lease garbage collector I0418 03:30:23.917967 1 shared_informer.go:318] Caches are synced for node_authorizer E0418 03:30:23.967617 1 controller.go:146] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" I0418 03:30:23.998345 1 apf_controller.go:379] Running API Priority and Fairness config worker I0418 03:30:23.998415 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0418 03:30:23.998421 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0418 03:30:23.998381 1 cache.go:39] Caches are synced for AvailableConditionController controller I0418 03:30:23.998629 1 shared_informer.go:318] Caches are synced for configmaps I0418 03:30:23.998666 1 shared_informer.go:318] Caches are synced for crd-autoregister I0418 03:30:23.998717 1 aggregator.go:166] initial CRD sync complete... I0418 03:30:23.998729 1 autoregister_controller.go:141] Starting autoregister controller I0418 03:30:23.998734 1 cache.go:32] Waiting for caches to sync for autoregister controller I0418 03:30:23.998740 1 cache.go:39] Caches are synced for autoregister controller I0418 03:30:23.998993 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0418 03:30:24.000357 1 controller.go:624] quota admission added evaluator for: namespaces I0418 03:30:24.172671 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0418 03:30:24.906525 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0418 03:30:24.911598 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0418 03:30:24.911627 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0418 03:30:25.480214 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0418 03:30:25.530637 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0418 03:30:25.627210 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0418 03:30:25.640229 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.78] I0418 03:30:25.641862 1 controller.go:624] quota admission added evaluator for: endpoints I0418 03:30:25.651462 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0418 03:30:26.575863 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0418 03:30:27.460813 1 controller.go:624] quota admission added evaluator for: deployments.apps I0418 03:30:27.485210 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0418 03:30:27.691236 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0418 03:30:31.218920 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0418 03:30:31.816726 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0418 03:30:51.546385 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0418 03:30:51.567773 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0418 03:30:51.576886 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0418 03:30:51.576943 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0418 03:30:51.600587 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0418 03:30:51.619746 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0418 03:30:51.619849 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0418 03:30:51.650434 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0418 03:30:51.676114 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0418 03:30:51.688792 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0418 03:30:51.709830 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0418 03:30:51.709988 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0418 03:30:51.760334 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0418 03:30:51.760400 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0418 03:30:51.787725 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0418 03:30:51.936525 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0418 03:30:51.936705 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0418 03:30:51.963031 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:52.022189 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:52.074103 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:52.176865 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:52.180707 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:53.025844 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:53.027412 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:53.096836 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0418 03:30:58.298280 1 trace.go:236] Trace[1046820809]: "Update" accept:application/json, */*,audit-id:2648d68f-b7fd-42a7-a97d-108401c4d412,client:199.204.45.78,protocol:HTTP/2.0,resource:secrets,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/secrets/sh.helm.release.v1.envoy-gateway.v1,user-agent:Helm/3.11.2,verb:PUT (18-Apr-2026 03:30:57.725) (total time: 572ms): Trace[1046820809]: ---"Conversion done" 20ms (03:30:57.750) Trace[1046820809]: ["GuaranteedUpdate etcd3" audit-id:2648d68f-b7fd-42a7-a97d-108401c4d412,key:/secrets/envoy-gateway-system/sh.helm.release.v1.envoy-gateway.v1,type:*core.Secret,resource:secrets 547ms (03:30:57.751) Trace[1046820809]: ---"Txn call completed" 501ms (03:30:58.257)] Trace[1046820809]: ---"Writing http response done" 39ms (03:30:58.298) Trace[1046820809]: [572.377157ms] [572.377157ms] END I0418 03:30:58.822534 1 controller.go:624] quota admission added evaluator for: jobs.batch I0418 03:30:59.490242 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0418 03:30:59.497322 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0418 03:30:59.505956 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0418 03:30:59.873342 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0418 03:31:00.079902 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0418 03:31:00.267714 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0418 03:31:00.669427 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0418 03:31:02.101813 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0418 03:31:02.286626 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0418 03:31:03.130449 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0418 03:31:03.278047 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0418 03:31:17.810950 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0418 03:31:17.810995 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0418 03:31:18.362232 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0418 03:31:26.848061 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.104.90.60"} W0418 03:31:26.877757 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.90.60:9443: connect: connection refused E0418 03:31:26.877814 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.90.60:9443: connect: connection refused I0418 03:31:39.107699 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0418 03:31:39.128365 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0418 03:31:39.143117 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0418 03:31:39.172242 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.105.8.100"} I0418 03:31:39.176381 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0418 03:31:39.180202 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.96.202.213"} I0418 03:31:39.220991 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0418 03:31:39.276126 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager W0418 03:31:59.558767 1 dispatcher.go:225] Failed calling webhook, failing closed webhook.cert-manager.io: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": context deadline exceeded I0418 03:31:59.559134 1 trace.go:236] Trace[1203012293]: "Create" accept:application/json, */*,audit-id:646a8598-ab03-4801-b81b-fb1cc6d7a623,client:10.0.0.185,protocol:HTTP/2.0,resource:certificates,scope:resource,url:/apis/cert-manager.io/v1/namespaces/cert-manager/certificates,user-agent:Go-http-client/2.0,verb:POST (18-Apr-2026 03:31:49.557) (total time: 10001ms): Trace[1203012293]: ["Call mutating webhook" configuration:cert-manager-webhook,webhook:webhook.cert-manager.io,resource:cert-manager.io/v1, Resource=certificates,subresource:,operation:CREATE,UID:1d5df56c-0de6-43cd-901b-709b672df6bc 10000ms (03:31:49.558)] Trace[1203012293]: [10.001407317s] [10.001407317s] END I0418 03:31:59.580377 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0418 03:32:05.778627 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0418 03:32:20.180674 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.104.123.178"} I0418 03:32:20.192570 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.104.76.233"} I0418 03:32:20.200753 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.103.149.59"} I0418 03:32:20.208143 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.96.148.92"} I0418 03:32:26.682048 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.699078 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.723148 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.741424 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.764436 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.790800 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.804103 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.845036 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.873349 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.885949 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0418 03:32:26.905510 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.930464 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:26.945919 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:28.286811 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0418 03:32:32.058005 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0418 03:32:32.065199 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0418 03:32:32.123372 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.111.149.26"} I0418 03:32:32.150248 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0418 03:32:34.419607 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0418 03:32:34.469938 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0418 03:32:35.115067 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0418 03:32:35.115217 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0418 03:32:35.115318 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0418 03:32:41.877409 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0418 03:32:46.596236 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.100.168.16"} I0418 03:32:46.974629 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0418 03:32:47.119923 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.105.13.97"} I0418 03:32:47.132206 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.99.65.178"} I0418 03:33:33.452524 1 trace.go:236] Trace[863698190]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c6f6acbe-1c8c-4a60-b5fa-84b8ca6d6bcd,client:199.204.45.78,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/percona-xtradb-haproxy-0.18a7560e5a40ba5d,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (18-Apr-2026 03:33:32.672) (total time: 780ms): Trace[863698190]: ["GuaranteedUpdate etcd3" audit-id:c6f6acbe-1c8c-4a60-b5fa-84b8ca6d6bcd,key:/events/openstack/percona-xtradb-haproxy-0.18a7560e5a40ba5d,type:*core.Event,resource:events 780ms (03:33:32.672) Trace[863698190]: ---"Transaction prepared" 556ms (03:33:33.231) Trace[863698190]: ---"Txn call completed" 220ms (03:33:33.452)] Trace[863698190]: ---"Object stored in database" 777ms (03:33:33.452) Trace[863698190]: [780.228377ms] [780.228377ms] END I0418 03:34:26.489691 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.101.76.134"} I0418 03:34:28.977964 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey" clusterIPs={"IPv4":"10.111.238.196"} I0418 03:34:28.985608 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey-metrics" clusterIPs={"IPv4":"10.100.181.222"} I0418 03:34:35.235447 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.105.163.237"} I0418 03:34:35.245217 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.97.229.46"} I0418 03:37:03.023071 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0418 03:37:06.687448 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0418 03:37:06.718092 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0418 03:37:10.511848 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0418 03:37:10.534344 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0418 03:37:10.564121 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0418 03:37:10.594074 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0418 03:37:10.603550 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0418 03:37:10.615473 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0418 03:37:10.629994 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0418 03:37:12.715643 1 controller.go:624] quota admission added evaluator for: nodefeatures.nfd.k8s-sigs.io I0418 03:37:23.275900 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0418 03:38:01.638067 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0418 03:38:15.887240 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0418 03:38:16.092501 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.096020 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.111004 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.327037 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0418 03:38:16.595723 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.600077 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.662277 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0418 03:38:16.682451 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:16.863810 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0418 03:38:54.133625 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.98.205.143"} I0418 03:38:54.142618 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.98.245.36"} I0418 03:38:54.151229 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.106.202.8"} I0418 03:38:54.159465 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.111.121.205"} I0418 03:38:54.170958 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.103.99.29"} I0418 03:38:54.178896 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.111.152.103"} I0418 03:38:54.225936 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0418 03:38:54.235738 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0418 03:38:54.268154 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0418 03:38:54.279035 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0418 03:38:54.299387 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0418 03:38:54.307740 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.307777 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.307936 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.307981 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.308049 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.308096 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.308218 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.308273 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.308365 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.308393 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.309240 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.309259 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.309268 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.309282 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.309299 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.309275 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.310106 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.310134 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.310542 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.310573 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.310737 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.310767 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.310828 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.310873 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.311550 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.311643 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.312454 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.312482 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.312575 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.312594 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused W0418 03:38:54.312657 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused E0418 03:38:54.312793 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.106.202.8:443: connect: connection refused I0418 03:39:04.975234 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki" clusterIPs={"IPv4":"10.97.125.72"} I0418 03:39:04.984793 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki-gateway" clusterIPs={"IPv4":"10.106.51.143"} I0418 03:39:09.552394 1 alloc.go:330] "allocated clusterIPs" service="monitoring/goldpinger" clusterIPs={"IPv4":"10.106.195.238"} I0418 03:39:12.304238 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-pushgateway" clusterIPs={"IPv4":"10.97.27.98"} I0418 03:39:17.271898 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.106.52.4"} I0418 03:39:18.129940 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.101.100.101"} I0418 03:39:24.936064 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0418 03:39:25.214902 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.101.43.81"} E0418 03:39:51.498864 1 upgradeaware.go:425] Error proxying data from client to backend: write tcp 199.204.45.78:33658->199.204.45.78:10250: write: broken pipe I0418 03:40:05.009238 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.110.144.255"} I0418 03:40:05.027168 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0418 03:43:01.573555 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-barbican" clusterIPs={"IPv4":"10.97.221.44"} I0418 03:43:35.020480 1 alloc.go:330] "allocated clusterIPs" service="openstack/barbican-api" clusterIPs={"IPv4":"10.96.238.197"} I0418 03:45:21.961869 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0418 03:45:21.977484 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.019045 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.019230 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0418 03:45:22.019549 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.027995 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.036514 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.045647 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.053024 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.061217 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.082447 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.100435 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.119116 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.138648 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.152844 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.193487 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.247069 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.282051 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:45:22.420021 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0418 03:46:02.715502 1 controller.go:624] quota admission added evaluator for: cephclusters.ceph.rook.io I0418 03:46:02.726691 1 controller.go:624] quota admission added evaluator for: cephobjectstores.ceph.rook.io I0418 03:46:36.551350 1 alloc.go:330] "allocated clusterIPs" service="openstack/rook-ceph-rgw-ceph" clusterIPs={"IPv4":"10.107.163.251"} I0418 03:46:37.703695 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-glance" clusterIPs={"IPv4":"10.111.183.102"} I0418 03:47:11.609878 1 alloc.go:330] "allocated clusterIPs" service="openstack/glance-api" clusterIPs={"IPv4":"10.111.69.130"} I0418 03:49:49.461942 1 alloc.go:330] "allocated clusterIPs" service="openstack/staffeln-api" clusterIPs={"IPv4":"10.103.8.136"} I0418 03:50:10.060865 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-cinder" clusterIPs={"IPv4":"10.101.141.134"} I0418 03:50:44.366061 1 alloc.go:330] "allocated clusterIPs" service="openstack/cinder-api" clusterIPs={"IPv4":"10.109.22.220"} E0418 03:53:14.104477 1 watch.go:287] unable to encode watch object *v1.WatchEvent: http2: stream closed (&streaming.encoderWithAllocator{writer:responsewriter.outerWithCloseNotifyAndFlush{UserProvidedDecorator:(*metrics.ResponseWriterDelegator)(0xc0328357d0), InnerCloseNotifierFlusher:struct { httpsnoop.Unwrapper; http.ResponseWriter; http.Flusher; http.CloseNotifier; http.Pusher }{Unwrapper:(*httpsnoop.rw)(0xc01da82460), ResponseWriter:(*httpsnoop.rw)(0xc01da82460), Flusher:(*httpsnoop.rw)(0xc01da82460), CloseNotifier:(*httpsnoop.rw)(0xc01da82460), Pusher:(*httpsnoop.rw)(0xc01da82460)}}, encoder:(*versioning.codec)(0xc025ee6d20), memAllocator:(*runtime.Allocator)(0xc0250f8240)}) I0418 03:55:32.254425 1 alloc.go:330] "allocated clusterIPs" service="openstack/placement-api" clusterIPs={"IPv4":"10.104.32.204"} I0418 03:58:05.459857 1 alloc.go:330] "allocated clusterIPs" service="openstack/neutron-coredns-metrics" clusterIPs={"IPv4":"10.103.59.108"} I0418 03:58:05.468458 1 alloc.go:330] "allocated clusterIPs" service="openstack/neutron-coredns" clusterIPs={"IPv4":"10.96.0.20"} I0418 03:58:09.430439 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-nova" clusterIPs={"IPv4":"10.98.98.170"} I0418 03:58:47.377674 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-novncproxy" clusterIPs={"IPv4":"10.109.33.29"} I0418 03:58:47.388444 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-api" clusterIPs={"IPv4":"10.106.150.87"} I0418 03:58:47.399908 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-metadata" clusterIPs={"IPv4":"10.96.79.56"} I0418 04:01:53.296813 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-neutron" clusterIPs={"IPv4":"10.111.62.87"} I0418 04:02:28.222599 1 alloc.go:330] "allocated clusterIPs" service="openstack/neutron-server" clusterIPs={"IPv4":"10.111.117.12"} I0418 04:05:23.542725 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-heat" clusterIPs={"IPv4":"10.102.114.160"} I0418 04:05:57.629945 1 alloc.go:330] "allocated clusterIPs" service="openstack/heat-cfn" clusterIPs={"IPv4":"10.100.233.253"} I0418 04:05:57.637976 1 alloc.go:330] "allocated clusterIPs" service="openstack/heat-api" clusterIPs={"IPv4":"10.101.164.178"} I0418 04:09:35.987414 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-octavia" clusterIPs={"IPv4":"10.103.89.110"} I0418 04:11:26.075829 1 controller.go:624] quota admission added evaluator for: sshkeys.secretgen.k14s.io I0418 04:11:39.075862 1 alloc.go:330] "allocated clusterIPs" service="openstack/octavia-api" clusterIPs={"IPv4":"10.107.211.98"} I0418 04:14:01.682587 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-magnum" clusterIPs={"IPv4":"10.98.95.195"} E0418 04:14:28.199947 1 upgradeaware.go:425] Error proxying data from client to backend: write tcp 199.204.45.78:48358->199.204.45.78:10250: write: connection reset by peer I0418 04:14:39.361601 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.394743 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.454100 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.496135 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.556363 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.570390 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.613836 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.658923 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.684003 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.711652 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.728107 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.775360 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.817958 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.835854 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.883928 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.914007 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.963195 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:39.985141 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:40.013540 1 handler.go:275] Adding GroupVersion openstack.k-orc.cloud v1alpha1 to ResourceManager I0418 04:14:40.153512 1 alloc.go:330] "allocated clusterIPs" service="orc-system/orc-controller-manager-metrics-service" clusterIPs={"IPv4":"10.106.197.62"} I0418 04:14:43.675777 1 handler.go:275] Adding GroupVersion clusterctl.cluster.x-k8s.io v1alpha3 to ResourceManager I0418 04:14:45.130434 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.148470 1 handler.go:275] Adding GroupVersion addons.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.172500 1 handler.go:275] Adding GroupVersion addons.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.204621 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.210322 1 handler.go:275] Adding GroupVersion runtime.cluster.x-k8s.io v1alpha1 to ResourceManager I0418 04:14:45.220922 1 handler.go:275] Adding GroupVersion ipam.cluster.x-k8s.io v1alpha1 to ResourceManager I0418 04:14:45.221037 1 handler.go:275] Adding GroupVersion ipam.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.230425 1 handler.go:275] Adding GroupVersion ipam.cluster.x-k8s.io v1alpha1 to ResourceManager I0418 04:14:45.230480 1 handler.go:275] Adding GroupVersion ipam.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.305127 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.339761 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.388341 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.443288 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.464347 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.504130 1 handler.go:275] Adding GroupVersion cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.601024 1 alloc.go:330] "allocated clusterIPs" service="capi-system/capi-webhook-service" clusterIPs={"IPv4":"10.111.6.126"} I0418 04:14:45.676902 1 controller.go:624] quota admission added evaluator for: providers.clusterctl.cluster.x-k8s.io I0418 04:14:45.896436 1 handler.go:275] Adding GroupVersion bootstrap.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:45.961005 1 alloc.go:330] "allocated clusterIPs" service="capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-webhook-service" clusterIPs={"IPv4":"10.111.160.151"} I0418 04:14:45.984374 1 handler.go:275] Adding GroupVersion bootstrap.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.393581 1 handler.go:275] Adding GroupVersion controlplane.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.433073 1 handler.go:275] Adding GroupVersion controlplane.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.455705 1 alloc.go:330] "allocated clusterIPs" service="capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-webhook-service" clusterIPs={"IPv4":"10.100.149.53"} I0418 04:14:46.814712 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.869968 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.877004 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1alpha1 to ResourceManager I0418 04:14:46.904472 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.927332 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1beta1 to ResourceManager I0418 04:14:46.963067 1 handler.go:275] Adding GroupVersion infrastructure.cluster.x-k8s.io v1alpha1 to ResourceManager I0418 04:14:47.035950 1 alloc.go:330] "allocated clusterIPs" service="capo-system/capo-webhook-service" clusterIPs={"IPv4":"10.99.98.78"} I0418 04:16:00.839206 1 alloc.go:330] "allocated clusterIPs" service="openstack/magnum-api" clusterIPs={"IPv4":"10.104.28.176"} I0418 04:18:14.183751 1 alloc.go:330] "allocated clusterIPs" service="openstack/magnum-registry" clusterIPs={"IPv4":"10.101.128.71"} I0418 04:18:27.142627 1 trace.go:236] Trace[1350941417]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:c1cd0974-9718-4492-9f18-c579673381f5,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (18-Apr-2026 04:18:26.612) (total time: 530ms): Trace[1350941417]: ---"About to write a response" 530ms (04:18:27.142) Trace[1350941417]: [530.157908ms] [530.157908ms] END I0418 04:18:27.142808 1 trace.go:236] Trace[107502086]: "Patch" accept:application/json, */*,audit-id:ff9eab3b-f51d-42dd-a4fe-3a7f1d397437,client:199.204.45.78,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/container-infra.18a75881fa86165a,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PATCH (18-Apr-2026 04:18:26.157) (total time: 985ms): Trace[107502086]: ["GuaranteedUpdate etcd3" audit-id:ff9eab3b-f51d-42dd-a4fe-3a7f1d397437,key:/events/openstack/container-infra.18a75881fa86165a,type:*core.Event,resource:events 985ms (04:18:26.157) Trace[107502086]: ---"initial value restored" 395ms (04:18:26.552) Trace[107502086]: ---"Txn call completed" 588ms (04:18:27.142)] Trace[107502086]: ---"Object stored in database" 589ms (04:18:27.142) Trace[107502086]: [985.407472ms] [985.407472ms] END I0418 04:18:27.142980 1 trace.go:236] Trace[1510503664]: "Update" accept:application/json, */*,audit-id:41dd325b-1231-4ae4-8905-54e7b29ddb4f,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (18-Apr-2026 04:18:26.592) (total time: 550ms): Trace[1510503664]: ["GuaranteedUpdate etcd3" audit-id:41dd325b-1231-4ae4-8905-54e7b29ddb4f,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 550ms (04:18:26.592) Trace[1510503664]: ---"Txn call completed" 549ms (04:18:27.142)] Trace[1510503664]: [550.165211ms] [550.165211ms] END I0418 04:18:27.142987 1 trace.go:236] Trace[1640131706]: "Get" accept:application/json, */*,audit-id:edb547b6-5eca-400d-9cc6-8dfd3852837e,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (18-Apr-2026 04:18:26.603) (total time: 539ms): Trace[1640131706]: ---"About to write a response" 539ms (04:18:27.142) Trace[1640131706]: [539.555533ms] [539.555533ms] END I0418 04:18:27.143068 1 trace.go:236] Trace[202515652]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:da0b787d-0068-4c97-84ab-fb4427d68946,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (18-Apr-2026 04:18:26.593) (total time: 549ms): Trace[202515652]: ["GuaranteedUpdate etcd3" audit-id:da0b787d-0068-4c97-84ab-fb4427d68946,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 549ms (04:18:26.593) Trace[202515652]: ---"Txn call completed" 548ms (04:18:27.142)] Trace[202515652]: [549.874074ms] [549.874074ms] END I0418 04:18:27.147868 1 trace.go:236] Trace[601055662]: "Get" accept:application/json, */*,audit-id:009fb1ec-a981-4681-9d94-d480dac17d14,client:10.0.0.21,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0/log,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:CONNECT (18-Apr-2026 04:18:26.593) (total time: 554ms): Trace[601055662]: ---"About to write a response" 552ms (04:18:27.145) Trace[601055662]: [554.744237ms] [554.744237ms] END I0418 04:18:27.935344 1 trace.go:236] Trace[700043601]: "Delete" accept:application/vnd.kubernetes.protobuf, */*,audit-id:7157acf7-634c-4921-ae80-0ed8faefe4cc,client:10.0.0.21,protocol:HTTP/2.0,resource:deployments,scope:resource,url:/apis/apps/v1/namespaces/openstack/deployments/percona-xtradb-pitr,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:DELETE (18-Apr-2026 04:18:27.215) (total time: 719ms): Trace[700043601]: [719.803739ms] [719.803739ms] END I0418 04:18:27.935381 1 trace.go:236] Trace[1694039893]: "Patch" accept:application/json, */*,audit-id:d67ce7ac-a77e-47b5-8e7c-3fd87b5aa87e,client:199.204.45.78,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/openstack/events/container-infra-registry.18a758829694fba4,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PATCH (18-Apr-2026 04:18:27.146) (total time: 789ms): Trace[1694039893]: ["GuaranteedUpdate etcd3" audit-id:d67ce7ac-a77e-47b5-8e7c-3fd87b5aa87e,key:/events/openstack/container-infra-registry.18a758829694fba4,type:*core.Event,resource:events 789ms (04:18:27.146) Trace[1694039893]: ---"initial value restored" 412ms (04:18:27.558) Trace[1694039893]: ---"Txn call completed" 375ms (04:18:27.935)] Trace[1694039893]: ---"Object stored in database" 375ms (04:18:27.935) Trace[1694039893]: [789.289656ms] [789.289656ms] END I0418 04:18:27.935703 1 trace.go:236] Trace[1627725271]: "Update" accept:application/json, */*,audit-id:72d983c2-35ad-47a1-aefc-f0548fc29d9e,client:10.0.0.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/capo-system/leases/controller-leader-election-capo,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:18:27.423) (total time: 512ms): Trace[1627725271]: ["GuaranteedUpdate etcd3" audit-id:72d983c2-35ad-47a1-aefc-f0548fc29d9e,key:/leases/capo-system/controller-leader-election-capo,type:*coordination.Lease,resource:leases.coordination.k8s.io 511ms (04:18:27.423) Trace[1627725271]: ---"Txn call completed" 510ms (04:18:27.935)] Trace[1627725271]: [512.155309ms] [512.155309ms] END I0418 04:18:46.567345 1 trace.go:236] Trace[325619703]: "GuaranteedUpdate etcd3" audit-id:,key:/ranges/serviceips,type:*core.RangeAllocation,resource:serviceipallocations (18-Apr-2026 04:18:45.910) (total time: 657ms): Trace[325619703]: ---"initial value restored" 233ms (04:18:46.143) Trace[325619703]: ---"Txn call completed" 423ms (04:18:46.567) Trace[325619703]: [657.015194ms] [657.015194ms] END I0418 04:18:46.822005 1 trace.go:236] Trace[707917112]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.78,type:*v1.Endpoints,resource:apiServerIPInfo (18-Apr-2026 04:18:45.758) (total time: 1062ms): Trace[707917112]: ---"initial value restored" 384ms (04:18:46.143) Trace[707917112]: ---"Txn call completed" 647ms (04:18:46.821) Trace[707917112]: [1.062993039s] [1.062993039s] END I0418 04:18:46.822250 1 trace.go:236] Trace[1551364302]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c55c3a01-2019-4820-a6f3-7c9b1401de05,client:10.0.0.21,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:18:46.241) (total time: 580ms): Trace[1551364302]: ["GuaranteedUpdate etcd3" audit-id:c55c3a01-2019-4820-a6f3-7c9b1401de05,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 580ms (04:18:46.241) Trace[1551364302]: ---"Txn call completed" 578ms (04:18:46.821)] Trace[1551364302]: [580.838274ms] [580.838274ms] END I0418 04:18:46.822478 1 trace.go:236] Trace[405637250]: "Update" accept:application/json, */*,audit-id:26ab4498-0163-4e5b-b665-dee6e6c984bc,client:10.0.0.169,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/capo-system/leases/controller-leader-election-capo,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:18:46.245) (total time: 576ms): Trace[405637250]: ["GuaranteedUpdate etcd3" audit-id:26ab4498-0163-4e5b-b665-dee6e6c984bc,key:/leases/capo-system/controller-leader-election-capo,type:*coordination.Lease,resource:leases.coordination.k8s.io 576ms (04:18:46.245) Trace[405637250]: ---"Txn call completed" 575ms (04:18:46.822)] Trace[405637250]: [576.858247ms] [576.858247ms] END I0418 04:18:46.827157 1 trace.go:236] Trace[908065202]: "Create" accept:application/vnd.kubernetes.protobuf, */*,audit-id:647c5416-1385-4658-93f2-c0b382d616fa,client:199.204.45.78,protocol:HTTP/2.0,resource:serviceaccounts,scope:resource,url:/api/v1/namespaces/kube-system/serviceaccounts/endpointslice-controller/token,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/kube-controller-manager,verb:POST (18-Apr-2026 04:18:46.146) (total time: 680ms): Trace[908065202]: ---"Write to database call succeeded" len:81 679ms (04:18:46.826) Trace[908065202]: [680.175373ms] [680.175373ms] END I0418 04:18:46.858643 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-manila" clusterIPs={"IPv4":"10.96.29.135"} I0418 04:18:46.858883 1 trace.go:236] Trace[1307860912]: "Create" accept:application/vnd.kubernetes.protobuf, */*,audit-id:e1447c03-9457-4049-9b4d-159b8c64e0e7,client:10.0.0.218,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/openstack/services,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (18-Apr-2026 04:18:45.909) (total time: 949ms): Trace[1307860912]: [949.445776ms] [949.445776ms] END I0418 04:18:47.085129 1 trace.go:236] Trace[1482707]: "Create" accept:application/vnd.kubernetes.protobuf, */*,audit-id:d73aacd8-e72b-4556-84e1-47bbb704d94f,client:199.204.45.78,protocol:HTTP/2.0,resource:endpoints,scope:resource,url:/api/v1/namespaces/openstack/endpoints,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/system:serviceaccount:kube-system:endpoint-controller,verb:POST (18-Apr-2026 04:18:46.572) (total time: 512ms): Trace[1482707]: [512.626183ms] [512.626183ms] END I0418 04:18:47.645264 1 trace.go:236] Trace[456332154]: "List(recursive=true) etcd3" audit-id:,key:/masterleases/,resourceVersion:0,resourceVersionMatch:NotOlderThan,limit:0,continue: (18-Apr-2026 04:18:47.086) (total time: 559ms): Trace[456332154]: [559.202437ms] [559.202437ms] END I0418 04:18:49.854332 1 trace.go:236] Trace[792967443]: "Update" accept:application/json, */*,audit-id:4b7a6ea7-5449-4a5f-99d2-61ed04d1655f,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:18:49.302) (total time: 551ms): Trace[792967443]: ["GuaranteedUpdate etcd3" audit-id:4b7a6ea7-5449-4a5f-99d2-61ed04d1655f,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 551ms (04:18:49.302) Trace[792967443]: ---"Txn call completed" 550ms (04:18:49.854)] Trace[792967443]: [551.640773ms] [551.640773ms] END I0418 04:18:49.855141 1 trace.go:236] Trace[2124299500]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:0ba4c759-847e-457d-9b9c-4f05d531e75c,client:199.204.45.78,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/local-path-storage/pods/helper-pod-create-pvc-6485ccb7-7aa7-4ece-8ab8-56c7eb2ff9cf/status,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (18-Apr-2026 04:18:49.302) (total time: 552ms): Trace[2124299500]: ["GuaranteedUpdate etcd3" audit-id:0ba4c759-847e-457d-9b9c-4f05d531e75c,key:/pods/local-path-storage/helper-pod-create-pvc-6485ccb7-7aa7-4ece-8ab8-56c7eb2ff9cf,type:*core.Pod,resource:pods 552ms (04:18:49.302) Trace[2124299500]: ---"Txn call completed" 548ms (04:18:49.854)] Trace[2124299500]: ---"Object stored in database" 549ms (04:18:49.854) Trace[2124299500]: [552.64775ms] [552.64775ms] END I0418 04:18:56.333672 1 trace.go:236] Trace[1122064233]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:99ae71cf-70cf-4adc-8eb8-3a9c3878580f,client:10.0.0.253,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/orc-system/leases/f35396c5.k-orc.cloud,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:18:55.804) (total time: 529ms): Trace[1122064233]: ["GuaranteedUpdate etcd3" audit-id:99ae71cf-70cf-4adc-8eb8-3a9c3878580f,key:/leases/orc-system/f35396c5.k-orc.cloud,type:*coordination.Lease,resource:leases.coordination.k8s.io 529ms (04:18:55.804) Trace[1122064233]: ---"Txn call completed" 528ms (04:18:56.333)] Trace[1122064233]: [529.346396ms] [529.346396ms] END I0418 04:19:26.730884 1 trace.go:236] Trace[563666382]: "Update" accept:application/json, */*,audit-id:856239c8-5e0e-4b58-a2cd-2f5493f998ab,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (18-Apr-2026 04:19:25.902) (total time: 827ms): Trace[563666382]: ["GuaranteedUpdate etcd3" audit-id:856239c8-5e0e-4b58-a2cd-2f5493f998ab,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 827ms (04:19:25.903) Trace[563666382]: ---"Txn call completed" 826ms (04:19:26.730)] Trace[563666382]: [827.914665ms] [827.914665ms] END I0418 04:19:26.731433 1 trace.go:236] Trace[782378643]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:f7397426-f725-4461-9a12-23efd0a38466,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (18-Apr-2026 04:19:25.906) (total time: 824ms): Trace[782378643]: ["GuaranteedUpdate etcd3" audit-id:f7397426-f725-4461-9a12-23efd0a38466,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 824ms (04:19:25.906) Trace[782378643]: ---"Txn call completed" 823ms (04:19:26.731)] Trace[782378643]: [824.635736ms] [824.635736ms] END I0418 04:19:26.731735 1 trace.go:236] Trace[2008042179]: "Update" accept:application/json, */*,audit-id:2ce12329-d4f2-4921-97f2-35bd179d7069,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (18-Apr-2026 04:19:25.906) (total time: 825ms): Trace[2008042179]: ["GuaranteedUpdate etcd3" audit-id:2ce12329-d4f2-4921-97f2-35bd179d7069,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 825ms (04:19:25.906) Trace[2008042179]: ---"Txn call completed" 824ms (04:19:26.731)] Trace[2008042179]: [825.51881ms] [825.51881ms] END I0418 04:19:26.731896 1 trace.go:236] Trace[288051033]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:039d9b46-ea1b-409b-b029-b74d829d45b7,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (18-Apr-2026 04:19:25.905) (total time: 826ms): Trace[288051033]: ["GuaranteedUpdate etcd3" audit-id:039d9b46-ea1b-409b-b029-b74d829d45b7,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 826ms (04:19:25.905) Trace[288051033]: ---"Txn call completed" 825ms (04:19:26.731)] Trace[288051033]: [826.543588ms] [826.543588ms] END I0418 04:19:26.732096 1 trace.go:236] Trace[1982580250]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8e85bc8e-2e89-4cc0-bb93-caaa3fb52b26,client:199.204.45.78,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/ingress-nginx/services/ingress-nginx-controller,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (18-Apr-2026 04:19:26.112) (total time: 619ms): Trace[1982580250]: ---"About to write a response" 619ms (04:19:26.731) Trace[1982580250]: [619.853316ms] [619.853316ms] END I0418 04:19:26.732282 1 trace.go:236] Trace[248183379]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:229a6b9a-3f47-4a0f-965e-e320ab019519,client:10.0.0.21,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (18-Apr-2026 04:19:25.996) (total time: 735ms): Trace[248183379]: ---"About to write a response" 734ms (04:19:26.731) Trace[248183379]: [735.839275ms] [735.839275ms] END I0418 04:19:26.798846 1 trace.go:236] Trace[54675021]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.78,type:*v1.Endpoints,resource:apiServerIPInfo (18-Apr-2026 04:19:25.761) (total time: 1037ms): Trace[54675021]: ---"initial value restored" 140ms (04:19:25.901) Trace[54675021]: ---"Transaction prepared" 829ms (04:19:26.731) Trace[54675021]: ---"Txn call completed" 67ms (04:19:26.798) Trace[54675021]: [1.037624321s] [1.037624321s] END I0418 04:19:27.527093 1 trace.go:236] Trace[593870889]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:5c4a15ee-c3c5-43e7-bd91-4ce4e7f3b959,client:::1,protocol:HTTP/2.0,resource:endpoints,scope:resource,url:/api/v1/namespaces/default/endpoints/kubernetes,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (18-Apr-2026 04:19:26.799) (total time: 727ms): Trace[593870889]: ---"About to write a response" 727ms (04:19:27.526) Trace[593870889]: [727.485868ms] [727.485868ms] END I0418 04:19:27.530270 1 trace.go:236] Trace[2039033384]: "Get" accept:application/json, */*,audit-id:b000c06e-70b6-458a-b119-9afef63fe6cf,client:10.0.0.21,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0/log,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:CONNECT (18-Apr-2026 04:19:26.733) (total time: 796ms): Trace[2039033384]: ---"About to write a response" 793ms (04:19:27.527) Trace[2039033384]: [796.235495ms] [796.235495ms] END I0418 04:19:28.493364 1 trace.go:236] Trace[1773288362]: "Update" accept:application/json, */*,audit-id:9bff3ca2-8bf7-45a1-8b4c-424fef63969f,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:19:27.971) (total time: 521ms): Trace[1773288362]: ["GuaranteedUpdate etcd3" audit-id:9bff3ca2-8bf7-45a1-8b4c-424fef63969f,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 520ms (04:19:27.972) Trace[1773288362]: ---"Txn call completed" 519ms (04:19:28.493)] Trace[1773288362]: [521.771971ms] [521.771971ms] END I0418 04:19:28.703172 1 trace.go:236] Trace[1544170730]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:2ad2fbc1-0c42-4e67-82e0-395ffb7a3ceb,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (18-Apr-2026 04:19:28.036) (total time: 666ms): Trace[1544170730]: ["GuaranteedUpdate etcd3" audit-id:2ad2fbc1-0c42-4e67-82e0-395ffb7a3ceb,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 666ms (04:19:28.036) Trace[1544170730]: ---"Txn call completed" 665ms (04:19:28.702)] Trace[1544170730]: [666.964854ms] [666.964854ms] END I0418 04:19:28.706900 1 trace.go:236] Trace[288696356]: "Update" accept:application/json, */*,audit-id:ede50abf-38b5-4df0-a45a-61db72495cf6,client:10.0.0.21,protocol:HTTP/2.0,resource:perconaxtradbclusters,scope:resource,url:/apis/pxc.percona.com/v1/namespaces/openstack/perconaxtradbclusters/percona-xtradb/status,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (18-Apr-2026 04:19:27.982) (total time: 724ms): Trace[288696356]: ["GuaranteedUpdate etcd3" audit-id:ede50abf-38b5-4df0-a45a-61db72495cf6,key:/pxc.percona.com/perconaxtradbclusters/openstack/percona-xtradb,type:*unstructured.Unstructured,resource:perconaxtradbclusters.pxc.percona.com 723ms (04:19:27.983)] Trace[288696356]: [724.524658ms] [724.524658ms] END I0418 04:19:29.679963 1 trace.go:236] Trace[2049391665]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:dab0d134-c1ee-4dd5-8cab-fc8461f6c831,client:10.0.0.253,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/orc-system/leases/f35396c5.k-orc.cloud,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:19:29.167) (total time: 512ms): Trace[2049391665]: ["GuaranteedUpdate etcd3" audit-id:dab0d134-c1ee-4dd5-8cab-fc8461f6c831,key:/leases/orc-system/f35396c5.k-orc.cloud,type:*coordination.Lease,resource:leases.coordination.k8s.io 511ms (04:19:29.168) Trace[2049391665]: ---"Txn call completed" 510ms (04:19:29.679)] Trace[2049391665]: [512.090238ms] [512.090238ms] END I0418 04:19:31.164905 1 trace.go:236] Trace[298778913]: "Update" accept:application/json, */*,audit-id:dc7f629b-3f77-47c0-8ecb-8722c2efa983,client:10.0.0.49,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (18-Apr-2026 04:19:30.402) (total time: 762ms): Trace[298778913]: ["GuaranteedUpdate etcd3" audit-id:dc7f629b-3f77-47c0-8ecb-8722c2efa983,key:/leases/cert-manager/cert-manager-cainjector-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 762ms (04:19:30.402) Trace[298778913]: ---"Txn call completed" 761ms (04:19:31.164)] Trace[298778913]: [762.789527ms] [762.789527ms] END I0418 04:19:31.596518 1 trace.go:236] Trace[1463989327]: "Get" accept:application/json, */*,audit-id:23a3defc-9905-4b9e-bd2a-7c8178dbbe5e,client:199.204.45.78,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (18-Apr-2026 04:19:30.495) (total time: 1101ms): Trace[1463989327]: ---"About to write a response" 1100ms (04:19:31.596) Trace[1463989327]: [1.101115075s] [1.101115075s] END I0418 04:19:31.596786 1 trace.go:236] Trace[340723128]: "Get" accept:application/json, */*,audit-id:d3191b79-407d-47ca-8030-67cebbbe9151,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (18-Apr-2026 04:19:30.796) (total time: 800ms): Trace[340723128]: ---"About to write a response" 800ms (04:19:31.596) Trace[340723128]: [800.223093ms] [800.223093ms] END I0418 04:19:52.535785 1 alloc.go:330] "allocated clusterIPs" service="openstack/manila-api" clusterIPs={"IPv4":"10.106.185.10"} I0418 04:22:38.367368 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon" clusterIPs={"IPv4":"10.103.4.90"} I0418 04:22:38.373139 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon-int" clusterIPs={"IPv4":"10.97.203.216"} I0418 04:29:26.694976 1 trace.go:236] Trace[1651758432]: "List" accept:application/json,audit-id:4f30b731-a96a-4d60-b7e0-4504a77fce22,client:199.204.45.78,protocol:HTTP/2.0,resource:customresourcedefinitions,scope:cluster,url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:Helm/3.11.2,verb:LIST (18-Apr-2026 04:29:26.160) (total time: 528ms): Trace[1651758432]: ---"Writing http response done" count:135 246ms (04:29:26.689) Trace[1651758432]: [528.198678ms] [528.198678ms] END