I0519 13:13:22.975792 1 options.go:220] external host was not specified, using 199.204.45.248 I0519 13:13:22.976579 1 server.go:148] Version: v1.28.13 I0519 13:13:22.976646 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0519 13:13:23.251597 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0519 13:13:23.259079 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0519 13:13:23.259124 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0519 13:13:23.259313 1 instance.go:298] Using reconciler: lease I0519 13:13:23.405273 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0519 13:13:23.405297 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0519 13:13:23.638308 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0519 13:13:23.638510 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0519 13:13:24.035939 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0519 13:13:24.060543 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0519 13:13:24.060575 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.060583 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.061141 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0519 13:13:24.061157 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0519 13:13:24.062045 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0519 13:13:24.062876 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0519 13:13:24.062893 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0519 13:13:24.062899 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0519 13:13:24.064329 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0519 13:13:24.064346 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0519 13:13:24.065289 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0519 13:13:24.065306 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.065310 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.065892 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0519 13:13:24.065907 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.065944 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0519 13:13:24.066606 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0519 13:13:24.068002 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0519 13:13:24.068015 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.068019 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.068425 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0519 13:13:24.068437 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.068441 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.069323 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0519 13:13:24.069336 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0519 13:13:24.070714 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0519 13:13:24.070727 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.070731 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.071109 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0519 13:13:24.071122 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.071126 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.072934 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0519 13:13:24.072947 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.072951 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.073916 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0519 13:13:24.074951 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0519 13:13:24.074963 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.074967 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.078060 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0519 13:13:24.078073 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0519 13:13:24.078077 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0519 13:13:24.078774 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0519 13:13:24.078786 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0519 13:13:24.078790 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0519 13:13:24.079293 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0519 13:13:24.079305 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0519 13:13:24.121876 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0519 13:13:24.121911 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0519 13:13:24.519007 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0519 13:13:24.519026 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0519 13:13:24.519158 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0519 13:13:24.519331 1 secure_serving.go:213] Serving securely on [::]:6443 I0519 13:13:24.519460 1 available_controller.go:423] Starting AvailableConditionController I0519 13:13:24.519479 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0519 13:13:24.519539 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0519 13:13:24.519590 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0519 13:13:24.519662 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0519 13:13:24.519715 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0519 13:13:24.519734 1 aggregator.go:164] waiting for initial CRD sync... I0519 13:13:24.519663 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0519 13:13:24.519542 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0519 13:13:24.519871 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0519 13:13:24.519956 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0519 13:13:24.520126 1 controller.go:78] Starting OpenAPI AggregationController I0519 13:13:24.520169 1 controller.go:80] Starting OpenAPI V3 AggregationController I0519 13:13:24.520199 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0519 13:13:24.520213 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0519 13:13:24.519873 1 system_namespaces_controller.go:67] Starting system namespaces controller I0519 13:13:24.520285 1 controller.go:134] Starting OpenAPI controller I0519 13:13:24.520301 1 establishing_controller.go:76] Starting EstablishingController I0519 13:13:24.524233 1 controller.go:85] Starting OpenAPI V3 controller I0519 13:13:24.524269 1 naming_controller.go:291] Starting NamingConditionController I0519 13:13:24.520772 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0519 13:13:24.520789 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0519 13:13:24.520802 1 crd_finalizer.go:266] Starting CRDFinalizer I0519 13:13:24.520805 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0519 13:13:24.524491 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0519 13:13:24.521227 1 gc_controller.go:78] Starting apiserver lease garbage collector I0519 13:13:24.521504 1 controller.go:116] Starting legacy_token_tracking_controller I0519 13:13:24.524599 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0519 13:13:24.521543 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0519 13:13:24.521784 1 gc_controller.go:78] Starting apiserver lease garbage collector I0519 13:13:24.555277 1 shared_informer.go:318] Caches are synced for node_authorizer I0519 13:13:24.619603 1 cache.go:39] Caches are synced for AvailableConditionController controller I0519 13:13:24.620780 1 apf_controller.go:379] Running API Priority and Fairness config worker I0519 13:13:24.620805 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0519 13:13:24.621018 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0519 13:13:24.621089 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0519 13:13:24.622658 1 controller.go:624] quota admission added evaluator for: namespaces I0519 13:13:24.624552 1 shared_informer.go:318] Caches are synced for crd-autoregister I0519 13:13:24.624590 1 aggregator.go:166] initial CRD sync complete... I0519 13:13:24.624602 1 autoregister_controller.go:141] Starting autoregister controller I0519 13:13:24.624612 1 cache.go:32] Waiting for caches to sync for autoregister controller I0519 13:13:24.624782 1 shared_informer.go:318] Caches are synced for configmaps I0519 13:13:24.624622 1 cache.go:39] Caches are synced for autoregister controller I0519 13:13:24.641141 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0519 13:13:25.527248 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0519 13:13:25.532889 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0519 13:13:25.532907 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0519 13:13:26.069052 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0519 13:13:26.113354 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0519 13:13:26.231584 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0519 13:13:26.240664 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.248] I0519 13:13:26.241685 1 controller.go:624] quota admission added evaluator for: endpoints I0519 13:13:26.246517 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0519 13:13:27.640394 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0519 13:13:27.649497 1 controller.go:624] quota admission added evaluator for: deployments.apps I0519 13:13:27.730390 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0519 13:13:27.933629 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0519 13:13:33.432828 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0519 13:13:33.981566 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0519 13:13:51.446121 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0519 13:13:51.502005 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0519 13:13:51.502057 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0519 13:13:51.525116 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0519 13:13:51.525162 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0519 13:13:51.567813 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0519 13:13:51.583387 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0519 13:13:51.601366 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0519 13:13:51.618090 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0519 13:13:51.686992 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0519 13:13:51.687037 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0519 13:13:51.707938 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0519 13:13:51.734518 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0519 13:13:51.734616 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0519 13:13:51.766071 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0519 13:13:51.875816 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:51.972151 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.100953 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0519 13:13:52.101072 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0519 13:13:52.133430 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.185612 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.197814 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.821757 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.866478 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:52.969656 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0519 13:13:58.392782 1 trace.go:236] Trace[1117681618]: "Update" accept:application/json, */*,audit-id:eca62a00-a5bb-4ca1-9003-f33a8416ab8e,client:199.204.45.248,protocol:HTTP/2.0,resource:secrets,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/secrets/sh.helm.release.v1.envoy-gateway.v1,user-agent:Helm/3.11.2,verb:PUT (19-May-2026 13:13:57.815) (total time: 577ms): Trace[1117681618]: ---"limitedReadBody succeeded" len:1381245 25ms (13:13:57.840) Trace[1117681618]: ---"Conversion done" 23ms (13:13:57.864) Trace[1117681618]: ---"Writing http response done" 30ms (13:13:58.392) Trace[1117681618]: [577.62142ms] [577.62142ms] END I0519 13:13:59.032466 1 controller.go:624] quota admission added evaluator for: jobs.batch I0519 13:13:59.798856 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0519 13:14:00.089073 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0519 13:14:00.199380 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0519 13:14:00.389168 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0519 13:14:00.588893 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0519 13:14:00.807514 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0519 13:14:00.990210 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0519 13:14:01.214913 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0519 13:14:01.396829 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0519 13:14:04.332277 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0519 13:14:04.503888 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0519 13:14:04.671738 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0519 13:14:04.671782 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0519 13:14:08.193174 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0519 13:14:17.013610 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.104.187.235"} W0519 13:14:19.013275 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.187.235:9443: connect: connection refused E0519 13:14:19.013317 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.104.187.235:9443: connect: connection refused I0519 13:14:30.893971 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0519 13:14:30.911092 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0519 13:14:30.926818 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0519 13:14:30.937605 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.107.77.210"} I0519 13:14:30.948021 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.106.5.212"} I0519 13:14:30.963476 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0519 13:14:30.996436 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0519 13:14:31.046671 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0519 13:14:41.574594 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0519 13:14:46.076106 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0519 13:14:58.896218 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.106.252.182"} I0519 13:14:58.903181 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.101.171.97"} I0519 13:14:58.910636 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.96.142.68"} I0519 13:14:58.921047 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.100.146.169"} I0519 13:15:05.774410 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.794258 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.816717 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.839879 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.863053 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.877392 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.897181 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.920691 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.960693 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:05.996616 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0519 13:15:06.026833 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:06.049210 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:06.064557 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:07.279859 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0519 13:15:11.436210 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0519 13:15:11.559512 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.105.219.94"} I0519 13:15:11.590723 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0519 13:15:13.690672 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0519 13:15:13.731154 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0519 13:15:14.352227 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0519 13:15:14.352328 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0519 13:15:14.352389 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0519 13:15:21.269479 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0519 13:15:23.463355 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.108.13.184"} I0519 13:15:23.841283 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0519 13:15:23.952632 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0519 13:15:24.107013 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.108.193.30"} I0519 13:15:24.122903 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.97.58.145"} I0519 13:16:43.513642 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.110.125.225"} I0519 13:16:45.398881 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.107.153.25"} I0519 13:16:46.224791 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.103.228.232"} I0519 13:16:54.150060 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.100.157.231"} I0519 13:16:54.160416 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.104.225.57"} I0519 13:19:24.009149 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0519 13:19:25.924874 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0519 13:19:25.932242 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0519 13:19:25.943483 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0519 13:19:25.957809 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0519 13:19:25.978966 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0519 13:19:25.989299 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0519 13:19:26.006761 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0519 13:19:38.171718 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0519 13:20:27.588884 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0519 13:20:42.022070 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0519 13:20:42.248309 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:42.268696 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:42.272473 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:42.606865 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0519 13:20:42.862299 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:42.889082 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:43.006556 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0519 13:20:43.020001 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:20:43.118847 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0519 13:21:28.370751 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.105.220.40"} I0519 13:21:28.386381 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.111.80.173"} I0519 13:21:28.401035 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.101.214.17"} I0519 13:21:28.410941 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.98.80.152"} I0519 13:21:28.422505 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.105.124.209"} I0519 13:21:28.442097 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.108.233.133"} I0519 13:21:28.525272 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0519 13:21:28.546269 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0519 13:21:28.600951 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0519 13:21:28.619400 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0519 13:21:28.643205 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0519 13:21:28.656934 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.656979 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.657027 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.657053 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.657091 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.657121 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.657125 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.657139 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.656939 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.658787 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.660139 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.660143 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.660155 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.660163 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.660226 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.660254 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.660488 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.660503 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.660552 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.660584 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.661362 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.661406 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.661520 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.661552 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.661713 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.661733 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.661861 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.661879 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.662135 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.662163 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused W0519 13:21:28.664930 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused E0519 13:21:28.664954 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.105.124.209:443: connect: connection refused I0519 13:21:42.839371 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0519 13:21:43.015254 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.109.225.148"} I0519 13:22:23.629391 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.98.136.87"} I0519 13:22:23.643778 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0519 13:24:16.950162 1 trace.go:236] Trace[1249936921]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.248,type:*v1.Endpoints,resource:apiServerIPInfo (19-May-2026 13:24:16.256) (total time: 693ms): Trace[1249936921]: ---"Transaction prepared" 643ms (13:24:16.901) Trace[1249936921]: [693.237406ms] [693.237406ms] END I0519 13:24:51.864273 1 trace.go:236] Trace[1490426985]: "Update" accept:application/json, */*,audit-id:c01f6528-9600-4b68-b2a6-f61a583de945,client:10.0.0.244,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.12.17 (linux/amd64) cert-manager/37b853ff34a6c093e946c657c189f40413c0f628,verb:PUT (19-May-2026 13:24:50.986) (total time: 877ms): Trace[1490426985]: ["GuaranteedUpdate etcd3" audit-id:c01f6528-9600-4b68-b2a6-f61a583de945,key:/leases/cert-manager/cert-manager-controller,type:*coordination.Lease,resource:leases.coordination.k8s.io 877ms (13:24:50.987) Trace[1490426985]: ---"Txn call completed" 876ms (13:24:51.863)] Trace[1490426985]: [877.427067ms] [877.427067ms] END I0519 13:24:51.866104 1 trace.go:236] Trace[34218914]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1f54a429-77e5-461c-a3f3-c8dafe24489c,client:10.0.0.53,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:24:51.035) (total time: 830ms): Trace[34218914]: ["GuaranteedUpdate etcd3" audit-id:1f54a429-77e5-461c-a3f3-c8dafe24489c,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 830ms (13:24:51.035) Trace[34218914]: ---"Txn call completed" 827ms (13:24:51.864)] Trace[34218914]: [830.428456ms] [830.428456ms] END E0519 13:24:56.871545 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:24:56.871628 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:24:56.871688 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 26.16µs, panicked: false, err: context deadline exceeded, panic-reason: E0519 13:24:56.871992 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 10.67µs, panicked: false, err: context deadline exceeded, panic-reason: E0519 13:24:56.872146 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:24:56.873497 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:24:56.873613 1 trace.go:236] Trace[1107695244]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:2283e181-355c-4f9f-9eab-3a19c1dfb8ba,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (19-May-2026 13:24:51.871) (total time: 5001ms): Trace[1107695244]: ["GuaranteedUpdate etcd3" audit-id:2283e181-355c-4f9f-9eab-3a19c1dfb8ba,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (13:24:51.871) Trace[1107695244]: ---"Txn call failed" err:context deadline exceeded 4998ms (13:24:56.871)] Trace[1107695244]: [5.001712839s] [5.001712839s] END E0519 13:24:56.873780 1 timeout.go:142] post-timeout activity - time-elapsed: 2.307725ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0519 13:24:56.873839 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:24:56.874941 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:24:56.875013 1 trace.go:236] Trace[1635945356]: "Update" accept:application/json, */*,audit-id:37b3ef40-13e4-4774-8999-af64d672919c,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:24:51.871) (total time: 5003ms): Trace[1635945356]: ["GuaranteedUpdate etcd3" audit-id:37b3ef40-13e4-4774-8999-af64d672919c,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (13:24:51.872) Trace[1635945356]: ---"Txn call failed" err:context deadline exceeded 4998ms (13:24:56.871)] Trace[1635945356]: [5.00310171s] [5.00310171s] END E0519 13:24:56.875115 1 timeout.go:142] post-timeout activity - time-elapsed: 3.2274ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0519 13:24:56.969257 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:24:56.969304 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:24:56.969406 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 152.782µs, panicked: false, err: context canceled, panic-reason: E0519 13:24:56.970439 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:24:56.970538 1 trace.go:236] Trace[507097903]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4e27182c-8009-4223-b7fa-abf0e1143ba9,client:10.0.0.33,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:24:51.968) (total time: 5001ms): Trace[507097903]: ["GuaranteedUpdate etcd3" audit-id:4e27182c-8009-4223-b7fa-abf0e1143ba9,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (13:24:51.968) Trace[507097903]: ---"Txn call failed" err:context canceled 4999ms (13:24:56.969)] Trace[507097903]: [5.001926421s] [5.001926421s] END E0519 13:24:56.970654 1 timeout.go:142] post-timeout activity - time-elapsed: 1.795498ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: I0519 13:24:57.383160 1 metrics.go:285] "Failed to get storage metrics" storage_cluster_id="etcd-0" err="context deadline exceeded" E0519 13:24:57.421202 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0519 13:24:57.421326 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:24:57.423159 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:24:57.423238 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:24:57.424329 1 trace.go:236] Trace[2053637967]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:0a6dfb68-8f96-41fd-9863-eec794bc9f0f,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (19-May-2026 13:24:52.421) (total time: 5002ms): Trace[2053637967]: [5.002646923s] [5.002646923s] END E0519 13:24:57.424540 1 timeout.go:142] post-timeout activity - time-elapsed: 3.602465ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0519 13:24:58.868934 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 12.17µs, panicked: false, err: context deadline exceeded, panic-reason: E0519 13:24:58.868952 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:24:58.870802 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:24:58.870901 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:24:58.872113 1 trace.go:236] Trace[250232475]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e8fc449c-d27c-4610-a027-42f4d5d5fa5b,client:10.0.0.53,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:24:53.869) (total time: 5002ms): Trace[250232475]: ["GuaranteedUpdate etcd3" audit-id:e8fc449c-d27c-4610-a027-42f4d5d5fa5b,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (13:24:53.869) Trace[250232475]: ---"Txn call failed" err:context deadline exceeded 4997ms (13:24:58.868)] Trace[250232475]: [5.002198714s] [5.002198714s] END E0519 13:24:58.872284 1 timeout.go:142] post-timeout activity - time-elapsed: 3.470453ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0519 13:24:58.875634 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:24:58.875805 1 trace.go:236] Trace[1161754516]: "Update" accept:application/json, */*,audit-id:b0219613-d059-41c1-ad92-75b7f3afcb01,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (19-May-2026 13:24:51.871) (total time: 7004ms): Trace[1161754516]: ["GuaranteedUpdate etcd3" audit-id:b0219613-d059-41c1-ad92-75b7f3afcb01,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (13:24:51.871) Trace[1161754516]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (13:24:58.875)] Trace[1161754516]: [7.004058951s] [7.004058951s] END E0519 13:24:58.965017 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:24:58.965227 1 trace.go:236] Trace[458294110]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:f3cebe7d-29ba-4ff9-ac9c-0c003fa21723,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (19-May-2026 13:24:51.962) (total time: 7002ms): Trace[458294110]: ["GuaranteedUpdate etcd3" audit-id:f3cebe7d-29ba-4ff9-ac9c-0c003fa21723,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (13:24:51.962) Trace[458294110]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (13:24:58.964)] Trace[458294110]: [7.002718168s] [7.002718168s] END E0519 13:25:00.876771 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:00.876982 1 trace.go:236] Trace[394336774]: "Update" accept:application/json, */*,audit-id:fbfd472e-30bf-4beb-acf5-0ede80547f35,client:10.0.0.43,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:24:53.872) (total time: 7004ms): Trace[394336774]: ["GuaranteedUpdate etcd3" audit-id:fbfd472e-30bf-4beb-acf5-0ede80547f35,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (13:24:53.873) Trace[394336774]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (13:25:00.876)] Trace[394336774]: [7.00405642s] [7.00405642s] END E0519 13:25:01.787998 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:01.788136 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:01.788323 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:01.789213 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:01.789264 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:01.789860 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0519 13:25:01.790185 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:01.790561 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout I0519 13:25:01.790716 1 trace.go:236] Trace[999144191]: "Get" accept:application/json, */*,audit-id:77abc332-141a-4f03-96bb-3e5fee0ce263,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:24:58.875) (total time: 2914ms): Trace[999144191]: [2.914943647s] [2.914943647s] END E0519 13:25:01.790931 1 timeout.go:142] post-timeout activity - time-elapsed: 2.898004ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0519 13:25:01.791046 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:01.791385 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0519 13:25:01.792135 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout I0519 13:25:01.792147 1 trace.go:236] Trace[198903180]: "Get" accept:application/json, */*,audit-id:1cb93526-9ba5-480c-aa71-ab5f29c2c3ad,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (19-May-2026 13:25:00.878) (total time: 913ms): Trace[198903180]: [913.357195ms] [913.357195ms] END E0519 13:25:01.792479 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0519 13:25:01.792580 1 timeout.go:142] post-timeout activity - time-elapsed: 4.454418ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: I0519 13:25:01.793227 1 trace.go:236] Trace[1080839253]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:62c32658-d2cd-41bd-b122-87a2aeb132c2,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (19-May-2026 13:24:58.874) (total time: 2918ms): Trace[1080839253]: [2.918706674s] [2.918706674s] END E0519 13:25:01.793456 1 timeout.go:142] post-timeout activity - time-elapsed: 3.349421ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0519 13:25:01.968908 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:01.969692 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:01.970199 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:01.971432 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:01.971512 1 trace.go:236] Trace[1830904070]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4d98c6ff-1af8-426c-8ad2-4e6cebaeadf9,client:10.0.0.33,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:24:56.970) (total time: 5000ms): Trace[1830904070]: [5.00075005s] [5.00075005s] END E0519 13:25:01.971727 1 timeout.go:142] post-timeout activity - time-elapsed: 2.845384ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0519 13:25:01.994574 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:01.994727 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:01.996857 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:01.996966 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:01.998172 1 trace.go:236] Trace[1880536035]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:acbfe4a4-4ac2-4124-9e11-ab093fe347ad,client:10.0.0.33,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (19-May-2026 13:24:52.755) (total time: 9242ms): Trace[1880536035]: [9.242894629s] [9.242894629s] END E0519 13:25:01.998384 1 timeout.go:142] post-timeout activity - time-elapsed: 4.230454ms, GET "/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" result: E0519 13:25:02.283259 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:02.283343 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:02.284446 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:02.284520 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:02.285556 1 trace.go:236] Trace[1446956008]: "Get" accept:application/json, */*,audit-id:404ebe99-474b-47e3-8fc6-1257db8c781c,client:10.0.0.165,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:24:52.278) (total time: 10006ms): Trace[1446956008]: [10.006975362s] [10.006975362s] END E0519 13:25:02.285692 1 timeout.go:142] post-timeout activity - time-elapsed: 2.460548ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0519 13:25:02.421222 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:02.421531 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:02.422602 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:02.422668 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:02.423757 1 trace.go:236] Trace[1622201432]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:fbc426e2-9fc3-4aa6-8d8a-52d7d082f2e2,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (19-May-2026 13:24:59.422) (total time: 3001ms): Trace[1622201432]: [3.00140848s] [3.00140848s] END E0519 13:25:02.423942 1 timeout.go:142] post-timeout activity - time-elapsed: 2.853474ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0519 13:25:03.082344 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:03.082675 1 trace.go:236] Trace[672123356]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d1ea4b8d-3a5f-4152-9703-519b9c6b801e,client:199.204.45.248,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (19-May-2026 13:24:56.079) (total time: 7003ms): Trace[672123356]: ["Create etcd3" audit-id:d1ea4b8d-3a5f-4152-9703-519b9c6b801e,key:/events/kube-system/kube-apiserver-instance.18b0fa5511269cc8,type:*core.Event,resource:events 7002ms (13:24:56.080)] Trace[672123356]: [7.003246039s] [7.003246039s] END E0519 13:25:03.869168 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:03.869335 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:03.870448 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:03.870540 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0519 13:25:03.871739 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:03.871861 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout I0519 13:25:03.872061 1 trace.go:236] Trace[637391373]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c448ba99-e0db-4bc5-9710-d1b6fc11e317,client:10.0.0.53,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:24:58.871) (total time: 5000ms): Trace[637391373]: [5.000532746s] [5.000532746s] END E0519 13:25:03.872851 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:03.872874 1 timeout.go:142] post-timeout activity - time-elapsed: 3.792358ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0519 13:25:03.873023 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:03.874153 1 trace.go:236] Trace[1520776913]: "Get" accept:application/json, */*,audit-id:54784a82-4e48-45fe-bb39-399574939fe2,client:10.0.0.43,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:25:00.879) (total time: 2994ms): Trace[1520776913]: [2.99474565s] [2.99474565s] END E0519 13:25:03.874234 1 timeout.go:142] post-timeout activity - time-elapsed: 2.365926ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: E0519 13:25:03.878290 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0519 13:25:03.878387 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0519 13:25:03.879405 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0519 13:25:03.879564 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0519 13:25:03.881007 1 trace.go:236] Trace[2130690970]: "Get" accept:application/json, */*,audit-id:fadaa640-fc87-4a24-8ba3-6c17da1d4d48,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (19-May-2026 13:24:53.877) (total time: 10003ms): Trace[2130690970]: [10.00333634s] [10.00333634s] END E0519 13:25:03.881178 1 timeout.go:142] post-timeout activity - time-elapsed: 2.891014ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0519 13:25:05.313405 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:05.313718 1 trace.go:236] Trace[1757559715]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:15d507b6-ca9d-4131-af01-1c992d04986d,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (19-May-2026 13:24:58.309) (total time: 7003ms): Trace[1757559715]: ["GuaranteedUpdate etcd3" audit-id:15d507b6-ca9d-4131-af01-1c992d04986d,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (13:24:58.309) Trace[1757559715]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (13:25:05.313)] Trace[1757559715]: [7.003944601s] [7.003944601s] END E0519 13:25:05.971864 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:05.972212 1 trace.go:236] Trace[1977800053]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:12fbb424-48ae-4906-84f7-9b511f674bbc,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (19-May-2026 13:24:58.967) (total time: 7004ms): Trace[1977800053]: ["GuaranteedUpdate etcd3" audit-id:12fbb424-48ae-4906-84f7-9b511f674bbc,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (13:24:58.967) Trace[1977800053]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (13:25:05.971)] Trace[1977800053]: [7.004413099s] [7.004413099s] END E0519 13:25:06.079170 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:06.079367 1 trace.go:236] Trace[1974133805]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:3ad48a4a-1923-4483-9b8d-7d14fc6ba481,client:199.204.45.248,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/monitoring/persistentvolumeclaims/alertmanager-kube-prometheus-stack-alertmanager-db-alertmanager-kube-prometheus-stack-alertmanager-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (19-May-2026 13:24:52.733) (total time: 13345ms): Trace[1974133805]: [13.345681375s] [13.345681375s] END E0519 13:25:06.079885 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0519 13:25:06.080328 1 trace.go:236] Trace[1136558951]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.248,type:*v1.Endpoints,resource:apiServerIPInfo (19-May-2026 13:24:56.258) (total time: 9821ms): Trace[1136558951]: [9.821816037s] [9.821816037s] END E0519 13:25:06.080361 1 controller.go:159] unable to sync kubernetes service: etcdserver: request timed out I0519 13:25:06.080486 1 trace.go:236] Trace[1438043747]: "Get" accept:application/json, */*,audit-id:0ecf062e-d944-49c6-91a7-ac5c76258655,client:10.0.0.157,protocol:HTTP/2.0,resource:endpoints,scope:resource,url:/api/v1/namespaces/openstack/endpoints/keystone-api,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (19-May-2026 13:24:52.603) (total time: 13477ms): Trace[1438043747]: [13.477355672s] [13.477355672s] END I0519 13:25:06.755889 1 trace.go:236] Trace[873452387]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:2124644c-3575-497c-9902-0515ff73954f,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (19-May-2026 13:25:00.137) (total time: 6618ms): Trace[873452387]: ["GuaranteedUpdate etcd3" audit-id:2124644c-3575-497c-9902-0515ff73954f,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 6618ms (13:25:00.137) Trace[873452387]: ---"Txn call completed" 6616ms (13:25:06.755)] Trace[873452387]: [6.618317409s] [6.618317409s] END I0519 13:25:06.756552 1 trace.go:236] Trace[1993385807]: "Update" accept:application/json, */*,audit-id:cf69af0a-0385-4184-b59b-a76e147085e3,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (19-May-2026 13:25:01.790) (total time: 4966ms): Trace[1993385807]: ["GuaranteedUpdate etcd3" audit-id:cf69af0a-0385-4184-b59b-a76e147085e3,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 4965ms (13:25:01.790) Trace[1993385807]: ---"Txn call completed" 4963ms (13:25:06.755)] Trace[1993385807]: [4.966243724s] [4.966243724s] END I0519 13:25:06.758255 1 trace.go:236] Trace[1873466140]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:bb9b81f4-a8a8-4a14-9638-a53439b9e94f,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (19-May-2026 13:25:01.791) (total time: 4966ms): Trace[1873466140]: ["GuaranteedUpdate etcd3" audit-id:bb9b81f4-a8a8-4a14-9638-a53439b9e94f,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 4966ms (13:25:01.791) Trace[1873466140]: ---"Txn call completed" 4964ms (13:25:06.757)] Trace[1873466140]: [4.966750881s] [4.966750881s] END I0519 13:25:06.760004 1 trace.go:236] Trace[1599944744]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e13a2eee-c2d0-4f48-970a-f1223313583a,client:199.204.45.248,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/ingress-nginx/services/ingress-nginx-controller,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (19-May-2026 13:25:05.158) (total time: 1601ms): Trace[1599944744]: ---"About to write a response" 1601ms (13:25:06.759) Trace[1599944744]: [1.601183346s] [1.601183346s] END I0519 13:25:06.762488 1 trace.go:236] Trace[777267598]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5546ffb6-0adc-4aff-873d-cea66a420471,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (19-May-2026 13:25:05.974) (total time: 787ms): Trace[777267598]: ["GuaranteedUpdate etcd3" audit-id:5546ffb6-0adc-4aff-873d-cea66a420471,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 787ms (13:25:05.975) Trace[777267598]: ---"Txn call completed" 785ms (13:25:06.761)] Trace[777267598]: [787.69541ms] [787.69541ms] END I0519 13:25:06.762883 1 trace.go:236] Trace[98949327]: "Get" accept:application/json, */*,audit-id:e403193a-2d77-42d9-87ca-72d0ef939907,client:10.0.0.17,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (19-May-2026 13:25:05.226) (total time: 1536ms): Trace[98949327]: ---"About to write a response" 1535ms (13:25:06.762) Trace[98949327]: [1.536079713s] [1.536079713s] END I0519 13:25:06.763132 1 trace.go:236] Trace[1250482074]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8e73061a-d90d-4f2b-bbe2-dd86a9928b1a,client:199.204.45.248,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (19-May-2026 13:25:05.316) (total time: 1446ms): Trace[1250482074]: ---"About to write a response" 1445ms (13:25:06.762) Trace[1250482074]: [1.446141812s] [1.446141812s] END I0519 13:25:06.763531 1 trace.go:236] Trace[1715347849]: "Update" accept:application/json, */*,audit-id:451728c4-a865-4ac4-a5f9-889acdd3e5fe,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (19-May-2026 13:25:03.879) (total time: 2883ms): Trace[1715347849]: ["GuaranteedUpdate etcd3" audit-id:451728c4-a865-4ac4-a5f9-889acdd3e5fe,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 2883ms (13:25:03.879) Trace[1715347849]: ---"Txn call completed" 2881ms (13:25:06.761)] Trace[1715347849]: [2.883804041s] [2.883804041s] END I0519 13:25:06.763600 1 trace.go:236] Trace[1744468956]: "Update" accept:application/json, */*,audit-id:7fe00af0-ef35-45e2-8456-7c756a55b92b,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (19-May-2026 13:25:01.789) (total time: 4973ms): Trace[1744468956]: ["GuaranteedUpdate etcd3" audit-id:7fe00af0-ef35-45e2-8456-7c756a55b92b,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 4973ms (13:25:01.790) Trace[1744468956]: ---"Txn call completed" 4970ms (13:25:06.762)] Trace[1744468956]: [4.973809449s] [4.973809449s] END I0519 13:25:06.763784 1 trace.go:236] Trace[2064207173]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4e0195ab-b33e-4bc2-865b-711f7496d5d4,client:199.204.45.248,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (19-May-2026 13:25:03.086) (total time: 3677ms): Trace[2064207173]: ["Create etcd3" audit-id:4e0195ab-b33e-4bc2-865b-711f7496d5d4,key:/events/kube-system/kube-apiserver-instance.18b0fa55e5ce26fe,type:*core.Event,resource:events 3676ms (13:25:03.087) Trace[2064207173]: ---"TransformToStorage succeeded" 3671ms (13:25:06.758)] Trace[2064207173]: [3.677541453s] [3.677541453s] END I0519 13:25:06.764017 1 trace.go:236] Trace[863346999]: "Create" accept:application/json, */*,audit-id:09d27ae4-5b8a-4ede-bac3-00ac8c535f31,client:10.0.0.53,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (19-May-2026 13:25:03.873) (total time: 2890ms): Trace[863346999]: ["Create etcd3" audit-id:09d27ae4-5b8a-4ede-bac3-00ac8c535f31,key:/events/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io.18b0fa56e184cd72,type:*core.Event,resource:events 2889ms (13:25:03.874) Trace[863346999]: ---"TransformToStorage succeeded" 2884ms (13:25:06.758)] Trace[863346999]: [2.890693056s] [2.890693056s] END I0519 13:25:06.765787 1 trace.go:236] Trace[1241663377]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:627c3eab-05a9-44da-8b83-b580a7fa13b8,client:199.204.45.248,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/kube-system/pods/kube-apiserver-instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (19-May-2026 13:25:02.097) (total time: 4668ms): Trace[1241663377]: ---"About to write a response" 4664ms (13:25:06.761) Trace[1241663377]: [4.668301055s] [4.668301055s] END I0519 13:25:06.768980 1 trace.go:236] Trace[120594130]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.248,type:*v1.Endpoints,resource:apiServerIPInfo (19-May-2026 13:25:06.259) (total time: 509ms): Trace[120594130]: ---"initial value restored" 502ms (13:25:06.761) Trace[120594130]: [509.438297ms] [509.438297ms] END I0519 13:26:13.344065 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon" clusterIPs={"IPv4":"10.97.79.105"} I0519 13:26:13.356666 1 alloc.go:330] "allocated clusterIPs" service="openstack/horizon-int" clusterIPs={"IPv4":"10.111.112.190"}