I0629 00:20:28.975813 1 options.go:220] external host was not specified, using 199.204.45.4 I0629 00:20:28.979354 1 server.go:148] Version: v1.28.13 I0629 00:20:28.979443 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" I0629 00:20:29.252486 1 shared_informer.go:311] Waiting for caches to sync for node_authorizer I0629 00:20:29.260195 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. I0629 00:20:29.260219 1 plugins.go:161] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. I0629 00:20:29.260405 1 instance.go:298] Using reconciler: lease I0629 00:20:29.449797 1 handler.go:275] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager W0629 00:20:29.449821 1 genericapiserver.go:744] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. I0629 00:20:29.726270 1 handler.go:275] Adding GroupVersion v1 to ResourceManager I0629 00:20:29.726482 1 instance.go:709] API group "internal.apiserver.k8s.io" is not enabled, skipping. I0629 00:20:30.178223 1 instance.go:709] API group "resource.k8s.io" is not enabled, skipping. I0629 00:20:30.189464 1 handler.go:275] Adding GroupVersion authentication.k8s.io v1 to ResourceManager W0629 00:20:30.189510 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.189518 1 genericapiserver.go:744] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.189922 1 handler.go:275] Adding GroupVersion authorization.k8s.io v1 to ResourceManager W0629 00:20:30.189936 1 genericapiserver.go:744] Skipping API authorization.k8s.io/v1beta1 because it has no resources. I0629 00:20:30.190740 1 handler.go:275] Adding GroupVersion autoscaling v2 to ResourceManager I0629 00:20:30.191529 1 handler.go:275] Adding GroupVersion autoscaling v1 to ResourceManager W0629 00:20:30.191545 1 genericapiserver.go:744] Skipping API autoscaling/v2beta1 because it has no resources. W0629 00:20:30.191550 1 genericapiserver.go:744] Skipping API autoscaling/v2beta2 because it has no resources. I0629 00:20:30.192926 1 handler.go:275] Adding GroupVersion batch v1 to ResourceManager W0629 00:20:30.192941 1 genericapiserver.go:744] Skipping API batch/v1beta1 because it has no resources. I0629 00:20:30.194655 1 handler.go:275] Adding GroupVersion certificates.k8s.io v1 to ResourceManager W0629 00:20:30.194717 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.194732 1 genericapiserver.go:744] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.195753 1 handler.go:275] Adding GroupVersion coordination.k8s.io v1 to ResourceManager W0629 00:20:30.195778 1 genericapiserver.go:744] Skipping API coordination.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.195874 1 genericapiserver.go:744] Skipping API discovery.k8s.io/v1beta1 because it has no resources. I0629 00:20:30.196931 1 handler.go:275] Adding GroupVersion discovery.k8s.io v1 to ResourceManager I0629 00:20:30.200053 1 handler.go:275] Adding GroupVersion networking.k8s.io v1 to ResourceManager W0629 00:20:30.200070 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.200076 1 genericapiserver.go:744] Skipping API networking.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.200611 1 handler.go:275] Adding GroupVersion node.k8s.io v1 to ResourceManager W0629 00:20:30.200625 1 genericapiserver.go:744] Skipping API node.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.200631 1 genericapiserver.go:744] Skipping API node.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.201681 1 handler.go:275] Adding GroupVersion policy v1 to ResourceManager W0629 00:20:30.201697 1 genericapiserver.go:744] Skipping API policy/v1beta1 because it has no resources. I0629 00:20:30.203542 1 handler.go:275] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager W0629 00:20:30.203557 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.203563 1 genericapiserver.go:744] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.204081 1 handler.go:275] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager W0629 00:20:30.204100 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.204107 1 genericapiserver.go:744] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.206493 1 handler.go:275] Adding GroupVersion storage.k8s.io v1 to ResourceManager W0629 00:20:30.206510 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.206516 1 genericapiserver.go:744] Skipping API storage.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.207839 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager I0629 00:20:30.208900 1 handler.go:275] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta2 to ResourceManager W0629 00:20:30.208912 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.208915 1 genericapiserver.go:744] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.211706 1 handler.go:275] Adding GroupVersion apps v1 to ResourceManager W0629 00:20:30.211718 1 genericapiserver.go:744] Skipping API apps/v1beta2 because it has no resources. W0629 00:20:30.211721 1 genericapiserver.go:744] Skipping API apps/v1beta1 because it has no resources. I0629 00:20:30.212574 1 handler.go:275] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager W0629 00:20:30.212584 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. W0629 00:20:30.212587 1 genericapiserver.go:744] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. I0629 00:20:30.213024 1 handler.go:275] Adding GroupVersion events.k8s.io v1 to ResourceManager W0629 00:20:30.213036 1 genericapiserver.go:744] Skipping API events.k8s.io/v1beta1 because it has no resources. I0629 00:20:30.247839 1 handler.go:275] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager W0629 00:20:30.247890 1 genericapiserver.go:744] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. I0629 00:20:30.691141 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0629 00:20:30.691202 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/etc/kubernetes/pki/apiserver.crt::/etc/kubernetes/pki/apiserver.key" I0629 00:20:30.691222 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0629 00:20:30.691550 1 secure_serving.go:213] Serving securely on [::]:6443 I0629 00:20:30.691692 1 apf_controller.go:374] Starting API Priority and Fairness config controller I0629 00:20:30.691770 1 system_namespaces_controller.go:67] Starting system namespaces controller I0629 00:20:30.691819 1 apiservice_controller.go:97] Starting APIServiceRegistrationController I0629 00:20:30.691832 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller I0629 00:20:30.691827 1 aggregator.go:164] waiting for initial CRD sync... I0629 00:20:30.691859 1 crdregistration_controller.go:111] Starting crd-autoregister controller I0629 00:20:30.691874 1 shared_informer.go:311] Waiting for caches to sync for crd-autoregister I0629 00:20:30.691884 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0629 00:20:30.691963 1 controller.go:78] Starting OpenAPI AggregationController I0629 00:20:30.692111 1 handler_discovery.go:412] Starting ResourceDiscoveryManager I0629 00:20:30.692177 1 available_controller.go:423] Starting AvailableConditionController I0629 00:20:30.692216 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller I0629 00:20:30.692490 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller I0629 00:20:30.692512 1 shared_informer.go:311] Waiting for caches to sync for cluster_authentication_trust_controller I0629 00:20:30.692564 1 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/etc/kubernetes/pki/front-proxy-client.crt::/etc/kubernetes/pki/front-proxy-client.key" I0629 00:20:30.692685 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" I0629 00:20:30.692714 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/etc/kubernetes/pki/front-proxy-ca.crt" I0629 00:20:30.692730 1 gc_controller.go:78] Starting apiserver lease garbage collector I0629 00:20:30.692730 1 controller.go:80] Starting OpenAPI V3 AggregationController I0629 00:20:30.692195 1 gc_controller.go:78] Starting apiserver lease garbage collector I0629 00:20:30.692695 1 controller.go:116] Starting legacy_token_tracking_controller I0629 00:20:30.693570 1 shared_informer.go:311] Waiting for caches to sync for configmaps I0629 00:20:30.693706 1 customresource_discovery_controller.go:289] Starting DiscoveryController I0629 00:20:30.693745 1 controller.go:134] Starting OpenAPI controller I0629 00:20:30.693774 1 controller.go:85] Starting OpenAPI V3 controller I0629 00:20:30.693797 1 naming_controller.go:291] Starting NamingConditionController I0629 00:20:30.693812 1 establishing_controller.go:76] Starting EstablishingController I0629 00:20:30.693831 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController I0629 00:20:30.694002 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController I0629 00:20:30.694026 1 crd_finalizer.go:266] Starting CRDFinalizer I0629 00:20:30.752617 1 shared_informer.go:318] Caches are synced for node_authorizer E0629 00:20:30.769823 1 controller.go:146] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" I0629 00:20:30.792811 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller I0629 00:20:30.792855 1 apf_controller.go:379] Running API Priority and Fairness config worker I0629 00:20:30.792876 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process I0629 00:20:30.792937 1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller I0629 00:20:30.792950 1 cache.go:39] Caches are synced for AvailableConditionController controller I0629 00:20:30.793061 1 shared_informer.go:318] Caches are synced for crd-autoregister I0629 00:20:30.793139 1 aggregator.go:166] initial CRD sync complete... I0629 00:20:30.793159 1 autoregister_controller.go:141] Starting autoregister controller I0629 00:20:30.793169 1 cache.go:32] Waiting for caches to sync for autoregister controller I0629 00:20:30.793178 1 cache.go:39] Caches are synced for autoregister controller I0629 00:20:30.793641 1 shared_informer.go:318] Caches are synced for configmaps I0629 00:20:30.795246 1 controller.go:624] quota admission added evaluator for: namespaces I0629 00:20:30.974968 1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io I0629 00:20:31.700797 1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000 I0629 00:20:31.709649 1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000 I0629 00:20:31.709681 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist. I0629 00:20:32.365095 1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io I0629 00:20:32.422704 1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io I0629 00:20:32.509789 1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"} W0629 00:20:32.518680 1 lease.go:263] Resetting endpoints for master service "kubernetes" to [199.204.45.4] I0629 00:20:32.520187 1 controller.go:624] quota admission added evaluator for: endpoints I0629 00:20:32.528352 1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io I0629 00:20:33.345376 1 controller.go:624] quota admission added evaluator for: serviceaccounts I0629 00:20:34.251568 1 controller.go:624] quota admission added evaluator for: deployments.apps I0629 00:20:34.284886 1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"} I0629 00:20:34.490224 1 controller.go:624] quota admission added evaluator for: daemonsets.apps I0629 00:20:38.439559 1 controller.go:624] quota admission added evaluator for: replicasets.apps I0629 00:20:38.587704 1 controller.go:624] quota admission added evaluator for: controllerrevisions.apps I0629 00:21:07.444203 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0629 00:21:07.480639 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0629 00:21:07.497671 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0629 00:21:07.526056 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0629 00:21:07.539989 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0629 00:21:07.559504 1 handler.go:275] Adding GroupVersion gateway.networking.x-k8s.io v1alpha1 to ResourceManager I0629 00:21:07.589067 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0629 00:21:07.589139 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0629 00:21:07.609360 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha2 to ResourceManager I0629 00:21:07.609421 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1alpha3 to ResourceManager I0629 00:21:07.647404 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0629 00:21:07.647460 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0629 00:21:07.684433 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0629 00:21:07.684486 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0629 00:21:07.762496 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0629 00:21:08.115379 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.210849 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1 to ResourceManager I0629 00:21:08.210928 1 handler.go:275] Adding GroupVersion gateway.networking.k8s.io v1beta1 to ResourceManager I0629 00:21:08.254788 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.284540 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.338994 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.344373 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.946129 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:08.952147 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:09.093538 1 handler.go:275] Adding GroupVersion gateway.envoyproxy.io v1alpha1 to ResourceManager I0629 00:21:14.940985 1 controller.go:624] quota admission added evaluator for: jobs.batch I0629 00:21:15.732259 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0629 00:21:15.741371 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0629 00:21:15.958249 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0629 00:21:16.136140 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0629 00:21:16.339946 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0629 00:21:16.531067 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0629 00:21:16.945924 1 handler.go:275] Adding GroupVersion cilium.io v2alpha1 to ResourceManager I0629 00:21:17.161454 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0629 00:21:18.553675 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0629 00:21:19.339292 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager I0629 00:21:19.801545 1 handler.go:275] Adding GroupVersion cilium.io v2 to ResourceManager W0629 00:21:26.167463 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found E0629 00:21:26.167522 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": service "envoy-gateway" not found I0629 00:21:26.713384 1 controller.go:624] quota admission added evaluator for: ciliumendpoints.cilium.io I0629 00:21:33.687090 1 alloc.go:330] "allocated clusterIPs" service="envoy-gateway-system/envoy-gateway" clusterIPs={"IPv4":"10.101.98.44"} W0629 00:21:33.715331 1 dispatcher.go:210] Failed calling webhook, failing open topology.webhook.gateway.envoyproxy.io: failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.101.98.44:9443: connect: connection refused E0629 00:21:33.715374 1 dispatcher.go:214] failed calling webhook "topology.webhook.gateway.envoyproxy.io": failed to call webhook: Post "https://envoy-gateway.envoy-gateway-system.svc:9443/inject-pod-topology?timeout=10s": dial tcp 10.101.98.44:9443: connect: connection refused I0629 00:21:47.385227 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0629 00:21:47.395082 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0629 00:21:47.413365 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0629 00:21:47.437053 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager" clusterIPs={"IPv4":"10.102.38.227"} I0629 00:21:47.445915 1 alloc.go:330] "allocated clusterIPs" service="cert-manager/cert-manager-webhook" clusterIPs={"IPv4":"10.99.59.152"} I0629 00:21:47.488586 1 handler.go:275] Adding GroupVersion acme.cert-manager.io v1 to ResourceManager I0629 00:21:47.568507 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager I0629 00:21:47.615606 1 handler.go:275] Adding GroupVersion cert-manager.io v1 to ResourceManager W0629 00:21:53.026564 1 dispatcher.go:217] Failed calling webhook, failing closed webhook.cert-manager.io: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": dial tcp 10.99.59.152:443: connect: connection refused I0629 00:21:58.025425 1 controller.go:624] quota admission added evaluator for: certificates.cert-manager.io I0629 00:22:03.836325 1 controller.go:624] quota admission added evaluator for: certificaterequests.cert-manager.io I0629 00:22:19.004030 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-admission" clusterIPs={"IPv4":"10.100.194.246"} I0629 00:22:19.013869 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-defaultbackend" clusterIPs={"IPv4":"10.105.171.68"} I0629 00:22:19.023804 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller-metrics" clusterIPs={"IPv4":"10.103.78.84"} I0629 00:22:19.033991 1 alloc.go:330] "allocated clusterIPs" service="ingress-nginx/ingress-nginx-controller" clusterIPs={"IPv4":"10.110.105.67"} I0629 00:22:25.588024 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.602864 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.620266 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.642137 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.665965 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.688979 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.708606 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.729811 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.753734 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.769701 1 handler.go:275] Adding GroupVersion rabbitmq.com v1alpha1 to ResourceManager I0629 00:22:25.789900 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.811537 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:25.829716 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:27.209921 1 handler.go:275] Adding GroupVersion rabbitmq.com v1beta1 to ResourceManager I0629 00:22:31.291031 1 controller.go:624] quota admission added evaluator for: networkpolicies.networking.k8s.io I0629 00:22:31.299320 1 controller.go:624] quota admission added evaluator for: poddisruptionbudgets.policy I0629 00:22:31.369523 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-messaging-topology-operator-webhook" clusterIPs={"IPv4":"10.105.51.79"} I0629 00:22:31.395554 1 controller.go:624] quota admission added evaluator for: issuers.cert-manager.io I0629 00:22:33.815960 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0629 00:22:33.858372 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0629 00:22:34.537846 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-10-0 to ResourceManager I0629 00:22:34.537953 1 handler.go:275] Adding GroupVersion pxc.percona.com v1-11-0 to ResourceManager I0629 00:22:34.538040 1 handler.go:275] Adding GroupVersion pxc.percona.com v1 to ResourceManager I0629 00:22:41.239664 1 controller.go:624] quota admission added evaluator for: perconaxtradbclusters.pxc.percona.com I0629 00:22:45.549750 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-cluster-operator" clusterIPs={"IPv4":"10.100.2.144"} I0629 00:22:45.826414 1 controller.go:624] quota admission added evaluator for: statefulsets.apps I0629 00:22:45.989067 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy" clusterIPs={"IPv4":"10.99.66.199"} I0629 00:22:46.004749 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-replicas" clusterIPs={"IPv4":"10.106.133.79"} I0629 00:24:21.337044 1 alloc.go:330] "allocated clusterIPs" service="openstack/percona-xtradb-haproxy-metrics" clusterIPs={"IPv4":"10.98.74.186"} I0629 00:24:24.008741 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey" clusterIPs={"IPv4":"10.107.136.71"} I0629 00:24:24.018643 1 alloc.go:330] "allocated clusterIPs" service="openstack/valkey-metrics" clusterIPs={"IPv4":"10.101.138.81"} I0629 00:24:31.059523 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak" clusterIPs={"IPv4":"10.97.70.83"} I0629 00:24:31.067706 1 alloc.go:330] "allocated clusterIPs" service="auth-system/keycloak-metrics" clusterIPs={"IPv4":"10.106.103.22"} I0629 00:27:00.982206 1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io I0629 00:27:04.843214 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0629 00:27:04.875150 1 handler.go:275] Adding GroupVersion nfd.k8s-sigs.io v1alpha1 to ResourceManager I0629 00:27:08.682935 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0629 00:27:08.692342 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0629 00:27:08.705937 1 handler.go:275] Adding GroupVersion secretgen.carvel.dev v1alpha1 to ResourceManager I0629 00:27:08.715563 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0629 00:27:08.724891 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0629 00:27:08.736258 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0629 00:27:08.744816 1 handler.go:275] Adding GroupVersion secretgen.k14s.io v1alpha1 to ResourceManager I0629 00:27:11.287997 1 controller.go:624] quota admission added evaluator for: nodefeatures.nfd.k8s-sigs.io I0629 00:27:20.855483 1 controller.go:624] quota admission added evaluator for: passwords.secretgen.k14s.io I0629 00:28:00.045881 1 controller.go:624] quota admission added evaluator for: secrettemplates.secretgen.carvel.dev I0629 00:28:14.532461 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0629 00:28:14.729267 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:14.794180 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:15.006473 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0629 00:28:15.316877 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:15.317851 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:15.422624 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1alpha1 to ResourceManager I0629 00:28:15.457145 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:15.555886 1 handler.go:275] Adding GroupVersion monitoring.coreos.com v1 to ResourceManager I0629 00:28:55.586685 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus" clusterIPs={"IPv4":"10.109.233.238"} I0629 00:28:55.593393 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-kube-state-metrics" clusterIPs={"IPv4":"10.101.72.190"} I0629 00:28:55.603697 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-grafana" clusterIPs={"IPv4":"10.96.180.243"} I0629 00:28:55.611872 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-alertmanager" clusterIPs={"IPv4":"10.96.27.75"} I0629 00:28:55.618898 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-operator" clusterIPs={"IPv4":"10.103.194.119"} I0629 00:28:55.625662 1 alloc.go:330] "allocated clusterIPs" service="monitoring/kube-prometheus-stack-prometheus-node-exporter" clusterIPs={"IPv4":"10.100.242.182"} I0629 00:28:55.684945 1 controller.go:624] quota admission added evaluator for: alertmanagers.monitoring.coreos.com I0629 00:28:55.699292 1 controller.go:624] quota admission added evaluator for: prometheusrules.monitoring.coreos.com I0629 00:28:55.745404 1 controller.go:624] quota admission added evaluator for: podmonitors.monitoring.coreos.com I0629 00:28:55.764976 1 controller.go:624] quota admission added evaluator for: servicemonitors.monitoring.coreos.com I0629 00:28:55.790677 1 controller.go:624] quota admission added evaluator for: prometheuses.monitoring.coreos.com W0629 00:28:55.798804 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.798855 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.799593 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.799663 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.799632 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.799759 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.799950 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.800000 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.800193 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.800210 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.800236 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.800255 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.800365 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.800433 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.801096 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.801134 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.801279 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.801316 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.801542 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.801571 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.801780 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.801803 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.801866 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.801899 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.802217 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.802246 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.803486 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.803509 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.803538 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused W0629 00:28:55.803563 1 dispatcher.go:210] Failed calling webhook, failing open prometheusrulemutate.monitoring.coreos.com: failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.803622 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused E0629 00:28:55.803516 1 dispatcher.go:214] failed calling webhook "prometheusrulemutate.monitoring.coreos.com": failed to call webhook: Post "https://kube-prometheus-stack-operator.monitoring.svc:443/admission-prometheusrules/mutate?timeout=10s": dial tcp 10.103.194.119:443: connect: connection refused I0629 00:29:07.013628 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki-gateway" clusterIPs={"IPv4":"10.102.149.151"} I0629 00:29:07.024511 1 alloc.go:330] "allocated clusterIPs" service="monitoring/loki" clusterIPs={"IPv4":"10.104.243.122"} I0629 00:29:11.556108 1 alloc.go:330] "allocated clusterIPs" service="monitoring/goldpinger" clusterIPs={"IPv4":"10.109.151.4"} I0629 00:29:14.489965 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-smartctl-exporter" clusterIPs={"IPv4":"10.100.162.80"} I0629 00:29:16.494102 1 alloc.go:330] "allocated clusterIPs" service="monitoring/prometheus-pushgateway" clusterIPs={"IPv4":"10.111.51.37"} I0629 00:29:21.558115 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached" clusterIPs={"IPv4":"10.105.179.64"} I0629 00:29:22.623982 1 alloc.go:330] "allocated clusterIPs" service="openstack/memcached-metrics" clusterIPs={"IPv4":"10.102.131.183"} I0629 00:29:29.940269 1 controller.go:624] quota admission added evaluator for: rabbitmqclusters.rabbitmq.com I0629 00:29:30.102990 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-keystone" clusterIPs={"IPv4":"10.109.4.192"} I0629 00:30:11.120509 1 alloc.go:330] "allocated clusterIPs" service="openstack/keystone-api" clusterIPs={"IPv4":"10.110.144.215"} I0629 00:30:11.140892 1 controller.go:624] quota admission added evaluator for: cronjobs.batch I0629 00:33:16.694701 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-barbican" clusterIPs={"IPv4":"10.111.81.24"} I0629 00:33:50.151420 1 alloc.go:330] "allocated clusterIPs" service="openstack/barbican-api" clusterIPs={"IPv4":"10.109.106.10"} I0629 00:35:45.136916 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0629 00:35:45.147934 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.164500 1 handler.go:275] Adding GroupVersion objectbucket.io v1alpha1 to ResourceManager I0629 00:35:45.164671 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.182876 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.195905 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.202460 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.211534 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.220012 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.239443 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.245323 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.294705 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.308379 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.330915 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.348745 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.384832 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.419802 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.457084 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:35:45.545871 1 handler.go:275] Adding GroupVersion ceph.rook.io v1 to ResourceManager I0629 00:36:26.110327 1 controller.go:624] quota admission added evaluator for: cephclusters.ceph.rook.io I0629 00:36:26.118284 1 controller.go:624] quota admission added evaluator for: cephobjectstores.ceph.rook.io I0629 00:36:59.838487 1 alloc.go:330] "allocated clusterIPs" service="openstack/rook-ceph-rgw-ceph" clusterIPs={"IPv4":"10.109.85.85"} I0629 00:37:01.733390 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-glance" clusterIPs={"IPv4":"10.98.206.59"} I0629 00:37:35.863919 1 alloc.go:330] "allocated clusterIPs" service="openstack/glance-api" clusterIPs={"IPv4":"10.110.147.130"} I0629 00:42:08.374197 1 alloc.go:330] "allocated clusterIPs" service="openstack/staffeln-api" clusterIPs={"IPv4":"10.104.158.34"} I0629 00:42:30.555079 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-cinder" clusterIPs={"IPv4":"10.102.187.151"} E0629 00:42:57.087388 1 upgradeaware.go:425] Error proxying data from client to backend: write tcp 199.204.45.4:53414->199.204.45.4:10250: write: broken pipe I0629 00:43:05.002027 1 alloc.go:330] "allocated clusterIPs" service="openstack/cinder-api" clusterIPs={"IPv4":"10.97.61.144"} I0629 00:47:56.063619 1 alloc.go:330] "allocated clusterIPs" service="openstack/placement-api" clusterIPs={"IPv4":"10.104.197.87"} I0629 00:48:38.199900 1 trace.go:236] Trace[204562672]: "Update" accept:application/json, */*,audit-id:6040908e-28cb-480a-830c-68517fe41f3e,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:37.279) (total time: 919ms): Trace[204562672]: ["GuaranteedUpdate etcd3" audit-id:6040908e-28cb-480a-830c-68517fe41f3e,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 919ms (00:48:37.280) Trace[204562672]: ---"Txn call completed" 918ms (00:48:38.199)] Trace[204562672]: [919.992208ms] [919.992208ms] END I0629 00:48:38.205417 1 trace.go:236] Trace[1541172852]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d5b8953f-961f-417e-a9b3-ebc5835f36d7,client:10.0.0.224,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:37.573) (total time: 632ms): Trace[1541172852]: ["GuaranteedUpdate etcd3" audit-id:d5b8953f-961f-417e-a9b3-ebc5835f36d7,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 631ms (00:48:37.573) Trace[1541172852]: ---"Txn call completed" 631ms (00:48:38.205)] Trace[1541172852]: [632.06215ms] [632.06215ms] END I0629 00:48:38.205418 1 trace.go:236] Trace[28144152]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:ec6f92ce-6e8f-4dde-bbbb-402800e5791a,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:48:37.376) (total time: 828ms): Trace[28144152]: ["GuaranteedUpdate etcd3" audit-id:ec6f92ce-6e8f-4dde-bbbb-402800e5791a,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 828ms (00:48:37.376) Trace[28144152]: ---"Txn call completed" 827ms (00:48:38.205)] Trace[28144152]: [828.85419ms] [828.85419ms] END I0629 00:48:38.205639 1 trace.go:236] Trace[1648108486]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c9ff433a-338f-4a0e-98a6-5488329ba01c,client:10.0.0.197,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:37.476) (total time: 728ms): Trace[1648108486]: ["GuaranteedUpdate etcd3" audit-id:c9ff433a-338f-4a0e-98a6-5488329ba01c,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 728ms (00:48:37.477) Trace[1648108486]: ---"Txn call completed" 727ms (00:48:38.205)] Trace[1648108486]: [728.652781ms] [728.652781ms] END I0629 00:48:38.208738 1 trace.go:236] Trace[251520000]: "Get" accept:application/json, */*,audit-id:24ec5292-b06a-4da9-a380-83d1b65ee81b,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:37.691) (total time: 517ms): Trace[251520000]: ---"About to write a response" 517ms (00:48:38.208) Trace[251520000]: [517.335773ms] [517.335773ms] END I0629 00:48:43.511284 1 trace.go:236] Trace[582942811]: "Get" accept:application/json, */*,audit-id:a3f70156-b8db-4586-99d1-faf26d9cdae2,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-endpoints,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:39.037) (total time: 4473ms): Trace[582942811]: [4.473847004s] [4.473847004s] END I0629 00:48:43.513362 1 trace.go:236] Trace[731240057]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:cc7e8978-b9be-4080-b9b9-07920b66b18d,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (29-Jun-2026 00:48:39.037) (total time: 4475ms): Trace[731240057]: ["GuaranteedUpdate etcd3" audit-id:cc7e8978-b9be-4080-b9b9-07920b66b18d,key:/leases/kube-system/kube-controller-manager,type:*coordination.Lease,resource:leases.coordination.k8s.io 4475ms (00:48:39.038) Trace[731240057]: ---"Txn call completed" 4473ms (00:48:43.513)] Trace[731240057]: [4.475444276s] [4.475444276s] END I0629 00:48:43.513591 1 trace.go:236] Trace[1379970917]: "Get" accept:application/json, */*,audit-id:f8c67845-a68b-461f-bf6c-2da9758321b4,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-user,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:39.037) (total time: 4475ms): Trace[1379970917]: [4.475807263s] [4.475807263s] END I0629 00:48:43.514113 1 trace.go:236] Trace[1863761081]: "Get" accept:application/json, */*,audit-id:e7e2019b-2f4a-4af2-8601-cee949d3fa1a,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-service,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:39.038) (total time: 4475ms): Trace[1863761081]: ---"About to write a response" 4474ms (00:48:43.513) Trace[1863761081]: [4.475082978s] [4.475082978s] END I0629 00:48:43.583392 1 trace.go:236] Trace[998455489]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:3774de36-c7ee-44a6-ae32-573de3fe328f,client:10.0.0.197,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:40.207) (total time: 3375ms): Trace[998455489]: ["GuaranteedUpdate etcd3" audit-id:3774de36-c7ee-44a6-ae32-573de3fe328f,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 3375ms (00:48:40.207) Trace[998455489]: ---"Txn call completed" 3374ms (00:48:43.583)] Trace[998455489]: [3.375712836s] [3.375712836s] END I0629 00:48:43.609641 1 trace.go:236] Trace[654526104]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:af0a523a-653e-4ea8-a877-b21050b337c3,client:10.0.0.224,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:40.208) (total time: 3401ms): Trace[654526104]: ["GuaranteedUpdate etcd3" audit-id:af0a523a-653e-4ea8-a877-b21050b337c3,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 3401ms (00:48:40.208) Trace[654526104]: ---"Txn call completed" 3400ms (00:48:43.609)] Trace[654526104]: [3.401282479s] [3.401282479s] END I0629 00:48:43.609780 1 trace.go:236] Trace[1127988]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d0adb621-b4a6-4ae2-ab7d-851a370148dd,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (29-Jun-2026 00:48:42.898) (total time: 710ms): Trace[1127988]: ["GuaranteedUpdate etcd3" audit-id:d0adb621-b4a6-4ae2-ab7d-851a370148dd,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 710ms (00:48:42.898) Trace[1127988]: ---"Txn call completed" 709ms (00:48:43.609)] Trace[1127988]: [710.990761ms] [710.990761ms] END I0629 00:48:43.609985 1 trace.go:236] Trace[418792498]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4eec5737-5ebf-4040-935c-6889fb4176a4,client:10.0.0.182,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:40.210) (total time: 3399ms): Trace[418792498]: ["GuaranteedUpdate etcd3" audit-id:4eec5737-5ebf-4040-935c-6889fb4176a4,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 3399ms (00:48:40.210) Trace[418792498]: ---"Txn call completed" 3398ms (00:48:43.609)] Trace[418792498]: [3.399353169s] [3.399353169s] END I0629 00:48:43.610187 1 trace.go:236] Trace[749083572]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:018fab57-b7da-42a8-9e83-983585c1988c,client:10.0.0.244,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:48:40.209) (total time: 3400ms): Trace[749083572]: ["GuaranteedUpdate etcd3" audit-id:018fab57-b7da-42a8-9e83-983585c1988c,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 3400ms (00:48:40.209) Trace[749083572]: ---"Txn call completed" 3399ms (00:48:43.610)] Trace[749083572]: [3.400338058s] [3.400338058s] END I0629 00:48:43.610796 1 trace.go:236] Trace[1127959205]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:cf38bf79-7394-4935-8e20-f001af8c4b55,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:48:41.037) (total time: 2573ms): Trace[1127959205]: ---"About to write a response" 2573ms (00:48:43.610) Trace[1127959205]: [2.573575618s] [2.573575618s] END I0629 00:48:43.611705 1 trace.go:236] Trace[191205505]: "Get" accept:application/json, */*,audit-id:025955ee-3ba2-440a-acce-316bd6fd1a4a,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:48:40.202) (total time: 3408ms): Trace[191205505]: ---"About to write a response" 3408ms (00:48:43.611) Trace[191205505]: [3.408914603s] [3.408914603s] END I0629 00:48:43.612044 1 trace.go:236] Trace[1284916239]: "Get" accept:application/json, */*,audit-id:363a1c82-852c-4925-8182-05013db25ccd,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:40.496) (total time: 3115ms): Trace[1284916239]: ---"About to write a response" 3115ms (00:48:43.611) Trace[1284916239]: [3.115565423s] [3.115565423s] END I0629 00:48:43.612125 1 trace.go:236] Trace[133507095]: "Get" accept:application/json, */*,audit-id:2325dd2a-51ab-4458-8837-4501ffbde7bd,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:40.215) (total time: 3396ms): Trace[133507095]: ---"About to write a response" 3396ms (00:48:43.612) Trace[133507095]: [3.3965223s] [3.3965223s] END I0629 00:48:43.614557 1 trace.go:236] Trace[722204938]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e5a7d3a4-e018-4745-9513-8ec533cc58e9,client:10.0.0.224,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:40.193) (total time: 3421ms): Trace[722204938]: ---"About to write a response" 3418ms (00:48:43.612) Trace[722204938]: [3.421197193s] [3.421197193s] END I0629 00:48:43.614592 1 trace.go:236] Trace[1809375158]: "Get" accept:application/json, */*,audit-id:8cc8fa1e-0c76-4322-a7e6-0ca037f4ad80,client:199.204.45.4,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-dbdg9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:48:41.046) (total time: 2567ms): Trace[1809375158]: ---"About to write a response" 2567ms (00:48:43.613) Trace[1809375158]: [2.567918621s] [2.567918621s] END I0629 00:48:43.872941 1 trace.go:236] Trace[218924980]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.4,type:*v1.Endpoints,resource:apiServerIPInfo (29-Jun-2026 00:48:42.854) (total time: 1018ms): Trace[218924980]: ---"initial value restored" 755ms (00:48:43.610) Trace[218924980]: ---"Transaction prepared" 141ms (00:48:43.751) Trace[218924980]: ---"Txn call completed" 120ms (00:48:43.872) Trace[218924980]: [1.018298574s] [1.018298574s] END E0629 00:49:14.708312 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 35.54µs, panicked: false, err: context deadline exceeded, panic-reason: E0629 00:49:14.708472 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:14.709706 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:14.709786 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:14.711015 1 trace.go:236] Trace[461014178]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:95d29571-6f3e-4163-91d8-7482be93f3d5,client:10.0.0.224,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:09.707) (total time: 5003ms): Trace[461014178]: ["GuaranteedUpdate etcd3" audit-id:95d29571-6f3e-4163-91d8-7482be93f3d5,key:/leases/openstack/08db1feb.percona.com,type:*coordination.Lease,resource:leases.coordination.k8s.io 5003ms (00:49:09.707) Trace[461014178]: ---"Txn call failed" err:context deadline exceeded 4999ms (00:49:14.707)] Trace[461014178]: [5.003524448s] [5.003524448s] END E0629 00:49:14.711246 1 timeout.go:142] post-timeout activity - time-elapsed: 3.116054ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0629 00:49:14.711309 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:14.711416 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 15.37µs, panicked: false, err: context deadline exceeded, panic-reason: E0629 00:49:14.712920 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:14.712991 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:14.714171 1 trace.go:236] Trace[670661463]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:efadc698-4c7b-4ba4-bcce-f2e09a5501ac,client:10.0.0.182,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:09.711) (total time: 5002ms): Trace[670661463]: ["GuaranteedUpdate etcd3" audit-id:efadc698-4c7b-4ba4-bcce-f2e09a5501ac,key:/leases/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (00:49:09.711) Trace[670661463]: ---"Txn call failed" err:context deadline exceeded 4998ms (00:49:14.711)] Trace[670661463]: [5.002587349s] [5.002587349s] END E0629 00:49:14.714448 1 timeout.go:142] post-timeout activity - time-elapsed: 3.189785ms, PUT "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0629 00:49:14.968370 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0629 00:49:14.968498 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:14.969866 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:14.969983 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0629 00:49:14.970908 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled I0629 00:49:14.971056 1 trace.go:236] Trace[152700815]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:5a0a3435-bcdb-4b99-8831-f9c670877a78,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:09.968) (total time: 5002ms): Trace[152700815]: [5.002498178s] [5.002498178s] END E0629 00:49:14.971335 1 timeout.go:142] post-timeout activity - time-elapsed: 2.750757ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0629 00:49:14.971345 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:14.973021 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:14.973096 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:14.974262 1 trace.go:236] Trace[879973325]: "Get" accept:application/json, */*,audit-id:cd7b7f80-a8ab-4765-95d1-17ab1f20645e,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:09.970) (total time: 5003ms): Trace[879973325]: [5.003338304s] [5.003338304s] END E0629 00:49:14.974388 1 timeout.go:142] post-timeout activity - time-elapsed: 3.561183ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: I0629 00:49:16.131116 1 metrics.go:285] "Failed to get storage metrics" storage_cluster_id="etcd-0" err="context deadline exceeded" E0629 00:49:16.650217 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:16.650326 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:16.651486 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:16.651608 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:16.653698 1 trace.go:236] Trace[198145581]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:66abd532-16b6-4c63-ab56-daba37366b65,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:11.778) (total time: 4875ms): Trace[198145581]: [4.875425537s] [4.875425537s] END E0629 00:49:16.653948 1 timeout.go:142] post-timeout activity - time-elapsed: 3.631194ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0629 00:49:16.684036 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:16.684075 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:16.684107 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 11.76µs, panicked: false, err: context canceled, panic-reason: E0629 00:49:16.685231 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:16.685348 1 trace.go:236] Trace[375275221]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c402fb85-d779-4542-9088-4873a96d0935,client:10.0.0.197,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:11.683) (total time: 5001ms): Trace[375275221]: ["GuaranteedUpdate etcd3" audit-id:c402fb85-d779-4542-9088-4873a96d0935,key:/leases/openstack/messaging-topology-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (00:49:11.683) Trace[375275221]: ---"Txn call failed" err:context canceled 4999ms (00:49:16.684)] Trace[375275221]: [5.00162685s] [5.00162685s] END E0629 00:49:16.685526 1 timeout.go:142] post-timeout activity - time-elapsed: 1.518121ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: E0629 00:49:16.711021 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:16.711054 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:16.711237 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 160.973µs, panicked: false, err: context deadline exceeded, panic-reason: E0629 00:49:16.713569 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:16.713695 1 trace.go:236] Trace[205060855]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8567ca85-8ae8-4693-b75b-459cf2f85d31,client:10.0.0.244,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:11.711) (total time: 5002ms): Trace[205060855]: ["GuaranteedUpdate etcd3" audit-id:8567ca85-8ae8-4693-b75b-459cf2f85d31,key:/leases/openstack/rabbitmq-cluster-operator-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (00:49:11.711) Trace[205060855]: ---"Txn call failed" err:context deadline exceeded 4999ms (00:49:16.711)] Trace[205060855]: [5.002455297s] [5.002455297s] END E0629 00:49:16.713890 1 timeout.go:142] post-timeout activity - time-elapsed: 2.900019ms, PUT "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0629 00:49:19.252865 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:19.253148 1 trace.go:236] Trace[964167319]: "Create" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:f45d8a24-1d9c-43dc-ab6c-5296fc42aca4,client:199.204.45.4,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:POST (29-Jun-2026 00:49:12.248) (total time: 7004ms): Trace[964167319]: ["Create etcd3" audit-id:f45d8a24-1d9c-43dc-ab6c-5296fc42aca4,key:/events/kube-system/kube-apiserver-instance.18bd66e2e7d86526,type:*core.Event,resource:events 7003ms (00:49:12.249) Trace[964167319]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:19.252)] Trace[964167319]: [7.0041986s] [7.0041986s] END E0629 00:49:19.706821 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:19.706986 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.708241 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.708344 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:19.709539 1 trace.go:236] Trace[908639374]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:4d82365c-8c61-441a-a2fb-4802850ec9b1,client:10.0.0.224,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:14.804) (total time: 4904ms): Trace[908639374]: [4.904788637s] [4.904788637s] END E0629 00:49:19.709650 1 timeout.go:142] post-timeout activity - time-elapsed: 2.837248ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0629 00:49:19.711515 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:19.711588 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.712662 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.712786 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:19.714361 1 trace.go:236] Trace[537161383]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:aab4f65b-38de-43e2-ba03-78dc9abd8dfb,client:10.0.0.182,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:14.712) (total time: 5001ms): Trace[537161383]: [5.001481047s] [5.001481047s] END E0629 00:49:19.714524 1 timeout.go:142] post-timeout activity - time-elapsed: 2.869698ms, GET "/apis/coordination.k8s.io/v1/namespaces/envoy-gateway-system/leases/5b9825d2.gateway.envoyproxy.io" result: E0629 00:49:19.731890 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.731951 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:19.731963 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.732027 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 202.674µs, panicked: false, err: context canceled, panic-reason: E0629 00:49:19.733720 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.733752 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.733757 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0629 00:49:19.735040 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0629 00:49:19.735125 1 timeout.go:142] post-timeout activity - time-elapsed: 3.361099ms, POST "/api/v1/namespaces/openstack/events" result: I0629 00:49:19.735139 1 trace.go:236] Trace[1108395334]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:0d32b991-2f4a-4d83-8ad8-101baff7d2ac,client:10.0.0.224,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:14.804) (total time: 4930ms): Trace[1108395334]: [4.930659273s] [4.930659273s] END E0629 00:49:19.735377 1 timeout.go:142] post-timeout activity - time-elapsed: 3.520712ms, GET "/api/v1/namespaces/openstack/pods/percona-xtradb-pxc-0" result: E0629 00:49:19.968683 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:19.968949 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.969957 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.970147 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:19.971235 1 trace.go:236] Trace[1790154620]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:cdbe1061-26de-40ce-b84b-7fc4c2415029,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:16.970) (total time: 3000ms): Trace[1790154620]: [3.000482499s] [3.000482499s] END E0629 00:49:19.971301 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:19.971546 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:19.971561 1 timeout.go:142] post-timeout activity - time-elapsed: 2.802357ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0629 00:49:19.972533 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:19.972690 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:19.973713 1 trace.go:236] Trace[1000303207]: "Get" accept:application/json, */*,audit-id:18495f1e-10d7-4197-a477-b7999b0bcc66,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:16.972) (total time: 3000ms): Trace[1000303207]: [3.000733083s] [3.000733083s] END E0629 00:49:19.973884 1 timeout.go:142] post-timeout activity - time-elapsed: 2.411399ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0629 00:49:20.006836 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:20.006934 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:20.006982 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:20.007920 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:20.008301 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:20.008361 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout E0629 00:49:20.009727 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout I0629 00:49:20.009797 1 trace.go:236] Trace[861685224]: "Get" accept:application/json, */*,audit-id:93835eb5-0bba-4860-8451-117861cfd839,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:10.006) (total time: 10003ms): Trace[861685224]: [10.003582575s] [10.003582575s] END E0629 00:49:20.009900 1 timeout.go:142] post-timeout activity - time-elapsed: 3.137044ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock" result: E0629 00:49:20.009799 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:20.011728 1 trace.go:236] Trace[1379381569]: "Get" accept:application/json, */*,audit-id:9757be04-e152-45d2-b4f0-d2d3a241712a,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:10.005) (total time: 10005ms): Trace[1379381569]: [10.005798892s] [10.005798892s] END E0629 00:49:20.011912 1 timeout.go:142] post-timeout activity - time-elapsed: 5.174725ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock" result: E0629 00:49:21.649908 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:21.650110 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:21.651896 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:21.652007 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:21.653042 1 trace.go:236] Trace[1072768810]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:3111eea5-cfea-4a09-8edd-eed2ca4f32e3,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:18.652) (total time: 3000ms): Trace[1072768810]: [3.000551159s] [3.000551159s] END E0629 00:49:21.653262 1 timeout.go:142] post-timeout activity - time-elapsed: 3.140524ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" result: E0629 00:49:21.683006 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:21.683168 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:21.684427 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:21.684516 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:21.685603 1 trace.go:236] Trace[1982219947]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:99c7a2b0-39fa-4b02-b5ff-d482981cd181,client:10.0.0.197,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:16.685) (total time: 4999ms): Trace[1982219947]: [4.999762161s] [4.999762161s] END E0629 00:49:21.685803 1 timeout.go:142] post-timeout activity - time-elapsed: 2.497761ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election" result: E0629 00:49:21.711595 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:21.711833 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:21.712737 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:21.713978 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:21.714148 1 trace.go:236] Trace[1245049471]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:a1836f16-dd31-467b-9938-5cb2bde5920b,client:10.0.0.244,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:16.715) (total time: 4998ms): Trace[1245049471]: [4.998464516s] [4.998464516s] END E0629 00:49:21.714445 1 timeout.go:142] post-timeout activity - time-elapsed: 2.560592ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0629 00:49:21.974612 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:21.974839 1 trace.go:236] Trace[1290127567]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9cb44305-4d7b-4931-a4fb-df419b6a21af,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (29-Jun-2026 00:49:13.727) (total time: 8247ms): Trace[1290127567]: ["GuaranteedUpdate etcd3" audit-id:9cb44305-4d7b-4931-a4fb-df419b6a21af,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 8247ms (00:49:13.727) Trace[1290127567]: ---"Txn call failed" err:etcdserver: request timed out 8245ms (00:49:21.974)] Trace[1290127567]: [8.247528599s] [8.247528599s] END E0629 00:49:23.642729 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:23.642890 1 trace.go:236] Trace[889308498]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:ef298441-40fb-4baf-b6fa-0e9b982980dd,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:16.639) (total time: 7003ms): Trace[889308498]: ["GuaranteedUpdate etcd3" audit-id:ef298441-40fb-4baf-b6fa-0e9b982980dd,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (00:49:16.639) Trace[889308498]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:23.642)] Trace[889308498]: [7.003540247s] [7.003540247s] END E0629 00:49:24.969620 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:24.969667 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:24.970209 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 465.569µs, panicked: false, err: context canceled, panic-reason: E0629 00:49:24.970848 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:24.972060 1 trace.go:236] Trace[424068450]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:197d3a4d-f688-432a-8293-ad0c2e98a752,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:PUT (29-Jun-2026 00:49:19.969) (total time: 5002ms): Trace[424068450]: ["GuaranteedUpdate etcd3" audit-id:197d3a4d-f688-432a-8293-ad0c2e98a752,key:/leases/kube-system/kube-scheduler,type:*coordination.Lease,resource:leases.coordination.k8s.io 5001ms (00:49:19.970) Trace[424068450]: ---"Txn call failed" err:context canceled 4998ms (00:49:24.970)] Trace[424068450]: [5.002114061s] [5.002114061s] END E0629 00:49:24.972259 1 timeout.go:142] post-timeout activity - time-elapsed: 2.718015ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0629 00:49:24.972433 1 finisher.go:175] FinishRequest: post-timeout activity - time-elapsed: 21.52µs, panicked: false, err: context deadline exceeded, panic-reason: E0629 00:49:24.972605 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:24.973498 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:24.973809 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:24.974697 1 trace.go:236] Trace[1753645449]: "Update" accept:application/json, */*,audit-id:a6221fe1-b91d-4798-9bfb-6f2f74f5dcd6,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:19.972) (total time: 5002ms): Trace[1753645449]: ["GuaranteedUpdate etcd3" audit-id:a6221fe1-b91d-4798-9bfb-6f2f74f5dcd6,key:/leases/kube-system/cilium-operator-resource-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 5002ms (00:49:19.972) Trace[1753645449]: ---"Txn call failed" err:context deadline exceeded 4997ms (00:49:24.972)] Trace[1753645449]: [5.002296824s] [5.002296824s] END E0629 00:49:24.974914 1 timeout.go:142] post-timeout activity - time-elapsed: 2.668404ms, PUT "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0629 00:49:26.601073 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:26.601304 1 trace.go:236] Trace[75144577]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:1f1aa395-6b82-4f29-be0f-852c1ee581cf,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:19.597) (total time: 7004ms): Trace[75144577]: ["GuaranteedUpdate etcd3" audit-id:1f1aa395-6b82-4f29-be0f-852c1ee581cf,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (00:49:19.597) Trace[75144577]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:26.600)] Trace[75144577]: [7.004095349s] [7.004095349s] END E0629 00:49:26.602279 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0629 00:49:26.734134 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:26.734354 1 trace.go:236] Trace[280846959]: "Create" accept:application/json, */*,audit-id:6ee9a50a-f351-4cc0-ad9e-b9dee4ce4adf,client:10.0.0.182,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events,user-agent:envoy-gateway/v0.0.0 (linux/amd64) kubernetes/$Format,verb:POST (29-Jun-2026 00:49:19.714) (total time: 7020ms): Trace[280846959]: ["Create etcd3" audit-id:6ee9a50a-f351-4cc0-ad9e-b9dee4ce4adf,key:/events/envoy-gateway-system/5b9825d2.gateway.envoyproxy.io.18bd66e4a4c5e40f,type:*core.Event,resource:events 7019ms (00:49:19.715)] Trace[280846959]: [7.020139326s] [7.020139326s] END E0629 00:49:27.012487 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out E0629 00:49:27.012569 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:27.012682 1 trace.go:236] Trace[2028929054]: "Update" accept:application/json, */*,audit-id:a2e2e8b9-687f-4d3b-b72f-98a280b59f88,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (29-Jun-2026 00:49:20.008) (total time: 7003ms): Trace[2028929054]: ["GuaranteedUpdate etcd3" audit-id:a2e2e8b9-687f-4d3b-b72f-98a280b59f88,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (00:49:20.008) Trace[2028929054]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:27.012)] Trace[2028929054]: [7.003972516s] [7.003972516s] END I0629 00:49:27.013855 1 trace.go:236] Trace[26902766]: "Update" accept:application/json, */*,audit-id:b3a578f5-f5b3-4249-87ed-7f199b6bdfe4,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (29-Jun-2026 00:49:20.008) (total time: 7005ms): Trace[26902766]: ["GuaranteedUpdate etcd3" audit-id:b3a578f5-f5b3-4249-87ed-7f199b6bdfe4,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (00:49:20.008) Trace[26902766]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:27.012)] Trace[26902766]: [7.005264483s] [7.005264483s] END E0629 00:49:28.039971 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:28.040230 1 trace.go:236] Trace[801885083]: "Update" accept:application/json, */*,audit-id:7f9536d7-0079-4e55-890a-97053c5521b1,client:10.0.0.34,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:PUT (29-Jun-2026 00:49:21.035) (total time: 7004ms): Trace[801885083]: ["GuaranteedUpdate etcd3" audit-id:7f9536d7-0079-4e55-890a-97053c5521b1,key:/leases/cert-manager/cert-manager-cainjector-leader-election,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (00:49:21.036) Trace[801885083]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:28.039)] Trace[801885083]: [7.004385864s] [7.004385864s] END E0629 00:49:28.649606 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:28.649759 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:28.650848 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:28.650941 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:28.652035 1 trace.go:236] Trace[1006146479]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:c79b52ac-68e3-4aa2-99e5-7f439b1acacd,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (29-Jun-2026 00:49:21.977) (total time: 6674ms): Trace[1006146479]: [6.674210055s] [6.674210055s] END E0629 00:49:28.652203 1 timeout.go:142] post-timeout activity - time-elapsed: 2.625403ms, GET "/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader" result: E0629 00:49:30.231404 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:30.231582 1 trace.go:236] Trace[1929928287]: "Update" accept:application/json, */*,audit-id:f12ea633-5495-4f0b-8510-da5fff9ab786,client:10.0.0.221,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.15.5 (linux/amd64) cert-manager/e8fdbe63fd94107a83d3c9f9c134f2d222ba9cff,verb:PUT (29-Jun-2026 00:49:23.227) (total time: 7004ms): Trace[1929928287]: ["GuaranteedUpdate etcd3" audit-id:f12ea633-5495-4f0b-8510-da5fff9ab786,key:/leases/cert-manager/cert-manager-controller,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (00:49:23.227) Trace[1929928287]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:30.231)] Trace[1929928287]: [7.004353133s] [7.004353133s] END E0629 00:49:30.648367 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:30.648536 1 trace.go:236] Trace[952560004]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:1b0d9b9e-bc02-424f-b1a3-78f93860c1a2,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:23.645) (total time: 7003ms): Trace[952560004]: ["GuaranteedUpdate etcd3" audit-id:1b0d9b9e-bc02-424f-b1a3-78f93860c1a2,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7003ms (00:49:23.645) Trace[952560004]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:30.648)] Trace[952560004]: [7.003447576s] [7.003447576s] END E0629 00:49:33.606486 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:33.606650 1 trace.go:236] Trace[2013915816]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6ffd8818-4a18-43c8-adb6-b432e4ca13db,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:26.603) (total time: 7003ms): Trace[2013915816]: ["GuaranteedUpdate etcd3" audit-id:6ffd8818-4a18-43c8-adb6-b432e4ca13db,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (00:49:26.603) Trace[2013915816]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (00:49:33.606)] Trace[2013915816]: [7.003090408s] [7.003090408s] END E0629 00:49:33.607235 1 controller.go:193] "Failed to update lease" err="etcdserver: request timed out" E0629 00:49:35.653646 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:35.653851 1 trace.go:236] Trace[1803622994]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:61fa3932-f380-4692-9e73-c11c6466c46d,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (29-Jun-2026 00:49:28.650) (total time: 7003ms): Trace[1803622994]: ["GuaranteedUpdate etcd3" audit-id:61fa3932-f380-4692-9e73-c11c6466c46d,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (00:49:28.650) Trace[1803622994]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (00:49:35.653)] Trace[1803622994]: [7.003078228s] [7.003078228s] END E0629 00:49:37.655146 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:37.655324 1 trace.go:236] Trace[310586967]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:f3ae32ba-d5b8-4dce-a1c0-5bfe5da51324,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:30.650) (total time: 7004ms): Trace[310586967]: ["GuaranteedUpdate etcd3" audit-id:f3ae32ba-d5b8-4dce-a1c0-5bfe5da51324,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 7004ms (00:49:30.651) Trace[310586967]: ---"Txn call failed" err:etcdserver: request timed out 7002ms (00:49:37.655)] Trace[310586967]: [7.004357664s] [7.004357664s] END I0629 00:49:38.155850 1 trace.go:236] Trace[1775334092]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:54ff6a50-9639-466b-a457-b2525751d6ac,client:199.204.45.4,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/kube-system/pods/kube-apiserver-instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (29-Jun-2026 00:49:18.277) (total time: 19878ms): Trace[1775334092]: ---"About to write a response" 19877ms (00:49:38.155) Trace[1775334092]: [19.878061351s] [19.878061351s] END I0629 00:49:38.156804 1 trace.go:236] Trace[1359553605]: "Get" accept:application/json, */*,audit-id:76f7537c-4ba2-46ca-9137-30db88066b0b,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-user,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:10.494) (total time: 27662ms): Trace[1359553605]: ---"About to write a response" 27662ms (00:49:38.156) Trace[1359553605]: [27.662552522s] [27.662552522s] END I0629 00:49:38.156954 1 trace.go:236] Trace[2067423426]: "Get" accept:application/json, */*,audit-id:5fec4681-8189-4c34-8982-ddfb93329f74,client:10.0.0.221,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-controller,user-agent:cert-manager-leader-election/v1.15.5 (linux/amd64) cert-manager/e8fdbe63fd94107a83d3c9f9c134f2d222ba9cff,verb:GET (29-Jun-2026 00:49:30.233) (total time: 7923ms): Trace[2067423426]: ---"About to write a response" 7923ms (00:49:38.156) Trace[2067423426]: [7.923239119s] [7.923239119s] END I0629 00:49:38.157207 1 trace.go:236] Trace[1781859439]: "Get" accept:application/json, */*,audit-id:d37d9e20-25d0-422e-9541-49a18f0ba4f3,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-endpoints,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:11.579) (total time: 26577ms): Trace[1781859439]: [26.577930619s] [26.577930619s] END I0629 00:49:38.157259 1 trace.go:236] Trace[912235275]: "Get" accept:application/json, */*,audit-id:58d8a4d7-c15b-4b08-aa72-0b89c24fdf11,client:10.0.0.34,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/cert-manager/leases/cert-manager-cainjector-leader-election,user-agent:cainjector/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:28.042) (total time: 10114ms): Trace[912235275]: ---"About to write a response" 10114ms (00:49:38.157) Trace[912235275]: [10.114856704s] [10.114856704s] END I0629 00:49:38.157408 1 trace.go:236] Trace[964401434]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:79ecaaf0-e6ee-4a0f-81e8-87a4666ab93d,client:199.204.45.4,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/persistence-rabbitmq-glance-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (29-Jun-2026 00:49:14.422) (total time: 23734ms): Trace[964401434]: ---"About to write a response" 23734ms (00:49:38.157) Trace[964401434]: [23.734481722s] [23.734481722s] END I0629 00:49:38.157513 1 trace.go:236] Trace[1168371956]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:d4eb9ef8-0ac5-4a6a-9947-13efd151b30b,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (29-Jun-2026 00:49:35.656) (total time: 2500ms): Trace[1168371956]: ---"About to write a response" 2500ms (00:49:38.157) Trace[1168371956]: [2.500807764s] [2.500807764s] END I0629 00:49:38.157597 1 trace.go:236] Trace[215283647]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:7730bfbb-2d0c-41f4-b512-be67e321c915,client:199.204.45.4,protocol:HTTP/2.0,resource:services,scope:resource,url:/api/v1/namespaces/ingress-nginx/services/ingress-nginx-controller,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:GET (29-Jun-2026 00:49:25.834) (total time: 12322ms): Trace[215283647]: ---"About to write a response" 12322ms (00:49:38.157) Trace[215283647]: [12.322773639s] [12.322773639s] END I0629 00:49:38.157699 1 trace.go:236] Trace[135140671]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:e7fd23ed-95bb-46a5-96f3-ccdeb963e87d,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:37.657) (total time: 500ms): Trace[135140671]: ["GuaranteedUpdate etcd3" audit-id:e7fd23ed-95bb-46a5-96f3-ccdeb963e87d,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 500ms (00:49:37.657) Trace[135140671]: ---"Txn call completed" 498ms (00:49:38.157)] Trace[135140671]: [500.329896ms] [500.329896ms] END I0629 00:49:38.157806 1 trace.go:236] Trace[1246017399]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:6ca942ab-ed27-40f6-a304-f222fc352cf0,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:33.608) (total time: 4549ms): Trace[1246017399]: ["GuaranteedUpdate etcd3" audit-id:6ca942ab-ed27-40f6-a304-f222fc352cf0,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 4549ms (00:49:33.608) Trace[1246017399]: ---"Txn call completed" 4548ms (00:49:38.157)] Trace[1246017399]: [4.54968139s] [4.54968139s] END I0629 00:49:38.157827 1 trace.go:236] Trace[874047130]: "Get" accept:application/json, */*,audit-id:adaf52b8-3222-4a92-8dd6-75309a606668,client:199.204.45.4,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-dbdg9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:10.114) (total time: 28043ms): Trace[874047130]: ---"About to write a response" 28043ms (00:49:38.157) Trace[874047130]: [28.043671591s] [28.043671591s] END E0629 00:49:38.159233 1 controller.go:193] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"apiserver-f3tcohoifagyom4bbi4wgeu7te\": the object has been modified; please apply your changes to the latest version and try again" I0629 00:49:38.178770 1 trace.go:236] Trace[642545484]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:aae2b6a2-f58f-4816-83b7-002da170d1dd,client:199.204.45.4,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/kube-system/events/kube-apiserver-instance.18bd66dc3d0986d9,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (29-Jun-2026 00:49:19.256) (total time: 18922ms): Trace[642545484]: ["GuaranteedUpdate etcd3" audit-id:aae2b6a2-f58f-4816-83b7-002da170d1dd,key:/events/kube-system/kube-apiserver-instance.18bd66dc3d0986d9,type:*core.Event,resource:events 18922ms (00:49:19.256) Trace[642545484]: ---"initial value restored" 18900ms (00:49:38.156)] Trace[642545484]: ---"Object stored in database" 21ms (00:49:38.178) Trace[642545484]: [18.922699692s] [18.922699692s] END I0629 00:49:38.185010 1 trace.go:236] Trace[914319343]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.4,type:*v1.Endpoints,resource:apiServerIPInfo (29-Jun-2026 00:49:13.377) (total time: 24807ms): Trace[914319343]: ---"initial value restored" 24783ms (00:49:38.161) Trace[914319343]: [24.807367497s] [24.807367497s] END I0629 00:49:38.364598 1 trace.go:236] Trace[865396948]: "Get" accept:application/json, */*,audit-id:ae029a9d-62ed-4366-a6b4-34e67311e61d,client:10.0.0.26,protocol:HTTP/2.0,resource:nodefeatures,scope:resource,url:/apis/nfd.k8s-sigs.io/v1alpha1/namespaces/monitoring/nodefeatures/instance,user-agent:nfd-worker/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:11.174) (total time: 27189ms): Trace[865396948]: ---"About to write a response" 27157ms (00:49:38.332) Trace[865396948]: [27.189727036s] [27.189727036s] END E0629 00:49:48.762217 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"context canceled"}: context canceled E0629 00:49:48.762334 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:48.763364 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:48.763438 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:48.764508 1 trace.go:236] Trace[1904671536]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:cb24b300-312c-4d02-a2c4-b425530ba62b,client:10.0.0.224,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com,user-agent:percona-xtradb-cluster-operator/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:43.762) (total time: 5002ms): Trace[1904671536]: [5.002341524s] [5.002341524s] END E0629 00:49:48.764728 1 timeout.go:142] post-timeout activity - time-elapsed: 2.522752ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/08db1feb.percona.com" result: E0629 00:49:49.015921 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0629 00:49:49.016024 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:49.017161 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:49.017193 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:49.018349 1 trace.go:236] Trace[1490374310]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:5e89f907-f2b1-48a3-b4c0-f90565e64355,client:10.0.0.244,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:44.016) (total time: 5002ms): Trace[1490374310]: [5.00216423s] [5.00216423s] END E0629 00:49:49.018473 1 timeout.go:142] post-timeout activity - time-elapsed: 2.433419ms, GET "/apis/coordination.k8s.io/v1/namespaces/openstack/leases/rabbitmq-cluster-operator-leader-election" result: E0629 00:49:49.589217 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0629 00:49:49.589329 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:49.590623 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:49.590733 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:49.591965 1 trace.go:236] Trace[2083417234]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:35a3a2f4-ff67-43c2-8301-18006ea3978d,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:kube-scheduler/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:44.589) (total time: 5002ms): Trace[2083417234]: [5.002351784s] [5.002351784s] END E0629 00:49:49.592110 1 timeout.go:142] post-timeout activity - time-elapsed: 2.620853ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler" result: E0629 00:49:49.742447 1 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded E0629 00:49:49.742717 1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout E0629 00:49:49.743943 1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout E0629 00:49:49.744013 1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout I0629 00:49:49.745114 1 trace.go:236] Trace[2064420436]: "Get" accept:application/json, */*,audit-id:17f03297-dc96-4db6-9860-5ad3cbacf3c4,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock,user-agent:cilium-operator-generic/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:44.741) (total time: 5003ms): Trace[2064420436]: [5.00311157s] [5.00311157s] END E0629 00:49:49.745309 1 timeout.go:142] post-timeout activity - time-elapsed: 2.669184ms, GET "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cilium-operator-resource-lock" result: E0629 00:49:50.669344 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:50.669628 1 trace.go:236] Trace[490859954]: "Update" accept:application/json, */*,audit-id:597baff5-0c42-4460-9d12-9eede5fb4984,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-svcs-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (29-Jun-2026 00:49:43.666) (total time: 7002ms): Trace[490859954]: ["GuaranteedUpdate etcd3" audit-id:597baff5-0c42-4460-9d12-9eede5fb4984,key:/leases/kube-system/plndr-svcs-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (00:49:43.666) Trace[490859954]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (00:49:50.669)] Trace[490859954]: [7.002966596s] [7.002966596s] END E0629 00:49:50.671377 1 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out I0629 00:49:50.671504 1 trace.go:236] Trace[1203953846]: "Update" accept:application/json, */*,audit-id:0d126d28-c0be-4163-ab18-600ebb42db3f,client:127.0.0.1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock,user-agent:kube-vip/v0.0.0 (linux/amd64) kubernetes/$Format,verb:PUT (29-Jun-2026 00:49:43.668) (total time: 7002ms): Trace[1203953846]: ["GuaranteedUpdate etcd3" audit-id:0d126d28-c0be-4163-ab18-600ebb42db3f,key:/leases/kube-system/plndr-cp-lock,type:*coordination.Lease,resource:leases.coordination.k8s.io 7002ms (00:49:43.669) Trace[1203953846]: ---"Txn call failed" err:etcdserver: request timed out 7001ms (00:49:50.671)] Trace[1203953846]: [7.002620068s] [7.002620068s] END I0629 00:49:50.950280 1 trace.go:236] Trace[1012301211]: "Update" accept:application/vnd.kubernetes.protobuf, */*,audit-id:962fd2b7-0ab7-499b-8b55-96fe5311d2d0,client:::1,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/apiserver-f3tcohoifagyom4bbi4wgeu7te,user-agent:kube-apiserver/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:48.384) (total time: 2566ms): Trace[1012301211]: ["GuaranteedUpdate etcd3" audit-id:962fd2b7-0ab7-499b-8b55-96fe5311d2d0,key:/leases/kube-system/apiserver-f3tcohoifagyom4bbi4wgeu7te,type:*coordination.Lease,resource:leases.coordination.k8s.io 2565ms (00:49:48.384) Trace[1012301211]: ---"Txn call completed" 2564ms (00:49:50.950)] Trace[1012301211]: [2.566040664s] [2.566040664s] END I0629 00:49:50.950414 1 trace.go:236] Trace[1119594036]: "Get" accept:application/json, */*,audit-id:23ff4eb5-d1ce-400d-9325-bc528ed3b831,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-endpoints,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:44.175) (total time: 6775ms): Trace[1119594036]: [6.775025239s] [6.775025239s] END I0629 00:49:50.950298 1 trace.go:236] Trace[1989833792]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8109e626-863a-4e5a-ba4b-d7febc3ab30d,client:199.204.45.4,protocol:HTTP/2.0,resource:persistentvolumeclaims,scope:resource,url:/api/v1/namespaces/openstack/persistentvolumeclaims/persistence-rabbitmq-keystone-server-0,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (29-Jun-2026 00:49:46.397) (total time: 4552ms): Trace[1989833792]: ---"About to write a response" 4552ms (00:49:50.950) Trace[1989833792]: [4.552637899s] [4.552637899s] END I0629 00:49:50.950587 1 trace.go:236] Trace[913150097]: "Get" accept:application/vnd.kubernetes.protobuf, */*,audit-id:53c7a30f-cb01-40e2-8e2a-757c2de8c4a1,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.28.13 (linux/amd64) kubernetes/024ab2a/leader-election,verb:GET (29-Jun-2026 00:49:46.276) (total time: 4673ms): Trace[913150097]: ---"About to write a response" 4673ms (00:49:50.950) Trace[913150097]: [4.673977972s] [4.673977972s] END I0629 00:49:50.950843 1 trace.go:236] Trace[300560296]: "Get" accept:application/json, */*,audit-id:6ce27c08-db25-40a5-a2a5-ccf6801956cb,client:199.204.45.4,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/openstack/pods/keepalived-dbdg9,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:44.216) (total time: 6734ms): Trace[300560296]: ---"About to write a response" 6733ms (00:49:50.950) Trace[300560296]: [6.734373841s] [6.734373841s] END I0629 00:49:50.950959 1 trace.go:236] Trace[293521269]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:8e5543e6-d3ed-412c-aff6-4ec37cef7d49,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/ingress-nginx/leases/ingress-nginx-leader,user-agent:nginx-ingress-controller/v1.12.1 (linux/amd64) ingress-nginx/51c2b819690bbf1709b844dbf321a9acf6eda5a7,verb:PUT (29-Jun-2026 00:49:45.697) (total time: 5253ms): Trace[293521269]: ["GuaranteedUpdate etcd3" audit-id:8e5543e6-d3ed-412c-aff6-4ec37cef7d49,key:/leases/ingress-nginx/ingress-nginx-leader,type:*coordination.Lease,resource:leases.coordination.k8s.io 5253ms (00:49:45.697) Trace[293521269]: ---"Txn call completed" 5251ms (00:49:50.949)] Trace[293521269]: [5.253545303s] [5.253545303s] END I0629 00:49:50.951304 1 trace.go:236] Trace[2066115062]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:9ae8c4aa-fb42-4afd-8f35-6d811a344d2c,client:199.204.45.4,protocol:HTTP/2.0,resource:pods,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/pods/envoy-gateway-78446f96c9-qhjtz,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:GET (29-Jun-2026 00:49:44.025) (total time: 6925ms): Trace[2066115062]: ---"About to write a response" 6924ms (00:49:50.950) Trace[2066115062]: [6.925361363s] [6.925361363s] END I0629 00:49:50.952935 1 trace.go:236] Trace[2130292700]: "Update" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:080a270b-1f71-422a-b755-c8cc1b55165f,client:199.204.45.4,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/instance,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PUT (29-Jun-2026 00:49:48.461) (total time: 2491ms): Trace[2130292700]: ["GuaranteedUpdate etcd3" audit-id:080a270b-1f71-422a-b755-c8cc1b55165f,key:/leases/kube-node-lease/instance,type:*coordination.Lease,resource:leases.coordination.k8s.io 2491ms (00:49:48.461) Trace[2130292700]: ---"Txn call completed" 2490ms (00:49:50.952)] Trace[2130292700]: [2.49176143s] [2.49176143s] END I0629 00:49:50.953428 1 trace.go:236] Trace[479330567]: "Get" accept:application/json, */*,audit-id:0a404887-096c-4a63-9bc4-1cbc278c9fa4,client:10.0.0.237,protocol:HTTP/2.0,resource:jobs,scope:resource,url:/apis/batch/v1/namespaces/openstack/jobs/placement-ks-user,user-agent:kubernetes-entrypoint/v0.0.0 (linux/amd64) kubernetes/$Format,verb:GET (29-Jun-2026 00:49:44.175) (total time: 6778ms): Trace[479330567]: ---"About to write a response" 6777ms (00:49:50.952) Trace[479330567]: [6.778048171s] [6.778048171s] END I0629 00:49:50.953533 1 trace.go:236] Trace[860591967]: "Get" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:382bf7d8-8eb9-4d1f-8490-a3ce373269b7,client:10.0.0.197,protocol:HTTP/2.0,resource:leases,scope:resource,url:/apis/coordination.k8s.io/v1/namespaces/openstack/leases/messaging-topology-operator-leader-election,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format/leader-election,verb:GET (29-Jun-2026 00:49:46.262) (total time: 4690ms): Trace[860591967]: ---"About to write a response" 4690ms (00:49:50.953) Trace[860591967]: [4.690687032s] [4.690687032s] END I0629 00:49:50.955710 1 trace.go:236] Trace[757695164]: "List" accept:application/json, */*,audit-id:6c0034cb-0b71-4f17-91cc-cd3fb359f68e,client:10.0.0.16,protocol:HTTP/2.0,resource:deployments,scope:namespace,url:/apis/apps/v1/namespaces/openstack/deployments,user-agent:rook/v0.0.0 (linux/amd64) kubernetes/$Format,verb:LIST (29-Jun-2026 00:49:49.649) (total time: 1306ms): Trace[757695164]: ["List(recursive=true) etcd3" audit-id:6c0034cb-0b71-4f17-91cc-cd3fb359f68e,key:/deployments/openstack,resourceVersion:,resourceVersionMatch:,limit:0,continue: 1305ms (00:49:49.649)] Trace[757695164]: [1.306085857s] [1.306085857s] END I0629 00:49:50.956120 1 trace.go:236] Trace[1151887590]: "Patch" accept:application/vnd.kubernetes.protobuf,application/json,audit-id:93ddc409-0bcf-4928-8169-bbf2cacf8cc4,client:199.204.45.4,protocol:HTTP/2.0,resource:events,scope:resource,url:/api/v1/namespaces/envoy-gateway-system/events/envoy-gateway-78446f96c9-qhjtz.18bd66e5a5dbd1a5,user-agent:kubelet/v1.28.13 (linux/amd64) kubernetes/024ab2a,verb:PATCH (29-Jun-2026 00:49:44.026) (total time: 6929ms): Trace[1151887590]: ["GuaranteedUpdate etcd3" audit-id:93ddc409-0bcf-4928-8169-bbf2cacf8cc4,key:/events/envoy-gateway-system/envoy-gateway-78446f96c9-qhjtz.18bd66e5a5dbd1a5,type:*core.Event,resource:events 6929ms (00:49:44.026) Trace[1151887590]: ---"initial value restored" 6924ms (00:49:50.950)] Trace[1151887590]: [6.929861576s] [6.929861576s] END I0629 00:49:50.961019 1 trace.go:236] Trace[1765400178]: "GuaranteedUpdate etcd3" audit-id:,key:/masterleases/199.204.45.4,type:*v1.Endpoints,resource:apiServerIPInfo (29-Jun-2026 00:49:48.196) (total time: 2764ms): Trace[1765400178]: ---"initial value restored" 2755ms (00:49:50.952) Trace[1765400178]: [2.764014908s] [2.764014908s] END I0629 00:51:11.334937 1 alloc.go:330] "allocated clusterIPs" service="openstack/ovn-ovsdb-sb" clusterIPs={"IPv4":"10.104.114.22"} I0629 00:51:11.346233 1 alloc.go:330] "allocated clusterIPs" service="openstack/ovn-ovsdb-nb" clusterIPs={"IPv4":"10.96.210.246"} I0629 00:51:25.464114 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-nova" clusterIPs={"IPv4":"10.97.198.227"} I0629 00:52:03.633205 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-api" clusterIPs={"IPv4":"10.111.134.254"} I0629 00:52:03.645998 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-novncproxy" clusterIPs={"IPv4":"10.107.184.221"} I0629 00:52:03.656857 1 alloc.go:330] "allocated clusterIPs" service="openstack/nova-metadata" clusterIPs={"IPv4":"10.108.58.212"} I0629 00:55:22.316217 1 alloc.go:330] "allocated clusterIPs" service="openstack/rabbitmq-neutron" clusterIPs={"IPv4":"10.107.61.232"} I0629 00:55:57.941520 1 alloc.go:330] "allocated clusterIPs" service="openstack/neutron-server" clusterIPs={"IPv4":"10.100.251.145"}